Brutor AI Gateway

Deployed is not done. Prove each AI System still does what you shipped, not just on day one, on day 201.

• Every task becomes a run. The gateway groups the calls of one task, across models, tools, skills and delegated agents, into a single run with an honest outcome: completed, errored or abandoned.
• The contract is enforced on every call. Per-run ceilings on cost, model calls, tokens, delegation depth and wall-clock are checked before each call, and every run is stamped with the contract it ran under.
• Normal is learned, not guessed. Cost per completed run, trajectory length, tool errors and outcomes, all recorded at the gateway, build each system’s own baseline, so drift is flagged with its likely cause.
• Silence is a signal. When a system stops running, liveness raises an alert, even though nothing errored and uptime stayed green.
• Drift gets a response. Tighten a limit, require approval or suspend the system, and the gateway enforces it on the very next call.

The contract, the signals and the assurance report live in the Brutor AI Control Plane. The full story lives on Agents, with the mechanisms shown end to end in the video series.

One API for every model and every modality, with the price list, the failure handling and the keys already in place.

• One API, every model. 311 preconfigured models across 39 providers, plus self-hosted Ollama, vLLM and KServe. Text, image, voice, transcription and video through the same endpoint.
• Anthropic Messages API, natively. /v1/messages and token counting served directly, so Claude-native SDKs and agent frameworks point at Brutor unchanged.
• Batch API. Thousands of requests submitted asynchronously at provider batch rates, up to 50% below on-demand.
• The catalog ships with the price list. Per-token, per-image, per-minute and per-second rates on every model, so budgets, caps and cost-based routing work on real numbers.
• Import from Hugging Face. Search the Hub inside the admin console and pull a model's configuration straight into your catalog.
• Central model defaults. Temperature, top-p and max tokens set per model, so every team gets consistent behavior without per-call tweaking.
• Resilience built in. Per-model circuit breakers, connect timeouts, retries and fallback chains. One provider having a bad day doesn't take your apps down.
• Model lifecycle management. Provider retirement dates tracked per model, with advance operator alerts, so a vendor sunsetting a model never surprises production.
• Secure API key vault. Provider keys encrypted at rest, managed centrally, never exposed to users or to client-side code.

One name your clients keep calling while the fleet changes underneath, and no way for routing to hand out access you never granted.

• A routing group is just a model name. Clients call one name; the group picks the real model. Change the fleet, change no client code.
• One group, many providers. Mix OpenAI, Anthropic and self-hosted models in a single group, and move traffic between them by config.
• Five selection strategies. Weighted random, least busy, lowest usage, lowest latency or lowest cost. Set per group, changed without a redeploy.
• Failure is a fallback, not an error. A 429, 5xx or timeout retries the same request down the chain, and the model that failed goes into cooldown until it recovers.
• Context-window fallback. A prompt too long for the chosen model is re-routed to one that fits, instead of failing back to the caller.
• Rate limits are part of the decision. Per-model RPM, TPM and concurrency are checked while selecting, so the router routes around a saturated model.
• Health checks keep the pool honest. Unhealthy models are excluded automatically on a configurable interval, and returned when they recover.
• Routing can never widen access. Every model in a group must be granted to the calling resource group in its own right. A group is a convenience, never a back door.
• Every route governed and recorded. The same guardrails, budgets and quotas apply whichever model is picked, and the log names the one that actually served the call.

Govern every tool your AI connects to: what exists, what a call is allowed to do, and who has to say yes.

• 80+ MCP servers out of the box. GitHub, HubSpot, Slack, Snowflake, Asana, Atlassian, AWS, Postgres, Terraform and more, one-click enable, with images mirrored to Brutor's own registry so production never depends on an external one.
• Bring your own. Any Docker image, GitHub repo or remote MCP URL. Mix containerized and remote servers in one governed catalog, Kubernetes-ready for production.
• Virtual MCP aggregation. Combine tools from many servers behind one endpoint, with per-group filtering and tool renaming. Agents see one clean toolbox; you keep per-server control.
• Enterprise MCP Registry (Beta) A standalone registry on the official MCP registry spec: federate public servers, publish your own, attach governance metadata to every record.
• OAuth proxy. Authentication handled automatically for every OAuth-protected server. Access tokens stay encrypted at the Gateway, never exposed to clients.
• Capability filtering. Enable individual tools, resources, prompts and resource templates per resource group. A capability that isn't enabled doesn't exist for that group.
• Three-state tool approval. Each tool is enabled, approval-required (blocks pending human review) or disabled. Auto-approve the trusted ones; gate the actions that need a human.
• Guardrails on both sides. Arguments scanned on the way in for PII, secrets and prompt injection; results scanned on the way back before they reach the agent, including mid-call elicitation prompts.
• Argument and semantic policies. Deterministic SQL, URL, shell, path and JSON analyzers that warn or deny on what a call would do, plus a plain-English rule an LLM judge evaluates. Shadow first, then enforce.
• Limits at group, server and tool scope. Token bucket, fixed-window quota and cooldown per tool, on top of daily call and budget caps merged up the group's ancestors. Only real tool calls consume budget.
• Data residency per server. A server's region is checked against the tenant's residency profile before it is reached, and a refusal is written to the audit trail with its reason.
• Every tool call recorded. One log row naming who called, which server, which tool and the decision. Calls sharing a task id become one run in the run ledger.

Encode how your organization actually does things, and let an agent follow it, one governed step at a time.

• Encode your processes as a unit. A skill packages instructions (SKILL.md), scripts, reference files and templates into one versioned, governed bundle: the procedure, not just a prompt.
• Progressive disclosure over MCP. Agents discover the catalog, load full instructions only when needed, then act step by step. A skill costs roughly 50–100 tokens of context until it is actually used.
• Works from any MCP client. Claude Code, IDE plugins, Goose or your own agent, with the same three levels and no Brutor SDK to adopt.
• The bundle never leaves the platform. Agents receive instructions, reference content and script output, never the code itself.
• Scripts run in Brutor's sandbox. No network exposure, a read-only filesystem and a hard timeout. Execution happens on the platform, not inside the agent.
• Every step governed independently. Guardrails on skill input and output, argument policies, per-skill quotas and human approval gates, on each action. No “approved once, trusted after.”
• Access by resource group. Skills are assigned to groups, not to individuals. A skill the caller's groups don't allow never appears in their catalog at all.
• Versioned and immutable. Skills move through draft → validated → published → deprecated → archived. A published version never changes, so callers always know what they're running.
• Author in the UI or sync from GitHub. Keep skill bundles in a repo and publish them through review, or write them directly in the Admin UI.

An agent is a principal you can name, permission, time-box and revoke, before it ever calls a model, a tool, or another agent.

Agent identity & policy

• A principal, not a shared key. Every agent gets its own agent-, a named human owner, and an optional anchor to your IdP, so a workload claim resolves to a governed identity you can hold to account.
• Default-deny. A newly created agent can call nothing at all until a grant says otherwise. Nothing is inherited from a group, and nothing is implied.
• Grants per action, per target. Written against llm_call, mcp_tool, a2a_call or skill_exec, each scoped by a glob over the model, tool, agent card or skill it covers.
• Allow, deny, or ask a human. The third effect routes into the same approval queue your people already answer, so an agent’s riskiest actions get a reviewer, not a refusal.
• Constrained and time-boxed. A grant can carry a time window, a maximum delegation depth, a rate and its own expiry. Identities carry a TTL and revoke in one click.
• Dry-run before you enforce. Run an agent in dry-run and Brutor records what would have been blocked, without blocking it. Flip to enforce once the log is clean.

Agent-to-agent (A2A)

• Native A2A v1.0. The full open standard, implemented in the Gateway with no vendor extensions, inbound and outbound, both governed.
• Signed Agent Cards. Published at /.well-known/agent-card/{tenant}/{name}.json, Ed25519-signed so a third party can verify them, and RBAC-filtered per caller.
• Signed delegation chains. Every multi-hop call carries HMAC provenance, so a hop cannot be spoofed, and depth is capped per group and per card, with a hard ceiling above that.
• Full task lifecycle. SSE token streaming, the complete task state machine, and per-task push notification configs for work that outlives the connection.
• Per-card rate limits. Frequency caps per resource group and tenant-wide per agent card, so one chatty partner agent can’t exhaust your capacity.
• The same rails as every other call. Same RBAC, budgets, guardrails and audit. No second control plane. A2A spend attributes back to the calling team.

Every hop logged, costed and audit-trailed, including the ones your agent makes on someone else’s behalf. See the full A2A architecture

Extend the model’s knowledge with your own document content, governed by the same rules as every other call.

• The retrieval pipeline, already built. Chunking, embedding, hybrid search, permission filtering and citation are all in the platform. You still run a vector store. What you skip is writing and maintaining the pipeline on top of it.
• Qdrant, deployed and owned by you. Qdrant ships in the deployment alongside PostgreSQL and Redis, and you operate it like any other datastore. Brutor manages what lives inside it: the per-tenant collection, the vector configuration and the short-lived scoped tokens.
• Sync from the tools you already use. Eight connectors (Confluence, Notion, Google Drive, Slack, GitHub, Jira, SharePoint and a web crawler) on a schedule, syncing only what changed.
• Or just upload the file. PDF, Word, Excel, PowerPoint, markdown, HTML and CSV, chunked, embedded and indexed automatically from the Portal or the Admin Console.
• Hybrid retrieval, not just vectors. Dense vectors and BM25 keyword matching, fused with reciprocal rank fusion, so a reworded question and an exact part number both find the right passage.
• Source permissions still apply. A synced document keeps the permissions it had at the source, and retrieval filters to what the person asking could have opened themselves. RAG doesn’t become a way around your ACLs.
• Per-resource-group knowledgebases. Each team, department or app gets its own isolated corpus, bound to the group like any other resource, so the same question returns different sources in Legal than in Support.
• Citations on every answer. Each response names the documents that grounded it, so a reader can check the source rather than trust the summary.
• Every retrieval on the record. Who asked what, when, which collections were searched and which sources came back, on the same audit trail as the model call it grounded.

Content checks on every surface the gateway carries, run in-process or handed to a provider you already trust.

• Six checks, on by default where it counts. PII, secrets, prompt injection, jailbreak, toxic content and your own banned-word list. PII, injection and jailbreak are enabled out of the box rather than waiting to be found in a settings page.
• Bring your own detector, per check. Each check runs built-in, or is delegated: Microsoft Presidio self-hosted in your cluster, AWS Bedrock Guardrails in your own AWS account, Lakera Guard, OpenAI moderation, or any endpoint of yours that speaks the contract.
• Built-in means in-process. The bundled detectors run inside the proxy: no second network hop, and nothing about the request leaves your deployment to be scanned.
• Most restrictive action wins. Run more than one detector on the same request and the strictest outcome applies. A provider can tighten the decision; it can never loosen it.
• Every surface, not just chat. LLM input and output, token-by-token streaming, embeddings, image, audio, batch, MCP tool calls, Skills and A2A traffic. The same checks, wherever the content is.
• Streaming guarded two ways. Choose buffered release, where nothing reaches the user until it has been checked, or live interruption, which cuts the stream mid-token when a violation appears.
• It fails closed, by default. If a provider times out or is unreachable, Brutor escalates: retry, then circuit-breaker, then fall back to the built-in detector, and refuses the call if it still can’t decide.
• Break-glass, and it expires. When you genuinely need to proceed through a failed check, the override is explicit, time-limited and on the record, not a config flag someone leaves on.
• Global, or per resource group. One security profile for the tenant, or a stricter one for the teams that need it, merged the same most-restrictive way as every other control.

Guardrails read the content. Policies decide what a call is allowed to do, and every one of them can be measured before it blocks anything.

• Argument policies: what would this do? Deterministic analyzers read the arguments before anything executes: SQL, URL, shell, path and JSON. Block the DELETE, allow the SELECT.
• Including the model’s own tool calls. Argument policies apply to the tool calls a model emits, to MCP input and to skill input, so a bad argument is caught wherever it was composed.
• Warn first, then deny. Every argument policy carries a severity. Run it at warn to see what it would have caught, and promote it to deny once you trust it.
• Semantic policies: what does it mean? Write the constraint in plain English, “must not exfiltrate customer PII”, and a model judges every governed request against it. Intent, not a regex.
• Shadow before enforce. A semantic policy runs in shadow first: real violations are recorded and nothing is blocked. Flip it to enforce when the evidence says it’s right.
• Agent policies: may this agent at all? Grants per action and per target against a default-deny identity, with allow, deny or approval-required, plus dry-run before enforce. Detailed in /06/.
• Policy-as-Code, not a wiki page. Export a resource group’s entire configuration as versioned YAML, who may use which model, tool, skill and agent, and keep it in the repo beside the code it governs.
• Validate → dry-run → apply. Check a bundle against the published schema, see exactly what would change and where, then apply it as a versioned bundle with a full event trail.
• Diff any two versions, roll back one. Compare bundles to see what actually moved between them, and return to an earlier version when a change turns out wrong.

One tree that mirrors your organization, where a team can always tighten a limit, and can never widen one.

• A tree shaped like your org chart. Organization → department → team → app or agent. Every model, MCP server, skill, guardrail, member, API key and limit is scoped to a node in it.
• Resources compose additively. A child can use what it is granted plus what it inherits from its parents, and each inheritance is gated, so a parent can still withhold rather than pass everything down.
• Limits compose restrictively. The effective value is the most restrictive across the node and every ancestor above it. A $500 cap under a $400 department is still $400.
• Which is what makes delegation safe. A team lead can carve their own budget smaller or lock a model down further without a ticket, and without any way to spend past the ceiling finance set above them.
• Caps on what actually costs money. Daily dollar limits, token limits and request limits, for LLM and MCP traffic alike, enforced in real time: the next call over the line is refused, not reconciled later.
• Run-level ceilings for agents. Cost, tokens, LLM calls, delegation depth and wall-clock per run, so one looping agent can’t consume a whole team’s daily allowance before anyone notices.
• Tenant-wide controls per resource. Set a limit directly on one model, MCP server or skill and it applies across every group, merged into the same most-restrictive calculation.
• Agents are members like anyone else. Give an agent its own identity, add it to groups, and it inherits exactly the same way a person does. Detail in /06/.
• 12+ RBAC roles, or make your own. Granular permissions with custom roles for any access pattern, and two separate authentication paths: portal users, and API keys for apps and agents.

Two separate trails: what the AI did, and who changed what it was allowed to do. Both are written as it happens, so the evidence exists before anyone asks for it.

The record

• Proxy logs: what the AI did. One row per call: who called, which model or tool answered, the decision taken, tokens and cost, and which guardrail or policy fired, including the calls that were refused, which are usually the ones you get asked about.
• Tamper-evident by construction. Every row hashes its own canonical bytes onto the previous row’s hash, and the writer Ed25519-signs the head of each flushed batch into a checkpoint. Forging one row means forging every hash after it and a signed checkpoint.
• Rotate keys without losing history. Each checkpoint stores the public key that signed it, so a key rotated out still verifies everything it signed while it was live.
• Full bodies, and a way through them. Request and response bodies retained and filterable by user, group, model, server and date range, with drill-down from a summary view to the single interaction behind it.
• The change trail: who changed what. Separate from the traffic: every resource carries its author, its editor and its version, so “who widened this, and when?” has an answer that doesn’t depend on anyone’s memory.
• Operator actions get their own row. Destructive console operations (counter resets, cache flushes, forced logouts, scheduler triggers, maintenance toggles) are logged individually, apart from ordinary configuration edits.

What compliance is derived from it

• Compliance tagging is opt-in. Declare your frameworks (GDPR Article 30, SOC 2, HIPAA, EU AI Act, ISO 42001) and tags are written onto the log row as calls happen. Declare none and the engine never runs at all.
• ISO/IEC 42001, generally available. Annex A control coverage with evidence derived from real traffic, a readiness rollup, a Statement of Applicability, and an auditor-facing evidence pack you can hand over as-is.
• EU AI Act, against a dated catalog. Risk classification per AI System against an obligation catalog that carries its own dates: Article 9 enforced as controls, Article 12 recorded as a by-product of running.
• AI Asset Registry with fact sheets. One inventory of every model, tool, skill and agent, each with a live dossier of usage, guardrail events and compliance activity. Capture an append-only snapshot when you need a fixed point-in-time record.
• Views your auditor can actually read. GDPR Article 30 processing record, SOC 2 control coverage and HIPAA PHI access, each a view over the same rows, not a separate collection effort.
• Export it where you keep evidence. Stream to S3 or any S3-compatible store, to Splunk HEC, or to a JSONL webhook. Logs live in your own database; long retention is wherever you choose to send them.

Two different caches doing two different jobs: one avoids the provider call entirely, the other makes the calls you can’t avoid cost less.

The response cache: don’t call the provider

• Layer 1: exact match. A SHA-256 hash of the request, looked up in Redis in about a tenth of a millisecond. The identical question asked twice is answered once.
• Layer 2: semantic match. The request is embedded and matched by cosine similarity in Qdrant, in roughly 5–50 ms. A question worded differently still finds the answer, above a similarity threshold you set.
• A hit costs nothing upstream. The provider is never called, so there are no tokens to pay for, and the call is still written to the audit trail like any other.
• Isolated per tenant and per group. One group’s answers never surface in another’s, with an optional shared pool when you do want FAQ-style questions answered once for everybody.
• Time bucketing for time-sensitive asks. “What’s revenue this month?” is bucketed so it doesn’t return last month’s answer, the thing that quietly makes naive response caching dangerous.
• Tool-using requests are skipped. By default the response cache neither looks up nor stores them: the answer depended on what a tool returned at that moment, so replaying it later would be wrong.

The prompt cache: pay for less of the call

• Provider-side, managed centrally. Brutor injects each provider’s own cache markers so the model keeps its computed state for a stable prompt prefix, and repeated calls skip prefill instead of paying to recompute it.
• Six providers, one setting. Anthropic, OpenAI, Bedrock Claude, Vertex Claude, Azure OpenAI and Google Gemini each expose prompt caching differently. Brutor tracks the capability per provider so you don’t hand-write markers per vendor.
• The economics, stated plainly. A cache read costs roughly 10% of input tokens; a cache write roughly 125%. Break-even is two reads, which is why it belongs on stable prefixes and not on everything.
• It wants your tool definitions. The opposite of the response cache: tool definitions are a large, stable prefix, and prompt caching is about input computation rather than output content. A tool-heavy agent is exactly the workload it pays off on.
• Long system prompts stop hurting. The system prompt, the tool schema and the retrieved context are the expensive, repeated part of an agent’s every turn, and the part this makes nearly free after the first call.
• Both surfaced in Mission Control. Hit rates, spend avoided and latency saved for each cache, so you can see whether a threshold or a TTL is actually earning its place.

Know what it costs, know who spent it, and be able to show what you saved, measured, not estimated by a vendor.

See where the money actually goes

• Attribution four ways, on one screen. Spend broken out by provider, by model, by team and by individual user, each with request counts and period-on-period movement, and each exportable.
• Agents and unattributed, counted honestly. Spend with no human behind it is shown as its own line rather than being quietly spread across users. On an agent estate that line is usually the big one.
• Governed and observed, side by side. Traffic Brutor enforced inline, and vendor spend imported for visibility, in the same total, so the AI you buy sits next to the AI you build instead of in a different spreadsheet.
• Cost composition over time. Weekly buckets by provider, so a change in the mix is visible as a shape rather than as a surprise at invoice time.
• Budget burn-down against real caps. Set a budget on a resource group and the burn-down is tracked against it here. The enforcement itself lives in /10/. This is where you watch it.

LLM economics

• Unit economics, not just totals. Cost per 1,000 requests, average cost per request and average tokens per request: the numbers that tell you whether a system got more expensive or simply got busier.
• Cost per 1,000 requests, over time. Charted month by month next to request volume, with token usage split into prompt and completion, so a rising bill reads as either more work or more expensive work.
• Cost avoided is a measured figure. The platform reports what caching actually saved, separating the gateway cache’s estimate from the provider prompt cache’s measured discount. Two different confidence levels, shown as two different numbers.
• Cache hit rate, in the same view. Hits, tokens saved and hit rate across the period, so a similarity threshold or a TTL can be judged on what it earned rather than on how it felt.

Spend less, and prove how much

• Two caches, two kinds of saving. The response cache avoids the provider call; the prompt cache makes the unavoidable calls cheaper. Both are covered in /12/.
• Route to the cheapest model that qualifies. Cost-based routing sends each call to the least expensive model still allowed by policy, so premium tiers are reserved for the queries that need them. Detail in /03/.
• Batch the work that can wait. Non-urgent workloads go through provider batch APIs at roughly half the on-demand rate: same governance, same audit trail, lower invoice.
• Alerts before the invoice. Usage and budget thresholds raise alerts with an acknowledgement workflow, and can notify an endpoint you already monitor rather than an inbox nobody reads.

Drill into any group, at any level

• The same view at every node. Organization, department, team or a single agent: requests, cost, tokens, MCP calls, agent calls, skill calls, unique users and error rate, with descendants rolled in or excluded.
• Broken down by what was called. Which models, which MCP tools, which agent cards and capabilities, each with call counts, duration and errors, so an expensive week has a named cause.
• Every surface counted, not just chat. LLM, MCP, agent and skill traffic in one place. A tool-heavy agent that barely touches a model still shows up as the workload it is.

Seven views over the same traffic, and one of them tells you about the AI that never came through the gateway at all.

See the traffic

• Seven tabs over one dataset. Traffic, Cost, LLM Economics, Governance, Caching, Agents and Reliability: the same requests, asked seven different questions, rather than seven disconnected dashboards.
• Broken down by surface. LLM, MCP, A2A and Portal traffic as a share of the whole. On most estates the MCP slice is the one nobody had a number for.
• Volume, failures and error rate over time. Total against failed on one axis, error rate on its own, plus an hour-of-day by day-of-week heatmap that makes a batch job or a runaway schedule obvious on sight.
• MCP and A2A get first-class views. Per-method call counts, average duration and success rate for MCP; per-capability and per-agent-card figures for A2A, including which peer is failing and how slowly.
• Who is actually calling. User-type distribution across API keys, portal users and anonymous callers, so “is this people or machines?” is answered before you start optimizing the wrong thing.
• Slice it any way you need. Filter by AI System, by governed or observed plane, and across 24 hours to all time or a custom range. Every panel exports.

See the governance posture, including what you don’t govern

• Enforcement outcomes, as numbers. Guardrail blocks as a count and as a share of requests, policy denials, PII detections, and how many requests were inspected at all: the difference between “nothing fired” and “nothing was checked”.
• Discovery coverage, stated as a fraction. Governed coverage shown as governed assets over discovered assets. A percentage under 100 is the honest headline, and most estates start well under it.
• Shadow AI, named and typed. Assets seen by collectors but never routed through Brutor (local AI clients, LLM endpoints, MCP servers, inference runtimes, agent frameworks) each with a confidence score.
• Onboard or dismiss, from the list. Every discovered asset carries an action. Shadow AI stops being a report you read and becomes a queue you work through.
• Un-approved model spend. Requests to models that are imported, disabled or simply not in your catalog, with what they cost: traffic the gateway never enforced, surfaced so you can decide whether to.
• Compliance coverage at a glance. Per-framework counts for EU AI Act, GDPR Article 30, HIPAA, ISO 42001 and SOC 2, showing plainly when a framework isn’t enabled rather than reporting a comfortable zero. Detail in /11/.

And take it where you already look

• Bring your own observability stack. Native OpenTelemetry hooks feed Prometheus, Grafana, Loki or any OTel-compatible backend you already run. Brutor doesn’t ask to be your only pane of glass.
• Alerts with somewhere to go. Usage-limit breaches, unhealthy models and budget thresholds raise alerts with an acknowledgement workflow, and notify an endpoint your team already watches.
• Infrastructure health, live. Real-time reachability for every MCP server and AI model, so a failing dependency shows up as a red component rather than as a mysterious rise in error rate.