Brutor AI Gateway

Every task recorded as a run

The calls that make up one task — across models, tools, skills and delegated agents — are grouped into a single run with an honest terminal state. That is the unit owners and auditors actually ask about.

Signals, learned per system

Cost per completed run, trajectory length, tool errors and outcome mix, each with its own history. Not thresholds you guessed — the system’s own normal, learned from what it did.

Including what nothing produced

Spend on runs that errored, were abandoned or exhausted their budget is called out separately. Money burned without an answer is the number no per-request dashboard shows you.

Liveness — the absence signal

The only signal that fires when traffic stops. A system that goes quiet, stalls or collapses in volume raises an alert, even though nothing errored and uptime stayed green.

Drift with a cause, or none claimed

Behaviour is compared against the system’s own baseline and changes are flagged with the likely reason — a model updated, a tool definition changed. “Unexplained” is reported as unexplained, never guessed.

Health that refuses to round up

One worst-of score per system, always decomposable into the signals behind it. A system with too little evidence is scored unknown, never green.

Signals with their own graphs

Each signal shows its current figure, its distribution and its shape over time — so “cost went up” is immediately either a step change, a slow drift, or one bad afternoon.

A cadence you can propose from evidence

Rather than asking you to invent a liveness expectation, Brutor can suggest one from the runs it has already observed — and shows plainly when the detector has not evaluated a system yet.

Every run, and every step inside it

Open the ledger to find the runs that matter, then open one to see the actions it took in order, with what each cost and what it returned.

A contract is generated, never authored

A hash-pinned snapshot of every control that resolves onto the system — models, MCP servers, skills, guardrails, policies, limits, autonomy. A policy document that can disagree with reality is worse than none.

You can read what was signed

Each version expands to show exactly what it froze, with inherited grants marked as such — because a v1 has no predecessor and therefore no diff, and nobody should approve a hash whose body they cannot see.

Enforced ceilings and graded bounds

Per-run ceilings on cost, model calls, tokens, delegation depth and wall-clock are checked at the gateway before every call. Window bounds — average cost per task, completion and error rates — are graded in the report and never refuse a call.

Stamped on every run

Each finished run records the contract it ran under, so “what governed this?” has a provable answer months later, and conformance is measurable rather than assumed.

Replay before you promote

Freeze a stratified suite from recorded traffic and rehearse a policy change against it offline — nothing dispatched upstream. Your audit log becomes an eval set, with no extra instrumentation.

Automatic response when it drifts

Arm a policy to downgrade autonomy, tighten a limit or suspend a system while a human investigates — with a blast-radius preview first, and a one-click restore after.

A verdict in words, not a score

Assured, conditionally assured or not assured, with what we know, what prevents full assurance, the recommended action, and why any of it matters — readable by someone who has never seen the product.

It states what it does not cover

Unmeasured is reported as unmeasured, never as a pass. A verdict measured against a system’s own history rather than an approved bound says exactly that, in the verdict itself.

An evidence maturity ladder

Observable, baseline established, controlled, evidence-backed, independently assured — with the specific thing blocking the next rung, and what to do about it.

Learned baselines, shown as learned

Every baseline carries how many runs it rests on and whether it is ready or still learning — so nobody mistakes a fortnight of data for an established norm.

Recommended controls, ranked

The report ends with what to do next, ordered by what would move the verdict — mint a contract, bind a replay suite, arm a response policy, revoke unused grants.

Snapshot it, or export it

Capture an immutable point-in-time snapshot, or export the whole thing as a signed certificate, JSON or PDF for whoever asked.

One API, every model

311 preconfigured models across 39 providers, plus self-hosted Ollama, vLLM and KServe. Text, image, voice, transcription and video through the same endpoint.

Anthropic Messages API, natively

/v1/messages and token counting served directly, so Claude-native SDKs and agent frameworks point at Brutor unchanged.

Batch API

Thousands of requests submitted asynchronously at provider batch rates — up to 50% below on-demand.

The catalog ships with the price list

Per-token, per-image, per-minute and per-second rates on every model, so budgets, caps and cost-based routing work on real numbers.

Import from Hugging Face

Search the Hub inside the admin console and pull a model’s configuration straight into your catalog.

Central model defaults

Temperature, top-p and max tokens set per model, so every team gets consistent behaviour without per-call tweaking.

Resilience built in

Per-model circuit breakers, connect timeouts, retries and fallback chains. One provider having a bad day doesn’t take your apps down.

Model lifecycle management

Provider retirement dates tracked per model, with advance operator alerts, so a vendor sunsetting a model never surprises production.

Secure API key vault

Provider keys encrypted at rest, managed centrally, never exposed to users or to client-side code.

A routing group is just a model name

Clients call one name; the group picks the real model. Change the fleet, change no client code.

One group, many providers

Mix OpenAI, Anthropic and self-hosted models in a single group, and move traffic between them by config.

Five selection strategies

Weighted random, least busy, lowest usage, lowest latency or lowest cost. Set per group, changed without a redeploy.

Failure is a fallback, not an error

A 429, 5xx or timeout retries the same request down the chain, and the model that failed goes into cooldown until it recovers.

Context-window fallback

A prompt too long for the chosen model is re-routed to one that fits, instead of failing back to the caller.

Rate limits are part of the decision

Per-model RPM, TPM and concurrency are checked while selecting, so the router routes around a saturated model.

Health checks keep the pool honest

Unhealthy models are excluded automatically on a configurable interval, and returned when they recover.

Routing can never widen access

Every model in a group must be granted to the calling resource group in its own right. A group is a convenience, never a back door.

Every route governed and recorded

The same guardrails, budgets and quotas apply whichever model is picked, and the log names the one that actually served the call.

80+ MCP servers out of the box

GitHub, HubSpot, Slack, Snowflake, Asana, Atlassian, AWS, Postgres, Terraform and more — one-click enable, with images mirrored to Brutor’s own registry so production never depends on an external one.

Bring your own

Any Docker image, GitHub repo or remote MCP URL. Mix containerized and remote servers in one governed catalog, Kubernetes-ready for production.

Virtual MCP aggregation

Combine tools from many servers behind one endpoint, with per-group filtering and tool renaming. Agents see one clean toolbox; you keep per-server control.

Enterprise MCP Registry (Beta)

A standalone registry on the official MCP registry spec: federate public servers, publish your own, attach governance metadata to every record.

OAuth proxy

Authentication handled automatically for every OAuth-protected server. Access tokens stay encrypted at the Gateway, never exposed to clients.

Capability filtering

Enable individual tools, resources, prompts and resource templates per resource group. A capability that isn’t enabled doesn’t exist for that group.

Three-state tool approval

Each tool is enabled, approval-required (blocks pending human review) or disabled. Auto-approve the trusted ones; gate the actions that need a human.

Guardrails on both sides

Arguments scanned on the way in for PII, secrets and prompt injection; results scanned on the way back before they reach the agent, including mid-call elicitation prompts.

Argument and semantic policies

Deterministic SQL, URL, shell, path and JSON analyzers that warn or deny on what a call would do, plus a plain-English rule an LLM judge evaluates. Shadow first, then enforce.

Limits at group, server and tool scope

Token bucket, fixed-window quota and cooldown per tool, on top of daily call and budget caps merged up the group’s ancestors. Only real tool calls consume budget.

Data residency per server

A server’s region is checked against the tenant’s residency profile before it is reached, and a refusal is written to the audit trail with its reason.

Every tool call recorded

One log row naming who called, which server, which tool and the decision. Calls sharing a task id become one run in the run ledger.

Encode your processes as a unit

A skill packages instructions (SKILL.md), scripts, reference files and templates into one versioned, governed bundle — the procedure, not just a prompt.

Progressive disclosure over MCP

Agents discover the catalog, load full instructions only when needed, then act step by step. A skill costs roughly 50–100 tokens of context until it is actually used.

Works from any MCP client

Claude Code, IDE plugins, Goose or your own agent — the same three levels, with no Brutor SDK to adopt.

The bundle never leaves the platform

Agents receive instructions, reference content and script output — never the code itself.

Scripts run in Brutor’s sandbox

No network exposure, a read-only filesystem and a hard timeout. Execution happens on the platform, not inside the agent.

Every step governed independently

Guardrails on skill input and output, argument policies, per-skill quotas and human approval gates — on each action. No “approved once, trusted after.”

Access by resource group

Skills are assigned to groups, not to individuals. A skill the caller’s groups don’t allow never appears in their catalog at all.

Versioned and immutable

Skills move through draft → validated → published → deprecated → archived. A published version never changes, so callers always know what they’re running.

Author in the UI or sync from GitHub

Keep skill bundles in a repo and publish them through review, or write them directly in the Admin UI.

A principal, not a shared key

Every agent gets its own agent-<ULID>, a named human owner, and an optional anchor to your IdP — so a workload claim resolves to a governed identity you can hold to account.

Default-deny

A newly created agent can call nothing at all until a grant says otherwise. Nothing is inherited from a group, and nothing is implied.

Grants per action, per target

Written against llm_call, mcp_tool, a2a_call or skill_exec, each scoped by a glob over the model, tool, agent card or skill it covers.

Allow, deny, or ask a human

The third effect routes into the same approval queue your people already answer — so an agent’s riskiest actions get a reviewer, not a refusal.

Constrained and time-boxed

A grant can carry a time window, a maximum delegation depth, a rate and its own expiry. Identities carry a TTL and revoke in one click.

Dry-run before you enforce

Run an agent in dry-run and Brutor records what would have been blocked, without blocking it. Flip to enforce once the log is clean.

The retrieval pipeline, already built

Chunking, embedding, hybrid search, permission filtering and citation are all in the platform. You still run a vector store — what you skip is writing and maintaining the pipeline on top of it.

Qdrant, deployed and owned by you

Qdrant ships in the deployment alongside PostgreSQL and Redis, and you operate it like any other datastore. Brutor manages what lives inside it: the per-tenant collection, the vector configuration and the short-lived scoped tokens.

Sync from the tools you already use

Eight connectors — Confluence, Notion, Google Drive, Slack, GitHub, Jira, SharePoint and a web crawler — on a schedule, syncing only what changed.

Or just upload the file

PDF, Word, Excel, PowerPoint, markdown, HTML and CSV, chunked, embedded and indexed automatically from the Portal or the Admin Console.

Hybrid retrieval, not just vectors

Dense vectors and BM25 keyword matching, fused with reciprocal rank fusion — so a reworded question and an exact part number both find the right passage.

Source permissions still apply

A synced document keeps the permissions it had at the source, and retrieval filters to what the person asking could have opened themselves. RAG doesn’t become a way around your ACLs.

Per-resource-group knowledgebases

Each team, department or app gets its own isolated corpus, bound to the group like any other resource — so the same question returns different sources in Legal than in Support.

Citations on every answer

Each response names the documents that grounded it, so a reader can check the source rather than trust the summary.

Every retrieval on the record

Who asked what, when, which collections were searched and which sources came back — on the same audit trail as the model call it grounded.

Six checks, on by default where it counts

PII, secrets, prompt injection, jailbreak, toxic content and your own banned-word list. PII, injection and jailbreak are enabled out of the box rather than waiting to be found in a settings page.

Bring your own detector, per check

Each check runs built-in, or is delegated: Microsoft Presidio self-hosted in your cluster, AWS Bedrock Guardrails in your own AWS account, Lakera Guard, OpenAI moderation, or any endpoint of yours that speaks the contract.

Built-in means in-process

The bundled detectors run inside the proxy — no second network hop, and nothing about the request leaves your deployment to be scanned.

Most restrictive action wins

Run more than one detector on the same request and the strictest outcome applies. A provider can tighten the decision; it can never loosen it.

Every surface, not just chat

LLM input and output, token-by-token streaming, embeddings, image, audio, batch, MCP tool calls, Skills and A2A traffic — the same checks, wherever the content is.

Streaming guarded two ways

Choose buffered release, where nothing reaches the user until it has been checked, or live interruption, which cuts the stream mid-token when a violation appears.

It fails closed, by default

If a provider times out or is unreachable, Brutor escalates — retry, then circuit-breaker, then fall back to the built-in detector — and refuses the call if it still can’t decide.

Break-glass, and it expires

When you genuinely need to proceed through a failed check, the override is explicit, time-limited and on the record — not a config flag someone leaves on.

Global, or per resource group

One security profile for the tenant, or a stricter one for the teams that need it — merged the same most-restrictive way as every other control.

Argument policies — what would this do?

Deterministic analyzers read the arguments before anything executes: SQL, URL, shell, path and JSON. Block the DELETE, allow the SELECT.

Including the model’s own tool calls

Argument policies apply to the tool calls a model emits, to MCP input and to skill input — so a bad argument is caught wherever it was composed.

Warn first, then deny

Every argument policy carries a severity. Run it at warn to see what it would have caught, and promote it to deny once you trust it.

Semantic policies — what does it mean?

Write the constraint in plain English — “must not exfiltrate customer PII” — and a model judges every governed request against it. Intent, not a regex.

Shadow before enforce

A semantic policy runs in shadow first: real violations are recorded and nothing is blocked. Flip it to enforce when the evidence says it’s right.

Agent policies — may this agent at all?

Grants per action and per target against a default-deny identity, with allow, deny or approval-required, plus dry-run before enforce. Detailed in /05/.

Policy-as-Code, not a wiki page

Export a resource group’s entire configuration as versioned YAML — who may use which model, tool, skill and agent — and keep it in the repo beside the code it governs.

Validate → dry-run → apply

Check a bundle against the published schema, see exactly what would change and where, then apply it as a versioned bundle with a full event trail.

Diff any two versions, roll back one

Compare bundles to see what actually moved between them, and return to an earlier version when a change turns out wrong.

A tree shaped like your org chart

Organization → department → team → app or agent. Every model, MCP server, skill, guardrail, member, API key and limit is scoped to a node in it.

Resources compose additively

A child can use what it is granted plus what it inherits from its parents — and each inheritance is gated, so a parent can still withhold rather than pass everything down.

Limits compose restrictively

The effective value is the most restrictive across the node and every ancestor above it. A $500 cap under a $400 department is still $400.

Which is what makes delegation safe

A team lead can carve their own budget smaller or lock a model down further without a ticket — and without any way to spend past the ceiling finance set above them.

Caps on what actually costs money

Daily dollar limits, token limits and request limits, for LLM and MCP traffic alike, enforced in real time — the next call over the line is refused, not reconciled later.

Run-level ceilings for agents

Cost, tokens, LLM calls, delegation depth and wall-clock per run, so one looping agent can’t consume a whole team’s daily allowance before anyone notices.

Tenant-wide controls per resource

Set a limit directly on one model, MCP server or skill and it applies across every group — merged into the same most-restrictive calculation.

Agents are members like anyone else

Give an agent its own identity, add it to groups, and it inherits exactly the same way a person does. Detail in /05/.

12+ RBAC roles, or make your own

Granular permissions with custom roles for any access pattern, and two separate authentication paths — portal users, and API keys for apps and agents.

Proxy logs — what the AI did

One row per call: who called, which model or tool answered, the decision taken, tokens and cost, and which guardrail or policy fired — including the calls that were refused, which are usually the ones you get asked about.

Tamper-evident by construction

Every row hashes its own canonical bytes onto the previous row’s hash, and the writer Ed25519-signs the head of each flushed batch into a checkpoint. Forging one row means forging every hash after it and a signed checkpoint.

Rotate keys without losing history

Each checkpoint stores the public key that signed it, so a key rotated out still verifies everything it signed while it was live.

Full bodies, and a way through them

Request and response bodies retained and filterable by user, group, model, server and date range, with drill-down from a summary view to the single interaction behind it.

The change trail — who changed what

Separate from the traffic: every resource carries its author, its editor and its version, so “who widened this, and when?” has an answer that doesn’t depend on anyone’s memory.

Operator actions get their own row

Destructive console operations — counter resets, cache flushes, forced logouts, scheduler triggers, maintenance toggles — are logged individually, apart from ordinary configuration edits.

Compliance tagging is opt-in

Declare your frameworks — GDPR Article 30, SOC 2, HIPAA, EU AI Act, ISO 42001 — and tags are written onto the log row as calls happen. Declare none and the engine never runs at all.

ISO/IEC 42001, generally available

Annex A control coverage with evidence derived from real traffic, a readiness rollup, a Statement of Applicability, and an auditor-facing evidence pack you can hand over as-is.

EU AI Act, against a dated catalog

Risk classification per AI System against an obligation catalog that carries its own dates — Article 9 enforced as controls, Article 12 recorded as a by-product of running.

AI Asset Registry with fact sheets

One inventory of every model, tool, skill and agent, each with a live dossier of usage, guardrail events and compliance activity. Capture an append-only snapshot when you need a fixed point-in-time record.

Views your auditor can actually read

GDPR Article 30 processing record, SOC 2 control coverage and HIPAA PHI access — each a view over the same rows, not a separate collection effort.

Export it where you keep evidence

Stream to S3 or any S3-compatible store, to Splunk HEC, or to a JSONL webhook. Logs live in your own database; long retention is wherever you choose to send them.

Layer 1 — exact match

A SHA-256 hash of the request, looked up in Redis in about a tenth of a millisecond. The identical question asked twice is answered once.

Layer 2 — semantic match

The request is embedded and matched by cosine similarity in Qdrant, in roughly 5–50 ms. A question worded differently still finds the answer, above a similarity threshold you set.

A hit costs nothing upstream

The provider is never called, so there are no tokens to pay for — and the call is still written to the audit trail like any other.

Isolated per tenant and per group

One group’s answers never surface in another’s, with an optional shared pool when you do want FAQ-style questions answered once for everybody.

Time bucketing for time-sensitive asks

“What’s revenue this month?” is bucketed so it doesn’t return last month’s answer — the thing that quietly makes naive response caching dangerous.

Tool-using requests are skipped

By default the response cache neither looks up nor stores them: the answer depended on what a tool returned at that moment, so replaying it later would be wrong.

Provider-side, managed centrally

Brutor injects each provider’s own cache markers so the model keeps its computed state for a stable prompt prefix — and repeated calls skip prefill instead of paying to recompute it.

Six providers, one setting

Anthropic, OpenAI, Bedrock Claude, Vertex Claude, Azure OpenAI and Google Gemini each expose prompt caching differently. Brutor tracks the capability per provider so you don’t hand-write markers per vendor.

The economics, stated plainly

A cache read costs roughly 10% of input tokens; a cache write roughly 125%. Break-even is two reads — which is why it belongs on stable prefixes and not on everything.

It wants your tool definitions

The opposite of the response cache: tool definitions are a large, stable prefix, and prompt caching is about input computation rather than output content. A tool-heavy agent is exactly the workload it pays off on.

Long system prompts stop hurting

The system prompt, the tool schema and the retrieved context are the expensive, repeated part of an agent’s every turn — and the part this makes nearly free after the first call.

Both surfaced in Mission Control

Hit rates, spend avoided and latency saved for each cache, so you can see whether a threshold or a TTL is actually earning its place.

Attribution four ways, on one screen

Spend broken out by provider, by model, by team and by individual user — each with request counts and period-on-period movement, and each exportable.

Agents and unattributed, counted honestly

Spend with no human behind it is shown as its own line rather than being quietly spread across users. On an agent estate that line is usually the big one.

Governed and observed, side by side

Traffic Brutor enforced inline, and vendor spend imported for visibility, in the same total — so the AI you buy sits next to the AI you build instead of in a different spreadsheet.

Cost composition over time

Weekly buckets by provider, so a change in the mix is visible as a shape rather than as a surprise at invoice time.

Unit economics, not just totals

Cost per 1,000 requests, average cost per request and average tokens per request — the numbers that tell you whether a system got more expensive or simply got busier.

Budget burn-down against real caps

Set a budget on a resource group and the burn-down is tracked against it here. The enforcement itself lives in /09/ — this is where you watch it.

Cost avoided is a measured figure

The platform reports what caching actually saved, separating the gateway cache’s estimate from the provider prompt cache’s measured discount. Two different confidence levels, shown as two different numbers.

Cache hit rate, in the same view

Hits, tokens saved and hit rate across the period, so a similarity threshold or a TTL can be judged on what it earned rather than on how it felt.

Two caches, two kinds of saving

The response cache avoids the provider call; the prompt cache makes the unavoidable calls cheaper. Both are covered in /11/.

Route to the cheapest model that qualifies

Cost-based routing sends each call to the least expensive model still allowed by policy, so premium tiers are reserved for the queries that need them. Detail in /02/.

Batch the work that can wait

Non-urgent workloads go through provider batch APIs at roughly half the on-demand rate — same governance, same audit trail, lower invoice.

Alerts before the invoice

Usage and budget thresholds raise alerts with an acknowledgement workflow, and can notify an endpoint you already monitor rather than an inbox nobody reads.

The same view at every node

Organization, department, team or a single agent — requests, cost, tokens, MCP calls, agent calls, skill calls, unique users and error rate, with descendants rolled in or excluded.

Broken down by what was called

Which models, which MCP tools, which agent cards and capabilities — each with call counts, duration and errors, so an expensive week has a named cause.

Every surface counted, not just chat

LLM, MCP, agent and skill traffic in one place. A tool-heavy agent that barely touches a model still shows up as the workload it is.

Seven tabs over one dataset

Traffic, Cost, LLM Economics, Governance, Caching, Agents and Reliability — the same requests, asked seven different questions, rather than seven disconnected dashboards.

Broken down by surface

LLM, MCP, A2A and Portal traffic as a share of the whole. On most estates the MCP slice is the one nobody had a number for.

Volume, failures and error rate over time

Total against failed on one axis, error rate on its own, plus an hour-of-day by day-of-week heatmap that makes a batch job or a runaway schedule obvious on sight.

MCP and A2A get first-class views

Per-method call counts, average duration and success rate for MCP; per-capability and per-agent-card figures for A2A, including which peer is failing and how slowly.

Who is actually calling

User-type distribution across API keys, portal users and anonymous callers — so “is this people or machines?” is answered before you start optimising the wrong thing.

Slice it any way you need

Filter by AI System, by governed or observed plane, and across 24 hours to all time or a custom range. Every panel exports.

Enforcement outcomes, as numbers

Guardrail blocks as a count and as a share of requests, policy denials, PII detections, and how many requests were inspected at all — the difference between “nothing fired” and “nothing was checked”.

Discovery coverage, stated as a fraction

Governed coverage shown as governed assets over discovered assets. A percentage under 100 is the honest headline, and most estates start well under it.

Shadow AI, named and typed

Assets seen by collectors but never routed through Brutor — local AI clients, LLM endpoints, MCP servers, inference runtimes, agent frameworks — each with a confidence score.

Onboard or dismiss, from the list

Every discovered asset carries an action. Shadow AI stops being a report you read and becomes a queue you work through.

Un-approved model spend

Requests to models that are imported, disabled or simply not in your catalog, with what they cost — traffic the gateway never enforced, surfaced so you can decide whether to.

Compliance coverage at a glance

Per-framework counts for EU AI Act, GDPR Article 30, HIPAA, ISO 42001 and SOC 2, showing plainly when a framework isn’t enabled rather than reporting a comfortable zero. Detail in /10/.

Bring your own observability stack

Native OpenTelemetry hooks feed Prometheus, Grafana, Loki or any OTel-compatible backend you already run — Brutor doesn’t ask to be your only pane of glass.

Alerts with somewhere to go

Usage-limit breaches, unhealthy models and budget thresholds raise alerts with an acknowledgement workflow, and notify an endpoint your team already watches.

Infrastructure health, live

Real-time reachability for every MCP server and AI model, so a failing dependency shows up as a red component rather than as a mysterious rise in error rate.