Your AI agent passed its tests. It now processes work throughout the day, calls tools and delegates to other agents. Can you leave it running without watching every task?
That takes clear limits, ongoing checks and a way to intervene. Here, unattended means the agent can perform approved work independently, with people responsible for designated decisions and exceptions.
Brutor’s AI control plane connects those controls to the evidence: what the agent was approved to do, what its governed calls actually did, and what happened when something changed. This is AI agent assurance in practice.
The code can stay the same while behaviour changes
Configuration, data and dependencies affect ordinary software too. With an AI agent, model outputs can also influence tool selection, recommendations and subsequent actions. A model update, an edited prompt or different tool responses can change its behaviour without a new release of the agent’s code.
Testing before release remains essential. Runtime assurance adds evidence about how the deployed system operates as those conditions change.
Two questions, with different answers
Brutor checks the agent against two references:
| Question | Reference |
|---|---|
| Is it within what we approved? | The operating contract: the approved resources, limits, controls and releases. |
| Has its behaviour changed? | The baseline: patterns learned from recorded runs, including actions per task, tool use, completion rates and cost. |
An illustrative example: the same release starts taking fifteen calls to complete work that usually took nine. Its tools are still approved and its spending remains within budget. Contract conformance can remain healthy while a sustained change from the baseline warrants investigation. Harder applications, changed tool data or an inefficient loop could explain it; the change alone does not establish the cause.
Conversely, an unapproved release may report itself while producing otherwise familiar traffic. That is a conformance issue even if no behavioural trend has shifted. You need both checks.
A running demo, with synthetic data
The Brutor Demo System is a loan pre-screening agent for a fictional Swedish lender, Borealis Consumer Finance AB. It runs every ten minutes on our platform. All applicant data is synthetic.
For each application, it:
- retrieves application, identity and credit information through two tool servers;
- runs an affordability-check skill;
- asks one model to assess risk and a second agent to screen for fraud;
- asks a stronger model to draft a recommendation and a customer letter;
- records the recommendation, subject to the approval rules.
In this demo, recording a decline or a recommendation for a loan above EUR 25,000 requires a human underwriter’s approval. That threshold is a demo policy.
Each application becomes a run: a task whose governed calls are grouped and recorded in sequence.
How Brutor keeps watch
1. Agree: approve the operating contract
Brutor records a versioned snapshot of the system’s resources, limits and controls, with a hash identifying that snapshot. Previous contract versions are retained, so later configurations can be compared with what was approved.
2. Route: enforce rules and record calls
The demo routes its model, tool, skill and agent-to-agent calls through the Brutor AI Gateway. It uses the gateway’s endpoints and adds headers identifying the run, step and release. No monitoring library is required.
The gateway enforces the configured access rules, policies and limits on each governed call and records the result. Assurance then compares the recorded activity and configuration with the approved contract. Immediate enforcement and checks over completed runs serve different purposes.
3. Compare: check runs and trends
Brutor brings the evidence together in seven signals:
| Signal | Question |
|---|---|
| Liveness | Is the agent running when expected? |
| Behaviour | Have its action counts, tools or models changed? |
| Reliability | Are runs completing or failing? |
| Cost | Is the cost per completed task changing? |
| Conformance | Does recorded activity match the approved contract? |
| Oversight | Are required human approvals receiving decisions? |
| Review | Are findings being reviewed? |
Overall health follows the weakest enforced signal, rather than an average. Strong cost and reliability scores cannot cancel an oversight failure. Where evidence is insufficient, the verdict is unknown.
Some checks evaluate individual runs; others need a reporting window and enough observations to detect a meaningful change.
4. Respond: raise a finding and apply the configured response
A finding records the issue, its severity and supporting evidence. Related occurrences update one finding with a recurrence count, keeping repeated problems together in the Assurance Inbox.
Configured notifications can send findings to Slack or a signed webhook. Notification delivery failures are also surfaced for investigation.
In the demo, a response policy can lower autonomy to approval_required when qualifying drift is detected. Side-effecting tool calls then require human approval. Restoring autonomy requires a human decision.
A real finding: an unapproved release
The demo reported release 0.1.0 while its active contract approved 0.2.0. Brutor raised an unapproved-release finding.
The distinction between declared and verified matters. This demo sends its version in a header. That identifies what the caller reports; it does not independently verify the code running. Brutor’s release tracking keeps declared values separate from verified workload-token claims.
Findings can also point to recorded changes near the time behaviour shifted, giving an operator a lead to investigate. Correlation is not proof of cause. Where no supporting change is known, Brutor leaves the investigation open.
Closing a finding requires a reason. In the screenshot, an earlier caller-mix change was accepted because the platform had changed its caller labels, and the baseline began relearning from that point.
Human oversight can fail too
On one day, the demo’s agent-related signals were within their configured thresholds, but its overall health was Degraded:
The agent had held the actions for approval. Most requests went unanswered. The gate was preventing those actions from proceeding, but the human review process was not keeping up. Brutor made that operational gap visible.
Silence needs attention as well. The demo is expected to run at least once every half hour. If it misses that expectation, its liveness signal raises a finding. An absence of traffic is not treated as evidence that everything is healthy.
What the evidence can tell you
The contract records what was approved. The run ledger records observed activity. Check results and the assurance report describe what was assessed and where evidence is insufficient. Findings record exceptions; resolutions record who reviewed them and what they decided.
Together, these support operational reviews and discussions with auditors and insurers. The scope still matters:
- Normal behaviour does not prove correct answers. Task quality and decision correctness still need evaluations and outcome checks.
- Coverage follows the governed path. Calls that bypass the gateway are outside that evidence; the report states its coverage limits.
- Detection has a timescale. Call-level controls act in the request path. Drift and liveness findings depend on their checks, thresholds and observation windows.
- Evidence supports an assessment. It does not by itself establish legal compliance or insurance cover, or replace testing before release.
The aim is to let an agent perform approved work while giving its owner evidence of how it is operating, notice of detected changes and controls to respond.
See it on your own agent
- Run the open-source Brutor Demo System against the Brutor trial.
- Read the companion white paper, AI Systems Assurance.
- Talk to us about defining an operating contract for your agents.
