An agent's audit trail should let someone outside the build team reconstruct any single decision: what came in, what the agent checked, which rule and tolerance applied, what it decided and why, what it changed in which system, and which person reviewed or overrode it. Logs written for engineers rarely do this on their own.
Who this is for
Finance, HR and operations leads who are about to put an agent into a live process, and the controllers or internal auditors who will be asked to sign off on it. If you are still at the pilot stage, this is the list to agree before the pilot, not after.
A log is not an audit trail
Most systems already log. The trouble is who the log was written for. Engineering logs answer questions like "did the call time out" and "which version was running". An auditor asks different questions. Why was this invoice paid? Who decided the variance was acceptable? Was the rule that allowed it the rule in force that day?
An audit trail is the set of records that answers the second kind of question for one case at a time. The test is simple. Pick a decision at random, hand its record to someone who has never seen the system, and ask them to explain what happened. If they need to call the build team, the trail is incomplete.
Agents make this harder than older automation did. A fixed script follows one path, so the path is the explanation. An agent chooses its next step, which means the record has to carry the reason for each choice, not only the outcome.
The eight fields every decision record needs
We write one record per decision, not one per run. A run that processes 300 invoices produces 300 records, each readable on its own.
| Field | What it holds | The question it answers |
|---|---|---|
| Case and time | A case ID, plus start and end time | Which decision are we looking at, and when did it happen? |
| Input | The document or request as received, with a fingerprint so it can be matched later | What exactly did the agent see? |
| Checks made | Every record the agent looked up: the purchase order, the goods receipt, the vendor master | What did it compare the input against? |
| Rule in force | The rule and tolerance applied, with their version and who approved that version | Was it allowed to do this, on that day? |
| Decision and reason | Proceed, hold or escalate, and a plain-language reason tied to the checks | Why this outcome and not another? |
| Action taken | What changed, in which system, under which service account | What did it actually do? |
| Human involvement | Who reviewed it, what they saw, what they changed, and when | Who is accountable for this case? |
| Exceptions | Anything that failed or was skipped, with the fallback used | Did anything go wrong quietly? |
Two of these carry most of the weight. Rule in force matters because tolerances change, and a decision is only correct against the rule that applied at the time. Human involvement matters because it is where accountability lands. A record that says "approved" without a named reviewer is a record nobody will defend.
A worked example: one invoice, two outcomes
Take a three-way match where, for this example, the finance lead has set and signed a price tolerance of 5%. Two invoices from the same vendor arrive on the same morning.
- Invoice A is 4% above the purchase order price. The record shows the PO and goods receipt the agent matched, the tolerance version in force, and the decision: proceed. The reason reads: "Price variance 4%, within the 5% tolerance approved by finance lead, version 3." No human touched it, and the record says so.
- Invoice B is 22% above the PO price. The record shows the same checks, the same tolerance version, and the decision: escalate. The reviewer's name, the time they opened the case, and their call (hold, request credit note) are all on the same record.
An auditor sampling either case gets the full story from one page. Neither record needs the engineer who built the agent.
What regulation already asks for
Most finance and operations agents are not classed as high-risk under the EU AI Act. Some HR agents can be. Annex III lists systems used "to make decisions affecting terms of work-related relationships" and to evaluate candidates, among others. If an agent falls in that list, the Act is specific about logs:
High-risk AI systems shall technically allow for the automatic recording of events (logs) over the lifetime of the system.EU AI Act, Article 12(1)
Article 26 puts duties on the company using the system, not only the vendor. Deployers must keep those logs for "at least six months" unless other law says otherwise, and must assign oversight to people "who have the necessary competence, training and authority". Retention rules for financial records under national law are often longer, so the longer period usually wins.
Outside the EU, the NIST AI Risk Management Framework makes the same point in general terms. It lists accountability and transparency as a characteristic of trustworthy AI, including "supporting attribution of the AI system's decisions". The eight fields above are one practical way to meet that.
This is not legal advice. If an agent touches hiring, performance or credit decisions, check its classification with counsel before go-live.
Where it is stored and who can change it
- The agent writes, but cannot edit. Records should be append-only. An agent that can rewrite its own history has no history.
- Keep it in your systems. The trail belongs to the business that is accountable for the decisions, not to the vendor that built the agent.
- Read access for audit, by default. Internal audit should be able to pull a sample without raising a ticket.
- Mind personal data. HR and customer cases carry personal details. Record what the decision needs, mask the rest, and apply the same retention rules as the source records.
Where this breaks
A trail can be complete and still useless. Three failure patterns come up often.
- Too much text. Dumping every model output into the record buries the reason. The reason field should be one or two sentences a controller can read in ten seconds.
- Reasons written after the fact. A reason generated when someone asks is a story, not a record. It has to be written at the moment of the decision, from the checks actually made.
- The right record of the wrong rule. A perfect trail of decisions made under a bad tolerance is still a trail of bad decisions. The record makes the problem visible. It does not fix it. Review the rules themselves on a schedule.
From our builds
When we design an agent, the decision record is one of the first things we write, before the agent itself. We sit with the process owner and the person who will audit the process, and agree what a good record looks like for three real cases: one clean, one borderline, one that should escalate. Then we build to that. The invoice walkthrough on our home page follows this shape. In our experience, the conversation about the record surfaces disagreements about the rule faster than any other step.
Questions people ask
Do we need to store the model's full reasoning?
No. Store the inputs, the checks made, the rule applied and a short reason tied to those checks. Long reasoning text is hard to review and can carry personal data you did not mean to keep.
How long should we keep agent decision records?
At least as long as the source records they relate to. For high-risk systems under the EU AI Act, deployers must keep logs for at least six months, and finance retention rules under national law are often longer.
Is a vendor dashboard enough?
Only if you can export every record, keep it in your own systems, and still read it after the contract ends. If the trail lives only in the vendor's tool, you do not control your own evidence.
Who should review the record design?
The process owner, because they own the rule, and someone from internal audit or finance control, because they will test it. Agree it with both before go-live.
Sources
- EU AI Act, Article 12: Record-keeping, European Commission AI Act Service Desk
- EU AI Act, Article 26: Obligations of deployers of high-risk AI systems, European Commission AI Act Service Desk
- EU AI Act, Annex III: High-risk AI systems, European Commission AI Act Service Desk
- NIST AI Risk Management Framework, section 3.4: Accountable and transparent, US National Institute of Standards and Technology
How this note was made: researched and drafted with AI assistance, then reviewed and edited by the Unwired team. Read our editorial policy.