Can you prove what your AI system did?
Most teams that put a model in front of their data cannot answer four questions: what did it see, why did it answer that, who was allowed to see it, and how would you know if it got worse. We build the parts that produce those answers, as engineering evidence rather than a policy document.
Talk to an engineerWhen teams call us
- An assistant answered a customer with something that was not in any approved source, and there is no record of what it retrieved.
- Someone asks which model version produced an answer from three months ago, and the honest reply is that nobody knows.
- Retrieval indexes everything together, so an answer for one user can be built from a document another user was not allowed to open.
- A prompt was improved last week and nobody can say whether the system is better or worse than before the change.
- Legal is asking for evidence of controls and engineering is sending screenshots.
- There is a compliance deadline, a policy document, and nothing in the system that produces the records the policy promises.
What we build
Records that answer ‘what did it see’
Every answer tied to the documents that produced it, with enough retention to still be there when someone asks. Article 12 of the EU AI Act requires high-risk systems to log automatically across their lifetime; Article 26 requires deployers to keep those logs at least six months. Both are engineering requirements, not policy ones.
Versioning that survives the question
Model, prompt, retrieval index and configuration recorded together with each output. Without this, ‘why did it say that’ is unanswerable three months later, because everything involved has changed since.
Permission-aware retrieval
Filtering applied at retrieval, not after generation. This is where most systems leak: one index for everything, permissions checked too late or not at all. By the time a model has read a document, restricting the answer is already too late.
Evaluation that catches a regression
A fixed question set with expected properties, scored on every change to model, prompt or index. So ‘we improved the prompt’ becomes a measurement instead of an opinion, and a degradation is visible before a user finds it.
Bounded cost and defined failure
Ceilings that stop spend, and explicit behaviour when the model is unavailable or unsure. A system that answers confidently when it should abstain is the expensive kind of wrong.
Typical projects
Evidence gap assessment, from USD 4,000
Two weeks, fixed price, starting at USD 4,000. We take your AI system and try to answer the four questions with what exists today. The output is a written report: what you can already prove, what you cannot, and what each gap takes to close. Self-contained, and yours to hand to anyone.
Retrofit records onto a system already running
Logging, versioning and retention added to something in production, without a rewrite. Reconstructing this later, from a system that has already been running for a year, costs several times what recording it from the start does.
Fix retrieval that ignores permissions
Move access control into the retrieval step, with tests that prove a user cannot receive an answer built from documents they cannot open. Usually the highest-consequence finding, and the one nobody looks for.
Build the evaluation harness
A fixed question set, expected properties, and scoring in CI, so every change to model, prompt or index is measured before deploy rather than after.
What you get
- Records your system produces on its own, not a document describing records it should produce
- Every output traceable to its model, prompt, index version and sources
- Retrieval that enforces permissions, with tests that demonstrate it
- An evaluation set that fails the build when quality drops
- A written description of the controls, in language an auditor and an engineer both accept
Questions we get asked
What does it cost?
The evidence gap assessment is two weeks at a fixed price, from USD 4,000, and it ends with a written report you own. The figure is a floor, not a quote: scope and depth change it, and we agree the number in writing before anything starts. Work that follows the assessment is quoted separately, and most of what an assessment finds is worth fixing whoever fixes it.
Are you lawyers?
No, and you should not take legal advice from us. Your counsel decides which obligations apply to you and how to interpret them. What we do is the part they cannot: building the systems that produce the evidence those obligations require. Most teams already have the legal reading and are missing the engineering.
When do the EU AI Act obligations actually apply?
2 August 2026 is the binding date for the high-risk obligations covering providers and deployers. The most consequential requirements for Annex III systems were deferred to 2 December 2027 by the Digital Omnibus on AI. That deferral is time to prepare, not time off: records describe decisions being made now, and reconstructing them later from a running system costs several times more.
What exactly does the record-keeping requirement mean?
Article 12 requires high-risk AI systems to technically allow automatic recording of events across the system’s lifetime. Article 26 requires deployers to keep the logs the system generates, where those logs are under their control, for at least six months unless other law requires longer. The practical consequence is that logging cannot be added at audit time: the system has to have been producing the records all along.
We are not in Europe. Does this apply to us?
It can. The regulation follows the user, not the company: serving users in the EU can bring you in scope regardless of where you are. Beyond that, enterprise customers increasingly ask for the same evidence in procurement — ISO 42001, SOC 2, data processing agreements — whether or not a regulator is asking.
Can an AI tool not just do this?
It can help build the pieces. It cannot be the evidence. The point of an audit trail is that something independent recorded what happened; a system that certifies itself proves nothing. That is a structural limit, not a gap that closes as models improve.
What if we have no AI system in production yet?
Then this is the cheapest it will ever be. Records, versioning and permission-aware retrieval designed in from the start are a design decision. Added afterwards, they are a migration.
Start a conversation
Tell us what you are building, or what is currently breaking. You will get a straight answer from an engineer, not a sales script.