The AI compliance software category grew out of GRC tooling, and it inherited that legacy more completely than most buyers realize. The underlying model is control attestation. You declare a control, you map it to a framework, somebody attests that it’s in place, and the tool tracks the attestation, the owner, and the renewal date.
For SOC 2 and ISO 27001 this works, because the controls in question are largely static configurations. Encryption is either on or it’s off. Access reviews either happened or they didn’t. A screenshot settles it.
AI systems don’t produce configurations. They produce decisions, one at a time, thousands of times a day. And each decision is contingent on data that changed since yesterday and a model version that will be deprecated by next year. When a regulator, a customer, or opposing counsel asks about any one of them, the question is never about whether you had a control in place or not. The question is what happened on March 14th at 9:42pm, and whether it was defensible.
No attestation answers that. Which means the AI compliance software you bought is tracking a different problem from the one you’ll be asked about. Let’s explore where the gaps are and what you need to do.
What does AI compliance tooling actually do today?
Most AI compliance software products do four things competently, and it’s worth being precise about what those four things are before criticizing the category (which I’ll inevitably be doing at some point) .
- They maintain a register of AI systems, usually populated by manual entry.
- They map controls to frameworks, so a single control satisfies a clause in the EU AI Act, a NIST AI RMF subcategory, and an ISO 42001 requirement simultaneously.
- They run workflow, routing risk assessments and approvals to the right people on a schedule.
- They produce reporting, giving a board or an auditor a view of coverage and gaps.
All four are actually useful. But none of them touch the running system. The register is populated by somebody typing. The control mapping describes intent. The workflow moves documents. And the reporting summarizes the documents. At no point does the tool read anything the AI system produced.
That’s the structural limit, and it’s not a product deficiency so much as a category boundary. Compliance software sits at the organizational layer. Evidence is generated at the runtime layer. Workflow tooling can't bridge that gap no matter how good the workflow is. Which is why buyers who evaluate these products on features rather than on data source end up surprised during their first real audit.
Why does control attestation break down for AI systems?
Attestation assumes the thing being attested to holds still. That assumption is load bearing, and it’s false for every system whose behavior depends on data that changes. For example, a control like quarterly access review is stable. It happened or it didn’t, and the state persists until the next review.
A control like human oversight of AI outputs is not stable in the same way. It can be true in January and functionally false by June because the reviewer was either reassigned, or the volume tripled, or the queue got batched, and none of those changes generate an event that triggers a re-attestation.
Stanford's 2026 AI Index illustrates how quickly things can change. On the Foundation Model Transparency Index, average scores rose from 37 to 58 between 2023 and 2024, but then fell to 40 in 2025, with persistent gaps around training data, compute, and post-deployment impact. If the disclosure posture of the models underneath your systems can move that far in a year, an annual attestation about how those systems behave is describing a state that no longer exists.
Regulators have already moved in a direction attestation can't follow. Article 12 of the EU AI Act requires high-risk systems to automatically record events across their lifetime at a level supporting traceability. And Article 26(6) requires deployers to retain those logs for at least 6 months. The obligation is on the system to generate records, not on a person to affirm that records would be generated.
Can a policy document count as evidence?
It counts as evidence of a policy. It doesn’t count as evidence of an outcome. And the distinction is where most programs discover they’re short. Picture the sequence in an actual examination. The examiner reads your AI policy and finds it reasonable. Then they select three decisions from the past year and ask you to walk through each one:
- What data did the system see?
- Which model version ran?
- What did the retrieval layer return and from which source documents?
- Who reviewed it, and what would have happened if they had disagreed?
Your compliance platform can produce the policy instantly. It can't produce any of the four answers because it never had access to them. The answers live in application logs that weren’t designed as evidence and are mutable. Oh, and they typically retain the prompt and the completion, while discarding the retrieved context and the index state that actually determined the output.
This is the substance behind AI auditability as a design property. A decision is reconstructible or it’s not, and that gets decided during the build rather than during the audit. Retrofitting covers monitoring and review going forward, but decisions already made without a decision-level record stay permanently irrecoverable.
Where should AI compliance evidence come from?
The useful evaluation question for any AI compliance software product is short. Where does this tool get its facts? Everything else on a feature comparison follows from the answer, and none of them substitutes for it.
If the answer is a person filling in a form, the tool is a document management system for compliance artifacts. That has value, and it’s worth buying for that value, but it won’t survive an evidence request.
If the answer is the system itself, through an API into decision records, retrieval logs, evaluation results, and entitlement checks, then the tool is doing compliance work that attestation can't do.
Four data sources make the difference in practice:
- Decision records joined by correlation ID covering input
- Retrieved context with source lineage
- Model version, configuration, and output
- And reviewer identity
Reviewer identity is confirmed with entitlement telemetry showing whose permissions were enforced at query time rather than at index time. Evaluation results are tracked as a time series against production configuration rather than a test environment. And override records showing not just that a human reviewed something, but how often they disagreed and what happened next.
Only one of those four is commonly available from a compliance platform today. The other three come from the data layer the systems run on, which is the argument for treating evidence as a platform property rather than a reporting feature. Our approach to audit-ready reporting in regulated industries starts from that constraint rather than working backwards from a dashboard.
Why do compliance tools fail their first agentic deployment?
Agentic systems break the register model, and the register model is the assumption underneath nearly every product in the category. A register assumes a system is a discrete thing with a name, an owner, a purpose, and a boundary.
An agent that calls four tools, queries three data sources, and hands off to another agent doesn’t have a boundary in that sense. Registering it as one row understates it. Registering every path through it is unmaintainable by hand. And ironically, hand maintenance is exactly what these tools rely on.
The permission surface is where this gets expensive. Most retrieval layers authenticate through a service account provisioned during a pilot with broad read access. This quietly collapses the entitlement model for every user downstream. No compliance tool detects that, because nothing about it appears in a form. It appears in the identity the query ran under, which is runtime data.
IBM's 2025 breach research found that 97% of organizations reporting an AI-related security incident lacked proper AI access controls. The same problem repeats as agents multiply. You can get more on the mechanics around this in our writing on secure AI deployment and on agentic systems specifically.
What should you ask an AI compliance vendor?
There are 5 questions that separate evidence generation from documentation management, and they’re worth asking in a demo rather than in a security review:
- Ask them to reconstruct a specific decision from a live system, not to show a dashboard about decisions.
- Ask what happens when a model version is deprecated by the provider and whether the tool depends on re-running inference.
- Ask whether their inventory is populated by discovery or by data entry, and what percentage of a typical customer's AI footprint they find that the customer had not registered.
- Ask whether they capture the identity that retrieval ran under.
- And ask what they store that a subpoena could rely on, given that mutable application logs are not evidence in any meaningful sense.
A vendor answering the first question with a coverage report is answering a different question. That’s not necessarily disqualifying, but it tells you what you’re buying and what you still have to build.
The gap is broadly acknowledged yet slowly addressed. Grant Thornton found that just 22% of Ops leaders reported a fully developed and implemented AI strategy. Most organizations are buying tooling ahead of having decided what the tooling is meant to prove, which is how you end up with a well-run process that generates the wrong artifacts.
What replaces the compliance checklist
Not a different checklist. A different source of truth. Which is a harder change to make and a much cheaper one to live with afterwards. The workable pattern puts evidence generation in the runtime, and uses compliance software for what it is good at.
Decision records, entitlement enforcement, evaluation history, and override tracking are all produced by the system as a byproduct of operating. The compliance layer reads them. And nobody attests to anything they can't see.
That inversion also fixes staleness. Anything a person has to remember to record won’t survive a QBR. And anything that degrades silently will…you guessed it, degrade silently. Evidence generated automatically is current by construction, which matters most where data quality itself is the upstream variable. Especially since bad data produces errors and erodes trust.
If I’m being honest, you likely have a compliance problem. It’s just downstream of the evidence problem, and buying software for the first one won’t fix the second. Uncontrolled AI is a business risk, and the control that matters is the one that leaves a record.
Let's talk soon.
FAQs
What is AI compliance software?
AI compliance software is tooling that maintains an inventory of AI systems, maps controls to regulatory frameworks such as the EU AI Act, NIST AI RMF, and ISO 42001, routes risk assessments and approvals through workflow, and reports coverage to boards and auditors. Most products in the category inherit their model from GRC tooling built for control attestation, which means they track declared controls rather than reading evidence from the AI systems themselves.
Is a policy document sufficient evidence for an AI audit?
No. A policy evidences the existence of a policy. An examiner selecting specific decisions will ask what data the system saw, which model version ran, what the retrieval layer returned and from which sources, and who reviewed the output. None of those answers exist in a policy document or in a compliance platform populated by manual entry. They exist only in decision-level records generated by the system at the time the decision was made.
What is the difference between AI governance software and AI compliance software?
The categories overlap heavily in practice and most vendors sell into both. Governance tooling tends to emphasize policy definition, approval workflow, and ownership assignment, while compliance tooling emphasizes framework mapping, control attestation, and audit reporting. The more useful distinction for buyers is not the label but the data source, since both categories are dominated by products that read from forms rather than from running systems.
What should you ask an AI compliance vendor during evaluation?
Ask them to reconstruct a specific decision from a live system rather than show a dashboard. Ask whether reconstruction depends on re-running inference against a model version the provider may deprecate. Ask whether inventory is populated by automated discovery or manual entry, and what share of a typical customer's AI footprint they discover unregistered. Ask whether they capture the identity that retrieval authenticated under. And ask what they store that would hold up as evidence, since mutable application logs generally do not.
Can compliance tooling detect shadow AI or unregistered systems?
Only if it performs runtime or network discovery rather than relying on manual registration. A register populated by data entry contains exactly the systems somebody chose to enter, which by definition excludes unsanctioned deployments. This is the most common blind spot in the category, and it matters because an incomplete inventory is usually the first finding in an examination rather than a minor gap.
.svg)