Every enterprise AI roadmap has a slide that says "Data Governance" in the middle somewhere. Nobody reads it. Everybody nods. Then the project starts, and within six weeks, the project is stuck in legal review, and everyone wonders what happened.
Data governance is the boring thing that is also the reason nothing ships.
This post is the practical checklist we use at Sprintt when we engage with an enterprise that wants to ship AI but keeps getting stopped. It is opinionated, it is narrow, and it works.
The governance question is really three questions
The phrase "data governance for AI" conflates three distinct problems that have distinct solutions. If you don't separate them, you will try to solve all three with the same policy, and you will fail at all three.
Question 1: Training data governance
"Can we use this data to train or fine-tune a model?"
This is the question that most policies were written for. It is mostly a legal and contractual question. Rights, licensing, consent, PII handling. The answer tends to be a no-go decision before any code is written.
For most enterprises: you are not training models. You are using frontier models via API. This question mostly doesn't apply to you, and treating it like it does is the source of half the friction.
Question 2: Inference-time data governance
"Can we send this data to a model at inference, in a prompt, to get an output?"
This is the question that actually matters for 95% of enterprise AI work. It is a runtime question. Every time a user asks an AI assistant a question, and your system composes a prompt that includes customer data, policy documents, or internal knowledge — that's inference-time data flow. That's where the real governance action lives.
Most existing policies do not address this explicitly, because they were written before prompts were a thing. This is the gap we spend most of our time helping clients close.
Question 3: AI output governance
"Can we trust what the model said, and is the business willing to stand behind it?"
This is the question downstream of the other two. Even if your inputs are perfect, the model's output has to be governed — reviewed, filtered, logged, attributable. "Who said this" becomes a real question when the answer is "our AI assistant," and your legal team will want an answer before you ship to real customers.
These three governance problems are different. The tooling is different. The stakeholders are different. The risk is different. Conflating them is why enterprise AI projects get buried under generic policy debates that never resolve.
The practical checklist
Below is the checklist we use with clients. It is not exhaustive. It is the subset that matters for most enterprise engagements.
Data classification, but simpler
Every enterprise has a data classification scheme. Most are too granular for AI use. For inference-time flow, you really only need three buckets:
- Red: Cannot be sent to a third-party AI model under any circumstance. (e.g., unredacted PII, protected health information, attorney-client privileged material, trade secrets under specific legal frameworks.)
- Yellow: Can be sent, but only through approved, contractually-bound channels with specific handling. (e.g., customer data under an active DPA, internal financial data, non-public product roadmap.)
- Green: Can be sent freely. (e.g., public marketing content, documented policies, non-sensitive operational data.)
The trick is not to elaborate the scheme. The trick is to be able to classify a new data source in thirty seconds using this scheme, and to have the tooling enforce the classification at the point of prompt composition.
If your system can't look at the data it's about to send and say "this is Red, blocked" in code, you don't have governance. You have a PDF.
The inference-time data flow diagram
Every AI system you ship in an enterprise needs a diagram that shows: what data leaves the tenancy, what model receives it, what contract governs the relationship, what retention applies, what logging is in place.
This is not a compliance document you file and forget. It is a running artifact that changes every time you add a new retrieval source or a new tool. If you haven't updated the diagram in three months, your system is probably out of compliance with its own documented design.
The diagram should answer, per data flow:
- What data classification bucket is this?
- Which model / provider / endpoint is it going to?
- What contract covers it? (MSA, DPA, BAA if applicable.)
- How long is it retained, by whom, under what key?
- Is it logged? Where? Who can see the logs?
- What is the incident-response path if it's exfiltrated?
Most "AI governance policies" are a preamble. This diagram is the substance.
Model provider selection isn't just about capability
Most teams pick a model based on benchmarks or cost. For enterprise deployments, the selection is 60% contracts.
Practical questions to ask every provider, on the record:
- Do you train on customer data by default? (Answer must be no.)
- What is the retention policy for prompts, outputs, and logs? (Ideally: zero-retention or configurable.)
- What subprocessors do you use? (You need the list. It will be longer than you expect.)
- What jurisdictions does the data flow through? (Matters for GDPR, data residency.)
- What audit and certifications do you hold? (SOC 2 Type II is table stakes; HIPAA BAA and ISO 27001 matter for specific contexts.)
- What is the breach-notification SLA?
If a provider dodges any of these questions or refuses to contract around them, they are not an enterprise provider, regardless of how good their model is. There are now multiple providers who will answer all of these to the satisfaction of a Fortune-500 legal team. There is no longer a "we have to use OpenAI because nothing else works" excuse.
The retrieval layer is where real governance lives
Every RAG system you build has a retrieval layer that pulls documents and sends them to a model. This layer is where most of your governance failures will happen.
The checklist for the retrieval layer:
- Access control carries over: if a user isn't authorized to see document X, the retrieval layer must not return document X even if it's semantically relevant. Use the same ACLs that govern direct access. Do not build a parallel permission model.
- Provenance is tracked: every retrieved document has a pointer back to its source. If the model's output includes the document's content, you can trace which document.
- Sensitive data is filtered pre-retrieval: PII, credentials, private keys should be stripped from documents at indexing time. Once they're in the vector store, they're harder to remove.
- Audit logs exist: every retrieval is logged with user, query, documents returned, and timestamp. This is your forensic trail if something goes wrong.
Output governance: the quiet killer
Your AI system produces output. That output is a statement your company is, in some sense, making. Legal will eventually ask: "who is responsible for what the AI said?"
The answer cannot be "the AI." It has to be a person or a process.
The basic pattern we use:
- Medium-stakes outputs go through a review layer: a rule-based filter that blocks obvious failures (profanity, protected-class discussion, regulated claims) before they reach the user.
- High-stakes outputs require human review: if the AI is drafting a customer contract, making a medical claim, or giving financial advice, a human approves before it ships.
- Every output is logged with full context: prompt, retrieval, output, model version. If a customer later says "your AI told me X," you need to be able to prove or disprove it.
This is not a limitation of AI. This is how every risk function in every regulated business has always worked. AI doesn't get a pass because it's new.
The pattern that gets you unstuck
Most enterprise AI programs we audit have the same underlying dysfunction. Governance is treated as a gate — the step that must be passed before the project can start. Which means governance reviews happen all at once, at the beginning, for an abstract future system, in front of stakeholders who are skeptical.
The gate never opens.
The pattern that works is the opposite: governance by instantiation. Pick a scoped use case. Define the data flows for that specific use case. Get governance sign-off on that use case. Ship it. Use what you learned to define the next use case.
Each approval is small. Each approval is concrete. Each approval happens against a running system, not a whiteboard diagram.
After the third or fourth use case, you have a body of approved patterns and a governance team that is bought in. After the tenth, you have a framework. Nobody starts with a framework. They earn one.
What to do Monday morning
If your AI roadmap is currently stuck in governance purgatory, here is what to do:
- Pick one scoped use case. Not the most strategic one. The most shippable one.
- Draw the data flow diagram. Just that one.
- Classify every data element as Red / Yellow / Green. Just for that use case.
- Identify the one or two governance gaps that block this use case. Nothing else.
- Take those two gaps to legal/compliance with a proposed remediation. Ask for approval only for this use case.
- Ship.
- Repeat, with a more ambitious use case next time.
The goal is not to solve AI governance for your organization. That project is infinite and never ships. The goal is to solve governance for the next AI system you ship, and the next one, and the next one. After enough iterations, your organization has a governance model that is grounded in real deployments, not abstractions.
That is how enterprises actually become AI-native. Not through a policy. Through accumulated delivery.
Sprintt helps enterprises work through governance blockers by scoping AI engagements to the smallest shippable unit and pressure-testing them through legal and compliance in parallel with build. If your AI roadmap keeps dying in review, book a 30-minute call.
