Overview

You need governance. So you build a process. Every AI project requires approval. Every model needs formal evaluation. Every output needs human review. Now nothing ships.

Or you go the other way: no governance. AI teams do whatever they want. Models get trained on biased data and ship to production. Customer data gets misused. Regulatory violations pile up. The company gets sued.

The right answer is between these extremes. Governance that's light enough to not kill innovation but substantial enough to catch real problems. Governance that prevents catastrophic failures, not all failures. Governance that's documented and defensible if regulators ask questions.

This lecture is about being smart about risk, not bureaucratic about process. We'll cover a framework that scales with risk. We'll discuss what governance actually looks like operationally. We'll talk about who owns it and how to build buy-in from engineering.

What Governance Actually Means (And What It Doesn't)

Let's be clear about terms because everyone uses them differently.

Governance is NOT:

• Paperwork for its own sake
  - Slow approvals and bottlenecks
  - Committees that block everything
  - Slowing down shipping
  - Preventing all risk (impossible)

Governance IS:

• Understanding what can go wrong
  - Having documented controls
  - Being able to explain decisions if something goes wrong
  - Catching real risks before they happen
  - Scaling oversight to match risk
  - Clear accountability for problems

Good governance characteristics:

• Takes less than a week for most projects
  - Catches real risks without being paranoid
  - Allows innovation and speed
  - Can be explained to a regulator and makes sense
  - Built into process, not bolted on

Bad governance characteristics:

• Takes months, kills shipping velocity
  - Prevents any risk (impossible, so it's constantly overruled)
  - Requires permission for everything from non-technical people
  - No one understands the point
  - Creates separate approval track

The Governance Principle: Governance is about understanding risk and having documented controls, not about preventing risk entirely or slowing down shipping. Scale governance to match risk. Low-risk projects move fast. High-risk projects take more time. The goal is acceptable risk, not zero risk.

The Governance Framework That Works

1. Risk Categorization (Essential First Step)

Not all AI risks are equal. Categorize by impact. Put 80% of effort on 20% of high-risk applications.

High Risk (Requires Formal Review):

• AI makes significant decisions affecting individuals (hiring, lending, medical diagnosis, insurance underwriting)
  - Output bias could cause real harm to protected groups
  - Uses sensitive personal or health data
  - Impacts life chances or legal status
  - Examples: hiring recommendations, loan approval, criminal risk assessment, medical diagnosis

Medium Risk (Requires Basic Documentation):

• Impacts customer experience or revenue but not individual life chances
  - Bias is problematic but not catastrophic
  - Uses customer data but not health/financial
  - Examples: content recommendations, pricing optimization, support routing

Low Risk (Light Documentation):

• Ranking/recommendations with no individual impact
  - Non-sensitive data
  - Bias doesn't cause harm
  - User can easily override
  - Examples: search ranking, suggested tags, content ranking, auto-save

2. Pre-Launch Requirements by Risk Level

HIGH-RISK REQUIREMENTS (4-8 Week Review):

• Model card: what it does, how it was built, known limitations, training data source
  - Bias evaluation: formally test for disparate impact on protected groups
  - Data audit: where did data come from? Is it representative? What are the biases in source data?
  - Human review process: how do we catch and correct errors in production?
  - Explainability: can we explain decisions to users?
  - Stakeholder review: product, legal, ethics, affected domain (HR for hiring, risk for lending, etc.)

MEDIUM-RISK REQUIREMENTS (2 Week Review):

• Model card (simpler version)
  - Basic bias testing (spot check for disparate impact)
  - Data source documentation
  - Monitoring plan
  - Product review

LOW-RISK REQUIREMENTS (2-3 Days):

• One-page model card
  - No formal evaluation required
  - Quick product check

3. Ongoing Monitoring (By Risk Level)

High-risk systems:

• Weekly monitoring for bias/performance drift
  - Monthly human review of actual outputs
  - Quarterly formal evaluation
  - Clear criteria for when to pull the plug (accuracy drops below X%, disparate impact detected, etc.)

Medium-risk systems:

• Monthly monitoring
  - Quarterly human review
  - Annual re-evaluation

Low-risk systems:

• Basic monitoring (alert if performance drops significantly)
  - Annual check-in

4. When to Pull the Plug (Clear Criteria)

Define upfront, before something goes wrong. Don't make crisis decisions.

• Accuracy drops below X% (define threshold pre-launch)
  - Disparate impact detected on protected group (define threshold)
  - Data drift shows model isn't applicable anymore
  - Error rate spikes above expected
  - Regulatory or compliance violation
  - Negative press or customer backlash

Have these discussions in the governance committee before deployment. If you have to decide in a crisis, you won't make a good call.

Failure Modes: When Governance Breaks

Failure Mode 1: Governance is Too Heavy

High-risk system approval takes 4 months. By the time approval happens, the business need has changed. The system is irrelevant. Nobody uses the platform because shipping takes too long.

Fix: Parallel approval. Teams start building while governance happens. Committee meets weekly, not monthly. Approval is "approved with monitoring" not "approved then deploy." Get feedback early, not late.

Failure Mode 2: Governance is Too Light

No governance. Teams build what they want. One team ships a system with serious bias. Customers complain. Regulatory issues. Now you're in crisis mode fixing something that good governance would've caught.

Fix: Right-size governance to risk. Low-risk systems: light touch. High-risk systems: real review. This isn't one-size-fits-all.

Failure Mode 3: Governance Becomes Theater

You have a governance committee. They meet and talk. But they don't actually make decisions. Teams go around them. The committee becomes powerless. Or worse, the committee blocks everything and becomes a bottleneck everyone hates.

Fix: The committee has to have real authority AND they have to use it wisely. They can say "no" but rarely. They accelerate shipping when it's safe. They catch real risks without being paranoid.

Real Case Study: Governance Enabling Speed

A Series C company had no governance. Teams shipped what they wanted. By year 3, they had 50 models in production. No one was monitoring them. One model had serious bias affecting a protected group. When discovered, they had to pull it. Crisis ensued.

They built governance: risk categorization, committee, requirements by risk level. High-risk (hiring, lending, moderation): formal review. Medium-risk (recommendations): documentation and monitoring. Low-risk (tags, ranking): light touch.

Result: They could move faster safely. High-risk systems took 6 weeks instead of 2 weeks for old model, but they didn't break in production. Medium-risk moved in 2 weeks. Low-risk moved same day. They doubled shipping velocity while reducing risk. Governance enabled speed, not stopped it.

Governance in Practice (Real Example)

Month 1: Pre-Launch Decision

Team says: "We want to launch AI for hiring recommendations. Help us think through governance."

You say: "That's high-risk. Hiring decisions affect life chances. Here's what we need: model card, bias testing on demographics (race, gender, age where data allows), data audit showing training data is representative, and a human review process for all recommendations."

They say: "Can we launch with just model cards?"

You say: "No. Bias testing is non-negotiable here. Here's why: if we accidentally create a system that recommends against women or minorities at higher rates, that's discrimination and regulatory violation. The testing catches that before it reaches customers."

They say: "Can we do a pilot first?"

You say: "Yes. Pilot with 1% of candidates, continuous monitoring, clear stopping criteria. If we see issues, we stop. If performance is good, we expand to 5%, then 10%."

Months 2-7: Iteration

Team builds model. Governance committee has bi-weekly reviews. They discover that the model flags certain candidates at different rates by demographic group. They iterate on data and model. Committee evaluates each iteration.

Month 8: Launch Decision

Bias testing shows no significant disparate impact. Human review process is in place. Monitoring is set up. Clear stopping criteria defined.

You say: "Launch 1% pilot. Monitor for one month. If metrics hold, expand to 5%."

The result: 8 weeks from start to 1% launch. That's reasonable for a decision affecting people's employment. It's not fast, but it's not bureaucratic.

The point: Governance isn't "no" all the time. It's "here's how we do this thoughtfully."

Who Owns Governance?

Not Legal. Not Compliance alone. They have a role but they don't own it.

The AI Governance Committee (3-5 people):

• Head of AI/ML (co-chair), understands technical capability and limitations
  - Senior product manager (co-chair), understands customer impact and business goals
  - Data scientist or engineer (technical perspective), can evaluate model approaches
  - General counsel or compliance (legal perspective), understands regulatory landscape
  - Domain expert from affected area, hiring expert for hiring AI, risk expert for lending AI, etc.

Committee responsibilities:

• Reviews high-risk applications before launch
  - Monitors ongoing applications quarterly
  - Updates governance guidelines based on what they learn
  - Escalates issues (model failure, bias detected, data compromise)
  - Makes go/no-go decisions for pilots and full launches

Meeting cadence: Monthly routine meeting. Emergency meetings as needed.

Key principle: The head of AI co-owns this with product. It's not something Legal imposes. It's something the engineering org does because it's the right thing. This builds buy-in.

The Ownership Principle: Governance works when the engineering org owns it (with legal and compliance supporting). It fails when Legal tries to impose it top-down. Make it collaborative, not adversarial.

Right-Sizing Governance to Risk: The biggest governance failure is treating all AI the same. High-risk systems (hiring, lending, medical) need rigorous review. Low-risk systems (ranking, tagging, content suggestions) need light oversight. Proportional governance prevents both bureaucratic paralysis and risky shortcuts.

What to Do Monday Morning

Step 1: Categorize your AI applications. Which are high-risk? Medium? Low? List them out. Get agreement from product and legal.

Step 2: Define requirements by risk level. What needs to happen before launch? What monitoring is needed? Be specific about thresholds.

Step 3: Assemble governance committee. AI lead, product, engineer, legal, domain expert. Get 30 minutes on their calendar monthly.

Step 4: Communicate to teams. "Here's the governance process. High-risk takes 8 weeks. Medium takes 2. Low takes a few days. It's not to slow you down. It's to make sure we get it right."

Step 5: Create templates and processes. Model card template. Bias testing checklist. Monitoring dashboard template. Make it easy to comply.

Step 6: Iterate governance itself. After 3-6 months, review. Is governance working? Are we catching risks? Is it too heavy? Are teams shipping? Adjust. Governance is a tool. If it's not working, change it. Don't let process become cargo cult.

FAQ: Governance Questions

Q: What if the board wants more governance than our teams can handle?

A: Show them the tradeoff. "More governance means slower shipping. Right now we're 4-8 weeks for high-risk. Doubling requirements pushes it to 12-16 weeks. Is that what we want?" Usually the answer is no. Help them understand you're balancing risk and speed.

Q: Should we require Legal approval on all AI?

A: No. Legal should be in the room for high-risk. But for low-risk, it's overkill and slows you down. Right-size governance to risk.

Q: What if we find bias in production and it reaches customers?

A: You pull the model. You communicate clearly to affected users. You fix it. You do post-mortem. This happens sometimes even with governance. The point isn't to make it impossible. It's to make it unlikely and to respond well when it happens.

Q: Who pays for governance (people, tools, time)?

A: Usually split: engineering pays for model cards and documentation (part of development cost, ~5-10% of project). Legal/compliance pays for governance committee (~0.5 FTE). Data science pays for monitoring infrastructure (~1 FTE for mature deployments). It's a shared responsibility. For a $2M AI project, expect governance to cost $100-200k. If it saves you from one lawsuit, it pays for itself 10x over.

Q: How do we keep governance from becoming theater?

A: Make it matter. Have the committee actually make decisions. Be willing to say "no" or push back on risky projects. Show that governance has teeth. Publicly celebrate teams that follow governance (they ship faster, safer). Call out teams that try to work around it. Otherwise it becomes compliance theater and teams will work around it.

Q: What if the board is skeptical and thinks governance will slow us down?

A: Show them data. "Without governance, we had bias incidents, regulatory issues, 3 months of crisis response. With governance, we catch risks early, we ship faster safely, and we have zero incidents." Frame it as risk management, not bureaucracy. Show that companies with good governance move faster, not slower, because they don't crash in production.

Key Insight

Governance scales with risk, not bureaucracy. Categorize AI as high-risk (hiring, lending, medical, needs bias testing, data audit, 6-8 weeks), medium-risk (recommendations, 2 weeks), or low-risk (ranking, tagging, 2-3 days). Governance committee includes AI lead, product, engineer, legal, and domain expert. This approach enables speed while catching real risks.

Governance as Responsibility

Good governance isn't about being cautious or protective. It's about being responsible. Understanding what can go wrong. Having controls. Being able to explain your decisions.

Do that and you can ship fast and safely. Regulators will trust you. Customers will trust you. You'll trust yourself.

On This Page

Introduction
What Governance Means
Framework
Failure Modes
Case Study
In Practice
The Committee
Monday Morning Action
FAQ
Key Takeaway
Responsibility

Chapter Details

Part ofCh 4: AI Governance and Compliance