Learning Objectives

After completing this lecture, you will be able to:

• Understand the key concepts of the human in the loop in a government context

• Connect the human in the loop to your agency's AI initiatives

• Identify next steps for applying these concepts in your role

Key Topics Covered

• Why humans must remain in decision-making
• performative oversight
• Government context for the human in the loop
• Practical applications and next steps

Why This Matters for Government

Government agencies face unique challenges when it comes to AI adoption. This lecture addresses these challenges head-on by providing all government employees with the knowledge and frameworks needed to navigate AI in the public sector responsibly and effectively.

As part of the L1 (AI Aware) curriculum, this lecture builds on the foundational principle that every AI system in government ultimately serves citizens. Whether you are working with AI tools daily or setting strategy for your agency, understanding the human in the loop is essential for responsible, effective government AI adoption.

Lecture URL: https://skill.re/learn/govt/the-human-in-the-loop.php

======================================================================

TRANSCRIPT: The Human in the Loop

======================================================================

What you will learn: Why humans must remain in decision-making. Automation bias. Meaningful vs. performative oversight.

Here's a dangerous assumption: if an AI system is accurate, it can make decisions without human input.

This assumption is wrong. Even highly accurate AI systems should not make important decisions alone. Humans need to remain in the loop.

This lecture is about why, and how to do it right.

WHY THIS MATTERS FOR GOVERNMENT

Government decisions are not just about accuracy. They're about judgment, context, values, and accountability.

AI can be very accurate at recognizing patterns. But human judgment is needed for:

• Context. The AI sees data. The human sees the person and their circumstances.

• Values. How do we balance competing goods? AI doesn't do this. Humans do.

• Exceptions. The person whose case doesn't fit the pattern. The exception that requires human compassion.

• Accountability. When a decision goes wrong, who's responsible? Not the algorithm. The human.

AUTOMATION BIAS

Automation bias is the tendency of people to rely on automated decisions even when they shouldn't.

How Automation Bias Manifests:

• "The AI said so." A human reviewer is presented with an AI recommendation. They assume it's right and don't carefully review it.

• Trusting accuracy metrics. "The system is 94% accurate." Therefore, it's probably right in this case.

• Deferring judgment. "I'm not sure, so I'll go with what the AI says."

• Reduced scrutiny. A decision made by a human gets careful review. A decision made by an AI gets less careful review (or no review).

Why Automation Bias Is Dangerous:

AI systems make errors. Humans make errors. But human errors are sometimes caught by other humans who ask questions. Automated errors are less often caught.

If a human just accepts the AI without question, the error becomes decision final.

MEANINGFUL HUMAN REVIEW

The solution is meaningful human review: genuine human oversight that can override the AI.

What Makes Review Meaningful?

• The human has authority. They can actually override the AI if they think it's wrong.

• The human has information. They have access to the same data the AI had, plus context the AI didn't have.

• The human has responsibility. If they override the AI and the decision goes wrong, they're responsible.

• The human has incentive to review carefully. They're not just rubber-stamping. They have motivation to get it right.

What Meaningful Review Looks Like:

• "Here's what the AI recommended and why. Here's what I think. They differ on [point]. Given this context [situation], I override the AI and recommend [alternative]."

• "The AI recommends [X]. But this person's circumstances fit a pattern the AI might not recognize. I'm escalating to a supervisor for further review."

• "The AI's reasoning seems sound, but it's based on aggregate data. This person might be an exception. I'm requesting additional information before deciding."

What Performative (Fake) Review Looks Like:

• "The AI said [X]. I agree. Approved." (No actual review.)

• "The AI said [X]. I checked and don't see any obvious errors. Approved." (Minimal review.)

• "The AI said [X]. I can't understand why, but it's probably right. Approved." (Abdication of responsibility.)

DESIGNING FOR MEANINGFUL HUMAN REVIEW

To make meaningful human review possible:

• Make AI decisions explainable. Humans can't review what they don't understand.

• Provide context to reviewers. Give them the data the AI saw, plus relevant context.

• Train reviewers. They need to understand the AI system well enough to evaluate its recommendations critically.

• Empower reviewers. Make it easy for them to override the AI. Make override authority clear.

• Monitor override patterns. If reviewers override the AI frequently on certain types of cases, investigate. Maybe the AI is systematically wrong about those cases.

• Build feedback loops. If a reviewer overrides the AI and the override turns out to be wrong, this should feed back into retraining the AI.

ANTI-PATTERNS / MISUSE RISKS

Anti-Pattern 1: "Human Review" That's Rubber-Stamping

An agency has humans "review" AI decisions, but they just check a box saying "approved" without actually reviewing.

Risk: The system functions as if it were fully automated, but the agency pretends there's human oversight.

Anti-Pattern 2: Responsibility Diffusion

"The AI recommended [X], but the human signed off on it, so it's not clear who's responsible if something goes wrong."

Risk: No one feels accountable. Errors aren't addressed.

Anti-Pattern 3: Inadequate Reviewer Training

Humans are supposed to review AI decisions, but they don't understand the AI system or its limitations.

Risk: They can't do meaningful review. They just accept the AI's recommendation.

Anti-Pattern 4: No Authority to Override

A human reviewer thinks the AI made a mistake, but they don't have authority to override it.

Risk: Errors go uncorrected.

Anti-Pattern 5: Viewing Humans as "Bottleneck"

"AI is faster and more efficient if we remove the human from the loop."

Risk: You remove checks and balances. You increase risk of systematic errors and harm.

PRACTICE / REFLECTION PROMPTS

• In your agency, when AI systems are used to make decisions, who reviews those decisions? What does that review involve?

• Have you ever disagreed with an AI system's recommendation? What did you do?

• What would need to be true for human review of AI decisions to be meaningful in your context?

KEY TAKEAWAYS

• Humans must remain in the loop for important decisions. AI should inform, not replace, human judgment.

• Meaningful review requires understanding, authority, and accountability. Not just someone checking a box.

• Automation bias is real. Humans tend to trust automated decisions too much.

• Transparency enables review. If the AI system's reasoning is opaque, meaningful review is impossible.

• Reviewers need training. They need to understand the AI system well enough to evaluate it critically.

• Authority matters. Reviewers need power to override the AI. Otherwise, their review is pointless.

TERMS / GLOSSARY ITEMS

Automation Bias: The tendency to rely on automated decisions even when they shouldn't be trusted.

Meaningful Human Review: Genuine oversight where a human can override an automated decision.

Performative Review: Review that goes through the motions but doesn't actually influence decisions.

Override: A decision by a human to reverse or change an AI recommendation.

Responsibility: Accountability for decisions and outcomes.

Your agency uses an AI system to determine benefit eligibility. The system reviews applications and makes a determination.

Scenario: Meaningful Human Review

• The AI reviews an application and recommends denial.

• The recommendation goes to a caseworker.

• The caseworker reviews: "The AI flagged lack of recent employment history. But this person has been caring for a seriously ill family member, which I know from the notes, and employment might not be feasible. The AI doesn't account for this. I'm overriding and approving."

• The caseworker's decision stands.

• If the person later complains, there's a human who made and can defend the decision.

Scenario: Performative Review

• The AI reviews an application and recommends denial.

• The recommendation goes to a caseworker.

• The caseworker skims the AI's reason. "Looks right." Signs off.

• The person is denied without any meaningful human review.

• When the person appeals, the human says, "The AI determined eligibility. I just confirmed it."

One is meaningful human review. One is theater.

10 minutes.

For an AI system your agency uses:

• Is there human review of AI decisions?

• Is it meaningful or performative?

• Do humans have authority to override?

• Are reviewers trained on the system?

• Who is accountable for decisions?

If review is performative, what would need to change to make it meaningful?

The goal is not to remove AI from government. The goal is to use AI well, with humans exercising judgment, taking responsibility, and catching errors that AI systems make.

That's how you get the benefits of AI while maintaining human values: dignity, context, accountability.

It's the hard way. But it's the right way.

Government AI CLUB Certification Program

Level 1: AI Aware | Algorithmic Fairness in Government | Lecture 5.4

A GOVT.CLUB initiative.

<- 1.5.2 Transparency: Citizens' Right to Know 1.5.4 When Government AI Goes Wrong ->

Start Your CLUB Certification

This lecture is part of L1: AI Aware—8 hours of comprehensive government AI training.

Explore CLUB Certification

Related Lectures

L1 1.5.1—Algorithmic Fairness in Government 15 min - Video + Cases

L1 1.5.2—Transparency: Citizens' Right to Know 10 min - Video + Reading

L1 1.5.4—When Government AI Goes Wrong 15 min - Case Studies

Frequently Asked Questions

What will I learn in The Human in the Loop?

In this 10 min video + scenarios lecture, you will Why humans must remain in decision-making. Automation bias. Meaningful vs. performative oversight

What level is The Human in the Loop?

This is a Level 1 (AI Aware) lecture, part of Chapter 1.5 \u2014 Ethics and Citizen Impact. It is designed for all government employees.

How long is lecture 1.5.3?

Lecture 1.5.3 (The Human in the Loop) takes 10 min. It is delivered as a video + scenarios format.

Do I need prerequisites for The Human in the Loop?

This lecture is part of L1 (AI Aware). Prerequisites: None.

What is the CLUB Certification?

CLUB (Community Leading Unified Benchmarks) is a maturity-based AI certification for government professionals with 5 levels (L1-L5), 215 lectures, and 25 chapters aligned with NIST AI RMF, OMB, and GAO frameworks.