Learning Objectives

After completing this lecture, you will be able to:

• Understand the key concepts of understanding ai bias in a government context

• Apply knowledge of benefits, policing, hiring

• Connect understanding ai bias to your agency's AI initiatives

• Identify next steps for applying these concepts in your role

Key Topics Covered

• How bias enters AI systems (training data, design choices, deployment context)
• Government-specific bias risks: benefits, policing, hiring
• Government context for understanding ai bias
• Practical applications and next steps

Why This Matters for Government

Government agencies face unique challenges when it comes to AI adoption. This lecture addresses these challenges head-on by providing all government employees with the knowledge and frameworks needed to navigate AI in the public sector responsibly and effectively.

As part of the L1 (AI Aware) curriculum, this lecture builds on the foundational principle that every AI system in government ultimately serves citizens. Whether you are working with AI tools daily or setting strategy for your agency, understanding understanding ai bias is essential for responsible, effective government AI adoption.

Lecture URL: https://skill.re/learn/govt/understanding-ai-bias.php

======================================================================

TRANSCRIPT: Understanding AI Bias

======================================================================

What you will learn: How bias enters AI systems through training data, design choices, and deployment context. Specific risks in government use cases: benefit determination, law enforcement, hiring.

Welcome. We're going to talk about something you've probably heard about in the news—AI bias—but we're going to go deeper than the headlines. Because understanding bias isn't just about abstract fairness. In government, bias in AI can determine whether someone gets a job, whether they qualify for vital services, whether they're investigated for a crime. It's about real consequences for real people.

Here's what's important to understand: AI bias is not a flaw in some AI systems. It's not something that only happens with poorly-built models or with negligent engineers. Bias can exist in even well-intentioned, carefully-designed systems. Why? Because bias comes from many sources, and it's baked into decisions at multiple stages.

In this lecture, we're going to unpack where bias comes from, why it's particularly dangerous in government contexts, and what you need to know to spot it, report it, and help prevent it.

WHY THIS MATTERS FOR GOVERNMENT

In government, AI bias is not just a fairness issue. It's a constitutional issue. It's an issue of equal protection under law.

The foundational principle is simple: government cannot systematically treat people differently based on protected characteristics like race, color, religion, sex, or national origin. This principle has been central to democratic governance for generations.

But here's the tension: AI systems can violate equal protection even when the algorithm itself makes no explicit reference to race, gender, or other protected characteristics. An AI system trained on historical data can learn to discriminate based on proxies for those characteristics. And the discrimination can be systematic and widespread, affecting thousands of people, before anyone notices.

In government specifically, AI bias matters because:

• Government decisions are binding - If a private company's AI gets your recommendation wrong, you go somewhere else. If a government AI denies you benefits or investigates you, you're stuck with that decision until you appeal.

• Government decisions are about essential goods - We're talking about employment, housing, food assistance, freedom, safety. These aren't luxuries. This is survival.

• Government has a duty of equal protection - This is not just ethically right; it's legally required.

• Bias can be invisible - A human hiring manager might consciously choose the best candidate. An AI system can systematically choose people of one demographic and nobody notices for months.

HOW BIAS ENTERS AI SYSTEMS

Bias doesn't just appear magically. It enters through three main pathways. Let's examine each one carefully.

Pathway 1: Training Data Bias

AI systems learn from historical data. That's the fundamental idea: you show the model thousands of examples, and it learns patterns.

But here's the problem: historical data often contains the biases, inequities, and discriminatory patterns of the past. Sometimes it reflects discrimination that was explicit. More often, it reflects systemic inequities baked into the data collection process itself.

A concrete example: Your government agency wants to use AI to help decide which job applicants to interview. You train the system on your hiring decisions from the past 5 years. The system learns: "These types of candidates got hired; these types didn't." But your past hiring decisions were made by humans who, consciously or unconsciously, had biases. Maybe they hired more men than women. Maybe they hired more people of certain backgrounds. Maybe they preferentially hired people from certain universities.

Now you train an AI on that data. The AI learns these patterns. It reproduces them. When you deploy it, you've systematized and scaled those biases. You're now using AI to discriminate in hiring in a more consistent, more widespread way than humans ever did alone.

Another example from actual government: A city police department wanted to use an AI system to predict where crime would occur, to allocate patrol resources. They trained the system on historical arrest data from the past 10 years. But arrests are not uniform across neighborhoods. They're not even uniform based on actual crime. They reflect where police patrol. Neighborhoods with more police presence have more arrests, not necessarily more crime. So the AI system learned: "Crime happens in the heavily-policed neighborhoods." It recommended putting even more police there. The result: even more arrests in those neighborhoods, which fed the model for the next iteration. A feedback loop of bias was created.

How to spot training data bias:

• Ask: Where did the training data come from?

• Ask: Does it reflect the population you're making decisions about, or does it reflect historic discrimination?

• Ask: Are there demographic groups underrepresented in the training data?

• Ask: Were the historical decisions in the training data themselves fair and unbiased?

Pathway 2: Design Choices and Feature Selection

Even if your training data were perfectly representative and unbiased, bias can still enter through the choices engineers and data scientists make in designing the system.

Here's how:

When you build an AI system, you choose what variables (features) to include. "We'll use age, employment history, education level, and credit score to predict loan repayment." Sounds straightforward. But some of those features might be proxies for protected characteristics.

For example, zip code is not itself a protected characteristic. But zip code is highly correlated with race in many countries due to segregation and redlining (historic policies that prevented people of certain races from getting mortgages in certain neighborhoods). So if you train an AI to use zip code as a predictor, you're potentially using a proxy for race.

Credit score is similar. Credit score is influenced by education, employment stability, and access to credit—all of which are influenced by historic and ongoing discrimination. Using credit score as a predictor can indirectly discriminate against groups that have been historically denied access to credit.

Another design choice: How do you define success? If you're building an AI system to predict which job applicants will be successful, you need to define what "successful" means. Tenure (how long they stay in the job)? Promotion? Performance reviews? Each definition has assumptions baked in.

For instance, if you define success as "gets promoted within 5 years," and if your organization has a history of promoting men more than women, then you've defined success in a way that biases the AI system toward characteristics of men.

How to spot design bias:

• Ask: What features does the system use? Are any of them proxies for protected characteristics?

• Ask: How was "success" defined? Does that definition itself reflect bias?

• Ask: Did anyone from the affected community review these choices?

• Ask: Would you make these same choices if you were being audited?

Pathway 3: Deployment and Context Bias

Even a perfectly designed, well-trained system can produce biased outcomes in how and where it's deployed.

Here's why: An AI system trained in one context might not work the same way in a different context. And deploying it uniformly across different populations can produce disparate impact.

Example: An AI hiring system is trained on data from a tech company, where success is defined by engineering metrics. The company deploys this system across all divisions, including customer service and HR. But the features that predict success in engineering (certain educational backgrounds, specific technical skills) might not predict success in customer service. Worse, if those features are correlated with demographics, the system could disadvantage certain groups in customer service even though it was unbiased in engineering.

Another example: A government agency uses a single AI system to predict welfare fraud across all benefit programs. But benefit fraud looks different across programs. Unemployment benefits fraud is different from food assistance fraud is different from housing assistance fraud. Using one system across all programs could result in the system being accurate for some populations and inaccurate for others.

Context bias also appears when you deploy AI without understanding local conditions. An AI system that works well in one region might work poorly in another if the population, economic conditions, or social dynamics are different.

How to spot deployment bias:

• Ask: Is the system being deployed uniformly across different populations or contexts?

• Ask: Has anyone tested whether it works the same way across those populations and contexts?

• Ask: Are there reasons to believe the system might work differently in some contexts?

• Ask: Who benefits from the current deployment? Who is harmed?

GOVERNMENT-SPECIFIC BIAS RISKS

Let's examine three critical areas where government AI bias poses particular dangers.

Risk Domain 1: AI in Benefit Determination

Government agencies determine eligibility for social programs: food assistance, housing assistance, disability benefits, unemployment insurance. In many countries, hundreds of thousands of people depend on getting these determinations right.

Bias in these systems affects who gets help. If an AI system is biased against a particular demographic, people in that demographic are less likely to receive benefits they're legally entitled to.

A real-world example: The state of Michigan used an AI system to detect fraudulent unemployment claims during COVID-19. The system was biased. It flagged far more claims from one demographic group than another, even after controlling for the actual characteristics of the claims. The result: thousands of people—disproportionately from certain groups—had their benefits wrongly denied or delayed. Some lost their homes. Some went without food. Families went hungry because an AI system was biased.

The bias entered through training data: The system was trained on past fraud cases. But past investigation and prosecution of fraud reflected investigator biases, not actual fraud rates. The AI learned those biases and reproduced them at scale.

Risk Domain 2: AI in Law Enforcement

Government agencies use AI for various law enforcement purposes: predictive policing (predicting where crimes will occur), risk assessment (predicting whether someone will reoffend if released), facial recognition (identifying suspects).

Bias in these systems has profound consequences: wrongful investigation, wrongful arrest, wrongful conviction, wrongful incarceration. These aren't hypothetical harms. People's freedom is at stake.

A famous case: COMPAS, a recidivism prediction tool used in U.S. courts. The system predicts whether someone, if released, is likely to reoffend. Judges use these predictions in sentencing decisions: "Will you be a danger to society if released?" Researchers audited the system and found it was biased. It systematically overpredicted recidivism for Black defendants and underpredicted it for white defendants. This meant Black defendants were more likely to be given longer sentences based on AI predictions of future dangerousness that were systematically wrong.

The bias in COMPAS came from training data: it was trained on historical arrest and conviction data. But arrests and convictions themselves are influenced by policing practices, prosecution decisions, and systemic inequities in the criminal justice system. The AI learned these patterns and reproduced them.

Risk Domain 3: AI in Hiring and Employment Decisions

Government agencies are major employers. They use AI systems to screen applications, schedule interviews, and recommend candidates.

Bias in hiring systems affects people's access to employment. If an AI system is biased against certain demographics, people in those groups are less likely to be hired, regardless of qualifications.

A real example: Amazon developed an AI recruiting tool that was biased against women. The system was trained on historical hiring data from a male-dominated tech industry. It learned to prefer characteristics of successful past hires, most of whom were men. It systematically downranked women applicants. Amazon eventually abandoned the system, but only after it had screened thousands of applications.

Government hiring systems have the same risk. If a government agency trains an AI on its historical hiring decisions, and if those decisions reflected bias, the AI will reproduce and scale that bias.

ANTI-PATTERNS / MISUSE RISKS

Here are the most common ways organizations fail to address AI bias.

Anti-Pattern 1: Testing for Overall Accuracy Only, Not Demographic Accuracy

An organization develops an AI system and measures its accuracy: "The system is 92% accurate." Great. But if you disaggregate the results, it might be 96% accurate for one demographic group and 78% for another. The overall metric hides the disparity.

The risk: The system discriminates against a particular group, but you don't notice because you're only looking at aggregate metrics. By the time someone audits the system by demographic group, thousands of people have been affected.

Anti-Pattern 2: Deploying Without Testing for Bias

An organization is excited about a new AI capability. They want to deploy quickly. They test the system on overall performance, it seems good, and they launch. They don't test for disparate impact across demographics. They don't look for proxies for protected characteristics.

The risk: The system is biased, you find out later, people have been harmed, and you face legal liability and damage to public trust.

Anti-Pattern 3: Assuming Historical Data Is Objective

An organization assumes that if they train an AI on historical data, the system will be objective because it's based on "what happened." They don't question whether what happened was fair.

The risk: They're training the AI to reproduce historic discrimination. The system becomes a tool for perpetuating past inequities.

Anti-Pattern 4: Using Proxies for Protected Characteristics Without Recognizing It

An organization deliberately or inadvertently uses variables that are proxies for protected characteristics (race, gender, etc.). They might think, "We're not using race directly, so it's not discriminatory." But if you use zip code, or certain names, or education from particular institutions, you might be indirectly using race.

The risk: The discrimination is harder to spot because it's obscured by proxy variables. It's also harder to defend or fix.

PRACTICE / REFLECTION PROMPTS

Reflect on these questions as they relate to your role and your agency:

• Think about an AI system used by your agency (or one you've heard about). Where did its training data come from? Did anyone examine whether that data reflects historical bias? What would you need to know to be confident the data is unbiased?

• If you were to audit an AI system for bias, what demographic groups would you compare? What metrics would you use? How would you know if the disparities you found were statistically significant vs. random variation?

• Have you experienced (or witnessed) a case where an automated decision seemed unfair or biased? What was the mechanism—training data bias, design choices, or deployment context? What should have been done differently?

• In your own work, what decisions do you make that could be affected by AI in the future? Which of those decisions affect vulnerable populations? What would you want to know about an AI system before you trusted it with those decisions?

KEY TAKEAWAYS

• Bias in AI is not a rare flaw—it's a structural risk that can exist in any system trained on historical data, which all systems are.

• AI bias enters through three main pathways: training data that reflects historical discrimination, design choices that embed bias, and deployment decisions that create disparate impact.

• Government AI bias is not just unfair; it's unconstitutional. It violates the equal protection principle that's foundational to democratic governance.

• Bias can be invisible if you only look at aggregate metrics. You must disaggregate performance by demographic group to spot it.

• Historical data is not objective. Data that reflects past discrimination will teach AI to discriminate in the future.

• Proxies for protected characteristics are still discriminatory. Using zip code instead of race, or name patterns instead of ethnicity, doesn't eliminate discrimination—it just obscures it.

• Testing for bias must be ongoing. A system that's unbiased when it launches can become biased as it's deployed, as the population it affects changes, or as the context shifts.

• In government, the stakes are high enough that you cannot afford to deploy AI without testing it for bias. Do the work upfront. It's worth it.

TERMS / GLOSSARY ITEMS

Algorithmic Bias: Systematic errors or unfairness in AI decision-making, particularly when performance differs across demographic groups.

Disparate Impact: When a neutral policy or system produces unequal outcomes for different demographic groups, often disadvantaging protected classes.

Training Data Bias: Bias that exists in the historical data used to train an AI system, often reflecting past discrimination or systemic inequities.

Proxy Variables: Features that are not themselves protected characteristics but are correlated with them (e.g., zip code as a proxy for race).

Performance Disaggregation: The practice of evaluating an AI system's accuracy separately for different demographic groups, rather than only looking at overall accuracy.

Feedback Loops: Situations where biased AI predictions influence real-world outcomes, which then become data for training the next iteration of the model, amplifying bias.

Equitable Deployment: Using AI systems in ways that account for different contexts and populations, ensuring the system works fairly across diverse groups.

Let me walk through a concrete scenario of how bias could enter a government AI system, and how to prevent it.

Your agency manages a job training program. You want to use AI to match job seekers with training tracks most likely to lead to employment. You have 10 years of historical data: thousands of people who completed training, with information about what they did, what they were trained in, and whether they got jobs afterward.

You build an AI system: it looks at characteristics of successful training participants and recommends similar people to specific training tracks.

Where could bias enter?

Training data bias: The historical data reflects who trained in the past. But who trained in the past might not be representative of all job seekers. Maybe certain demographics were steered toward certain training tracks. Maybe certain demographics faced barriers to completing training. If the training data overrepresents one group, the AI will learn their characteristics as "success" and underrepresent others.

Design choices: You choose to use "highest education level achieved" as a feature. But education access is influenced by race, income, and family background. You're using a proxy for socioeconomic status and, indirectly, for race. If you weight this feature heavily, the AI will preferentially recommend training for people with more education, which might systematically disadvantage people from less-advantaged backgrounds.

Deployment: You deploy the system uniformly. But the training might work differently in different regions. A track that successfully leads to employment in an urban area might fail in a rural area. If you use the same recommendations everywhere, you might get biased results in some regions.

How do you prevent this?

• Examine your training data. Did all demographic groups have equal access to training in the past? If not, your data is biased.

• Choose features carefully. Avoid proxies for protected characteristics. If you use education level, weight it appropriately—recognize that it reflects past opportunity, not just merit.

• Test for bias. After building your model, disaggregate results by demographic group. "Is our system equally accurate at predicting success for all groups?" If not, why?

• Deploy contextually. Test the system in a few regions before going statewide. Monitor whether it works the same way everywhere.

• Monitor over time. After deployment, continue measuring performance by demographic group. If disparities emerge, investigate and fix.

That's how you prevent bias from entering an AI system.

Spend 15 minutes on this exercise.

Pick an AI system (real or hypothetical) used by your agency or one you're familiar with. Answer these questions:

• Where did the training data come from? Who collected it? Who does it represent?

• Are there any demographic groups underrepresented in the training data? If so, why?

• What features does the system use? Are any of them proxies for protected characteristics?

• How is success defined? Does that definition itself reflect bias?

• Has anyone tested this system for bias? If so, what did they find? If not, why not?

• If you disaggregate the system's performance by demographic group, are the results equal? If not, what explains the difference?

For any question where you don't have an answer, that's a gap you've identified. That's your starting point for improvement.

AI bias is not new. Humans have been biased for thousands of years. What's new is that AI allows us to scale bias—to apply biased decisions to thousands or millions of people automatically and consistently.

But here's the flip side: AI also gives us tools to detect bias. We can measure. We can disaggregate. We can audit. We can find where bias is hiding. And we can fix it.

The key is to do the work. Don't assume your system is unbiased. Test it. Look at who wins and who loses. Be honest about what you find. And then improve.

In government, this is not optional. This is fundamental to equal protection, to democratic legitimacy, and to the trust that citizens place in you.

Thank you for taking seriously the responsibility to prevent and address AI bias in your work.

Government AI CLUB Certification Program

Level 1: AI Aware | Government AI Policy Landscape | Lecture 2.2

A GOVT.CLUB initiative.

<- 1.2.4 The Blueprint for an AI Bill of Rights 1.2.6 The AI Decision Framework: When to Use and When Not To ->

Start Your CLUB Certification

This lecture is part of L1: AI Aware—8 hours of comprehensive government AI training.

Explore CLUB Certification

Related Lectures

L1 1.2.1—Government AI Policy Landscape 20 min - Video + Reading

L1 1.2.2—Data Sensitivity and Classification 15 min - Video + Checklist

L1 1.2.3—PII and AI: The Bright Red Lines 15 min - Video + Scenarios

Frequently Asked Questions

What will I learn in Understanding AI Bias?

In this 15 min video + interactive lecture, you will How bias enters AI systems (training data, design choices, deployment context). Government-specific bias risks: benefits, policing, hiring

What level is Understanding AI Bias?

This is a Level 1 (AI Aware) lecture, part of Chapter 1.2 \u2014 Responsible AI Use. It is designed for all government employees.

How long is lecture 1.2.5?

Lecture 1.2.5 (Understanding AI Bias) takes 15 min. It is delivered as a video + interactive format.

Do I need prerequisites for Understanding AI Bias?

This lecture is part of L1 (AI Aware). Prerequisites: None.

What is the CLUB Certification?

CLUB (Community Leading Unified Benchmarks) is a maturity-based AI certification for government professionals with 5 levels (L1-L5), 215 lectures, and 25 chapters aligned with NIST AI RMF, OMB, and GAO frameworks.