Learning Objectives

After completing this lecture, you will be able to:

• Understand the key concepts of data quality and ai performance in a government context
• Complete hands-on exercises that reinforce practical skills
• Connect data quality and ai performance to your agency's AI initiatives
• Identify next steps for applying these concepts in your role

Key Topics Covered

-
Garbage in, garbage out -- quantified

-
Data cleaning, labeling, representativeness

-
Measuring data quality

Why This Matters for Government

Government agencies face unique challenges when it comes to AI adoption. This lecture addresses these challenges head-on by providing analysts, project leads, team supervisors with the knowledge and frameworks needed to navigate AI in the public sector responsibly and effectively.

As part of the L2 (AI Practitioner) curriculum, this lecture builds on the foundational principle that every AI system in government ultimately serves citizens. Whether you are working with AI tools daily or setting strategy for your agency, understanding data quality and ai performance is essential for responsible, effective government AI adoption.

======================================================================

TRANSCRIPT: Data Quality and AI Performance

======================================================================

What you will learn: How data quality directly impacts AI performance; quantifying data quality (completeness, accuracy, consistency, uniqueness); data cleaning and labeling practices; bias in data; strategies for improving data quality; monitoring data quality over time.

There's a phrase in machine learning: "Garbage in, garbage out." It's true. No matter how sophisticated your model is, if your training data is poor quality, your model will be poor quality.

This lecture is about understanding data quality deeply, measuring it, and improving it. You'll learn what makes data good versus bad, how to quantify quality, and what to do about quality problems.

By the end, you'll be able to audit your agency's data and develop a plan to improve it.

WHY THIS MATTERS FOR GOVERNMENT

Government AI systems are only as good as their data. If you're training a model on historical benefit decisions and those decisions contain errors or biases, the model learns from those errors and biases. If you're analyzing compliance data and that data is incomplete, your analysis is incomplete.

Understanding data quality is fundamental to responsible government AI.

DIMENSIONS OF DATA QUALITY

Quality has multiple dimensions:

Completeness: Do you have all the data you need? Are there missing values? Missing entire records?

Accuracy: Is the data correct? Are there errors, typos, out-of-range values?

Consistency: Is data represented consistently? If you have "state" recorded as both abbreviations and full names, that's inconsistent.

Uniqueness: Are there duplicates? If the same person appears multiple times with slightly different information, that's a uniqueness problem.

Timeliness: Is the data current? If your data is from 2020 but you're analyzing current situations, timeliness is a problem.

Validity: Does the data meet defined standards? If you have a field that should only contain numbers and it contains letters, that's a validity problem.

BIAS IN DATA

Overview

Data can contain bias at multiple stages:

Collection bias: The way data is collected might systematically exclude or over-represent certain groups.

Labeling bias: If humans label data, they might be biased in their labeling.

Historical bias: If you're training on historical data, and that data reflects past discrimination, the model learns the discrimination.

Measurement bias: The way something is measured might systematically favor or disfavor certain groups.

PRACTICAL USE CASES

Use Case 1: Benefits Eligibility Data Quality

Scenario: A benefits agency trains a model to predict whether an applicant is likely to be eligible. The training data comes from 5 years of historical benefit decisions.

Quality issues:

-- Incomplete data: Some applications have missing information (some applicants didn't fill in all fields)

-- Inconsistency: Some income fields are annual, some monthly

-- Duplicates: Some applicants applied multiple times; records aren't de-duplicated

-- Bias: Historical decisions might reflect biases in how decisions were made

Approach to improve data quality:

-- Impute missing values (fill in missing income with statistical estimates based on other applicants)

-- Standardize income to annual amounts

-- De-duplicate records

-- Analyze whether historical biases are present; if so, account for them or exclude biased data

Use Case 2: Compliance Monitoring Data Quality

Scenario: An agency monitors whether organizations comply with regulations. The monitoring data comes from inspections, self-reports, and complaints.

Quality issues:

-- Different data from different sources (inspections might be more accurate than self-reports)

-- Incompleteness (some compliance areas might not be monitored consistently)

-- Bias (organizations in certain regions might be inspected more frequently)

Approach:

-- Weight data based on source (inspection data weighted more heavily than self-reports)

-- Identify and account for inspection bias (adjust analysis to account for some regions being over-inspected)

ANTI-PATTERNS

Anti-Pattern 1: Using Raw Data Without Cleaning

RISK: Data quality problems propagate through analysis and modeling.

HOW TO AVOID

-- Always audit data before using it.

-- Clean data (remove duplicates, fix inconsistencies, handle missing values).

-- Document what cleaning you did so auditors can understand what you changed.

Anti-Pattern 2: Ignoring Biases in Historical Data

RISK: Training on biased historical data, the model replicates and amplifies historical biases.

HOW TO AVOID

-- Before training, audit data for biases.

-- If biases are present, decide: exclude biased data, or account for it analytically.

-- Test the trained model on diverse groups to detect amplified biases.

KEY TAKEAWAYS

• Data quality has multiple dimensions: completeness, accuracy, consistency, uniqueness, timeliness, validity.
• Garbage in, garbage out. Poor data quality produces poor AI models, regardless of model sophistication.
• Data can contain biases introduced at collection, labeling, measurement, or historical stages.
• Before using data for AI, audit it for quality issues and biases.
• Data cleaning (removing duplicates, handling missing values, standardizing formats) is essential preparatory work.
• Historical data might contain past biases. Account for these when training models.

TERMS AND GLOSSARY

• Data Quality: The degree to which data is suitable for its intended use. Includes completeness, accuracy, consistency, timeliness, validity.
• Data Cleaning: The process of correcting errors, removing duplicates, handling missing values, and standardizing data before analysis.
• Imputation: Filling in missing values with estimates based on other data.
• Deduplication: Removing duplicate records.
• Measurement Bias: Systematic errors introduced by how something is measured.
• Historical Bias: Biases present in historical data reflecting past discrimination or unfair practices.

Data quality is the foundation of AI quality. Before deploying any AI system, you must understand and improve the quality of your data.

You've completed Lecture 2.1.6. You now understand data quality deeply and how to improve it. Next lecture (2.1.7: The AI System Lifecycle), we'll look at the full lifecycle of AI systems: development, training, testing, deployment, monitoring, retirement.

Government AI CLUB Certification Program

Level 2: AI Ready | Data Quality and AI Performance | Lecture 2.1.6

A GOVT.CLUB initiative.

<- 2.1.3 Generative AI Deep Dive
2.1.5 The AI System Lifecycle ->

Start Your CLUB Certification

This lecture is part of L2: AI Practitioner -- 40 hours of comprehensive government AI training.

Explore CLUB Certification

Related Lectures

L2
2.1.1 -- Supervised vs. Unsupervised vs. Reinforcement Learning
60 min - Video + Interactive

L2
2.1.2 -- How Transformers and LLMs Work
60 min - Video + Diagrams

L2
2.1.3 -- Generative AI Deep Dive
60 min - Video + Demos