Learning Objectives

After completing this lecture, you will be able to:

• Understand the key concepts of ai confidence and hallucination in a government context

• Connect ai confidence and hallucination to your agency's AI initiatives

• Identify next steps for applying these concepts in your role

Key Topics Covered

• Why AI sounds certain even when wrong
• Recognizing hallucination patterns
• Building healthy skepticism without paralysis

Why This Matters for Government

Government agencies face unique challenges when it comes to AI adoption. This lecture addresses these challenges head-on by providing all government employees with the knowledge and frameworks needed to navigate AI in the public sector responsibly and effectively.

As part of the L1 (AI Aware) curriculum, this lecture builds on the foundational principle that every AI system in government ultimately serves citizens. Whether you are working with AI tools daily or setting strategy for your agency, understanding ai confidence and hallucination is essential for responsible, effective government AI adoption.

Lecture URL: https://skill.re/learn/govt/ai-confidence-and-hallucination.php

======================================================================

TRANSCRIPT: AI Confidence and Hallucination

======================================================================

What you will learn: Why AI sounds certain even when wrong. Recognizing hallucination patterns. The confidence-accuracy gap.

Here's something that will surprise many people: AI systems don't actually know whether they're right or wrong. They generate plausible-sounding text. They do this with high confidence regardless of accuracy.

This is dangerous. If an AI tells you something with absolute certainty, and it turns out to be completely false, you've been misled.

In this lecture, we're going to understand why AI systems behave this way, and how to recognize when they're likely hallucinating.

WHY THIS MATTERS FOR GOVERNMENT

Government work requires accurate information. If AI systems confidently assert false information, and you rely on that information, you make bad decisions.

Understanding AI confidence and hallucination is critical to using AI safely.

HOW AI SYSTEMS WORK

Understanding how AI systems generate text helps explain why they hallucinate.

AI systems (like large language models) work by predicting the next word, based on patterns in training data.

Given "The capital of France is," the model predicts "Paris" because that's the most common next word given that context.

Given "A study showed that eating [blank] reduces heart disease," the model predicts a plausible-sounding word based on patterns in medical writing. It might predict "vegetables" (correct) or "blueberries" (reasonable) or "unicorn dust" (if that appeared in training data context that somehow related to health).

The model doesn't "know" what's true. It predicts what's likely based on patterns.

Here's the critical point: the model generates text with the same level of confidence regardless of whether it's accurate or hallucinated.

When the model says "Studies show that X reduces Y by 50%," it's not asserting this with knowledge. It's predicting this is a likely sentence given the context.

WHAT IS HALLUCINATION?

Hallucination is when an AI system generates false information, often presented with confidence.

Examples of Hallucinations:

• Making up citations: "Smith (2019) found that AI improves government efficiency by 40%." No such paper exists. The model invented it.

• Inventing statistics: "Studies show that 73% of government employees prefer remote work." The model made up this statistic.

• False historical claims: "The Affordable Care Act was passed in 2008." (It was 2010.) The model was wrong and stated it confidently.

• Nonsensical logic: "To reduce traffic, cities should require cars to be green." The model generated a sentence that sounds like a policy but is nonsensical.

Why Hallucinations Happen:

AI systems are trained to generate plausible-sounding text. "Plausible-sounding" and "accurate" are not the same thing.

Sometimes the most plausible-sounding next word is wrong.

Sometimes the AI was trained on incorrect information.

Sometimes the AI is forced to generate something it has no information about, so it makes it up.

THE CONFIDENCE-ACCURACY GAP

This is critical: AI system confidence and AI accuracy are not correlated.

An AI can be:

• Wrong but confident ("I'm certain the capital of France is Rome")

• Right but uncertain ("Maybe the capital of France is Paris?")

• Right and confident (usually happens when it's trained clearly on something)

You cannot use the AI's confidence as a measure of its accuracy.

This is particularly dangerous with complex claims. "The effectiveness of job training programs in reducing unemployment" is complex. The AI might:

• Invent specific statistics with complete confidence

• Present them as fact

• Sound authoritative

• Be completely wrong

PATTERNS THAT INDICATE HALLUCINATION

While you can't use confidence as a signal, there are patterns that indicate hallucination is more likely:

• Specific numbers without sources. "X reduces Y by 47%." If the AI doesn't cite where this number comes from, it might be hallucinated.

• Made-up citations. The AI cites a study. You look it up. It doesn't exist. Hallucination.

• Internal contradictions. The AI contradicts itself within the same response. This suggests it's generating text without actually understanding it.

• Confident assertions about obscure topics. "The best approach to increase civic participation in your city is..." The AI is probably hallucinating. It doesn't know your city. It's generating plausible-sounding text.

• Highly specific claims about your context. "Your agency should implement X specific policy." The AI has no information about your agency. It's hallucinating.

• Claims that seem almost right but slightly off. Sometimes hallucinations are close to truth but not quite. "The Affordable Care Act was passed in 2009." Close to 2010, but wrong.

MITIGATING HALLUCINATION RISK

You can't eliminate hallucination risk, but you can reduce it:

• Ask for sources. "What sources support this claim?" Sometimes the AI will admit it doesn't have sources.

• Ask for uncertainty. "How confident are you in this? What's uncertain?" This sometimes helps the AI be more honest about uncertainty.

• Verify everything important. Don't trust AI claims about facts without checking.

• Be most skeptical of specific claims. Specific statistics, dates, names are more likely to be hallucinated.

• Check pattern: If an AI consistently hallucinates on certain topics, be extra skeptical of its output on similar topics.

• Compare outputs. Ask the same question to different AI systems. Do they agree? Disagreement can indicate hallucination.

ANTI-PATTERNS / MISUSE RISKS

Anti-Pattern 1: Trusting Confident-Sounding AI Outputs Without Verification

The AI sounds confident. You assume it's right.

Risk: Relying on hallucinated information for important decisions.

Anti-Pattern 2: Sharing AI-Generated Citations Without Verifying Them

The AI cites a study in your report. You share your report without checking if the citation is real.

Risk: Your credibility is damaged when someone checks and finds the citation doesn't exist.

Anti-Pattern 3: Asking AI About Topics Where It Definitely Hallucinates

You ask the AI about your specific agency's policies, budget, or procedures. It hallucinates plausible-sounding information.

Risk: You act on false information about your own agency.

PRACTICE / REFLECTION PROMPTS

• Ask an approved AI tool a specific factual question. Does it provide sources? Can you verify the facts?

• Deliberately ask an AI to do something it might hallucinate on (like describe your agency's specific procedures). What does it produce? How confident is it?

• In your work, what types of claims would be most dangerous if hallucinated?

KEY TAKEAWAYS

• AI systems don't know whether they're right or wrong. They generate text based on patterns.

• Confidence is not correlated with accuracy. A wrong answer can be stated with complete confidence.

• Hallucinations are real and common, especially for specific claims like statistics or citations.

• Specific numbers, citations, and claims about your context are most likely to be hallucinated.

• You cannot rely on AI confidence as a signal of accuracy.

• Verify important claims, especially specific statistics, citations, and claims about your specific situation.

TERMS / GLOSSARY ITEMS

Hallucination: When an AI generates false information, often with confidence.

Confidence-Accuracy Gap: The lack of correlation between how confident an AI sounds and whether it's actually right.

Plausible-Sounding: Text that sounds reasonable and likely but might be false.

Token Prediction: The process by which language models generate text word-by-word based on patterns.

You ask an AI: "What's the average cost of implementing a municipal bike-sharing program?"

The AI responds: "According to research by the International Transportation Association (2021), the average cost of implementing a municipal bike-sharing program is approximately $1.2 million for a city of 500,000 people. This includes infrastructure, bikes, and first-year operational costs."

This sounds authoritative and specific. But:

• The "International Transportation Association" might not exist (or might not have published this specific research)

• The $1.2 million figure is specific and sourced, but you can't verify it

• The confidence is high but might be hallucinated

What to do:

• Search for the "International Transportation Association 2021" research. Does it exist?

• Search for actual bike-sharing cost data. What do you find?

• Ask the AI for more sources.

• Cross-check with what you find independently.

• Only rely on the AI's number after you've verified it.

That's how you mitigate hallucination risk.

10 minutes.

Think about something you know well (a city you've lived in, an organization you've worked in, a topic you've studied). Ask an AI about it in detail.

Does the AI get it right? Does it hallucinate?

This exercise trains you to recognize when AI is accurate vs. hallucinating.

Hallucination is not a flaw that will go away. It's fundamental to how AI systems work. Learn to recognize it. Verify important claims. Don't let confident-sounding text fool you into trusting information you haven't checked.

Government AI CLUB Certification Program

Level 1: AI Aware | Your Agency's Approved AI Tools | Lecture 3.6

A GOVT.CLUB initiative.

<- 1.3.5 AI for Research and Data Organization 1.4.1 How AI Changes the Threat Landscape ->

Start Your CLUB Certification

This lecture is part of L1: AI Aware—8 hours of comprehensive government AI training.

Explore CLUB Certification

Related Lectures

L1 1.3.1—Your Agency's Approved AI Tools 20 min - Hands-On Lab

L1 1.3.2—Prompt Engineering Basics 25 min - Hands-On Lab

L1 1.3.3—Evaluating AI Outputs 20 min - Hands-On Lab

Frequently Asked Questions

What will I learn in AI Confidence and Hallucination?

In this 15 min video + demos lecture, you will Why AI sounds certain even when wrong. Recognizing hallucination patterns. Building healthy skepticism without paralysis

What level is AI Confidence and Hallucination?

This is a Level 1 (AI Aware) lecture, part of Chapter 1.3 \u2014 Practical AI Skills. It is designed for all government employees.

How long is lecture 1.3.6?

Lecture 1.3.6 (AI Confidence and Hallucination) takes 15 min. It is delivered as a video + demos format.

Do I need prerequisites for AI Confidence and Hallucination?

This lecture is part of L1 (AI Aware). Prerequisites: None.

What is the CLUB Certification?

CLUB (Community Leading Unified Benchmarks) is a maturity-based AI certification for government professionals with 5 levels (L1-L5), 215 lectures, and 25 chapters aligned with NIST AI RMF, OMB, and GAO frameworks.