Learning Objectives

After completing this lecture, you will be able to:

• Understand the key concepts of how transformers and llms work in a government context
• Connect how transformers and llms work to your agency's AI initiatives
• Identify next steps for applying these concepts in your role

Key Topics Covered

-
Attention mechanisms, tokenization, training, fine-tuning, RLHF

-
Conceptual understanding without code

-
Government context for how transformers and llms work

-
Practical applications and next steps

Why This Matters for Government

Government agencies face unique challenges when it comes to AI adoption. This lecture addresses these challenges head-on by providing analysts, project leads, team supervisors with the knowledge and frameworks needed to navigate AI in the public sector responsibly and effectively.

As part of the L2 (AI Practitioner) curriculum, this lecture builds on the foundational principle that every AI system in government ultimately serves citizens. Whether you are working with AI tools daily or setting strategy for your agency, understanding how transformers and llms work is essential for responsible, effective government AI adoption.

======================================================================

TRANSCRIPT: How Transformers and LLMs Work

======================================================================

What you will learn: Attention mechanisms and how they enable understanding context; tokenization and why text is broken into pieces; the transformer architecture conceptually; training, fine-tuning, and RLHF; why LLMs produce different outputs; how to interpret confidence signals.

Welcome to Lecture 2.1.2. You've learned that machine learning comes in three paradigms. Today, we dive deep into the specific architecture powering most generative AI you're using in government: transformers and large language models.

Why spend an entire lecture on this? Because transformers are fundamentally different from older ML approaches. To use generative AI responsibly in government -- to evaluate systems, understand limitations, prompt effectively, and know when to trust outputs -- you need to understand how they work.

This lecture is conceptual, not technical. I'm not teaching you to build transformers. I'm teaching the concepts: how these systems understand context, why they hallucinate, what fine-tuning means, what RLHF does, why the same prompt sometimes produces different answers.

By the end, you'll reason about transformer-based systems clearly. You'll know what questions to ask vendors. You'll understand prompt engineering. You'll have realistic expectations about what these systems can and can't do for government.

WHY THIS MATTERS FOR GOVERNMENT

Most generative AI your agencies are adopting uses transformer architecture. ChatGPT is a transformer. Claude is a transformer. Google's Gemini is a transformer. Government agencies are increasingly deploying these systems for document drafting, policy analysis, constituent response, and knowledge management.

The challenge: these systems are powerful but not predictable like traditional software. They don't follow hard rules. They produce probabilistic outputs. They sometimes generate plausible-sounding answers that are false. They work differently than the deterministic systems government has relied on for decades.

Without understanding how transformers work -- even conceptually -- you can't evaluate whether an LLM is suitable for your use case. You can't detect whether it's genuinely good at a task or just sounds confident. You can't design effective prompt engineering. You can't build appropriate human oversight.

Understanding transformers is prerequisite knowledge for responsible AI governance in government.

ATTENTION -- THE KEY INNOVATION

The transformer architecture is built on "attention." Here's the intuitive idea:

When you read a sentence, you don't treat all words equally. Some words are more important for understanding meaning. Some words have special relationships to other words.

Example: "The bank executive announced the merger. She was excited."

"She" refers to "executive," not "bank." Your brain automatically knows this because you're paying attention to which words matter for the reference. You're attending to the relationship between "She" and the nearby word "executive."

Similarly, when a model reads text, it needs to understand relationships between words. This is what attention does. It learns to weight different parts of input based on their relevance to understanding each word.

Technical idea (simplified): Attention works like this. For each word in the input, the model asks: "Which other words in this sequence are most important for understanding me?" It computes a numerical score for each possible relationship. It learns to weight inputs by those scores. Words with high attention weights influence the output more.

Example: In "The bank executive announced the merger. She was excited," when processing "was," the attention mechanism gives high weight to "She" and "executive" (relevant to determining state of being), lower weight to "bank" and "merger" (less directly relevant).

Why this matters: Attention lets the model understand context. It determines that "bank" can mean a financial institution or the side of a river, and which meaning applies based on surrounding words. It tracks references across long distances in text.

Without attention, older language models (recurrent networks) struggled with long-distance relationships. They'd forget important context from earlier sentences. Transformers, built on attention, can handle much longer contexts and understand complex interdependencies.

TOKENIZATION -- WHY TEXT GETS BROKEN INTO PIECES

Before a transformer processes text, it doesn't see words. It sees tokens. And tokens aren't always words.

Here's why: The model works with numbers, not text. "Hello" must become numbers. One option: assign each word a number (word 1, word 2, etc.). But dictionaries are huge (500,000+ English words). New words are invented constantly. Misspellings and proper nouns present challenges.

Instead, tokenization breaks text into subword units. "Running" becomes "run" + "ning." "Unprecedented" becomes "unprece" + "dented." Common letter sequences get single tokens. Rare sequences get broken into smaller pieces.

Why: This lets the model handle rare words and new words. "Cryptocurrency" might not be in training vocabulary, but "crypt" + "o" + "currency" might be, so the model can process it.

Implications for government:

-- The same word might tokenize differently depending on language or context. "Colour" (British) and "color" (American) tokenize differently, potentially affecting how models treat them.

-- Acronyms and abbreviations tokenize unpredictably. "OMB" might be one token, or O + M + B as three.

-- This is why prompts sometimes produce weird outputs on acronyms or specialized terms. The model is processing them as token sequences, not meaningful units.

-- When preparing documents for LLM analysis, understanding tokenization helps. Using full terms instead of abbreviations, or defining abbreviations in prompts, helps models understand better.

THE TRANSFORMER ARCHITECTURE

A transformer is built in layers. Each layer refines understanding through attention. Simplified picture:

Layer 1 (Bottom): Raw text comes in as tokens. Attention weights are learned based on which tokens are relevant to each other.

Layer 2: Output from Layer 1 (refined representations where words are understood in context) goes into another attention layer. Relationships are refined further. The model can now attend to abstract concepts, not just token proximity.

Layer 3, 4, 5, etc.: This repeats. Each layer refines understanding. By the top layer, the model has a rich representation of text where word meanings are resolved by context, relationships are clear, intent is captured, and reasoning has developed.

Then, at the very top, there's a prediction layer. The model outputs the most likely next word.

Why predict the next word? Because that's the training task. The model is trained on billions of examples like:

-- Input: "The capital of France is"

-- Correct output: "Paris"

The model learns: "Given context X, the most likely next token is Y."

Then at inference (when you're using it), you give it a prompt like "The capital of France is" and it predicts the next token. That token becomes part of output. Then you feed the model "The capital of France is Paris" and ask for the next token. It predicts a period or space. You repeat until the model predicts a stop token or reaches a length limit.

This is why generation is probabilistic. At each step, the model doesn't deterministically produce the next token; it generates a probability distribution over possible tokens and samples one (or picks the highest probability). Different samples at different steps create different outputs.

TRAINING AND FINE-TUNING

A large language model like GPT-4 is trained on billions of words from the internet, books, and public sources. Training happens in two phases:

Phase 1 (Pre-training): The model is exposed to massive amounts of text. For each sequence of tokens, the model tries to predict the next token. When it gets it wrong, its parameters adjust. This happens billions of times. The model learns patterns of language: grammar, facts, reasoning, even coding.

Phase 2 (Fine-tuning): After pre-training, the model is specialized for specific tasks. For example, OpenAI fine-tunes GPT-4 using examples of good conversations. Training shows the model examples of good responses to prompts and adjusts parameters to increase likelihood of producing similar responses. This is where instruction-following and safety measures are added.

Why does fine-tuning matter for government? Because it means different organizations can specialize the same base model for their needs. You could fine-tune a base model on:

-- Your agency's internal documentation and processes, creating a model specialized for your operations.

-- Your regulatory corpus, creating a model specialized in compliance.

-- Your domain language and terminology, creating a model that understands your context.

Fine-tuning is more efficient than training from scratch (you reuse pre-training) but requires substantial work:

• Curating examples of good input-output pairs.
• Training (hours to days on good hardware).
• Validation and iteration.

For government, fine-tuning is attractive because it could make models more accurate and domain-specific. But it requires quality curated data and ongoing maintenance.

RLHF -- REINFORCEMENT LEARNING FROM HUMAN FEEDBACK

Overview

After fine-tuning, many organizations apply RLHF. Here's how it works:

Step 1: The model generates multiple responses to the same prompt.

Step 2: Human raters rank the responses (best, second-best, worst).

Step 3: A separate reward model is trained to predict: "Given a prompt and response, is this a good response?" This model learns from the rankings.

Step 4: The original LLM is fine-tuned using reinforcement learning, where the reward signal is the reward model's assessment. The LLM adjusts to maximize this reward.

Why? Because RLHF helps align the model's behavior with human preferences. A model good at next-token prediction might still generate rambling, inaccurate, or unhelpful responses if that's statistically likely in training data. RLHF pushes the model toward helpfulness, harmlessness, and honesty.

For government: RLHF is particularly important. It's how safety constraints get added. It's also where problems arise: if human raters' preferences reflect biases, or if the reward signal is misaligned with actual government values, RLHF can amplify those problems.

Understanding RLHF is important because it means LLM behavior isn't purely learned from data; it's also learned from human preferences. Different organizations apply RLHF differently, resulting in different models with different behavior even starting from the same base.

PRACTICAL USE CASES WITH GOVERNMENT SCENARIOS

Use Case 1: Policy Analysis and Briefing Generation (Fine-tuning a Base Model)

Scenario: A policy analysis office spends 40% of time drafting briefing documents. The office has 500 historical briefing documents, internal standards for structure (executive summary, context, analysis, recommendations), internal terminology and policy acronyms, preferences for specific evidence standards.

Approach: Fine-tune a base LLM on these 500 documents. After fine-tuning, the model understands your briefing structure and format, your terminology and context, your evidence standards and what constitutes credible support.

Application: When a staffer provides background information on a topic, the model drafts a briefing in your organization's style. It's not autonomously making policy decisions; it's accelerating drafting. Staffers still review, edit, and sign off.

Challenge: Fine-tuning requires high-quality examples. If your historical briefings contain errors or biases, the model learns them. Quality control is essential.

Use Case 2: Constituent Service Response (RLHF and Prompt Engineering)

Scenario: A state legislative office receives hundreds of constituent inquiries daily on diverse topics: licensing, benefits, permits, complaints. Staff currently manually responds to most inquiries.

Approach: Deploy an LLM (possibly base model, possibly fine-tuned) with RLHF to generate initial draft responses. Responses are reviewed by human staff before sending.

Application: A constituent asks: "I applied for my driver's license three months ago and haven't received it. What should I do?"

The LLM generates a draft response: "Motor vehicle licensing processing times vary by state and workload. Yours should take 2-4 months from application. Since you're at the 3-month mark, your license is likely being processed. You can call our office at [number] to check status or submit a service request at [website]. If you applied in error or need to expedite, contact our licensing unit directly."

The staff member reviews, might add a link to the licensing website, and sends it. The model handles the tedious part; humans ensure accuracy and empathy.

Challenge: RLHF in this context requires human preferences to be clear. Does your office prefer concise answers or detailed ones? Do you prioritize being helpful or conservative? Those preferences should drive the RLHF process.

Use Case 3: Technical Jargon Translation

Scenario: A benefits agency publishes eligibility documents written for other benefits programs. The documents contain legal language and program jargon that ordinary citizens don't understand.

Approach: Use an LLM (possibly with light fine-tuning on simplified language examples) to translate dense policy language into plain language. Examples of RLHF might rank how well each version preserves meaning while being understandable.

Application: Original (complex): "Applicants whose modified adjusted gross income exceeds 400% of the federal poverty level as adjusted for household composition shall not be eligible for supplementary nutrition assistance."

LLM output: "You probably don't qualify for nutrition help if your income is more than 4 times the poverty limit for your family size. (Check our website to see what the poverty limit is for your family.)"

The model handles translation; editors verify accuracy.

Challenge: Accuracy is critical. If the model simplifies too much, it changes meaning. Tight human oversight is needed, at least initially.

ANTI-PATTERNS AND MISUSE RISKS

Anti-Pattern 1: Treating LLM Outputs as Authoritative Without Verification

RISK: The model produces confident-sounding but false outputs (hallucinations), used in government decisions.

WHY IT HAPPENS: LLMs are very good at producing fluent, coherent text. They "sound right." When a model confidently states "The federal poverty level in 2024 is $28,500 for a family of four," it's hard not to believe it, especially under time pressure.

But the model hasn't accessed any data source. It's predicting the most likely next token given its training data (which ended in April 2024, say). If it saw that statistic in training, it might reproduce it. If it didn't, it invents something plausible.

WHAT GOES WRONG: A government document relying on the model's answer publishes the wrong poverty level, affecting benefit determinations. A policy office cites made-up statistics. A compliance check uses inaccurate legal interpretations.

EXAMPLE: A model is asked, "What is the current OMB guidance on AI procurement?" The model, trained on data from 2023, generates a detailed response that sounds authoritative. But it's combining guidance from different eras and adding details that don't exist. An agency uses this in a procurement decision, later discovering it's non-compliant.

HOW TO AVOID

-- Never use LLM outputs for factual claims without verifying against authoritative sources.

-- Build a validation workflow: LLM drafts, human checks facts.

-- For legal or compliance questions, consult your legal team; don't rely on the model.

-- Use the model for synthesis and drafting, not for fact-finding.

-- Be especially cautious with statistics, dates, percentages, and direct quotes. These are exactly what models hallucinate.

-- Explicitly tell users: "This is a draft. All facts must be verified."

Anti-Pattern 2: Fine-tuning on Biased or Non-Representative Data

RISK: The fine-tuned model amplifies biases present in the training data.

WHY IT HAPPENS: Fine-tuning uses smaller, curated datasets. If those datasets are small, they might not represent diverse scenarios. If they reflect historical decisions, they might reflect historical biases. If you fine-tune on policy decisions from the last five years, and those decisions reflected certain biases, the model will learn them.

WHAT GOES WRONG: A model fine-tuned on historical hiring decisions learns patterns correlated with protected characteristics. A model fine-tuned on historical compliance decisions systematically favors or disfavors certain applicants or organizations. A model trained on historical benefit determinations learns to approve certain demographic groups at higher rates.

EXAMPLE: A government agency fine-tunes a model on its 500 most recent performance reviews. Over the years, supervisors have been systematically more harsh in reviews for female employees. The fine-tuned model, learning from this data, produces harsher-sounding performance reviews for women, perpetuating the bias.

HOW TO AVOID

-- Before fine-tuning, audit your training data for biases. Are certain groups represented fairly?

-- Use diverse examples. Deliberately include scenarios with underrepresented groups.

-- After fine-tuning, test the model on diverse scenarios. Does it behave differently for protected groups?

-- Include human oversight in the feedback loop. If RLHF is part of your process, ensure raters are trained to avoid bias.

-- Document what data you fine-tuned on, so auditors can evaluate whether biases are present.

-- Consider retraining regularly on new, diverse data to prevent biases from crystallizing.

Anti-Pattern 3: Assuming Consistency; Deploying Without Testing on Your Specific Inputs

RISK: The model works well on generic examples but fails on your domain-specific language, acronyms, or edge cases.

WHY IT HAPPENS: Transformers are trained on broad internet data. They understand general language well. But government language is specialized. You use acronyms (OMB, NIST, SLA), terminology (non-appropriated funds, obligated authority), and genres (memos, briefings) that might be underrepresented in the model's training.

Without testing, you don't know whether the model handles your specific inputs well. It might work great on generic policy questions but struggle with your acronyms.

WHAT GOES WRONG: You deploy a model to draft briefings. It handles 90% of tasks fine but fails on documents involving specific acronyms or domain terms. Staff spends time correcting outputs that should have been caught before deployment. Staff might not catch all errors.

EXAMPLE: A model is deployed to analyze compliance reports. The reports use the abbreviation "FY" (fiscal year). The model sometimes misinterprets "FY2024" as a technical term, producing confused summaries. Staff should have tested this before deployment.

HOW TO AVOID

-- Before deployment, test the model on representative examples from your domain.

-- Include edge cases: rare acronyms, complex sentence structures, unusual terminology.

-- Test on examples where the model is likely to struggle.

-- Gather feedback from actual users (staff) on whether outputs are helpful.

-- Define "acceptable error rate" for your use case. If 5% of outputs need significant revision, that might be unacceptable.

-- Start with low-stakes use cases (drafting that gets reviewed) before high-stakes use cases (autonomous decisions).

Anti-Pattern 4: Over-Trusting Prompt Engineering as a Governance Solution

RISK: Prompt engineering can improve outputs, but it's not a substitute for real governance, testing, or human oversight.

WHY IT HAPPENS: Once teams discover that careful prompts produce better outputs, they assume good prompts solve the problem. They engineer increasingly complex prompts, thinking sophisticated instructions will make the model work better.

But prompt engineering has limits. You can't prompt a model out of fundamental limitations. If the model's training data doesn't include certain knowledge, prompting won't help. If the model is prone to particular biases, a more elaborate prompt won't eliminate them. Also, prompt engineering is often brittle -- a prompt that works on version 1 of a model might not work on version 2.

WHAT GOES WRONG: A team spends weeks perfecting a prompt for contract analysis. The prompt works well for test cases. Then they deploy it, and it encounters contracts slightly outside the test distribution, producing worse results than expected. Or they upgrade to a new model version, and the prompt no longer works.

EXAMPLE: A team engineers an elaborate prompt like: "You are a contract analyst for a government agency. You have 10 years of experience. Analyze this contract for risk, considering all of the following: [long list of considerations]. Provide a structured output with: Summary, Risks, Recommendations. Be concise. Prioritize government interests. Avoid recommending approval unless you're highly confident." The prompt works well on test cases. But in production, the model sometimes still misses risks or makes aggressive recommendations. The prompt didn't actually solve the problem; it just masked it on test cases.

HOW TO AVOID

-- Use prompts to improve outputs, but don't rely on prompts alone.

-- Build human oversight (humans review outputs before use).

-- Test prompts on diverse cases, not just a curated test set.

-- When you upgrade models, re-test prompts. Don't assume they still work.

-- Combine prompt engineering with fine-tuning if possible. Prompts are surface-level; fine-tuning is deeper.

-- Document your prompts so they can be audited and improved.

-- Recognize that prompts are part of governance, but not the whole governance.

PRACTICE AND REFLECTION PROMPTS

Prompt 1: Attention in Your Work

Think about a complex document you read recently. Identify one moment where you had to pay attention to which words were related. Write a sentence or two explaining what "attention" meant in that context. Then think: does an LLM need the same kind of attention? Why or why not?

Prompt 2: Tokenization Impact

Find a specialized acronym or term common in your agency. Look it up in a tokenizer (OpenAI provides a free tokenizer tool online). How many tokens does it take? How does the model likely interpret it? Would defining the term in a prompt help? Write a brief note on whether tokenization is affecting your communication with LLMs.

Prompt 3: Fine-tuning Assessment

Does your agency have domain-specific language, formats, or standards that a generic LLM might not know? List 3-5 examples. Do you think fine-tuning a base model on your documents would improve its outputs? What would be required (data quality, training time, expertise)?

Prompt 4: Hallucination Vulnerability

Think of a use case where your agency might deploy an LLM. What factual errors would be most damaging? What oversight would catch them? What percentage of errors would make the system unusable? Document this; it's your baseline for acceptable performance.

Prompt 5: RLHF and Values

If your agency were to use RLHF to align an LLM with your values, what would those values be? What kind of responses would be "good" vs. "bad"? How would you ensure human raters understood and applied those values consistently?

KEY TAKEAWAYS

• Transformers use attention mechanisms to understand relationships between parts of text, enabling context-dependent understanding impossible in older architectures.
• Tokenization breaks text into subword units before processing. This affects how models interpret acronyms and specialized terms. Understanding tokenization helps you write better prompts.
• Transformers are built in layers. Each layer refines understanding through attention. By the top layer, the model has rich, context-dependent representations.
• LLMs generate text probabilistically, predicting one token at a time. This is why the same prompt can produce different outputs; different token samples at different steps create variation.
• Fine-tuning specializes a pre-trained model for specific domains or tasks. For government, fine-tuning on agency documents, policies, and formats can make models more accurate and useful for your context.
• RLHF aligns models with human preferences. It's how safety measures get added and how models become more helpful. If human preferences are biased, RLHF amplifies those biases.
• LLM outputs look authoritative but are not inherently accurate. Hallucinations are common. All factual claims must be verified. Use models for drafting and synthesis, not for fact-finding.
• Prompt engineering can improve outputs, but it's not a substitute for real governance, testing, or human oversight.

TERMS AND GLOSSARY

• Attention Mechanism: The core innovation of transformers. It learns to weight relationships between words based on relevance, enabling context-dependent understanding.
• Tokenization: Breaking text into subword units (tokens) before processing. Allows models to handle rare words, new words, and misspellings.
• Token: A subword unit of text. "Running" might be one or two tokens depending on the tokenizer. Models process text as token sequences.
• Transformer: An architecture built on attention mechanisms. Processes entire sequences in parallel (unlike older networks). Enables understanding of long-distance relationships.
• Pre-training: The first phase of LLM training, where models learn patterns from massive amounts of raw text by predicting the next token.
• Fine-tuning: The second phase of LLM training, where a pre-trained model is specialized for specific tasks or domains using curated examples.
• RLHF (Reinforcement Learning from Human Feedback): A training technique where human raters rank model outputs, a reward model learns to predict quality, and the original model is fine-tuned to maximize reward.
• Hallucination: When an LLM generates confident-sounding but false outputs. The model isn't lying; it's predicting plausible text that happens to be inaccurate.
• Large Language Model (LLM): A transformer-based model trained on billions of words to predict text. Examples: GPT-4, Claude, Gemini.

You now understand the conceptual architecture of the systems most of your government agencies are deploying. This matters because it shapes how you use them, how you govern them, how you test them.

Key insight: Transformers are pattern-matching systems, not reasoning systems. They're very good at pattern-matching (which is why they're useful for many government tasks). But they're not actually thinking. They're predicting the most likely continuation of a pattern. This is both their strength and their limitation.

Their strength: They can synthesize information, draft documents, answer questions, explain concepts, because all of those are pattern-completion tasks.

Their limitation: They don't inherently know what's true. They predict what's statistically likely in their training data. If something is statistically likely but false, they'll produce it confidently.

This is why government needs to build governance around LLMs that includes human oversight. Not because the models are malicious or stupid, but because their nature is probabilistic pattern-matching, not truth-seeking.

Also, now that you understand transformers, you understand why vendor choices matter. Different vendors apply different fine-tuning, different RLHF, different safety measures. A model from Vendor A and Vendor B might produce different outputs for the same prompt, not because one is "right" and one is "wrong," but because they were trained and fine-tuned differently.

Take two minutes: Think about one LLM you've used recently. ChatGPT, Claude, Gemini, or a government-specific tool.

Recall: What did you ask it to do? How accurate was the output? What would have happened if you'd used it without verification?

Reflect: Now that you understand that LLMs predict the most likely next token given context, does that change how you think about the accuracy of the output? What kind of oversight would be appropriate for this use case?

Note: This reflection will inform how you approach governance in later lectures.

You've completed Lecture 2.1.2. You now understand transformers and LLMs at a conceptual level. You understand attention, tokenization, pre-training, fine-tuning, and RLHF. You know why they hallucinate, why they produce different outputs, and what their limitations are.

Next lecture (2.1.3: Generative AI Deep Dive), we'll zoom out. We'll look at different types of generative AI beyond just language: image generation, code generation, video generation. We'll understand diffusion models and how image generation works differently than language generation. We'll explore government applications and risks for each modality.

After that comes a full chapter on multimodal AI -- systems that work across text, images, audio, and video simultaneously. These are the systems that will enable government to analyze satellite imagery, process hours of video footage, or transcribe and summarize citizen calls.

Keep building. Each lecture is a foundation for the next.

Government AI CLUB Certification Program

Level 2: AI Ready | Transformers and LLMs | Lecture 2.1.2

A GOVT.CLUB initiative.

<- 2.1.1 Supervised vs. Unsupervised vs. Reinforcement Learning
2.1.3 Generative AI Deep Dive ->

Start Your CLUB Certification

This lecture is part of L2: AI Practitioner -- 40 hours of comprehensive government AI training.

Explore CLUB Certification

Related Lectures

L2
2.1.1 -- Supervised vs. Unsupervised vs. Reinforcement Learning
60 min - Video + Interactive

L2
2.1.3 -- Generative AI Deep Dive
60 min - Video + Demos

L2
2.1.4 -- Data Quality and AI Performance
60 min - Video + Lab