Emerging Architectures: Agentic AI and Compound Systems

Learning Objectives

After completing this lecture, you will be able to:

• Understand the key concepts of emerging architectures: agentic ai and compound systems in a government context
• Connect emerging architectures: agentic ai and compound systems to your agency's AI initiatives
• Identify next steps for applying these concepts in your role

Key Topics Covered

-
Agent frameworks

-
Orchestration patterns

-
Multi-agent systems

-
Government implications

Why This Matters for Government

Overview

Government agencies face unique challenges when it comes to AI adoption. This lecture addresses these challenges head-on by providing cios, chief ai officers, agency technology leaders with the knowledge and frameworks needed to navigate AI in the public sector responsibly and effectively.

As part of the L4 (AI Architect) curriculum, this lecture builds on the foundational principle that every AI system in government ultimately serves citizens. Whether you are working with AI tools daily or setting strategy for your agency, understanding emerging architectures: agentic ai and compound systems is essential for responsible, effective government AI adoption.

Lecture URL: https://skill.re/learn/govt/emerging-architectures-agentic-ai-and-compound-systems.php

======================================================================

TRANSCRIPT: Emerging Architectures: Agentic AI and Compound Systems

======================================================================

You will understand agentic AI systems that act autonomously with oversight, compound systems

that combine multiple AI models, and orchestration patterns for managing this complexity in

government environments.

An agentic AI system is different from a classifier or a chatbot. It makes decisions. It takes

actions. It might send emails, update databases, or initiate processes. And then it does it again

tomorrow without you asking.

That's powerful. And it's scary.

For government, agency systems offer real opportunities: automating routine decisions, escalating

complex ones, freeing staff for higher-value work. But they also raise governance questions we

haven't had to answer before.

By the end of this session, you'll understand how agent systems work, what oversight they need,

and how to build compound systems that combine multiple AI models safely.

What is Agentic AI?

Traditional AI:

• User provides input
• AI processes it
• AI returns output
• User decides what to do with the output

Agentic AI:

• Goal is set (human or system)
• Agent receives goal
• Agent plans steps to achieve goal
• Agent executes steps (with human oversight)
• Agent monitors progress
• Agent replans if needed
• Agent achieves goal or escalates to human

The key difference: The agent doesn't just respond. It acts repeatedly toward a goal.

Examples:

• Benefit processing: Goal is to process a benefit application. Agent retrieves data, checks

eligibility, sends notifications, updates database. All without human intervention.

• Fraud investigation: Goal is to investigate a flagged transaction. Agent gathers evidence,

queries multiple databases, generates a report, recommends action.

• Hiring: Goal is to screen resumes. Agent reads submissions, evaluates against criteria,

schedules interviews, sends communications.

Why It Matters for Government

Government work is repetitive. Caseworkers spend 60% of their time on routine tasks:

• Checking eligibility
• Retrieving information from multiple systems
• Filling out forms
• Sending notifications

Agentic AI can automate all of that. Caseworkers focus on complex cases, exceptions, human judgment.

CORE CONCEPTS

An agentic system has several components:

GOAL SPECIFICATION

What is the agent trying to achieve? This must be precise.

Bad goal: "Process benefit applications"

Better goal: "Determine if applicant is eligible for program X based on criteria Y and Z, and if

eligible, initiate benefit payment within 48 hours"

PLANNING

Given a goal, how does the agent break it down into steps?

Example: Goal is to determine eligibility.

• Retrieve applicant income from tax records
• Retrieve household composition from social security
• Retrieve existing benefits from welfare system
• Apply eligibility rules
• Return decision with confidence

TOOLS

What can the agent act on?

Examples:

• Retrieve data from a database
• Call an API
• Send an email
• Update a record
• Escalate to human

MONITORING

How does the agent know if it's on track? What does it do if something goes wrong?

Examples:

• Data retrieval fails -> Retry 3 times, then escalate
• Eligibility is ambiguous -> Escalate to human for judgment
• New information suggests ineligibility -> Revise decision

HUMAN OVERSIGHT

Where does a human get involved?

• Approval before action: Human approves before agent sends notifications
• Exception handling: Agent escalates ambiguous cases to human
• Regular audits: Sample agent decisions regularly
• Appeal mechanism: Affected parties can request human review

CORE CONCEPTS

Most interesting government problems require multiple AI models, not just one.

Example: Benefit fraud detection system

Model 1: Income verification

• Input: Reported income, tax records
• Output: Likelihood reported income is accurate

Model 2: Benefit eligibility

• Input: Income, household composition, existing benefits
• Output: Whether applicant is eligible

Model 3: Fraud detection

• Input: Application patterns, historical data, reported vs. actual income
• Output: Probability of fraud

Model 4: Risk assessment

• Input: All the above, plus caseworker judgment
• Output: Risk level (low/medium/high)

These models chain together. Output of one becomes input to the next.

ORCHESTRATION PATTERNS

How do these models work together?

Sequential: Model 1 -> Model 2 -> Model 3. Output of one feeds the next.

Useful when you need a pipeline of decisions.

Parallel: Run models independently, combine results.

Useful when models answer different questions.

Conditional: If Model 1 returns X, use Model 2. If Model 1 returns Y, use Model 3.

Useful when different paths apply to different cases.

CONSENSUS

When multiple models disagree, how do you decide?

Options:

• Majority vote: 2 out of 3 say fraud. Decision: fraud.
• Weighted voting: Different models have different weights based on historical accuracy.
• Ensemble: Combine all predictions into a single prediction.
• Escalation: If models disagree beyond threshold, escalate to human.

FALLBACK

What if a model fails?

Options:

• Use previous version of the model
• Use a simpler, more reliable model
• Use historical decision patterns
• Escalate to human

Pattern 1: Goal Specification That's Too Vague

You tell the agent to "process benefit applications efficiently." The agent interprets this as

"approve all applications quickly." Now you have massively overpaying benefits.

Mitigation: Goals must be specific and measurable. "Process applications in 95% accuracy on eligibility determination."

Pattern 2: Insufficient Human Oversight

You build an agentic system but don't include human oversight. The system makes a decision that's

clearly wrong, but nobody catches it until hundreds of applications have been processed.

Mitigation: Mandatory human approval for a percentage of decisions (even if they seem routine).

Automated escalation for ambiguous cases. Regular audits.

Pattern 3: Compounding Model Errors

Model 1 is 90% accurate. Model 2 is 90% accurate. You chain them. Combined accuracy is 81%. You

didn't account for error compounding.

Mitigation: Test the full pipeline, not just individual models. Measure end-to-end accuracy.

Pattern 4: Unchangeable Agent Decisions

The agent made a decision. It's wrong. But undoing it is complex because the agent initiated

multiple downstream processes. Now you're stuck.

Mitigation: All agent actions must be reversible or auditable. Clear rollback procedures.

• AGENT IDENTIFICATION

For your agency:

• What processes are most repetitive and rule-based?
• Which could be automated with an agentic system?
• What are the risks if they're automated?
• COMPOUND SYSTEM DESIGN

For a specific use case:

• What models would you need?
• How would they chain together?
• What oversight would you require?
• GOVERNANCE FRAMEWORK

For an agentic system:

• Who approves goals?
• What percentage of agent decisions need human review?
• What escalation paths exist?
• How do affected parties appeal decisions?
• Agentic AI is powerful but requires careful governance. It makes repeated decisions without

waiting for human input.

• Goals must be specific. Vague goals lead to unexpected behavior.
• Humans must be in the loop, at least for oversight. Not every decision needs human approval,

but some always should.

• Compound systems multiply complexity. Test them end-to-end, not just individual models.
• Reversibility matters. You need to be able to undo agent decisions if they're wrong.

Agentic AI: AI system that autonomously pursues a goal, taking repeated actions and replanning

as needed.

Compound System: System combining multiple AI models that work together to solve a problem.

Tool: Any action an agent can take (API call, database query, email, etc.).

Orchestration: How compound systems coordinate between multiple models.

Escalation: Process of sending ambiguous or risky decisions to humans for review.

Start with low-risk use cases: Routine data processing, document classification, notification

sending. Prove the pattern works. Then expand to higher-risk cases.

Invest in governance infrastructure early. It's easier to add rules when building than to retrofit

them.

• If you could delegate one job function entirely to an agentic AI system, what would it be?
• What could go wrong? How would you detect it?
• What would employees affected by this automation need to transition to new roles?
• How would you test such a system before rolling it out?

Agentic AI is the next frontier for government AI adoption. Done well, it can transform how

government delivers services. Done poorly, it can cause significant harm.

The agencies that succeed will be those that build robust governance first, then capability.

Government AI CLUB Certification Program

Level 4: AI Architect | Emerging Architectures: Agentic AI and Compound Systems | Lecture 4.1.8

A GOVT.CLUB initiative.

<- 4.1.8 Technical Debt Management in AI
4.2.1 Drafting Agency AI Policies ->

Start Your CLUB Certification

This lecture is part of L4: AI Architect -- 120 hours of comprehensive government AI training.

Explore CLUB Certification

Related Lectures

L4
4.1.1 -- Designing Agency-Wide AI Platforms
180 min - Lecture + Workshop

L4
4.1.2 -- Multi-Cloud, Multi-Model Strategy
150 min - Lecture + Strategy Exercise

L4
4.1.3 -- Data Mesh and Data Fabric for Government
150 min - Lecture + Architecture