Learning Objectives

By the end of this lecture you will (1) characterize 'mission-critical' government functions as those where failure causes loss of life, significant harm to the economy, loss of critical capabilities, or irreversible damage to public trust; (2) apply the Presidential Policy Directive 21 (PPD-21) framework of 16 critical infrastructure sectors and the 2024 update via National Security Memorandum 22 (NSM-22), together with sector-specific risk management agency (SRMA) responsibilities; (3) describe AI applications in specific mission-critical domains: DoD command and control, ODNI/IC analytic support, FAA air traffic management, FEMA disaster response, DOE energy grid management, CDC public health emergency response, IRS tax administration, VA veterans health, and SSA disability adjudication; (4) apply OMB M-24-10 safety-impacting AI minimum practices and NSM-10 national security AI considerations; (5) evaluate operational design including human-in-the-loop (HITL), human-on-the-loop (HOTL), and full automation boundaries, especially for safety-critical decisions; (6) interpret test, evaluation, verification, and validation (TEVV) requirements including DoDI 5000.87 software acquisition pathway, DoD Responsible AI Strategy, and FDA-style predetermined change control for non-defense AI; (7) design resilience including failure modes and effects analysis (FMEA), fail-safe and graceful-degradation design, fallback procedures, and incident response coordinated with the National Response Framework; and (8) apply lessons from historical case studies including Patriot missile battery Gulf War 2003, Air France 447 automation handoff, Tesla Autopilot incidents, and MAX 737 MCAS as warnings, alongside successful federal deployments.

What Makes a Government Function Mission-Critical

Mission-critical is a term frequently used loosely. For federal AI purposes we apply four criteria. First, loss of life implication: failure can directly or indirectly cause death or serious injury. Second, economic harm: failure can cause catastrophic economic damage to the nation, a sector, or key infrastructure. Third, critical capability loss: failure prevents the government from performing a legally or politically required function. Fourth, irreversible trust damage: failure erodes public trust in ways that are difficult to rebuild. PPD-21 (February 2013) identified 16 critical infrastructure sectors: Chemical, Commercial Facilities, Communications, Critical Manufacturing, Dams, Defense Industrial Base, Emergency Services, Energy, Financial Services, Food and Agriculture, Government Services and Facilities, Healthcare and Public Health, Information Technology, Nuclear Reactors, Transportation Systems, and Water and Wastewater. NSM-22 (April 30, 2024) replaced PPD-21 and restated the framework. The CISA Sector Risk Management Agency (SRMA) for each sector provides coordination. DHS Safety and Security Guidelines for Critical Infrastructure Owners and Operators (April 2024) specifically addresses AI across these sectors. OMB M-24-10 designates safety-impacting AI as AI that can materially affect life, health, safety, critical infrastructure, or the environment; these are the federal AI systems most commonly aligned with mission-critical functions.

Defense Command and Control

DoD command and control integrates AI through CDAO Task Force Lima for generative AI, Project Maven for intelligence from imagery, ADVANA for data integration, and the Joint All-Domain Command and Control (JADC2) concept. DoDD 3000.09 (updated January 2023) governs autonomy in weapons systems, requiring appropriate levels of human judgment over the use of force. DoD's Responsible AI Strategy and Implementation Pathway emphasizes Test and Evaluation, Verification and Validation (TEVV). DoDI 5000.87 provides the Software Acquisition Pathway. AUKUS Pillar 2 accelerates cooperation with UK and Australia on AI. Specific safety considerations include distinguishing between advisory AI, operator-confirmed AI, and fully autonomous systems; DoD policy disfavors full autonomy for lethal force absent specific authorization. Historical cases inform design: Patriot missile battery in 2003 Iraq War accidentally engaged friendly aircraft due to interaction between automation and human oversight; Aegis USS Vincennes in 1988 Iran Air 655 incident showed automation bias under stress; Air France 447 in 2009 showed automation handoff risk. Test ranges including Yuma, Nellis, and the Sea Range provide operational test environments. Red-teaming against AI-enabled adversaries is increasingly a requirement.

Intelligence Community Analytic Support

The IC uses AI for imagery analysis, signal intelligence exploitation, counterintelligence, and open-source intelligence triage. ODNI's AI Ethics Principles (2020) and the subsequent Risk Management Framework guide implementation. ICD 503 governs risk management for IC systems. Classification and compartment controls require air-gapped or cross-domain deployment. Analytic judgment remains human-led; AI accelerates triage and pattern detection but does not replace the analyst's sourced, reasoned judgment per Analytic Tradecraft Standards (ICD 203). Specific deployments include National Geospatial-Intelligence Agency (NGA) programs for imagery analysis, National Security Agency (NSA) signal-intelligence AI, Defense Intelligence Agency (DIA) adversary analysis, CIA Open Source Enterprise, and FBI counterintelligence AI. The Cyber Threat Intelligence Integration Center (CTIIC) provides coordination. Red-teaming is crucial given state-sponsored adversary efforts to manipulate or evade analytic AI. Foreign-origin AI components are screened carefully; CFIUS review of vendor transactions applies.

FAA Air Traffic Management

FAA uses AI for arrival/departure prediction, weather-impact analysis, runway assignment optimization, and conflict detection support. The Airport Surface Surveillance Capability (ASSC) integrates multiple sensors with ML-based conflict prediction. NextGen modernization includes AI-adjacent capabilities. NTSB investigations of automation-related incidents including Air France 447, Asiana 214, and others drive design. FAA certification standards (Part 25, 23, 27, 29) are being updated for AI-enabled avionics through guidance such as AC 20-174 and RTCA DO-178C/DO-254 with ML extensions via RTCA SC-205 work. EASA has parallel efforts including the AI Roadmap. Human-in-the-loop is the standard in safety-critical avionics and air traffic control. The Aviation Safety Reporting System (ASRS) captures operational safety data. Federal aviation AI deployments emphasize graceful degradation: if the AI is unavailable or uncertain, controllers and pilots retain full capability.

FEMA Disaster Response

FEMA applies AI for damage assessment through the Individuals and Households Program, predictive analytics for resource positioning, satellite imagery analysis with NGA and USGS collaboration, and crisis communication through the Integrated Public Alert and Warning System (IPAWS). The Stafford Act provides authority. The National Response Framework and the National Incident Management System (NIMS) structure federal response. AI is currently advisory in disaster response; final resource decisions remain with human incident commanders. Key risks include model drift as climate change shifts disaster patterns, equity concerns as damage assessment algorithms have historically favored higher-value properties in some studies, and data quality across varied incident types. FEMA has published its AI Strategy and coordinates with DHS S&T on AI for emergency management. Cross-agency coordination includes USDA for agricultural disasters, EPA for environmental disasters, HHS/CDC for health emergencies.

Energy Grid and DOE

Department of Energy and its National Laboratories (Argonne, Oak Ridge, Lawrence Livermore, Lawrence Berkeley, Los Alamos, Pacific Northwest, Sandia, Idaho, Brookhaven, SLAC, Jefferson Lab, Ames, NREL, PPPL, Fermilab, and SRNL) apply AI extensively. Power grid applications include demand forecasting, state estimation, stability assessment, and distribution management. Federal Energy Regulatory Commission (FERC) oversight and NERC Critical Infrastructure Protection (CIP) standards apply. The Cybersecurity for Energy Delivery Systems (CEDS) program addresses cyber-physical risks. AI for grid must meet stringent reliability requirements; false triggers can cause outages, and missed indicators can cascade. The 2003 Northeast blackout and the 2021 Texas winter storm illustrate the consequences of grid failure. Nuclear grid applications have additional NRC oversight for safety-related systems. Distribution utilities adopt AI more slowly due to capital cycles. Gridwise-type coordination structures emerging under DOE Grid Modernization Initiative.

CDC and Public Health Emergency

CDC applies AI for syndromic surveillance (National Syndromic Surveillance Program), forecasting through the Center for Forecasting and Outbreak Analytics (CFA, established 2021), genomic epidemiology for SARS-CoV-2 and other pathogens, and vaccine effectiveness monitoring. The Public Health Service Act and 42 CFR 70-71 authorize quarantine and isolation. The Pandemic and All-Hazards Preparedness Act supports preparedness planning. COVID-19 exposed weaknesses: variable EHR data quality, limited data sharing with state/local, latency in reporting. CFA's design addresses these through durable data pipelines, academic partnerships through Insight Net, publicly available code and evaluation, and integration with MMWR communication. Collaboration with HHS ASPR, NIH, FDA, state public health authorities, and the WHO is essential. Forecast accuracy during novel outbreaks is limited; CFA uses ensemble approaches and explicitly communicates uncertainty. Equity considerations include ensuring surveillance covers minority-serving institutions, tribal health, rural areas, and LEP populations.

IRS Tax Administration

IRS uses AI and advanced analytics for audit selection (Discriminant Function System updates), earned income tax credit review, identity verification, tax fraud detection, and taxpayer service. The Taxpayer First Act (2019) and the Inflation Reduction Act (2022) funded modernization. IRS procurement history includes the ID.me facial recognition rollback of 2022 as a cautionary tale. Current identity verification runs through Login.gov with optional credential service providers. Taxpayer rights are enumerated in the Taxpayer Bill of Rights. Fairness across income brackets, demographic groups, and tax preparer populations is actively evaluated; GAO and TIGTA audits apply. The National Taxpayer Advocate provides independent review. Audit selection AI must be contestable; taxpayers have appeal rights under the Internal Revenue Code. Generative AI for IRS customer service (Taxpayer Experience) is advancing cautiously; hallucination is particularly dangerous in tax advice.

SSA Disability Adjudication

SSA adjudicates millions of disability claims annually through Disability Determination Services at state level and ALJ hearings. AI assists in case processing, medical evidence retrieval, and ALJ decision support. The Social Security Act and 20 CFR Part 404 and 416 govern. Administrative Procedure Act due process applies. Historical criticism of SSA disability adjudication includes variability across ALJs and processing delays; AI could help standardize but also risks amplifying historical biases. Federal Advisory Committee Act structures oversight. NAS reports and GAO audits have examined SSA AI. Equity considerations are substantial: claimants face long processing times during which some die; disparate impact on rural, LEP, and disability-population subgroups is a recognized risk. Transparency to claimants of AI involvement and human oversight is a due-process concern under Mathews v. Eldridge and related jurisprudence.

Human Oversight Levels

For mission-critical AI, the operating model specifies human oversight level. Human-in-the-loop (HITL): human authorizes each AI decision before it takes effect. Appropriate for high-stakes individual decisions including benefits adjudication and weapons release. Human-on-the-loop (HOTL): AI operates autonomously but human can intervene; appropriate for time-sensitive applications where human cannot authorize each step, such as air defense. Fully autonomous: no human intervention during execution; reserved for applications with pre-authorized rules, bounded scenarios, and robust safety cases. OMB M-24-10 minimum practices require human oversight for safety-impacting and rights-impacting AI. DoDD 3000.09 requires appropriate human judgment over lethal force. FAA certification typically requires pilot or controller authority retention. The key design question is not whether human oversight exists, but whether the human has meaningful ability to understand, evaluate, and override - meaningful in terms of time, information, and cognitive load. Automation complacency is a well-documented failure mode; training, workflow design, and audits mitigate.

Resilience and Fallback

Mission-critical AI must have resilience designed in. Failure modes and effects analysis (FMEA) identifies how the AI can fail and the consequence. Fail-safe design ensures that failure defaults to a safe state; fail-operational design maintains operation in degraded mode. Graceful degradation preserves useful function even when AI is unavailable or uncertain. Fallback procedures specify what humans and systems do without AI. Redundancy at data, model, and infrastructure layers increases availability but also attack surface; it must be managed. Cross-domain dependencies (power, communications, cyber) are assessed. The National Institute of Standards and Technology's SP 800-160 on systems security engineering and NIST AI RMF Measure function address these. After-action reviews following incidents (near-miss or actual) feed continuous improvement. Joint exercises across agencies including FEMA's Tier I and Tier II exercises and DoD's Large Scale Exercises test end-to-end resilience including AI components.

Mission-Critical AI Anti-Patterns

(1) Autonomy Creep: a system designed for advisory use gradually becomes authoritative without explicit re-authorization; Patriot battery 2003 illustrates. (2) Automation Complacency: humans defer to AI without critical review; Air France 447 and Asiana 214 are aviation examples; MAX 737 MCAS is a software example. (3) Single-Point Dependency: mission-critical function depends on one vendor, one region, one data source; a failure cascades. (4) Testing-in-Production: deploying without TEVV; most federal AI failures have this as a contributor. (5) Metric-Only Safety: measuring accuracy without considering rare but catastrophic failure modes. (6) No Fallback: no documented procedure for AI unavailability. (7) Missing Incident Response: no playbook for AI-specific failure. (8) Over-Trust of Vendor Safety Case: vendor claims about safety accepted without independent verification. (9) Certification-Free Operation: flying without certification basis; federal safety-critical typically requires certification. (10) Ignoring Analogous Failures: repeating known failure patterns. Each anti-pattern has documented precedent; a mature mission-critical AI program audits against this list.

Summary and Next Steps

Mission-critical government functions span defense, intelligence, air traffic, disaster response, energy, public health, tax administration, veteran health, and disability adjudication. Each has specific authorities, specific safety disciplines, and specific AI patterns. OMB M-24-10 safety-impacting AI minimum practices and NSM-10 national security AI policy apply broadly; sector-specific rules overlay. Human oversight levels (HITL, HOTL, autonomous) must be chosen deliberately. Resilience requires FMEA, fail-safe and graceful degradation, fallback procedures, and cross-agency exercise. Anti-patterns including autonomy creep, automation complacency, single-point dependency, and missing fallback are well documented and avoidable. The next lecture, AI in Defense and National Security, goes deeper into the defense and intelligence specific considerations.