Learning Objectives

After completing this lecture, you will be able to design, operate, and supervise cybersecurity for AI systems in a federal environment consistent with FISMA, the NIST Cybersecurity Framework 2.0, NIST Special Publication 800-53 Rev. 5, NIST AI Risk Management Framework 1.0, FedRAMP, CISA secure-by-design guidance, Executive Order 14028 on Improving the Nation's Cybersecurity, and Executive Order 14110 on Safe, Secure, and Trustworthy Development and Use of AI.

Specifically, you will be able to:

1. Describe the AI-specific threat model, including model theft, model inversion and reconstruction, training-data poisoning, adversarial examples, prompt injection, prompt leakage, indirect prompt injection via retrieved content, supply-chain compromise of model weights, and abuse of agentic AI systems.
2. Map AI cybersecurity controls to NIST SP 800-53 Rev. 5 families and to the AI-specific overlay emerging from NIST's AI RMF and the MITRE ATLAS framework for Adversarial Threat Landscape for AI Systems.
3. Identify the CISA secure-by-design principles that apply to AI systems, and translate them into concrete engineering practices at the data, training, inference, and monitoring layers.
4. Apply Executive Order 14028 (2021) obligations for federal software supply chain to AI components, including Software Bill of Materials (SBOM) generation for models and data, attestations to NIST Secure Software Development Framework, and federal incident response through CISA.
5. Execute FISMA incident-response obligations for AI-specific incidents: adversarial input exploitation, training-data poisoning discovery, model theft or leak, prompt-injection exploitation, compromise of a retrieval source, or compromise of an agentic tool.
6. Apply the CJIS Security Policy, CMMC/CMMC 2.0 for Controlled Unclassified Information in the defense industrial base, IRS Publication 1075 for federal tax information, and HIPAA/HITRUST for health data to AI systems that handle these data categories.
7. Design red-team exercises specifically for AI, including adversarial example generation, jailbreak testing, prompt-injection testing, data-poisoning simulation, and agentic-tool abuse testing, aligned with NIST's AI red-teaming guidance and the AI Safety Institute's expectations.
8. Communicate AI cybersecurity risk to agency leadership, the AI Governance Board, OMB, and congressional oversight in terms that connect to established federal cybersecurity vocabulary while surfacing AI-specific concerns.

Key Topics Covered

This lecture covers cybersecurity for AI systems in federal environments. Topics include:

AI-specific threat model. The ways AI systems fail differently than traditional software: adversarial examples (inputs crafted to mislead the model); training-data poisoning (malicious data introduced to corrupt a model); model theft (extraction of model weights through repeated queries or direct exfiltration); model inversion (reconstruction of training data from the model); prompt injection (malicious instructions embedded in input); indirect prompt injection (malicious instructions embedded in retrieved content such as web pages, documents, or emails); prompt leakage (exposure of system prompts and proprietary context); agentic tool abuse (misuse of AI-controlled tools to perform unauthorized actions); jailbreaks (bypass of safety guardrails); and supply-chain compromise of model weights, training data, or pre-trained components.

Federal control frameworks. How existing federal cybersecurity frameworks apply to AI: FISMA as the enabling statute; the NIST Cybersecurity Framework 2.0 as the overarching posture; NIST SP 800-53 Rev. 5 as the control catalog; NIST SP 800-37 Rev. 2 for risk management; NIST SP 800-171 for Controlled Unclassified Information in non-federal systems; FedRAMP for cloud services; and the NIST AI RMF for AI-specific considerations.

AI security overlays. The AI-specific controls that extend SP 800-53: adversarial-input monitoring; model integrity verification; prompt-injection resistance; agent-tool permissioning; data-poisoning detection; model-level access control; model-inference audit logging; and supply-chain attestation for model artifacts and training data.

CISA secure-by-design. How Cybersecurity and Infrastructure Security Agency's secure-by-design principles apply to AI: secure defaults, ownership of security outcomes by manufacturers, and transparency to customers. Agency deployments demand these properties of vendors.

Executive Order 14028 software supply chain. How EO 14028 and OMB M-22-18 obligations apply to AI: SBOM for model components, attestation to NIST SSDF, vulnerability disclosure programs, and federal incident reporting through CISA.

MITRE ATLAS. The Adversarial Threat Landscape for AI Systems framework, which catalogs adversary tactics and techniques against AI systems for use in threat modeling and red-team exercises.

Red-teaming AI. How to run AI red-team exercises, including adversarial example generation, jailbreak probes, prompt-injection testing, indirect-prompt-injection testing through retrieval sources, data-poisoning simulation, agent-tool abuse scenarios, and reporting.

Incident response for AI. How to apply FISMA incident response to AI-specific incidents, including classification under US-CERT taxonomy, reporting to CISA, notification to OMB, and coordination with the agency's Chief Information Security Officer and Senior Agency Official for Privacy when applicable.

Classified and sensitive-data considerations. How CJIS Security Policy, IRS Publication 1075, HIPAA/HITRUST, DoD Impact Levels, and compartmented-information rules apply to AI systems that process those data categories.

Vendor and supply-chain risk. How to evaluate AI vendors for cybersecurity posture, from FedRAMP status to SBOM production to vulnerability-disclosure practices to exit rights.

Why This Matters for Government

AI systems are software, so they inherit every traditional cybersecurity concern. They are also a new class of software that fails in ways traditional controls do not catch. Federal cybersecurity, long organized around FISMA, NIST SP 800-53, and the NIST Cybersecurity Framework, now has to extend to address AI-specific failure modes without losing the durable discipline that FISMA provides.

The federal government has already experienced, and publicly reported, adversarial AI incidents. The National Institute of Standards and Technology's Face Recognition Vendor Test has documented years of demographic-disparity and adversarial-image performance variation across vendors, informing DHS, CBP, TSA, and state and local customer deployments. The Cybersecurity and Infrastructure Security Agency, jointly with the National Cyber Security Centre of the United Kingdom, has issued guidance on secure AI system development (November 2023) that federal agencies now rely on for vendor evaluation. OMB M-24-10 requires agencies to inventory AI systems and apply minimum practices that include cybersecurity. Executive Order 14028 (May 2021) on Improving the Nation's Cybersecurity imposes software-supply-chain obligations that now extend to AI model weights, training data, and pre-trained components. Executive Order 14110 (October 2023) adds AI-specific security obligations including red-teaming requirements for advanced AI models.

The practical consequence is that an L2 practitioner must know two vocabularies and keep them synchronized. The first vocabulary is traditional federal cybersecurity: FISMA, ATO, SP 800-53, SP 800-37, FedRAMP, incident reporting via CISA, the Cyber Incident Reporting for Critical Infrastructure Act (CIRCIA) when applicable, FIPS 140-3 for cryptography, and FIPS 199/200 for categorization. The second vocabulary is AI-specific security: MITRE ATLAS tactics and techniques; NIST AI RMF Govern/Map/Measure/Manage; AI red-teaming practices; prompt-injection and indirect-prompt-injection defenses; model theft and model inversion defenses; training-data poisoning defenses; agentic-tool permissioning; and model/data/weight supply-chain integrity.

Real federal examples illustrate the stakes. In 2024, CISA and allied cyber agencies issued joint guidance on deploying AI systems securely, reflecting pattern-of-practice across sectors. NIST's AI Safety Institute began pre-deployment testing of advanced models under EO 14110 reporting expectations. DHS's CISA stood up AI-specific advisories that agencies are expected to monitor. The Department of Defense's JAIC and subsequent CDAO published AI-specific cybersecurity practices. IRS Publication 1075 (Safeguards Program) compliance work now includes AI systems that touch federal tax information, with explicit obligations for access control, logging, encryption, and training. HIPAA Security Rule compliance, likewise, now includes AI systems that handle PHI, with explicit obligations around access, logging, risk analysis, and breach notification.

The failure modes are not theoretical. Model theft has been demonstrated at scale against commercial LLMs, allowing reconstruction of proprietary models from API access. Training-data poisoning has been demonstrated in academic literature, allowing an adversary to insert undetectable backdoors. Prompt-injection attacks have moved out of the laboratory into active exploitation, including via indirect injection through retrieval sources like websites, PDFs, and emails that agentic systems read. Agentic systems with tool access have been abused to perform actions the operator did not intend. Each of these is a real incident category that federal agencies have had to incorporate into incident response.

This lecture therefore connects the two vocabularies, shows how the AI-specific concerns overlay on existing FISMA discipline, and walks through worked incident scenarios so the practitioner can recognize, report, and remediate AI cybersecurity incidents using the authorities already in place.

WHY THIS MATTERS FOR GOVERNMENT

Federal cybersecurity depends on a pattern: identify, protect, detect, respond, recover. The NIST Cybersecurity Framework codifies that pattern. AI systems have to fit inside it while also introducing new concerns at each stage.

IDENTIFY.

For AI, identification includes asset inventory (model weights, training data, evaluation data, retrieval indices, prompts/system prompts, agent tool definitions), supply-chain inventory (base models, fine-tune data, pre-trained components), data-classification (FTI under IRS Pub 1075, CJI under CJIS, PHI under HIPAA, CUI under 32 CFR 2002), and risk assessment that includes AI-specific failure modes catalogued in MITRE ATLAS.

PROTECT.

Protection for AI systems layers AI-specific controls onto SP 800-53. Access control (AC family) must cover model weights and inference APIs. Audit logging (AU family) must cover prompts, inference, retrieval, and tool calls. Configuration management (CM family) must cover model versions and retrieval indices. System and communications protection (SC) must cover prompt boundary enforcement and output filtering. System and information integrity (SI) must cover input validation, adversarial-input detection, and prompt-injection resistance. Personnel security (PS) must address insider risk for model-weight access. Supply chain risk management (SR) must cover model provenance and training-data provenance.

DETECT.

Detection for AI includes continuous monitoring of inference traffic for anomalous patterns, monitoring of outputs for policy violations and hallucination indicators, monitoring of retrieval sources for compromise (indirect prompt injection), and telemetry on agent tool calls for unexpected patterns. Detection integrates with the agency's Security Operations Center and with CISA's continuous diagnostics and mitigation (CDM).

RESPOND.

Response for AI follows FISMA incident-response procedures with AI-specific playbooks. Incident classes include: adversarial input exploitation; prompt injection; indirect prompt injection from a compromised retrieval source; prompt leakage; model theft/extraction; training-data poisoning discovery; model-inversion attack; agent-tool abuse; and compromise of base model or fine-tune data. Each class has a specific containment, eradication, and recovery pattern.

RECOVER.

Recovery includes rolling back to a known-good model version, rolling forward with a patched model, rotating compromised prompts and credentials, scrubbing retrieval indices, restoring agent tool permissions, and notifying affected individuals and oversight bodies consistent with the Privacy Act, FISMA, and any sector-specific law.

THE AI-SPECIFIC THREAT CATALOG

The practitioner should memorize the short list of AI-specific threats. Adversarial examples: inputs crafted to be classified incorrectly. Training-data poisoning: malicious data inserted during training. Model theft: extraction of weights via repeated queries or via direct exfiltration. Model inversion: reconstruction of training data from the model. Prompt injection: malicious instructions in input. Indirect prompt injection: malicious instructions in retrieved content. Prompt leakage: exposure of system prompts. Jailbreak: bypass of safety guardrails. Agentic tool abuse: misuse of AI-controlled tools. Supply-chain compromise: corrupted base model or training data.

For each, there is a MITRE ATLAS identifier, a defense pattern, a monitoring pattern, and an incident-response pattern. The rest of this lecture walks through each.

Related Lectures

L2 2.2.1 -- How AI Fails: Failure Modes and Risk Profiles

L2 2.2.3 -- Privacy Engineering for AI

L2 2.2.4 -- Equity and Fairness in Government AI

L2 2.2.5 -- Human Oversight of AI

L3 3.3.3 -- Monitoring AI in Production

L3 3.3.4 -- Incident Response for AI Systems

L4 4.1.x -- Advanced Adversarial Testing

L4 4.2.x -- Red-Teaming AI Systems at Scale

L3 3.5.x -- FedRAMP for AI Services

L3 3.6.x -- Supply Chain Risk Management for AI