Learning Objectives

After completing this lecture on Building Institutional Knowledge for government AI, you will be able to:

• Design knowledge-capture systems that outlast individual administrations and personnel turnover, using OMB M-24-10 inventory requirements, the Federal Records Act (44 USC 3101), and the Presidential and Federal Records Act of 2014 as anchors.
  - Implement AI use case inventories, model cards, data sheets, AI Impact Assessments, lessons-learned repositories, and agency-specific knowledge bases aligned to NIST AI RMF GOVERN functions.
  - Integrate knowledge management across roles: Chief AI Officer, CIO, CISO, CPO, Chief Data Officer (CDO under the OPEN Government Data Act), Chief Records Officer, and program-office leads.
  - Apply GAO Green Book principles for communicating risk and performance upward and sideways so decisions are informed by what the agency already knows.
  - Avoid institutional amnesia by capturing failure modes from IRS ID.me, Michigan MIDAS, Houston HISD EVAAS, SyRI, and Dutch toeslagenaffaire; by running structured after-action reviews; and by maintaining 'agency AI memory' through transitions between political leadership.
  - Connect institutional knowledge to FOIA (5 USC 552), Presidential Records Act handling, Congressional oversight expectations, and GAO audit evidence requirements.

Key Topics Covered

Building institutional knowledge for government AI covers:

1. Knowledge artifacts: use case inventories (published under OMB M-24-10), model cards, data sheets, AI Impact Assessments, risk registers, post-deployment reviews, and incident reports.
   2. Records and retention: Federal Records Act, National Archives and Records Administration (NARA) schedules, General Records Schedule 3.2 for information technology, and agency-specific schedules, applied to AI development artifacts.
   3. Governance continuity: CAIO Council, GSA AI CoE, CIO Council's CISO Council, and cross-agency communities of practice that preserve knowledge across administrations.
   4. Workforce knowledge: IPAs under the Intergovernmental Personnel Act, Presidential Innovation Fellows, US Digital Service tours, Title 5 details, and academic affiliations that carry knowledge in and out.
   5. Tooling: repositories (public-facing agency AI pages, internal wikis, controlled document management), standardized templates, and automation to capture telemetry and evaluation evidence.
   6. Failure capture: structured after-action reviews of incidents, IG and GAO findings, litigation outcomes (e.g., Houston HISD EVAAS, SyRI, K.W. v. Armstrong), public scandals (IRS ID.me, MIDAS, toeslagenaffaire) integrated into agency training and policy.
   7. Public accountability: FOIA readiness, Congressional testimony preparation, engagement with civil society, and academic partnerships under CRADAs and cooperative agreements.

Why This Matters for Government

Institutional knowledge is the memory of a federal agency: the accumulated record of why specific AI systems exist, what authority they operate under, what data they consume, what tradeoffs were made at design time, which evaluation methods were applied, what incidents occurred, how those incidents were remediated, and what an Inspector General, a GAO auditor, a congressional staffer, a journalist, a judge, or a successor political appointee would need to see if they asked. Agencies that capture institutional knowledge deliberately run their AI programs across administrations, personnel turnover, contractor transitions, and litigation without the failure modes that plague agencies where knowledge lives in someone's inbox. Agencies that do not capture it pay enormous costs later. The Veterans Health Administration's National Artificial Intelligence Institute (NAII), NIST's AI Safety Institute, GSA's Federal AI Community of Practice, and mature program offices at NASA, NIH, and NOAA all demonstrate institutional-memory practices that survive transitions. Weaker-memory agencies demonstrate the inverse.

The statutory backbone is the Federal Records Act (44 USC 31), the Presidential and Federal Records Act Amendments of 2014 that closed electronic records loopholes, the National Archives and Records Administration (NARA) general and agency-specific records schedules including General Records Schedule 3.2 for information technology, the Paperwork Reduction Act (44 USC 35), and the Freedom of Information Act (5 USC 552). The OPEN Government Data Act of 2019 created Chief Data Officers in each agency and required agencies to maintain enterprise data inventories. For AI specifically, OMB Memorandum M-24-10 (March 2024) requires agencies to publish an annual AI use-case inventory, designate a Chief AI Officer (CAIO), complete AI Impact Assessments for rights-impacting and safety-impacting AI, and apply minimum risk-management practices that explicitly include documentation. NIST AI RMF 1.0 (January 2023) places documentation at the center of every function: GOVERN 1 on policies and documentation, GOVERN 5 on decision records, MAP 1 on context including organizational history, MEASURE 4 on feedback integration, and MANAGE 4 on continuous monitoring, learning, and incident response. EO 14110 (October 2023) and its implementing guidance further require documentation of testing, red-teaming, and dual-use foundation model reporting. ISO/IEC 42001 (AI management systems) and ISO/IEC 23894 (AI risk management guidance) both institutionalize documentation and continuous improvement.

The case record shows the cost of amnesia. The IRS ID.me crisis of 2022 traced in part to a gap in institutional memory between the 2017 to 2020 identity-proofing pilot discussions and the 2021 to 2022 rollout: the fairness concerns raised earlier did not reach the deployment decision with the force needed. Michigan's MIDAS continued generating false fraud accusations for nearly two years despite internal concerns raised early; those concerns had no durable institutional channel to reach state leadership. Houston Federation of Teachers v. Houston ISD (S.D. Tex. 2017) surfaced that the district could not reconstruct how specific teachers had been scored by the EVAAS model; the court questioned whether due process could be provided when the institution itself did not know. K.W. v. Armstrong (D. Idaho) involved a Medicaid disability-benefits allocation algorithm the state could not fully explain; the court ordered disclosure and reform. SyRI in the Netherlands was struck down in 2020 in part because the government had not preserved and published the evidence that would have demonstrated necessity and proportionality. The Dutch childcare benefits (toeslagenaffaire) scandal was made worse by institutional reluctance to surface internal warnings across ministries, leading to the resignation of the Rutte III cabinet in January 2021 and multiple parliamentary commissions. The UK Post Office Horizon scandal showed that when institutional memory is weak and documentation is controlled by vendors, the public can be harmed on an enormous scale before the record can be reconstructed. Each case is a variation on the same theme: when an agency cannot reconstruct what it did, why, and with what evidence, oversight and remediation become ruinously expensive.

Building institutional knowledge is therefore not a records-management sideline; it is a first-order governance practice for any CAIO under M-24-10. Working in partnership with the Chief Records Officer, the Chief Data Officer, the Senior Agency Official for Privacy, the Chief Information Security Officer, and the General Counsel, the CAIO has to design the artifacts, the retention rules, the review cadence, the public-facing inventory, the internal learning practices, and the transition protocols so that the agency's AI program has durable, auditable memory. Every rights-impacting model should have a living dossier: purpose, authority, intended use, data provenance, model lineage, evaluation history with disparate impact analysis tied to NIST AI RMF MEASURE 2.11 and EEOC Uniform Guidelines, incidents, remediation, waivers, and retirement criteria. Every incident should result in an after-action review (AAR) captured against a standard template, shared inside the agency, and where appropriate published. Every administration transition should involve a structured handoff that covers AI posture, pending decisions, open IG and GAO findings, litigation status, budget state, personnel continuity, and external partnership commitments. The CAIO Council convened by OMB, along with GSA's AI Center of Excellence and 18F, provides interagency memory infrastructure that survives individual administrations. USDS and Presidential Innovation Fellows alumni networks, IPA rotations, and academic affiliations under CRADAs also act as knowledge vehicles. The rest of this seminar turns those principles into a concrete operating model: dossier contents, records schedules, inventory governance, lessons-learned repositories, onboarding packages for new CAIOs, and a transition playbook that makes the agency's AI memory robust to churn.

Overview

Every administration arrives in a federal agency with urgency and leaves with lessons. Without institutional knowledge systems, those lessons walk out the door with each political appointee and each experienced career employee. AI systems magnify that risk. They encode decisions, data choices, and tradeoffs that are hard to reconstruct years later without disciplined capture. Federal agencies with strong institutional knowledge practices, the VA's National AI Institute, NIST's AI Safety Institute, GSA's Federal AI Community of Practice, and mature program offices at NASA, NIH, and NOAA, run their AI programs across transitions without the failure modes that plague agencies where knowledge lives in somebody's inbox.

The statutory backbone is the Federal Records Act (44 USC 31) and NARA's general and agency-specific records schedules. For AI specifically, OMB M-24-10 (March 2024) requires agencies to publish an annual AI use case inventory, designate a Chief AI Officer, complete AI Impact Assessments for rights- and safety-impacting AI, and apply minimum practices. The OPEN Government Data Act (2019) created Chief Data Officers and requires agencies to maintain data inventories. The Paperwork Reduction Act (44 USC 35) governs information collections and their documentation. The Presidential and Federal Records Act of 2014 closed electronic records loopholes. NIST AI RMF GOVERN 1 focuses on policies and documentation; GOVERN 5 on documentation of decisions; MAP 1 on context including organizational history; MEASURE 4 on feedback integration; MANAGE 4 on continuous monitoring, learning, and incident response.

The cases make the cost of amnesia concrete. IRS ID.me's 2022 crisis traced in part to institutional memory loss between the 2017-2020 identity-proofing pilot discussions and the 2021-2022 rollout. Michigan MIDAS continued generating false accusations for nearly two years despite internal concerns raised early; those concerns had no institutional channel to reach leadership. Houston HISD's EVAAS teacher evaluation litigation, Houston Federation of Teachers v. Houston ISD (S.D. Tex. 2017), surfaced that the district could not reconstruct how specific teachers had been scored; institutional records were thin. SyRI and toeslagenaffaire in the Netherlands were made worse by institutional reluctance to surface and share internal warnings across ministries. The K.W. v. Armstrong decision in Idaho involved a disability-benefits algorithm the state could not fully explain because institutional knowledge had not been captured. Each case is a variation on the same theme: when an agency cannot reconstruct what it did, why, and with what evidence, oversight and remediation become very expensive.

Building institutional knowledge is therefore not a records-management sideline; it is a first-order governance practice. The CAIO, working with the Chief Records Officer, CDO, CPO, and CISO, has to design the artifacts, the retention rules, the review cadence, the public-facing inventory, and the internal learning practices so that the agency's AI program has memory. Every rights-impacting model should have a living dossier: purpose, authority, intended use, data provenance, model lineage, evaluation history, incidents, remediation, waivers, and retirement criteria. Every incident should result in a published (where appropriate) after-action review. Every administration transition should involve a structured handoff that covers AI posture, pending decisions, and open risks.

LECTURE #

DURATION: 120 minutes seminar with institutional-memory exercise
FORMAT: Lecture plus structured after-action review (AAR) drill
AUDIENCE: CAIOs, CROs, CDOs, program executives, historians of agency practice
PREREQUISITES: L1-L4 foundation; familiarity with FRA, NARA, M-24-10

THE ARCHITECTURE OF AGENCY AI MEMORY

Section 1: THE DOSSIER
Every rights- or safety-impacting AI system gets a living dossier. Contents include the statutory and regulatory authority; intended use and scope; user population; data sources with provenance, license, and SORN references; model architecture and version lineage; training and evaluation history including fairness metrics tied to NIST AI RMF MEASURE 2.11; known limitations and failure modes; waivers; incidents with after-action outcomes; retirement criteria; and responsible role IDs. The dossier is updated quarterly and on material change.

Section 2: RECORDS AND RETENTION
NARA General Records Schedule 3.2 covers IT records. Agencies draft specific schedules for AI artifacts. Retention should match the downstream exposure: for rights-impacting AI, long enough to support litigation, GAO review, and affected-individual remedy. Federal Records Act and the Electronic Records Archives demand preservation of the evidence of decisions, not just the decisions themselves. Email, chat (Slack, Teams), code repositories, notebooks, and model weights all create records when they document decisions.

Section 3: USE CASE INVENTORY AS A MEMORY TOOL
OMB M-24-10 requires the inventory; wise CAIOs make it the gateway to the dossier. Each inventory entry links to the public-facing summary, the internal dossier, the most recent AIA, evaluation results, and incident counts. Internal controls prevent inventory rot: quarterly owner certification, automated checks for dossier presence, sample audits by the CAIO office.

Section 4: LESSONS-LEARNED REPOSITORY
A shared repository, inside the agency and, where appropriate, across agencies via the CAIO Council, captures incidents anonymized when needed. Entries include: event description, root cause analysis aligned to NIST AI RMF MEASURE, corrective actions, and policy updates. The CAIO Council compiles inter-agency patterns to warn of emerging issues. IRS ID.me lessons should live in the IRS inventory and be shared to Treasury, SSA, VA, and DHS components.

Section 5: WORKFORCE KNOWLEDGE TRANSFER
Structured onboarding for new CAIOs: briefings with prior CAIOs (Schedule C or career), review of dossiers, IG and GAO findings history, Congressional correspondence, and budget context. IPA assignments with universities are not just talent pipelines; they are knowledge vehicles. PIFs and USDS alumni networks preserve memory. Sabbatical exchanges with NIST AISI, MITRE, and RAND bring outside perspective back.

Section 6: PUBLIC ACCOUNTABILITY AS MEMORY DISCIPLINE
FOIA readiness forces agencies to know what they have. Congressional engagement builds memory by requiring agencies to reconstruct and explain decisions. Civil society engagement through comment periods and advisory boards, academic partnerships under CRADAs, and GAO audits all act as external memory prompts. Agencies that treat these interactions as adversarial lose memory; agencies that treat them as learning opportunities gain it.

Section 7: TRANSITION PLAYBOOK
Every administration transition should produce a standard AI posture package: active rights-impacting systems; known risks and residuals; pending waivers; IG and GAO correspondence; litigation posture; budget state; personnel continuity map; external partnerships; and a priority stack. The CAIO Council and GSA AI CoE are natural keepers of cross-agency transition continuity.

COMMON FAILURE MODES
(i) Knowledge lives in email and Slack, not in durable records. (ii) Dossiers exist for pilots but not for legacy systems. (iii) Inventory lists systems but not evidence. (iv) After-action reviews are not done or not published internally. (v) Transitions lose context in the last six weeks of an administration. (vi) IG findings are addressed narrowly, not systemically. (vii) No owner is accountable for long-horizon knowledge health.

CASES
IRS ID.me: institutional memory gap between 2018 identity-proofing research and 2022 rollout; remediation included publishing fairness reports and strengthening CAIO processes.
Michigan MIDAS: internal concerns were not escalated; the state later adopted statutory limits on automated adjudication.
Houston HISD EVAAS: records could not reconstruct teacher-level scoring; court questioned due-process adequacy.
Toeslagenaffaire: siloed ministries could not share warnings; the commission report called for structural changes to Dutch government knowledge systems.
K.W. v. Armstrong (Idaho Medicaid algorithm): state could not explain the model; court ordered disclosure and reform.

Start Your CLUB Certification

This lecture is part of L5 AI Visionary, 160 hours aligned to NIST AI RMF, OMB M-24-10, EO 14110, ISO/IEC 42001, and GAO Green Book. Explore CLUB Certification at skill.re/govt.

Related Lectures

L5
5.5.1 -- Publishing on Government AI
180 min - Workshop

L5
5.5.2 -- Speaking and Presenting on Government AI
180 min - Workshop + Practice

L5
5.5.3 -- Mentoring Next-Generation Leaders
120 min - Seminar + Program Design