A senior auditor from Schellman walks into the AI Officer's conference room on a Tuesday morning in May 2026 and asks one question: "Show me how you know your Article 4 program is producing literacy, not just attendance." The slide deck the team prepared the night before, coverage rates by tier, completion percentages, hours-of-training-delivered, answers a different question. The auditor sits patiently while the AI Officer scrolls. Forty minutes later, the assessment-evidence finding is drafted. The completion records were comprehensive. The assessment evidence was not. This lesson is the L2 playbook for designing the assessment methodology, completion-tracking infrastructure, refresh cadence, and evidence-retention architecture that prove Article 4 compliance to a Schellman, A-LIGN, BSI, KPMG, DNV, or LRQA auditor, and to a national market surveillance authority that arrives without prior notice. Lesson 033 covered curriculum design. This lesson covers the second-half operational reality: assessment by tier, completion infrastructure, refresh-cadence design, retention per Article 18, the audit walk-through, and the six common mistakes that turn assessment into an audit finding.

Why Completion Tracking Alone Is Not Audit Evidence

The most common finding in first-generation Article 4 audit reviews is a variation on the same sentence: "The organization tracks training completion but does not produce evidence of understanding." The finding is almost always factually correct. The program assigns the e-learning module, the LMS records completion, the dashboard reports "94% coverage," and the management-review packet shows the trend line moving up. None of that answers the question the auditor is asking. The auditor is testing whether the program produces, in Article 4's words, a "sufficient level of AI literacy", and the only operational test for sufficiency is assessment-evidence demonstrating the learner can apply the content to a representative judgment.

The distinction matters because Article 4's standard is functional. The text requires literacy "to ensure, to their best extent, a sufficient level", and sufficient is calibrated to the role. Tier 1: ability to govern the program and approve risk appetite. Tier 2: operational competence to run intake-to-deployment without prohibited-use exposure. Tier 3: system-specific competence as human-in-the-loop with escalation. Tier 4: access to the five-element notice. None of those standards can be evidenced by attendance alone.

The Commission's living repository of AI literacy practices reinforces the distinction. Multiple late-2025 entries emphasize "evidence of comprehension" or "competence demonstration" as the operational evidence of literacy. ISO/IEC 42001:2023 Annex A.4, the cross-walked control for Article 4 evidence, refers to "competence" rather than "training delivery," and clause 7.2 of the ISO 42001 main body requires the organization to "evaluate the effectiveness" of competence-building. NIST AI RMF Govern 1.4 similarly references "personnel and partners" with "the requisite knowledge and skills" rather than receipt of training events.

The operational implication: every defensible Article 4 program in 2026 carries an assessment layer behind the completion layer. Assessment varies by tier, scenario-rich at Tier 2, knowledge-check at Tier 3, sample-based at Tier 1, optional comprehension-check at Tier 4, but the principle is constant. Completion produces coverage evidence. Assessment produces competence evidence. The audit-defensible program produces both.

Assessment Methodology - Tier by Tier

Tier 1 - Executive Sample-Based Scenario Assessment

Population. Typically 10-50 people across CEO, Executive Committee, Board, Audit Committee, Risk Committee, and (in financial services) the SMF holders covering AI accountability. Small enough that psychometric assessment is meaningful only at cohort level, the defensible approach is sample-based scenario assessment, not a graded examination.

Methodology. Six to ten short scenario-based questions delivered as part of the 90-minute briefing, typically as the interactive segment immediately following the content delivery. Each scenario is calibrated to a Tier 1 standing topic. Representative examples:

• Article 5 prohibition scenario. "Marketing proposes an emotion-recognition system in the call center to monitor agent stress. Walk through evaluation against Article 5(1)(f), what the narrow medical/safety carve-out permits, and the escalation pathway if Compliance flags concern."
• Article 27 FRIA trigger scenario. "The credit-cards business wants to deploy a new model for adverse-action explanations. Walk through whether a FRIA is required under Article 27, who needs to be involved, the timeline, and the board's role in reviewing the FRIA output before go-live."
• Article 73 reporting-clock scenario. "A GDPR Article 22 complaint surfaces a systematic Article 5(1)(a) concern in fraud-detection. The Article 73 clock for fundamental-rights infringement is 2 business days. Walk through the board's role, the escalation pathway, the regulator-coordination playbook, and the disclosure decision."
• Article 99 penalty-quantification scenario. "An Article 5 prohibition finding for our €12B-turnover firm carries a €840M worst-case Article 99(2) exposure. Walk through how the board reviews mitigation status, the realistic-case after mitigation, and the personal-accountability story for the Article 47 declaration signer."

Sample size. 100% participation. Scenarios are 30-90 second responses delivered orally or via in-session polling (Mentimeter, Slido, LMS in-session response). The pass threshold is not punitive, the goal is not to flunk a board director, but response quality is tracked, and weakness patterns drive curriculum updates for the next refresh.

Evidence retention. Cohort-level response aggregation retained per the firm's board-materials retention schedule. Anonymized weakness patterns retained for management-review. Individual responses anonymized after aggregation; named-attendance and named-participation records retained.

Tier 2 - Deployer Scenario-Rich Assessment with 80% Threshold

Population. Typically 50 to 500 across the AI Officer's direct team, business-function heads deploying AI, Procurement Directors signing AI contracts, senior legal staff, Internal Audit's AI specialists, the InfoSec AI-risk leads, and the DPO office. Large enough for psychometric meaningful assessment at the individual level.

Methodology. The Tier 2 assessment runs after each of the four 45-60 minute modules plus a capstone scenario at the end. Each module assessment combines 8-12 multiple-choice items (knowledge recall) with 2-3 short-scenario items (applied judgment). The capstone scenario integrates topics from all four modules into a single end-to-end case: typically a new high-risk-system intake walked through tiering memo, FRIA, procurement-contract review, deployment approval, and the first 30 days of post-market monitoring. 80% pass threshold applies to aggregate score across modules plus capstone. A first-attempt failure triggers a remediation pathway: targeted review of the weak module, a 30-day re-take window, and an escalation to the people-manager if the second attempt fails.

Item-design principles. Multiple-choice items are designed to require operational application, not pure recall. A weak item, "What is the Article 73 reporting clock for ordinary serious incidents?", tests recall only. A defensible item, "A vendor-deployed translation AI in the customer-service workflow has produced what looks like systematic accuracy degradation affecting a specific protected-class language. The system was flagged at 11:30 AM Tuesday. The first formal escalation occurred at 4:00 PM the same day. Walk through the harm-severity decision tree, identify the applicable Article 73 clock, and identify the regulator-coordination next step.", tests application.

Scenario items require judgment under partial information. A defensible scenario provides 3-5 paragraphs of operational context, asks the learner to identify the relevant obligations, propose a sequence of actions, and identify the escalation pathway. Scoring uses a rubric with 3-5 dimensions per scenario. Inter-rater reliability between Tier 2 assessment reviewers is monitored, disagreement above 15% triggers rubric refinement.

Sample assessment items (representative).

• "The HR business head proposes an off-the-shelf AI screening tool. The vendor references a conformity declaration but does not include the Annex IV technical file. Walk through the five-clause procurement framework gaps and the specific clauses you would require before signing."
• "A business head wants to fine-tune a GPAI model on customer-conversation transcripts. Walk through the substantial-modification analysis (Article 25(1)(b)), the provider-status implications, the Annex XII downstream-deployer-information considerations, and the GDPR overlay for the training-data use."
• "A customer-service AI has surfaced systematic bias against a protected class in post-market monitoring this week. The system is high-risk under Annex III §5(a). Walk through the Article 72 post-market monitoring response, the Article 73 reporting analysis, the FRIA refresh requirement, and the corrective-action workflow."

Pass threshold and remediation. 80% on the aggregate score (modules plus capstone). First-attempt failure triggers a 30-day remediation pathway: targeted review of the weak content, optional 1-on-1 review with the AI Officer's training lead, and a re-take. Second-attempt failure escalates to the people-manager and triggers a discussion about role suitability. The remediation pathway is documented for audit evidence.

Evidence retention. Individual-level assessment results retained per the firm's ISO 42001 record-retention schedule (typically 10 years aligned to Article 18). Cohort-level pass-rate trends retained and reported to management review. Item-level statistics retained for curriculum-quality analysis (item-difficulty, item-discrimination, distractor-analysis). Capstone-scenario response samples retained for quality review and used as exemplars for subsequent cohorts.

Tier 3 - User Knowledge-Check Assessment with 80% Threshold

Population. Typically thousands to tens of thousands across front-line employees who deploy or operate AI systems. Too large for live scenario-rich assessment but large enough for psychometrically meaningful individual-level knowledge-check assessment delivered through the LMS.

Methodology. Each of three 20-30 minute modules ends with a knowledge-check of 10-15 items, predominantly multiple-choice with 1-2 short-scenario items per module. Total assessment length is 30-45 items across the curriculum. 80% pass threshold applies to the aggregate. Delivered immediately after each module's content; the learner cannot complete the module without the assessment. First-attempt failure triggers an automatic re-take after a 24-hour cooling-off period; second-attempt failure escalates to the people-manager.

Item-design principles. Items written at the operational-application level appropriate to the front-line role. A defensible recruiter item: "A candidate asks how the screening AI evaluated their application. Walk through which Article 26(9) elements you must provide, which elements you would refer to the FAQ, and which elements require HR Legal coordination." Distractors surface common misconceptions ("Refer all questions to Legal," "Decline to discuss because the system is proprietary"). Scenario items are role-specific, a chatbot-operator scenario differs from a credit-analyst scenario. Scenarios are written and reviewed by the relevant business function in collaboration with the AI Officer's content team, the operational mechanism by which Article 4's "context" requirement is satisfied at the assessment layer.

Refresh on system change. A material system change (new model version, new use case, new vendor) triggers refresh of the assessment items relevant to that system. Refresh-cadence triggers are tracked through the AI inventory's system-change events. The LMS auto-assigns the refresh module and assessment to the affected user population.

Evidence retention. Individual-level results retained per the firm's record-retention schedule (typically 10 years aligned to Article 18 for high-risk-system-related records, 5 years for non-high-risk). Cohort-level pass-rate trends reported to management review. System-specific assessment-result trends surface system-level curriculum-quality issues (e.g., a system with persistently low pass rates suggests the user-facing documentation or training content needs improvement).

Tier 4 - Read-and-Acknowledge with Optional Comprehension Check

Population. Affected persons: job candidates, monitored employees, credit applicants, insured persons, students, members of the public subject to public-sector AI. External to the firm; the firm controls notice delivery but not recipient engagement.

Methodology. Default is read-and-acknowledge: notice delivered at point of interaction; recipient acknowledges receipt (checkbox in portal, recorded verbal acknowledgment, signed paper); acknowledgment timestamped and retained. For higher-stakes interactions (workplace monitoring, employment-decision AI in regulated industries), an optional comprehension-check: 2-4 items embedded in the acknowledgment workflow confirming understanding of the five notice elements. The check is voluntary, a failure cannot block the underlying transaction, but the comprehension-rate metric is tracked and reported to management review.

Sample comprehension-check items (representative, for a workplace-monitoring notice).

• "The notice you just reviewed describes a system that monitors which of the following? (a) keystroke patterns and application focus time during working hours; (b) your personal email and social media during working hours; (c) your location 24/7; (d) your communications with the works council." The correct answer reinforces the notice's actual scope and surfaces misconceptions.
• "If you believe a decision affected you and you want to ask for an explanation, the right approach is to (a) contact HR with the date and decision; (b) file a complaint with the labor inspectorate; (c) wait for the next performance review; (d) contact the works council." Multiple answers may be valid; the rubric reinforces all of the legitimate pathways.

Evidence retention. Acknowledgment records retained per the firm's privacy-and-employment schedule (typically 2-7 years). Comprehension-check results at cohort level for management review; individual results only where required by the underlying transaction (e.g., credit-decision evidence).

Completion-Tracking Infrastructure - LMS, HRIS, and Reporting

The assessment-and-completion-tracking infrastructure is the operational backbone that makes the audit-evidence package producible on demand. The architecture has four components: the LMS or learning-experience platform, the HRIS integration for assignment, the analytics layer for reporting, and the export-and-archive layer for audit-evidence production.

LMS or LXP selection. Dominant 2026 enterprise platforms: Cornerstone Learning (dominant in financial services and global enterprises), Workday Learning (default for Workday HCM customers), Docebo (popular for customer-facing and partner-facing literacy), SAP SuccessFactors Learning (default for SAP HCM customers), and custom LXPs built on headless infrastructure (several global banks and insurers). Article 4 selection criteria: HRIS-integration depth for role-based assignment; assessment-tooling sophistication for scenario-rich Tier 2 items and the capstone; reporting and analytics for cohort-level pass-rate trends and item-level statistics; SSO integration for access; multi-language support; content-authoring tooling; mobile delivery for high-turnover Tier 3 populations; refresh-cadence automation; export-and-archive capability for audit-evidence production; vendor stability and security posture.

HRIS integration for assignment. The HRIS-LMS integration is the most important architecture decision for an Article 4 program. It enables: role-based auto-assignment of the appropriate tier module on hire or role change; new-hire onboarding integration with a 30-day completion target gated to first AI-system access; role-change refresh trigger for any movement into a Tier 2 deployer or decision-maker role; director-onboarding trigger for any new board member or Executive Committee appointee for Tier 1; vendor-staff-augmentation trigger for Tier 3 where the contract grants AI-system access; system-change refresh trigger driven by the AI inventory's system-change events.

Without HRIS integration, completion-tracking evidence is manual, lagging, and fragile. With it, evidence is real-time, attributable, and audit-ready. The integration is built once during program launch and refreshed quarterly as roles, vendors, and systems evolve.

Analytics layer. Feeds two consumers: the AI Officer team's monthly operational report and the AI Governance Committee's quarterly management review. Monthly operational report: coverage rate by tier (overall, BU, geography, role); assessment pass-rate trend with cohort comparison; first-attempt-failure rate with remediation-pathway throughput; refresh-cadence status (annual, on-event, role-change, on-system-change); Tier 4 notice delivery volumes; complaint-pathway volumes and response times. Quarterly review aggregates the monthly views, adds trend analysis, identifies corrective actions, and reports against prior-quarter commitments.

The annual board or Risk Committee briefing distills the quarterly reports into a 3-5 slide management-review summary. This is the upward-reporting evidence the ISO 42001 Stage 2 auditor tests under clause 9 (performance evaluation) and clause 9.3 (management review). It is also the audit-committee-briefing evidence the external auditor references under their attestation engagement.

Export-and-archive layer. When an auditor or regulator requests evidence, the program produces it in 24-48 hours, not 2-4 weeks. The export-and-archive layer supports this through pre-built audit-evidence packages: the curriculum-design document; the LMS completion record for the requested population and date range; the assessment results for the requested population and date range; the refresh-cadence evidence; the management-review evidence. Packages are exported in formats the auditor accepts (PDF for documents, CSV or Excel for completion-record extracts, structured JSON for system integrations where the auditor uses their own analytics tooling) and retained in the firm's audit-evidence archive per the record-retention schedule.

Refresh-Cadence Architecture - Annual, On-Event, Role-Change, System-Change

The four-cadence refresh architecture is the operational pattern that survived first-generation Article 4 audit reviews. Each cadence answers a different audit question.

Annual refresh. The calendar-driven Q1 baseline keeping the curriculum current with regulatory developments, organizational changes, and portfolio evolution. Includes: content review against prior-year regulatory developments (Commission guidance, Member State implementing laws, ENISA/EBA/EIOPA/ESMA sectoral guidance, ISO/NIST standard updates); tier-classification review against AI-inventory evolution; assessment-item review with rotation of stale items and addition of new scenarios; delivery-format review including any LMS-platform or content-tooling updates; management-review of prior-year coverage and assessment trends. Outputs: refreshed curriculum-design document, module content, and assessment item bank. Version-controlled so the auditor can trace curriculum content at any point in time to a specific version.

On-event refresh. Triggered by material regulatory developments or organizational events that change the literacy content. Dominant 2026 triggers: Commission guidance updates (especially the rolling Article 50 series); new enforcement actions (the first Article 99 penalties expected in 2027 will drive immediate on-event refresh); Member State implementing-law publications; ISO 42001 standard updates; NIST AI RMF profile updates; major organizational events (M&A, divestiture, business-line launch or wind-down); major-incident learnings from internal post-mortem. Typically a 30-90 minute targeted briefing to the affected tier within 30-60 days. Tier 1: delivered at the next regular board meeting or via a dedicated session. Tier 2/3: LMS-delivered with assessment where content materially changes operational competence.

The on-event refresh evidence is the differentiator in audit reviews. A program that refreshed within 30 days of the Omnibus VII political agreement (May 2026) and documented the refresh is materially more defensible than a program that waited for the next annual cycle. The auditor will ask: "Show me your on-event refresh evidence for Omnibus VII." The defensible answer includes the briefing materials, the attendance record, the assessment results where applicable, and the management-review evidence acknowledging the refresh completion.

Role-change refresh. Triggered by HRIS events moving an individual into or within a tier. Dominant triggers: movement into a Tier 2 role (e.g., a new business-function head, a new Procurement Director, a new General Counsel office hire with AI scope); movement into a Tier 3 role with AI-system access (e.g., a new recruiter, a new credit analyst, a new customer-service agent); movement out of a tier with retention of system access (e.g., a Tier 2 deployer moves to an advisory role but retains operational access, the refresh is calibrated to the new responsibilities). Auto-assigned by the HRIS-LMS integration within 7 days of the role-change event; gated to completion within 30 days for Tier 2 and Tier 3.

System-change refresh. Triggered by material changes to AI systems in the inventory. Dominant triggers: new high-risk-system deployment; substantial modification under Article 25(1)(b) (which transfers provider status); new model version where the version materially changes performance, scope, or risk profile; new vendor for a system in use; new use case for a system in use. Auto-assigned to the user population of the affected system within 7 days; 60-day completion window. The window is calibrated to allow deployment runway; for systems with shorter runway (e.g., emergency response to a major-incident remediation), the window can be compressed.

Cadence interaction. The four cadences operate in parallel; an individual learner can be assigned multiple refresh modules in a short period (e.g., an annual refresh in Q1 plus an on-event Omnibus VII refresh in Q2 plus a system-change refresh in Q3). LMS analytics tracks cadence-completion-rate separately so the management-review can identify cadence-specific gaps (e.g., "system-change refresh at 71% vs. the 85% target, investigate vendor onboarding delays").

Evidence Retention - Article 18, ISO 42001 A.7, Sectoral Overlays

The retention architecture for Article 4 evidence integrates EU AI Act Article 18 (record-keeping by providers), ISO/IEC 42001:2023 Annex A.7 (documentation retention), and sectoral retention overlays.

Article 18 baseline. Article 18 requires providers of high-risk AI systems to keep the technical documentation (Annex IV), the documents on the quality management system (Article 17), the documentation concerning changes approved by notified bodies, the conformity assessment certificates, the documents concerning the post-market monitoring, the documents concerning cooperation with national competent authorities, and the EU declaration of conformity, for 10 years after the AI system has been placed on the market or put into service. Article 4 evidence is not explicitly named in Article 18 but is operationally relevant under several Article 18 categories: specifically the quality-management-system documentation (Article 17 includes "documentation, design and verification activities" plus "examination, test and validation procedures" which encompass literacy-program design) and the post-market monitoring (Tier 3 user-operator training and assessment evidence supports the human-oversight integrity that post-market monitoring relies on).

Defensible default for high-risk-system-related Article 4 evidence: 10-year Article 18 retention. For non-high-risk Article 4 evidence (e.g., Tier 3 for users of minimal-risk generative-AI tools): 5-7 years aligned to the firm's broader compliance-training retention standard. Rationale documented in the curriculum-design document and reviewed annually.

ISO/IEC 42001:2023 Annex A.7 documentation retention. Annex A.7 covers AI system development data, traceability, and documentation retention. The control applies to literacy program evidence as part of the AIMS (AI Management System) documentation. The Stage 2 auditor tests the retention practice against the firm's documented retention schedule and against the underlying obligations. A schedule that says "10 years" but cannot produce 10-year-old evidence is a finding; a schedule that says "5 years" with documented rationale aligned to the underlying obligation may be acceptable depending on the obligation analysis.

Sectoral overlays. Financial-services firms align the retention schedule with applicable sectoral law: EBA Guidelines on ICT and security risk management (typically 5 years); MiFID II record-keeping (5-7 years); insurance Solvency II reporting record-keeping (typically 7 years); central-bank supervisory record requirements. Healthcare deployers align with medical-records retention (varies by Member State, typically 10-30 years). Public-sector deployers align with the relevant administrative-law retention requirements. Where multiple overlays apply, the defensible default is the longest required period.

GDPR interaction. Retention of individual-level assessment results triggers GDPR considerations. Lawful basis is typically the legitimate interest of operating the AI Management System plus the legal obligation under Article 4 itself plus (for personnel) the contractual basis of the employment relationship. Retention period justified against the obligation; data-subject rights are honored (subject to the AIMS-documentation exemption for legitimate-interest data); data-minimization is applied (item-level responses can be aggregated and anonymized after the operational-window has passed where the underlying obligation no longer requires individual-level evidence).

Records the auditor will request. The defensible retention architecture produces, on demand: the curriculum-design document at any version in scope; the LMS completion record for the requested population and date range; the assessment results for the requested population and date range; the refresh-cadence evidence (annual, on-event, role-change, system-change); the Tier 4 notice templates and per-interaction delivery records; the management-review evidence (quarterly AI Governance Committee plus annual board/Risk Committee briefings); the corrective-action evidence; the integration evidence with the broader AIMS documentation.

The Schellman / A-LIGN / BSI / KPMG Audit Walk-Through

The external auditor's Article 4 review during an ISO 42001 Stage 2 engagement follows a predictable pattern. Understanding the pattern in advance lets the program prepare evidence packages that match the auditor's request sequence and produce them in the audit-window.

Opening question. Almost always some variation on: "Walk me through how your organization satisfies Article 4 of the EU AI Act." The defensible 5-7 minute response covers: the four-tier curriculum structure; the audience scoping by tier; the assessment methodology by tier; the completion-tracking infrastructure; the refresh-cadence architecture; the management-review loop; the cross-walks to ISO 42001 Annex A.4, Annex A.7, Annex A.8, and clause 7.2. Ends with an offer to produce specific evidence on request.

Curriculum-design document request. The defensible document includes: program purpose and scope; the regulatory and standards basis (Article 4, ISO 42001 Annex A.4/A.7/A.8, NIST AI RMF Govern 1.4, applicable sectoral and U.S. overlays); tier definitions and audience scoping; content outline per tier with learning objectives and assessment methodology; delivery format per tier; refresh-cadence architecture; LMS-integration architecture; audit-evidence-package definition; management-review process; version-control history. Typically 15-40 pages depending on enterprise size.

Completion-tracking sample request. Typical samples: "Show me the completion record for everyone in a Tier 2 role as of last quarter-end"; "Show me the completion record for new hires in Tier 3 roles in the last six months"; "Show me the role-change refresh completion record for the past year"; "Show me the on-event refresh completion record for the Omnibus VII briefing." The LMS export-and-archive layer produces these in CSV or PDF format within the audit-window.

Assessment-results sample request. The auditor requests results for a sample of Tier 2 and Tier 3 learners. The defensible response includes: individual-level pass-rate; the assessment-item set the learner completed; the pass-rate trend for the learner's cohort; the curriculum-quality analysis driven by the cohort's results (item-difficulty, item-discrimination, distractor-analysis). For Tier 1: cohort-level scenario-response evidence plus management-review summary.

Refresh-cadence walk-through. Typical asks: "Show me your on-event refresh evidence for Omnibus VII"; "Show me the role-change refresh evidence for moves into Tier 2 roles in the past year"; "Show me the system-change refresh evidence for the new model version deployed in Q1." Evidence: briefing materials, HRIS-LMS assignment records, completion records, assessment results where applicable, management-review acknowledgment.

Tier 4 notice walk-through. Typical asks: "Show me the Tier 4 notice template for HR screening AI candidates"; "Show me the per-interaction delivery evidence for the past quarter"; "Show me the comprehension-check results where they apply"; "Show me a sample complaint-pathway response." Evidence: notice template, timestamped acknowledgment records, cohort-level comprehension-check results, complaint-record sample with the firm's response.

Management-review walk-through. Typical asks: "Show me the AI Governance Committee quarterly meeting minutes covering the Article 4 program"; "Show me the corrective-action register"; "Show me the annual board or Risk Committee briefing materials." The defensible response shows documented agenda items covering coverage, assessment trend, refresh-cadence status, Tier 4 notice coverage, complaint-pathway volumes, corrective-action status; named attendance; decisions; action items with named owners and target dates; prior-quarter corrective-action closure rate.

Sampling methodology. For populations larger than 100, the auditor typically samples. The defensible sampling methodology is documented in advance: population definition, sampling frame, sample size (typically 25-60 records for thousands-scale populations), selection method (random or stratified), test execution. The program supports the sampling by producing the population frame and sampled individual records within the audit-window.

Six Common Mistakes - The Assessment-and-Evidence Failure Catalog

Mistake 1 - Treating Completion as Competence

The dominant first-generation failure: treating LMS completion as audit evidence. "We trained 8,000 people" answers the wrong question; the auditor asks "what fraction of those 8,000 can apply the content to a representative judgment." Fix: psychometrically meaningful assessment at Tier 2/3, sample-based scenario assessment at Tier 1, optional comprehension-check at Tier 4. Without an assessment layer, completion-tracking is fragile.

Mistake 2 - No Assessment Layer at All

The harder-to-remediate variant: no assessment layer at all, only attendance or completion. Remediation cost is meaningful (item authoring, scenario design, rubric development, inter-rater reliability, LMS configuration). Defensible launch sequence builds assessment in parallel with curriculum, not as a follow-on.

Mistake 3 - One-Time Launch with No Refresh Architecture

A 2024 or early-2025 single-release program with no refresh architecture is a 2026 audit finding regardless of launch quality. The landscape shifted with Omnibus VII (May 2026), the Commission's repository updated quarterly through 2025, sectoral guidance evolved, and the firm's AI portfolio changed. Fix: four-cadence refresh (annual, on-event, role-change, system-change) with on-event responsiveness as the differentiator.

Mistake 4 - No On-Event Refresh for Regulatory Developments

A program that runs the annual refresh on schedule but does not refresh on regulatory developments is a near-miss finding. The 2026 Omnibus VII test: a program that did not refresh Tier 1 and Tier 2 within 60 days of the political agreement will be asked to explain the gap. Fix: on-event trigger architecture: defined trigger events (Commission guidance, Member State implementing laws, enforcement actions, ISO/NIST updates, major-incident learnings) with named owners, target windows, management-review tracking.

Mistake 5 - Weak Retention Practice

The program with a retention schedule that says "10 years" but cannot produce 10-year-old evidence is the worst-case retention finding. The variant, a retention schedule with no documented rationale, is more common. Fix: retention architecture aligned to Article 18 (10 years for high-risk-related records), the firm's sectoral overlays (varies), and the broader ISO 42001 Annex A.7 documentation retention; documented rationale for non-default retention periods; export-and-archive that can produce historical evidence on demand; GDPR analysis covering individual-level retention.

Mistake 6 - No Management-Review Evidence

The program with comprehensive completion and assessment evidence but no management-review evidence is a clause-9 finding under ISO 42001 Stage 2. The auditor cannot verify the program is operating and improving without the management-review loop. Fix: AI Governance Committee quarterly review with documented agenda, attendance, decisions, and corrective actions; annual board or Risk Committee briefing as the upward-reporting evidence; corrective-action register with named owners and target dates; prior-period closure-rate reporting; integration with the broader AIMS performance-evaluation evidence under ISO 42001 clauses 9.1 (monitoring, measurement, analysis, evaluation), 9.2 (internal audit), and 9.3 (management review).

Key Takeaways

• Completion tracking is the coverage evidence. Assessment is the competence evidence. The audit-defensible Article 4 program produces both, the auditor's opening test is whether the program can demonstrate understanding, not just attendance.
• Tier 1, sample-based scenario assessment. 6-10 short scenario questions delivered live during the 90-minute briefing; 100% participation; cohort-level response aggregation; pass-threshold tracked but not punitive; weakness patterns drive curriculum updates.
• Tier 2, scenario-rich assessment with 80% threshold. 8-12 multiple-choice plus 2-3 short-scenario items per module plus a capstone end-to-end scenario; first-attempt-failure triggers 30-day remediation; second-attempt-failure escalates to people-manager; rubric-based scoring with inter-rater-reliability monitoring.
• Tier 3, knowledge-check assessment with 80% threshold. 10-15 items per module across 3 modules; system-specific scenarios written by the relevant business function; first-attempt-failure triggers 24-hour-cooled re-take; system-change events trigger assessment-item refresh.
• Tier 4, read-and-acknowledge with optional comprehension check. Timestamped acknowledgment at point of interaction; 2-4 voluntary comprehension-check items where the underlying transaction supports it; comprehension-rate tracked at cohort level for management-review reporting.
• Completion-tracking infrastructure is LMS + HRIS + analytics + export-and-archive. Cornerstone, Workday Learning, Docebo, SAP SuccessFactors, or custom LXP; HRIS-driven assignment is the most important architecture decision; monthly operational reporting plus quarterly management-review plus annual board briefing.
• Refresh cadence is annual + on-event + role-change + system-change. Annual baseline in Q1; on-event triggered by regulatory developments (Omnibus VII as the 2026 dominant example) within 30-60 days; role-change within 30 days; system-change within 60 days.
• Retention is 10 years for high-risk-related records (Article 18 baseline), 5-7 years for non-high-risk aligned to the firm's broader retention standard; ISO 42001 Annex A.7 documentation retention as the cross-walked control; sectoral overlays for financial-services, healthcare, and public-sector deployers; GDPR data-minimization for individual-level data.
• The Schellman / A-LIGN / BSI / KPMG / DNV / LRQA audit walk-through is predictable: opening walk-through, curriculum-design document, completion-tracking sample, assessment-results sample, refresh-cadence walk-through, Tier 4 notice walk-through, management-review walk-through. Prepare the evidence packages to match.
• The six common mistakes are the assessment-and-evidence failure catalog. Completion-equals-competence assumption; no assessment layer; one-time launch; no on-event refresh; weak retention; no management-review evidence. Design around them from day one.