The Fed examiner returned to Acme Bank in May and asked the second question every MRM team in 2026 is bracing for: "Show us the independent validation report for the Claude 4 adverse-action letter fine-tune and the loan-default outreach agent." The 2L lead handed over a 47-page document. The examiner read for forty minutes and then identified three structural gaps that would later be written into the supervisory letter. Gap one: the LLM-as-judge used in the eval suite was the same model family as the model under validation, Claude judging Claude, no independence at the judge layer. Gap two: the eval suite included 800 of the 12,000 letters used in the fine-tune training set, eval contamination invalidating the held-out performance numbers. Gap three: there was no adversarial robustness section: no Promptfoo runs, no Garak coverage, no PyRIT probes, no documented red-team. The validator had measured what was easy to measure and skipped what mattered most. The 2L escalated to the Chief Risk Officer that afternoon. Lesson 068 is the discipline that prevents that conversation: Independent Model Validation (IMV) for generative AI in 2026, the three-lines-of-defense application that makes effective challenge real, the five validation components that survive non-determinism, the validator-independence test, the 15-section validation-report template, the cadence triggers, and the cross-walk that lets one validation packet satisfy SR 11-7, PRA SS1/23, EU AI Act Article 9 / 14 / 17 / 26(5), NIST AI RMF Govern + Manage, and ISO 42001 A.6.2.6 + A.6.2.7 + A.3 simultaneously.

Three Lines of Defense Applied to Generative AI - The Independence Chassis

SR 11-7 (April 2011) defined "effective challenge" as validation by parties independent of the model development process, with appropriate competence, authority, and an incentive structure that lets them reach negative conclusions when warranted. The Three Lines of Defense (3LoD) framework, endorsed by the Basel Committee, the Institute of Internal Auditors, and the PRA in SS1/23, is the organisational chassis that makes that independence operational. For generative AI in 2026, each line carries an adapted role and a distinct deliverable.

First line: model owner, data science, and prompt engineering. The 1L builds the system and operates first-line monitoring. For a 2026 LLM or agentic deployment, the 1L extends well beyond a single ML engineer: it includes the prompt engineer (who versions and tests the system prompt), the RAG corpus owner (who manages content and refresh), the eval-suite owner (who curates benchmark and slice data), the production engineer (who runs the inference stack), and the model owner (who is accountable to the business for performance and risk). The 1L's deliverables are the development package: model card, system card, system-prompt version history, eval results, deployment runbook, ongoing-monitoring dashboard. The 1L performs first-line testing before handoff to 2L, but first-line testing by the team that built the system is not validation. It is self-assessment. The distinction matters: a self-assessment is necessary; it is not sufficient.

Second line, Model Risk Management / Independent Validators. The 2L is the validation function. Its deliverable is the Independent Model Validation (IMV) report. The 2L cannot have built the model, cannot report to the 1L, must have the technical depth to evaluate evidence and re-run experiments, and must be able to reach the conclusion that the model should not deploy without that conclusion being overruled by 1L pressure. The 2L's competence requirement in 2026 is non-trivial: validators need evaluation methodology under non-determinism, adversarial-testing literacy (Promptfoo, Garak, PyRIT, Inspect), statistical training for sample-size and confidence-interval calculations on stochastic outputs, prompt-engineering literacy sufficient to read and critique system prompts, and the ability to commission independent eval data when supplied eval data is suspect. The 2L's authority requirement is structural: the 2L head must report to the Chief Risk Officer (or a CRO-equivalent), not to the Chief Data Officer or the Chief AI Officer who own the 1L. The reporting line is the independence test.

Third line, Internal Audit. The 3L provides periodic independent assurance over the operation of the framework. Internal audit reports to the audit committee of the board. The 3L does not validate models; the 3L audits whether the 2L is performing validation effectively, whether the 1L is producing the artefacts the 2L can validate, and whether the governance process is operating. For 2026 AI portfolios, internal audit needs an AI competence baseline, typically one or more auditors with technical AI background who can read a validation report critically and identify the gaps an audit committee would expect them to find. A 3L finding that the 2L's adversarial-test coverage is inadequate is the kind of finding that the audit committee, the CRO, and the regulator all want flagged early.

The Acme Bank application. 1L for the Claude 4 adverse-action workflow comprises the credit-operations team's prompt engineer, the compliance-policy team's letter-template curator (who effectively owns the eval data), and the AI Engineering production lead. 2L is the MRM Validation team, headed by a Director who reports to the CRO; the validator assigned to this model has six years of model-validation experience, completed an internal LLM-evaluation curriculum in Q4 2025, and has no reporting relationship into AI Engineering. 3L is Internal Audit's Technology and Model Risk practice, with one senior auditor maintaining the AI competence baseline. The validation report's sign-off signature comes from the 2L Director, with the 1L owner's name listed as the owner-of-record but explicitly not as a signatory. That signature placement is what makes the document an IMV report and not a 1L self-assessment.

What Counts as "The Model" for Validation - Eight Components, Not One

The single most consequential 2026 IMV decision is scope. A validation report that covers "the LLM" without covering the system prompt, the RAG corpus, the retrieval pipeline, the tools, the memory store, the judge models, and the eval suite is a partial validation, and a partial validation will be returned to the validator by any competent examiner. The 2026 PRA Dear CEO letter and the 2024-2025 OCC interagency guidance both explicitly framed scope as a validation question.

Component 1 - Base model. The foundation model (Claude 4, GPT-5, Gemini 2.x) or fine-tuned variant. Validation evidence includes vendor model card, vendor system card, vendor eval results, third-party audit reports where available, and the vendor's published incident history. For fine-tuned variants, validation extends to the fine-tune training data quality, the fine-tune objective, the fine-tune evaluation, and the comparison against the base model on the bank's use case.

Component 2 - System prompt. The deterministic component of the deployment. The system prompt is hash-pinned to the inventory row; the validator reviews the prompt's instructions, role definition, constraint enumeration, refusal triggers, output format specification, and known-injection defences. A change to the system prompt triggers re-validation of the affected scope (often partial, a constraint addition may need only adversarial re-testing rather than full benchmark re-run).

Component 3 - RAG corpus. Every document the model can retrieve. Validator assesses corpus provenance, content accuracy (sampled), coverage relative to the use case, freshness, refresh cadence, and drift-monitoring. Acme's adverse-action workflow has no RAG; the loan-default outreach agent has a hardship-program catalogue corpus that is part of the validation scope.

Component 4 - Retrieval pipeline. Embedding model (vendor, version, dimensionality), chunking strategy, retrieval-k, reranker presence and configuration, hybrid-search (BM25 + dense) settings. The retrieval pipeline determines what reaches the model; a flawed retrieval pipeline can make a perfectly good base model produce wrong answers.

Component 5 - Tools and functions. The agentic surface. Validator enumerates the tool allowlist, per-tool parameter ranges, per-tool authentication scope, per-tool rate limits, per-tool human-in-the-loop checkpoints. Validator tests for tool misuse, tool hallucination, tool-call injection, and tool-chain failure modes (OWASP Agentic Top 10 ASI02 + ASI04).

Component 6 - Memory store. Persistent state across sessions or users. Validator reviews memory schema, write/read permissions, retention, reset cadence, poisoning defences, GDPR Article 15/17 disclosure-and-deletion plumbing, and cross-user leakage controls.

Component 7 - Judge models. Every LLM-as-judge used in evaluation or guardrailing is itself a model. The validator must (a) verify the judge is not the same model family as the model under test (to preserve independence at the judge layer), (b) measure the judge's own calibration against human labels on a held-out set, and (c) include judge-model performance in the validation report. The Acme Gap 1 example, Claude judging Claude, is the single most common 2026 IMV finding.

Component 8 - Eval suite. The benchmark and test data used to measure performance. Validator must verify the eval suite was not contaminated by training data (the Acme Gap 2 example), covers the production input distribution, includes stratified slices for fairness analysis, includes adversarial probes, includes worst-case examples, and uses defensible thresholds. An eval suite that the 1L curated alone is suspect; the 2L should commission held-out eval data or independently sample production traffic for spot-validation.

Five Validation Components - Conceptual Soundness through Limitations and Remediation

SR 11-7 organises validation around three dimensions (conceptual soundness, ongoing monitoring, outcomes analysis). The 2026 expansion for generative AI splits this into five components that map onto a single 15-section validation report.

Component 1 - Conceptual soundness. Is the chosen approach appropriate for the problem? Could a simpler, more deterministic approach achieve the same outcome with lower risk? For the Acme adverse-action workflow, the validator evaluates whether fine-tuning Claude 4 on 12,000 letters is a more defensible choice than (a) template-based generation with deterministic reason-code-to-paragraph mapping, (b) a smaller fine-tuned model with cheaper inference, or (c) the base model with a strong system prompt and no fine-tuning. The conceptual-soundness section documents alternatives considered and the reasoning for the chosen design. A "we picked Claude because we like Anthropic" answer fails this section.

Component 2 - Process verification. Was development sound? Were the right inputs used, with documented provenance? Was the training methodology defensible? Was the eval data held out properly? Process verification reviews the development artefacts: training data lineage, fine-tune logs, hyperparameter selection rationale, eval set construction, and the 1L's first-line testing. For LLM systems, process verification specifically covers system-prompt change history (does the prompt have version control? are changes peer-reviewed?), RAG corpus governance (who can add documents? what is the review process?), and eval suite contamination checks (did training data leak into eval?).

Component 3 - Outcomes analysis. Does the system perform as intended? This is the bulk of the validation work in 2026. Outcomes analysis runs the system against benchmark eval suites, adversarial probes, fairness slices, hallucination measurement, refusal-appropriateness tests, robustness checks, cost/latency profiles under SLA, and reproducibility tests. The specific 2026 methods are documented in the next section. Outcomes analysis is where the validation either confirms 1L's claims or finds the gaps.

Component 4 - Ongoing monitoring design. How will deployment-time performance be measured? Ongoing monitoring is part of the IMV scope because a model that performs well at validation but cannot be monitored in production is not validated for production. The validator reviews the monitoring KPI set, the threshold definitions, the alerting design, the dashboard implementation, the drift-detection mechanism, the incident-response runbook, and the cadence of human review. The 1L proposes; the 2L challenges. Acme's adverse-action workflow uses compliance-officer override rate as a primary monitoring KPI; the validator confirms that 5% sample review is statistically powered to detect a 2pp degradation within four weeks.

Component 5 - Limitations and remediation. What residual risk remains? What is the validator's recommendation for residual-risk acceptance? Every validation report identifies limitations; the 2L's role is to make those limitations explicit and to recommend either acceptance (with documented rationale), additional controls (mitigants), or non-approval. The Acme adverse-action workflow report identifies four limitations: (a) fairness measurement is constrained by the historical training data which itself reflects historical lending decisions; (b) judge-model calibration drift over time has not been characterised; (c) the production input distribution may shift if the upstream credit model is retrained; (d) the eval set covers English-language letters only. Each limitation has a recommended mitigant or acceptance with the CRO's signature on the residual-risk memo.

2026 Generative-AI Validation Methods - What Outcomes Analysis Actually Tests

The five validation components are the structure; the methods below are the substance of outcomes analysis for 2026 LLM and agentic systems. Each method has a tool, a metric, and a threshold.

Benchmark eval on held-out test set with stratified slices. The 2L commissions or curates a held-out test set that was not used in development. For the Acme adverse-action workflow, the 2L sampled 1,500 letters from the production-eligible population that were explicitly excluded from the fine-tune training set, stratified by denial-reason category, applicant demographic (where lawfully held), state of residence, and letter complexity. Performance is measured per slice, not as an aggregate, to surface fairness and coverage gaps.

Adversarial robustness eval. Promptfoo provides scenario-driven testing across providers and policies; NVIDIA Garak provides automated probe-based scanning for hallucination, leakage, jailbreak, and toxicity; Microsoft PyRIT provides an extensible Python framework for red-team automation; UK AISI Inspect provides safety-evaluation infrastructure. A 2026 Tier 1 validation that does not run all four (or document why a tool was omitted) is incomplete. The Acme Gap 3 example, no adversarial test, was the structural gap that made the original report indefensible.

Hallucination-rate measurement. RAGAS Faithfulness scores measure whether outputs are grounded in retrieved context (RAG systems); TruLens provides programmatic evaluation hooks; LLM-as-judge frameworks with calibrated thresholds measure factual accuracy where ground truth is available. The judge must be a different model family from the model under test, and judge calibration against human-labelled samples must be reported. The Acme report's faithfulness target is >0.95 on the held-out adverse-action letters with judge calibration documented at 0.91 agreement with the compliance officer reviewers.

Fairness and bias slice analysis. Intersectional subgroups (race x gender x age band, where lawfully held) measured against parity thresholds. For adverse-action letters, fairness is measured on the letter's tone, clarity, completeness of reason explanation, and refusal rate by protected class. A 3pp parity threshold across protected classes is the Acme operating standard, with monthly monitoring.

Safety eval. Coverage matrix across OWASP LLM Top 10 (2025) for LLM systems, OWASP Agentic Top 10 (December 2025) for agentic systems, and NIST AI 600-1's 12 GenAI risks for the full GenAI risk landscape. Each safety category has applicability, probe set, observed result, mitigation, and residual.

Cost and latency profile under SLA. Production-realistic load testing measures p50/p95/p99 latency and per-call cost. The Acme adverse-action workflow operates at low volume (200 letters/day) with relaxed SLA; the loan-default outreach agent operates at higher volume with stricter latency requirements. Cost and latency are part of the validation packet because economic infeasibility at scale is a production risk.

Reproducibility eval. Where the provider supports it, seed pinning and temperature pinning are used to measure variance across runs. Even without exact reproducibility, the validator measures distributional stability, running the same input N times and characterising output variance against acceptable bounds.

Edge-case and worst-case probing. Deliberately constructed inputs that probe the boundary of acceptable behaviour: extreme denial reasons, complex multi-applicant cases, ambiguous regulatory framing. These are the inputs the 1L is least likely to have tested and the 2L is most likely to surface failures.

The Validator Independence Test, Report Structure, and Cadence Triggers

Independence is the SR 11-7 principle that makes validation real. Three operational tests determine whether a 2L validator is independent for purposes of a specific model.

Test 1 - Did the validator build the model? If yes, the validator is conflicted and cannot validate. Independence is per-model: a 2L validator who built model A can validate model B but not model A. This rules out the convenient pattern of rotating 1L engineers into the 2L for validation duties on systems they helped build.

Test 2 - Does the validator report to the 1L? If yes, the validator is conflicted by reporting line and cannot validate. The 2L head must report to the CRO or equivalent, not to the executive who owns the 1L. The Acme structure (2L Director reports to CRO; 1L Director reports to CAIO) passes this test.

Test 3 - Does the validator have authority to deliver a negative conclusion? If the validator's bonus, performance review, or career advancement is tied to the model being approved, the validator is functionally compromised even if structurally independent. The validator must be able to recommend non-approval without career penalty. This is the cultural test, and it is the hardest to verify, but the audit committee can verify it through targeted interviews with 2L staff.

The 15-section IMV report structure. A defensible 2026 IMV report has 15 sections: (1) Executive summary: conclusion, key findings, residual-risk recommendation. (2) Scope: what was validated (eight components), what was excluded, why. (3) Data and evidence reviewed, list of artefacts examined, with versions and hashes. (4) Conceptual soundness assessment, approach evaluation, alternatives considered. (5) Process verification findings, development discipline review. (6) Outcomes analysis findings, overall performance summary. (7) Benchmark results, held-out test set performance with stratified slices. (8) Adversarial test results: Promptfoo, Garak, PyRIT, Inspect coverage and findings. (9) Fairness slice results, intersectional parity analysis. (10) Hallucination rate: faithfulness, groundedness, judge calibration. (11) Ongoing monitoring plan: KPIs, thresholds, dashboards, cadence. (12) Limitations identified, explicit residual-risk list. (13) Residual risk acceptance recommendation: accept, mitigate, or reject; rationale. (14) Conditions of use: operating constraints, scope restrictions, human-review requirements. (15) Sign-off, 2L Director signature; date; review cadence; trigger conditions for re-validation.

Cadence triggers. Validation is not a one-time event. The 2026 IMV cadence has three drivers. Pre-deployment IMV is required for all Tier 1 models and recommended for Tier 2, no Tier 1 deployment ships without 2L sign-off. Annual IMV refresh is the baseline cadence for Tier 1 and Tier 2 in production; the refresh re-runs outcomes analysis with current data, reviews the year's incidents and changes, and re-issues the residual-risk acceptance. Trigger-based IMV fires on any of: substantial modification (system prompt change beyond a defined threshold, RAG corpus replacement, tool addition, base-model upgrade), material incident (any Article 73 reportable event or equivalent), upstream model upgrade (vendor changes Claude version, OpenAI changes GPT version), or examination finding. The trigger-based cycle is what catches drift and modification risk between annual refreshes.

Common 2026 IMV findings, the gap list every examiner is checking. The 2024-2025 supervisory letters and the Fed's 2026 horizontal review surfaced six gaps that appear repeatedly: (1) judge-model is the same model family as the model under test, no independence at the judge layer; (2) no adversarial test was conducted (or the adversarial test was a single Garak run without scope documentation); (3) eval suite leaked into training data, held-out performance numbers are not actually held-out; (4) no slice analysis, performance reported as aggregate only, masking fairness gaps; (5) RAG corpus drift unmonitored, corpus changes between validation and operation without re-validation; (6) system prompt versioning absent, current prompt does not match the prompt that was validated. The Acme Bank gaps that opened the lesson are three of these six; the other three appear in roughly half of 2026 horizontal-review samples.

Acme Bank Worked IMV and Cross-Walks to Article 9 / 14 / 17 / 26(5), NIST RMF, ISO 42001

The remediated Acme IMV report for the Claude 4 adverse-action workflow walks all five components and produces a defensible packet. The remediation took six weeks and three additional 2L validators; the final report's executive summary recommends approval conditional on three remediations being in place within 60 days.

Conceptual soundness. Fine-tuning Claude 4 is appropriate for the adverse-action use case because (a) template-based generation cannot achieve the plain-language readability that ECOA/Regulation B examiner expectations have evolved toward, (b) a non-fine-tuned base model produced unacceptable variance in compliance officer override rate during pilot (14% versus 1.8% for the fine-tune), and (c) a smaller fine-tuned model showed comparable quality but failed adversarial robustness probes. Alternatives are documented; the choice is reasoned.

Process verification. Training data lineage is documented (12,000 compliance-approved letters from 2023-2025 with two-of-three compliance reviewer concurrence). Fine-tune methodology used standard supervised fine-tuning with documented hyperparameters. Eval set construction was re-done after the Gap 2 finding; 1,500 letters held out from training. System-prompt version control is in git with peer review required for any change. RAG is not in scope (no RAG component). The 1L's first-line testing is documented and the 2L was able to replicate the headline results.

Outcomes analysis. Benchmark performance on the held-out 1,500 letters: compliance accuracy 98.2% (target >97%); faithfulness 0.96 (target >0.95); plain-language readability score 0.91 (target >0.85); refusal-rate 0.3% (target <0.5%). Stratified by denial-reason category, applicant demographic (where lawfully held), and state: parity within 2.4pp across protected classes (target <3pp); state-level variance within tolerance. Adversarial: Promptfoo policy suite passed with 2 advisories; Garak probes passed with one finding (a prompt-injection vector through embedded denial-reason free-text was patched with input sanitisation); PyRIT red-team identified one failure mode that was remediated; Inspect safety eval clean. Judge model was changed from Claude (same family) to GPT-5 for letter-quality judging and to a deterministic regex-based check for compliance-element completeness, with judge calibration of 0.93 against human reviewer labels documented. Hallucination measurement: faithfulness on retrieval-augmented adverse-action workflows is not applicable (no RAG); factual accuracy against ground-truth denial-reason codes is 99.6%. Cost/latency: p99 latency 4.2 seconds at 200 letters/day; cost per letter $0.04; well within SLA.

Ongoing monitoring. KPIs: compliance officer override rate (target <2%, monitored daily, alert at >3% rolling 7-day); fairness parity by protected class (monthly review by compliance-and-fairness committee); faithfulness score on sampled outputs (weekly); refusal-rate (daily); input-distribution drift versus baseline (weekly); vendor-version polling (weekly with re-validation trigger on any Claude version bump). Dashboard implementation in Acme's MRM platform; alerting routes to the 1L on-call with daily summary to the 2L.

Limitations and residual risk. Four limitations documented: (a) fairness signal limited by the historical letter training set's inherent reflection of historical lending decisions, mitigant: ongoing monitoring by protected class and quarterly review by the fairness committee, with retraining trigger if parity drifts beyond 4pp; (b) judge-model calibration drift not characterised over time, mitigant: quarterly re-calibration of judge against fresh human labels, documented in the model file; (c) production input distribution may shift on upstream credit-model retraining, mitigant: input-distribution monitoring with re-validation trigger; (d) eval set covers English only, mitigant: Spanish-language eval set commissioned for Q3 2026 with deployment to Spanish-speaking customers gated on that eval. Sign-off: approval recommended conditional on (i) RAG corpus integrity monitoring being in place within 60 days (forward-looking for the loan-default outreach agent's hardship-catalogue), (ii) Spanish-language eval set commissioning, (iii) judge-model calibration cadence documented in the model file. CRO signs the residual-risk acceptance memo; the model deploys with the conditions of use written into the model file.

Cross-walks. EU AI Act Article 9(2)(b) ongoing risk management is satisfied by the IMV's ongoing-monitoring section plus the trigger-based re-validation cadence. Article 14 human oversight expectations are embedded in the IMV's conditions of use (sample review rate, compliance officer override authority); a finding in the IMV report that the oversight design is insufficient triggers a 1L remediation cycle before deployment. Article 17 quality management system is operationalised through the 2L's existence as an ISO-mappable QMS process; the validator's report is an Article 17 record. Article 26(5) post-market monitoring obligation for deployers is evidenced by the ongoing-monitoring section of the IMV report and the operational metrics fed back into annual refresh. NIST AI RMF Govern 4.1 (organisational accountability) is the CRO sign-off; Govern 5.1 (external stakeholder engagement) is the cross-walk discipline; Manage 1.1 (risk management) is the validation report itself; Manage 1.3 (responses to risks) is the conditions of use; Manage 4.1 (post-deployment monitoring) is the ongoing-monitoring plan. NIST AI 600-1 applicability matrix marks the GenAI risks that the validation explicitly tested (confabulation/hallucination, harmful bias, information integrity) and the risks excluded with rationale (CBRN, dangerous capabilities not applicable to letter drafting). ISO 42001 A.6.2.6 testing is the outcomes analysis section; A.6.2.7 post-deployment monitoring is the ongoing-monitoring section; A.3 leadership accountability for independence is the reporting-line structure (2L to CRO).

Penalty exposure. Under SR 11-7 / OCC 2011-12, inadequate IMV is the warning that consent-order territory is one examination cycle away, Matters Requiring Attention (MRA) escalate to Matters Requiring Immediate Attention (MRIA), and persistent failures move to consent order with public disclosure and remediation obligations. Under PRA SS1/23, the equivalent escalation is the section 166 skilled-person review (firm pays; public record). Under EU AI Act Article 99(3), Article 9 (risk management) failures and Article 17 (QMS) failures carry administrative fines up to €15 million or 3% of worldwide annual turnover, whichever is higher. The validation packet that satisfies SR 11-7 effective challenge satisfies Article 9 + Article 17 + Article 26(5) simultaneously, and the cross-walk is the efficiency that lets a 2L team of 20 cover a 200-model bank portfolio rather than maintaining four parallel validation regimes.

Key Takeaways

• Independent Model Validation under SR 11-7 requires a 2L validation function that is structurally independent, technically competent, and culturally empowered to deliver a negative conclusion. The three-test independence check, did the validator build the model? does the validator report to the 1L? does the validator have authority to recommend non-approval without career penalty? operationalises the SR 11-7 effective-challenge principle.
• Three Lines of Defense applied to GenAI: 1L is the model owner, prompt engineer, RAG corpus owner, eval-suite owner, production engineer (build + first-line monitoring); 2L is Model Risk Management / Independent Validators reporting to the CRO (effective challenge, validation report, sign-off); 3L is Internal Audit reporting to the audit committee (process effectiveness audit). 3L needs AI competence on staff for 2026 portfolios.
• "The model" for validation purposes is eight components, not one: base model, system prompt, RAG corpus, retrieval pipeline, tools/functions, memory store, judge models, eval suite. A validation that does not cover all eight is partial and will be returned by a competent examiner.
• Five validation components: (1) conceptual soundness (is the approach appropriate? what alternatives were considered?); (2) process verification (was development sound and documented?); (3) outcomes analysis (does it perform: benchmarks, adversarial, fairness, hallucination, cost/latency, reproducibility, edge cases?); (4) ongoing monitoring design (KPIs, thresholds, cadence, drift detection); (5) limitations and remediation (residual risk acceptance with CRO sign-off).
• 2026 validation methods for outcomes analysis use Promptfoo (scenario testing), NVIDIA Garak (probe-based scanning), Microsoft PyRIT (red-team automation), UK AISI Inspect (safety evaluation), RAGAS Faithfulness, TruLens, judge models from a different family with documented calibration, stratified intersectional fairness analysis, OWASP LLM Top 10 + Agentic Top 10 + NIST AI 600-1 12-risk coverage, and held-out eval data the 1L did not curate alone.
• The six common 2026 IMV findings are: judge-model same family as the model under test, no adversarial test, eval suite leaked into training, no slice analysis, RAG corpus drift unmonitored, system prompt versioning absent. The Acme Bank examiner gaps were three of these six.
• The 15-section IMV report template covers executive summary, scope, data and evidence, conceptual soundness, process verification, outcomes analysis, benchmark results, adversarial test results, fairness slice results, hallucination rate, ongoing monitoring plan, limitations, residual-risk acceptance recommendation, conditions of use, and sign-off. Cadence: pre-deployment IMV required for Tier 1; annual refresh baseline; trigger-based IMV on substantial modification, incident, upstream upgrade, or examination finding.
• One validation packet satisfies four frameworks. SR 11-7 effective challenge maps to EU AI Act Article 9(2)(b) ongoing risk management, Article 14 oversight, Article 17 QMS, and Article 26(5) post-market monitoring; to NIST AI RMF Govern 4.1/5.1 + Manage 1.1/1.3/4.1; to ISO 42001 A.6.2.6 testing + A.6.2.7 post-deployment monitoring + A.3 leadership. Penalty exposure: SR 11-7 escalation MRA → MRIA → consent order; EU AI Act Article 99(3) up to €15M or 3% of worldwide turnover for Article 9 + 17 failures. The cross-walk is the efficiency that makes a 2L team scalable.