Respond to the NAIC AI Systems Evaluation Tool Information Request - Exhibits A, B, C, D
The NAIC AI Systems Evaluation Tool (AISET) information request landing on a carrier's desk in 2026 is the most consequential regulatory artifact in insurance AI governance - a four-exhibit, structured request that examines the carrier's AI program in its entirety. Exhibit A (AI quantification) inventories every AI System with category, function, line of business, and risk classification. Exhibit B (governance memo) narrates the §4 program: written program, accountable executives, committee structure, third-party AI standards, testing and validation. Exhibit C (high-risk system detail) examines Tier 1 systems in depth - model documentation, testing results, fairness analysis, drift monitoring, third-party AI evaluation. Exhibit D (AI data) inventories the data feeding the AI Systems with lineage, ECDIS sources, third-party data acquisitions. The 12-state pilot ran early 2026 through September 2026; re-exposure September-October 2026; adoption at NAIC Fall National Meeting November 2026. Carriers responding to the pilot information request - California, Colorado, Connecticut, Florida, Iowa, Louisiana, Maryland, Pennsylvania, Rhode Island, Vermont, Virginia, Wisconsin - build the response packet now; non-pilot carriers in remaining 38 states build for 2027 deployment of AISET as the standardized examination tool. This lesson is the response-packet build: how to map each Exhibit to existing artifacts (Written Program, algorithm registry, peer-review reports, vendor scorecards), how to draft narratives that extract from documentation rather than rebuild from scratch, and how to coordinate the cross-functional response across CRO, CCO, Chief Actuary, Chief Claims Officer, CUO, CISO, General Counsel.
The AISET Information Request Anatomy and Timing
AISET arrives as a state DOI information request typically through SERFF or direct examiner contact. The request includes four exhibit templates with detailed sub-questions, a response timing window (typically 60-90 days from receipt for the initial pilot; permanent adoption may compress to 30-60 days), and instructions for narrative format, supporting documentation, and confidentiality treatment. The carrier coordinates response through CCO with cross-functional input from CRO, Chief Actuary, Chief Claims Officer, CUO, CISO, General Counsel; the final response packet is signed by the CRO and CCO on behalf of the carrier.
Response packet structure: cover letter (signed by CRO and CCO); Exhibit A response with narrative and attached registry extract; Exhibit B response with §4 program documentation; Exhibit C response with Tier 1 system detail including model documentation, peer-review reports, fairness test results, drift monitoring; Exhibit D response with data inventory, ECDIS sources, third-party data lineage; appendix with supporting documentation (Written Program, AI committee charter, RACI, vendor scorecards, incident history summary, training program documentation, regulatory notification log). Length: typically 80-200 pages depending on AI portfolio size and complexity.
The 12 pilot states (CA, CO, CT, FL, IA, LA, MD, PA, RI, VT, VA, WI) coordinated the pilot through the NAIC Big Data and AI (H) Working Group calendar. Pilot timing: Q1 2026 launch through September 2026; re-exposure for public comment September-October 2026; adoption expected at NAIC Fall National Meeting November 2026. Post-adoption: standardized examination tool deployable by any state DOI; remaining 38 states adopt at varying paces 2027-2028. Carriers in pilot states have remediation runway as part of pilot feedback; post-pilot carriers face examination without runway.
Exhibit A - AI Quantification From the Algorithm Registry
Exhibit A requests AI quantification: counts of AI Systems by business function, risk classification, line of business, vendor versus internal-build, FCRA-triggering versus not, MHPAEA-applicable versus not. Sub-questions probe AI category (predictive, generative, agentic, hybrid), decisioning role (autonomous, decision-support, productivity), consumer-facing versus internal. The exhibit's purpose: give the regulator a quantitative picture of the carrier's AI deployment scale and risk distribution.
Carrier with operationally complete algorithm registry extracts Exhibit A through query: SELECT COUNT(*), risk_tier, business_function, line_of_business FROM registry GROUP BY tier, function, line. Result is the quantitative answer; narrative wraps the numbers with methodology notes (definitions, scope decisions, edge case handling). Total Exhibit A response: 4-8 pages including tables.
Sample Exhibit A summary for a $1.5B specialty carrier: total AI Systems 47 (Tier 1 - 18; Tier 2 - 21; Tier 3 - 8). By business function: UW 14 (Cytora, Federato, Convr, Indico, Hyperscience); claims 16 (Tractable, CCC, Shift, Hi Marley, Five Sigma, Snapsheet, EagleView, ClaimXperience); pricing/actuarial 8 (Akur8, Earnix, internal ML); distribution 5 (Send, Outmarket, Vlocity, internal AMS overlays); other 4 (chatbots, productivity tools). By line: personal auto 6, homeowners 4, commercial property 7, GL 5, workers' comp 4, BOP 3, specialty 6, L&H 8, cross-line 4. Vendor versus internal: 38 vendor, 9 internal. FCRA-triggering: 14 (pricing, accelerated UW, fraud referrals). MHPAEA-applicable: 3 (utilization review L&H, behavioral overlay tools). Generative AI: 12 (drafting, summarization, chat). Agentic AI: 3 (Cytora Autopilot pilot, Shift Claims agentic SIU pilot, Federato agentic UW pilot).
Narrative wrap explains: definition of AI System (excludes deterministic rules engines), risk-tier classification methodology, scope of in-production vs. pilot, treatment of vendor sub-models versus carrier-level system. Without methodology, examiner cannot interpret counts; with methodology, counts are defensible.
Exhibit B - Governance Memo From §4 Program Documentation
Exhibit B is the narrative core: the §4 governance program as a coherent narrative. Sub-questions: program existence and approval status; accountable executive mapping; committee structure and cadence; risk classification taxonomy; algorithm inventory linkage; testing and validation summary; third-party AI standards; incident response posture; training program; continuous improvement.
Carrier with operationally complete §4 program extracts Exhibit B by section. Program existence - reference Written Program with board attestation date. Accountable executives - extract RACI mapping with named individuals. Committee structure - reference AI committee charter with membership and cadence. Risk classification - extract Tier 1/2/3 definitions with examples. Algorithm inventory - reference registry with field set and refresh cadence. Testing and validation - extract Tier-based testing cadence (Tier 1 quarterly fairness, monthly drift, ASOP 56 peer review). Third-party AI - extract vendor lifecycle (diligence, contracting, monitoring, exit) with concentration analysis. Incident response - reference runbook structure with Severity matrix. Training - reference development plan (CPCU, AIC, AIAI, CAS, SOA). Continuous improvement - reference post-incident review process and annual program refresh.
Total Exhibit B response: 15-25 pages depending on carrier complexity. The narrative is dense but factual; each statement references underlying artifact in appendix or registry. Examiner reading Exhibit B should be able to ask "show me the registry entry for the Akur8 model" and receive immediate response from Exhibit C; "show me the last AI committee meeting minutes" and receive immediate response from appendix.
Sample Exhibit B narrative opening: "The Company maintains a written AI Systems Program approved by the Board's Risk Committee on November 14, 2024 (initial) and refreshed on November 13, 2025. The Program comprises 14 sections covering AI System definitions, risk classification taxonomy, accountable executive mapping, AI committee charter, algorithm registry standards, third-party AI standards, testing and validation by risk tier, documentation standards, training requirements, incident response, reporting, exception management, and annual review. Chief Risk Officer [Name] is the accountable executive for the Program overall, with named accountability across pricing (Chief Actuary [Name]), claims (Chief Claims Officer [Name]), underwriting (Chief Underwriting Officer [Name]), and security (Chief Information Security Officer [Name]). The AI committee meets monthly with quarterly deep-dive sessions; the most recent meeting occurred on [Date] with documented minutes referenced in Appendix [X]." Each sentence references documentation.
Exhibit C - High-Risk System Detail From Registry and Peer Reviews
Exhibit C examines Tier 1 (high-risk) systems in depth. Sub-questions per system: system identification, accountable executive, model documentation, testing and validation results, fairness analysis, drift monitoring, third-party AI evaluation, incident history, version control. The exhibit is the deepest part of the AISET response - typically the longest section of the response packet.
Carrier with operationally complete registry extracts Exhibit C entry-by-entry. Each Tier 1 system response: 3-5 pages. For a carrier with 18 Tier 1 systems, Exhibit C is 54-90 pages. Each entry maps to a registry entry plus supporting documentation (model card, peer-review report, fairness test report, drift dashboard extract, vendor scorecard).
Sample Exhibit C entry for Akur8 personal auto pricing - building on registry entry AK-PL-AUTO-001 from prior lesson. Sections: (1) System identification - name, version, vendor, in-production date, deployment scope. (2) Accountable executive - Jane Smith, Chief Actuary, FCAS, MAAA, attestation March 15, 2026. (3) Algorithm owner - Robert Chen, AVP Personal Lines Pricing, ACAS. (4) Risk classification rationale - consumer-adverse rating decision, FCRA workflow integration, Colorado Reg 10-1-1 applicable, 14-state multi-jurisdiction. (5) Model documentation - GLM with elastic-net regularization; 47 features; training data lineage from carrier policy database, ISO PA, LexisNexis MVR, Verisk Vehicle; model card URL. (6) Testing and validation - quarterly fairness testing (last result March 12, 2026: disparate impact ratio 0.94 across BISG-race, gender, age band, urban/rural; equality of opportunity test; calibration parity by class); monthly drift monitoring (PSI 0.07, calibration stable, AUC 0.732); ASOP 56 external peer review January 2026 with report referenced. (7) Fairness and bias analysis - BISG monitoring not modeling; SHAP analysis showing no protected-class proxy in top 20 features; NY DFS proxy test refreshed February 2026 with finding of no material proxy. (8) Drift monitoring - registry dashboard with PSI yellow at 0.10 and red at 0.25; algorithm owner notified on threshold breach; AI committee informed at exceedance. (9) Third-party AI - Akur8 vendor with §4 attestation March 8, 2026; SOC 2 Type II March 2026 (no material gaps); sub-processor disclosure (AWS US-East-1, OpenAI for non-model platform features); 30-day model-update notification; annual audit rights. (10) Incident history - October 2025 drift exceedance with re-calibration response (Severity 2); March 2026 customer-disputed adverse-action notice with FCRA workflow review (no §4 finding). (11) Version control - v3.2 deployed March 4, 2026 (training data refresh through Q4 2025, one feature removed for proxy concern, AI committee approval March 10, 2026); v3.1 archived; full change history available.
Exhibit D - AI Data From Data Lineage and ECDIS Inventory
Exhibit D inventories the data feeding the AI Systems: data sources, lineage, third-party data acquisitions, ECDIS (External Consumer Data and Information Sources) for L&H, GLBA NPI handling, HIPAA PHI handling for L&H, data residency, retention policies. Sub-questions probe: source identification, freshness, representativeness, fairness analysis by source, vendor data acquisitions and contractual terms.
Carrier with documented data lineage in registry entries extracts Exhibit D by aggregation. Sources commonly named: carrier policy and claims databases (internal); ISO ratings, claim search, loss data (vendor); LexisNexis Risk Solutions (consumer reports, MVR, public records); Verisk (vehicle data, geocoding, weather); FEMA NRI (natural hazards); EagleView, Vexcel (aerial); D&B, Moody's Orbis (commercial business data); Milliman IntelliScript, ExamOne ScriptCheck, LexisNexis MedAdvisor (L&H prescription); MIB (L&H medical); ECRI, IBM Watson (clinical); LexisNexis ECDIS for L&H; OpenAI / Anthropic / Google for foundation model data; carrier-built training datasets for fine-tuning.
For each source: provenance, contract or license terms, data freshness, refresh cadence, fairness analysis status, GLBA handling for NPI, HIPAA handling for L&H PHI, residency, retention. Length: 10-20 pages with tables. The ECDIS inventory specifically required for Colorado Reg 10-1-1 L&H compliance also serves Exhibit D for the L&H portion.
Cross-Functional Response Coordination
AISET response packet requires cross-functional coordination. CCO leads response project; CRO is final signatory with CCO; Chief Actuary contributes Exhibit C content for pricing/reserving/capital systems and ASOP 56 peer-review references; Chief Claims Officer contributes Exhibit C content for claims systems (Tractable, CCC, Shift, Hi Marley, Five Sigma); Chief Underwriting Officer contributes Exhibit C content for UW systems (Cytora, Federato, Convr, Indico); CISO contributes data security and vendor security content; General Counsel reviews legal-privilege posture and confidentiality treatment; Chief Data Officer contributes data lineage content for Exhibit D; IT provides registry extract capability.
Project plan typical 60-90 day response window: Week 1-2: response project kickoff; review information request specifics; coordinate cross-functional team; assign exhibit ownership. Week 2-4: extract registry data for Exhibit A; draft Exhibit B from §4 program documentation; pull Tier 1 system documentation for Exhibit C. Week 4-6: draft full exhibit responses; identify gaps requiring remediation or narrative explanation; vendor coordination for vendor-specific content. Week 6-8: cross-functional review; General Counsel privilege review; CRO and CCO review; refinement. Week 8-10: final review by executive team; CRO and CCO sign-off; submission. Week 10-12: examiner follow-up questions; rapid-response capability with documented escalation.
Confidentiality Treatment and Discovery Implications
AISET response packet contains proprietary model documentation, vendor contractual terms, fairness test results, incident history. State DOI examination materials are typically confidential under state insurance laws; AISET responses inherit that treatment. General Counsel ensures confidentiality marking, request handling, and tracking of any materials shared with third parties (e.g., outside counsel).
Discovery implications: bad-faith litigation, market-conduct exam findings, public records requests may seek AISET response. State law variations on examination-material disclosure produce different exposure across states. Carriers maintain privileged investigation files separately from AISET response when applicable. General Counsel coordinates discovery posture across regulatory examination, civil litigation, and public records framework.
Post-Pilot Evolution and 2027 Deployment
The pilot's September-October 2026 re-exposure period collects public comment on AISET design, exhibit structure, sub-question content. Adoption at NAIC Fall National Meeting November 2026 finalizes the tool. Carriers in pilot states benefit from feedback opportunity; carriers outside the pilot benefit from observing the pilot's learnings before facing examination.
2027 deployment trajectory: pilot states deploy in steady-state examination; remaining states adopt at varying paces with NY, Texas, Connecticut, Nevada, California, and large states likely leading; smaller states follow over 2027-2028. Some states may add state-specific extensions to AISET (Colorado Reg 10-1-1 inventory tie-in; NY DFS Circular Letter 2024-7 proxy-test integration). Carriers should monitor state-specific extensions through state DOI bulletin tracking.
Key Takeaways
- AISET is a four-exhibit structured information request from state DOI through SERFF or direct contact. Exhibit A (AI quantification) inventories systems; Exhibit B (governance memo) narrates §4 program; Exhibit C (high-risk system detail) examines Tier 1 systems; Exhibit D (AI data) inventories data feeding systems.
- Response packet structure: cover letter signed by CRO and CCO; four exhibit responses with narratives and extracts; supporting documentation appendix. Total length 80-200 pages depending on AI portfolio complexity.
- 12 pilot states - CA, CO, CT, FL, IA, LA, MD, PA, RI, VT, VA, WI - coordinated pilot Q1 2026 through September 2026. Re-exposure September-October 2026; adoption NAIC Fall National Meeting November 2026; remaining 38 states deploy 2027-2028.
- Exhibit A extracts from registry via query. Counts by business function, risk tier, line, vendor/internal, FCRA-triggering, MHPAEA-applicable. Sample $1.5B specialty carrier: 47 systems (Tier 1: 18; Tier 2: 21; Tier 3: 8).
- Exhibit B narrates §4 program in 15-25 pages. Each statement references underlying artifact in appendix or registry. Examiner should be able to drill from any narrative point to source documentation.
- Exhibit C examines each Tier 1 system in depth, 3-5 pages per system. Carrier with 18 Tier 1 systems produces 54-90 pages of Exhibit C. Each entry maps to registry plus supporting documentation.
- Exhibit D inventories data sources: carrier internal, ISO, LexisNexis, Verisk, FEMA NRI, EagleView, D&B, Milliman IntelliScript, MIB, ECDIS for L&H, foundation model data. 10-20 pages with provenance, contract, freshness, fairness analysis, GLBA/HIPAA handling, residency, retention.
- Cross-functional coordination through CCO; CRO and CCO sign. Chief Actuary, Chief Claims Officer, CUO, CISO, Chief Data Officer, General Counsel contribute. 60-90 day project plan with executive review and rapid-response capability for examiner follow-up.
- Carriers with operationally complete §4 program and registry extract response through query and formatting. Carriers without face writing-from-scratch with field gaps that surface during examiner follow-up. Mechanical build passes; memo-only build fails.
Skill.re