A regional carrier's coverage attorney in Hartford, Connecticut spent six hours in February 2026 writing a memo on a faulty-workmanship Anti-Concurrent-Cause analysis. Three of the four citations the LLM had inserted were not real cases. The fourth was real but stood for the opposite proposition the LLM described. A New Jersey homeowners-claims adjuster sent an adverse-determination letter citing ISO endorsement HO 04 90 04 24 - an edition that does not exist; the real form is HO 04 90 09 22. A fraud-network alert from a third-party platform produced a 38-claim cluster that was 82% Hispanic-surnamed insureds - and the model's variable selection had used Bayesian Improved Surname Geocoding as a feature without the bias-test exhibit Colorado SB 21-169 demands. A Five Sigma auto-coverage summary on a Texas BI claim doubled the open IBNR by misreading "12,500" as "125,000" when the loss-run column header read $thousands. A Tractable photo estimate returned $19,400 ACV on a 2019 Highlander; the insured produced three Atlanta dealer listings averaging $23,800. Each of these is a real 2026 failure pattern. Each was preventable. This lesson walks through six categories of where AI fails on the insurance desk - hallucinated coverage, phantom endorsements, fabricated case citations, biased fraud signals, mis-read scanned data, and over-trusted photo estimates - with each failure mapped to a named guardrail. The cardinal rule from the next lesson rests on this one: verification is the discipline that makes the L1 categorization and the L1 mechanics work in practice.

Failure 1 - Hallucinated ISO and AAIS Endorsement Numbers

The most common 2026 LLM failure on the insurance desk is fabrication of a coverage form number or edition that does not exist. The model produces "CG 00 03 12 19" - there is no ISO CGL coverage form CG 00 03; the CGL is CG 00 01 (with the 12 19 or 04 13 or earlier edition date). The model produces "HO 04 90 04 24" - the carrier's actual special-form endorsement library has HO 04 90 09 22 and earlier editions. The model produces "BOP 00 04" - there is no BOP 00 04; the Businessowners Coverage Form is BOP 00 03. Each output is fluent, plausible, and wrong.

The mechanic from Lesson 2 explains why: the model samples the next token from a probability distribution. Real form numbers (CG 00 01, CG 21 47, CG 24 04, HO 00 03, HO 00 05, HO 04 90, CA 00 01, WC 00 00 00 A) appear in the training corpus, so the model produces plausible-but-fabricated neighbors. The edition date pattern (04 13, 12 19, 09 22, 04 24) follows a recognizable format, so the model invents new dates that have not been filed.

The named guardrail. The L3 RAG architecture grounds the LLM in the carrier's form-of-record library. The system prompt forbids any ISO/AAIS form citation that is not in the retrieved set. An output classifier scans for form-number strings and validates each against the catalog. The verifier - the adjuster, underwriter, or producer - cross-checks any cited form against the dec page and the endorsement schedule before the artifact ships. The L4 incident-response playbook includes "hallucinated form number" as a named trigger; the L2 verification checklist makes the cross-check non-negotiable.

The Hartford coverage attorney's six-hour rewrite is the canonical case. The original LLM-drafted memo cited CG 00 03 04 13 four times. Each citation looked correct. The verifier (the attorney) did not catch the fabrication on first read because the prose around the citation was internally consistent. The fix - RAG over the ISO catalog plus an output classifier rejecting non-catalog form strings - is structural, not behavioral. A better attorney does not catch every fabrication. A better architecture does.

Failure 2 - Fabricated Case Citations and Misquoted Exclusion Language

The Hartford attorney's other failure was case-law fabrication. The LLM cited "Liberty Mutual v. ABC Construction, 2024 Conn. App. 432" - there is no such case in the Connecticut Appellate Court reporter for 2024. The model invented a name, a year, and a citation that looks valid because cases are named in a recognizable pattern (party v. party, citation format Vol. Reporter Page).

Case-law fabrication is particularly damaging because legal-research tools - Westlaw, LexisNexis, Bloomberg Law, Casetext - exist precisely to surface real cases. The LLM operating without RAG against those databases is generating prose that mimics the surface form of a case citation. Coverage attorneys, adjusters, and underwriters who paste an LLM-drafted coverage opinion into a Reservation-of-Rights letter or a denial letter risk citing a case that opposing counsel will reveal does not exist - converting a borderline coverage dispute into a bad-faith exposure.

The parallel failure is misquoted exclusion language. The LLM drafts a CGL coverage analysis that quotes the faulty-workmanship exclusion as "Your work … as a result of failure to perform" - and the actual exclusion language in CG 00 01 04 13 is a specific construction the LLM softened or rephrased. The discrepancy between drafted and actual exclusion language is the foothold opposing counsel uses in a coverage dispute.

The named guardrail. Paste the actual form excerpt and the actual case excerpt into the LLM's context window; instruct the model to quote only from the provided text; reject any citation not appearing in the retrieved Westlaw/Lexis result set. The L3 RAG architecture indexes the carrier's claim-handling manual, the ISO form catalog by edition, and a venue-state case-law corpus (refreshed on a documented cadence). The L2 verification step on every ROR, coverage opinion, and reserve memo is to validate every citation and every quoted clause against the source - before the letter leaves the carrier or the agency.

Failure 3 - Biased Fraud Signals and Proxy-Variable Drift

A 2026 fraud-network alert at a top-25 personal-lines carrier flagged a 38-claim soft-tissue cluster across the metro Atlanta area. The Shift-style score was elevated; the network analysis grouped the claims by shared attorney, shared medical clinic, and geographic proximity. When the SIU manager reviewed the cluster before referral, 82% of the named insureds had Hispanic surnames. The carrier's L4 compliance officer pulled the model's variable selection: Bayesian Improved Surname Geocoding (BISG) had been included as a feature in the network-detection model without a documented bias-test exhibit, and the model's geographic features were collapsing legitimate immigrant-community claim patterns into the fraud-cluster signal.

This is a proxy-variable failure. BISG is a defensible-but-careful technique (the Colorado Reg 10-1-1 implementation guidance and the academic literature both treat it as permissible when paired with bias testing); using it as an unweighted feature in a fraud model produces disparate impact even when no human ever saw the surname. NY DFS Circular Letter 2024-7's proxy test demands written analysis of any variable that may act as a stand-in for a protected class. Colorado SB 21-169 quantitative bias testing requires the disparate-impact ratio analysis with a documented mitigation if below threshold.

The named guardrail. The L3 bias-detection workflow (covered in L3 Ch6) tests every deployed fraud model on protected-class proxies, computes the disparate-impact ratio per cluster, and remediates when below threshold. The L4 algorithm inventory entry for the fraud model names the variables, the bias-test exhibit, and the mitigation. The L1 lesson here is the categorization: a fraud signal correlated with a protected class is an NAIC §4 violation even if no human ever saw the cluster - the carrier owns the model's output. The SIU manager's review is the human checkpoint, but the model design upstream is what prevents the cluster from forming in the first place.

Failure 4 - Mis-Read Scanned Data and the Doubled IBNR

A Texas commercial property carrier's reserving actuary received a quarterly Schedule P preview drafted by the carrier's enterprise LLM. The Q3 2025 paid-loss line on the BI accident year showed $4.2M. The actual paid-loss line was $420K. The LLM had read the loss-run column header (which said "$ thousands") and applied the multiplier wrong - producing a number 10x too high. The downstream IBNR estimate doubled. The actuarial team caught it in roundtable; the supervisor caught it in second review. The version sent to the chief actuary would have triggered an unnecessary reserve strengthening.

This is mis-read scanned data - or, more precisely, mis-applied unit semantics. The LLM extracted the cell value correctly; it failed to honor the column header that said "$ thousands" and instead read the value as raw dollars. The same failure happens on loss runs with mixed-currency columns (USD vs. CAD), on SOVs with mixed-unit area columns (square feet vs. square meters), and on policy limits stated in some carriers' templates as "1000" meaning $1,000,000.

The named guardrail. The L2 structured-output prompting forces the LLM to surface unit assumptions ("paid loss interpreted as USD raw dollars; column header reads $ thousands - please confirm"). The L3 verification workflow runs a numeric-reasonableness check on every extracted total against benchmarks (a quarterly paid-loss line 10x the prior period's should trigger an automatic flag). The L4 algorithm-inventory entry for the IDP layer and the LLM that summarizes its output documents the unit-handling discipline. Hyperscience and Indico's IDP have improved unit handling materially in 2026; the LLM summarization layer is the second checkpoint.

Failure 5 - Five Sigma Coverage Summary on the Wrong Policy Edition

The Atlanta adjuster's water-damage claim in the program's running scenario surfaces this failure. Five Sigma's auto-drafted coverage summary cited ISO HO 00 03 - the basic Special Form - when the dec page shows HO 00 05, the Comprehensive Form. HO 00 03 and HO 00 05 have materially different open-perils-vs-named-perils logic for personal property. The Five Sigma summary also missed an ordinance-or-law endorsement that bears directly on the rebuild estimate; the endorsement attachment was outside the context window the Five Sigma workflow loaded.

This is a context-window failure compounded by an edition-recognition failure. The mechanic is from Lesson 2: anything outside the context window is invisible; ambiguous artifacts inside the window produce plausible-but-wrong outputs. The fix is structural - the L3 RAG architecture loads the dec page, the full endorsement schedule, and the carrier's form catalog by edition into the context; the system prompt forbids coverage opinion without explicit citation to a loaded form.

The named guardrail. The L2 verification checklist on every Five Sigma coverage summary verifies the dec page (cross-check policy form and edition), the endorsement schedule (cross-check each endorsement against the form), and the sublimit table (cross-check against the actual policy). The override discipline documents the corrected coverage position in the file note. The L4 algorithm-inventory entry for Five Sigma names the failure mode and the override discipline. The L5 board narrative ties the Five Sigma deployment lift to the verification discipline - not to the tool alone.

Failure 6 - Over-Trusted Photo Estimates and the ACV Dispute

The Atlanta adjuster's Tractable photo estimate on the 2019 Toyota Highlander returned $19,400 ACV against an outstanding loan of $22,800. The insured produced three Atlanta-area dealer listings averaging $23,800. The dispute is a real predictive-model failure mode: Tractable's comparable-vehicle set may not represent the local Atlanta market accurately, the vehicle's condition may diverge from the photo-derived assessment (interior wear, mechanical maintenance, prior accident history), and the model's confidence band may not include the legitimate top of the local market range.

Tractable's published Admiral Seguros case shows 70-75% digital completion - meaning 25-30% of estimates require an adjuster override or dispute resolution. The over-trust failure pattern is the adjuster who closes the file on the AI estimate without engaging the dispute, producing a customer-effort failure, a complaint to the state DOI, and potential bad-faith exposure under Texas Insurance Code §541, Florida §624.155, or California Cumis/Brandt. The under-trust failure pattern - the adjuster who second-guesses every Tractable estimate - produces zero cycle-time lift.

The named guardrail. The L2 verification step on every Tractable estimate documents the comparable-vehicle set, the local-market range, and the disposition of any insured dispute. The L3 escalation workflow surfaces disputes above a threshold to a senior adjuster or a salvage-and-total-loss specialist. The L4 algorithm-inventory entry for Tractable names the override discipline and the bad-faith exposure framework. The L5 narrative ties the Tractable lift to the dispute-handling discipline - and the metric the chief claims officer watches is not "AI ACV accepted" but "AI ACV accepted with documented insured concurrence."

The Failure-Mode Table the L1 Learner Carries to the Desk

Six categories, each with a named guardrail, each anchored to a real 2026 desk incident.

Hallucinated ISO/AAIS form numbers. Guardrail: RAG over the carrier's form-of-record library; system prompt forbids non-catalog citations; output classifier rejects unmatched form strings. Verifier checks every citation against the dec page and endorsement schedule.

Fabricated case citations and misquoted exclusion language. Guardrail: RAG over Westlaw/Lexis venue-state corpus and the carrier's claim-handling manual; paste actual form excerpts and case excerpts into context; verifier validates every citation against the source.

Biased fraud signals and proxy-variable drift. Guardrail: bias-test exhibit per deployed fraud model; disparate-impact ratio per cluster; SIU manager review before referral; Colorado SB 21-169 + NY DFS proxy-test memo; algorithm-inventory entry documenting variables and mitigation.

Mis-read scanned data and unit semantics. Guardrail: structured-output prompting forcing unit-assumption surfacing; numeric-reasonableness checks against benchmarks; IDP layer's unit-handling discipline; L4 algorithm-inventory entry on the summarization layer.

Five Sigma / Hi Marley / generative coverage summary on the wrong edition or with missing endorsements. Guardrail: RAG architecture loading dec page + endorsement schedule + form catalog; system prompt forbids coverage opinion without explicit citation; adjuster override discipline; documented file note.

Over-trusted Tractable / CCC / EagleView photo estimates. Guardrail: documented comparable-set review; local-market range cross-check; escalation workflow on disputes above threshold; bad-faith exposure framework in TX/FL/CA documented in the L3 escalation chapter.

Why These Failures Are Not Bugs and Do Not Go Away with a Better Model

The temptation in 2026 is to read failure cases as "we need a better model." Better models hallucinate less, mis-read less, and produce fewer biased signals - but they still produce all three failure modes. The mechanic from Lesson 2 explains why: generative models sample from probability distributions, and the distributions can place high probability on plausible-but-false continuations regardless of model size. Predictive models are bounded by training data, and disparate impact emerges from data patterns the model is trained to extract. Better tooling is necessary but not sufficient.

The structural answer is verification chains and guardrails. RAG architectures ground generative output. Output classifiers reject unmatched citations. Bias-test exhibits surface disparate impact before deployment. Numeric-reasonableness checks catch unit-handling failures. Override workflows handle predictive edge cases. The L2 verification checklist makes the discipline operational; the L3 workflow design embeds it; the L4 algorithm inventory documents it; the L5 board narrative ties combined-ratio movement to the discipline.

The carriers that get the AM Best 41%/60% transformation outcome are the ones that built the guardrails into the operating model - not the ones that bought the best tools. The Hartford attorney's six hours, the New Jersey adjuster's wrong-edition letter, the Atlanta fraud cluster, the Texas IBNR doubling, the Five Sigma HO 00 03 vs. HO 00 05 miss, the Tractable ACV dispute - every one of them is a guardrail problem solved by structure, not by buying a different LLM.

Key Takeaways

• Six 2026 failure categories on the insurance desk: hallucinated ISO/AAIS form numbers (CG 00 03 doesn't exist; CG 00 01 does), fabricated case citations and misquoted exclusion language, biased fraud signals from proxy-variable drift (the 82% Hispanic-surname Atlanta cluster), mis-read scanned data and unit semantics (the doubled IBNR from "$ thousands" mis-applied), Five Sigma / Hi Marley coverage summaries on the wrong policy edition with missing endorsements, and over-trusted Tractable / CCC / EagleView photo estimates (the $19,400 vs. $23,800 ACV dispute).
• Each failure is structural, not behavioral. The Hartford coverage attorney's six hours, the New Jersey adjuster's wrong-edition letter, the Texas reserving actuary's doubled IBNR - none of them is solved by a better verifier; all of them are solved by RAG architectures, output classifiers, bias-test exhibits, numeric-reasonableness checks, and override workflows.
• Hallucinated form numbers and case citations have the same mechanical root: the LLM samples plausible-token sequences that follow the pattern of real form numbers and real case names. RAG over the carrier's form-of-record library and a Westlaw/Lexis corpus is the structural answer; system prompts that forbid non-catalog citations are the operational anchor.
• The 82% Hispanic-surname fraud cluster is an NAIC §4 violation even if no human ever saw the cluster. The carrier owns the model's output. Colorado SB 21-169 quantitative bias testing and NY DFS Circular Letter 2024-7's proxy-test memo are the regulatory hooks; Bayesian Improved Surname Geocoding is defensible but careful and requires the bias-test exhibit and mitigation documented.
• Unit-semantics failures on scanned loss runs and SOVs (the doubled IBNR; mixed USD/CAD columns; "1000" meaning $1,000,000) are caught by structured-output prompting forcing unit-assumption surfacing and by numeric-reasonableness checks against prior-period benchmarks. The L3 verification workflow enforces both.
• Five Sigma's wrong-edition failure (HO 00 03 vs. HO 00 05) is a context-window failure compounded by an edition-recognition failure. The fix is loading the dec page, the full endorsement schedule, and the form catalog; the system prompt forbids coverage opinion without explicit citation to a loaded form; the adjuster's override discipline documents corrections in the file note.
• Tractable's $19,400 vs. $23,800 ACV dispute is a predictive-model edge case requiring escalation and documented comparable-set review. Over-trust and under-trust are both failure modes; the L4 metric is "AI ACV accepted with documented insured concurrence," not raw acceptance rate.
• Better models hallucinate less but still hallucinate; better predictive models reduce disparate impact but still produce it. The structural answer is verification chains and guardrails. The carriers achieving the AM Best 41%/60% transformation outcome built the guardrails into the operating model; the L2 verification checklist, L3 workflow design, L4 algorithm inventory, and L5 board narrative are the operating-model surfaces that make the L1 mechanics actionable.