Feeding an ACORD 125 + ACORD 140 + SOV + 5-year loss run into a chat tool and expecting a structured submission summary back without a five-line preamble, a structured artifact target, and a verification constraint produces predictable failure: the named insured's address gets paraphrased into the wrong city, the per-location TIV sums to a number 3% off the SOV total, the roof-age field gets fabricated on locations where the source doesn't carry it, an ISO endorsement number gets cited that doesn't exist in the form library, and the loss-run severity narrative invents two cat events that never happened. The Dallas 8:14 a.m. submission packet on Acme Warehousing - 47 buildings, $182M TIV, three over $25M, treaty $25M facultative threshold, cat aggregate 78% consumed - is the canonical test case for whether the prompt produces an artifact the underwriter can sign or an artifact that lands in the §4 examiner's evidence file. This lesson walks the actual prompt against the actual ACORD-shape data, names the five common failure modes, and prescribes the constraint patterns that close each failure structurally. By the end, you can feed any ACORD 125 + 140 + SOV + loss-run packet into Claude / GPT / Gemini / a carrier enterprise LLM and get back a structured submission summary that doesn't hallucinate named insured address, doesn't invent sublimit, doesn't fabricate roof age, and doesn't cite a non-existent ISO endorsement number.

The ACORD-Shape Input and What the LLM Actually Sees

The ACORD 125 (Commercial Insurance Application - Applicant Information) carries named insured, mailing address, FEIN, NAICS code, contact, prior carrier, prior policy number, loss-history flag, and the broker information. The ACORD 140 (Property Section) carries per-location: street address, occupancy, ISO construction class, year built, square footage, sprinklered Y/N, central station alarm Y/N, protection class, distance-to-hydrant, distance-to-fire-station, and the building/BPP/BI values. The SOV (Statement of Values) carries per-location: location number, full address, building value, BPP value, BI value, EDP value (where applicable), construction, occupancy, year built, square footage, total values. The 5-year loss run carries: date of loss, cause of loss, status (open/closed/reopened), paid indemnity, paid ALAE, outstanding reserve, total incurred, recovery type, subrogation status, deductible. Together: roughly 15-50 pages of structured data depending on building count.

The LLM does not see PDFs natively. The carrier's intake stack (Convr, Send, Indico, Hyperscience) OCRs the ACORD packet into a JSON or CSV payload; the cleaned data goes into the prompt context. Without the OCR layer, the LLM is reading a flattened text representation that has lost structural alignment between columns; with the OCR layer, the LLM sees rows. The L3 RAG architecture sits between the OCR and the prompt: ACORD fields map to canonical names; ISO construction codes load from a reference table; the appetite guide loads from the carrier's published guide; the treaty schedule loads from the reinsurance team's current cession schedule.

The Dallas packet as concrete input. The Acme Warehousing submission OCRs into: ACORD 125 with Acme Warehousing Inc. at 4720 Industrial Blvd, Dallas TX 75247, FEIN 75-3xxxxxx, NAICS 493110 (General Warehousing and Storage), broker Roper-Smith Insurance Brokers in Plano TX. ACORD 140 with 47 locations covering 12 ZIP codes across Dallas-Fort Worth Metroplex. SOV totals $182M TIV with building values from $0.4M to $48M, three locations above $25M. Loss run shows two non-cat fire losses ($142K and $87K incurred, 2022 and 2024), four wind-related losses ($14K-$61K, two in 2022 and two in 2024). The L3 RAG enrichment joins: FEMA NRI risk index per location, EagleView roof age per address, D&B credit on Acme Warehousing Inc., and the carrier's appetite guide showing class 5 light non-combustible quotes with restrictions while class 1 (frame) habitational declines in coastal counties.

The Five Failure Modes and the Constraints That Close Them

Five failure modes recur across ACORD-input submission-summary tasks. Each has a documented hallucination surface; each has a constraint that closes it structurally.

Failure 1 - Named-insured address paraphrase. The LLM substitutes a near-match address (different ZIP, different city, different street number) when the source has a complete address. The mechanism: the LLM's training prior on common commercial addresses overrides the specific source. Constraint that closes it: "Quote named insured address verbatim from ACORD 125 field. Do not paraphrase. If the OCR produced uncertain characters, flag with [verify against source PDF] rather than guess."

Failure 2 - TIV summation drift. The LLM produces a TIV total that differs from the SOV by 1-5% because it estimates rather than sums. The mechanism: large numerical arithmetic across many rows degrades model accuracy. Constraint that closes it: "Do not compute TIV total - the SOV carries the total. Quote the SOV-stated total verbatim. If per-location values are needed, quote each line; do not compute aggregates."

Failure 3 - Roof-age fabrication. The LLM fills the roof-age field with a plausible number (15 years, 20 years, 8 years) on locations where the source data doesn't carry it. The mechanism: trained pattern-completion on incomplete inputs. Constraint that closes it: "Roof age is not in ACORD 125, ACORD 140, or SOV unless explicitly stated. If you need roof age, request it from the producer or pull from EagleView retrieval. If neither is available, mark roof age [not in source - request supplemental]."

Failure 4 - ISO endorsement-number fabrication. The LLM cites a plausible-looking endorsement code (CG 24 35, CG 22 79, CP 12 19) that may or may not exist or may not be the right edition. The mechanism: ISO code patterns are dense in the training corpus, and the model interpolates. Constraint that closes it: "Cite ISO forms by exact form number and edition only when the source explicitly carries them. Do not infer endorsement numbers from coverage descriptions. If a coverage feature is described without a form citation, name the feature and mark [form citation: verify against ISO form library]."

Failure 5 - Loss-run severity narrative invention. The LLM produces a 'severity narrative' on the loss run that invents cat events, severity trends, or per-occurrence amounts not in the source. The mechanism: narrative generation defaults to coherent-story patterns that smooth over data gaps. Constraint that closes it: "Quote loss-run figures verbatim from the source. Severity narrative must cite specific losses by date, cause, and amount; do not aggregate or characterize beyond what the source supports. If a trend assertion requires data the source does not provide (e.g., industry severity benchmarks), mark the assertion [unsupported by source - separate analysis required]."

The Full Submission-Summary Prompt on the Acme Packet

Apply the L2 Lesson 1 architecture: five-line preamble + five-part body + reason-code clause. The actual prompt:

System message. "You are a licensed P&C underwriter in the commercial-property line at a regional admitted carrier writing in TX, OK, AR, LA, MS, NM. NAIC Model Bulletin §4 reason-chain discipline applies. Cite policy forms by exact edition. Every adverse-decision recommendation must enumerate variables and flag protected-class proxies. If uncertain, mark [verify] and state the source document. Do not invent ISO forms, endorsement numbers, NCCI codes, carrier appetite specifics, or loss-run details. Do not paraphrase named-insured addresses. Do not compute TIV totals - quote them from the SOV. Do not fabricate roof age. Do not infer endorsement numbers from coverage descriptions. Loss-run severity narrative must cite specific losses by date, cause, and amount only."

User message. "Produce a structured submission summary for the Acme Warehousing Inc. account in the format that drops into Federato. Context: ACORD 125 attached (named insured Acme Warehousing Inc., FEIN 75-3xxxxxx, NAICS 493110, broker Roper-Smith Insurance Brokers, Plano TX). ACORD 140 attached (47 locations, ISO construction codes per location, sprinklers/alarms per location, protection class per location). SOV attached (47 locations, TIV $182M, three locations above $25M with values $26.4M, $31.8M, $48.2M). 5-year loss run attached (two non-cat fire 2022 + 2024, four wind 2022 + 2024). FEMA NRI per location loaded via RAG. EagleView roof age per address loaded via RAG (32 of 47 locations have EagleView data; 15 do not). Carrier appetite guide loaded via RAG. Treaty schedule: $25M single-risk facultative threshold; cat aggregate 78% consumed for the quarter. Artifact: structured submission summary with (1) named insured + broker identification verbatim from ACORD 125; (2) SOV total verbatim from source plus per-location TIV sums quoted verbatim; (3) construction profile by ISO class with count + TIV per class; (4) protection profile (sprinklered, alarmed, protection class distribution); (5) FEMA NRI summary per location with the three highest-risk locations named; (6) EagleView roof-age data where available; locations without EagleView marked [no EagleView data - request from producer]; (7) loss-run summary by year + cause + amount verbatim, no severity narrative beyond stated facts; (8) appetite-match summary against carrier appetite guide with per-location appetite status; (9) treaty-cession recommendations for the three locations above $25M; (10) cat-aggregate impact summary; (11) missing-information requests; (12) reason-code summary for any adverse recommendations. Constraint: do not paraphrase named-insured address; do not compute TIV beyond quoting source figures; do not fabricate roof age on the 15 locations without EagleView; do not invent endorsement codes; do not produce severity narrative beyond cited losses. Format: structured headings per artifact section, tables for per-location data, bullet lists for missing-info and reason codes, length 1,200-1,800 words."

Output structure. The output drops into Federato as a submission record with each section mapping to a Federato data field. Named insured + broker → identification fields. SOV totals → exposure fields. Construction and protection profiles → underwriting attributes. FEMA NRI summary → cat-exposure fields. Roof-age data → supplemental risk attributes (with the [no EagleView data] flag triggering Federato's missing-data workflow). Loss-run summary → historical loss attributes. Appetite-match summary → appetite workflow. Treaty-cession recommendations → reinsurance workflow. Missing-info requests → broker-communication queue. Reason codes → §4 documentation chain.

What the Prompt Buys vs. an Unconstrained Prompt

Run the same Acme packet through an unconstrained prompt - "summarize this submission for the underwriter" - and observe the documented failure modes. The named-insured city gets paraphrased to "Plano, TX" (the broker's city, not the insured's). The TIV total comes back $185.4M (a 1.9% drift from the $182M actual). The roof-age field gets filled on every location with values ranging 8-22 years based on construction-year minus roof-replacement-cycle math the model invented. Two ISO endorsement numbers appear in the coverage section - CG 22 79 and CP 12 19 - neither of which the broker's quote sheet contains. The loss-run severity narrative cites a 2023 cat event "consistent with regional weather patterns" that never occurred on the Acme account. The output reads coherent. The output is wrong on five surfaces.

The constrained prompt produces output that quotes Dallas TX 75247 verbatim, quotes $182M TIV verbatim, marks the 15 EagleView-less locations [no EagleView data], cites no endorsement number without source confirmation, and produces loss-run summary citing only the two fire losses and four wind losses by date and amount as the source provides. The underwriter signs the output because every assertion traces back to a source; the §4 examiner's potential evidence file stays empty because every adverse recommendation includes the variables-driving-decision narrative; the carrier's E&O posture stays defensible because the artifact is auditable.

The Supplemental Prompts Around the Submission Summary

The structured submission summary is the anchor artifact, but a complete L2 UW packet workflow includes supplementary prompts that share the same context and the same constraint posture. Each prompt customizes on artifact target.

Appetite memo prompt. Takes the submission summary as input and produces a one-page appetite memo recommending bind/refer/decline/quote-with-restriction with reason codes. Customizes the Artifact line: "appetite memo with overall recommendation, supporting facts from submission summary, treaty-cession analysis, reason codes for any restrictions, missing-info requests, and producer outreach plan if quote-with-restriction." L2 Lesson 5 walks this in detail.

Missing-information request prompt. Takes the submission summary + the carrier's required-data checklist and produces a structured request to the broker. Customizes the Artifact line: "missing-info request in email format, listing each missing field with the source document that would carry it (COPE worksheet, supplemental construction details, roof inspection report, ordinance-or-law analysis, business-interruption worksheet), the carrier's deadline for response, and the underwriting impact of each missing item."

COPE supplemental request prompt. Takes the per-location construction/protection data and produces a COPE-specific supplemental request for the locations with ambiguous classification. Customizes the Artifact line: "COPE supplemental for locations flagged [verify construction] or [verify protection], requesting wall construction, roof construction, floor construction, sprinkler type + coverage area, alarm specifics, and any recent loss-prevention investments."

Treaty-cession analysis prompt. Takes the three locations above $25M + the treaty schedule and produces the cession analysis. Customizes the Artifact line: "treaty-cession analysis with facultative cession recommendations per location above $25M, surplus-treaty layer impact, cat-aggregate impact against the 78% consumed posture, and reinsurance team coordination note." L2 Lesson 6 walks this in detail.

Why the Five Failure Modes Recur Across ACORD-Input Tasks

The five failure modes are not random. Each ties to a structural property of how LLMs handle insurance ACORD-shape input. Understanding the mechanism makes the constraints durable rather than ad-hoc.

Named-insured paraphrase happens because the model's training prior on commercial addresses (frequency-weighted toward major cities, common ZIPs, recognizable streets) overrides specific source detail when the OCR layer has uncertainty. The fix is verbatim quoting from the named field; the L3 fix is OCR confidence-band loading so the model knows which characters are uncertain.

TIV summation drift happens because large arithmetic across many rows degrades; the model interpolates a plausible total. The fix is the rule that the model does not compute aggregates; the source carries them. The L3 fix is structured data ingestion where the SOV total is a named field, not a sum the model has to reproduce.

Roof-age fabrication happens because pattern completion on commercial-property data fills missing fields with construction-year-minus-cycle math. The fix is naming roof age as not-in-source and requiring [no EagleView data - request from producer]. The L3 fix is RAG loading EagleView per address with explicit nulls on locations EagleView doesn't cover.

ISO endorsement-number fabrication happens because ISO code patterns are dense in the training corpus and the model interpolates. The fix is naming the coverage feature without the citation when source doesn't provide it. The L3 fix is RAG loading the actual ISO form library against the carrier's bound endorsement schedule so the citation comes from retrieval.

Loss-run severity narrative invention happens because narrative generation smooths data gaps with coherent-story patterns. The fix is restricting to cited losses by date + cause + amount only. The L3 fix is structured loss-run ingestion where each loss is a row with explicit fields, and severity narrative is generated against the row set rather than against a paraphrased summary.

What Changes When the L3 RAG Architecture Is Mature

The constraints in this lesson are the L2 prompt-level discipline. The L3 RAG architecture sits beneath, loading retrieval sources that make each failure mode structurally harder to trigger. EagleView retrieval per address closes roof-age fabrication. ISO form-library retrieval against the carrier's bound endorsement schedule closes ISO-number fabrication. Structured loss-run ingestion as row-based JSON closes severity-narrative invention. Named-field ingestion on ACORD 125 with OCR confidence per field closes address-paraphrase. SOV total as named field closes TIV-summation drift.

The L2 prompt discipline survives even when L3 RAG is immature; L3 RAG compounds the discipline when mature. A carrier with L2 prompt discipline alone produces auditable output most of the time; a carrier with L2 + L3 produces auditable output structurally. The L4 algorithm-inventory entry on the submission-summary prompt template names: template version, retrieval-source list, constraint set, prompt-log retention policy, override-pattern observations, and incident-response runbook for output failures.

Key Takeaways

• Five failure modes recur across ACORD-input submission-summary tasks. (1) Named-insured address paraphrase; (2) TIV summation drift; (3) Roof-age fabrication; (4) ISO endorsement-number fabrication; (5) Loss-run severity narrative invention. Each has a documented hallucination surface and a constraint that closes it structurally.
• The full submission-summary prompt on the Acme Warehousing packet applies the L2 Lesson 1 architecture. Five-line preamble + five-part body + reason-code clause; system message restricts hallucination on five named surfaces; user message loads ACORD 125 + 140 + SOV + loss run + FEMA NRI + EagleView + appetite guide + treaty as context; artifact is 12-section Federato-shaped record.
• Constrained prompts produce auditable output; unconstrained prompts produce coherent wrong output. The Acme packet unconstrained produces wrong city, 1.9% TIV drift, fabricated roof ages, invented endorsement codes, and severity narrative citing non-existent cat events. The constrained version quotes verbatim, marks missing data, and cites only what the source provides.
• The L3 RAG architecture compounds L2 prompt discipline. EagleView per address closes roof-age fabrication; ISO form-library retrieval closes endorsement-number fabrication; structured loss-run row ingestion closes severity invention; SOV-total as named field closes TIV drift; OCR confidence per ACORD field closes address paraphrase.
• Supplemental prompts around the submission summary share the same context and constraint posture. Appetite memo (L2 Lesson 5), missing-info request, COPE supplemental request, treaty-cession analysis (L2 Lesson 6). Each customizes the Artifact line; preamble + constraints + reason-code clause stay constant.
• The named-insured paraphrase mechanism is the model's training prior overriding source detail. Verbatim quoting from named fields is the prompt-level fix; OCR confidence-band loading is the L3 fix; the artifact stays defensible against §4 examination because every assertion traces back to a source.
• The Acme Warehousing packet - 47 buildings, $182M TIV, three over $25M, treaty $25M facultative threshold, cat aggregate 78% consumed - is the L2 canonical test case. Outputs Federato submission record with 12 sections; missing-data flags route to producer outreach; treaty-cession recommendations route to reinsurance; reason codes route to §4.4 documentation.
• The constraint posture in this lesson scales beyond commercial property. ACORD 126 (Commercial GL) needs the same constraint set; ACORD 130 (Workers Comp) needs class-code verification against NCCI manual; ACORD 137 (Commercial Auto Section) needs MVR + DOT number verification; ACORD 175 (Surplus Lines) adds stamping-and-tax verification. The five failure modes are LOB-agnostic; the artifact-specific constraints customize.