Pasting an SSN, a date of birth, a driver's license number, a recorded statement transcript, a named loss run, an MIB code hit, or an electronic health record snippet into a public LLM - ChatGPT consumer, Claude.ai consumer, Gemini consumer, Microsoft Copilot consumer, Meta AI - is a GLBA Safeguards Rule event. For L&H carriers and others handling PHI, it is also a potential HIPAA breach under §164.402 - and depending on the data and the consumer's residency, a CCPA/CPRA, CPA, VCDPA, CTDPA, UCPA, or TIPA notification event. The consequences are real: state DOI fine, OCR HHS enforcement, plaintiff-bar class action under state UDAP statutes, and personal-license-board exposure for the producer or adjuster who pasted. The carrier-approved alternatives are Microsoft Copilot for M365 with Enterprise terms, Azure OpenAI with data-residency and no-training contractual terms, Google Vertex AI with no-training and BAA-eligible deployment, AWS Bedrock with HIPAA-eligible tier and BAA, and on-premise deployments. The BAA chain, the no-training paragraph, the data-residency clause, and the audit-rights provision are the four contractual artifacts that separate compliant carrier AI from a breach event. This lesson is the rules, the consequences, the alternatives, and the file note.

What Counts as NPI and What Counts as PHI

NPI (Nonpublic Personal Information) under GLBA includes any personally identifiable financial information provided by a consumer to a financial institution, resulting from any transaction or service performed for the consumer, or otherwise obtained by the financial institution. For insurance, this includes: name + address + SSN combination, driver's license number, date of birth, financial account information, insurance application data, premium and payment records, claim history with identifiers, loss-run entries naming the insured, recorded statements, EUO transcripts, medical records when received by a P&C carrier in connection with a BI claim, MVR records linked to a named insured, MIB code hits, and any model output that derives from any of these inputs and can be linked to an individual.

PHI (Protected Health Information) under HIPAA is individually identifiable health information held by a covered entity or business associate. For L&H carriers operating as covered entities (health plans, certain group LTD plans, certain Medicare Advantage / Medicare Supplement operations), this includes diagnostic codes (ICD-10), procedure codes (CPT), prescription history, treating-physician notes, functional-capacity evaluations, mental-health-and-substance-use disorder records, lab results, imaging reports, claims-and-payment records with health context, and any inference derived from health data that could re-identify the individual.

Most insurance professionals encounter both NPI and PHI in their work. The L&H underwriter touching MIB + Rx + ECDIS is handling NPI and PHI. The P&C BI adjuster handling medical records subpoenaed under defense-counsel BAA is handling PHI received as a business associate. The producer drafting a renewal narrative including loss history is handling NPI. The actuary running a model with insured-level features is handling NPI in aggregate but in identifiable form during model build and validation.

The Public LLM Problem - Why Pasting Is a Breach Event

Public LLMs - ChatGPT consumer (OpenAI), Claude.ai consumer (Anthropic), Gemini consumer (Google), Microsoft Copilot consumer free tier, Meta AI, and various wrapper apps using these underlying APIs - are designed for general-purpose consumer use. Their default terms allow the provider to use submitted content for model training, evaluation, and product improvement. The data may be retained, reviewed by human contractors, and used to shape future model behavior. None of this is hidden - it's in the terms of service - and none of it is compatible with GLBA, HIPAA, or any state-specific insurance confidentiality framework.

The breach event mechanics: an insurance professional pastes an SSN, a DOB, a loss-run row naming the insured, or a recorded-statement transcript into ChatGPT consumer to summarize, classify, or rewrite. The content enters OpenAI's systems. The data is potentially retained for training. It is potentially viewed by human contractors during quality review. It leaves the carrier's control entirely. The act is a §314.4 GLBA Safeguards Rule violation because the carrier did not maintain control over the NPI; it is a §164.502 HIPAA breach if PHI was included; it is a state-law breach trigger in jurisdictions where the data is linked to a state resident; and it is a §4 NAIC bulletin issue because the AI process produced an outcome (the summary, the rewrite) without satisfying the bulletin's third-party AI safeguards.

The single most consequential operating mistake an underwriter, adjuster, producer, or actuary can make in 2026 is pasting confidential insurance data into a consumer-grade LLM "just to try something." The breach is real, the consequences are real, and the file note is real - assuming anyone notices. Most carrier compliance programs in 2026 are running shadow-AI-detection scans to find this exact behavior; some are using Microsoft Purview or comparable DLP tools to block paste actions to consumer-AI URLs at the browser level. The mature posture is to assume the behavior happens, block it where possible, train against it continuously, and provide approved alternatives that take the friction out of compliance.

The Named Consequences

The consequences of pasting confidential data into a public LLM span multiple enforcement directions.

State DOI enforcement. The carrier's market-conduct exam can flag the breach as a §4.2 third-party AI failure under the NAIC bulletin and as a Safeguards Rule failure. State DOI consent orders typically include restitution, corrective action plans, and ongoing oversight. Recent state-level enforcement actions on AI-related data handling have produced fines in the $500K–$5M range plus consent-order remediation.

OCR HHS enforcement (HIPAA). If PHI was involved, OCR investigation is likely. Settlement range $2M–$15M for major breaches with 2–3 year corrective action plans, sometimes higher. OCR's Wall of Shame publication for 500+ individual breaches creates persistent reputational damage independent of the financial settlement.

FTC GLBA enforcement. The FTC enforces GLBA Safeguards Rule for non-bank financial institutions including insurance entities. FTC enforcement actions typically include consent orders with multi-year compliance monitoring, financial penalties (case-by-case), and required third-party audits.

State Attorney General enforcement. CCPA gives the California AG (and the California Privacy Protection Agency) enforcement authority with administrative fines up to $7,500 per intentional violation. The CCPA private right of action provides $100–$750 per consumer per incident statutory damages for breaches affecting certain data categories. Comparable enforcement under CPA (CO), VCDPA (VA), CTDPA (CT), UCPA (UT), TIPA (TX), and the rest of the state privacy patchwork.

Plaintiff-bar class actions. Under state UDAP (Unfair and Deceptive Acts and Practices) statutes, plaintiffs can drive class actions independent of regulator enforcement. Class settlements in the $5M–$50M range for major insurance breaches are documented through 2024–2026.

License-board exposure for the individual. Producer licenses (state-by-state), adjuster licenses (Texas under Tex. Ins. Code §4101, Florida under DFS Division of Agent & Agency Services, NY under DFS, California under CDI, Louisiana under LDI, plus the 30+ adjuster-licensing states), and professional designations (CPCU, AIC, CIC, AINS, ARM, AIAI, FCAS, ASA, FSA, MAAA) all have ethics provisions that an AI-driven data breach can violate. The individual practitioner who pasted faces personal license-board action and designation-suspension exposure that is separate from any carrier consequence.

The Carrier-Approved Alternatives - What Works

Every mature carrier in 2026 maintains a list of approved enterprise AI deployments. The list typically includes Microsoft Copilot for M365 with Enterprise terms, Azure OpenAI Service with data-residency and contractual no-training provisions, Google Vertex AI / Gemini for Enterprise, AWS Bedrock with HIPAA-eligible deployment and BAA, and on-premise deployments using open-weight models. The approval process for each follows a §4.2 third-party AI workflow plus a GLBA §314.4(f) vendor-oversight assessment plus, where PHI may be involved, a HIPAA §164.504(e) BAA execution.

Microsoft Copilot for M365 with Enterprise terms

Microsoft Copilot for M365 (the enterprise tier embedded in Microsoft 365 / Office 365 / Outlook / Teams under E3 / E5 / Business Premium licensing) operates under Microsoft Enterprise Agreement terms that include contractual no-training, data-residency where configured, encryption in transit and at rest, and integration with Microsoft Purview for data-loss prevention. The Enterprise terms align with GLBA Safeguards Rule §314 requirements and support HIPAA BAA execution. For most carrier general-purpose AI work - summarizing, drafting, rewriting - Microsoft Copilot for M365 with Enterprise terms is the default approved tool. The "consumer" version of Copilot (the free or personal-tier version accessed at copilot.microsoft.com without enterprise licensing) is not approved for confidential data.

Azure OpenAI Service with no-training and data-residency

Azure OpenAI Service provides programmatic access to OpenAI models (GPT-4, GPT-4o, GPT-5 tier where available) inside Microsoft Azure with contractual provisions for no-training (data submitted to Azure OpenAI is not used to train base models), data-residency (the carrier can configure which Azure region processes the data - US East, US West, EU, etc.), and HIPAA BAA eligibility for PHI workloads. For programmatic AI integration into carrier systems (Cytora, Federato, Akur8 integrations; Hyperscience document processing; in-house RAG systems over the claim manual or underwriting guide), Azure OpenAI is the standard enterprise-grade alternative.

Google Vertex AI / Gemini for Enterprise

Google Vertex AI provides comparable programmatic access to Gemini and Anthropic models with no-training contractual terms, data-residency configuration, and HIPAA BAA support. Carriers operating in Google Cloud environments use Vertex AI for the same programmatic integration patterns Azure OpenAI supports. Gemini for Enterprise is the Google-side equivalent of Microsoft Copilot for M365 - embedded AI in Google Workspace with enterprise-grade controls.

AWS Bedrock with HIPAA tier and BAA

AWS Bedrock provides programmatic access to multiple AI models (Anthropic Claude, Meta Llama, Mistral, Amazon Titan, Cohere) inside AWS with contractual no-training, data-residency, and HIPAA BAA execution. For carriers using AWS as their primary cloud, Bedrock is the equivalent enterprise alternative. Anthropic's Claude family on AWS Bedrock is widely used in 2026 for carrier RAG, document processing, and structured-output workflows.

On-prem and air-gapped deployments

For the most sensitive workloads - Bermuda Form coverage analysis on confidential litigation files, certain L&H workflows in privacy-strict jurisdictions, certain MGA-fronting bordereau where treaty data residency is contractually constrained - carriers deploy open-weight models (Llama 3.3, Mistral, certain Anthropic-licensed deployments) on owned infrastructure or air-gapped environments. The deployment removes the third-party-data-transfer risk entirely but adds operational complexity (model maintenance, infrastructure scaling, version management).

The BAA Chain for L&H Workflows

For L&H carriers operating as HIPAA covered entities, every AI vendor handling PHI signs a Business Associate Agreement. The chain extends through sub-processors: Munich Re's risk-assessment platform signs a BAA with the carrier and a BAA with Microsoft Azure as its compute provider; Azure has a BAA with Anthropic where Claude is the underlying model; Anthropic has internal controls. Each link must execute the BAA; a single gap creates HIPAA exposure for every entity in the chain. The Exhibit D AI Data response under the NAIC AI Systems Evaluation Tool asks for the BAA chain explicitly; carriers maintaining the chain in production-ready documentation pass §4.2 reviews; carriers without documented chains expose themselves at the next examination.

The No-Training Paragraph - Why It Matters

The single most important contractual provision in any insurance AI agreement is the "no-training" paragraph. The provision states that data submitted by the carrier to the AI vendor's service is not used to train the vendor's underlying base models. This matters because (a) carrier data fed to model training becomes embedded in the model's parameters and could be regurgitated to other customers in subsequent queries (a privacy and confidentiality breach), (b) carrier-specific information could become competitively exposed if it shapes vendor outputs to competitors, and (c) the carrier loses control over its own data lineage. Microsoft's Enterprise terms, Azure OpenAI Service terms, Google Vertex AI terms, AWS Bedrock terms, and Anthropic's enterprise contracts all include no-training provisions; consumer-tier products do not.

The contract paragraph carriers expect in 2026 typically reads (paraphrased across vendors): "Vendor shall not use Customer Data submitted to the Service to train, fine-tune, or improve any of Vendor's base models or any model offered to other customers. Customer Data shall be processed only for the purpose of providing the Service to Customer. Vendor's logging and monitoring activities shall be limited to security, abuse prevention, and operational metrics, and shall not include human review of Customer Data except as required for security incident response with prior notice to Customer where feasible." The exact wording varies; the substance is consistent across enterprise tiers.

The Data-Residency Clause

For state-specific compliance (NY DFS, Colorado, CCPA/CPRA) and for cross-border issues (carriers operating in Bermuda, Lloyd's of London, Canadian provinces, EU jurisdictions), the data-residency clause specifies where data is processed and stored. The clause typically pairs with the vendor's regional configuration - Microsoft Azure US East / US West / EU Central; AWS US East / EU West; Google Cloud US Central / Europe West. Carriers operating across multiple jurisdictions configure separate environments to maintain residency. The residency clause supports state-privacy-law compliance, treaty-data confidentiality, and the Exhibit D AI Data response on data lineage.

The Audit Rights Provision

The contract addendum includes audit rights - the carrier's right to audit the vendor's security and compliance controls on a defined cadence (typically annual onsite or remote audit, plus SOC 2 Type II attestation, ISO 27001 certification, or comparable security attestation). Audit rights operationalize the §314.4(f) GLBA vendor-oversight requirement and the NAIC bulletin §4.2 third-party AI requirement. Without contractual audit rights, the carrier cannot demonstrate ongoing oversight.

The File Note and the Shadow AI Problem

Every AI-touched insurance artifact in 2026 carries a file-note entry naming the AI involved, the prompt run (or its archival reference), the model version, and the practitioner's verification step. The discipline is the §4.4 documentation pillar applied at the artifact level. The file note is the audit trail when the artifact surfaces in a market-conduct exam, a discovery request, a bad-faith suit, or an OCR investigation.

The shadow AI problem is the gap between the file note and the actual practitioner behavior. Underwriters, adjusters, producers, and actuaries use consumer-grade AI tools (ChatGPT consumer, Claude.ai consumer, Gemini consumer) when the enterprise tools are slower, friction-ier, or less capable. The shadow use produces breach events that the file note system doesn't capture because the practitioner doesn't disclose the consumer-tool use. Carrier compliance programs in 2026 are investing heavily in (a) DLP tooling to detect and block paste actions to consumer-AI URLs, (b) browser-level monitoring to detect consumer-AI use, (c) regular training reinforcing the rule, (d) periodic surveys of practitioner AI use to identify shadow-AI patterns, and (e) easy-access enterprise alternatives that reduce the friction-gap motivating shadow use.

What This Means for the People on the Desk

For the underwriter: every submission summary, every appetite memo, every reason code, every variable-attribution narrative drafted with AI assistance uses an approved enterprise tool - Microsoft Copilot for M365 with Enterprise terms, Cytora or Federato workbench AI, Azure OpenAI in an in-house integration, never consumer-grade ChatGPT or Claude.ai. The SOV, the COPE narrative, the loss run, and the broker submission packet stay inside the approved environment.

For the adjuster: every FNOL summary, every coverage analysis, every reservation-of-rights letter, every SIU referral, every EUO outline drafted with AI assistance uses an approved tool. Recorded statement transcripts and medical records are particularly sensitive and require the BAA chain to be intact. The file note documents the AI involvement.

For the producer: every client communication, every renewal narrative, every carrier comparison drafted with AI assistance uses an approved tool. The agency-built chatbot must operate under approved enterprise terms or be removed. The producer's individual use of ChatGPT consumer to draft a client email containing a named insured's loss history is the most common shadow-AI breach scenario.

For the actuary: every model card, every SERFF filing memo, every reserve opinion drafted with AI assistance uses an approved tool. Model training data - typically containing insured-level NPI and sometimes PHI - never leaves the approved environment. The actuarial certification names the AI involvement and the data environment.

Key Takeaways

• Pasting an SSN, DOB, driver's license, recorded statement, named loss run, MIB hit, or medical record into a public LLM (ChatGPT consumer, Claude.ai consumer, Gemini consumer) is a GLBA Safeguards Rule event. For L&H or PHI handling, it's also a potential HIPAA breach under §164.402.
• The consequences span 5+ enforcement directions: State DOI consent order ($500K-$5M range), OCR HIPAA settlement ($2M-$15M for major breaches), FTC GLBA enforcement, state AG enforcement (CCPA fines $7,500/intentional violation; $100-$750 per consumer private right of action), plaintiff-bar class actions ($5M-$50M range), plus personal producer/adjuster license-board exposure under Tex. Ins. Code §4101, FL DFS, NY DFS, CDI, LDI, and professional designation suspension (CPCU, AIC, AIAI, FCAS, FSA).
• Carrier-approved alternatives are Microsoft Copilot for M365 (Enterprise terms), Azure OpenAI Service, Google Vertex AI / Gemini for Enterprise, AWS Bedrock with HIPAA tier, and on-prem deployments. Each provides contractual no-training, data-residency, and BAA where applicable.
• The four critical contractual provisions are: no-training paragraph, data-residency clause, audit rights, and BAA chain. Without all four, the vendor relationship is not §4.2 / §314.4(f) / §164.504(e) compliant.
• BAA chain integrity is enforced end-to-end. A single gap (Munich Re's BAA with Azure missing, or Azure's BAA with Anthropic missing) creates HIPAA exposure for every entity in the chain. The Exhibit D AI Data response asks for the chain explicitly.
• Every AI-touched insurance artifact carries a file-note entry naming the AI involved, the prompt run, the model version, and the verification step. The §4.4 documentation pillar applied at the artifact level. The file note is the audit trail when the artifact surfaces in market-conduct exam, discovery, or OCR investigation.
• Shadow AI is the operational gap. Practitioners use consumer-grade tools when enterprise tools are slower or friction-ier. Carrier compliance programs invest in DLP blocking, browser-level monitoring, training, surveys, and friction-reduced enterprise alternatives to close the gap.
• The single most consequential operating mistake in 2026 is "just trying" a consumer LLM with real insurance data. One paste of a recorded statement transcript, a loss run with named insureds, or an MIB hit can produce a multi-million-dollar breach exposure across 5+ enforcement directions plus personal license-board action.