Evaluating an insurance AI vendor in 2026 is a structured nine-dimension scoring exercise, not a vibes-based procurement. The carrier, MGA, or agency that buys without running the full evaluation absorbs operational risk that compounds across the contract term - vendor incidents the carrier did not anticipate, regulatory exposure the vendor disclaimed, exit costs the contract did not surface, and concentration risk the CRO would have flagged at signing. The nine dimensions: data security (SOC 2 Type 2, ISO 27001, penetration test posture), NAIC §4 third-party-AI conformance attestation, BAA and DPA for HIPAA and state privacy, model documentation depth (model cards, lineage, training data disclosure), fairness testing methodology and cadence, exit clauses (data export, model export, transition support), sub-processor disclosure and consent rights, model-update notification and approval rights, and audit rights with on-site or remote inspection authority. This lesson runs the nine-dimension eval on sixteen insurance AI vendors in 2026 - Guidewire, Duck Creek, Sapiens, Cytora, Federato, Akur8, Earnix, Shift, Tractable, CCC, Hi Marley, Five Sigma, Coalition, Send, Indico, Hyperscience - and produces the scoring template the procurement, security, legal, actuarial, and underwriting functions all sign before a contract reaches the CRO's desk.

The Nine-Dimension Evaluation Framework

Each dimension scores 1-5, with 1 being "non-compliant or undocumented," 3 being "industry-standard with documented evidence," and 5 being "best-practice with auditable trail." Total possible score 45. A vendor scoring below 30 is not ready for production deployment; a vendor scoring 30-37 requires contractual remediation before signing; a vendor scoring 38-45 is production-ready with normal operational oversight.

Dimension 1 - Data Security. SOC 2 Type 2 is table stakes; the question is what the audit revealed in the management letter, what remediations are in flight, and whether ISO 27001 certification has also been obtained. Penetration test cadence (quarterly internal, annual external by named third party), vulnerability disclosure program, and incident-history disclosure for the trailing 24 months. Score 5 requires SOC 2 Type 2, ISO 27001, quarterly internal pentest, annual external pentest by a recognized firm (Bishop Fox, NCC Group, IOActive), and zero material incidents in trailing 24 months.

Dimension 2 - NAIC §4 Third-Party AI Conformance. The NAIC Model Bulletin on AI was finalized in late 2023 and adopted by 30+ states by mid-2026. §4 governs governance of third-party AI vendors used by insurance carriers. Conformance attestation includes: documented model risk management program at the vendor, algorithm inventory the carrier can review, bias-testing methodology disclosure, drift-monitoring instrumentation, fairness-testing cadence, and incident-response coordination with the carrier. Score 5 requires the vendor to provide annual §4 conformance attestation signed by a vendor-side CRO or equivalent and to participate in the carrier's annual third-party AI audit.

Dimension 3 - BAA and DPA. Business Associate Agreement under HIPAA for any vendor touching PHI (life and health, workers' comp medical, BI claims medical records review). Data Processing Agreement under GLBA Safeguards Rule §314.4(f) and state privacy regimes (CCPA/CPRA, CPA, VCDPA, CTDPA, UCPA, TIPA). Score 5: vendor provides current BAA on request, DPA addresses every state-privacy regime in the carrier's write states, sub-processor list is current and notification rights are contractual.

Dimension 4 - Model Documentation. Model cards documenting purpose, training data, performance characteristics, known limitations, fairness testing, and intended use. Lineage from raw data through model training to model deployment. Training data disclosure to the level the carrier needs for Colorado Reg 10-1-1 explainability and AISET Exhibit C. Score 5: vendor produces model cards on every production model the carrier consumes, lineage instrumented end-to-end, training data disclosure sufficient for state DOI examination.

Dimension 5 - Fairness Testing. Methodology (disparate impact across protected classes, equalized odds, calibration parity), cadence (quarterly minimum, monthly preferred), reporting (carrier receives results), and remediation process (what happens when a fairness issue surfaces). Score 5: quarterly fairness testing across all protected classes the vendor's models touch, monthly drift monitoring on fairness metrics, contractual remediation timeline (typically 30-90 days), shared escalation to the carrier's AI committee.

Dimension 6 - Exit Clauses. Data export rights (the carrier's data in machine-readable format, complete, on reasonable timeline), model export rights (where applicable - typically the carrier-specific tuning or feature engineering, not the vendor's core IP), transition support (named hours and rates, transition period of 6-24 months), and post-contract data destruction with attestation. Score 5: all four elements contractual, exit-clause testing performed annually as a tabletop exercise.

Dimension 7 - Sub-Processor Disclosure. Current list of sub-processors (cloud infrastructure, third-party data sources, third-party model providers, support functions), notification rights for material changes, consent rights for high-risk additions (a new sub-processor in a non-US jurisdiction, a new third-party model provider). Score 5: list updated quarterly, notification 60 days in advance of material change, consent rights for high-risk additions, audit rights down the sub-processor chain for material vendors.

Dimension 8 - Model-Update Notification. When the vendor materially updates a model the carrier consumes - retraining, feature changes, performance characteristic shifts - the carrier needs notification, an evaluation window, and (for high-risk models) approval rights. Score 5: 30-90 day notification for material model updates, evaluation window with sample data, approval rights for models used in pricing, underwriting, or claims-affecting decisions.

Dimension 9 - Audit Rights. On-site or remote audit rights at carrier's reasonable request, frequency (annual minimum), scope (vendor controls, sub-processor controls, model governance, incident response), and dispute resolution if vendor refuses an audit. Score 5: contractual annual audit, expanded audit rights for incidents, sub-processor audit by reference, audit cost shared on a defined formula.

Vendor Eval Results - The Sixteen 2026 Incumbents

The following scores reflect typical 2026 mid-market specialty carrier evaluations and may vary based on specific contract negotiations and recent vendor changes. Scores are indicative, not authoritative - the carrier's own evaluation against current vendor disclosures is the only authoritative result.

Policy Administration and Platform Vendors

Guidewire (PolicyCenter, ClaimCenter, BillingCenter, Predict, Cyence). Composite typical score: 38-42. Strengths: SOC 2 Type 2, ISO 27001, mature §4 attestation, deep BAA/DPA framework, well-documented model cards on Predict and Cyence, contractual model-update notification, robust audit rights. Weaknesses: exit clauses can be heavy on transition costs; sub-processor disclosure improving but historically thin on third-party data partners.

Duck Creek. Composite typical: 36-40. Similar profile to Guidewire on core controls; the AI-specific dimensions (4, 5, 8) score slightly lower because Duck Creek's AI product depth is narrower than Guidewire's. Strengths: SOC 2 Type 2, BAA/DPA strong, audit rights contractual. Weaknesses: model documentation depth varies by product, fairness testing methodology less mature than Guidewire Predict.

Sapiens (IDIT, CoreSuite). Composite typical: 33-37. Strengths: international privacy regime coverage (GDPR depth helps US state privacy alignment), SOC 2 Type 2. Weaknesses: §4 conformance documentation thinner than US-native peers, model-update notification needs contract strengthening.

Underwriting Workbench and Triage Vendors

Cytora. Composite typical: 35-40. Strengths: dedicated insurance focus, SOC 2 Type 2, strong §4 attestation in 2025-2026, model cards on appetite and triage models. Weaknesses: sub-processor disclosure improving but still requires carrier-side push; fairness testing methodology developing on novel commercial-lines models.

Federato. Composite typical: 36-41. Strengths: workbench architecture limits direct model exposure (Federato orchestrates rather than originates many decisions), SOC 2 Type 2, NAIC §4 attestation, contractual audit rights, healthy exit clauses given the workbench's role. Weaknesses: model-update notification requires careful contracting because Federato's workflow updates can affect downstream decisions; sub-processor list includes multiple foundation-model providers requiring careful disclosure.

Pricing Vendors

Akur8. Composite typical: 37-42. Strengths: model documentation depth (Akur8's transparent-pricing architecture is differentiator), §4 attestation strong, fairness testing methodology established, model-update notification process documented. Weaknesses: exit clauses on carrier-specific model tuning require careful negotiation; data export rights on training-data-derived features need contractual specification.

Earnix. Composite typical: 35-40. Strengths: enterprise-grade controls (Earnix is the older incumbent with mature governance), SOC 2 Type 2, ISO 27001, strong audit rights. Weaknesses: model documentation depth historically less transparent than Akur8's; fairness testing methodology requires contractual specification for US carriers.

Claims Vendors - Estimating

Tractable. Composite typical: 36-41. Strengths: SOC 2 Type 2, ISO 27001, well-documented model cards on auto and property estimating models, strong fairness testing on auto estimating (gender and geographic disparate-impact testing), contractual model-update notification. Weaknesses: sub-processor list includes multiple image-processing infrastructure providers requiring careful disclosure; exit clauses on the carrier-specific estimate-tuning require negotiation.

CCC Intelligent Solutions. Composite typical: 35-39. Strengths: ecosystem maturity, SOC 2 Type 2, ISO 27001, deep BAA/DPA framework, audit rights well-established. Weaknesses: §4 conformance attestation evolving; model documentation depth varies by product; fairness testing methodology requires explicit contractual statement.

Claims Vendors - Workflow and Messaging

Hi Marley. Composite typical: 34-38. Strengths: SOC 2 Type 2, focused on claims messaging (narrower model surface than full claims workbench), BAA available. Weaknesses: §4 attestation maturing; sub-processor disclosure improving but requires carrier-side push; model-update notification needs explicit contractual language.

Five Sigma. Composite typical: 35-39. Strengths: comprehensive claims platform reduces vendor count, SOC 2 Type 2, evolving §4 attestation. Weaknesses: integration breadth means sub-processor list is large and requires careful review; model-update notification process is in development; audit rights need contractual strengthening.

Fraud and Cyber Vendors

Shift Technology. Composite typical: 37-41. Strengths: SOC 2 Type 2, ISO 27001, mature fairness testing on fraud models (carefully constructed to avoid disparate-impact concerns), strong §4 attestation, robust audit rights. Weaknesses: model documentation depth on novel fraud-detection techniques can be challenging because of IP sensitivity; sub-processor disclosure requires careful negotiation.

Coalition. Composite typical: 35-39 (note: Coalition operates as both carrier and tech provider, so the eval applies to its tech-provider role). Strengths: cyber-native posture, SOC 2 Type 2, ISO 27001, mature incident-response coordination. Weaknesses: §4 conformance is evolving given Coalition's hybrid carrier-tech model; data export rights on incident-response data require careful contractual specification.

Distribution and Data Vendors

Send. Composite typical: 33-37. Strengths: focused on surplus-lines and wholesale workflow, SOC 2 Type 2. Weaknesses: §4 attestation requires development for AI-driven triage features; sub-processor disclosure thinner than enterprise peers; model documentation depth requires contractual specification.

Indico Data. Composite typical: 35-39. Strengths: SOC 2 Type 2, ISO 27001, focused on IDP with relatively transparent model architecture, model documentation depth strong on document classification. Weaknesses: fairness testing methodology requires explicit contractual statement; model-update notification process needs strengthening.

Hyperscience. Composite typical: 36-40. Strengths: enterprise-grade SOC 2 Type 2, ISO 27001, FedRAMP authorization, mature audit rights, strong sub-processor disclosure. Weaknesses: model documentation depth varies by use case; fairness testing methodology less developed than insurance-native vendors (Hyperscience serves multiple industries).

How to Run the Eval - The Procurement Workflow

The nine-dimension evaluation is a cross-functional exercise. Procurement coordinates; security signs on Dimensions 1, 3, 7, 9; legal signs on Dimensions 3, 6, 7, 9; data science and actuarial sign on Dimensions 4, 5, 8; the business owner (CUO, CCO, chief actuary) signs on the overall fit-for-purpose. The CRO signs the final score and the contractual remediations.

The workflow takes 4-8 weeks for a new vendor. Week 1-2: vendor questionnaire distributed (the carrier's standardized form covering all nine dimensions). Week 3-4: vendor responses reviewed by each signer function; gaps identified; clarification requests sent. Week 5-6: vendor on-site or video meeting to walk through artifacts (SOC 2, §4 attestation, model cards, exit-clause language); remediation negotiation begins. Week 7-8: composite score finalized, remediations contractually drafted, CRO sign-off, contract executed.

Renewal evaluations run on a 2-4 week cycle because the artifacts are largely known and the focus is on year-over-year changes. The renewal eval surfaces: SOC 2 audit changes, §4 attestation updates, sub-processor changes, incident history in the trailing 12 months, model updates that occurred, fairness testing results, and any contract changes the vendor proposes.

MGA and Agency Edition

The framework applies with adjustments for MGAs and agencies. MGAs evaluate vendors on behalf of their capacity providers (the carrier whose paper they write); the capacity provider's vendor list, BAA/DPA framework, and §4 conformance flow through to the MGA. An MGA buying a vendor without checking against the capacity provider's vendor approval list creates compliance exposure for both parties. Agencies and brokerages evaluate vendors against their E&O posture and their carrier-relationship implications - a vendor handling PII at scale (Send, Coalition's distribution side, large agency-side platforms) needs to flow with the agency's data-broker registration where applicable (Texas DR 21-104 and similar state requirements).

For agencies under $100M revenue, the practical eval is lighter - Dimensions 1, 3, 6 are non-negotiable; Dimensions 4, 5, 8 can be lighter; Dimensions 2, 7, 9 depend on the vendor's exposure to insured data and decisions. For agencies above $100M revenue (where the 84% AI adoption rate sits), the full nine-dimension eval is the standard, run by a procurement function with named ownership.

The Vendor Concentration Question

Beyond per-vendor scoring, the portfolio-level question is concentration. A carrier with Cytora, Federato, and Akur8 all in production has three vendors in the underwriting-and-pricing flow; if Cytora goes offline, what happens to Federato's enrichment, and what happens to Akur8's pricing inputs? The concentration question requires mapping vendor dependencies and stress-testing failure scenarios. The CRO treats any vendor with more than 25% of a critical decision flow (intake, triage, pricing, estimating, fraud) as a concentration risk equivalent to a top-five reinsurer's. Annual operational due diligence, contractual exit-clause testing, and contingency-vendor identification are ongoing requirements.

Key Takeaways

• Nine-dimension evaluation: data security, NAIC §4 conformance, BAA/DPA, model documentation, fairness testing, exit clauses, sub-processor disclosure, model-update notification, audit rights. Each scored 1-5; total 45. Below 30 not production-ready; 30-37 contractual remediation; 38-45 production-ready.
• The 2026 sixteen-vendor eval covers Guidewire, Duck Creek, Sapiens, Cytora, Federato, Akur8, Earnix, Shift, Tractable, CCC, Hi Marley, Five Sigma, Coalition, Send, Indico, Hyperscience. Typical 2026 scores range 33-42; rankings shift with contract negotiations and recent vendor changes.
• SOC 2 Type 2 is table stakes, not a differentiator; the differentiator is what's in the management letter and what remediations are in flight. Penetration test cadence and incident history matter equally.
• NAIC §4 third-party AI conformance attestation must flow from vendor to carrier annually; without it, the carrier carries unsupported regulatory exposure. Vendor refusal to provide §4 attestation is a contractual deal-breaker.
• Exit clauses are the single most commonly under-negotiated dimension. Data export rights, model export rights (where applicable), named-hour-and-rate transition support, post-contract data destruction with attestation. Annual exit-clause tabletop testing surfaces gaps.
• Sub-processor disclosure and consent rights are operationally important because vendors increasingly use foundation-model providers (OpenAI, Anthropic, Google). A new sub-processor in a non-US jurisdiction or a new third-party model provider requires consent rights, not just notification.
• Cross-functional sign-off: procurement coordinates; security on 1/3/7/9; legal on 3/6/7/9; data science and actuarial on 4/5/8; business owner on fit-for-purpose; CRO on composite. 4-8 weeks for new vendor; 2-4 weeks for renewal.
• Vendor concentration is a portfolio-level CRO concern. Any vendor with more than 25% of a critical decision flow is concentration risk equivalent to a top-five reinsurer's. Annual operational due diligence, exit-clause testing, contingency-vendor identification ongoing.