The chief lending officer at a $4 billion regional bank sat across a conference table from a fintech vendor's sales team in March 2026, watching a demo that showed loan decisions rendered in under three seconds. The model's accuracy figures were impressive, the interface was clean, and the vendor's team talked fluently about "built-in explainability." When the CLO asked a single follow-up question, the room shifted. "If a regulator asks us to reconstruct why a specific applicant was denied on a specific date six months ago," she said, "what exactly will our adverse-action file contain, and which member of your team will sign an affidavit that those reasons accurately reflect the model's actual logic?" The vendor's lead rep paused, then said they could "export a reason-code report." The CLO set her pen down. She had been through a fair-lending exam two years earlier. She knew the difference between a report and a record, and between a reason-code export and a legally defensible explanation. The bank's procurement process now included twelve mandatory questions before any vendor demo began. This lesson is that procurement process, translated into the analytical framework every lending risk leader needs before signing a contract with an AI vendor. (The scenario above is a composite illustration reflecting patterns common across the industry; it does not describe a specific institution.)

Why Explainability Is a Procurement Requirement, Not a Feature

The most expensive misconception in lending AI procurement is treating explainability as a product feature on par with processing speed, integration options, or user interface design. Feature checklists invite vendors to check a box. Procurement requirements force vendors to answer specific, verifiable questions and accept contractual obligations tied to the answers.

The legal basis for treating explainability as a requirement is ECOA (the Equal Credit Opportunity Act, 15 U.S.C. 1691 et seq.) and its implementing regulation, Regulation B (Reg B, 12 CFR Part 1002, implemented by the Consumer Financial Protection Bureau). Under ECOA and Reg B, any creditor who denies an application for credit, changes the terms of existing credit unfavorably, or takes certain other adverse actions must provide the applicant with a written notice containing the specific reasons for the adverse action. The word "specific" matters: a reason like "model output did not meet threshold" is not specific. A reason like "your debt-to-income ratio of 47 percent exceeded our maximum of 43 percent" is specific. The institution that signs the adverse-action notice is legally responsible for the accuracy of the stated reasons. That responsibility does not transfer to the vendor who built the model.

OCC Bulletin 2026-13, the April 2026 interagency model-risk guidance issued jointly by the OCC (Office of the Comptroller of the Currency), the Federal Reserve, and the FDIC, reinforces this at the governance level. Under 2026-13, any AI system used in credit decisioning is a model subject to the institution's model risk management program. The institution must be able to document the model's inputs, outputs, validation status, and fair-lending testing results. A vendor contract that provides a model with opaque logic and a performance certification but no validation access does not satisfy the institution's model-risk obligations. The examiner who asks for the model-risk file will ask to see the institution's independent validation, not the vendor's self-certification.

Explainability in this context has three distinct dimensions that a procurement process must address separately. The first is individual-decision explainability: for any specific application, the institution must be able to reconstruct what the model used as inputs, what it produced as output, and what the file-grounded reasons for an adverse decision are. The second is population-level explainability: the institution must be able to analyze the model's outcomes across applicant groups and demonstrate that the model does not produce disparate impact on protected classes under ECOA, the Fair Housing Act, or applicable state law. The third is institutional explainability: in a regulatory examination, model-risk review, or litigation, the institution must be able to explain how the model works, what its limitations are, and how those limitations were addressed in the governance program. A vendor who satisfies one dimension but not the others is not providing a compliant product, regardless of what the demo slide deck says.

The Twelve Questions Every Vendor Must Answer

The procurement framework below is organized around twelve questions in four categories. Each question addresses a specific compliance or governance requirement. The right answer to each question is not always a clean "yes"; sometimes the right answer is a detailed explanation of limitations with a credible remediation plan. What disqualifies a vendor is not complexity or limitation but opacity: an inability or unwillingness to answer the question at all.

Category One: Individual-Decision Explanation

Question 1: For any specific denied application, what will our adverse-action file contain, and how was that content generated?

This is the most fundamental question in the evaluation. Ask the vendor to show you the exact output format of the adverse-action reasons the model produces, walk you through how those reasons are generated (from model weights, from SHAP values, from a separate rules engine, from a large language model), and demonstrate with a sample file what a complete adverse-action record looks like. The answer should specify: the reason codes produced, the data fields that drove each reason, the method used to connect model output to reasons, and the traceability path from the stated reason back to the applicant's file data.

A vendor who says "we provide an adverse-action reason code output" without being able to explain the generation method has given you an answer that is technically true and practically insufficient. Adverse-action reason codes generated by a post-hoc explanation method (such as SHAP, which stands for SHapley Additive exPlanations, a technique that assigns each input feature a contribution value to the model's output) can be accurate. Reason codes generated by a separate rules engine layered on top of a black-box model may not reflect the model's actual logic. Reason codes generated by a large language model that summarizes the file without reference to the actual score drivers may be fabrications. Ask how the reasons are generated. Verify that the method is auditable.

Question 2: How does the vendor handle cases where the model's explanation conflicts with the model's actual decision?

Post-hoc explanation methods are imperfect by design. SHAP values are an approximation of feature contributions, not a precise audit trail. There are documented cases where SHAP-based reason codes are inconsistent with what the model would have done if a specific feature had changed. Ask the vendor: what testing has been done to verify that the stated adverse-action reasons actually reflect the model's decision logic? Is there a process for identifying inconsistencies between the explanation and the decision? What happens when an inconsistency is found? The institution is responsible for the accuracy of adverse-action reasons under Reg B. A vendor who cannot describe how they validate the accuracy of their explanation output has left the accuracy risk with the institution and the explanation risk with the vendor.

Question 3: What is the retention period and format for adverse-action documentation, and how is it queryable?

ECOA and Reg B require institutions to retain records for specific periods (generally 25 months for mortgage applications). Ask the vendor: in what format are the model's input data, output scores, and adverse-action reasons retained? Who has access to that retained data? Can it be queried by application ID, date range, model version, and protected-class indicator? Can it be exported for an examiner? A vendor who retains raw output logs but cannot produce queryable records organized by application has provided storage, not governance. The distinction matters enormously when an exam sample arrives.

Category Two: Fair-Lending and Disparate Impact

Disparate impact refers to a neutral policy or practice that produces a statistically significant difference in outcomes across protected classes (such as race, sex, national origin, or religion under ECOA), regardless of intent. A model that uses no protected-class variables directly can still produce disparate impact through proxy variables (variables that correlate strongly with protected-class membership, such as ZIP code, which correlates with race in many markets). Testing for and mitigating disparate impact is a core fair-lending obligation.

Question 4: What testing for disparate impact has the vendor conducted, on what data, using what methodology, and what were the results?

This question distinguishes vendors who have actually done the work from vendors who have produced a slide deck about their commitment to fairness. The right answer provides: the demographic datasets used for testing (which protected classes, what data sources, what sample size), the statistical methodology (comparative file review, regression analysis, matched-pair testing), the specific outcome variables tested (approval rate, pricing, counter-offer rate), and the actual results including any disparities identified and the remediation taken. A vendor who provides a fairness certification without an underlying methodology, or who describes testing performed only on their own proprietary benchmark dataset, has not provided adequate evidence for a regulated institution's fair-lending risk assessment.

Question 5: Has the vendor conducted or documented a less-discriminatory alternative search?

The less-discriminatory alternative (LDA) search is the documented process of determining whether a model configuration that produces disparate impact can be replaced with an alternative that achieves a comparable business objective with less disparate impact. The LDA concept comes from civil rights law and has been integrated into banking fair-lending enforcement: when a regulator identifies disparate impact, the institution's defense includes demonstrating that it searched for a less-discriminatory alternative and either found one (in which case it should have adopted it) or documented why none was viable. A vendor who has done this work will be able to describe it. A vendor who has not done this work is leaving the LDA obligation entirely to the institution, which must then conduct the search itself on a model whose internals it may not be able to access.

Question 6: What ongoing fair-lending monitoring does the vendor provide, and who is responsible for acting on monitoring results?

A model that passes its pre-deployment disparate-impact test can develop disparate impact over time as the application population shifts, economic conditions change, or the model is retrained on new data. Under OCC 2026-13, the institution must monitor its AI models for fair-lending risk on an ongoing basis. The question for a vendor is: what does ongoing monitoring look like in this product? Do they provide periodic disparate-impact reports? Do they flag model drift that could affect fair-lending outcomes? Who is contractually obligated to take action when a monitoring result shows a concern, and what does "taking action" mean in the contract terms? The vendor can provide monitoring data. The obligation to respond to that data and to investigate and remediate disparities belongs to the institution. The contract must make this allocation explicit.

Category Three: Model Risk and Third-Party Governance

Question 7: What documentation does the vendor provide to support the institution's independent model validation?

OCC 2026-13 requires that AI models used in credit decisioning be independently validated. For a vendor model, independent validation means the institution (or a third-party validator it retains) must test the model for accuracy, stability, and fitness for the institution's specific use case. The vendor's documentation must be sufficient to support this testing. Ask specifically: does the vendor provide the model architecture description (even at a high level), the training data characteristics (industries, time periods, geographic distribution), the feature list with definitions, the performance metrics from internal testing, and the known limitations? A vendor who provides only an API and a performance certification cannot be independently validated to the standard OCC 2026-13 requires. Some vendors provide a detailed validation support package; others argue their architecture is proprietary. Evaluate the actual package, not the vendor's characterization of it.

Question 8: How will the vendor notify the institution of material model changes, and what is the process for reassessing the institution's model-risk documentation after a change?

Under OCC 2026-13, the institution must reassess its model-risk documentation following any material change to a vendor model. A material change includes retraining on new data, changes to the feature set, changes to the scoring logic, and changes to the explanation methodology. Ask the vendor: what constitutes a material change in your product terms? What notification will you provide and on what timeline? What documentation will you provide to support the institution's reassessment? A vendor who reserves the right to retrain or update the model without notifying the institution has created a governance gap that the institution cannot close without contract remediation. This is one of the most common and consequential gaps in existing AI vendor contracts.

Question 9: What are the vendor's data security, privacy, and data-use provisions for applicant data processed through the model?

Credit application data includes MNPI (material non-public information), income data, and information subject to GLBA (the Gramm-Leach-Bliley Act, which governs the privacy and security of consumer financial information). When an institution sends application data to a vendor's API for scoring, the institution retains responsibility for the security and appropriate use of that data. Ask the vendor: is applicant data retained after scoring, and if so for how long and for what purpose? Is applicant data used to retrain the model? What subprocessors have access to the data? What data breach notification obligations does the vendor accept? What certifications (SOC 2 Type II, ISO 27001) cover the scoring environment? These are not negotiable points; they are GLBA compliance requirements for the institution, and the vendor contract must reflect adequate protections.

Category Four: Institutional Explainability and Examiner Readiness

Question 10: Can the vendor produce a model-risk file that satisfies OCC 2026-13 requirements, or does the institution need to build that file from vendor-supplied documentation?

A model-risk file under OCC 2026-13 requires development documentation, independent validation, fair-lending testing results, ongoing monitoring records, and change documentation. A vendor who provides all of these in a format the institution can adopt is providing significant governance support. A vendor who provides a performance certification and expects the institution to construct the rest of the file from limited disclosures is creating a significant governance burden and risk for the institution. Ask the vendor to walk you through what they provide and what the institution must produce independently. Budget the staff time to build what the vendor does not provide before signing.

Question 11: Has the vendor's model been used by another regulated institution that has been through a model-risk or fair-lending examination while using this product? What were the results?

Reference checks for AI lending vendors should include specific examination history. A vendor whose product has been in production at a regulated institution through a completed model-risk or fair-lending exam, with no findings related to the vendor model, has demonstrated something a demo cannot: that the governance package works under regulatory scrutiny. Ask for references at institutions of comparable size and complexity, with permission to ask specifically about examination outcomes. A vendor who cannot provide examination-tested references is either new to the regulated market or has examination history they do not want to disclose. Both conditions are relevant to the procurement decision.

Question 12: What is the vendor's contractual commitment to producing a witness or documentation if the institution faces litigation, a regulatory action, or an examination finding related to this model?

When an institution faces a fair-lending investigation or adverse-action litigation, the vendor's cooperation in explaining how the model works is not a matter of goodwill; it is often a matter of whether the institution can mount a defense. A vendor contract that has no provision for vendor cooperation in regulatory examinations or litigation, or that contains broad indemnification in the vendor's favor, has left the institution holding governance obligations it cannot fulfill without the vendor's cooperation. This is a legal and procurement matter, but the risk officer and CLO should understand what the contract says before it is signed. The phrase "vendor accountability does not transfer" in OCC 2026-13 guidance means that the institution cannot walk into an exam and say the vendor is responsible. It does not mean the institution does not need the vendor's cooperation to explain the model.

The Explainability Spectrum: What Good Looks Like at Each Level

Not all lending AI products are designed with the same explainability architecture. Understanding the spectrum of approaches helps procurement teams evaluate vendor claims against actual capabilities rather than against a generic "explainability" label.

At the highest explainability level are intrinsically interpretable models: logistic regression, decision trees, and scorecard models (point-based linear models where each variable's contribution to the score is fixed and documented). These models are transparent by design: for any applicant, you can trace the exact path from input values to output score by reading the model's coefficients or rules. They were the industry standard in credit scoring for decades. Their limitation is that they may capture less predictive signal than more complex models, though the evidence on this trade-off in credit scoring is more nuanced than vendor marketing suggests. For an institution that prioritizes examination-ready explainability above all else, an intrinsically interpretable model is the lowest-risk choice.

At the next level are gradient-boosted ensemble models (such as XGBoost and LightGBM, machine learning methods that combine many decision trees to make predictions) with post-hoc explanation tools like SHAP. These models are not intrinsically interpretable: their decisions emerge from the interaction of many trees trained on many features, and there is no clean coefficient table to read. SHAP provides an approximation of each feature's contribution to any given prediction, but this approximation has documented limitations: it can be inconsistent across similar cases, it can be gamed by adversarial inputs, and it does not precisely simulate what would have happened if a feature value had changed. For a regulated institution, a SHAP-based explanation method is acceptable if the institution has validated the accuracy of the explanations against a sample of actual decisions and has documented the validation. It is not acceptable if the institution is simply relying on the vendor's assertion that the explanations are accurate.

At the lowest explainability level for credit-decisioning purposes are large language models (LLMs) and other foundation models used directly to generate credit recommendations or adverse-action reasons. These models produce fluent, persuasive text. They can sound like they are citing the applicant's file when they are actually pattern-matching on training data. Their outputs can be accurate and useful as drafts for human review, but as the direct source of adverse-action reasons without human verification, they are among the highest-risk tools in the lending technology landscape. The vendor who says their LLM "explains the decision" without specifying what grounding mechanism connects the LLM's output to the actual model's decision logic is describing a high-risk architecture that requires intensive human review before any output is used in a credit record.

Between these levels are hybrid architectures: a primary scoring model combined with a rules engine that generates adverse-action reason codes, or a scoring model combined with a RAG (Retrieval-Augmented Generation, the technique of grounding a generative AI output in retrieved documents or data at generation time) system that generates reason text from the actual LOS file data. These hybrid approaches can achieve both predictive sophistication and auditable explanations, but they require understanding two components rather than one. Procurement evaluation should assess both: the scoring model's explainability and the explanation generator's accuracy.

Running the Explainability Audit Before Contract Signing

A procurement evaluation that relies only on vendor presentations has a fundamental limitation: the vendor controls the information flow. An explainability audit during the procurement process gives the institution independent evidence about the vendor's claims.

The explainability audit for a lending AI vendor has four components that can be completed before contract signing, using a pilot dataset or vendor-provided sample data.

Component one: adverse-action reason accuracy test. Take a sample of 25 to 50 declined applications from the vendor's pilot or demo environment. For each declined file, review the vendor-provided adverse-action reasons alongside the actual application data. Verify independently: does each stated reason accurately describe a feature of the actual application that crosses a documented threshold? Would a credit analyst, reading the reason without knowing the model's output, agree that this is a specific, accurate description of why this applicant was declined? A reason code like "credit utilization ratio exceeded threshold" is verifiable: you can check the application's credit report data and confirm the ratio. A reason code like "risk profile inconsistent with credit standards" is not specific and is not Reg B compliant. Document the accuracy rate across the sample.

Component two: consistency test. Using the vendor's explanation methodology (SHAP, rule-based, or other), construct two applicant profiles that differ by one variable and see if the explanation correctly identifies that variable as the difference. If applicant A has a DTI (debt-to-income ratio) of 46 percent and is declined, and applicant B is identical except for a DTI of 39 percent and is approved, the explanation for A's decline should cite the DTI as a primary driver. If it cites other features instead, the explanation methodology is inconsistent in a way that matters for regulatory accuracy.

Component three: protected-class disparate impact pre-screen. Before deploying any lending AI model, the institution should run a disparate-impact analysis on a representative dataset. If the vendor provides a pilot environment, use your institution's own application data (with applicant identity protected) to test the model's approval and adverse-action rates across demographic groups. The four-fifths rule (also called the 80-percent rule, a statistical benchmark from employment discrimination law that the CFPB has applied in fair-lending contexts: if a protected class's approval rate is less than 80 percent of the highest approval rate among any group, a disparity meriting investigation exists) is a useful first screen, though it is not the only relevant test and does not substitute for a more rigorous statistical analysis. The point of the pre-screen is to identify potential disparate-impact concerns before you sign a multi-year contract.

Component four: model-risk documentation review. Before contract signing, request the vendor's full model documentation package and have your model-risk or internal audit team review it against OCC 2026-13 requirements. Specifically: is there an independent validation report? Does it address disparate impact? Does it describe the model's known limitations? Does the vendor's documentation contain enough information for the institution to conduct its own independent validation, or does it only contain enough information to rely on the vendor's validation? The difference between these two documentation packages is the difference between a model-risk defensible deployment and one that requires substantial institution-side governance build before deployment.

Contractual Protections That Explainability Requires

A vendor who performs well on the twelve questions above and the explainability audit will still need to have those commitments reflected in the contract. Verbal assurances from a sales team do not survive personnel turnover, acquisitions, or product pivots. The following contractual provisions are minimum requirements for a lending AI contract at a regulated institution.

Material change notification. The vendor must provide written notice of any material change to the model (retraining, feature changes, scoring logic changes, explanation methodology changes) at least 30 days before implementation, or immediately following an emergency change with written notice. The institution must have the right to require documentation supporting a re-validation assessment following any material change.

Examination cooperation. The vendor must agree to cooperate with regulatory examinations involving the model, including providing documentation, technical explanations, and in-person or written responses to examiner questions. The scope of this cooperation and its cost (many vendors charge for examination support services) should be defined in the contract, not left to post-signature negotiation.

Fair-lending testing obligations. The vendor must conduct periodic fair-lending testing and provide the results to the institution on an agreed schedule (at minimum annually, more frequently for high-volume models). The contract should specify the methodology, the demographic groups tested, and the format of the results. The institution's obligation to respond to adverse results should also be defined: who is responsible for investigating a disparity finding, and what remediation options does the institution have, including the right to terminate the contract if a disparity cannot be remediated?

Data return and deletion. On contract termination, the vendor must return or confirm destruction of all applicant data, in a format that allows the institution to meet its Reg B record-retention requirements. An institution that cannot retrieve its adverse-action records after a vendor relationship ends has a compliance gap that begins the day the relationship terminates.

Liability allocation. The contract should be clear about who bears the cost of regulatory findings, customer remediation, and litigation arising from the model's outputs. Many vendor contracts are written to allocate all such costs to the institution. This may be acceptable if the institution fully controls the model's deployment and use. It is not acceptable if the vendor controls material aspects of the model's behavior, explanation output, or monitoring results. A one-sided indemnification in the vendor's favor, in a contract where the vendor controls the model's logic, is a risk-allocation problem the chief risk officer and general counsel should address before signing.

Red Flags That Disqualify a Vendor Regardless of Demo Quality

Some vendor characteristics are disqualifying regardless of how impressive the product demonstration is. These red flags indicate a vendor that is either unaware of the regulated environment they are selling into or aware of it and choosing to avoid accountability for it.

The first red flag is inability to name the explanation methodology. A vendor who cannot describe, specifically and technically, how their adverse-action reasons are generated does not have an explainability product. They have an explanation-shaped feature that someone added to the output without governance.

The second red flag is resistance to independent validation. A vendor who characterizes independent validation as unnecessary, who says their internal testing satisfies regulatory requirements, or who makes access to documentation conditional on contract signing is not compatible with a regulated institution's model-risk program. OCC 2026-13 requires independent validation. A vendor who objects to it is objecting to a regulatory requirement, not to the institution's pickiness.

The third red flag is the absence of fair-lending testing documentation. A vendor who says they have tested for fairness without being able to provide a methodology description, the demographic groups tested, the outcomes measured, and the actual results has not done the work. Fairness statements without data are marketing, not governance.

The fourth red flag is a contract that contains no material change notification clause. A vendor who is unwilling to commit contractually to notifying the institution of model changes is a vendor who plans to make changes without notifying the institution. Under OCC 2026-13, the institution is responsible for its model-risk documentation following any material model change. A vendor who will not provide notification has made it impossible for the institution to meet this obligation.

The fifth red flag is a claim that the vendor's product eliminates the need for the institution's model-risk program. No vendor product eliminates the institution's model-risk obligations. OCC 2026-13's framework places those obligations on the institution, not the vendor, and they cannot be delegated by contract. A vendor who suggests otherwise either does not understand the regulatory environment or is making a claim they cannot support in an examination. In either case, the institution is better positioned by understanding this limitation before procurement than after.

Key Takeaways

• Explainability in lending AI is a regulatory requirement, not a product feature: ECOA and Reg B require specific, accurate adverse-action reasons for every credit denial, and the institution that signs the notice is legally responsible for the accuracy of those reasons regardless of which vendor built the model.
• The twelve procurement questions described in this lesson address four categories: individual-decision explanation (how reasons are generated and retained), fair-lending and disparate impact (what testing was done and what the results were), model risk and third-party governance (what documentation supports independent validation and what notification rights the institution has), and institutional explainability (what the model-risk file contains and whether the vendor has been through an examination).
• OCC Bulletin 2026-13 requires the institution to independently validate vendor AI models, retain model-risk documentation, and reassess that documentation after material model changes. A vendor who will not support this process is not compatible with the institution's regulatory obligations.
• The explainability spectrum runs from intrinsically interpretable models (logistic regression, scorecards) through SHAP-explained ensemble models to LLM-based explanations. Each level requires a different validation approach, and the institution must understand which architecture the vendor is using before evaluating the vendor's explainability claims.
• The pre-contract explainability audit (adverse-action reason accuracy test, consistency test, disparate-impact pre-screen, and model-risk documentation review) gives the institution independent evidence about the vendor's claims before signing a multi-year contract.
• Contractual protections for explainability include material change notification, examination cooperation clauses, fair-lending testing obligations, data return and deletion provisions, and liability allocation terms. Verbal assurances from a sales team are not substitutes for enforceable contract terms.
• Five red flags disqualify a vendor regardless of demo quality: inability to name the explanation methodology, resistance to independent validation, absence of fair-lending testing documentation, no material change notification clause, and claims that the vendor's product eliminates the institution's model-risk program.
• The goal of the procurement process is not to find a vendor who checks an explainability box but to find a vendor whose product, governance support, and contractual commitments allow the institution to meet its obligations under ECOA, Reg B, and OCC Bulletin 2026-13 with a defensible record an examiner can read.