The paystub lands in the queue at 9:14 on a Wednesday morning. It belongs to a thirty-one-year-old registered nurse applying for a $320,000 purchase mortgage. The processor opens the PDF, which was scanned on a home scanner at a slight angle, and the image quality is just good enough to frustrate manual keying: the numbers are there, but the font is small, the column headers are cut off at the right margin, and there are two separate pay periods on a single page because the employer switched payroll cycles mid-year. Under the old workflow, this file would sit in a queue until a processor had forty-five minutes to key every field by hand, verify the math, and flag the mid-year cycle change for the underwriter. Under an AI-assisted document extraction workflow, a model reads the PDF, identifies every labeled field, constructs a structured output, and hands the processor a draft in under ninety seconds. The job of the processor is no longer to key the numbers. The job is to verify them, specifically, against the source lines of the document, before any figure touches the loan origination system (LOS, the software platform that stores and routes every field from application to close). That shift, from keying to verifying, is what this lesson is about.

What Document Extraction AI Actually Does

Document extraction is a specific category of AI, distinct from the scoring models that assign credit risk and distinct from the generative models that draft narrative prose. An extraction model uses a combination of optical character recognition (OCR, the technology that converts scanned images into machine-readable text) and layout analysis to identify the position, label, and value of every field on a financial document. When it encounters a paystub, it is not making up income figures from training data about typical paystubs. It is reading this document: this employer name, this pay period, this gross pay line, this year-to-date figure. That distinction matters, because the failure modes of extraction AI are different from the failure modes of generative AI, and conflating them leads to the wrong verification strategy.

A generative model can invent a number that never appeared anywhere in the file because it fills gaps by predicting what a plausible completion looks like. An extraction model produces errors in a different pattern: it misreads a character, assigns a value to the wrong field, confuses two adjacent columns, or fails to recognize a field because the layout deviates from its training data. The extraction error looks like a transposition (a 6 read as a 0, producing $86,420 from a source line that reads $86,470), a column shift (gross pay read from the year-to-date column), or a missed field (a second employer on page two that the model did not detect because the header was partially obscured).

Understanding this distinction shapes the verification workflow. You are not checking whether the AI invented a plausible income. You are checking whether the AI correctly read the specific characters and field assignments on this specific document. The verification is a cross-reference, not a plausibility check. Does the extracted gross monthly income of $7,215 match the gross monthly figure on the paystub? Where on the page is that line? Which pay period does it represent?

The documents you encounter most often in a lending context, and the specific extraction challenges each one presents, are worth knowing individually before you build any verification workflow around them.

Paystubs present layout variability as the primary extraction challenge. Every employer uses a different payroll processor, and every payroll processor has a slightly different paystub format. Field labels are not standardized: "Gross Pay" on one paystub may appear as "Total Earnings," "Regular Pay," or an unlabeled first row on another. Year-to-date figures appear in different columns, sometimes before the current-period figures and sometimes after. When a borrower switches employers or pay cycles mid-year, the year-to-date figure on the later stub reflects only the time at the current employer, and an extraction model that does not recognize the cycle change may pass a partial year-to-date figure as if it were a full-year figure.

W-2 forms are more standardized because the IRS (Internal Revenue Service) prescribes the layout, but scanned copies introduce rotation, skew, and contrast issues that degrade OCR accuracy on the small box-and-number grids. The specific boxes that matter most in lending, Box 1 (wages, tips, and other compensation) and Box 5 (Medicare wages), are both in the upper portion of the form, and partial scans or double-sided print bleed can cut off the top rows. Employer names and EINs (Employer Identification Numbers, nine-digit federal tax identifiers) are frequent extraction errors because they contain alphanumeric strings with no standard length or format.

1040 federal income tax returns present the highest extraction complexity of the three primary income documents, because the relevant income lines depend on the borrower's filing situation. A salaried borrower's income is on line 1a. A self-employed borrower's income runs through Schedule C (business profit and loss), Schedule E (supplemental income and loss), and Schedule F (farm income), each of which may carry positive or negative amounts that the lender calculates into a qualifying income figure using a formula that differs by loan type. An extraction model that reads only line 1a of a return with Schedule C income will produce a completely wrong qualifying income figure without generating any error flag, because the extracted value is a real number on the page, correctly read, just not the right number for the calculation.

Bank statements are the least structured of the four primary asset and income documents, because there is no standardized format across institutions. Deposit patterns, balance figures, large deposits that require explanation, and recurring direct-deposit amounts all need to be identified from transaction narratives that are institution-specific. An extraction model reading a credit union's statement encounters different column headers, different narrative formats, and a different pagination structure than it encounters reading a large bank's statement. Asset and reserve verification from bank statements requires not just extracting the end-of-period balance but identifying and flagging large or irregular deposits that guidelines require the borrower to explain.

Writing Extraction Prompts That Produce Verifiable Output

The quality of an AI extraction depends substantially on the quality of the instruction. A prompt that asks the model to "extract the income information" will produce an output organized around the model's general understanding of income, which may or may not match the fields your LOS requires and may omit fields the model did not identify as income-related. A well-constructed extraction prompt specifies exactly which fields to extract, what to do when a field is not clearly visible, and how to structure the output so that every extracted value is traceable to its source location.

Here is an example extraction prompt for a paystub, written for a processor who will verify the output before entering it into the LOS:

You are processing a scanned paystub for a mortgage application. Extract only the following fields, reading directly from the document: (1) Borrower name as printed on the stub; (2) Employer name as printed on the stub; (3) Pay period start date; (4) Pay period end date; (5) Pay frequency (weekly, biweekly, semimonthly, or monthly); (6) Current period gross pay; (7) Year-to-date gross pay; (8) Current period base pay (if listed separately from gross); (9) Year-to-date base pay (if listed separately). For each field, provide the extracted value and the line label exactly as it appears on the document. If a field is not clearly visible or is not present, write "Not visible" rather than estimating. Do not calculate annualized income. Do not carry forward information from context. Read only this document.

Notice what this prompt does and does not ask for. It requests specific labeled fields, not a general summary. It requires the model to reproduce the source line label alongside each extracted value, which is the information the verifier needs to confirm the extraction is correct. It explicitly prohibits estimation for missing fields and prohibits annualized income calculation, both of which are sources of hallucinated values in extraction workflows. It constrains the model to the document, not to prior knowledge about typical paystubs.

The structured output from this prompt, when the model executes it well, looks like this:

Borrower Name: Elena Vargas [from "Employee Name" field]
Employer Name: Lakewood Regional Medical Center [from "Employer" field]
Pay Period Start: 10/01/2025 [from "Period Begin" field]
Pay Period End: 10/15/2025 [from "Period End" field]
Pay Frequency: Semimonthly [from "Pay Type" field]
Current Period Gross Pay: $3,607.50 [from "Gross Pay" field, current column]
Year-to-Date Gross Pay: $72,150.00 [from "Gross Pay" field, YTD column]
Current Period Base Pay: $3,607.50 [from "Regular Earnings" field, current column]
Year-to-Date Base Pay: $72,150.00 [from "Regular Earnings" field, YTD column]

The field label in brackets is the verification anchor. When the processor opens the paystub PDF and confirms that the "Gross Pay" field in the current column reads $3,607.50, the extraction is verified. The label tells the processor exactly where to look. An extraction output without source labels requires the verifier to search the whole document, which is slower, less reliable, and more likely to miss a mismatch.

For a W-2, the extraction prompt changes to reflect the standardized box structure:

You are processing a scanned W-2 for a mortgage application. Extract only the following boxes, reading directly from the document: Box 1 (Wages, tips, other compensation); Box 2 (Federal income tax withheld); Box 5 (Medicare wages and tips); Box 12 (if coded with C, D, E, or W, extract both the code and the dollar amount); Box 14 (extract any entry with its label); Employer name and address; Employer EIN; Employee name and address; Employee SSN (last four digits only). For each field, provide the extracted value and the exact box number or label. Write "Not present" if the box is blank. Do not interpret or calculate. Do not reconstruct information from other boxes.

The W-2 prompt adds a specific instruction about Box 12 codes, because those codes affect qualifying income calculations under many loan guidelines, and the instruction to extract the code alongside the dollar amount ensures the verifier has the information to look up the treatment. The SSN instruction limits extraction to the last four digits, a data minimization practice consistent with privacy standards.

The Cross-Check Workflow: Tying Every Number to Its Source Line

Extraction without verification is data entry with extra steps. The production gain from AI-assisted extraction only materializes when the verification step is designed to be fast, specific, and systematically logged. A processor who spends twelve minutes manually keying a paystub and a processor who spends four minutes verifying an AI extraction both need the same end result: every field in the LOS backed by a source line in the document. The difference is eighteen minutes per file at volume, which over a month of closings is the difference between clearing fifty files and clearing eighty.

The cross-check workflow has four stages, each of which the processor completes before any extracted value enters the LOS.

Stage one: field count. Count the number of fields the extraction returned and confirm it matches the number of fields your LOS requires for this document type. If the extraction returned eight fields and the LOS requires ten, two fields are missing. Identify which fields are missing before you move on. Missing fields should not be filled in by estimation; they should be flagged for a follow-up request to the borrower or for manual identification from the document.

Stage two: source line verification. For each extracted value, locate the corresponding source line in the document using the label the extraction output provided. Confirm the number exactly. Do not round. Do not approximate. If the extracted value is $3,607.50 and the source line reads $3,607.50, the extraction is verified. If the source line reads $3,670.50, there is a transposition error. The difference is $63 per pay period, which over a semimonthly pay schedule compounds to $1,512 in annualized income. On a $320,000 mortgage application, that difference can affect debt-to-income calculations and qualifying income by enough to change the decision.

Stage three: field assignment verification. Confirm that the model assigned each value to the correct field. The most common field assignment error in paystub extraction is the confusion between current-period figures and year-to-date figures when both appear on the same line in adjacent columns. A model that reads a paystub in a format where current-period amounts are in the left column and year-to-date amounts are in the right column may transpose the column assignment if the column headers are faint, cut off, or formatted in a way the model does not recognize. Check specifically: is the number in the "Year-to-Date Gross Pay" field the larger number (as it should be, covering more pay periods) or the smaller number (suggesting a column swap)?

Stage four: internal consistency check. Once all fields are verified against their source lines, run a quick internal consistency check. For a paystub with confirmed pay frequency, multiply the current-period gross pay by the number of pay periods in the year and compare to the year-to-date gross pay. If the borrower has worked the full year at this employer, the year-to-date should be close to the annualized figure. A significant discrepancy, say more than two pay periods, should prompt investigation: did the borrower start mid-year, change employers, or is there a data entry or extraction error in one of the fields?

For the nurse's paystub described in the opening, the internal consistency check would catch a mid-year cycle change immediately, because the year-to-date gross would not match a full-year multiple of the current period. That flag goes to the underwriter with the extracted data, not after the LOS is already populated with potentially incorrect annualized income figures.

The 1040 Extraction Problem: Qualifying Income Is Not on Line 1a

Federal income tax returns present a category of extraction error that is qualitatively different from the field-assignment and character-recognition errors on paystubs and W-2s. On a 1040, the qualifying income for a salaried borrower really is on line 1a, and an extraction that reads that line correctly is complete for that borrower type. But for self-employed borrowers, for borrowers with rental income, for borrowers with S corporation or partnership distributions, and for borrowers with farm income, the relevant income is distributed across schedules that the extraction model may or may not retrieve, and the qualifying income formula requires a calculation that the extraction model is not being asked to perform.

The practical implication is that a 1040 extraction workflow needs to be designed with income type in mind, not with a one-size-fits-all field list. Before running an extraction on a tax return, the processor should identify which schedules are present. A standard 1040 with no attached schedules has a different extraction prompt than a 1040 with Schedules C, E, and SE (Schedule SE calculates the self-employment tax). Here is an extraction prompt designed for a Schedule C borrower:

You are processing a federal 1040 income tax return and Schedule C for a mortgage application. The borrower is self-employed. Extract the following items, reading directly from the document pages provided: From the 1040: Line 1a (wages), Line 11 (AGI), Line 15 (taxable income), Filing status (single, married filing jointly, etc.), Tax year. From Schedule C: Line 7 (gross receipts), Line 28 (total expenses), Line 31 (net profit or loss), business name, business type/activity. For each field, provide the value and the exact line number and form (for example, "Schedule C, Line 31"). Write "Not present" for any line that is blank or does not appear. Do not calculate qualifying income. Do not add or combine lines. Do not interpret whether the business is profitable. Report only what is written on the document.

The qualifying income calculation for a self-employed borrower is performed separately, by the underwriter or by a dedicated income calculation tool, using the extracted figures as inputs. The extraction model's job is to read the numbers from the document accurately, not to apply the lending guideline formula that converts those numbers into a qualifying income figure. Separating these two tasks, extraction and calculation, is one of the most important design principles in an AI-assisted income analysis workflow, because it keeps the potential for a single AI error from propagating through the entire income analysis undetected.

Asset Document Extraction: Bank Statements and Investment Accounts

Asset verification presents a different set of extraction challenges because the relevant information is not always in a labeled field. A lender verifying checking and savings account balances needs the ending balance on the most recent statement, but a lender verifying reserves, sourcing down-payment funds, or identifying large deposits that require explanation needs to identify specific transaction patterns within the statement history. Extraction models that work well for labeled-field documents struggle with pattern-based extraction on transaction records.

For balance verification from bank statements, the extraction prompt is relatively straightforward:

You are processing a bank statement for a mortgage application asset verification. Extract the following: Institution name; Account type (checking, savings, money market); Account number (last four digits only); Statement period (start and end date); Beginning balance for the statement period; Ending balance for the statement period; Any statement text indicating whether the account is a joint or individual account. For each field, state the value and where on the document it appears (for example, "Statement header, ending balance field"). Write "Not present" for any field not found.

For large-deposit sourcing, the extraction task is more complex because you are asking the model to identify transactions that meet a threshold, not to extract a labeled field. A useful prompt for large-deposit identification looks like this:

You are processing a 60-day bank statement history for a mortgage application. Identify all individual deposits of $1,000 or more during the statement period. For each deposit, provide: the date of the deposit; the amount; the description or payee name exactly as it appears in the statement; and the page and line number or approximate location in the statement where the deposit appears. List them in date order. Do not include recurring deposits below $1,000. Do not summarize or combine deposits. List each one separately even if the same payee appears multiple times.

The $1,000 threshold in this example is illustrative; the actual threshold depends on the lender's guidelines and the loan amount. The important features of the prompt are: it sets an explicit threshold, requires the source description exactly as written (not paraphrased), requires a page location reference, and prohibits summarization or combining, because a lender who needs to source a $4,200 gift deposit needs to see it as a single $4,200 item, not merged into an aggregate of April deposits.

For investment account statements, mutual fund accounts, and retirement accounts used for reserve calculation, the specific fields depend on whether the funds are fully or partially accessible, and the extraction prompt should reflect the lender's reserve policy. A 401(k) account that the borrower intends to use for down-payment reserves requires a different extraction than a brokerage account, because the 401(k) value is subject to a vesting haircut and an early-withdrawal tax penalty that reduce the qualifying reserve amount. Extracting the total account balance from a 401(k) statement without noting the account type and applicable restrictions produces a reserves figure that the underwriter cannot use without additional information.

Failure Modes and the Verification Rules They Demand

Understanding the specific ways extraction AI fails is the foundation of a verification protocol that actually catches errors. There are five failure modes that account for the majority of extraction errors in mortgage and consumer lending document processing.

Character misread. The most common single-character errors in paystub and W-2 extraction are the zero-six pair (0 confused with 6 in low-contrast scan areas), the one-seven pair (1 confused with 7 in certain fonts), and the five-six pair (5 confused with 6 in small print). These errors produce income figures that are plausible and close to correct but off by small amounts. On a field-by-field verification, they are caught immediately because the verifier is comparing the extracted value to the specific character at the specific location on the document. They are not caught by a plausibility check alone, because a figure of $7,580 is perfectly plausible when the actual figure is $7,530 or $7,680.

Column transposition. As discussed earlier, confusion between current-period and year-to-date columns on paystubs is the most consequential column error. A second common column error is the confusion between regular pay and overtime on stubs that itemize both. Overtime income is treated differently from base income in many lending guidelines, and an extraction that combines them into a single gross pay figure without the breakdown prevents the underwriter from applying the correct overtime-qualifying formula.

Page miss. A multi-page document in which the extraction model processes only the first page, or misses a page because the PDF is assembled out of order, produces silently incomplete output. The extracted fields look complete for a single-page document, but the second employer on page two, the carryforward amount from Schedule C, or the continuation sheet with the large deposit are absent. The page-count check at the start of the cross-check workflow catches this: confirm the number of pages the model processed against the number of pages in the document before you rely on any extracted values.

Label mismatch. When a document's field labels differ from the labels the model expects, the model may assign a value to the closest matching label it knows rather than flagging the field as unrecognized. A paystub that labels gross pay as "Total Comp" may have its gross pay extracted as a different field, or extracted correctly but labeled in the output with the generic "Gross Pay" label rather than the document's actual label. The source-label verification in the cross-check workflow catches this: if the extraction output says the value came from a "Gross Pay" field but the document has no field with that exact label, the processor investigates whether the assignment is correct.

Scanned document degradation. PDF documents that are generated by scanning paper copies, particularly second-generation scans (a scan of a copy of the original), introduce OCR errors that no amount of model improvement entirely eliminates. Missing digits, fused characters, and rotated or skewed text all degrade extraction accuracy. For degraded documents, the correct workflow is to request a fresh scan or a bank-direct digital statement rather than verifying a degraded extraction, because the verification effort on a badly degraded document approaches the effort of manual keying.

Governance, Privacy, and OCC 2026-13 Requirements

AI-assisted document extraction is a model in the sense that OCC Bulletin 2026-13 (the April 2026 interagency guidance that brought AI and generative AI tools under model-risk, fair-lending, third-party, and board-governance expectations) uses that term. If the extraction tool influences a credit decision by producing the income and asset figures that flow into the underwriting calculation, it is part of the credit decisioning chain, and the institution's model-risk program applies to it.

In practical terms, this means the institution should maintain documentation of the extraction tool's accuracy on the document types it processes, with periodic validation against manual extraction on a sample of production files. Validation does not require a large sample to be meaningful: a monthly review of fifty randomly selected files, with each extracted field compared to the source document, produces a field-level accuracy rate that the model-risk file can track over time. If accuracy on a specific document type, such as W-2s from certain employers or 1040s with multiple schedules, drops below a threshold the institution defines, the protocol should escalate to more intensive verification for that document type until the degradation is investigated and resolved.

Privacy considerations arise because document extraction workflows handle borrowers' most sensitive personal and financial information: Social Security numbers, tax returns, bank account numbers, employer information, and income figures. Extraction models should be configured to handle the minimum information necessary for their function, extraction outputs should not persist beyond the verification workflow unless specifically required, and the LOS integration should route verified values directly rather than storing raw extraction outputs in accessible logs.

The UDAAP (Unfair, Deceptive, or Abusive Acts or Practices, the broad consumer protection standard under the Dodd-Frank Act) dimension of extraction accuracy is worth noting for compliance officers. An extraction error that produces a lower income figure than the document supports, and that lower figure causes a denial or a less favorable offer, could constitute an unfair act or practice if it reflects a systematic pattern rather than a one-time error. Institutions that use AI extraction at scale should monitor for income understatement errors, not just income overstatement errors, because both have consequences for borrowers and both can trigger regulatory scrutiny.

The Community Reinvestment Act (CRA, the statute requiring banks to meet the credit needs of the communities they serve) adds a geographic dimension to the governance question. If extraction accuracy is lower on documents submitted by borrowers in certain geographies, such as lower-quality scans from areas with less reliable mail or print services, and that accuracy difference affects qualifying income figures and denial rates, the institution may face CRA examination findings tied to the extraction tool's differential performance. Validation studies that break accuracy down by document quality tier, not just overall, help identify this pattern before it becomes an exam finding.

The Bank Secrecy Act (BSA) and Anti-Money Laundering (AML) programs at the institution are not directly triggered by income and asset extraction for mortgage underwriting, but the data hygiene practices established in the extraction workflow, particularly the rule against using inferred or estimated values when source data is available, are consistent with the documentation standards BSA/AML programs require for Customer Due Diligence (CDD, the process of verifying customer identity and assessing risk). Institutions that build strong source-citation habits in their extraction workflows are building skills that transfer directly to the documentation demands of BSA/AML compliance.

Key Takeaways

• Document extraction AI reads labeled fields from scanned and digital documents. Its errors are character misreads, column transpositions, page misses, label mismatches, and degraded-scan failures, not the plausible inventions characteristic of generative AI. The verification strategy targets each of these specific failure modes, not general plausibility.
• Extraction prompts should specify exactly which fields to extract, require the source label for each value, prohibit estimation for missing fields, and constrain the model to the document rather than to general knowledge about typical documents. A well-constructed prompt produces verifiable output; a vague prompt produces output that is faster to generate but harder to check.
• The cross-check workflow has four mandatory stages: field count confirmation, source line verification for each value, field assignment verification for current-period versus year-to-date figures, and internal consistency check. All four stages complete before any extracted value enters the LOS. Skipping any stage produces a gap in the verification record that an examiner will identify.
• For 1040 tax returns, qualifying income for self-employed borrowers, rental income earners, and S corporation shareholders is not on a single line. Extraction and income calculation are separate tasks. The extraction model reads the numbers from the schedules; a separate calculation, performed by the underwriter or by a dedicated income tool, produces the qualifying income figure. Never ask the extraction model to calculate qualifying income.
• Bank statement extraction for large-deposit sourcing requires the model to identify threshold-meeting deposits by date and amount, with source descriptions exactly as written and page location references. Summarizing or combining deposits prevents the lender from sourcing individual amounts as required by loan guidelines.
• OCC Bulletin 2026-13 applies to extraction tools that influence credit decisions. The institution's model-risk program should track field-level accuracy by document type, validated on a sample of production files. Accuracy degradation on specific document types should trigger enhanced verification protocols and investigation before it affects denial rates.
• UDAAP exposure arises from systematic income understatement errors, not only overstatement. Institutions should monitor extraction accuracy for both directions of error, because a pattern of understatement that produces denials or worse offers for affected borrowers is an unfair practice regardless of whether the model intended to harm those borrowers.
• The shift from keying to verifying is a permanent change in the processor's role. The value the processor adds is no longer the ability to type accurately from a source document. It is the ability to confirm that an AI extraction correctly represents the source document, flag the discrepancies that matter, and maintain the verification record that makes every LOS field traceable to its source line.