The supervisor pulled three court reports off the unit's queue on a Thursday afternoon, the way she did before every dependency review docket. All three had been drafted with the agency's AI documentation tool. The first two checked out. The third was a termination-of-parental-rights report, the most consequential document a caseworker ever files, and on page six it read: "The mother failed to complete the court-ordered substance use assessment referred on March 12." The supervisor stopped. She remembered this case. The mother had completed the assessment; the result was in the file. She opened the case-management system and found it: assessment completed March 28, returned with a recommendation for outpatient services the mother had started. The AI had not just gotten a date wrong. It had reversed the single fact on which a judge might decide whether a child went home or never went home again. The caseworker who drafted the report was experienced, careful, and exhausted, carrying 29 families. He had read the draft twice. He had not caught it, because the sentence read exactly like every true sentence around it. The supervisor caught it because she had a method, and the method did not depend on remembering the case or trusting the draft. This lesson is that method.

Why Careful Reading Is Not a Method

By the time a practitioner reaches this lesson, they already know the failure modes. They know that a large language model (LLM, the AI system that generates text from prompts and documents) produces statistically plausible prose rather than retrieving and checking facts. They know the three ways it goes wrong inside a case record: invented observations, misapplied or wrong policy, and fabricated history. What they often do not have is a repeatable procedure that catches those failures reliably, every time, under caseload pressure, on a Thursday at five o'clock with a docket the next morning.

The instinct most workers bring is "read it carefully." That instinct fails, and it fails for a specific reason that has to be understood before any method can replace it. Careful reading is a coherence check. The brain reads for sense, flow, and plausibility, and it is extraordinarily good at that. But hallucinations are coherent. They flow. They are plausible. The fabricated sentence about the failed assessment did not read as wrong because it was not wrong in any way the reading brain detects. It was grammatically clean, professionally worded, contextually appropriate, and consistent with the kind of sentence that appears in termination reports. Reading for sense will never catch it, because to the sense-making part of the mind it makes perfect sense.

The caseworker in the opening read the draft twice and missed the reversal. That is not a story about a careless worker. It is the expected result of applying a coherence check to a coherence failure. You cannot catch a plausible falsehood by checking whether it is plausible. You catch it by leaving the text entirely and going to the source. The shift this lesson teaches is from reading the draft to interrogating the draft against the record, claim by claim, in a way that does not rely on memory, does not rely on the draft sounding right, and does not slow down so much that it erases the time AI was supposed to return.

You cannot catch a plausible falsehood by checking whether it is plausible. The only reliable check leaves the draft and goes to the source.

The Claim-Extraction Pass

The method has a name for its first move: claim extraction. Before verifying anything, the practitioner converts the AI draft from a piece of prose into a list of discrete, checkable factual claims. This sounds mechanical because it is. The point of making it mechanical is to defeat the coherence trap. When the document is a flowing narrative, the eye glides. When it is a numbered list of atomic claims, each one stands alone and demands a yes or no.

A factual claim is any statement that asserts something happened, exists, was observed, was said, or governs the case. "The home was clean and the children appeared well cared for" is two claims, not one: the state of the home, and the condition of the children. "The mother completed parenting classes in the fall and tested negative on a hair-follicle screen" is two claims with two separate sources. The extraction pass breaks compound sentences apart, because a sentence that is half true and half fabricated will pass a glance and fail only when its halves are separated.

Consider a routine court report of about 1,200 words. A disciplined extraction pass on that report typically yields somewhere between 25 and 60 discrete claims: dates of contact, observations from visits, services referred, services completed, statements attributed to the parent or child, policy or statutory standards invoked, and prior-history references. On the unit in the opening story, the supervisor's rule was simple: no AI-drafted court report goes to a docket until every extracted claim has a source notation next to it. The first time a worker did this, it took 40 minutes on top of drafting. By the fifth report it took 12, because the worker had learned to draft in a way that made claims easy to trace, and because the case-management system entries were already organized in the worker's mind. The method is slow once and fast after that.

Walk through what the extraction pass looks like on a single paragraph, because the abstraction only becomes a skill when it is concrete. Suppose the AI draft reads: "Worker conducted a home visit on April 3 and observed the home to be clean and adequately stocked with food. The two children, ages 6 and 9, appeared well and reported attending school regularly. The mother stated she had completed her outpatient treatment and provided documentation." A glance reads this as one smooth, reassuring paragraph. The extraction pass breaks it into at least seven discrete claims: the visit occurred on April 3; the home was clean; the home was adequately stocked with food; the children's ages are 6 and 9; the children appeared well; the children reported regular school attendance; and the mother stated she completed treatment with documentation. Each of these now has a yes-or-no source question attached. The April 3 date traces to the contact log. The home condition and the children's appearance trace to the field notes. The school attendance traces either to the field notes (if the child said it) or to a school record. The treatment completion traces to the documentation in the case-management system. If even one of these seven, say the food in the home, is not in the field notes, the smooth paragraph that read as entirely true was carrying a fabrication that no amount of rereading would have surfaced.

There is a tooling shortcut, and it must be used carefully. A worker can ask the AI itself to list every factual claim it made in the draft. This is useful as a starting inventory because the model is good at parsing its own output into discrete statements. But it carries a trap that the practitioner must name out loud: the model that hallucinated a claim can also omit that claim from its own extraction list, or mislabel a fabricated claim as supported. The AI's claim list is a draft of the claim list, subject to the same verification as everything else. It speeds the extraction; it never replaces the human's own pass, and a claim the human spots that the AI's list missed is exactly the kind of claim most likely to be invented.

The Three-Source Traceback

Once the claims are extracted, each one is traced to a source. The discipline here is that there are exactly three legitimate source types, each matched to a failure mode, and a claim that cannot be traced to its correct source type is presumed unverified and must be removed or rewritten. The worker is not allowed to mark a claim "true" from memory, from the draft's confidence, or from asking the AI. Memory is the thing that failed the caseworker in the opening; the draft's confidence is the disease; and the AI confirming its own claim is asking the arsonist whether the fire was set.

Observations Trace to Field Notes

Every claim about what the worker saw, heard, or did during a contact traces to the worker's own contemporaneous field notes from that contact, not to the worker's memory and not to the AI draft. If the draft says the kitchen had no food in the refrigerator and the field notes do not contain that observation, the claim is removed, even if the worker thinks it is probably true. The case record documents what was observed and recorded, not what was likely. This is why the method depends on field-note discipline upstream: a worker whose raw notes are three words and a vibe has nothing to trace against, and the traceback collapses into the same memory check that already failed.

Policy Traces to the Current Authoritative Source

Every claim that invokes a rule, threshold, standard, or statutory provision traces to the current authoritative source: the state policy manual, the federal regulation, the agency operating procedure, or the statute, in its version in effect on the date of the determination. The AI is never the source for policy, because policy misapplication is precisely the failure mode where the model sounds most authoritative while being wrong, and because rules change after the model's training data was collected. A SNAP (Supplemental Nutrition Assistance Program, the federal food-assistance benefit) income threshold indexed to the federal poverty level updates annually; a state may have adopted a more generous option than the federal floor the model defaults to. The traceback for a policy claim ends at a citation the worker has opened and read, not a citation the model produced.

History Traces to the Case-Management System

Every claim about the case's past, a prior incident, a prior service, a prior finding, a prior contact, traces to a specific entry in the case-management system: an intake record, a service log, a completed-assessment document, a prior court order. "A CPS (child protective services) report was received in March 2024" must correspond to an actual intake record with that date. The reversal in the opening, the completed assessment recorded as failed, was a history claim, and the three-source method would have caught it the instant the worker tried to trace "failed to complete the assessment" to the system and found instead a record of completion dated March 28. The method does not require the worker to remember the truth. It requires the worker to look.

Observations to field notes, policy to the current authoritative source, history to the case-management record. A claim that will not trace to its source type does not belong in the document.

Triage: The Highest-Risk Claims First

Tracing every claim is the standard, but not every claim carries the same weight, and a method that treats a contact date and a substantiation finding as equally urgent will run out of time on the wrong things. The method layers a triage on top of the traceback so that the verification effort concentrates where a fabrication does the most damage. This is not permission to skip claims; it is an ordering, so that if an interruption ends the verification session early, the claims most capable of separating a family have already been checked.

Three categories sit at the top of the triage. First, any claim that bears directly on a consequential decision in front of a decision-maker: a removal, a return home, a substantiation, a termination, a benefit denial. The reversed-assessment sentence sat in exactly this tier. Second, any negative characterization of a parent, child, or household: substance use, non-compliance, neglectful conditions, a "history of." These are the claims that shape how a judge or a guardian ad litem reads the family, and they are the claims a model tends to manufacture because the documents it learned from are full of them. Third, any policy or eligibility claim that determines whether a person receives or loses a benefit, where a wrong threshold denies food, medical coverage, or shelter.

Lower in the triage sit claims that are real but less load-bearing: the spelling of a school name, the precise time of a visit, formatting of an address. They still get traced, but they are not where the first ten minutes go. A supervisor reviewing a worker's verification can ask a single diagnostic question: show me the source for the three claims that most affect the decision. If the worker can point to a field note, a regulation, and a system record for each, the verification was real. If the worker says they "looked fine," the verification did not happen, regardless of how clean the document looks.

The Cross-Examination Test

The most useful mental model for catching hallucinations against the record comes from the place these documents eventually go: a hearing. A court report, a safety assessment, an eligibility determination is a document that may be challenged by an attorney, a parent's advocate, or a fair-hearing officer whose entire job is to find the claim that cannot be substantiated. The verification method should anticipate that adversary. For each high-risk claim, the worker asks: if opposing counsel pointed at this sentence and said "show me where this is in the record," could I produce the source, on the stand, today?

This test does real work because it converts a vague sense of confidence into a concrete artifact. "I'm fairly sure she didn't complete the assessment" is not an answer a worker can give under cross-examination. "Here is the service log entry dated March 28 showing completion" is. The cross-examination test forces the worker to locate the artifact, and the act of locating it is the verification. When the artifact cannot be found, the worker has discovered a hallucination, or at minimum an unsupported claim that must come out of the document before it is filed, because a claim that cannot survive cross-examination should never have been put in front of a court in the first place.

There is a due-process dimension here that elevates this from good practice to obligation. A parent in a dependency proceeding has the right to challenge the evidence against them. A claimant denied benefits has the right to a fair hearing. When an AI-drafted document carries a fabricated claim into that proceeding, it does not merely risk an inaccurate record; it places into a due-process forum a statement the family is entitled to test and that will not withstand testing. The worker who catches it preserves the integrity of a process the family is owed. The worker who misses it has, however unintentionally, entered false evidence into a system built on the premise that the evidence is real.

Building the Discipline Into the Workflow

Individual method is necessary and insufficient. A unit of 15 workers each carrying 25 to 30 families cannot rely on every worker performing a flawless claim-by-claim traceback every time, especially when the AI draft is 95 percent correct and the temptation to trust the 95 percent and file is strongest exactly when the caseload spikes. The five percent is where the harm lives, and it does not announce itself. The discipline has to live in the workflow, not only in the worker's intentions.

That means a verification artifact that travels with the document. On the unit in the opening, every AI-assisted court report carried a short verification log: the extracted high-risk claims, the source traced to each, and the worker's sign-off, with supervisory review of the top-tier claims before filing. This is not bureaucracy for its own sake; it is the audit trail that lets the agency answer a court or an advocate who asks how the document was checked. It also changes worker behavior, because a claim that has to be sourced on a log is a claim the worker actually sources, where a claim that only has to "look fine" is a claim the worker skims.

Quantify the time so the trade-off is honest rather than aspirational. Suppose the AI tool saves a worker roughly 30 minutes per court report by producing a complete first draft, and the worker files about eight such reports a week. That is four hours returned. A disciplined claim-by-claim verification, once the worker is fluent, runs about 12 to 15 minutes per report, or roughly two hours a week across those eight reports. If the agency lets the worker keep the remaining two hours for direct work with families, the deployment is honest: faster documentation, verified records, time returned. If instead the agency raises the worker's caseload to absorb all four returned hours, the worker now has eight more reports' worth of verification with no time to do it, and the only way to keep up is to stop verifying. The number that determines whether this method survives contact with reality is not the model's accuracy. It is how many of the returned hours the agency leaves on the worker's calendar for verification.

It also means protecting the time the method requires. The genuine benefit of AI documentation is the hours it returns by drafting faster. If an agency captures all of those hours by raising caseloads or expecting more output, it has not reduced documentation risk; it has guaranteed that verification gets skipped, because there is no time to do it. A defensible deployment spends part of the returned time on verification and part on direct work with families, and treats the verification step as non-optional work, scheduled and supervised, not as something diligent workers do if they can find a spare twenty minutes they do not have. The method in this lesson is only as real as the time the agency gives workers to run it.

Key Takeaways

• Careful reading does not catch hallucinations because hallucinations are coherent, fluent, and plausible; a coherence check cannot detect a coherence failure. The reliable method leaves the draft and traces claims to independent sources.
• The method begins with a claim-extraction pass that converts the prose into a numbered list of discrete, atomic factual claims, breaking compound sentences apart so a half-true sentence cannot pass on a glance. A typical 1,200-word court report yields roughly 25 to 60 claims.
• Each claim traces to exactly one of three source types matched to the failure modes: observations to the worker's contemporaneous field notes, policy to the current authoritative manual or regulation, and history to a specific case-management system entry. Memory, the draft's confidence, and the AI itself are never valid sources.
• Triage concentrates verification on the highest-risk claims first: those bearing on a consequential decision, negative characterizations of a family, and eligibility claims that grant or deny a benefit. The lower-risk claims still get traced, but not before the load-bearing ones.
• The cross-examination test, "could I produce the source for this claim on the stand today," converts confidence into a concrete artifact; when the artifact cannot be found, a hallucination or unsupported claim has been discovered and must be removed before filing.
• Catching hallucinations against the record is a due-process safeguard: a parent has the right to challenge the evidence and a claimant has the right to a fair hearing, and a fabricated claim places into that process a statement that cannot survive the testing the family is owed.
• The discipline must live in the workflow through a verification log that travels with the document and through supervisory review of top-tier claims, not solely in individual diligence under caseload pressure where the temptation to trust a 95 percent-correct draft is strongest.
• Agencies must protect the time the method requires by spending part of the hours AI returns on verification rather than capturing all of it as higher caseload; an unverifiable workflow has moved documentation risk, not reduced it.