โ†
AI for Public Safety & First Responders
Capable ยท M10 ยท lesson 10 of 19 ยท queued
Preview โ€” browse every lesson free. Enroll to mark lessons complete, open partner links and save your progress. Login & enroll โ†’
Getting Accurate Output Under an Evidence Standard
๐Ÿ“–
now learning

Getting Accurate Output Under an Evidence Standard

15 min

A detective in a mid-sized department had twelve hours of recorded interview audio from three separate witnesses to a commercial robbery. The investigation had been open for six weeks. The case was going to grand jury in eleven days. She opened her agency's AI summarization tool, uploaded the audio transcripts, and asked for a summary of each witness account. The summaries came back in four minutes, clean and professional, organized by witness and topic. She reviewed them for five minutes, found them plausible, and incorporated them into her grand jury preparation file.

Eight days later, the defense attorney received discovery. He spent two hours with the audio transcripts and the AI-produced summaries. He found three factual claims in the summaries that could not be located in any of the transcripts. Not subtle misreadings. Absent facts, attributed to the witnesses with confidence. The model had filled three gaps with details that fit the pattern of a robbery investigation but were not present in the actual recordings.

The detective had not prompted badly. She had given the model the transcripts. She had asked for a summary tied to those transcripts. The model had still hallucinated three facts. The difference between the banking context and this one is that the consequence is not a repurchase demand. It is a Brady violation (Brady v. Maryland, 373 U.S. 83 (1963), the constitutional requirement that prosecutors disclose material exculpatory evidence to the defense), a suppression motion, and potentially a collapsed prosecution. This lesson is about the standard of accuracy that public-safety work demands and the specific prompting and verification discipline that meets it.

The Evidence Standard Defined

In most professional contexts, accuracy means "mostly right" or "close enough for the purpose." In public-safety AI work, accuracy means something more specific: every factual claim in a document that will be disclosed to a prosecutor or defense attorney must be traceable to a source. Not probable. Not consistent with how this type of incident usually unfolds. Traceable to a specific timestamp in the footage, a specific line in the CAD (computer-aided dispatch) entry, or a specific page in the case file.

This is what the lesson calls the evidence standard. It is not a novel invention. It is the standard that sworn police reports have always been held to. Officers know it instinctively: a report that says "the subject stated" must reflect what the subject actually stated, not what a subject in a similar situation typically states. What is new is that AI tools can violate that standard invisibly, producing a claim that is false but stated with the same confident professional tone as a claim that is true. The standard has not changed. The risk has increased.

The evidence standard is not a quality bar. It is the legal and constitutional minimum: every factual claim must be traceable to the record, or it does not belong in the document.

This standard applies regardless of how confident the AI output sounds, regardless of whether the output is plausible, and regardless of how time-pressured the reviewer is. There is no circumstance under which "it sounded right" is a sufficient verification standard for a sworn report or a grand jury preparation file. The standard is traceable to the record, and the record is the BWC (body-worn camera) footage, the CAD entry, and the case file in the RMS (records management system).

Why the Evidence Standard Is Harder with AI

Before AI drafting tools, the officer was both the generator and the verifier of the report. The act of writing a report from your own memory, your notes, and the footage was itself a form of verification. You were not reading a stranger's account of your incident and checking it. You were constructing the account from your own sources. Errors still occurred, but they were your errors, generated by your own memory and judgment, and you were in the best possible position to catch them during the writing process itself.

With AI drafting, you are now reading a stranger's account of your incident and checking it. The model is the stranger. It has never been to your incident. It has your transcript and your CAD notes, but it applies to those materials the full weight of its training on how incidents like this usually go. Most of the time, that produces a reasonable draft. Some of the time, it produces a confident error. The officer's job has shifted from "construct this account from your sources" to "verify that this stranger's account of your sources is accurate." That is a different cognitive task, and it is harder in one specific way: you are looking for errors in a document that was designed to read as if it has no errors.

The evidence standard is the response to that challenge. It is not a mood. It is a method. Every factual claim, one by one, checked against the source.

Forcing the Model to the Recording

The single most effective technique for improving AI accuracy in public-safety contexts is grounding: attaching the actual source documents and instructing the model to draw only from them. This technique was introduced in the previous lesson on prompting basics. This lesson goes deeper into how and why grounding works, and what to do when grounding alone is not enough.

Grounding changes the model's behavior in a specific way. Without source material, the model generates from its training distribution: what a typical report of this type looks like. With source material explicitly attached and a "draw only from these materials" instruction included, the model shifts to organizing and synthesizing the text you provided. It still generates language (it is always generating language), but the raw material it is synthesizing is your incident record rather than its statistical memory of similar incidents.

The practical effect is a reduction in the gap-fill hallucination, the most common failure mode in public-safety AI work. A gap-fill hallucination is a detail the model invented because the source material was silent on that point. Grounding does not eliminate gap-fills, but it dramatically narrows the space in which they occur. Instead of filling gaps across the entire narrative, a grounded model is more likely to flag gaps explicitly or to stick to what the transcript contains.

To force the model to the recording, use these techniques:

Provide the full transcript, not a summary. If you summarize the footage before giving it to the model, you have already filtered the material. The model will be synthesizing your summary, and any gaps in your summary become gaps in the model's output. Give the model the raw transcript. Let it do the synthesis. You will review the output against the raw transcript.

Name the source explicitly in the prompt. "Based only on the attached transcript of BWC footage from incident 2026-03814, camera ID BWC-441" is more constraining than "based on the transcript attached." The more specific your source reference, the more clearly the model understands what constitutes the authoritative record for this prompt.

Issue an explicit prohibition on inference. "Do not infer, estimate, or add details not explicitly present in the provided transcript. If a detail is absent, write 'not present in provided materials' rather than inferring." This prohibition directly targets the gap-fill behavior. Many officers are surprised to find that explicitly naming and forbidding the behavior they are trying to prevent actually reduces how often the model exhibits it.

Ask for citations within the output. "For each factual claim in the narrative, include a citation to the footage timestamp or CAD note line on which it is based, in brackets after the claim." Requiring the model to cite its sources produces two benefits: the model's output is easier to verify (you know exactly where to look for each fact), and the act of producing citations requires the model to process its relationship to the source material more carefully. Output with citations has fewer uncited fabrications than output without citations, across most tested models.

The CAD Entry as an Anchor

The BWC footage is the richest source of incident information, but it is not the most reliable source of objective facts. A conversation captured on BWC can be misheard, misread in transcript, or cut off by camera angle. The CAD entry, by contrast, is a system record. The received time, the dispatch time, the arrival time, the call type, and the address were entered by a telecommunicator (the dispatcher who fielded the call) in real time and logged in the system. These facts are not subject to interpretation. They are the official record of the call.

When you are prompting for a report narrative, include both the BWC transcript and the CAD export. Ask the model to use the CAD entry as the authoritative record for objective facts (times, addresses, call type) and the BWC transcript as the authoritative record for what was said and done during the contact. This division of labor reduces the risk that the model will use the BWC audio to determine a time that is better established by the CAD record, or that it will use vague language about location when the CAD entry has a precise address.

The CAD entry is also your first line of defense against the sequence error: a hallucination that occurs not in the facts themselves but in the order in which they are presented. A model can include accurate facts from the footage but arrange them in an inaccurate sequence, making it appear that Officer A made contact before Officer B when the CAD timestamps show the reverse. Grounding the timeline to the CAD timestamps prevents this class of error.

Forcing the Model to the Case File

For investigative documents, the source material is broader than a single BWC recording. A detective summarizing a murder investigation may have a hundred documents in the case file: interview transcripts, forensic reports, property receipts, photo logs, lab results, and prior field interview records. An AI tool that has access to all of those documents can produce a comprehensive summary dramatically faster than a detective working manually. An AI tool that has access to some of those documents and fills the rest from training patterns is a problem waiting to happen.

The rules for forcing the model to the case file are an extension of the rules for forcing the model to the recording:

Specify the scope explicitly. "Summarize the state of this investigation using only the documents listed below. Do not draw on any information not present in these documents." If you give the model a list of the documents you are including, it has a reference it can use to flag when it is being asked to produce a claim it cannot locate in the provided materials.

Separate the documents by type in your prompt. "Interview transcripts: [list]; Forensic reports: [list]; Property receipts: [list]." Organizing the input helps the model organize the output by source type, which makes your verification pass easier and faster.

Ask the model to flag anything it cannot source. Just as with the recording, instruct the model to write "not present in provided case file materials" rather than inferring. This is especially important in a case file context, where the model's training includes a large volume of material about how investigations of this type typically unfold. The temptation to fill from training patterns is strongest when the source material is incomplete.

Treat AI summaries of case files as preliminary, not final. A case file summary produced by AI is a starting point for your own review, not a finished document. The summary will be faster and more organized than manual summarization. It will also contain some number of errors. Your job is to verify those errors against the case file before the summary reaches a prosecutor, a grand jury preparation document, or a disclosure package. The word "preliminary" should appear in the document header until you have completed your verification pass.

The Quote Problem

Among all the hallucination types in public-safety AI work, the invented quote is the most dangerous and the most reliably produced by AI models without specific constraints. When a model summarizes an interview, it tends to produce representative quotes: short, clear statements attributed to the interviewee that capture the gist of what they said. Many of those statements will be close paraphrases of actual statements. Some will be fabrications that sound like what this type of witness in this type of case usually says.

A fabricated quote attributed to a witness is a catastrophic document error. In a criminal case, witness statements are the evidentiary core of the prosecution. The defense has a right to test every statement attributed to every witness. If an AI-produced document contains a quote that cannot be found in the recorded transcript, the defense has the basis for an impeachment that was never actually available to them, based on a statement the witness never made. That is not just an error. It is a Brady problem and potentially a case-ending problem.

The defense against invented quotes is a specific prompt instruction and a specific verification protocol. The prompt instruction: "Do not use quotation marks anywhere in the output. Summarize all witness statements in indirect speech only (e.g., 'The witness stated that...' rather than 'The witness said "..."'). If a direct quote is required, you must include the footage timestamp for that specific statement in brackets." Removing the quotation mark as a textual affordance reduces the model's tendency to generate invented attributed speech. Requiring timestamps for any retained direct quotes gives you a specific verification target.

The verification protocol for quotes: before any AI-produced summary containing attributed statements reaches a disclosure document, every attributed statement must be traced to a specific point in the recording. Not approximately. Specifically. If the transcript says at 14:23:07 the witness stated "I saw him run north on Fifth," and your AI output says the witness stated they saw the subject run north on Fifth (citing timestamp 14:23:07), that is verified. If your AI output says the witness stated the subject was carrying something in his left hand and you cannot find that statement in the transcript, that is an invented claim. Delete it. Note the gap. Verify separately from your own review of the recording.

The Skeptic's Prompt: Asking the Model to Check Itself

One of the most powerful and least used techniques in public-safety AI prompting is the follow-up verification prompt: after the model produces its initial draft, issue a second prompt that asks the model to audit its own output against the evidence standard.

The follow-up prompt looks like this:

"Review the draft narrative you just produced. For each factual claim (all statements of fact about time, location, persons, actions, or statements by any party), identify: (1) whether the claim is directly supported by the provided BWC transcript or CAD notes, with the specific timestamp or line reference; (2) whether the claim is an inference from provided materials; or (3) whether the claim cannot be traced to the provided materials. List all claims in categories 2 and 3 separately."

This prompt asks the model to perform a self-audit. The results are imperfect (the model cannot always accurately characterize its own sources), but they are useful. The claims the model identifies as inferences or unsupported are a prioritized list for your verification pass. You know exactly which parts of the draft need the most scrutiny.

This technique is sometimes called the skeptic's prompt because it inverts the default posture of AI use: instead of asking the model to be helpful and produce clean output, you are asking it to be skeptical of its own output and surface its weaknesses. In a context where a Brady violation could result from a single unverified claim, the skeptic's posture is the right one.

The Three-Source Rule

A practical rule for verification that has emerged from AI-assisted report writing in early-adopting agencies is the three-source rule. For any factual claim that the AI includes in a narrative, verify it against at least one of the three primary sources before finalizing the document:

Source 1: The BWC footage or audio recording. This is the richest and most authoritative account of the contact itself. For any claim about what was said, done, or observed during the contact, the footage is the primary source.

Source 2: The CAD entry. This is the authoritative source for objective facts: times, addresses, call types, and dispatch notes. For any claim about when something happened or how the call was categorized, the CAD entry is the primary source.

Source 3: The case file in the RMS. This is the repository for all prior and related documentation: prior incident reports involving the same address or parties, prior field interview cards, prior arrest records relevant to the case. For investigative documents that reference prior case history, the RMS is the primary source.

A claim verified against one of these three sources meets the evidence standard. A claim that cannot be verified against any of the three is unverified and should not appear in a sworn document. The three-source rule is not complex. It takes practice, not genius, to apply it consistently. What it requires is the discipline to apply it every time, not just when you suspect something is wrong.

When the Evidence Standard and Speed Are Both Real

Officers in early-adopting agencies frequently describe the same tension: the AI tool saves real time, but the verification standard required to use it responsibly takes time. Where is the net benefit?

The answer is that the verification pass with a grounded AI draft is substantially faster than writing the report from scratch. Consider the arithmetic. An officer who writes a ten-minute domestic disturbance report manually spends roughly thirty minutes on the task. (Officers spend 30 to 40% of a shift on paperwork, and on a twelve-hour shift, that is three to five hours.) With an AI draft from BWC footage, the model produces a structured narrative in under a minute. The officer's task shifts from construction to verification: reviewing each factual claim against the footage and the CAD entry. A well-grounded draft with citation markers can be verified in eight to twelve minutes. That is a real time savings, not an approximation.

The savings compound when the tool is used consistently. An officer who develops a reliable prompting and verification habit will, over a full patrol year, spend significantly fewer hours on reports than an officer who writes every report from scratch. Those hours go back to patrol. Axon's testing officers reported an 82% decrease in report-writing time with Draft One, their AI-assisted narrative tool. Even at a fraction of that number, the time savings are meaningful.

The evidence standard does not eliminate those savings. It prevents the savings from coming at the cost of accuracy. A verification pass that catches an invented quote before it reaches the prosecutor's file is not an overhead cost. It is the job. The report is evidence. Verifying it is part of the authorship obligation. The AI draft makes that obligation faster to discharge, not optional to discharge.

There is one situation where the speed case breaks down: if the AI draft is so poorly grounded that it requires more time to correct than to replace. If you open an AI draft and find that it is full of invented details, sequence errors, and unsupported claims, the fastest path forward is sometimes to close it and write the report from your notes. A badly grounded draft is a liability, not a time-saver. This is why the prompting work in the previous lesson matters. A well-constructed prompt produces a draft that is faster to verify. A badly constructed prompt can produce a draft that is slower to verify than writing from scratch.

Key Takeaways

  • The evidence standard means every factual claim in a disclosed document must be traceable to a specific source: a timestamp in the footage, a line in the CAD entry, or a document in the case file. "Plausible" and "consistent" are not sufficient.
  • Grounding (attaching source materials and instructing the model to draw only from them) is the single most effective technique for improving AI accuracy. Give the model the raw transcript, not a summary. Name the source explicitly. Prohibit inference.
  • Requiring in-line citations within AI output produces a verified output more reliably than uncited output, and makes the verification pass faster and more targeted.
  • The CAD entry is the authoritative source for objective facts (times, addresses, call type). The BWC footage is the authoritative source for what was said and done during the contact. Use both.
  • The invented quote is the most dangerous hallucination type in public-safety work. The defenses are: prohibiting quotation marks in the prompt, requiring timestamps for any retained direct quotes, and verifying every attributed statement against the recording before it reaches a disclosure document.
  • The skeptic's prompt (asking the model to audit its own output and identify unsupported claims) gives you a prioritized list for your verification pass. Use it for high-stakes documents.
  • The three-source rule: every factual claim must be verified against at least one of the three primary sources (BWC footage, CAD entry, RMS case file) before the document is finalized.
  • A well-grounded AI draft with an eight-to-twelve-minute verification pass is faster than writing from scratch, not slower. The evidence standard and the time savings are compatible. The evidence standard and a careless review pass are not.