The switching order summary looked authoritative: feeder designations, substation names, equipment IDs, and a recommended sequence for restoring service to 4,200 customers. The only problem was that transformer T-47A, the one the AI said to energize first, did not exist. The actual transformer was T-47C, sitting on a different bus. The operator caught it. But only because she cross-checked against the model book.

Why Grid Summaries Are High-Risk Hallucination Territory

A generative AI model produces text the way a very well-read intern might produce a summary: confidently, fluently, and without always knowing the difference between what is in the source document and what it is pattern-matching from related knowledge. In most contexts, a confident-but-wrong summary costs you a few minutes of verification. In a grid context, a confident-but-wrong summary can cost you a switching error on a 230 kV bus, an incorrect restoration estimate sent to regulators, or a compliance narrative that misquotes the standard it is citing.

The specific failure mode of hallucination in a grid context is not random noise. It is structured, confident error. The model produces an asset ID that follows the right naming convention but refers to a non-existent device. It cites a NERC standard by the right number but quotes a requirement that was in a prior version and was changed in the current enforceable text. It gives a restoration time estimate that sounds reasonable based on historical patterns but does not account for the specific access constraint on this particular circuit. The dangerous part is not that the output is wrong. The dangerous part is that it is wrong in a way that looks right.

Three categories of grid data are the highest-risk surfaces for hallucination: asset identifiers (equipment IDs, feeder designations, substation names, bus numbers), regulatory citations (NERC standard numbers and their current requirement text), and real-time system state (current loading, equipment status, switching position). Each of these categories has an authoritative source that the AI may or may not have access to, and each requires a different cross-check technique.

A hallucinated transformer ID is not a typo. It is a confident fabrication that follows the right pattern. Pattern-matching the naming convention is not the same as knowing the device exists.

The Three Authoritative Sources Every Grid Summary Must Reference

Catching hallucinations in grid summaries requires knowing what the AI should have said, which means knowing where the ground truth lives. For utility operations and planning work, three authoritative data systems define the ground truth that any AI-generated grid summary must be cross-checked against.

The SCADA/EMS real-time database. The Energy Management System (EMS) and its SCADA subsystem maintain the real-time state of the bulk power system: current MW and MVAR flows, equipment operating status, alarm conditions, and switching positions. Any AI summary that characterizes real-time system state must be checkable against the EMS. If the summary says a 138 kV line is energized and lightly loaded, an operator can look at the EMS in seconds and confirm or refute that claim. The ground truth is the EMS, not the AI's interpretation of it.

The GIS (Geographic Information System) and model book. The GIS and the network model book (sometimes called the equipment model, the bus-branch model, or simply "the model") contain the utility's equipment catalog: every substation, every feeder, every transformer, every bus, and every switch, with their correct designations and connectivity. This is the authoritative source for equipment existence and naming. An AI that invents a transformer ID may produce a plausible-sounding name, but it is an ID that the GIS and the model book do not contain. The cross-check is simple but must be habitual: any asset identifier in an AI-generated summary must be verified against the model book before the summary is acted on.

The current enforceable regulatory text. NERC standards, FERC orders, state PUC rules, and tariff language change. The AI's training data has a cutoff date; it does not automatically know that a standard was revised, that a FERC order superseded a prior requirement, or that a state commission issued an emergency rule. Any AI summary that cites a regulatory requirement must be verified against the current, published version of that requirement before it is used in a compliance document, a rate case filing, or an operator procedure. The source for NERC standards is the NERC Standards Library. The source for FERC orders is the official FERC docket system.

A Practical Detection Protocol for Grid Summaries

A detection protocol is a fast, repeatable process that every professional who uses AI-generated grid summaries can apply before acting. It does not require technical expertise in AI; it requires operational expertise in the grid. Here is a five-step protocol built on what experienced operators and planners actually do when they are being careful.

Step 1: Scan for asset identifiers. Before reading the summary for content, scan it for every piece of asset-specific information: equipment IDs, feeder numbers, substation names, bus designations, transformer identifications. Highlight them. These are the highest-risk hallucination targets. Each one must be verified against the model book or GIS before the summary drives any action.

Step 2: Pull the parallel SCADA view. For any summary characterizing real-time system state, open the corresponding EMS display while reading the summary. Compare each specific claim about equipment status or loading against what the EMS is actually showing. Discrepancies are red flags that require resolution before action. The operator is not trying to decide whether the AI is "generally right"; the operator is confirming each specific claim individually.

Step 3: Check regulatory citations against published text. For any summary that references a standard, requirement, or order, look up the cited provision in the official source. Read the actual text. Do not assume the AI's paraphrase is accurate. A single changed word in a regulatory requirement can alter what compliance looks like. This step is especially important for any document that will be filed with NERC, FERC, or a state commission.

Step 4: Apply the implausibility test. Does any specific claim in the summary seem surprising relative to what you know about this system? An experienced operator who has worked a feeder for several years has intuitions about what equipment exists, what loads are typical, and what switching sequences are standard. The implausibility test is a rapid filter: if something in the AI output surprises you, that surprise is a signal to verify that specific claim before proceeding. Hallucinations often trigger this intuitive alarm in experienced professionals; the discipline is to act on the alarm rather than override it because the overall summary looks good.

Step 5: Document what you checked. When the summary drives an action, record which claims you verified, against what source, and what you found. This documentation is not just good practice; it is the audit trail that lets you explain after an event why you trusted the AI summary, or lets you show a regulator that the AI was used as a starting draft and not as an autonomous authority. A 30-second note in the outage log or the switching record is sufficient: "Verified T-47C against model book; confirmed energized per EMS; switching sequence consistent with switching procedure."

The SCADA-GIS Crosswalk Technique in Practice

The most systematic approach to hallucination detection for grid summaries involving equipment is what experienced operations engineers call the SCADA-GIS crosswalk: a structured comparison of every asset mentioned in the summary against two independent authoritative sources.

Imagine an operations engineer is reviewing an AI-generated storm restoration summary that covers 47 equipment restoration steps across three distribution substations. She does not have time to manually look up all 47 items before crews start moving. But she can apply the crosswalk efficiently by working in a specific order.

First, she checks the substation names. Three substations: Millbrook, Heron Lake, and Cedar Ridge. She verifies all three exist in GIS and are in the affected territory. Check.

Second, she checks the feeder designations. Seven feeders are named. She looks up all seven in the distribution system map. Six are correct. One, feeder 4C-22, does not exist in the current GIS. The actual feeder is 4C-23. The AI transposed a digit. That feeder leads to 800 customers. If crews are dispatched to the wrong feeder designation, there is a delay at minimum and potentially a switching error.

Third, she checks three transformer identifiers called out for specific switching steps. All three check out against the model book.

The crosswalk took eight minutes for a 47-item summary. The feeder transposition was the only error. She corrects it, notes it in the log, and proceeds. The AI summary was useful: it organized the sequence, drafted the rationale, and identified the priority order. The crosswalk caught the one error that would have caused a problem.

This is the appropriate human-AI workflow for grid summaries in operations. The AI handles the organizational and narrative work; the human handles the ground-truth verification. Neither does the whole job alone.

Regulatory Citation Hallucinations: A Special Compliance Risk

Regulatory citation hallucinations deserve separate attention because they create compliance exposure in a way that equipment ID errors typically do not. If an AI summary misidentifies a transformer and an operator catches it during the crosswalk, the consequence is a corrected work order. If an AI summary cites the wrong version of a NERC reliability standard in a compliance narrative that gets filed with NERC, the consequence may be a finding that the utility was citing superseded requirements, suggesting the compliance program is not tracking current standards.

The specific failure mode to watch for: the AI cites a NERC standard by its correct number but quotes the requirement from a prior version. For example, CIP-003-9 was a significant revision; a model trained on data that includes prior CIP-003 versions may accurately cite CIP-003 but describe requirements from CIP-003-7 rather than CIP-003-9. The citation looks right. The standard number is correct. But the compliance requirement described is not the currently enforceable text.

The same risk exists for FERC orders. FERC has issued numerous orders modifying interconnection procedures, transmission planning requirements, and market rules. An AI summary citing "FERC Order 2023" may be citing the right order number but describing the original language rather than the current text as modified by subsequent orders or compliance filings. In a rate case, quoting modified order text incorrectly is precisely the kind of error that opposing counsel will find.

The detection technique is simple but must be disciplined: whenever an AI document cites a NERC standard number, a FERC order, or a state tariff provision by specific section, look up that specific section in the official current text and compare it to what the AI quoted. This takes 90 seconds. It catches virtually all regulatory citation hallucinations. The problem is that it is easy to skip when the summary looks authoritative and you are working under time pressure. Building it into the step-by-step verification protocol, rather than treating it as optional, is the governance answer.

Worked Example: The Regulatory Summary That Almost Filed Wrong

A NERC compliance analyst at a mid-size IOU used a generative AI tool to draft a self-certification narrative for CIP-003-9. The AI produced a well-organized document: it identified the applicable requirements, described the utility's controls, and explained how those controls met the standard. The analyst reviewed it for clarity and logic. It read well. She was about to route it for signature.

Before routing, she ran her standard regulatory check: she pulled up the current CIP-003-9 text from the NERC Standards Library and compared the AI's description of the requirements against the actual enforceable text. The AI had described the requirement for transient cyber asset protections using language from an earlier version of the standard. The current enforceable text had a more specific scope: it applied to low-impact BCS sites, not just medium-impact sites, as the AI's description implied. Her utility had low-impact BCS sites that were now in scope under the correct reading of CIP-003-9 but were not covered in the AI's draft.

If she had filed the draft as written, her self-certification would have attested to compliance with a requirement it did not actually satisfy for her low-impact sites. The NERC compliance examiner reviewing the filing might have caught the scope error. Or they might not have, and the gap would have appeared as a violation during the next audit.

The revision took 45 minutes: she updated the scope description, added the low-impact site controls to the narrative, and had a second reviewer verify the specific language against the NERC Standards Library. The filing was accurate. The close call came from a simple discipline: checking the AI's regulatory citations against the official current text before filing.

Building the Verification Habit at the Team Level

Individual discipline is necessary but not sufficient. When an organization deploys AI tools broadly across operations and planning functions, the quality of hallucination detection depends on whether the organization has made verification a standard practice, not a personal choice. Three organizational mechanisms convert individual discipline into team-level reliability.

The first is the verified summary format. Rather than asking each individual to figure out what to verify, the organization defines a standard output format for AI-generated grid summaries that includes explicit citation slots: "Equipment IDs used: [list]. Verified against model book by: [name], [date]. SCADA state confirmed against EMS display: [yes/no]. Regulatory citations verified against: [specific source and version]." When the format requires verification evidence, the absence of that evidence is immediately visible to the reviewer. A summary without completed verification fields does not get routed for signature or action.

The second is the calibration event. Once or twice a year, the team reviews a set of AI-generated summaries that contains known errors, including hallucinated asset IDs and misquoted regulatory requirements, and each team member applies the detection protocol independently. The calibration event identifies who is catching the errors and who is missing them. It creates a shared reference for what verified-versus-unverified practice looks like. It also keeps the skills fresh: hallucination detection is a perishable skill if the AI output is generally good and team members start assuming verification is unnecessary.

The third is the incident review process. Whenever a hallucination makes it through to a filed document, a switching order, or a customer communication, the team conducts a brief review: which verification step failed to catch it, and what change to the protocol would have caught it? This is not a blame process; it is a quality improvement process applied to the human-AI workflow, the same way any operational team reviews near-misses. The hallucination that gets caught by a downstream reviewer is a near-miss. The hallucination that appears in a NERC self-certification that is filed is the event.

A utility that deploys AI broadly for grid work without these three mechanisms is relying on individual vigilance alone. That is not a governance strategy; it is a hope. And in a regulated, safety-critical environment, hope is not an audit answer.

Key Takeaways

• Grid summaries are high-risk hallucination territory because AI errors in this domain are structured and confident, following the right patterns while describing things that do not exist or requirements that have changed.
• The three authoritative sources for cross-checking grid summaries are the SCADA/EMS real-time database (for system state), the GIS and model book (for equipment existence and correct identifiers), and the current enforceable regulatory text (for standard requirements and order language).
• A five-step detection protocol covers all major hallucination categories: scan for asset identifiers, pull the parallel SCADA view, check regulatory citations against published text, apply the implausibility test, and document what was verified.
• The SCADA-GIS crosswalk is a structured technique for verifying every equipment identifier in a grid summary against two independent authoritative sources; it is efficient when applied in a defined order (substation names first, then feeder designations, then specific equipment IDs).
• Regulatory citation hallucinations are a special compliance risk because the AI may cite the right standard number while describing requirements from a superseded version; the detection technique is to look up the specific cited section in the official current text before any compliance document is filed.
• The appropriate human-AI workflow for grid summaries is for the AI to handle organizational and narrative work while the human handles ground-truth verification; neither does the whole job alone, and the verification step must be documented as part of the audit trail.
• Verification steps performed under time pressure are the ones most likely to be skipped; building the detection protocol into standard operating procedures rather than treating it as optional is the governance answer.