Picture a senior planner at a regional transmission organization. She asks an AI assistant to summarize the key cost-allocation provisions of a recently filed tariff revision. The assistant returns three polished paragraphs, cites specific attachment numbers, and even includes a table. Everything looks right. Two of the five numbers in the table are wrong. Not slightly off: structurally wrong, because the model was drawing on a prior version of the tariff it had seen during training. She almost filed that summary with the commission. Understanding exactly why that happened, at a mechanical level, is what this lesson teaches.

What Generative AI Actually Is

The term "generative AI" describes a class of machine learning models that produce new outputs, typically text, given a prompt. The most widely deployed examples are large language models (LLMs), which include the systems behind ChatGPT, Claude, Gemini, and the AI assistants increasingly embedded in utility software platforms. Understanding how these work at a conceptual level, without any mathematics, is the foundation for using them reliably in grid work.

A large language model is, at its core, a very large statistical table of relationships between words, phrases, and concepts. During training, the model was shown an enormous quantity of text: web pages, books, technical documents, regulatory filings, code, and much more. From that training corpus, it learned that certain words and phrases tend to follow other words and phrases in certain contexts. When you give it a prompt, it uses those learned relationships to predict, one piece at a time, what text should come next. The technical unit is called a token, roughly corresponding to a word or word fragment. The model generates tokens sequentially, each one predicted from the preceding context.

This is not understanding. The model has no internal representation of the power system, no model of physical causation, and no concept of what a transmission line actually does. It has a very sophisticated map of how words about transmission lines are used relative to other words in the documents it was trained on. When a question resembles a question it has seen before, it produces text that resembles the answers it has seen. When a question is novel, it produces text that is statistically plausible given what it knows, which may or may not be accurate.

The Training Cutoff Problem

Every large language model has a training data cutoff: a date after which it saw no new information during training. For grid work, this matters in a specific and concrete way. Tariffs get revised. NERC standards get updated. FERC issues orders. State commissions issue decisions. Regulatory costs get reallocated. The model does not know about changes that occurred after its cutoff, and it will not tell you it is drawing on outdated information, because it has no mechanism for recognizing that a revision occurred. It will produce text about the latest filing it saw during training with the same confidence it would produce text about the current filing.

This is not a problem unique to utilities. But the consequences are uniquely serious in a regulated industry where specific version-sensitive requirements carry legal and financial weight. The planner in the opening scenario encountered exactly this failure: the model's training data included a prior tariff version, and nothing in its architecture caused it to flag the possibility that a newer version existed.

Tokens and Why They Matter for Grid Output

The token-by-token generation process creates predictable failure patterns that every energy professional using an AI assistant should know about.

Because the model generates the next token based on the preceding context, it is vulnerable to "path dependency": once it has generated text that commits to a certain claim (say, a particular tariff attachment number), subsequent tokens will tend to be consistent with that claim, even if the claim is wrong. The model does not backtrack and reconsider. It commits forward. This is why AI output can contain errors that are internally consistent: the invented attachment number in paragraph one spawns corresponding invented references in paragraphs two and three, because consistency is what the model has learned to produce.

Tokens also determine what the model can "see" at once. LLMs process text in chunks called context windows. An early GPT-3 model had a context window of roughly 4,000 tokens (about 3,000 words). Current models handle 100,000 or more tokens. For grid work, context window size matters when you are asking a model to analyze a large document: a 200-page interconnection study or a complete NERC standards manual. If the relevant information is near the edges of the context window, model performance can degrade in ways that are difficult to detect without testing.

Understanding tokenization also explains why AI models sometimes struggle with precise numerical calculations. A number like "345,000" is stored as tokens that represent its visual appearance in text, not as a value in a numerical computation. When a model appears to be doing arithmetic, it is actually pattern-matching against how arithmetic is expressed in the text it was trained on. Errors in AI-generated numerical tables and calculations in utility documents are common, and they are a direct consequence of this architecture.

Why Confidence and Accuracy Are Uncoupled

This is the most important architectural fact for a utility professional to internalize: the fluency and confidence of a generative AI output is structurally independent of its accuracy.

The model is optimized during training to produce text that looks like correct, authoritative, well-structured writing. It learned this by training on large quantities of text that was, in fact, correct, authoritative, and well-structured. But the model was never trained with a separate objective of "flag your uncertainty." It was trained to produce text that resembles the text in its training corpus. Correct text, in most domains, sounds confident. So the model produces confident text, whether or not the underlying facts are right.

Some AI systems layer additional mechanisms on top of the base model to produce uncertainty signals. These can be useful. But they are not part of the core generation architecture, and they are imperfect. In practice, the discipline for grid professionals is simple and non-negotiable: assume AI output may be wrong, regardless of how confident it sounds, and verify every factual claim against a primary source.

Fluency is not accuracy. An AI output that reads like something a senior engineer would write may have been written by a model that has no idea what a senior engineer knows. The model learned the style; it did not acquire the judgment.

How Generative AI Fits Into the Grid Workflow

Given this architecture, where does generative AI actually belong in a planner's or operator's workflow?

The honest answer is: in the draft layer, with mandatory human verification before the output moves into any consequential process. Here is what that looks like in practice for each of the major grid use cases.

Load forecast narrative drafting. A planner uses an AI assistant to turn a quantitative forecast (a table of hourly MW values) into the prose narrative that accompanies the forecast in a planning presentation or IRP filing. The AI produces fluent, well-structured text that describes the forecast's key features, assumptions, and uncertainties. The planner reviews every specific number, every claim about growth drivers, and every reference to a previous forecast or historical period. The AI saves two hours of drafting; the planner's verification takes thirty minutes.

Interconnection study report drafting. An interconnection engineer uses an AI assistant to produce the boilerplate sections of a study report: scope description, methodology overview, regulatory framework, and appendix formats. These sections are structurally predictable and largely independent of the specific study results. The engineer provides the actual power-flow findings, and the AI drafts the surrounding document. The engineer reviews every output, with particular attention to tariff citations, applicable standards, and any interpolated technical conclusions. The AI draft is treated as a first pass, not a deliverable.

Compliance narrative drafting. A NERC compliance lead uses an AI assistant to draft the evidence narrative for a self-certification. The AI produces text describing the applicable requirement and what the organization does to meet it. The compliance lead checks every requirement citation against the current standard text, every description of controls against the actual documented procedures, and every specific claim against the evidence record. Any sentence the compliance lead cannot independently verify is either rewritten with verified content or removed.

Regulatory monitoring. A regulatory affairs professional uses an AI assistant to summarize a FERC order or state commission decision. The AI produces a clear, readable summary. The professional checks every specific holding cited, every docket number referenced, and every compliance deadline mentioned against the primary document. The summary is a reading aid, not a substitute for reading.

Retrieval-Augmented Generation: The Grounding Approach

The training cutoff problem and the hallucination risk can be significantly reduced, though not eliminated, by a technique called Retrieval-Augmented Generation (RAG). In a RAG architecture, the AI system is connected to a specific, controlled document database. When you ask a question, the system first retrieves the relevant documents from that database (the current tariff, the applicable NERC standard, the specific asset record) and then generates its answer using those retrieved documents as context. The model is grounding its output in current, controlled-access documents rather than in its training memory.

For utility applications, this is the architecture that makes generative AI substantially more reliable for regulatory work. An AI assistant that is RAG-connected to the current FERC-filed tariff, the current NERC standards library, and your organization's documented asset records is structurally less likely to hallucinate a superseded provision than one that is working from training memory alone. It can still make errors (retrieval can miss relevant sections, and the model can still misinterpret retrieved text), but the failure mode is less catastrophic.

This matters for procurement decisions. When evaluating AI tools for regulatory drafting, compliance support, or tariff analysis, the question "is this RAG-connected to current regulatory sources, and who maintains that database?" is more important than the demo's impressive fluency. A beautifully fluent answer derived from a stale document store is still a stale answer.

A Worked Example: Watch the Hallucination Form

Walk through this scenario step by step to see how a hallucination forms in practice.

A planning manager pastes a paragraph about Order 2023 into an AI chat window and asks: "Summarize the key cost allocation changes for storage resources connecting under Attachment TT in PJM." The model has been trained on large quantities of FERC orders, PJM tariff text, and interconnection practitioner commentary. It produces a confident four-paragraph summary with specific percentage figures for cost sharing.

What is happening inside the model? It is retrieving its strongest statistical associations for the concepts "Order 2023," "Attachment TT," "PJM," "storage," and "cost allocation." These associations were formed during training on documents from before its cutoff date. PJM filed an Order 2023 compliance tariff revision after the model's training cutoff. The model has no knowledge of the revision. It is generating text that is statistically consistent with what it learned about the pre-revision tariff. The percentages it produces are plausible given that training context, but they do not reflect the current filed tariff.

Now watch an informed planner use the same tool productively. She asks the same question but adds: "Note: I will verify all specific provisions against the current PJM OATT. Flag any provision where the answer may depend on post-2023 filings." The model, prompted to flag uncertainty, will often acknowledge the training cutoff risk for recent regulatory questions. The planner then takes the structural framework the AI provided (the categories of cost allocation issues to look for) and looks up each one in the current PJM OATT on the PJM website. The AI organized her research; it did not replace it.

This is the shift that separates a professional who uses AI productively from one who uses it dangerously: the AI produces a structured draft or a research scaffold, and the professional's domain expertise, verified against current primary sources, is what makes the output reliable.

What This Means for the Working Planner

The lesson here is not that generative AI is unusable for utility work. It is already in use, productively, at utilities and ISOs that have developed the discipline to use it correctly. The lesson is that the discipline is specific and learnable, and that the professionals who learn it early will do significantly better work, faster, than those who rely on the tool naively.

The architectural facts matter because they tell you exactly where to apply your verification effort. Token-by-token generation with path dependency tells you to check the early paragraphs of any AI-drafted document most carefully, because early errors propagate consistently. The training cutoff tells you to treat any regulatory citation as potentially outdated and to verify it against the current primary source. The uncoupled confidence tells you that the tool's fluency is never evidence of accuracy. And the RAG architecture tells you what to ask vendors: "Is your tool connected to current regulatory sources, how are those sources maintained, and who is responsible for keeping them current?"

There is also an important message here for professionals who feel intimidated by the pace of AI adoption in their organizations. You do not need to understand the mathematics of transformer architectures to be an effective, critical user of generative AI tools. You need to understand the four architectural facts in this lesson: training cutoffs, path dependency, uncoupled confidence, and the RAG grounding approach. Armed with those, you can ask the right questions, set the right verification standards, and produce reliable work in a world where AI drafting is becoming the default first step. That is what this program is building toward: not technical mastery of machine learning, but professional mastery of AI-assisted practice in a reliability-first, regulated-industry context.

A Practical Checklist for AI-Drafted Regulatory Content

Before any AI-drafted utility document is used in a filing, submission, or operational decision, a qualified professional should apply this checklist. It takes the architectural insights from this lesson and translates them into a repeatable practice.

1. Date check. Is the document's regulatory content (tariff citations, standard versions, commission orders) potentially affected by changes after the AI tool's training cutoff? If yes, every citation requires primary-source verification.
2. Numerical audit. Every specific number in the document (MW values, percentages, cost figures, dates) gets traced to the primary source that was supposed to produce it. AI-generated numbers are pattern-matched, not computed.
3. Asset identifier check. Every equipment identifier, substation name, line segment number, or asset code gets verified in the current asset management system or GIS. Hallucinated identifiers in operational documents send crews to wrong locations.
4. Citation verification. Every regulatory citation (NERC standard version, tariff attachment number, FERC order number, docket reference) gets verified against the current primary document. Note whether the citation is to the currently enforceable version.
5. Early-paragraph scrutiny. Given path dependency, the first paragraph of each major section receives the most careful review. An error in the opening framing will propagate consistently through the section.
6. Provenance note. The document's review record notes that AI was used in drafting, which specific sections, and what verification steps were applied. This creates the audit trail that demonstrates human accountability.

This checklist is not a burden that slows down AI-assisted work. It is the framework that makes AI-assisted work reliable. With practice, it becomes second nature, and the combined productivity of AI drafting plus systematic verification typically outperforms the previous all-manual workflow even with the verification step included.

Key Takeaways

• Large language models generate text token by token by predicting what word or phrase is most likely to come next, given learned statistical relationships. They do not reason, they do not understand, and they do not have knowledge of physical causation.
• Every LLM has a training data cutoff. It has no knowledge of regulatory changes, tariff revisions, or NERC standard updates after that date, and it will not flag this limitation unless prompted to do so.
• Confidence and accuracy are structurally uncoupled in generative AI. The model produces fluent, authoritative-sounding text regardless of whether its content is correct. Fluency is never a proxy for accuracy in grid work.
• Path dependency means that once a model commits to an incorrect claim in its output, subsequent tokens will tend to be consistent with that error, producing internally coherent but incorrect documents.
• Numerical calculations in AI output are especially unreliable because numbers are represented as text tokens, not as values. Every figure in an AI-generated table should be verified against the primary data source.
• Retrieval-Augmented Generation (RAG) significantly reduces hallucination risk by grounding model output in current, controlled documents. For utility regulatory work, RAG-connected tools connected to live regulatory sources are substantially more reliable than base-model assistants working from training memory alone.
• The productive use pattern for generative AI in planning is: AI drafts the structure, the planner verifies the facts. The AI is a draft accelerator, not a source of truth.