A state commission auditor requests the forecasting records for the prior 18 months. The utility's witness reaches for the folder and finds signed paper printouts for some months, email attachments for others, and nothing at all for a three-week stretch in August when the forecasting lead was on vacation. The commission's attorney notes the gap. The rate case's procurement justification is now in question not because the forecasts were wrong, but because the utility cannot prove they were reviewed.

Why Audit-Trail Architecture Matters

Every AI-assisted forecast that drives a procurement decision is a regulated act. In a rate case, an IRP proceeding, or a NERC reliability audit, the utility must be able to demonstrate that its forecasting process was rigorous, that a qualified human reviewed each output before it influenced a dollar commitment, and that the model used was appropriate for the conditions. A utility that cannot produce this evidence has the same regulatory exposure whether its forecasts were accurate or not. The audit trail is not a documentation exercise. It is the foundation of regulatory defensibility.

The audit trail for an AI-assisted forecasting pipeline must answer five questions for any historical forecast date:

1. What inputs did the model use (weather data, DER registry, SCADA history, and at what vintage)?
2. Which model version produced the output (training data cutoff, architecture, ensemble composition)?
3. Were any manual overrides or adjustments applied, and what was the stated rationale?
4. Who reviewed the output, what checklist did they follow, and when did they sign off?
5. What was the actual load outcome compared to the forecast, and was the MAPE logged?

If the answer to any of these five questions is "I don't know" or "we don't have that record," the audit trail is broken. A broken audit trail for a regulated procurement decision is a material governance failure.

The Anatomy of a Complete Sign-Off Record

The sign-off record is the atomic unit of forecast governance. It is the document that proves a human reviewed a specific model output before it drove a specific decision. It must be structured, version-controlled, and permanently retained.

A complete sign-off record contains the following fields:

Run identification: A unique run ID (universally unique identifier or UUID), the forecast date and horizon (the operating day and the 24 or 48-hour interval range covered), the pipeline run timestamp, and the weather file vintage and source identifier.

Model identification: The model version identifier for each ensemble member, the training data cutoff date for each member, and the ensemble weights used in the reconciliation. If a weighted combination was used, the weights for the specific run are recorded, not the default weights.

Data quality gate results: The pass/fail status of each quality gate check, the actual values that were checked (for example, weather file age in hours, SCADA completeness percentage, DER registry age in hours), and the timestamp of each check. If any check failed and was overridden, the name of the authorizing person and their stated rationale are recorded here.

Step-load guardrail results: Whether the guardrail fired (yes/no), the specific interconnection events checked, any adjustments applied (magnitude and interval range), the source document for each adjustment (the interconnection agreement identifier and schedule reference), and the name of the person who reviewed and applied the adjustment.

Manual adjustments: A log of all adjustments to the model's output beyond the step-load guardrail: the interval range affected, the adjustment magnitude, the reason (free text), and the name of the person making the adjustment. This field must be explicitly populated with "none" if no adjustments were made, not left blank. A blank field is ambiguous; an explicit "none" is a statement.

Review checklist completion: A binary flag for each checklist item, recording that the reviewer specifically addressed that item. The checklist items include: peak alignment with weather analogs, uncertainty band appropriateness, DER offset consistency with the registry, grid condition awareness (open outages, topology changes), and step-load event verification. The reviewer cannot complete the sign-off until all checklist items are marked.

Sign-off attestation: The reviewer's name (not a shared role account), their job title, the timestamp of sign-off, and if a second reviewer is required (for step-load events above a materiality threshold), the second reviewer's name and sign-off timestamp.

Output reference: The unique identifier of the forecast output file that was delivered to the energy desk or market interface, linking the sign-off record to the specific forecast that drove the downstream decision.

Model Version Control and the Governance Register

The sign-off record references model version identifiers that must exist in a durable, accessible model governance register. If the model version referenced in a sign-off record cannot be found in the governance register, the audit trail is incomplete, even if the sign-off record itself is complete. The model governance register is the second pillar of the audit-trail architecture.

Each entry in the model governance register covers a model version lifecycle event: deployment, retraining initiation, validation, and retirement. A complete entry for a model version deployment contains:

• The model version identifier (matching what appears in sign-off records)
• The deployment date and the name of the forecasting manager who approved deployment
• The training data period (start and end dates) and the training data source reference
• A summary of the holdout validation results, including MAPE by regime
• A summary of the parallel run results (if a prior model version was being replaced)
• The specific drift event or calendar trigger that prompted the retraining, if applicable
• The retirement date of the prior model version
• The next mandatory performance review date and the mandatory retraining date

The governance register is not a spreadsheet that lives in someone's personal drive. It is a shared, version-controlled document in a system that preserves the audit history of all changes. When a new entry is added, the prior entries are immutable. When the next mandatory review date is updated, the prior date is preserved in the history. This immutability is what allows the register to serve as audit evidence rather than just an internal tracking tool.

Override Logging: The Most Commonly Missing Piece

The most common audit trail gap in production forecasting pipelines is not missing sign-off records. It is missing override logs. Forecasters make manual adjustments to model outputs regularly: applying step-load corrections, adjusting for known operational conditions, compensating for detected drift. These adjustments are legitimate and expected. What is not acceptable is applying an adjustment without documenting it in the sign-off record.

Why does this happen? Forecasters learn to apply certain adjustments habitually, and habit discourages documentation. A forecaster who has been adding 50 MW to the evening Zone C forecast every day for three weeks may stop thinking of it as an "adjustment" and start thinking of it as the forecast. When the sign-off record template is filled out, the manual adjustment field is left blank because the forecaster does not consciously register the habitual modification as a deviation from the model output.

The technical solution is a diff check: before the sign-off record is submitted, the pipeline automatically compares the delivered forecast to the model's raw output. Any interval where the delivered number differs from the raw output by more than a tolerance (for example, 10 MW) is flagged and requires a manual adjustment entry with rationale before the sign-off can be completed. This check cannot be satisfied by typing "minor adjustment" in the rationale field; it requires a specific magnitude, a specific reason, and a specific period.

The governance solution is making override logging a professional norm rather than a bureaucratic requirement. Forecasters who understand that the override log is what allows the audit trail to accurately reflect what happened, not just what the model said, are more likely to maintain it consistently than forecasters who see it as a compliance box to check.

The Commission and Auditor Lens: What They Look For

When a state commission or a NERC auditor examines a utility's AI-assisted forecasting governance, they are not primarily looking for technical model quality. They are looking for evidence of a controlled, accountable process. The specific things they look for:

Consistency of process: Does the sign-off record look the same in January as it does in August? Are the same fields populated, the same checklist items completed, the same quality gate results logged? Inconsistency is a red flag: it suggests the process is followed when convenient and skipped when pressed for time.

Named accountability: Every sign-off record should have a specific named human being who reviewed it. Shared role accounts ([email protected]) are not acceptable because they do not identify who specifically exercised judgment. If the sign-off record shows "Forecasting Team" as the reviewer, an auditor will ask which member of the team actually reviewed it, and if nobody can answer, the accountability chain is broken.

Handling of exceptions: How did the utility handle the days when something unusual happened? The quality gate failure log, the step-load guardrail adjustments, and the manual override log tell the story of how the utility responded to non-routine situations. A utility with perfect sign-off records but no quality gate failure log may raise the question of whether the quality gate was functioning at all (since some failures are expected in any real pipeline).

Continuity during personnel changes: The August gap in the scenario at the opening of this lesson is a continuity failure. When the forecasting lead goes on vacation, who covers? The governance framework must include a designated backup who is trained on the sign-off process and authorized to complete sign-off records. The backup's sign-offs should be identifiable as backup-authorized (not simply the same process applied by a different person without context), and the governance record should show the backup authorization period.

Connection to outcomes: Does the utility track whether its AI-assisted forecasts were accurate? The MAPE feedback log, linked to the sign-off records through the run ID, demonstrates that the utility closes the loop between forecast and outcome. A utility that signs off on forecasts but never reviews their accuracy has no way to demonstrate that the sign-off process was improving decisions rather than just creating paper.

Worked Example: Building the Audit Trail for a Summer Peak Procurement

On July 18, the utility's day-ahead forecast shows a peak demand of 13,950 MW for the following afternoon, with a 13,650 to 14,250 MW confidence band. The energy desk uses this forecast to commit to a 500 MW bilateral capacity purchase to cover the afternoon peak. Four months later, the commission opens a docket questioning whether the procurement was necessary (actual peak on July 19 was 13,420 MW, 530 MW below the committed 13,950 MW).

The audit trail for this procurement decision needs to answer the commission's question: was the forecast reasonable given what was known on July 18?

The sign-off record for July 18 shows: weather file vintage 4.2 hours at 5:00 a.m. sign-off (within the 6-hour quality gate), SCADA completeness 99.6 percent, DER registry age 18 hours. Ensemble weights: model A at 0.55, model B at 0.35, model C (statistical baseline) at 0.10. Step-load guardrail: no large interconnections in the 48-hour window. Manual adjustment: the forecaster added 80 MW to the 3 p.m. to 5 p.m. intervals, noting "NWP model run from 0Z shows 96 degrees at hour 15, up from 93 degrees in 12Z run; adjusting for expected increased HVAC load based on temperature sensitivity of 10 MW per degree." Checklist: all items marked. Sign-off by J. Rivera at 5:47 a.m.

The model version record shows: Model Version 2.4, trained on data through April 30, 2026, holdout MAPE 1.4 percent overall, 1.7 percent for summer peak days.

The MAPE feedback log shows: actual July 19 peak was 13,420 MW. The point forecast of 13,950 MW was 530 MW (3.8 percent) above actual. The 0Z weather run that J. Rivera used to justify the 80 MW upward adjustment showed a temperature that was ultimately not reached (actual afternoon high was 89 degrees, not the 96 degrees forecast). The weather forecast itself was wrong.

The audit trail tells a defensible story: the forecaster followed the documented process, made a reasonable adjustment based on a weather model revision that later proved incorrect, and the resulting error is attributable to the weather forecast provider's accuracy on that day, not to a failure of the forecasting governance process. The commission's question is answered: the procurement was a reasonable response to the information available at 5:47 a.m. on July 18.

Retention, Access, and the Regulatory Timeline

Audit trail records must be retained for as long as the decisions they document can be examined. For rate-case-related procurement decisions, the relevant standard is typically the commission's record retention requirement for rate case evidence, which in most jurisdictions is 7 to 10 years from the date of the final rate order. For NERC reliability audits, the standard varies by standard but is typically three to six years. The practical standard for a forecasting governance system is to retain all records for 10 years, which covers the most demanding regulatory retention requirement in most jurisdictions.

Access controls are a governance control in their own right. Sign-off records must be readable by auditors but not editable after completion. The model governance register must be editable only by authorized personnel, with every change logged with the editor's identity and timestamp. Weather files and DER registry files used in historical runs must be archived in a read-only format that preserves their exact content at the time of the run, verifiable through cryptographic hashing.

The governance system must have a defined response time to regulatory requests. A commission subpoena or a NERC audit request may require the utility to produce specific records within 24 to 72 hours. A governance system where records are stored in multiple locations (some in the energy desk's SharePoint, some in the forecasting team's file server, some in the IT department's archive) cannot meet this requirement reliably. A single integrated system with a defined query interface that can retrieve all records for a specified date range within minutes is the standard that serious governance architecture should target.

Finally, the governance records must survive personnel and system changes. A governance system that depends on a single person's knowledge of where the records are stored is not durable. Document the governance system architecture: where records are stored, how to retrieve them, who has access, and how access is granted and revoked. Test the retrieval process at least annually by simulating a regulatory request for records from a specific date range and timing the retrieval. If the retrieval takes more than an hour, the governance architecture needs improvement.

Key Takeaways

• The audit trail for an AI-assisted forecast must answer five questions for any historical date: what inputs were used, which model version ran, were overrides applied and why, who reviewed and signed off, and what was the actual outcome? A gap in any answer is a governance failure.
• The sign-off record is the atomic unit of forecast governance. It must include named human accountability, a completed checklist, explicit override documentation (including "none" when no overrides were applied), and a link to the output file that was delivered to the decision-maker.
• The model governance register is the second pillar. It must document every model version's lifecycle from deployment to retirement, with training data periods, validation results, and the drift events or calendar triggers that drove retraining decisions.
• Override logging is the most commonly missing piece in production pipelines. The diff check (comparing the delivered forecast to the raw model output) is the technical mechanism that makes undocumented habitual overrides impossible.
• Commissions and NERC auditors look for consistency of process, named accountability, documented exception handling, continuity during personnel changes, and evidence that accuracy outcomes are tracked. A utility that can show all five is in a strong position regardless of whether individual forecasts were accurate.
• The audit trail is not retrospective documentation. It is built in real time, run by run, as a by-product of the governed pipeline. A pipeline that requires the governance records to be reconstructed after the fact is not governed; it is documented after the fact.
• Continuity of the governance process during vacations, personnel changes, and operational disruptions requires a designated backup procedure that is tested and documented before it is needed. An August gap in the sign-off record is not a vacation problem; it is a governance design failure.