Overview

Your AI assistant generates a comprehensive nginx configuration for your web tier. It includes SSL pinning, caching directives, rate limiting, and module directives that all sound plausible. The syntax is correct. It passes a config validator. One of your engineers deploys it, and the service becomes unreachable. Investigation reveals that the configuration includes three directives that don't exist in your nginx version, and one references a non-existent SSL certificate file path. The AI was confident, specific, and completely wrong.

This is hallucination: AI generating artifacts that look legitimate but contain fundamental errors, nonexistent features, impossible parameter combinations, version-mismatched syntax, or fabricated command-line flags. Unlike syntax errors that fail loudly, hallucinations often fail silently or produce subtle, hard-to-debug problems.

Purpose

Hallucination detection for IT artifacts is different from general hallucination detection. In a conversation, an AI hallucination might be factually wrong but harmless. In IT Operations, a hallucination in a configuration or script can silently corrupt data, block critical traffic, or create security vulnerabilities. You need detection methods that catch AI-generated artifacts where the syntax is correct but the content is false.

This lesson covers how hallucinations manifest in IT artifacts, how to detect them before they reach production, and how to build validators and cross-referencing systems that catch fabricated configurations. By the end, you'll have detection tooling that catches hallucinated features, parameters, and references that manual review would miss.

Why This Matters

Hallucinations are harder to catch than syntax errors because they look legitimate. A hallucinated nginx directive sounds plausible because it follows the naming patterns you see in real directives. A fake Kubernetes API field looks right because it fits the YAML structure. A nonexistent command-line flag is spelled correctly. Everything passes until it fails in production.

The danger is asymmetrical: a human reviewer might read "ssl_session_timeout 100d" and not question it without checking documentation. The configuration will load, the service will start, but the setting won't actually work. It will fail silently and inconsistently.

From a risk perspective, hallucinated configs are worse than bad configs that fail loudly. When something fails loudly, you know there's a problem and you fix it. When something fails silently, it creates technical debt and security gaps that aren't discovered until they cause expensive problems.

From an operational perspective, if your team starts seeing AI-assisted configs that are sophisticated-looking but internally fabricated, they lose trust in AI assistance entirely. They'll reject otherwise good suggestions because they can't distinguish between hallucinations and legitimate improvements.

Core Concepts

Key insight: Hallucinations are systematic but not random. AI doesn't generate completely nonsensical directives. It extrapolates from patterns it has seen. It creates plausible-sounding alternatives when it doesn't know the actual syntax. It references features that exist in similar tools but not in the specific version you're using. These patterns are detectable.

1. Types of Hallucinations in IT Artifacts

Nonexistent directives or parameters: The AI generates a configuration option that doesn't exist in any version of the software.

Real directive that exists
proxy_set_header X-Real-IP $remote_addr;

Hallucinated directive (plausible but doesn't exist)
proxy_set_header_if_not_empty X-Real-IP $remote_addr;

The hallucinated version follows the naming pattern of real directives, so it looks legitimate. But no version of nginx has this directive.

Version-mismatched syntax: The AI generates syntax that's correct in a newer version but doesn't work in your current version.

Kubernetes 1.20 syntax (real)
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy

Kubernetes 1.16 syntax (real)
apiVersion: networking.k8s.io/v1beta1
kind: NetworkPolicy

Hallucinated: API version that doesn't exist
apiVersion: networking.k8s.io/v1.2
kind: NetworkPolicy

Your cluster runs 1.16, but the AI generated 1.20 syntax with a hallucinated intermediate version.

Impossible parameter combinations: Parameters that are valid individually but cannot be used together.

-- Both parameters exist, but they're mutually exclusive
ALTER TABLE users ROW_FORMAT=COMPRESSED COMPRESSION=ZSTD;
-- (ZSTD compression is only available with InnoDB, but ROW_FORMAT=COMPRESSED is MyISAM-specific)

Fabricated file paths or references: The configuration references files, certificates, or services that don't exist in your environment.

resource "aws_security_group" "app" {
ingress {
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["10.0.0.0/8"]
}
}

resource "aws_lb_listener" "https" {
certificate_arn = "arn:aws:acm:us-east-1:123456789012:certificate/12345678-1234-1234-1234-123456789012"
# ↑ This specific certificate doesn't exist; the ARN is plausible but wrong
}

Missing required dependencies: The configuration references features that require libraries, modules, or extensions that aren't installed.

Requires 'requests' library
import requests
response = requests.get('https://api.example.com')

AI-generated code that assumes 'requests' is installed
# but your container doesn't have it, and the script doesn't check

Logic errors disguised as configuration: The configuration is syntactically correct but implements the wrong logic.

AI generates: "backup only files modified in last 7 days"
find /data -mtime -7 -type f -exec tar -cf /backup/backup.tar {} \;
# ↑ This is wrong; it creates 1 tar per file instead of 1 tar with all files
# Correct version:
find /data -mtime -7 -type f | tar -cf /backup/backup.tar -T -

Key insight: The most dangerous hallucinations look exactly like correct configurations but are subtly wrong in ways that matter.

2. Detection Strategy: Cross-Referencing Against Official Documentation

The most reliable way to catch hallucinations is to verify every claim the AI makes against the official documentation or reference implementation.

┌─────────────────────────────────────────┐
│ AI-Generated Configuration │
└─────────────────────────────────────────┘
↓
┌─────────────────────────────────────────┐
│ 1. Extract all directives, parameters │
└─────────────────────────────────────────┘
↓
┌─────────────────────────────────────────┐
│ 2. For each element: │
│ - Check against official docs │
│ - Verify version compatibility │
│ - Check dependencies │
│ - Validate parameter combinations │
└─────────────────────────────────────────┘
↓
┌─────────────────────────────────────────┐
│ 3. Report discrepancies │
└─────────────────────────────────────────┘

This requires maintaining reference data: the official documentation for each tool you use, in each version you deploy. This is maintenance overhead, but it's cheaper than production incidents.

3. Pattern Recognition for Hallucinated Directives

Hallucinations follow predictable patterns. You can detect them by looking for:

Naming pattern violations: Real directives follow consistent naming conventions within each tool. Nginx uses underscores (proxy_set_header). Kubernetes uses camelCase (proxySetHeader). MySQL uses underscores (max_connections). When you see a directive that breaks the naming pattern, it's likely hallucinated.

Real nginx directives: proxy_set_header, proxy_cache, proxy_pass
Hallucinated pattern: proxy-set-header (uses hyphen instead of underscore)

Overly specific parameters: AI tends to invent parameters that are more specific than they need to be. Real tools usually have simpler, more general parameters.

Real: ssl_protocols TLSv1.2 TLSv1.3;
Hallucinated: ssl_protocols_strict_mode TLSv1.2 TLSv1.3 verify_hostname=true;

Parameter units without documentation: Real configurations specify units (seconds, bytes, percentage) in documented ways. AI sometimes invents unit specifications that don't match the tool's syntax.

Real: timeout 30s;
Hallucinated: timeout 30 seconds; // or timeout 30s (UTC);

Features that sound too powerful: When an AI generates a directive that would be incredibly useful but you've never heard of, it's often hallucinated. Real tools typically document their powerful features extensively.

Hallucinated: autoscale_based_on_prediction tomorrow;
// (Sounds amazing, doesn't exist)

Key insight: If a configuration feature is powerful enough to solve a major problem you've had, verify it extensively before trusting it. Hallucinations often target real pain points.

4. Automated Validation Against Reference Data

Build validators that check AI-generated artifacts against reference data you maintain:

Reference Data Structure:
├── nginx/
│ ├── 1.18/
│ │ ├── directives.json (valid directives)
│ │ ├── parameters.json (valid parameters for each directive)
│ │ ├── dependencies.json (which directives require which modules)
│ │ └── versions_compat.json (which features exist in which versions)
│ ├── 1.20/
│ │ ├── directives.json
│ │ └── ...
├── kubernetes/
│ ├── 1.16/
│ │ ├── api_versions.json
│ │ ├── fields_per_kind.json
│ │ └── deprecations.json
└── ...

Then validate:

For each directive in the AI-generated config:
directive="proxy_set_header"
version="nginx 1.18"

Check: does this directive exist in this version?
grep -q "$directive" "reference_data/${tool}/${version}/directives.json" || \
report_hallucination "Directive doesn't exist"

Check: does the parameter combination make sense?
validate_parameter_combination "$directive" "$param1" "$param2" || \
report_hallucination "Invalid parameter combination"

Check: what modules does this directive require?
required_modules=$(jq ".directives[\"$directive\"].requires_modules" \
"reference_data/${tool}/${version}/dependencies.json")

Verify those modules are loaded in the config
for module in $required_modules; do
grep -q "load_module.*$module" "$config" || \
report_hallucination "Missing required module: $module"
done

5. Reference Cross-Checking: When AI References External Systems

When AI-generated configs reference external systems (certificate paths, database endpoints, API keys), verify those references actually exist:

If config references a certificate file:
ssl_certificate /etc/ssl/certs/example.com.crt;
# Verify the file exists
[ -f /etc/ssl/certs/example.com.crt ] || \
report_hallucination "Certificate file doesn't exist"

If config references a DNS name:
upstream backend {
server internal-api.service.local:8080;
}
# Verify the DNS name resolves
dig +short internal-api.service.local @dns-server || \
report_hallucination "DNS name doesn't resolve"

If config references a service account:
serviceAccountName: ai-generated-account
# Verify the service account exists
kubectl get serviceaccount ai-generated-account || \
report_hallucination "Service account doesn't exist"

6. The "Trust but Verify" Toolchain

Build a toolchain that systematically verifies AI claims:

Stage 1: Syntax validation (quick, catches obvious errors)

• Does it parse?
• Are all bracket types balanced?
• Do quoted strings close properly?

Stage 2: Reference validation (medium, catches hallucinated directives)

• Do all directives exist in the reference documentation?
• Are parameters valid for those directives?
• Are all version-specific features available in your version?

Stage 3: Integration validation (slow, catches real-world failures)

• Load the config into a test instance
• Verify all referenced external resources exist
• Test actual behavior against expected behavior

Stage 4: Human review (requires judgment)

• Are there edge cases the validation missed?
• Does the logic make sense given your operational context?
• Are there subtle bugs that automated validation can't catch?

Practical Use Cases

Use Case 1: Detecting Hallucinated Kubernetes Manifests

Scenario: Your AI assistant generates a Kubernetes deployment manifest that includes several custom resource definitions. The YAML parses correctly, but some fields might not exist in your cluster's API version.

Without detection:

• Deploy the manifest
• Kubernetes accepts part of it, rejects part of it
• Some pods start, others fail
• Troubleshooting is confusing because the error messages are vague

With detection:

#!/usr/bin/env python3
# validate_k8s_manifest.py
import yaml
import subprocess
import json

def get_api_schema(resource_kind, api_version):
"""Get the schema for a specific Kubernetes resource from cluster"""
cmd = f"kubectl explain {resource_kind} --recursive"
result = subprocess.run(cmd, shell=True, capture_output=True, text=True)
return result.stdout

def validate_manifest(manifest_file, cluster_version):
"""Validate AI-generated manifest against cluster version"""

with open(manifest_file) as f:
docs = yaml.safe_load_all(f)

failures = []

for i, doc in enumerate(docs):
if not doc:
continue

kind = doc.get('kind')
api_version = doc.get('apiVersion')

Check 1: API version exists
cmd = f"kubectl api-resources | grep -i {kind}"
if subprocess.run(cmd, shell=True).returncode != 0:
failures.append(f"Resource type not available: {kind}")
continue

Check 2: Fields are valid for this kind/version
spec = doc.get('spec', {})
schema = get_api_schema(kind, api_version)

for field in spec.keys():
if field not in schema:
failures.append(
f"{kind}.spec.{field} doesn't exist in {api_version}"
)

Check 3: Referenced resources exist
if kind == 'Deployment':
svc_account = doc.get('spec', {}).get('template', {}).get(
'spec', {}
).get('serviceAccountName')
if svc_account:
cmd = f"kubectl get serviceaccount {svc_account}"
if subprocess.run(cmd, shell=True).returncode != 0:
failures.append(f"ServiceAccount not found: {svc_account}")

if failures:
print("HALLUCINATIONS DETECTED:")
for f in failures:
print(f" - {f}")
return False
return True

if __name__ == '__main__':
import sys
valid = validate_manifest(sys.argv[1], sys.argv[2] if len(sys.argv) > 2 else None)
sys.exit(0 if valid else 1)

Result: Manifests that contain hallucinated API fields are rejected before deployment. The AI learns to generate only valid fields.

Use Case 2: Detecting Hallucinated MySQL Configuration

Scenario: AI generates a MySQL configuration with directives that sound plausible but don't exist in your MySQL version.

Without detection:

• Add config to my.cnf
• MySQL starts but ignores the unrecognized directives
• Performance doesn't improve as expected
• Troubleshooting takes hours to determine the directives weren't actually applied

With detection:

#!/bin/bash
# validate_mysql_config.sh

MYSQL_VERSION=$(mysql --version | grep -oE '[0-9]+\.[0-9]+\.[0-9]+')
CONFIG_FILE="$1"

Reference data: valid MySQL directives for each version
VALID_DIRECTIVES="$(curl -s https://dev.mysql.com/doc/${MYSQL_VERSION}/en/server-system-variables.html \
| grep -oE 'name="[a-z_]+"' | cut -d'"' -f2 | sort -u)"

echo "Validating against MySQL $MYSQL_VERSION"

HALLUCINATIONS=0
while IFS='=' read -r key value; do
# Skip comments and empty lines
[[ "$key" =~ ^[[:space:]]*# ]] && continue
[[ -z "$key" ]] && continue

key=$(echo "$key" | xargs) # trim whitespace

Check if this directive exists in this MySQL version
if ! echo "$VALID_DIRECTIVES" | grep -q "^${key}$"; then
echo "HALLUCINATION: Unknown directive '$key' for MySQL $MYSQL_VERSION"
HALLUCINATIONS=$((HALLUCINATIONS + 1))
fi
done < "$CONFIG_FILE"

if [ $HALLUCINATIONS -gt 0 ]; then
echo "Found $HALLUCINATIONS hallucinated directives"
exit 1
fi

echo "Configuration validated successfully"
exit 0

Use Case 3: Detecting Hallucinated Terraform References

Scenario: AI generates Terraform that references AWS resources that don't exist or uses incorrect ARN formats.

Without detection:

• Deploy Terraform
• terraform apply fails with cryptic error messages
• Team spends hours debugging incorrect ARN formats or missing resources

With detection:

#!/bin/bash
# validate_terraform_references.sh

terraform_file="$1"

echo "Checking for hallucinated resource references..."

Extract all resource references from Terraform
RESOURCE_REFS=$(grep -oE 'aws_[a-z_]+\.[a-z_]+' "$terraform_file" | sort -u)

HALLUCINATIONS=0

for ref in $RESOURCE_REFS; do
resource_type=$(echo "$ref" | cut -d. -f1)

Check: is this a valid AWS resource type?
if ! aws ec2 describe-instances --dry-run 2>&1 | grep -q "$resource_type" && \
! [[ "$resource_type" =~ ^aws_(instance|security_group|s3_bucket|rds|vpc)$ ]]; then
echo "HALLUCINATION: Unknown resource type '$resource_type'"
HALLUCINATIONS=$((HALLUCINATIONS + 1))
fi
done

Check for ARN format errors
ARN_REFS=$(grep -oE 'arn:aws:[a-z-]+:[a-z0-9-]*:[0-9]{12}:[a-z/:-]+' "$terraform_file")
for arn in $ARN_REFS; do
# ARN must have exactly 6 colon-separated parts
if [ "$(echo "$arn" | grep -o ':' | wc -l)" -ne 5 ]; then
echo "HALLUCINATION: Invalid ARN format '$arn'"
HALLUCINATIONS=$((HALLUCINATIONS + 1))
fi
done

if [ $HALLUCINATIONS -gt 0 ]; then
echo "Found $HALLUCINATIONS hallucinated references"
exit 1
fi

echo "Terraform references validated"
exit 0

Examples

Example 1: Hallucination Detection Checklist for nginx Configuration

#!/bin/bash
# Check nginx config for common hallucinations

CONFIG="$1"
NGINX_VERSION=$(nginx -v 2>&1 | grep -oE '[0-9]+\.[0-9]+')

check_directive() {
local directive="$1"
local min_version="$2"

Check if directive exists in config
if grep -q "^\s*${directive}\s" "$CONFIG"; then
# Check if this version supports it
if (( $(echo "$NGINX_VERSION < $min_version" | bc -l) )); then
echo "HALLUCINATION: '$directive' not available in nginx $NGINX_VERSION (requires $min_version+)"
return 1
fi
fi
return 0
}

Check for known hallucinated directives in nginx
echo "Checking for hallucinated nginx directives..."

These directives don't exist in any version of nginx
hallucinated_directives=(
"proxy_set_header_secure"
"auto_ssl"
"rate_limit_by_ip_country"
"compress_if_cacheable"
"health_check_path"
)

for directive in "${hallucinated_directives[@]}"; do
if grep -q "^\s*${directive}\s" "$CONFIG"; then
echo "HALLUCINATION: Directive '$directive' does not exist in any nginx version"
fi
done

Check for parameter combinations that are impossible
echo "Checking for impossible parameter combinations..."

gzip and gzip_static can't both be on
if grep -q "gzip on" "$CONFIG" && grep -q "gzip_static on" "$CONFIG"; then
echo "WARNING: gzip and gzip_static can conflict in some scenarios"
fi

Check for SSL directives that reference non-existent files
echo "Checking for hallucinated SSL file references..."
while IFS= read -r line; do
if [[ "$line" =~ ssl_certificate ]]; then
cert_path=$(echo "$line" | grep -oE '/[^ ;]+')
if [ ! -f "$cert_path" ]; then
echo "HALLUCINATION: Certificate file doesn't exist: $cert_path"
fi
fi
done < "$CONFIG"

echo "Hallucination check complete"

Example 2: Cross-Reference Validator for Kubernetes

#!/usr/bin/env python3
"""
Cross-reference validator for Kubernetes manifests
Checks for hallucinated API fields, resource references, etc.
"""

import yaml
import subprocess
import json
import sys

class K8sHallucinationDetector:
def __init__(self):
self.api_resources = {}
self.load_api_resources()

def load_api_resources(self):
"""Load available API resources from cluster"""
cmd = "kubectl api-resources -o json"
result = subprocess.run(cmd, shell=True, capture_output=True, text=True)
resources = json.loads(result.stdout)

for resource in resources['resources']:
kind = resource['kind']
group = resource.get('group', '')
self.api_resources[kind] = {
'group': group,
'verbs': resource.get('verbs', []),
'version': resource.get('version', 'v1')
}

def get_schema_fields(self, kind, api_version):
"""Get valid fields for a resource kind"""
cmd = f"kubectl explain {kind}"
result = subprocess.run(cmd, shell=True, capture_output=True, text=True)

fields = set()
for line in result.stdout.split('\n'):
if line.startswith('FIELDS:'):
in_fields = True
elif line and in_fields:
field_name = line.split()[0]
if field_name:
fields.add(field_name)

return fields

def detect_hallucinations(self, manifest_file):
"""Scan manifest for hallucinated fields and references"""
with open(manifest_file) as f:
docs = yaml.safe_load_all(f)

hallucinations = []

for doc in docs:
if not doc:
continue

kind = doc.get('kind')
api_version = doc.get('apiVersion')

Check 1: Kind exists in API
if kind not in self.api_resources:
hallucinations.append(f"Unknown resource kind: {kind}")
continue

Check 2: Fields are valid
valid_fields = self.get_schema_fields(kind, api_version)

for field in doc.keys():
if field not in valid_fields and field not in ['kind', 'apiVersion', 'metadata']:
hallucinations.append(
f"{kind}: Hallucinated field '{field}' doesn't exist"
)

Check 3: Metadata references are valid
metadata = doc.get('metadata', {})
namespace = metadata.get('namespace')

if namespace:
cmd = f"kubectl get namespace {namespace}"
if subprocess.run(cmd, shell=True).returncode != 0:
hallucinations.append(
f"Referenced namespace doesn't exist: {namespace}"
)

return hallucinations

if __name__ == '__main__':
detector = K8sHallucinationDetector()
hallucinations = detector.detect_hallucinations(sys.argv[1])

if hallucinations:
print("HALLUCINATIONS DETECTED:")
for h in hallucinations:
print(f" - {h}")
sys.exit(1)
else:
print("No hallucinations detected")
sys.exit(0)

Example 3: Documentation Cross-Reference Validator

#!/usr/bin/env python3
"""
Cross-reference AI-generated config against official documentation
"""

import requests
from bs4 import BeautifulSoup
import yaml
import json

def get_nginx_directives(version="latest"):
"""Fetch list of valid nginx directives from documentation"""
url = f"https://nginx.org/en/docs/ngx_core_module.html"
response = requests.get(url)
soup = BeautifulSoup(response.content, 'html.parser')

directives = set()
for code in soup.find_all('code'):
if code.find_parent('dt'):
directive_name = code.text.strip()
directives.add(directive_name)

return directives

def validate_nginx_config(config_file):
"""Validate nginx config against official documentation"""
valid_directives = get_nginx_directives()

hallucinations = []

with open(config_file) as f:
for line in f:
line = line.strip()
if line.startswith('#') or not line:
continue

Extract directive name
directive = line.split()[0].rstrip('{};')

if directive not in valid_directives:
hallucinations.append(f"Unknown directive: {directive}")

return hallucinations

if __name__ == '__main__':
import sys
hallucinations = validate_nginx_config(sys.argv[1])

if hallucinations:
print("Hallucinations found:")
for h in hallucinations:
print(f" - {h}")
sys.exit(1)
else:
print("Configuration validated against official documentation")
sys.exit(0)

Anti-Patterns

Anti-Pattern 1: "It looks plausible, so it's probably correct"

This is how hallucinations survive review. A configuration that looks plausible, follows naming conventions, has reasonable parameter values, addresses a known problem, might still be entirely fabricated. Plausibility is not correctness. Verify systematically against documentation, not just visual inspection.

Anti-Pattern 2: Testing only the happy path

You test that a hallucinated config loads without errors. But a hallucination might load fine while doing nothing useful. The real test is: does it actually work as intended? Does the SSL configuration actually secure the connection? Does the rate-limiting directive actually limit rates? Testing should verify actual behavior, not just acceptance.

Anti-Pattern 3: Trusting the AI because previous outputs were correct

Just because the AI generated three good configs doesn't mean the fourth is good. Each artifact needs independent validation. AI consistency is not predictable. An AI that's correct 95% of the time is still wrong 1 in 20 times, and you need to catch that 1.

Anti-Pattern 4: Assuming syntax validation catches hallucinations

A hallucinated directive can have perfectly correct syntax. The validator will accept it as valid YAML, valid JSON, valid bash, etc. Hallucinations specifically exploit the gap between syntactic correctness and semantic correctness. You need validation that goes beyond syntax.

Anti-Pattern 5: Manual review instead of automated validation

Humans miss hallucinations because they look plausible. Automated validators that cross-reference against documentation catch them consistently. Don't rely on humans to remember all the rules. Automate the rules and let humans focus on judgment calls.

Anti-Pattern 6: One-time hallucination checking

Check every generated artifact, every time. Don't assume that checking once and deploying repeatedly is safe. Configuration changes, your tool versions upgrade, and what was hallucinated in one version might become hallucinated differently in another.

Human Judgment Checkpoints

1. Evaluating hallucination severity

Some hallucinations are critical (nonexistent directives that break functionality), others are subtle (parameters that are ignored but don't cause failures). Humans need to judge which hallucinations require blocking deployment and which ones just need documentation. A warning-level hallucination might be acceptable with proper logging; a critical one requires rejection.

2. Deciding when to check documentation vs. trusting the tool

There's a balance between over-validating (which takes time and resources) and under-validating (which misses problems). Humans need to decide: for this artifact, does the risk justify full cross-reference validation? Or is a lighter-weight syntax check sufficient? This depends on the criticality of the artifact and your risk tolerance.

3. Recognizing plausibility as a trap

When an AI-generated directive sounds incredibly useful and solves a problem you've been struggling with, that's when you most need to verify it carefully. Hallucinations often target real pain points because the AI understands what would be useful. Humans need to recognize that usefulness is not evidence of correctness.

Key Takeaways

Build reference data for every tool: Maintain lists of valid directives, parameters, API fields, and resource types for the specific versions you deploy. This data is your ground truth for hallucination detection.

Implement three layers of validation: Syntax validation (catches typos), reference validation (catches hallucinated directives), and integration validation (catches version mismatches and missing dependencies).

Cross-reference AI output against documentation: For every claim the AI makes about what's valid, verify it against official documentation. Automate this so it scales.

Pattern-match for hallucinations: AI hallucinations follow patterns (naming convention violations, overly specific parameters, features that sound too powerful). Train your validators to flag these patterns.

Trust external systems less than documentation: If the AI references external resources (files, DNS names, service accounts), verify those exist. Hallucinated references are particularly dangerous because they fail silently at runtime.

Make hallucination detection part of your pipeline: Don't treat it as optional review. Make validation gates required before deployment, and fail the build when hallucinations are detected.

Document what you find: When you discover a new type of hallucination, add detection for it to your validators. Over time, your validators get better at catching the specific types of hallucinations your AI tends to produce.