Hook

You run a script that AI generated. It looks reasonable when you read it, proper syntax, right structure. But halfway through execution, it fails because it's using a flag that doesn't exist in your version, or it references a package that was never installed, or it applies a config rule that contradicts your infrastructure. The output was confidently wrong. No error message warned you. The problem was that you didn't catch the subtle mistake before deploying it.

Purpose

This lesson teaches you to recognize the red flags that signal AI-generated technical content is wrong before you deploy it. You'll develop a mental checklist for evaluating configs, scripts, documentation, and troubleshooting steps. This isn't about becoming paranoid. It's about building patterns that catch the silent failures that happen when plausible-looking code is actually broken.

Why This Matters for IT Professionals

AI is confident. It generates well-structured output, uses correct terminology, sounds authoritative. This makes it dangerous. A human expert tends to signal uncertainty ("I'm not 100% sure" or "this might not work in your environment"). AI often doesn't. It gives you a command that looks right and you run it without verification, only to find later it fails.

In IT Operations, the cost of "this looked right but wasn't" is high. A security rule that silently doesn't work costs you. A backup script that looks like it's working but actually isn't recoverable costs you. A troubleshooting step that confidently recommends the wrong action costs you.

Building a checklist for "does this look right?" saves you from deploying broken technical output. You catch mistakes during review, not during the 2 a.m. incident when your infrastructure is down.

Core Concepts

Key Insight: Plausible Syntax Doesn't Mean Correct Functionality

AI can generate syntactically correct code that does the wrong thing. A config file can have perfect YAML syntax but incorrect logic. A script can run without errors but produce incorrect results.

Red flags:

• Syntax looks right, but you've never seen that flag before: "I don't recognize that PowerShell parameter." Run Get-Help on the cmdlet and verify the flag exists and does what you think.
• The command matches your version, but the output doesn't match documentation: You run the command, and the output format is different from what you expected or what the docs show.
• The logic looks right, but the edge cases feel missing: A script that handles the happy path but doesn't check if directories exist, files are readable, or permissions allow the operation.

Example:

This is syntactically correct Bash, but subtly wrong
tar -czf /backup.tar.gz /data --exclude=/data/temp --exclude=/data/logs

Looks reasonable. But the --exclude paths are relative to the tar working directory, not absolute paths. This won't exclude what you think it will. The syntax is valid; the logic is broken.

Key Insight: Version-Specific Syntax Is a Common Failure Mode

AI trains on data from many versions of software. When you don't specify your version, it might pull syntax from a different version than you're running. The result looks right, but doesn't work.

Red flags:

• Flags or parameters you can't find in your version's documentation: openssl s_client -showcerts works in OpenSSL 1.1, but in OpenSSL 3.0 the flag is -show_certs. Similar name, different flag. If you don't verify, you get an error.
• Package or module names that don't exist: A Python script that assumes requests is installed, but it's not in your base install. Or a PowerShell command that assumes a module is loaded, but it doesn't exist in your Windows version.
• Configuration options that are obsolete or deprecated: A Docker config that uses an old networking mode that's no longer supported, or a Kubernetes manifest that uses an API version that's removed in your version.

Example:

This command works in Kubernetes 1.20, but not 1.28
kubectl run --image=nginx mynginx --port=80 --expose

In older versions, --expose created a Service automatically. That flag was removed in newer versions. If your cluster is 1.28, this fails with "unrecognized flag."

Key Insight: Hallucinated Tools and Packages Are Hard to Spot

AI sometimes invents tools or packages that sound real but don't exist. Or it references a package that exists but doesn't have the functionality AI claims it has.

Red flags:

• A tool or package name you've never heard of: This could be real, or it could be hallucinated. Before you install something AI suggested, search for it in your package manager or vendor docs. Does it actually exist?
• A command-line flag that sounds plausible but doesn't exist: "Use grep -Q to suppress output." But grep -Q doesn't exist (you want grep -q, lowercase). The difference is one letter, but it breaks the command.
• A package that claims to do something that sounds too convenient: "Use the ansible-auto-fix module to automatically remediate security issues." Sounds great, but does it actually exist? Search Ansible Galaxy. It probably doesn't.

Example:

Use the netstat-enhanced tool to show port statistics.

This tool might not exist. If you try to install it, the package manager won't find it. Before you trust a tool recommendation, verify it exists in your package manager or the vendor's documentation.

Key Insight: Mixed Contexts Are Silent Killers

AI sometimes mixes contexts without realizing it. A script that assumes Linux but you're running Windows. A config that assumes root access but you're running as a regular user. A troubleshooting step that assumes you have internet access but you don't.

Red flags:

• Instructions that assume a different OS than you specified: You said "Windows Server 2022," but the script uses Linux commands like ps aux. Even worse, it might be subtle: a PowerShell command that assumes a Unix-style path separator.
• Commands that assume tools or access you don't have: "Run sudo make install" but you don't have sudo. Or "Download this package from GitHub" but you're air-gapped.
• Config that references resources that don't exist in your environment: A Kubernetes manifest that assumes a StorageClass called "fast-ssd" exists, but you only have "standard."

Example:

This is PowerShell, but it assumes Unix-style paths
Get-ChildItem -Path /var/log/myapp

Windows PowerShell uses backslashes for paths, not forward slashes. This will fail. AI mixed contexts without realizing it.

Key Insight: Confident but Unverifiable Claims Are Red Flags

AI sometimes makes claims that sound authoritative but are hard or impossible to verify.

Red flags:

• "This is the standard way to..." without qualification: There's rarely one "standard" way. There are usually tradeoffs. If AI presents something as standard without caveats, question it.
• "All versions of X do this" when you know different versions behave differently: This signals AI isn't being careful about version differences.
• "No permissions needed" or "No dependencies" when that sounds too good to be true: Usually there's a catch. Verify what permissions and dependencies are actually required.

Example:

To set up Kubernetes RBAC, run this single command:
kubectl apply -f rbac.yaml

This might be true, but it glosses over: what's in rbac.yaml? Does it match your cluster? Does it have the permissions your workload needs? The claim is confident but vague enough to hide problems.

Key Insight: Missing Error Handling Is a Code Red Flag

Scripts and code that don't handle errors are fragile. If something goes wrong, they keep running and cause downstream problems.

Red flags:

• Scripts with no error checking: A bash script that doesn't use set -e or check exit codes. If a command fails, the script keeps running.
• No checks for required files or directories: Code that assumes /data/temp exists without checking, then fails silently when it doesn't.
• No validation of input parameters: A script that accepts command-line arguments without checking they're not empty or invalid.
• Assumptions about availability: Code that assumes a file can be read, a network service is reachable, a directory can be written to, without checking first.

Example:

#!/bin/bash
# Backup script with no error handling
tar -czf /backup.tar.gz /data
scp /backup.tar.gz admin@backup-server:/backups
rm /backup.tar.gz

If tar fails, the script keeps running. If scp fails, it still deletes the local backup. If the backup server is down, you've deleted a working backup. Error handling would catch these.

Key Insight: Incomplete or Missing Documentation of Assumptions

Good technical output explains its assumptions. Bad output doesn't.

Red flags:

• No explanation of what the code assumes about the environment: Does it assume specific OS versions? Specific packages installed? Internet access?
• No explanation of edge cases or limitations: A config that works for the happy path but has caveats the output doesn't mention.
• No explanation of what testing was done: Has this actually been tested, or is AI generating it from first principles? If untested, you're the first tester.

Example:

Here's a Terraform module for RDS:
[40 lines of code]

But nowhere does it say: "This assumes you already have a VPC and security groups set up," or "This has been tested with AWS provider version 5.0+," or "The backup retention is set to 7 days; adjust if your requirements differ."

Good output would include these caveats.

Practical Use Cases

Use Case 1: Reviewing a Database Query

Scenario: AI generated a SQL query for a PostgreSQL report. It looks reasonable, but you need to verify it's correct before running it on production data.

Red flags to check:

1. Does the table/column structure match your database? The query references columns like customer_id and order_total. Are those the exact column names in your tables? Case-sensitive?
2. Is the join logic correct? If the query joins tables, trace through the logic. Does it produce the right result or could it create duplicates?
3. Does it handle NULL values correctly? A query that uses = to compare to a column might miss NULLs. Is that intentional?
4. Is the performance acceptable? A query that's correct but scans the whole table on production data might time out or lock resources.
5. Are there any deprecated SQL features? PostgreSQL versions differ. Is the query using features that exist in your version?

Example with issues:

SELECT c.customer_name, COUNT(o.order_id) as order_count
FROM customers c
LEFT JOIN orders o ON c.customer_id = o.customer_id
GROUP BY c.customer_name
ORDER BY order_count DESC;

Red flag: GROUP BY c.customer_name works, but it's not the best practice (should group by c.customer_id to ensure uniqueness). Also, if you run this on a table with millions of orders, it might be slow without an index on o.customer_id.

What you must do:

1. Run EXPLAIN ANALYZE on the query to see the execution plan.
2. Test on a non-production database first with realistic data volume.
3. Verify the results make sense (spot-check a few customers and their order counts manually).

Use Case 2: Reviewing a Kubernetes Manifest

Scenario: AI generated a Kubernetes deployment for your application. It looks reasonable, but Kubernetes manifests have many subtle things that can go wrong.

Red flags to check:

1. Does the API version match your Kubernetes version? apiVersion: apps/v1 is current, but older versions use extensions/v1beta1.
2. Does the image reference match your registry? If you're using a private registry, the image path needs to be correct. Is it ghcr.io/... or your private registry?
3. Are the resource requests and limits reasonable? If the limits are too low, the pod will be evicted. If they're too high, they're wasteful.
4. Does the liveness and readiness probe make sense? A probe that hits /health is reasonable, but what if your app takes 30 seconds to start? The default probe timeout might be too short.
5. Are there any security issues? Is the container running as root? Is securityContext missing? Are environment variables containing secrets?

Example with issues:

apiVersion: v1
kind: Pod
metadata:
name: myapp
spec:
containers:
- name: app
image: myapp:latest
ports:
- containerPort: 8080

Red flags:

• image: myapp:latest is not in a registry path. Where does this come from? Probably a local build, not a real image.
• No resource limits. The container could consume all cluster resources.
• No liveness or readiness probe. The orchestrator doesn't know if the app is healthy.
• apiVersion: v1 and kind: Pod together mean this won't scale or restart on failure. You want Deployment, not Pod.

What you must do:

1. Verify the image path points to an image that actually exists in your registry.
2. Run kubectl dry-run=client -f manifest.yaml to validate the syntax.
3. Test in a non-production cluster first.
4. Check that resource requests match what your infrastructure can provide.

Use Case 3: Reviewing a Bash Script

Scenario: AI generated a Bash script for a routine system task. You need to verify it's safe before deploying it to production servers.

Red flags to check:

1. Does it have error handling? Does it check exit codes? Does it fail safely if a command fails?
2. Does it validate input? If it takes command-line arguments, does it check they're not empty or invalid?
3. Does it assume things exist? Does it check if directories exist before writing to them, or files exist before reading them?
4. Are there any unsafe patterns? Like rm -rf $VARIABLE/ where $VARIABLE could be empty and wipe /?
5. Does it work with both spaces and special characters in filenames? If files might have spaces, are paths quoted?

Example with issues:

#!/bin/bash
# Cleanup script

for file in /tmp/myapp/*; do
if [ -f "$file" ]; then
rm "$file"
fi
done

echo "Cleanup complete"

Red flags:

• No error handling. If rm fails (permissions, read-only filesystem), the script keeps running.
• No check if /tmp/myapp/ exists. If it doesn't, the for loop might behave unexpectedly.
• echo "Cleanup complete" happens regardless of success or failure. Misleading.

Better version:

#!/bin/bash
set -e # Exit on any error

if [ ! -d "/tmp/myapp" ]; then
echo "Error: /tmp/myapp does not exist" >&2
exit 1
fi

rm -f /tmp/myapp/* && echo "Cleanup complete" || { echo "Error: cleanup failed" >&2; exit 1; }

What you must do:

1. Trace through the script logic. What happens if the first command fails?
2. Test it on a non-production system first.
3. Check that the script is idempotent (safe to run twice).
4. Verify it works with the exact data/files you'll use in production.

Examples

Example 1: Detecting Version-Specific Syntax Issues

Scenario: You asked AI for a Kubernetes rolling update strategy, and it provided:

apiVersion: apps/v1
kind: Deployment
spec:
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 0

This looks reasonable, but you're running Kubernetes 1.8 (very old).

Red flag: You should check if maxUnavailable: 0 works in Kubernetes 1.8. It doesn't. In that version, you had to use maxUnavailable: 1 or a percentage.

What you must do: Before deploying, check the Kubernetes API docs for version 1.8 to confirm these fields work as described.

Example 2: Detecting Missing Context Assumptions

Scenario: AI generated a Python script for automated remediation:

import requests

response = requests.get('https://api.example.com/status')
data = response.json()
if data['status'] == 'error':
# remediate

This looks reasonable, but hidden issues:

Red flags:

• No error handling if the API is unreachable.
• No timeout set, so the script could hang forever.
• No authentication, but the API might require it.
• Assumes the response is always JSON, even on errors (it might be HTML error page).

What you must do:

1. Test this script against your actual API.
2. Verify it handles network failures gracefully.
3. Add error handling for non-JSON responses.
4. Verify authentication is included if needed.

Example 3: Detecting Hallucinated Tools

Scenario: AI suggested you use openssl-batch to generate certificates in bulk.

Red flag: Search your system: which openssl-batch. It doesn't exist. This is likely a hallucination.

What you must do:

1. Before installing or using any tool AI recommends, verify it exists.
2. Search your package manager: apt search openssl-batch or brew search openssl-batch.
3. Check the official documentation for the tool.
4. If it doesn't exist or is outdated, ask AI for an alternative approach using actual tools.

Anti-Patterns

Anti-Pattern 1: Trusting Output Because It Looks Professional

Don't assume AI output is correct because it has proper syntax and formatting. Plausible-looking output can be wrong.

Anti-Pattern 2: Not Testing Before Production

Don't deploy code or configs to production without testing them first, even if they came from a trusted AI and look correct.

Anti-Pattern 3: Assuming Versions Match Without Verifying

Don't assume a command works in your version just because it's the right general category. Verify syntax and flags actually exist in your version.

Anti-Pattern 4: Skipping Error Handling Review

Don't assume error handling is present. Review scripts and code to confirm they check exit codes, validate input, and fail safely.

Anti-Pattern 5: Not Asking Questions When Something Seems Off

If AI output has something you don't understand or looks suspicious, ask AI to explain it. Force it to justify choices and document assumptions.

Human Judgment Checkpoints

Before you trust AI output:

• 
  Can I verify this works with my specific version/environment? If you're unsure, test it.
• 
  Are there flags, tools, or packages I don't recognize? Look them up. Don't assume they exist just because AI mentioned them.
• 
  Does the output make assumptions I didn't specify? If it assumes things about your infrastructure that you didn't mention, flag them.
• 
  What happens if this fails? Does the code handle errors gracefully, or will it cause downstream problems?
• 
  Have I tested this on non-production data first? Always. This catches mistakes before they affect production.
• 
  Can I explain why this works, not just that it does? If you can't articulate the logic, you don't understand it well enough to defend it.

Key Takeaways

• Plausible-looking syntax doesn't guarantee correct functionality. Verify the logic, not just the structure.
  - Hallucinated tools and packages are a real problem. Before you trust a tool recommendation, verify it actually exists.
  - Version-specific syntax is a common failure mode. If AI gave you a command and you don't recognize a flag, verify it exists in your version before running it.
  - Missing error handling is a code red flag. Review scripts to ensure they check exit codes, validate input, and fail safely.
  - Mixed contexts (Linux vs. Windows, internet vs. air-gapped, with tools vs. without) are silent killers. Always verify the output matches your actual environment.