Verification Checklists
Overview
AI generates a script. It looks good. You're tempted to run it in production. But you have a moment of doubt: Did the AI consider edge cases? Is this idempotent? Does it handle errors? Is it secure?
This is why you need verification checklists. A checklist is a concrete list of things you verify before using AI-generated technical content. It's not "review the code". It's specific checks: "Does this script handle the case where the file doesn't exist?" "Are there hardcoded credentials?" "Does this config work with our version of the system?"
Checklists convert vague review into concrete verification. They ensure you don't miss critical issues. They're the difference between "I reviewed this and it looks okay" and "I verified this against specific requirements and it's safe."
Purpose
Verification checklists have two purposes:
- Safety: Ensure AI-generated content doesn't introduce security vulnerabilities, compliance violations, or operational failures
- Consistency: Ensure all AI-generated content meets the same quality bar, regardless of who generates it or when
A checklist is specific to content type. A script checklist is different from a config checklist, which is different from a documentation checklist.
Why This Matters
Without verification checklists, you review AI content inconsistently:
- Some reviews are thorough, some are cursory. Depends on your mood, time pressure, and how "good" the content looks at first glance.
- You miss systematic issues. You review the logic but forget to check for hardcoded credentials. Or you check for credentials but forget error handling.
- You make the same discovery twice. You realize a script isn't idempotent, fix it, then months later find another non-idempotent script and fix it again.
- You can't onboard reviews to others. New team members don't know what to look for when reviewing AI content. Inconsistent quality results.
- Incidents reveal gaps in reviews. A script fails in production and you realize you should have tested the error case. Now you know to check for it in the future, but you didn't know before.
With verification checklists, reviews are consistent, systematic, and aligned with known risks.
Key Insight: Checklists Prevent Review Fatigue
Review is cognitively expensive. After reviewing 5 scripts, your attention wanes. Checklists combat this by making review mechanical: check item 1, check item 2, check item 3. Checkmarks are easier than thinking.
Checklists also make reviews defensible. If something goes wrong, you can show: "I checked X, Y, Z. Item A was on the checklist and was verified."
Core Concepts
1. Checklist Design Principles
Good checklists are:
- Specific: "Check error handling" is vague. "Does the script handle the case where the file doesn't exist?" is specific.
- Testable: Can you verify it? If the checklist item is "looks reasonable," you can't test it. If it's "script exits with code 1 on missing file," you can test it.
- Short: Checklists that are 50 items long don't get used. Keep them focused. If you need 50 items, break into multiple checklists.
- Organized: Group related items. Don't mix security items with performance items randomly.
- Evolving: Add to the checklist as you discover gaps. "We found a script that wasn't idempotent, add idempotency check to the checklist."
Key insight: Checklists work because they're concrete, not abstract.
2. Checklist Types by Content Type
Different content types need different checklists:
Script/Code Checklist:
- Does it have error handling?
- Is it idempotent?
- Are there hardcoded values that should be configurable?
- Does it use secure credential handling (not hardcoded passwords)?
- Is it tested?
Configuration Checklist:
- Does this version of the config work with our system version?
- Are there security implications (open ports, excessive permissions)?
- Are there dependencies this config assumes?
- Does this break any existing functionality?
- Is this config version-controlled friendly?
Documentation Checklist:
- Is the information accurate (verified against actual systems)?
- Is it complete (would a new person be able to follow this)?
- Is it audience-appropriate (technical level matches intended audience)?
- Are there outdated references or screenshots?
- Is it searchable (good headings, keywords)?
Procedure Checklist:
- Can someone with your typical skill level follow this without getting stuck?
- Are there decision points where it's unclear which path to take?
- Are there gotchas or prerequisites that are missing?
- Is the success criteria clear?
- Is rollback documented?
Key insight: Each content type has unique risks that warrant specific checks.
3. Building a Script Verification Checklist
Scripts are high-risk because they execute. They can delete data, change configs, break systems.
SCRIPT VERIFICATION CHECKLIST
□ SYNTAX AND EXECUTION
□ Script is syntactically valid (runs without errors)
□ Script exits cleanly (exit code 0 on success, non-zero on error)
□ Shebang is correct (#!/bin/bash or #!/usr/bin/python3, etc.)
□ SAFETY AND ERROR HANDLING
□ Script handles missing files gracefully (doesn't crash)
□ Script handles permission errors (file not readable/writable)
□ Script has meaningful error messages (not just "error")
□ Script logs its actions (for troubleshooting if something fails)
□ Script doesn't use hardcoded credentials (uses environment
variables or external secrets)
□ IDEMPOTENCY
□ Running script twice produces same result as running once
□ Script doesn't create duplicate data if re-run
□ Script is safe to re-run (doesn't undo previous work then redo it)
□ SECURITY
□ No hardcoded passwords or API keys
□ Script doesn't write secrets to logs
□ Temporary files are created securely (not world-readable)
□ Script uses sudo/elevated privileges only when necessary
□ Script validates input (doesn't blindly execute user-provided values)
□ COMPATIBILITY
□ Script works with target OS/version
□ Script doesn't assume tools that might not be installed
□ Script uses portable commands (not Linux-specific if intended
for BSD/macOS)
□ DOCUMENTATION
□ Script has comments explaining non-obvious logic
□ Script has usage/help information
□ Script documents required prerequisites
□ TESTING
□ Script tested with happy path (normal case)
□ Script tested with error cases (missing file, permission denied)
□ Script tested in staging (not first run in production)
□ Script tested with different inputs (edge cases: empty strings,
special characters, large values)
PASSED ALL CHECKS: _____ (approval signature)
This checklist is concrete. Each item is something you can verify by running the script or reading the code.
4. Checklist for Configurations
Configs are also high-risk because they affect system behavior. A bad config can disable a service, expose data, or break functionality.
CONFIGURATION VERIFICATION CHECKLIST
□ COMPATIBILITY
□ Config format matches system version (YAML for K8s 1.20+, etc.)
□ Config uses supported features (doesn't use deprecated APIs)
□ Config doesn't reference resources that don't exist
□ FUNCTIONALITY
□ Config enables required functionality (service starts, responds)
□ Config doesn't break existing functionality (other services
still work)
□ Config matches documented requirements
□ SECURITY
□ No exposed secrets in config (uses secret management)
□ Permissions are restrictive (not world-readable if sensitive)
□ Config doesn't open unintended network ports
□ Config doesn't disable security controls
□ PERFORMANCE
□ Config doesn't set unrealistic resource limits (too low causes
failures, too high wastes resources)
□ Config uses reasonable timeout values (not too short, not infinite)
□ Config doesn't cause unnecessary overhead
□ TESTING
□ Config tested in staging environment
□ Config validated against schema (no syntax errors)
□ Config tested with expected traffic/load
□ Rollback tested (can we revert quickly if needed)
PASSED ALL CHECKS: _____ (approval signature)
This checklist addresses config-specific risks.
5. Building Documentation Verification Checklist
Documentation is lower risk than code/config, but wrong documentation causes bigger problems over time (people follow bad procedures, knowledge accumulates incorrectly).
DOCUMENTATION VERIFICATION CHECKLIST
□ ACCURACY
□ Information verified against actual system (not guessed or assumed)
□ Procedures were actually tested by someone
□ Screenshots/examples reflect current state
□ References to URLs/systems are current (not outdated)
□ COMPLETENESS
□ Prerequisites are documented (what must be true before starting)
□ Success criteria are documented (how do you know it worked)
□ Troubleshooting section exists for known issues
□ Related documentation is linked
□ CLARITY
□ Target audience would understand this (vocabulary matches skill level)
□ Steps are in logical order
□ Assumptions are stated (don't assume knowledge)
□ Gotchas/common errors are highlighted
□ USABILITY
□ Content is scannable (good headings, not wall of text)
□ Code examples are properly formatted and copyable
□ Links are tested (don't 404)
□ Table of contents or navigation exists for long docs
□ MAINTENANCE
□ Document includes version/date (when was this last updated)
□ Owner is documented (who maintains this)
□ Instructions for updating the document exist
PASSED ALL CHECKS: _____ (approval signature)
This checklist is specific to documentation risks.
Practical Use Cases
Before: You review AI content inconsistently; some issues slip through.
After: You use checklists; systematic verification catches issues before they cause problems.
Use Case 1: Script Review with Checklist
AI generates a Python script to backup databases. It looks reasonable. You use the script checklist:
SCRIPT CHECKLIST FOR: backup_databases.py
□ SYNTAX AND EXECUTION
✓ Script is syntactically valid
✓ Script exits with code 0 on success
✗ Shebang line is missing (add #!/usr/bin/env python3)
□ SAFETY AND ERROR HANDLING
✓ Script handles missing database gracefully (tries, logs error)
✓ Script has error messages (uses logging)
✗ Script uses hardcoded password for database connection (CRITICAL:
use environment variable DB_PASSWORD instead)
✗ Backup path is hardcoded (should be configurable)
□ IDEMPOTENCY
✓ Running twice produces same backup (overwrites previous)
□ SECURITY
✗ Credentials are hardcoded (fix required)
✗ Backup files are world-readable (fix: chmod 600 backup.sql)
[Continue through checklist...]
ISSUES FOUND:
1. Add shebang line (minor)
2. Remove hardcoded credentials (CRITICAL)
3. Make backup path configurable (medium)
4. Restrict backup file permissions (CRITICAL)
5. Add test case for missing database (minor)
ACTION: Request fixes for critical items before using in production.
The checklist caught critical issues (hardcoded credentials, world-readable files) that you might have missed in a casual review. You send this back to AI with specific fixes needed.
Use Case 2: Configuration Review with Checklist
AI generates a Kubernetes config for your service. You use the config checklist:
CONFIG CHECKLIST FOR: app-deployment.yaml
□ COMPATIBILITY
✓ Uses apps/v1 API (current Kubernetes version)
✓ Image exists in registry (verified image pull)
✓ ServiceAccount exists (referenced by deployment)
□ FUNCTIONALITY
✓ Deployment creates pods (replicas: 3)
✓ Service exposes pods (selector matches deployment label)
✗ Readiness probe is missing (service might receive traffic before
app is ready)
✗ Liveness probe is missing (if app hangs, pod won't restart)
□ SECURITY
✓ No secrets in plain text (uses secretKeyRef)
✗ securityContext is missing (container runs as root)
✗ Resource limits are missing (no memory/CPU constraints)
□ PERFORMANCE
✓ Replicas: 3 (appropriate for service tier)
✗ Resource requests are high (2 CPU, 1Gi memory, verify this is
realistic)
[Continue...]
ISSUES FOUND:
1. Add readiness probe (recommended)
2. Add liveness probe (recommended)
3. Add securityContext: runAsNonRoot (CRITICAL security)
4. Add resource limits (CRITICAL, unlimited pods = cluster DoS)
5. Verify resource requests are realistic (medium)
ACTION: Address critical issues before deploying to production.
The checklist caught missing security controls (root access, no limits) and reliability features (no probes) that could cause problems.
Use Case 3: Documentation Review with Checklist
AI generates a procedure for restoring from backup. You use the documentation checklist:
DOCUMENTATION CHECKLIST FOR: "Restore Database from Backup"
□ ACCURACY
✓ Procedure tested by ops team (last tested 2026-02-15)
✗ Restore command in step 3 is outdated (db_restore was replaced
with db-restore-tool)
✓ Expected recovery time matches reality (30-60 min)
□ COMPLETENESS
✓ Prerequisites documented (backup file, DB access, disk space)
✗ Success criteria missing ("how do you know restore succeeded?")
✗ Troubleshooting section missing
□ CLARITY
✓ Language is accessible to target audience (sysadmins)
✓ Steps are in order
✗ Step 5 mentions "verify replication lag" but replication isn't
mentioned in prerequisites
□ USABILITY
✓ Commands are in code blocks
✓ Estimated time is provided
✗ No rollback procedure if restore fails
[Continue...]
ISSUES FOUND:
1. Update restore command name (critical, procedure won't work)
2. Add success criteria (medium)
3. Add troubleshooting section (medium)
4. Add rollback procedure (medium)
5. Mention replication in prerequisites (minor)
ACTION: Fix critical items before publishing. Medium items should be
addressed before Q2 documentation review.
The checklist caught that the procedure won't actually work (command is wrong), and identified gaps (no success criteria, no troubleshooting).
Examples
Complete Script Checklist Template
SCRIPT VERIFICATION CHECKLIST
Script Name: _________________
Purpose: _________________
Created By: _________________ Date: _________
Reviewed By: _________________ Date: _________
=== SYNTAX AND EXECUTION ===
□ Script is syntactically valid (can be parsed/executed)
□ Script has proper shebang line (#!/bin/bash, #!/usr/bin/env python3, etc.)
□ Script runs to completion without syntax errors
□ Script exits with correct exit codes (0 for success, non-zero for failure)
□ No infinite loops or hangs when run normally
=== ERROR HANDLING ===
□ Script handles missing input/files gracefully (doesn't crash)
□ Script handles permission errors (file not readable, etc.)
□ Script has meaningful error messages (not just "error" or "failed")
□ Script logs actions and errors (for troubleshooting)
□ Script doesn't silently fail (errors are visible)
□ Script handles network failures (if applicable)
□ Script handles timeouts (if applicable)
=== IDEMPOTENCY ===
□ Running script twice produces same result as running once
□ Script won't create duplicate data on re-run
□ Script won't partially execute if interrupted then re-run
□ Script is safe to re-run in production without side effects
=== SECURITY ===
□ No hardcoded passwords, API keys, or credentials
□ No secrets in logs or error messages
□ Temporary files created securely (not world-readable)
□ Script doesn't require sudo unless necessary
□ Input validation (doesn't blindly execute user-provided values)
□ Safe file permissions (scripts not world-writable)
□ No privilege escalation without clear reason
=== COMPATIBILITY ===
□ Script works with target OS and version
□ Script doesn't assume tools that might not be installed
□ Script uses portable commands (not OS-specific if intended for multiple OSes)
□ Dependencies are documented
=== DOCUMENTATION ===
□ Script has clear comments explaining logic
□ Script has usage/help output
□ Prerequisites are documented
□ Expected output is documented
□ Known limitations are documented
=== TESTING ===
□ Tested with normal input (happy path)
□ Tested with missing files/input
□ Tested with permission errors
□ Tested with different environments (dev, staging, prod)
□ Tested with edge cases (empty strings, special characters, large values)
□ Tested in staging before production use
=== PERFORMANCE ===
□ Script doesn't have obvious performance issues
□ Script doesn't consume excessive resources
□ Loops and recursion are reasonable (won't cause timeouts)
=== NOTES AND ISSUES ===
[List any concerns, required fixes, or decisions made during review]
APPROVAL:
□ All critical items passed
□ All required items passed
□ Minor issues documented above
Reviewer: _________________ Date: _________ Approval: Yes / No
If No: List required changes before approval:
_________________________________________________________________
This is a comprehensive, reusable template for reviewing scripts.
Anti-Patterns
Anti-Pattern 1: Checklist That's Too Vague
□ Script looks okay
□ No obvious security issues
□ Works correctly
These aren't checklist items, they're gut feelings. You can't verify them consistently.
Prevention: Make items specific and testable. "Does the script handle missing files?" instead of "Script looks okay."
Anti-Pattern 2: Checklist That's Never Updated
You discover the AI script isn't idempotent. You fix it. But you don't add "idempotency check" to your checklist. Next month, you find another non-idempotent script.
Prevention: When you find a gap in your review, add it to the checklist. Checklist is a living document.
Anti-Pattern 3: Checklist That's Never Used
You build a great checklist. Then you're in a hurry and skip the checklist "just this once." Something breaks. Now you're forced to use the checklist.
Prevention: Make checklists part of your process. You don't deploy without checking. Period.
Anti-Pattern 4: Checklist With Too Many Items
Your script checklist has 100 items. Reviews take 2 hours.
Prevention: Consolidate related items. If your checklist is >20 items, break it into multiple checklists by topic.
Anti-Pattern 5: Checking Things That Don't Matter
Your documentation checklist includes "color scheme is appealing." This is opinion, not risk.
Prevention: Focus on things that affect safety, compliance, functionality, or usability. Skip subjective preferences.
Human Judgment Checkpoints
Checklist completeness: "Are we checking everything we should?" After you find an issue, did you add it to the checklist?
Checklist accuracy: "Are these checklist items relevant to our actual environment?" A generic checklist might have items that don't matter for you.
Check severity: "Is this check critical, recommended, or optional?" Mark items accordingly so reviewers know which to prioritize.
Review time: "Does this checklist take a reasonable time to complete?" If reviews take >1 hour, the checklist is too long.
Key Takeaways
Build checklists for content types at risk. Scripts, configs, procedures. These warrant checklists. Blog posts, maybe not.
Make checklist items specific and testable. "Does error handling exist?" not "Script looks good."
Organize checklists by category. Group related items so reviews are systematic, not random.
Evolve checklists based on discovered issues. Each time you find a problem, consider: should this be on the checklist?
Keep checklists focused. 10-20 items is better than 50. Long checklists don't get used.
Use checklists as team standard. Everyone reviews using the same checklist so quality is consistent.
Checklists prevent review fatigue. Mechanical checking is easier than thinking through everything.
Document the "why" for non-obvious checks. "Why do we check for idempotency?" Because a script that isn't idempotent can fail if re-run, which happened to us in 2024."
Signatures on checklists create accountability. Reviewer signs off after checking. If something goes wrong, the review is traceable.
Combine checklists with testing. Checklists catch some issues; testing catches others. Both are needed.
Skill.re