The Documentation Problem

Every team knows the truth: documentation is essential and nobody writes it. You make an architectural decision, you forget to document why. You build a complex system, you skip the runbook. A year later, someone asks "why is it designed this way?" and the answer is "I don't know, ask whoever wrote it," and that person left the company six months ago.

The problem is that documentation is work. You have to remember what you decided, articulate it clearly, and explain the trade-offs. By the time you might write it, you're already three features ahead and documentation feels like overhead.

This is where AI becomes genuinely helpful. Not because it writes perfect documentation (it doesn't), but because it eliminates the friction of starting, organizing, and drafting. You provide the context. The AI handles structure and clarity. You edit and verify. The documentation gets written.

What AI is Good At

• Organizing thoughts into clear structure
  - Explaining technical concepts in multiple ways
  - Creating comprehensive outlines
  - Suggesting what you might have forgotten to document
  - Generating examples and diagrams descriptions
  - Writing in a consistent voice across multiple documents

What it's not good at:

• Knowing why you made a decision (you have to tell it)
  - Understanding what your actual constraints are (you have to specify)
  - Catching technical inaccuracies (you have to verify)
  - Writing decision rationales that explain your business context

You bring the knowledge. The AI brings clarity and completeness.

The Documentation Flywheel: Capture key information while it's fresh (in AI prompts). The AI drafts structure and content. You review and correct. Publish. Future team members find answers instead of bugging you.

Architecture Decision Records (ADRs)

An ADR documents a significant architectural decision, why you made it, what you considered, and what tradeoffs you accepted. Teams that maintain ADRs stay coherent because new people understand how to build on existing systems. They understand not just what was chosen, but why competitors' approaches were rejected. This is where organizational learning lives.

The ADR Template

The standard ADR has sections: Title, Status, Context, Decision, Consequences, Alternatives, and Notes.

Using AI to Generate ADRs

When you've made an architectural decision, capture it immediately while it's fresh:

"Create an ADR for this decision:

Decision: We're using a monolithic architecture with database replication instead of splitting into microservices.

Context: We have 10M daily requests. Current monolith handles it fine. Three teams work in the codebase. Deployments take 20 minutes. We have 4 backend engineers.

Why this choice: Microservices would add operational burden we can't support. Monolith with replication solves our immediate scaling needs without multiplying complexity.

Alternatives we considered: Full microservices (too complex for our team size), NoSQL database (unnecessary, SQL works fine), more aggressive caching (tried, didn't help much).

What we need to watch: As we grow, we might hit monolith limits around deployment speed and team coordination. We'll revisit in 12 months.

Create a professional ADR for this decision."

The AI produces a complete, well-formatted ADR. You review, correct any inaccuracies, and publish.

ADR Review and Evolution

Decisions change. When they do, update the ADR:

"This ADR is now superseded. We're moving from monolith to microservices because we hit the team coordination bottleneck we predicted. Update the ADR to reflect this, explain what triggered the decision, and create a new ADR for the microservices approach."

Future teams see the full history of thinking, not just the current state. This prevents re-litigating decisions. Someone joins and says "why don't we just use microservices?" You point them to the ADR from 2 years ago that explains why, with numbers.

Case Study: ADR As Organizational Memory

A Series B company maintained zero ADRs. By Series C (2023), they'd made so many architectural decisions that conflicting assumptions existed across teams. One team assumed they'd stay monolithic. Another was building microservices. They spent two months reconciling during Series C fundraising (worst possible timing). Then they created ADRs for the 20 biggest decisions using AI to draft and structure. Within one year, a decision that would have caused two months of debate took one week because everyone knew the history and rationale. They tracked that engineers spent 8 hours/week previously on "understanding what we decided and why." Post-ADR, that number dropped to 1 hour/week. The 7 hours/week freed up was redirected to new development.

More importantly: when they hired 40 new engineers for Series C, onboarding to the system architecture took 4 weeks instead of 8 (2 weeks of that was reading and asking about the 20 ADRs). New engineers could understand not just "we use PostgreSQL" but "we evaluated MongoDB and DynamoDB and chose PostgreSQL because [reasons that still matter]." This prevented them from repeating the same evaluation with each new hire. The ADRs became the institutional memory that outlasted specific people.

System Documentation

A good system document explains: what it does, how it works, what assumptions it makes, what scale it operates at, and how to add to it.

Pattern 1: System Overview Document

"Create a system overview document for our payment processing service:

What it does: Processes payments from users, validates transactions, integrates with Stripe and PayPal, updates user balances, logs all transactions.

How it works: REST API accepts payment requests, validates input, calls payment provider, stores result in database, publishes events for notification service.

Scale: 10,000 payments daily, peak 100/second.

Key components:
- Payment API (Node.js)
- Database (PostgreSQL)
- Payment provider integrations
- Event bus (Redis)

Assumptions: Payments can take up to 5 minutes to settle, duplicates must be handled, failures should retry.

Document this as if new engineers need to understand it without asking questions."

The AI produces a comprehensive overview. You verify accuracy, add diagrams or examples, and publish.

Pattern 2: Runbooks

Runbooks document how to operate a system when something goes wrong.

"Create a runbook for our payment service. Include:

1. Common issues and fixes:
   - High latency: check database queries, payment provider status
   - Failed payments: verify provider integrations, check logs
   - Duplicate transactions: replay reconciliation, contact provider
2. Emergency procedures:
   - If database is down: what can we still do?
   - If payment provider is down: how do we handle incoming requests?
   - If events aren't processing: how do we recover?
3. Rollback procedures:
   - How to revert a deployment
   - How to restore from a recent backup
   - How to handle in-flight transactions during rollback
4. Contact and escalation:
   - Who to page for what issue
   - When to call the payment provider

Document this in a way a 2am on-call engineer can follow it."

The AI creates a practical, thorough runbook. You review with operations, add your specific contact info and escalation paths, and publish. A good runbook can reduce MTTR (mean time to resolution) by 30-50% because the on-call engineer doesn't have to think. They follow steps.

When This Goes Wrong: Documentation Debt

A payment processing company skipped creating runbooks, thinking "the engineers know this." When a critical outage happened at 2am on a Saturday, the on-call engineer (new to the team, 3 months tenure) spent 45 minutes figuring out what to do instead of 10 minutes following a runbook. Cost: 35 minutes of downtime on a critical system, affecting 10,000 customers, with peak transaction loss of $12,000/minute (35 minutes = $420K in failed transactions and refunds). A runbook would have cost 4 hours to write. The outage cost the company far more than the engineering cost of documentation.

Worse: after the incident, they spent a week writing a runbook (during peak season, which was really bad timing). And they still had to handle customer escalations from the outage. A post-incident review noted: "If we'd created a 4-hour runbook before, this would have cost us $0 in downtime. Instead, we spent $420K in losses plus a week of crisis management."

When This Goes Wrong: ADRs Written But Not Followed

A company created 15 ADRs as part of a documentation effort. They were well-written. But they weren't linked from the relevant code or docs. New engineers didn't know they existed. Decisions that had been made were re-litigated because the ADRs weren't discoverable. Six months later, an engineer asked "why don't we use Kafka instead of our current event bus?" Someone dug up a 2-year-old ADR that explained exactly why. The ADR had already made that decision. The company had to re-explain the reasoning instead of referring to the existing decision.

Lesson: ADRs only work if they're discoverable. Link them from relevant code. Reference them in discussions. Put them in onboarding. Make them part of "how we talk about decisions." Otherwise they're just files in a repository that nobody reads.

When This Goes Wrong: Documentation Too Out-of-Date to Be Useful

A company created comprehensive system documentation. Then they didn't update it for 18 months while the system evolved. Payment integrations were added (not documented). Authentication was refactored (docs still described old flow). Monitoring was added (docs didn't mention how to read metrics). A new engineer followed the documentation and spent a day building something that already existed. New engineers learned to ignore the docs and ask experienced engineers instead. The documentation became a liability. It was actively misleading. After three months of junior engineers being confused, they finally updated the docs. The fix took a week. The cost of that week's lost productivity across the team: roughly $50K.

Lesson: Documentation is either fresh or it's worse than nothing. Pick systems that are critical and update them when you change them. Don't document everything, document what matters most (critical systems, major decisions, onboarding). Keep that subset fresh.

API Documentation

Most API docs are generated by tools like Swagger/OpenAPI. But the human parts (explanations, examples, gotchas) are often neglected.

Pattern: API Guide

"Create a user guide for our payment API. Include:

1. Quick start: minimal example to make a payment
   2. Authentication: how to authenticate requests
   3. Error handling: what errors can happen and what they mean
   4. Retry logic: which errors are retryable, backoff strategy
   5. Webhooks: how to receive payment status updates
   6. Rate limits: what the limits are and how to handle them
   7. Common problems: IDempotency key usage, handling timeouts, duplicate handling

Code examples in Node.js, Python, and Go.

Explain this for developers who are new to payments and might not know best practices."

The AI generates comprehensive API documentation. You verify examples work, add your specific endpoints, and publish.

Decision Documentation Workflow

Decisions happen continuously. Document them systematically:

1. Capture the Context

When you're making a decision, capture what matters:

• The problem you're solving
  - Constraints (time, people, budget, technology)
  - Options considered
  - The choice and why

2. Ask AI to Structure It

Give the AI the raw information. Ask it to create an ADR, a decision memo, or a system document.

3. Review and Correct

Read the AI output. Fix inaccuracies. Add context. Verify examples work.

4. Publish and Link

Put it where people can find it. Link from related documents. Make it part of onboarding.

5. Update Over Time

As your system evolves, update the docs. AI can help with updates too: "This document needs updating because [change]. Update it."

Key Insight

Documentation is most valuable when it captures thinking, not when it documents what the code does. AI helps you capture thinking (why decisions were made, what was considered, what constraints matter) and turn it into reference material.

What to Do Monday Morning

• Identify one significant decision your team made in the last month. Write down the context, why you chose it, and what you considered. Ask the AI to create an ADR. Review and publish.
  - Pick your most complex system. Describe how it works to the AI. Ask it to create a system overview document. Review for accuracy. Publish for onboarding.
  - Create a runbook for one of your critical systems. List common issues and fixes. Ask the AI to turn it into a structured runbook. Review with ops. Publish.
  - When an outage happens or a junior engineer gets confused, capture the question in documentation. Ask AI to help structure it. Prevent the same question twice.

FAQ

Q: Should AI-generated docs be published as-is?

A: Never. Review for accuracy, correctness, and whether it matches your context. AI provides structure and first draft. You provide correctness and domain knowledge. The review is non-negotiable.

Q: How do we keep documentation from getting stale?

A: Treat it like code. Review updates in PRs. Update when you change systems. Old documentation is worse than no documentation (it's actively misleading). Ask AI to help update when you change systems.

Q: Where should documentation live?

A: Somewhere searchable and version-controlled. GitHub wikis, Notion, Confluence, or your own doc site. The location matters less than whether people can find it and it's treated as living code.

Q: How much detail is too much?

A: High-level docs (why, what, how to use) are useful. Deep implementation details (which function calls which) are usually not. Code is the source of truth for implementation. Docs are the source of truth for "why did we design it this way?"

Q: Can we use AI docs for compliance and audits?

A: Only after human review. For compliance, you need to verify that documentation is accurate, complete, and current. AI provides first drafts. Humans sign off on accuracy.

Q: What if we have conflicting ADRs or decisions that contradict each other?

A: That's actually a sign your decision-making process needs work. Conflicting decisions mean you're not communicating them well. Use ADRs to surface these conflicts. When you find one, pick one decision to win and update the losing ADR to explain why the other won. This prevents teams from working at cross-purposes.

Q: How do we know if our documentation is actually being used?

A: Track it. Which docs get read? How often? Do new engineers reference them during onboarding? Use analytics or simple metrics (doc views, questions resolved by docs). If documentation isn't reducing the questions you get, it's not useful. Fix or delete it.

Q: Should we document decisions we later reversed?

A: Yes. Mark the ADR as "superseded" and explain what changed. This prevents re-litigating the decision. Future engineers understand: "We tried approach X (see ADR from 2023), it didn't work because Y, so we switched to approach Z (see new ADR from 2024)." This history prevents repeating mistakes.

On This Page

Watch the Lecture
The Documentation Problem
Architecture Decision Records
System Documentation
API Documentation
Decision Documentation Workflow
What to Do Monday Morning
FAQ

Chapter Details

Part of