Overview

Your dashboard shows: 45,000 lines of code shipped this quarter. Your team is thrilled. Your CFO is confused. Why? Because lines of code is the worst metric for productivity.

In fact, 45,000 lines of code shipped might mean you're actually less productive than last quarter when you shipped 20,000 lines. More code = more maintenance = more bugs = slower shipping next quarter. You should have shipped 20,000 lines of quality code, not 45,000 lines of cruft.

The best engineering organizations don't measure lines of code. They measure outcomes. And now that AI is in the mix generating code at superhuman speeds, your metrics need to change completely. AI can generate code fast. But can your team deploy fast? Can they maintain it? Are customers happy? That's what matters.

Why Your Current Metrics Are Wrong

Lines of Code

Wrong because: More code is usually worse. Simpler solutions are usually better. AI can generate 1000 lines of code in seconds, but that doesn't mean shipping happened.

Commits per Week

Wrong because: One good commit might change your entire system's performance. Ten commits might be trivial fixes that hint at bad architecture.

Bugs Fixed

Wrong because: You want bugs not to happen in the first place, not to measure how many you fixed.

Pull Request Cycle Time

Wrong because: Fast cycles could mean rubber-stamping code without thinking. Slow cycles could mean thorough review. The metric doesn't tell you which is happening.

What you actually care about:

• Are features shipping faster than before?
  - Is quality going up or down?
  - Are teams happy and staying?
  - Are production incidents decreasing?

That's what you should measure.

The Measurement Trap: You measure what's easy to measure, not what matters. Easy to measure: commits, LOC, PRs. Actually matters: outcomes, quality, reliability, team retention. Measure what matters.

The Metrics That Actually Work

Deployment Frequency

How often can you deploy to production? This correlates strongly with shipping velocity and team happiness. It's a proxy for: Can you make a change and ship it confidently?

What it shows: If you can deploy multiple times per day, you have: good architecture (changes are isolated), good testing (changes are validated automatically), good culture (people aren't afraid to ship). If you can only deploy once per quarter, you're slow and risk-averse.

Measure: Deployments per day. Track it weekly. Benchmark: good = 1-3x per day, excellent = 5-10x per day.

Why This Matters with AI

AI makes code generation fast. But if your deployment pipeline is slow, the speed of code generation doesn't matter. Measure how often you can actually ship.

Lead Time for Changes

From "we have an idea" to "it's in production," how much time passes?

What it shows: Are you removing friction from the pipeline? Is your CI/CD fast? Are decisions made quickly? Are there approval bottlenecks?

Measure: Days from commit to production. Benchmark: good = 1-3 days, excellent = 80%. If it's lower, you have planning or execution issues.

Production Incident Rate and Severity

How many incidents happen per month? How bad are they?

What it shows: Is quality going up or down? Are you shipping faster without sacrificing reliability?

Measure: Incidents per 10k deployments. Categorize by severity. Track monthly.

Code Review Efficiency

Not "cycle time," but actual efficiency: What percentage of comments result in changes? How often do issues show up in production that code review should have caught?

What it shows: Is your code review finding real problems or just adding process?

Measure: Review comments that result in changes / total comments. Should be >40%. Below that, review is mostly style.

Case Study: Metrics-Driven Team Improvement

An engineering team at a logistics company tracked only lines of code for years. They were "shipping 200k LOC per quarter" and felt productive. But customer complaints about bugs were steady. In Q4 2024, they switched to DORA metrics to measure what actually mattered. Baseline measurements:

• Deployment frequency: 0.2/day (one deployment every 5 days). Each deploy was a major event, required weeks of planning.
  - Lead time: 14 days from commit to production.
  - Change failure rate: 22% (roughly 1 in 5 deploys caused incidents requiring rollback or hotfixes).
  - MTTR: 3.5 hours when incidents happened.
  - Team satisfaction: 58/100 on internal survey. Many engineers reported "waiting for approvals" as the biggest time sink.

They diagnosed the problem: deployments were infrequent and high-stakes because each one bundled 2-3 weeks of changes. Review was slow (only 2 people could approve deploys). Testing was manual. Changes couldn't be rolled back easily.

Over 6 months (Q1-Q2 2025) with process change + AI adoption:

• Deployment frequency increased to 1.8/day (+800%). They deployed small changes frequently instead of big batches.
  - Lead time dropped to 2 days (-85%).
  - Change failure rate dropped to 8% (-64%). Small deploys meant fewer bugs. Better testing caught issues before deploy.
  - MTTR dropped to 45 minutes (-79%). When issues did happen, they were isolated to recent changes, so root cause was fast to find.
  - Team satisfaction jumped to 89/100 (+31 points). Engineers no longer felt bottlenecked.

Interestingly: lines of code shipped actually decreased (from 200k to 160k per quarter) because they were optimizing for quality over quantity. But productivity (measured by outcomes) skyrocketed. Customers reported fewer bugs. The team shipped more features per engineer despite shipping fewer total lines of code.

Financial impact: the team went from "shipping quarterly, high-risk" to "shipping daily, low-risk." Estimated cost of incidents avoided: $400K per year (based on their incident costs and reduced incident count). Engineering time freed from "wait for approvals" and incident response: approximately 20% of team capacity (roughly $600K in freed-up payroll that could go to new features).

Key insight: Lines of code was hiding the real problem (slow deployments, high failure rate, bottlenecked approvals). Real metrics exposed it. Once they could see the problem, they fixed it.

Team Satisfaction

This is a hard metric to measure, but critical. Monthly survey: "On a scale of 1-10, how productive do you feel? How much are you enjoying the work? Would you refer a friend to this team?"

What it shows: Is the team actually happy or just going through the motions?

Measure: Monthly NPS equivalent. Track it. Upward trend is good.

How AI Changes What You Measure

AI Usage Metrics

New things to track:

• AI pair programming hours: Is the team actually using AI for pairing? (Tool logs can tell you.)
  - Code generation acceptance rate: When AI generates code, how much of it is accepted vs. rejected? This tells you if AI is helpful or hallucinating.
  - AI-assisted code review: What percentage of code issues were caught by AI before human review?
  - Specification quality: Are your specs executable and testable before implementation? (Percentage of PRs that match spec without rework.)

AI Doesn't Change the Core Metrics, But Amplifies Them

If you have good processes, AI makes them better. If you have bad processes, AI makes them worse (amplifies mistakes). So measure the same things, but expect them to move faster now.

With AI:

• Deployment frequency should increase 2-3x
  - Lead time should drop 50%
  - MTTR should improve (better code = fewer incidents)
  - Change failure rate should improve (AI-powered testing and validation)

If they're not moving in these directions, your process isn't AI-native yet.

When Metrics Go Wrong: Gaming and Misalignment

A financial services company started measuring "code quality score" (based on static analysis findings). Engineers started gaming it, fixing low-severity issues (variable naming style, minor warnings) to boost the score while ignoring architectural problems. The metric made code look better (quality score increased 35%) while actual quality degraded (production bugs increased 18%). Lesson: make metrics hard to game. Focus on outcomes (incidents, bugs) rather than proxies (quality scores, test coverage). If it's easy to game, engineers will game it.

Another company optimized for "deployment frequency" and ended up shipping broken code constantly because they skipped testing. High deployment frequency looked good on the dashboard, but MTTR was 4 hours and change failure rate was 38%. They'd optimized for the wrong thing. The fix: bundle metrics together (deployment frequency + change failure rate together tells the real story).

When Metrics Go Wrong: Ignoring Context

A team was proud of their 1.5/day deployment frequency. But they didn't track what happened after deployment. They had 35 production incidents per month (compared to 3 for a peer team). The metrics didn't tell the story: they were deploying frequently but with poor validation. The peer team deployed 0.3/day but with 99% quality. Which team was more productive? The metrics alone didn't answer that. The fix: track complementary metrics (frequency + quality together) so you see the full picture.

Building Your Metrics Dashboard

Start Simple

Pick three metrics that matter:

• Deployment frequency: How often do you ship? (per day)
  - Lead time for changes: From idea to production, how long? (hours/days)
  - Change failure rate: What percentage of deployments cause incidents? (%)

Track them weekly. That's it for now. Get baseline data for a month. After a month, you'll see patterns.

Baseline Interpretation

After one month of tracking, interpret your baselines:

• Deployment frequency 5 days? You have planning/approval bottlenecks.
  - Change failure rate > 20%? You don't have good testing or validation.

These baselines become your starting point. Now you can measure improvement.

Add Context Metrics

After a month, add:

• MTTR (how fast can you recover?)
  - Feature completion rate (planning accuracy)
  - Team satisfaction (monthly survey)

Make It Visible

Post your metrics publicly. Every team sees them. Weekly standup reviews them. This isn't for blame. It's for learning. "Why did lead time increase this week? What did we learn? How do we improve?"

Avoid Gamification**

Don't tie metrics to bonuses. Don't create leaderboards. Metrics are for learning, not for performance management. The moment you make it about individual performance, people game the system.

What to Do Monday Morning

• List all the metrics your organization currently tracks (lines of code, commits, PR reviews, etc.)
  - For each one, ask: "Does this correlate with actual productivity? Can it be gamed?"
  - Identify three to five that actually correlate with shipping value (deployment frequency, lead time, failure rate, customer satisfaction)
  - Set up a dashboard that tracks them daily or weekly. Make it visible to the team.
  - Establish current baselines (measure for 2 weeks without changing anything)
  - Share baselines with your team. Ask: "What's our goal? Where do we want to be in 6 months?"
  - Track metrics religiously. Review them weekly in standup. Celebrate improvements.

FAQ

Q: What about individual developer metrics?

A: Don't use them for evaluation. They promote the wrong behaviors. Measure team outcomes instead. If individuals are underperforming, that shows up in the team's metrics. Address it there.

Q: How do we measure quality?

A: Incidents, bug escape rate, MTTR. Those are your quality metrics. Code coverage is a proxy, but a weak one.

Q: What if we're shipping fast but quality is dropping?

A: That's a process problem. Your incident rate is going up. That's your signal to slow down, improve testing, or add validation gates. The metrics tell you something's wrong.

Q: Should we measure AI's impact separately?

A: Not really. Just measure your overall metrics before and after AI adoption. If they're moving right, AI is working. If not, your process isn't AI-native yet.

Q: What if metrics don't correlate with revenue or customer satisfaction?

A: Then you're measuring the wrong thing. DORA metrics correlate with business outcomes for most tech companies, but not all. Track the connection explicitly: "As deployment frequency increased 2x, customer satisfaction increased by Y%." If the correlation is weak, adjust your metrics.

Q: Isn't tracking all these metrics overhead?

A: Good tools make it automatic (pull from CI/CD, incident management, etc.). You shouldn't be manually calculating. If you are, you've picked the wrong metrics.

Q: How do we handle metrics when teams have different codebases and tech stacks?

A: Use the same metrics for all teams (deployment frequency, lead time, MTTR, change failure rate) but interpret baselines differently. A Java team might deploy 0.5/day while a Node.js team deploys 2/day (different compilation/testing speeds). The important thing is: is each team improving over time? Are they moving in the right direction for their context?

Q: What if our metrics say we should deploy daily, but our business needs big quarterly releases?

A: That's a real tension. You can deploy frequently while releasing quarterly (dark launches, feature flags, canary deployments). Measure how long features sit in "done but not released" state. That's waste. The goal is: features ship the moment they're ready, even if there's a quarterly coordinated announcement. Decouple the metrics (deployment frequency) from the business timeline (release cadence).

Measure deployment frequency, lead time, change failure rate, and team satisfaction. Everything else is secondary. These four metrics tell you if you're shipping fast, shipping reliably, and keeping your team happy. That's productivity. AI amplifies your ability to improve all four metrics, but you have to measure them to know you're improving.

On This Page

Watch the Lecture
Why Current Metrics Are Wrong
Metrics That Actually Work
How AI Changes What You Measure
Building Your Dashboard
Monday Morning Action
FAQ

Chapter Details

Part ofChapter 2