The short answer
The question is whether work reaches production faster without more of it breaking. Those two together are the test, and they are measurable from data you already have in version control and your issue tracker.
Volume measures answer a different question and reward the wrong behaviour.
What to measure
| Measure | What it tells you |
|---|---|
| Lead time, start to production | Whether work actually flows faster |
| Review turnaround | Whether the constraint moved to review |
| Change failure rate | Whether speed cost quality |
| Time to restore after a failure | Whether problems are found quickly |
| Rework rate | Whether first attempts are landing |
The second row is the one that catches the common outcome. Faster writing and slower review can leave lead time unchanged while everyone feels busier.
What not to measure
- Lines of code, which rewards verbosity
- Commits, which rewards splitting work
- Suggestions accepted, which measures the tool not the outcome
- Licences purchased, which measures spending
- Self-reported time saved, which is unreliable in both directions
The third is the one vendors report and it tells you about adoption rather than about delivery.
Establish a baseline first
- Pull lead time and failure rate for the period before adoption.
- Note anything else that changed at the same time, such as team size.
- Give it long enough that a learning period does not dominate.
- Compare like work, since a quarter of migration work is not a quarter of feature work.
- Accept that attribution is imperfect and say so.
Without step one, any later number is uninterpretable. Teams routinely adopt tooling and then discover they cannot say whether it helped.
Ask the team as well
Quantitative measures miss things that matter: whether the unpleasant work got easier, whether people spend less time stuck, whether onboarding is quicker.
Those are legitimate benefits even when lead time is unchanged. They are also worth naming honestly as what you got, rather than claiming a throughput improvement the data does not show.