Usage is not success. Licence activations, lines of AI code and self-reported speed all go up in every rollout, including the ones that fail. This framework measures whether the method took hold in the people who know the system best, whether the guardrails held while it did, and whether it will survive when the outside help leaves.

Why usage is not success

Experienced developers in a randomised trial were 19% slower with AI while believing they were 20% faster (https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/). Perception is not a measurement.
Under a throughput mandate, merged PRs per developer went from 21.2 to 44.3, PRs grew 37% larger, human review coverage fell from 89% to 68% and per-reviewer load doubled. Reverts stayed flat, because review became the bottleneck, not because quality held (https://arxiv.org/abs/2607.01904).
Google reports 75% of new code AI-generated and about 10% velocity gain (https://www.fastcompany.com/91531519/google-ceo-says-75-of-the-companys-code-is-ai-generated). Volume and value decouple.
DORA 2025: AI is positively associated with throughput and negatively with delivery stability. AI amplifies what is already there (https://dora.dev/dora-report-2025/).
Access alone did nothing in the Microsoft study; adoption spread through social networks (https://arxiv.org/abs/2607.01418).

So the questions are different. Did the long-tenured developers adopt, and did they stay? Did change failure, rework, review coverage and reviewer load stay within tolerance? Is the context layer maintained by the teams themselves?

The funnel

Every level is computed from git and the roster. Every level is a stricter definition than the one before, and each is computed independently rather than as a subset of the previous.

Level	Definition	Source
Enabled	`enabled_since` in the roster is on or before today, or the developer has any AI-assisted commit	roster
Tried	At least one AI-assisted commit, ever	trailer
Active	At least one AI-assisted commit in the trailing `activeWindowDays` (default 28)	trailer
Engaged	At least two distinct days with an AI-assisted commit in the last 28 days. The GitHub and DX definition (https://docs.github.com/en/copilot/concepts/copilot-usage-metrics/copilot-metrics)	trailer
Frequent	At least twelve AI-active days in the last four weeks, that is three per week	trailer
Habitual	At least three commits in the last 28 days, at least half of them AI-assisted, and an AI-active day in at least three of the last four weeks	trailer
Contributor	A commit touching the context layer (`AGENTS.md`, `docs/`, `decisions/`, skills, generated rule files) in the last 90 days	git paths
Trainer	Marked `trainer` in the roster	roster

The distance between tried and engaged is where rollouts die. A developer who tried once and stopped is the real failure signal, and the retention metric below measures exactly that.

Days to habit: from enabled_since (or first AI commit if unset) until the first day on which the developer had an AI-active day in three of the trailing four weeks. Reported as a median per tenure band.

Four metric families

Each metric below lists its formula, source, target, and its Goodhart counter: the metric that goes wrong when someone optimises for the first one. Read them in pairs. Targets are the defaults in uctx.config.json or fixed in the scorecard; they are starting points, not laws.

Leading: adoption

Metric	Formula	Source	Default target	Goodhart counter
AI-active rate	active / enabled	trailer, roster	70%	Habitual rate. One commit a month is not adoption
Legacy active rate	legacy active / legacy enabled, legacy meaning tenure at or above `legacyTenureYears`	trailer, roster	60%	Adoption gap. If only new hires adopt, the number that matters is still zero
Adoption gap	recent active rate minus legacy active rate	trailer, roster	at most 5 points	None; this is itself a counter
Engaged rate	engaged / enabled	trailer	60%	Frequent and habitual rates
Habitual rate	habitual / enabled	trailer	40%	Revert and rework rates; habit with rising rework is a habit worth breaking
Retention at week 8	Over cohorts by week of first AI commit, weighted by size: share still AI-active in the 28 days ending 8 weeks after the cohort start	trailer	70%	Retention at week 12
Days to habit, legacy	Median over legacy developers	trailer, roster	42 days, one train-the-trainer cycle	Retention; a fast habit that fades is churn

Lagging: outcomes

Metric	Formula	Source	Target	Goodhart counter
AI-assisted commit share	AI commits / commits, four-week average	trailer	50%, informational	Rework and revert rates. Volume is not value
Merged PRs per active developer	merged PRs / active developers per week, four-week average	prs.json	Hold at or above baseline	PR size, human review coverage, pickup time, reviewer load
PR cycle time	Median hours from first commit to merge, four-week average	prs.json	Hold within 15% of baseline	Must not worsen while throughput rises
Hours saved per week	Survey mean, 13-week average	survey.csv	3 hours	Cycle time. Perception, always paired with a system metric

DX's Q2 2026 benchmarks saw AI-authored merged code go from 27% to 52% in one quarter while median PR size went from 44 to 72 lines and DXI fell 2% (https://newsletter.getdx.com/p/ai-in-engineering-q2-2026-benchmarks). Those three numbers together are the lesson: the first alone would have looked like success.

Guardrails

Metric	Formula	Source	Tolerance	Goodhart counter
Protected tests edited	AI-assisted commits deleting lines under protected paths, last four weeks	git	Must be zero	None; a violation is a broken gate
Revert rate, AI	AI reverts / AI commits, eight-week average, against the pre-rollout revert rate	git	Hold; amber above 25% worse, red above 50% worse	Change failure rate
Rework within 30 days, AI	Per commit, share of its files touched again by another commit within 30 days; mean over AI commits, eight-week average	git	Hold within 15%	Revert rate. Rework is how faster becomes twice
Human review coverage	PRs with at least one non-bot review / merged PRs, four-week average	prs.json	95%	Reviewer load. Coverage held by exhausting reviewers is not held
Reviews per reviewer per week	Reviews / distinct human reviewers, four-week average	prs.json	Hold within 30%	PR size
Median PR size	Lines added plus deleted, four-week average	prs.json	Hold within 30%	Review coverage. Big PRs get shallow reviews
Change failure rate, AI	Reverted AI PRs / AI PRs, eight-week average	prs.json	Hold within 25%. DORA models a rise from 5% to 6% after adoption; more is a problem (https://dora.dev/ai/roi/report/)	Revert rate
Audit-trail completeness	Mean of three coverages over explicit AI commits: Context pinned to a sha, Verification present, Reviewed-by present	trailer	100%	None. This is the compliance record

DX's change-failure-rate bands for orientation: elite 0 to 5%, high 10 to 15%, medium 16 to 30% (https://getdx.com/blog/change-failure-rate/).

Sustainability

Metric	Formula	Source	Target	Goodhart counter
Context contributors	contributors / enabled	git paths	25%	Doc freshness. Edits that do not keep documents current are noise
Organic adoption	AI-active developers never in a led cohort / enabled	trailer, roster	20%	Retention in the organic segment
Teams with an active trainer	Teams with at least one trainer who is AI-active / teams	roster, trailer	100%	Organic adoption. A trainer nobody follows is a title
Stale layer documents	Explaining documents more than 30 days behind their code	git	0	Context contributors
Context coverage	Code areas with recent commits that have a nested `AGENTS.md` or a layer rule / all such areas	git, config	80%	Root file size; coverage bought with a bloated root is worse than none
Trailer coverage	Commits with an explicit `AI-assisted:` trailer / commits	trailer	90%	None; see the floor below

Status rules

Each scorecard item is green, amber, red or n/a.

Direction	Green	Amber	Red
up	current at or above target	current at or above 70% of target	below
down	current at or below target	current at or below 130% of target	above
hold	current within tolerance of baseline	current worse than baseline by more than the tolerance	worse by more than twice the tolerance

An item with no current value, or a hold item with no baseline, is n/a and does not affect the verdict.

Verdict rules

The verdict is computed, not chosen. In order:

Baseline. The last complete week is before baselineUntil. No AI-assisted work is expected; targets do not apply yet.
Failing. Two or more guardrails red. Throughput without control is amplification of the legacy bottleneck, not adoption.
At risk. Exactly one guardrail red. Fix that before pushing adoption further.
Success. Legacy active rate green, week-8 retention green or n/a, organic adoption green or amber, context contributors not red, and no guardrail amber.
On track. Legacy active rate amber or green after eight or more weeks; or fewer than eight weeks in, whatever the numbers, because the dip is expected.
At risk. Eight or more weeks in, nothing broken, and legacy adoption not moving. Re-check the context layer and the pairing.

The J-curve

DORA's 2026 ROI report describes a J-curve: a learning curve, a verification tax and process adaptation depress outcomes before they improve, and brownfield high-complexity work gains 5 to 10% when it does improve (https://dora.dev/ai/roi/report/). Expect three to six months of wash for senior developers on mature code. The verdict rules encode this: nothing is called at risk for adoption in the first eight weeks, and hold-type guardrails use a tolerance band rather than a hard line. What the J-curve does not excuse is a red guardrail. A dip in throughput is expected; a rise in rework is not.

Per team, never per person

Every number is aggregated to team, tenure band and cohort. Individuals exist in memory during collection and are never written out. The reasons are practical and legal:

Individual tracking gets gamed and creates backlash, and it is the single fastest way to lose the long-tenured developers the rollout most needs (https://www.augmentcode.com/guides/ai-adoption-strategy-senior-vs-junior-developer-implementation).
DX's framework says not to use code-generation volume to evaluate individuals (https://getdx.com/blog/ai-measurement-framework-guide/).
Dashboards used for individual evaluation may fall under the EU AI Act's worker-management provisions (https://www.augmentcode.com/guides/eu-ai-act-2026).

If a team is one person, merge it in the roster.

The trailer-coverage floor

Every adoption number is derived from commit trailers. A commit without a trailer is counted as not AI-assisted. One pilot measured 65% AI-assisted by trailers against about 80% true use (https://barazany.dev/blog/how-we-measured-65-percent-ai-written-code-and-why-lines-dont-matter). Until trailer coverage is above 90%, every adoption metric is a floor, and the dashboard says so on the Context health tab. Install the hooks on every clone and make the trailer a CI gate before reading the adoption charts as measurements.

Inferred AI use, from vendor Co-Authored-By or Assisted-by trailers in older history, counts toward adoption and is reported separately under data quality. It does not count toward audit-trail completeness.

Setting the baseline

The Dutch word is nulmeting: the measurement before anything changes. Set metrics.since to a date well before the rollout and metrics.baselineUntil to the day access opens. Eight weeks is a good baseline; four is the minimum for the hold-type guardrails to have a reference.

During the baseline:

Install the hooks so that human-only commits carry AI-assisted: no and the trailer coverage number is real.
Fill in the roster: every developer, tenure, team, and an enabled_since for the wave they will join.
Import PR data so that cycle time, review coverage, reviewer load and PR size have a pre-rollout reference.
Run the first pulse survey.

Then uctx metrics collect and record the verdict, which will read baseline.

Targets per phase

Three phases, each with its own question. Set the targets in uctx.config.json per phase and change them in a reviewed pull request.

Phase	Dutch	Weeks	Question	What green means
Zero step	nulmeting	0 to 8	What is the reference?	Trailer coverage above 90%, roster complete, PR data present, verdict reads baseline
Build-up	opbouw	8 to 26	Is the method taking hold in the people who know the system?	Legacy active rate climbing toward 60%, adoption gap closing, week-8 retention above 70% for the first cohorts, every guardrail green or amber, days to habit for legacy developers at or under 42
Continuous	doorlopend, train the trainer	26 onward	Will it survive without outside help?	Every team has an active trainer, organic adoption above 20%, context contributors above 25%, no stale layer documents, guardrails green, verdict reads success for two consecutive collections

Lower the adoption targets for the build-up phase if the baseline shows a workforce with mostly long tenure; raise the sustainability targets for the continuous phase once the first are met. Never lower a guardrail tolerance to reach a verdict. If a guardrail is red, the correct response is to slow adoption, not to widen the band.

What to report, and to whom

Weekly, to the teams: their own funnel and guardrails, and the organisation's, side by side. Monthly, to leadership: the verdict, its reasons, and the two or three scorecard items that moved. Quarterly: the pulse survey next to the system metrics, with the gaps between perception and measurement named. Never: a ranking of people.