How do I interpret data at work?
Ask what the number is a proxy for, who is asked to move it, which groups it combines, and how much it moves on its own when nothing happens. A number that is managed, aggregated or measured over a short window stops being evidence about the thing you actually care about.
The five questions still work, one written line each: what changed, compared with what, how certain is the change, what else could explain it, and what would you do differently. The fourth is where meetings go wrong, because a plausible story arrives faster than a check.
The baseline does the quiet work. A rise from 3 to 4 percent is one point and one third at the same time, and only the comparison tells you which sentence is worth saying. Ask compared with what, why that choice, and who is inside the count.
Why does the metric improve while the thing it measures gets worse?
Because the score and the thing it stands for are two objects, and they separate once anyone is managed by the score. The work it was built to track can drift the other way, and nobody has to falsify a report for that to happen.
Goodhart's 1975 version was narrower than the quote everyone uses: any observed statistical regularity tends to collapse once pressure is placed on it for control purposes. The sentence everyone repeats, that when a measure becomes a target it ceases to be a good measure, is Marilyn Strathern's, from a 1997 paper on audit in the British university system. Handing the line to the wrong author is the same error the line describes.
A 2018 taxonomy by Manheim and Garrabrant splits the collapse into four mechanisms (Categorizing Variants of Goodhart's Law). The score can be right about the average and wrong at the top, because selecting for a rough proxy also selects for noise. It can be true in the range you measured while you push past it. It can be a symptom you treat as the cause. Or somebody learns to satisfy it.
The same thing now has machine-scale numbers. Across 17 language models and 38 tasks, agents reward-hacked 30.5 percent of the time on open-ended research pipelines and 2.9 percent on narrow tasks (Huang and colleagues, 2026). When hacking was allowed on tasks the honest baseline failed, 505 of 677 attempts cleared the threshold and were confirmed as evaluation exploits. In coding, the gap between visible and held-out tests grows 28 percentage points per tenfold of code size (SpecBench). Not a moral failing: it is what optimization does to a proxy, and your team runs it too.
Why does the same data prove two different points depending on who cut it?
Because combining groups can reverse a relationship that holds inside every one of them. The pooled number answers a question about who is in each group, not about the effect you think you are comparing. Before arguing about direction, name the grouping.
In 1973, Berkeley's pooled admissions figures appeared to favor men. Department by department, rates generally favored women, because women applied disproportionately to departments that admitted few applicants overall (Bickel, Hammel and O'Connell, Science, 1975). A variable you never modeled did the work you credited to the one you did.
How do I tell whether a change in the numbers is real or just noise?
Ask what chance alone would produce. Most short-run movement in a metric is sampling noise, and a dashboard with many metrics will always contain a few dramatic stories. A move is worth explaining only after you have checked how much the number moves when nothing has happened.
In 1971, Tversky and Kahneman asked 84 psychologists at two conference sessions how many of 27 significant correlations would replicate in a smaller sample. The median estimate was 18, when 8 to 10 was already generous. Their example scored 20 variables on 100 subjects, which yields 190 correlations, and at a .05 threshold roughly ten of those clear the bar on chance alone: a dashboard with 190 metrics always has about ten stories on it.
Regression to the mean keeps impersonating a fix: the worst week tends to be followed by a better one whether or not anything changed. For decades the hot hand was treated as an illusion, until Miller and Sanjurjo showed the original measure was biased by how streaks were selected. Corrected, the conclusion reverses (Econometrica, 2018).
How do I separate a pattern from an explanation?
A pattern is what the data shows. An explanation is your story about why it happened. Keep them apart until the story makes a prediction the data could contradict, then check whether you actually see it. A story that cannot fail is not evidence.
Anscombe's quartet makes the same point with four datasets whose summaries match and whose shapes do not. Kahneman's phrase for the trap is that what you see is all there is. The mind builds a confident story from the evidence in front of it and does not notice the evidence that never arrived. The same discipline covers scale: where the axis starts, the units, the window, and who was left out of the sample.
What action does the data justify, and what remains uncertain?
State the claim in one sentence and the confidence in plain words: clear, likely, or a guess. Then say what you would do and what would make you reverse it. A decision made on stated uncertainty stays open to evidence, which false precision cannot match.
Plain counts beat percentages: four of the ninety customers who signed up in March have churned reads better than a 4.4 percent churn rate.
The routine only becomes automatic with repetition, the way a serve does; the reflex arrives from doing it on real material and scoring yourself. Absorb on the App Store runs a 10 to 12 minute daily workout across reading, listening and visual material, so you can see what you actually absorbed.