Guide · foundations
Prediction is not intervention
The most common way to get a confidently wrong answer from data is to build a model that is extremely good at the wrong question. This is not a modelling mistake — the model is fine. It is a mistake about what kind of question was asked.
The example everybody has met
Umbrellas and rain
Count umbrellas on the street and you can predict rainfall very well. The correlation is real, it is stable, it will hold up in cross-validation, and it will hold up next year. As a forecast, umbrellas are excellent.
Now take a decision with it. Confiscate every umbrella in the city and the forecast says the rain stops. Obviously it does not. The model was never wrong — it answered “given that I see umbrellas, what should I expect?” correctly, every time. It simply has nothing to say about “what happens if I change the umbrellas”, because nothing in the data ever changed them for a reason unrelated to the weather.
The example is silly on purpose. In real work the same structure arrives dressed as something reasonable: hospitals with more staff have worse outcomes, students in smaller classes score lower, customers who receive more support emails churn more. Every one of those is a true, stable, predictive relationship. Not one of them is an instruction.
The test, in one sentence
If the answer would change depending on why the variable took the value it did, the question is causal and a predictive model cannot answer it — however accurate that model is.
Accuracy is not evidence either way
A model that predicts held-out data perfectly has demonstrated that the world it was trained on keeps behaving like itself. A decision proposes to make the world stop behaving like itself. That is precisely the case the test set cannot contain.
The distinction, sharpened
Three rungs, and only the first is free
Pearl's ladder is the standard way to keep these apart. The rungs are not degrees of difficulty — they are different questions, and no amount of data from a lower rung answers a higher one on its own.
Seeing — P(Y | X)
“Among the households that had a filter, what was the illness rate?” This is a fact about the data. It requires no assumptions beyond the sample being what it claims to be, and machine learning is extremely good at it.
answerable from the data alone
Doing — P(Y | do(X))
“If we gave a household a filter, what would the illness rate be?”
The do is not notation for emphasis: it means the value was set
from outside, breaking whatever normally determines it. In the data, filters
were taken up by households that chose to; under do, they are
handed out regardless. Those are different populations of filter-users.
needs the data and a claim about mechanism — this is what identification supplies
Imagining — P(Yx | X = x′, Y = y′)
“This household got a filter and stayed ill. Would they have been ill without it?” A question about a specific unit in a world that did not happen. Strictly harder again, and mostly out of scope here — but worth knowing it is a separate rung, because a great deal of everyday reasoning quietly lives on it.
needs a full model of the mechanism, not just its graph
Nearly every question a decision-maker asks is on rung two. Nearly every tool that gets pointed at it is built for rung one. That mismatch is the subject of this whole shelf.
What it looks like when it goes wrong
The same data, two answers, both computed correctly
This is not hypothetical. On the identify page, the same 5,000 rows are fitted twice — once ignoring a common cause and once adjusting for it — and the two estimates are on opposite sides of the truth.
Ignoring the common cause
95% Wald. The truth is 2.0, and the interval does not contain it. Nothing about the arithmetic is wrong.
Adjusting for it
Same rows, same estimator. The only difference is that something was conditioned on — and which something is not a question the data can answer.
The uncomfortable part
Both fits report a tight interval. Neither reports doubt. If you had only ever seen the first one, nothing in the output would tell you it was wrong — which is why the decision about what to adjust for has to be made before fitting, from a claim about the world, and written down where somebody can disagree with it.
In practice
Four questions worth asking out loud
Guide 1 of 12