Breiman 2001 - Critique of Data Models - Goodness of Fit, Multiplicity and Predictive Accuracy

Summary

§5-6 are the case against relying on data models alone. A fitted model’s conclusions are “about the model’s mechanism, and not about nature’s mechanism,” so a poor emulation of nature gives wrong conclusions. The standard checks cannot tell: omnibus goodness-of-fit tests have little power unless the alternative’s direction is specified (Bickel, Ritov and Stoker 2001; a 7-dimension simulation did not reject linearity until the nonlinearity was extreme), and residual analysis fails beyond four or five dimensions. Because these checks give yes/no answers, many models pass, and different passing models tell different stories. Breiman’s remedy is to measure how well the model box emulates nature’s box by predictive accuracy on held-out data (cross-validation, Stone 1974; test sets) and to widen the toolkit beyond discriminant analysis, logistic and linear regression.

Overview

This note covers the paper’s main methodological complaint. It links directly to the vault’s material on overfitting and false discovery: significance computed under an unchecked model, data-dependent model tinkering that invalidates the test, and an $R^2$ inflated by many parameters.

Main Content

§5 The use of data models (p. 202)

The template of applied statistics: “Faced with an applied problem, think of a data model.” It rests on the belief that a statistician can invent a reasonably good parametric class for a complex mechanism devised by nature. But when a model is fit to draw quantitative conclusions:

Breiman's core logical point (§5, p. 202)

• “The conclusions are about the model’s mechanism, and not about nature’s mechanism.”
• It follows that “If the model is a poor emulation of nature, the conclusions may be wrong.”

He calls these “truisms” that “have often been ignored in the enthusiasm for fitting data models.” A few decades earlier even residual analysis and goodness-of-fit tests were often skipped; “The belief in the infallibility of data models was almost religious.”

§5.1 An example: model (R) (pp. 202-203)

Model (R) — the Gaussian linear regression data model (§5.1, p. 202)

Data are independent draws from

$$y=b_0+\sum _{m=1}^M{b}_m{x}_m+\epsilon ,\epsilon \sim N(0,{\sigma }^2),$$

with coefficients $\{b_m\}$ and ${\sigma }^2$ to be estimated. Given that the data are generated this way, one can derive tests, confidence intervals, distributions of the residual sum of squares and asymptotics.

Breiman’s complaints about practice under model (R):

• Significance levels for coefficients were derived “with little consideration as to whether the data on hand could have been generated by a linear model.” Hundreds or thousands of articles claimed proof of something because a coefficient was significant at 5%.
• Goodness of fit was shown mostly by $R^2$, “often closer to zero than one” and inflatable by using too many parameters; nothing else was done.

Faculty salary and gender discrimination (§5.1, pp. 202-203)

Setup. A well-known member of a university statistics department assessed gender discrimination in faculty salaries. Response: salary. Predictors: 25 variables describing academic performance (papers published, journal quality, teaching record, evaluations, etc.) plus a binary gender indicator.

Analysis. Linear regression; the gender coefficient was significant at 5%, which “was accepted as gospel” as strong evidence of discrimination.

Breiman’s questions, which come before any model. Can the data gathered answer the question posed? Is inference justified when the sample is the entire population? Should a data model be used at all? “The deficiencies in analysis occurred because the focus was on the model and not on the problem.”

Mosteller and Tukey (1977) had already warned that “The whole area of guided regression is fraught with intellectual, statistical, computational, and subject matter difficulties.” David Freedman’s critiques of regression (1994), path models (1987) and data modelling (1991, 1995) are cited as rare exceptions.

§5.2 Problems in current data modeling (p. 203)

Low power of goodness-of-fit tests and residual analysis (§5.2, p. 203)

1. Simulation. Breiman set up a simulated regression problem in seven dimensions with a controlled amount of nonlinearity. “Standard tests of goodness-of-fit did not reject linearity until the nonlinearity was extreme.”
2. Theory. Bickel, Ritov and Stoker (2001): goodness-of-fit tests “have very little power unless the direction of the alternative is precisely specified.” Hence omnibus tests, which test many directions at once, have little power and will not reject until lack of fit is extreme.
3. Data-dependent tinkering. If the model is modified on the basis of the data — variables deleted, nonlinear combinations added — “then goodness-of-fit tests are not applicable.”
4. Residual analysis. At a 1993 Berkeley seminar William Cleveland, “one of the fathers of residual analysis,” admitted it could not uncover lack of fit in more than four to five dimensions. Published residual-analysis papers are confined to two or three variables. In higher dimensions, interactions can produce passable residual plots for a variety of models; “A residual plot is a goodness-of-fit test, and lacks power in more than a few dimensions. An acceptable residual plot does not imply that the model is a good fit to the data.”
5. Arbitrariness. In Landwehr, Pregibon and Shoemaker (1984), a detailed residual analysis of a three-variable logistic model, each of the four discussants proposed other methods, leaving “an unsettled sense about the arbitrariness of residual analysis.”
6. Neglect. Many JASA application articles that fit data models discuss fit very little; how well the model fits is “of secondary importance compared to the construction of an ingenious stochastic model.”

Proof sketch of point 2 (intuition). An omnibus test spreads its power over every direction of departure. With a fixed sample and a $p$-dimensional space of alternatives, power against any one direction falls as $p$ grows, so only gross misfit is detected. A test aimed at a specific alternative concentrates its power there.

§5.3 The multiplicity of data models (pp. 203-204)

The premise. One goal of statistics is to extract information about the mechanism producing the data. “The greatest plus of data modeling is that it produces a simple and understandable picture of the relationship between the input variables and responses” (p. 203). Logistic regression is popular in classification because it gives a linear combination of variables whose weights indicate variable importance: “The end result is a simple picture of how the prediction variables affect the response variable plus confidence intervals for the weights.” The multiplicity argument that follows undermines exactly this picture.

The problem. Two statisticians fit different models to the same data, both apply standard checks and are both convinced of fit — “Yet the two models give different pictures of nature’s mechanism and lead to different conclusions.” McCullagh and Nelder (1989): “Data will often point with almost equal emphasis on several possible models, and it is important that the statistician recognize and accept this.” Because fit checks are yes/no and lack power in more than a few dimensions, many models are acceptable, and “There is no way, among the yes–no methods for gauging fit, of determining which is the better model.” Mountain and Hsiao (1989) note the difficulty of encompassing all rival models and the dubious finite-sample validity of encompassing tests.

Medical journals had adopted the Cox model (survival) and logistic regression (survive/non-survive) as de facto publication standards, as 5% significance had become in the social sciences. “That different survival models, equally well fitting, could give different conclusions is not an issue.” This section sets up §8 (Breiman 2001 - Rashomon Effect and the Multiplicity of Good Models).

§5.4 Predictive accuracy (p. 204)

Emulation accuracy as the measure of fit (§5.4, p. 204)

Put a case $\mathbf{x}$ down nature’s box to get $y$ and down the model box to get $\hat{y}$. “The closeness of $y$ and $\hat{y}$ is a measure of how good the emulation is.” For a data model: fit the parameters, use the model to predict, and measure prediction quality. Unmeasured variables appear as noise, so prediction is rarely perfect, but the extent of emulation measures how well the model reproduces the phenomenon.

McCullagh and Nelder note that in-sample agreement is biased by the number of parameters; Breiman agrees — too many parameters overfit and give “a biased estimate of accuracy” — but bias can be removed with cross-validation (Stone 1974) or, with more data, a held-out test set. Mosteller and Tukey (1977): “Cross-validation is a natural route to the indication of the quality of any data-derived quantity … We plan to cross-validate carefully wherever we can.” Predictive-accuracy estimates are rare in JASA; publishing them would set standards for comparing models, as is common in machine learning. The glossary formalises this as generalization error (Breiman 2001 - Final Remarks and Glossary of Algorithmic Terms).

§6 The limitations of data models (pp. 204-205)

• Multivariate tools are “frozen at discriminant analysis and logistic regression in classification and multiple linear regression in regression.” Nobody believes multivariate data are multivariate normal, but that model fills textbooks.
• For uncontrolled observations of complex physical, chemical or biological systems, assuming a statistician-chosen parametric model “can result in questionable conclusions that cannot be substantiated by appeal to goodness-of-fit tests and residual analysis.” Simple parametric models on medical or financial data lose accuracy and information relative to algorithmic models (§11).
• “If all a man has is a hammer, then every problem looks like a nail.” Some problems have stopped looking like nails. Breiman conjectures that the response has been more complicated data models, such as Bayesian methods with MCMC, which lose the advantage of a simple, clear picture of nature’s mechanism.
• Insisting on a data model “imposes an a priori straight jacket”; the best solution might be a data model or an algorithmic one — “The data and the problem guide the solution.”
• Perhaps most damaging: statisticians ruled themselves out of the most interesting problems created by computers’ ability to store and manipulate data, which nonstatisticians are now solving.

Examples

Goodness-of-fit versus held-out error on a marketing regression

Setup. A media-mix regression with 10 channels, adstock and saturation is fit to 156 weeks. Residual plots against fitted values and against time look unremarkable; Durbin-Watson is acceptable.

Breiman’s argument. With 10+ transformed regressors, the residual plot has little power (point 4 above), and if the adstock decay and saturation shapes were chosen after looking at the data, nominal fit tests do not apply (point 3). A better check: refit on weeks 1-104 and predict weeks 105-156 against a naive baseline (e.g. last-year-same-week or a trend-plus-seasonality model). If the model barely beats the baseline, its channel coefficients deserve little trust. The rejoinder makes the same argument explicitly for causal-factor claims (p. 229).

Caveat. Held-out accuracy says little about whether channel coefficients are causal: two specifications can forecast equally well while attributing sales very differently (§5.3 and §8). Experiments are the check on that (Chan Perry 2017 - Challenges in Media Mix Modeling - Overview).

Connections

• The salary example is the kind of single significant coefficient whose credibility Ioannidis 2005 - Why Most Published Findings Are False - Overview analyses through PPV, and the §5.2 point that tinkering with the model on the basis of the data makes goodness-of-fit tests inapplicable is the mechanism behind Researcher Degrees of Freedom and Simmons Nelson Simonsohn 2011 - False-Positive Psychology - Overview.
• The Bayesian counterpart of “check by prediction” is Posterior Predictive Checking and leave-one-out comparison in Model Selection Using Predictive Performance; Model Selection and Overfitting covers in-sample bias from parameters.
• §6’s complaint about Bayesian/MCMC models getting more complex is answered in part by the Bayesian Workflow - Overview, which relies heavily on predictive checks.
• The multiplicity argument is formalised as the Rashomon set in Fisher Rudin Dominici 2019 - Model Class Reliance - Overview.

See Also

• Breiman 2001 - Statistical Modeling The Two Cultures - Overview
• Breiman 2001 - Rashomon Effect and the Multiplicity of Good Models — §8 continues §5.3
• Breiman 2001 - Discussion by Cox, Efron, Hoadley and Parzen and Rejoinder — Cox calls this critique partly a caricature
• Model Checking
• Overfitting and Information Criteria