Chipman 2010 - Extensions, Discussion and Practical Use

Summary

BART is modular: add linear terms, random effects or a second parametric component, or go multivariate (SUR-style), by adding one MCMC step per component. The discussion stresses the three ingredients (sum-of-trees, regularization prior, backfitting MCMC), the robustness of the defaults, and open issues (spurious splits inflate variable usage). This note adds a practitioner reading with caveats not in the paper.

Overview

Section 1 positions BART against boosting (sequential residual fitting), bagging/random forests (independent randomized trees, averaged) and Bayesian single-tree model averaging (posterior-weighted averages of whole-function trees). BART is none of these: it is a single model with all $m$ trees jointly posterior-sampled, and “BART is not equivalent to posterior averaging of single tree fits of the entire function $f$“.

Main Content

Section 1 comparisons

Method	How trees combine	Where regularization comes from
Boosting (Freund-Schapire; Friedman 2001)	Sequentially fit residuals	Learning-rate shrinkage, tree depth cap
Bagging / random forests (Breiman)	Independent trees, average	Averaging, randomization
Bayesian single-tree averaging (CGM98 and others)	Posterior-weighted average of whole-function trees	Tree prior
BART	Additive sum, jointly sampled by backfitting	Prior shrinking each tree (depth penalty and $1/\sqrt{m}$ leaf scale)

Section 7 extensions

Modular extensions (eqs. 28-29)

Partially parametric: $Y=h_1(x)+h_2(z)+\epsilon$, $h_1$ a sum of trees, $h_2$ parametric (linear terms or random effects) with one extra MCMC step for its parameters. Multivariate: $Y_i=h_i(x_i)+{\epsilon }_i$, $({\epsilon }_1..{\epsilon }_p)\sim N(0,\Sigma )$ with each $h_i$ a sum of trees; draw each $h_i^{\ast }\mid \{h_j^{\ast }{\}}_{j\ne i},\Sigma$, then $\Sigma \mid h^{\ast }$. With common $x_i$ this generalizes multivariate regression; with different $x_i$ it generalizes Zellner’s SUR.

Reported follow-on work (as of 2008-2009): SBART (spatial component added, improved merging of data sets over census methods); TF-DNA binding prediction (BART compared favorably with linear regression, MARS, neural nets, SVM and boosting); BACT (BART for credit risk / insolvency); CBART (probit BART for phishing-email detection, outperforming logistic regression, random forests, SVM, CART, neural nets and BART); a hockey penalty analysis showing referee bias (Abreveya & McCulloch). The authors note these provide “further evidence for the remarkable potential of BART”; treat as the authors’ summary of third-party reports.

Section 8 discussion points

• Trees account for parts of the fit, enforced by the prior; hyperparameters are interpretable and defaults “off the shelf”.
• Backfitting MCMC converges quickly; draws give means, intervals, partial dependence.
• Accuracy: competitive with boosting, lasso, MARS, neural nets, random forests; robust to hyperparameters; effective in high-$p$ with low-dimensional signal; model-free variable selection via small $m$.
• Open prior question: the default $\alpha =0.95$ puts only 5% on a single-node tree; spurious splits don’t hurt accuracy but inflate variable-usage frequencies, so a smaller $\alpha$ may help selection.

Examples

Partially parametric BART for a response with known structure (PyMC sketch, untested).

with pm.Model() as m:
    f_x = pmb.BART("f_x", X=X_nonlinear, Y=y, m=100)    # flexible part
    beta = pm.Normal("beta", 0, 1, shape=Z.shape[1])    # parametric part (e.g. promo dummies)
    mu = f_x + pm.math.dot(Z, beta)
    pm.Normal("y", mu, pm.HalfNormal("s", 1), observed=y)

Practitioner caveats (editorial, not from the paper)

Not claims of the 2010 paper

• Tree models are piecewise constant: they extrapolate flat beyond the data, so a BART-based counterfactual under a spend level never seen in the data inherits that flatness (compare Hooker Mentch Zhou 2021 - Extrapolation Mechanism in Forests and Neural Nets).
• Posterior intervals reflect the prior and model, not frequentist coverage (the paper itself shows 87-93% average coverage of nominal 90% intervals, lower near the edge of the $x$ space).
• Use the default only after checking sensitivity to $m$, $k$ and the noise prior; the Friedman $p=1000$ result shows defaults can fail badly.

Relevance to marketing measurement

BART as the outcome model inside an uplift or doubly-robust estimator is described in Nonparametric Causal Inference; as nuisance control for baseline demand in an MMM it can sit next to parametric media terms via the partially parametric form above (eq. 28). It does not by itself impose diminishing returns or non-negativity of media effects.

Connections

Random forest causal analogues: Honest Trees and Causal Forests, Generalized Random Forests - Local Moment Equations. Posterior predictive checks of BART fits: Posterior Predictive Checking.

See Also

Chipman 2010 - BART - Overview, Gaussian Process Regression.