Summary
Before fitting a model, the Bayesian workflow involves choosing a starting point, constructing the model modularly, scaling parameters for interpretability, performing prior predictive checks, and deciding on the generative scope of the model. These steps (Section 2 of Gelman et al., 2020) set the foundation for everything that follows.
Choosing an Initial Model
The starting point for most analyses is adapting a model from a textbook, case study, or published paper applied to a similar problem. This is analogous to the software design pattern concept in engineering. Templates save time in model building and computing, and they provide useful starting points and comparison baselines.
Sometimes we start simple and add features; other times we start with a complex model and strip it down. The key insight is that model choice is provisional — we expect to revise.
Modular Construction
A Bayesian model is built from modules which can be viewed as replaceable placeholders:
- Swap a normal distribution for a -distribution or mixture
- Replace a linear regression function with nonlinear splines or Gaussian processes
- Start with a weak prior, then strengthen it if the posterior shows unrealistic values
This modular thinking reduces pressure during initial model building because you can always go back and generalize or add information as necessary. See Hierarchical Models for a common modular pattern.
Scaling and Transforming Parameters
Parameters should be put on interpretable, roughly unit scales. For example, if a parameter is expected to be around 50, model on so that 0 is interpretable. This serves two purposes:
- Ease of interpretation for setting priors
- Effective hierarchical modeling — partial pooling works better on scale-free parameters
Common transformations include logarithmic, logit, and standardization ().
Prior Predictive Checking
Prior predictive checks simulate data from the model before observing real data, using only the prior and the generative model . This reveals whether chosen priors imply sensible data ranges.
High-Dimensional Prior Implications
Even independent priors on individual logistic regression coefficients become strongly informative as the number of predictors increases — pushing predicted probabilities toward 0 or 1. Joint priors on outcomes can control this.
Generative vs. Partially Generative Models
Fully Bayesian analysis requires a generative model — a joint distribution . However, many practical models are only partially generative (e.g., regression models conditioned on predictors without modeling itself). The distinction matters for predictive simulation and Model Checking.