Ji 2024 - Asymmetric Laplace Working Likelihood

Summary

Linear quantile regression models $Q_{\tau }(y\mid x)=x^{\prime }\beta (\tau )$ and estimates $\beta (\tau )$ by minimising the check loss. Yu and Moyeed (2001) turned this into Bayesian inference by pretending errors are asymmetric Laplace: its likelihood is the exponentiated negative check loss. The AL is not a plausible data-generating distribution, so the posterior is a loss-based (“quasi”) posterior whose mode is right and whose spread is arbitrary.

Overview

Alternatives to the AL (nonparametric likelihoods: Kottas and Krnjajić 2009, Reich et al. 2010; empirical likelihoods: Otsu 2008, Yang and He 2012) exist but the AL dominates in software: R brms, bayesQR, Stata bayes:qreg.

Main Content

Check loss

${\rho }_{\tau }(u)=u\{\tau -I(u<0)\}=\{\begin{array}{ll}u\tau & u\ge 0\\ -u(1-\tau )& u<0\end{array}$. Quantile estimator: $\hat{\beta }(\tau )=\arg {\min }_{\beta }{\sum }_{i=1}^n{\rho }_{\tau }(y_i-x_i^{\prime }\beta )$ (Koenker and Bassett 1978; linear programming).

Asymmetric Laplace density

$$f_{\text{ALD}}(y_i\mid {\mu }_i,\sigma ,\tau )=\frac{\tau (1-\tau )}{\sigma }\exp \left\{-{\rho }_{\tau }\left(\frac{y_i-{\mu }_i}{\sigma }\right)\right\},{\mu }_i=x_i^{\prime }\beta (\tau ).$$

For fixed $\sigma$, maximising the likelihood in $\beta$ is exactly minimising $\sum {\rho }_{\tau }(y_i-{\mu }_i)$. The AL is right-skewed for $\tau <0.5$ and left-skewed for $\tau >0.5$, i.e. its skewness depends on the quantile you happen to ask about - impossible for one data-generating process (Koenker and Machado 1999: “rather implausible”; Sriram et al. 2013 “misspecified”; Sriram 2015, Yang et al. 2016 “working likelihood”; Geraci 2019 “pseudo-likelihood”).

Scale-mixture-of-normals representation (Kotz et al. 2001; Kozumi and Kobayashi 2011)

${ϵ}_i=y_i-{\mu }_i=\sigma \left({\theta }_1{\nu }_i+{\theta }_2\sqrt{{\nu }_i}{z}_i\right)$ with ${\theta }_1=\frac{1-2\tau }{\tau (1-\tau )}$, ${\theta }_2^2=\frac{2}{\tau (1-\tau )}$, $z_i\sim N(0,1)$, ${\nu }_i\sim \text{Exp}(1)$ independent. This gives conditionally Gaussian full conditionals and Gibbs/EM algorithms; with Stan it is not needed.

Role of $\sigma$. $\sigma$ “weights the information in the data relative to that in the prior” (Syring and Martin 2019); it controls posterior spread. Practice: $\sigma =1$ with no discussion (Yu and Moyeed 2001; Yu and Stander 2007 call it a nuisance parameter), an estimated $\sigma$ with inverse Gamma prior (Kozumi and Kobayashi 2011; Yue and Rue 2011; Wang 2012; Sriram 2015; bayesQR and bayes:qreg use IG(0.01,0.01)), uniform$(0,10)$ (Reich et al. 2010) or half-$t$(3) (brms). Sriram et al. (2013) prove posterior consistency of the point estimate with $\sigma =1$ or with a prior on $\sigma$. The AL also enables censoring (Yu and Stander 2007), random effects/latent variables (Geraci and Bottai 2007, 2014; Wang 2012) and joint longitudinal-survival models (Huang and Chen 2016).

Units trap. Because $\sigma$ has units of $y$, "$\sigma =1$" means something different when $y$ is in dollars versus thousands of dollars; the same prior-data weighting changes with units (cf. Fig. 8: $\sigma =100$ with the original response is equivalent to $\sigma =1$ with a response rescaled).

Examples

Python: AL log-likelihood equals negative check loss

import numpy as np
def check_loss(u, tau): return u * (tau - (u < 0))
def al_loglik(y, mu, sigma, tau):
    u = (y - mu) / sigma
    return np.log(tau * (1 - tau) / sigma) - check_loss(u, tau)
rng = np.random.default_rng(1); y = rng.standard_normal(500)
tau = 0.9
grid = np.linspace(0, 2, 401)
ll = [al_loglik(y, m, 1.0, tau).sum() for m in grid]
loss = [check_loss(y - m, tau).sum() for m in grid]
print(grid[np.argmax(ll)], grid[np.argmin(loss)], np.quantile(y, tau))   # all agree

Connections

Frequentist: Quantile Regression. The corrected posterior: Ji 2024 - Yang-Wang-He Sandwich Adjustment, Ji 2024 - Infinitesimal Jackknife Standard Errors, Ji 2024 - Gibbs Posterior and Learning-Rate Calibration.

See Also

Ji 2024 - Valid Standard Errors for Bayesian Quantile Regression - Overview