Tsagris 2018 - Dirichlet Regression with Zeros - Zero Adjusted Dirichlet Regression

Summary

ZADR treats each pattern of non-zero components as a sub-population with probability ${\theta }_b$ (MLE $n_b/n$). Within each, the non-zero sub-composition is Dirichlet. A common coefficient vector and common precision $\varphi$ are shared across patterns through selection matrices $Q_b$, so rare zero patterns need not be estimated separately. This works because sub-compositions of a Dirichlet are Dirichlet with the corresponding parameters.

Overview

Existing strategies: (i) imputation of rounded zeros (Palarea-Albaladejo and Martin-Fernandez 2008; robust version of Martin et al. 2012; robCompositions), which changes all components of affected vectors and handles large point masses poorly; (ii) explicit zero modelling: Aitchison (1982, pp. 271-273), Zadora et al. (2010), Stewart and Field (2011), Bear and Billheimer (2015), Scealy and Welsh (2011, square-root transform to the sphere with a Kent distribution), Leininger et al. (2013, spatial power transform). ZADR extends the second family to Dirichlet regression with covariates.

Main Content

Zero-pattern mixture (Eqs. 5-6)

Let $G$ index the non-zero components and ${\theta }_b=P[G=g_b]$, ${\sum }_{b=1}^B{\theta }_b=1$, over $B$ populations (subsets of non-zero components). Then

$$f_{\mathbf{Y}}(\mathbf{y})=\sum _{b=1}^B{f}_{\mathbf{Y},G}(\mathbf{y},g_b)=f_{b^{\ast }}({\mathbf{y}}_{b^{\ast }}){\theta }_{b^{\ast }},$$

where $b^{\ast }$ is the pattern of $\mathbf{y}$ and ${\mathbf{y}}_{b^{\ast }}$ its $D_{b^{\ast }}$ non-zero components, with ${\mathbf{Y}}_{b^{\ast }}\sim \text{Dir}({\varphi }_{b^{\ast }},{\mathbf{a}}_{b^{\ast }}^{\ast })$.

Full-pattern log-likelihood (Eq. 7) and MLE of $\theta$

$$\ell =\sum _b[n_b\log {\theta }_b+n_b\log \Gamma ({\varphi }_b)-\sum _{j\in S_b}\sum _{i=1}^{D_b}\log \Gamma ({\varphi }_b{a}_{bij}^{\ast })+\sum _{j\in S_b}\sum _{i=1}^{D_b}({\varphi }_b{a}_{bij}^{\ast }-1)\log y_{bij}],$$

$S_b$ = observations with pattern $b$. The MLE is ${\hat{\theta }}_b=n_b/n$; the rest decouples into $B$ separate Dirichlet regressions. In practice many patterns have few observations, so estimation is unreliable.

ZADR with shared parameters (Eq. 8)

Following Bear and Billheimer (2015), assume the regression coefficients (and $\varphi$) are the same across patterns: ${\mathbf{Y}}_b\sim \text{Dir}(\varphi ,Q_b{\mathbf{a}}^{\ast })$ with a common ${\mathbf{a}}^{\ast }$ from Eq. (1). The selection matrix $Q_b$ picks the non-zero positions, e.g. for $D=4$ with zeros in positions 1 and 3:

$$Q_1=\left(\begin{array}{cccc}0& 1& 0& 0\\ 0& 0& 0& 1\end{array}\right).$$

With $Q_b{\mathbf{a}}^{\ast }[i]$ the $i$-th element of $Q_b{\mathbf{a}}^{\ast }$:

$$\ell =n\log \Gamma (\varphi )+\sum _b{n}_b\log {\theta }_b-\sum _b\sum _{j\in S_b}\sum _{i=1}^{D_b}\log \Gamma (\varphi Q_b{\mathbf{a}}_j^{\ast }[i])+\sum _b\sum _{j\in S_b}\sum _{i=1}^{D_b}(\varphi Q_b{\mathbf{a}}_j^{\ast }[i]-1)\log y_{bij}.$$

This is called simple ZADR. Replacing $\varphi$ by ${\varphi }_j^{\ast }=e^{{\mathbf{x}}_j^{\top }\mathbf{\gamma }}$ (Eq. 9) gives the full ZADR with covariate-dependent precision; the models are compared by likelihood-ratio test.

Why it works

A Dirichlet with some components set to zero has non-zero components that are Dirichlet with the same parameters (marginalising/closing a Dirichlet gives a Dirichlet). The extra assumption needed for consistency: the vectors containing zeros, modelled by lower-dimensional Dirichlets, share the full Dirichlet’s parameters (not proved by the authors).

Maximisation (Sec. 3.3)

The log-likelihood is the sum of the zero-free part and the part from vectors with zeros. Algorithm:

1. Apply ALR to zero-free vectors: $z_i=\log (y_i/y_1)$, $i=2,\dots ,D$ (Eq. 10).
2. Least squares $\hat{\mathbf{B}}=({\mathbf{X}}^{\top }\mathbf{X}{)}^{-1}{\mathbf{X}}^{\top }\mathbf{Z}$ (Eq. 11): the Aitchison model.
3. Use $\hat{\mathbf{B}}$ as starting values for maximising the Dirichlet regression likelihood (2) (initial coefficients; for model 4 random starting $\varphi$).
4. Use these as starting values for maximising the ZADR log-likelihood (8); obtain final estimates.

Authors report no empirical sensitivity to starting values; the good start reduces local-maximum risk.

Examples

Python: ZADR log-likelihood contribution for one vector

import numpy as np
from scipy.special import gammaln
def zadr_loglik_row(y, a_star, phi):
    """y: composition with zeros; a_star: full-length mean vector summing to 1."""
    nz = y > 0                      # Q_b picks non-zero positions
    alpha = phi * a_star[nz]
    return gammaln(phi) - gammaln(alpha).sum() + ((alpha - 1) * np.log(y[nz])).sum()
print(zadr_loglik_row(np.array([0.7, 0.0, 0.3]), np.array([0.5, 0.2, 0.3]), 10.0))

Note: this is the Dirichlet kernel of the observed sub-composition, with $\Gamma (\varphi )$ replaced by $\Gamma ({\sum }_{i\in nz}\varphi a_i)$ in an exact sub-Dirichlet; the paper’s Eq. 8 keeps $\Gamma (\varphi )$ because the common $\varphi$ is the total of the full parameter. Check against the paper before using in production.

Connections

Compared against imputed log-ratio regression in Tsagris 2018 - Dirichlet Regression with Zeros - Examples and Simulation Results. Counterpart conceptual discussion: Greenacre 2023 - Wide Data, Zeros and Quasi-Coherence.

See Also

Tsagris 2018 - Dirichlet Regression with Zeros - Overview