Concept - Latent Class and Dirichlet Process Models for Multiple Systems Estimation

Summary

LCMCR (Manrique-Vallier 2016) models the $2^K$ list-inclusion table as a mixture of product-Bernoulli latent classes with a truncated Dirichlet-process (stick-breaking) weight prior, and treats the unseen individuals by data augmentation. It gives flexible heterogeneity without picking list interactions, but Aleshin-Guendel (2020) shows its families are non-identifiable when the number of classes exceeds $K/2$, and Silverman (2020) shows unstable results on the UK and Netherlands data.

Overview

The log-linear family (Silverman 2020 - Poisson Log-Linear Model for Multiple Lists) models dependence between lists with explicit interaction terms, which become unwieldy and sparse for many lists. The latent-class approach instead assumes dependence arises from unobserved heterogeneity: given an individual’s latent class, lists include them independently ($M_{th}$; Aleshin-Guendel 2020 - Mth Model and Identifiability Definition).

Main Content

Latent class capture-recapture model

Each individual $i=\mathrm{1..}N$ has class $c_i\sim \text{Categorical}({\nu }_1..{\nu }_J)$ and inclusion indicators $x_{ik}\mid c_i=j\sim \text{Bernoulli}({\lambda }_{jk})$ independently over lists $k=\mathrm{1..}K$. The observed cells are those with some $x_{ik}=1$; $N$ includes $N-n$ unseen individuals with $x_i=0$. Marginal cell probability: ${\pi }_h={\sum }_j{\nu }_j{\prod }_k{\lambda }_{jk}^{h_k}(1-{\lambda }_{jk}{)}^{1-h_k}$; the missing cell is ${\pi }_0={\sum }_j{\nu }_j{\prod }_k(1-{\lambda }_{jk})$.

Dirichlet-process version (LCMCR; details to verify)

Replace finite $J$ by a stick-breaking prior: ${\nu }_j=v_j{\prod }_{l<j}(1-v_l)$, $v_j\sim \text{Beta}(1,\alpha )$, truncated at $J^{\ast }$ classes, with a hyperprior on the concentration $\alpha$ and Beta priors on ${\lambda }_{jk}$; the population size has a prior of the form $p(N)\propto 1/N$ (improper) or a bounded alternative, and inference is by Gibbs sampling with data augmentation of the unobserved individuals (as I recall; confirm exact hyperparameters against the paper and the R package LCMCR). Reported summary: posterior of $N$, including the uncertainty from the augmented zero-pattern count.

Identification bound (from Aleshin-Guendel 2020)

With $J^{\ast }$ distinct classes and $K$ lists, the family is identified iff $2J^{\ast }\le K$. The LCMCR default $J^{\ast }=5$ requires $K\ge 10$; the recommended $J^{\ast }=K$ never is. Practical reading: with $K\le 9$ lists the posterior for $N$ is partly set by the prior over class structure and the truncation level.

Computation. In a probabilistic-programming implementation, classes are summed out per cell using logsumexp rather than sampled (Concept - Marginalising Latent Discrete Counts for HMC); the unseen cell contributes the term ${\pi }_0$ through $n\sim \text{Binomial}(N,1-{\pi }_0)$ combined with a prior on $N$. Label switching is handled by ordering constraints, and the cell-level likelihood avoids per-individual class indicators when counts are aggregated (a $2^K$ table has at most $2^K-1$ distinct patterns).

Diagnostics. (a) Vary $J^{\ast }$ below and above $K/2$ and compare posterior intervals for $N$ (stability). (b) Merge two small lists and re-estimate: a good estimator should not move much (this exposed Netherlands instability in Silverman 2020 - Madigan-York and Dirichlet Process Benchmarks). (c) Prior-versus-posterior comparison for $N$.

Examples

PyMC-style sketch (untested): finite latent class MSE on a table

import numpy as np, pymc as pm, pytensor.tensor as pt, itertools
K, J = 4, 2                                   # keep 2J <= K for identification
H = np.array([h for h in itertools.product([0,1], repeat=K) if any(h)])
counts = np.array([...])                      # observed cell counts, one per row of H
n = counts.sum()
with pm.Model() as lcm:
    nu = pm.Dirichlet("nu", np.ones(J))
    lam = pm.Beta("lam", 1, 1, shape=(J, K))
    # log cell probabilities per class: (J, 2^K-1)
    logc = (H[None] * pt.log(lam)[:, None] + (1-H[None]) * pt.log1p(-lam)[:, None]).sum(-1)
    logpi = pm.math.logsumexp(pt.log(nu)[:, None] + logc, axis=0)
    log_pi0 = pm.math.logsumexp(pt.log(nu) + pt.log1p(-lam).sum(1))
    # conditional multinomial likelihood of observed cells, then add N | n via a prior on N
    pm.Potential("cells", pt.sum(counts * (logpi - pt.log1p(-pt.exp(log_pi0)))))
    pm.Deterministic("dark_per_observed", pt.exp(log_pi0) / (1 - pt.exp(log_pi0)))

Posterior draws of ${\pi }_0$ give $N=n/(1-{\pi }_0)$ (point) and, with a Negative-Binomial predictive for the unseen count, a full interval. Not tested; check with simulated data where the truth is known.

Relevance to marketing measurement

Latent class models for overlapping audiences (heavy vs light users) mirror the $M_{th}$ idea: dependence between data sources is explained by unobserved segments rather than pairwise interaction terms. The identification warning transfers directly: with only a few sources you cannot support many latent segments.

Connections

• Nonparametric priors: Nonparametric Models Overview.
• Heterogeneity as a hierarchical model: Hierarchical Models.
• Why identifiability matters here: Aleshin-Guendel 2020 - Latent Class Identifiability Theorem and LCMCR.

See Also

• Concept - Closed-Population Capture-Recapture and Otis Models
• Concept - Unseen Species and Latent Class Links