Barker 2018 - Relative Abundance, Alder Flycatcher Example and Discussion

Summary

If the goal is relative abundance after controlling for detection covariates, Poisson regression (log link, optionally with site random effects) extracts what N-mixtures reliably deliver. On alder flycatcher data, site-covariate effects agree across N-mixture, Poisson, random-effects Poisson and double-Poisson fits, while baseline abundance and detection are model-dependent.

Overview

With constant $p$ (or covariate-explained $p$), replicate counts give a minimum variance unbiased estimate ${\bar{n}}_i$ of ${\mu }_i=E[n_{ij}]$ (Gauss-Markov), so relative abundance is reliably estimable. Absolute abundance requires a decomposition $\mu =pN$ that counts alone cannot support.

Main Content

Poisson count model for relative abundance (Eq. 7-9)

$n_{ij}\sim \text{Poisson}({\mu }_{ij})$ with ${\mu }_{ij}=p{N}_i=e^{{\alpha }_p+{\alpha }_N+z_i{\beta }_N}$ ($p<1$ is relaxed: without marking, multiple counting makes $E[n_i]<N_i$ unnecessary). With $t_i={\sum }_j{n}_{ij}$ and $T={\sum }_i{t}_i$ the likelihood factorises as

$$[\{n_{ij}\}\mid \{{\mu }_{ij}\}]=\underset{[\{n_{ij}\}\mid \{t_i\}]}{\underbrace{\prod _i\frac{t_i!}{{\prod }_j{n}_{ij}!}{\left(\frac{1}{J}\right)}^{t_i}}}\times \underset{[\{t_i\}\mid T,{\beta }_N]}{\underbrace{\frac{T!}{{\prod }_i{t}_i!}\prod _i{\left(\frac{{\xi }_i}{{\sum }_h{\xi }_h}\right)}^{t_i}}}\times \underset{[T\mid \varphi ]}{\underbrace{\frac{e^{-\varphi }{\varphi }^T}{T!}}},$$

where ${\xi }_i=e^{z_i{\beta }_N}$ and $\varphi =J{e}^{{\alpha }_p}{\sum }_i{N}_i$. The intercepts ${\alpha }_p$ and ${\alpha }_N$ are confounded in $\varphi$. The partial likelihood $[\{t_i\}\mid T,{\beta }_N]$ gives MLEs of ${\beta }_N$ (the relative abundances).

With visit covariates $w_{ij}$ and coefficients ${\beta }_p$ the model ${\mu }_{ij}=e^{{\alpha }_p+{\alpha }_N+w_{ij}{\beta }_p+z_i{\beta }_N}$ factorises the same way (Eq. 10), with ${\xi }_i\propto e^{z_i{\beta }_N}{\sum }_j{e}^{w_{ij}{\beta }_p}$: detection covariates ${\beta }_p$ and relative abundance ${\beta }_N$ are both estimable; ${\alpha }_p$ and ${\alpha }_N$ remain confounded. It is a standard Poisson regression, so deviance goodness-of-fit and random-effect extensions are available (partial pooling for relative abundance rather than as an identification device).

Alder flycatcher example (Sec. 4.1)

Data: subset of Chandler et al. (2009), $I=50$ sites, $J=3$ visits, visit covariates time $t$ and date $d$, site covariates woody cover $w$ and plant vertical structure $s$ (all standardised).

Models fitted

• N-mixture (R unmarked): $n_{ij}\sim \text{Bin}(N_i,p_{ij})$, $N_i\sim \text{Pois}({\nu }_i)$, $\text{logit}(p_{ij})={\alpha }_p+{\beta }_{1}t+{\beta }_{2}d$, $\log {\nu }_i={\alpha }_{\nu }+{\gamma }_{1}w+{\gamma }_{2}s$.
• Poisson regression (glm): $n_{ij}\sim \text{Pois}(p_{ij}{\nu }_i)$, $\log (p_{ij})={\alpha }_p+{\beta }_{1}t+{\beta }_{2}d$, $\log {\nu }_i={\alpha }_{\nu }+{\gamma }_{1}w+{\gamma }_{2}s$, so $\log {\mu }_{ij}=\alpha +{\beta }_{1}t+{\beta }_{2}d+{\gamma }_{1}w+{\gamma }_{2}s$, $\alpha ={\alpha }_p+{\alpha }_{\nu }$.
• Random-effects Poisson (lme4::glmer): site intercept ${\alpha }_i\sim N({\mu }_{\alpha },{\sigma }_{\alpha }^2)$.
• Random-effects double Poisson: $n_{ij}\sim \text{DoublePoisson}({\mu }_{ij},{\theta }_{ij})$, $\log {\mu }_{ij}=m_{ij}+{\epsilon }_i$, with $\log \log {\theta }_{ij}=m_{ij}$ chosen to mimic the N-mixture’s Poisson-like behaviour at small $\mu$ and under-dispersion otherwise; fitted by integrate and optim.

Table 2 (reproduced)

Model	Log-likelihood	No. parameters	AIC
N-mixture	-131.11	6	274.21
Poisson regression	-139.57	5	289.14
Random-effects Poisson	-139.12	6	290.24
Random-effects double Poisson	-130.76	6	273.52

Interpretation. AIC differences appear large (double Poisson best, N-mixture 0.7 behind, Poisson models about 15-17 worse), yet Figure 3 shows the site-covariate effects ${\gamma }_1$ (woody cover) and ${\gamma }_2$ (plant structure) are similar in all four models: they are relative-abundance effects. The visit coefficients ${\beta }_1,{\beta }_2$ and the intercept differ superficially only because the N-mixture uses a logit link for $p$ and the Poisson a log link. Inference on ${\alpha }_p$ vs ${\alpha }_{\nu }$ is very different across models: completely confounded under Poisson, identified under the N-mixture only by the repeated-binomial assumption.

Discussion (Sec. 5)

• The appeal (inference without marking) comes with a “critical loss of information” (Sec. 2); alternatives like negative binomial or zero-inflated Poisson for $N_i$ have analogous non-identifiable counterparts (binomial/negative-binomial example in the supplement).
• “The problem is not an inability to fit the N-mixture model, but rather an inability of the data to discriminate among models that have vastly different implications for inference about $N$.”
• Only by assuming the alternatives do not hold and that $p$ is constant or fully covariate-explained can $N$ be inferred; with sparse data little residual information remains to check.
• Misconception: that alternatives cannot model abundance and detectability covariates simultaneously. Random-effect Poisson models can (Link and Sauer 1997), as the partition of Eq. 10 shows. Using the same covariate for abundance and detection in N-mixtures depends entirely on the binomial structure.
• Message: counts under imperfect detection can reliably serve only as indices. If absolute abundance is needed, spend effort on additional data such as recaptures of marked animals.

Examples

Relevance to marketing measurement

Observed conversions in a channel factor as exposure-opportunity times response rate. With only aggregate counts, the product is identified and its pieces are not, unless a separate source (a lift test, geo experiment, or user-level panel) anchors one factor. This mirrors the “relative abundance only” conclusion and the case for combining experiments with observational data. An analogy; not a claim of the paper.

# Relative-abundance Poisson regression (statsmodels), counts n[i,j], site covariate z[i]
import numpy as np, statsmodels.api as sm
rng = np.random.default_rng(3)
I, J = 50, 3
z = rng.normal(size=I); N = rng.poisson(np.exp(2+0.5*z)); n = rng.binomial(N[:, None], 0.3, (I, J))
X = sm.add_constant(np.repeat(z, J)); y = n.ravel()
print(sm.GLM(y, X, family=sm.families.Poisson()).fit().params)  # slope ~0.5 recovered; intercept = alpha_N + alpha_p

Connections

• Random-effect Poisson = partial pooling: Hierarchical Models, Partial Pooling as Multiple Comparisons Correction.
• Dispersion beyond Poisson: Monsters and Mixtures. GLM background: Generalized Linear Models.
• PyMC N-mixture sketch with marginalisation: Concept - Marginalising Latent Discrete Counts for HMC.

See Also

• Barker 2018 - Reliability of N-Mixture Models - Overview
• Concept - Closed-Population Capture-Recapture and Otis Models