Simpson 2016 - Convergence Theory

Summary

Two approximations are analysed: the quadrature in the likelihood (Thm 1: Hellinger error $O(p^{-k})$) and the finite-element field (Thm 2: posterior-expectation error almost $O(h^1)$). The classical lattice-counts approximation converges only at $O(p^{-1})$ (Cor. A1) while the off-grid likelihood on the same lattice would be $O(p^{-2})$ (Remark A2); with Karhunen-Loeve fields arbitrarily high order is possible (Cor. A2).

Overview

Posterior distributions are compared in Hellinger distance, following the Bayesian-inverse-problems framework of Stuart (2010), Cotter et al. (2010) and Dashti & Stuart (2011).

Main Content

Theorem 1 (quadrature error)

Assume $Z(\cdot )$ has almost surely $k$ square-integrable derivatives, and the $p$-point integration scheme in eq. 3 has deterministic error of order $p^{-k}$. Then the Hellinger distance between the posteriors from the true and approximate likelihoods is $O(p^{-k})$.

Theorem 2 (SPDE approximation)

Let $\Omega$ be a convex polygon in ${\mathbb{R}}^2$, $h$ the maximum mesh edge length, $Z$ solving eq. 5 with $\alpha =2$, $G$ uniformly Lipschitz. Then the error in the posterior expectation $\mathbb{E}\{G(Z)\mid y\}$ due to the SPDE (finite-element) approximation is $O(h^{1-ϵ})$ for any $ϵ>0$.

Appendix structure:

• A.1 Likelihood approximation (Thm A3): general bound for fixed Gaussian field on a Lipschitz domain: $|\Phi (Z)-{\Phi }^p(Z)|\le C\Vert Y\Vert \psi (p)\Vert Z{\Vert }_{H^{\gamma }}$, then Hellinger bound via Thm 2.4 of Cotter et al. (2010).
• Corollary A1 (lattice): for $\alpha \ge 2$ the $(p+1)\times (p+1)$ lattice approximation converges in Hellinger distance at rate $O(p^{-1})$. Proof idea: midpoint rule integral is $O(p^{-\gamma })$, $d/2<\gamma \le 2$, but approximating $Z(s_k)$ by the cell centroid value costs $|Z(s_k)-Z({\tilde{s}}_{ij})|\le C{p}^{-1}\Vert Z{\Vert }_{H^{1+d/2}}$ by Taylor and Sobolev embedding, dominating.
• Remark A2: with eq. 3 on the same lattice the rate is $O(p^{-2})$ for smooth fields.
• A.2-A.4: extension of Dashti & Stuart (2011) Thm 2.6 to finite-dimensional approximations (Thm A4) and proof of Thm 2 via finite-element results (Thm A5).
• A.5 Higher order: truncated Karhunen-Loeve expansion of $({\kappa }^2-\Delta {)}^{-\alpha /2}$ eigenfunctions with tensor Gaussian quadrature; Corollary A2: error $e_G\le C{N}^{-t}$ for all $t<\alpha -d/2$ (arbitrarily high order for smooth fields).

Caveat: results hold for fixed hyperparameters; effect of approximating $\kappa ,\tau$ is open.

Examples

Conceptual: if binning costs $O(p^{-1})$, halving the error needs doubling $p$ (4x cells in 2-D), versus 2x cells… for $O(p^{-2})$ halving error needs $\sqrt{2}p$ (2x cells). Numerics are not tabulated in the paper.

Connections

Why HSGP-style models (spectral truncation) are efficient: Cor. A2 is the Karhunen-Loeve/spectral analogue (Hilbert Space Gaussian Processes).

See Also

Simpson 2016 - Going Off Grid - Overview