Enhancing sampling in computational statistics using modified hamiltonians

Abstract

The Hamiltonian Monte Carlo (HMC) method has been recognized as a powerful sampling tool in computational statistics. In this thesis,we showthat performance ofHMCcan be dramatically improved by replacing Hamiltonians in theMetropolis test with modified Hamiltonians, and a complete momentum update with a partial momentum refreshment. The resulting generalized HMC importance sampler, whichwe called Mix & Match Hamiltonian Monte Carlo (MMHMC), arose as an extension of the Generalized Shadow Hybrid Monte Carlo (GSHMC) method, previously proposed for molecular simulation. The MMHMC method adapts GSHMC specifically to computational statistics and enriches it with new essential features: (i) the e icient algorithms for computation of modified Hamiltonians; (ii) the implicit momentum update procedure and (iii) the two-stage splitting integration schemes specially derived for the methods sampling with modified Hamiltonians. In addition, di erent optional strategies formomentumupdate and flipping are introduced as well as algorithms for adaptive tuning of parameters and e icient sampling of multimodal distributions are developed. MMHMChas been implemented in the in-house so ware package HaiCS (Hamiltonians in Computational Statistics) written in C, tested on the popular statistical models and compared in sampling e iciency with HMC, Generalized Hybrid Monte Carlo, Riemann Manifold Hamiltonian Monte Carlo, Metropolis Adjusted Langevin Algorithm and RandomWalk Metropolis-Hastings. The analysis of time-normalized e ective sample size reveals the superiority of MMHMC over popular sampling techniques, especially in solving high-dimensional problems.