Gosia Lazuka, Andreea Simona Anghel, et al.
SC 2024
Existing rigorous convergence guarantees for the Hamiltonian Monte Carlo (HMC) algorithm use Gaussian auxiliary momentum variables, which are crucially symmetrically distributed. We present a novel convergence analysis for HMC utilizing new analytic and probabilistic arguments. The convergence is rigorously established under significantly weaker conditions, which among others allow for general auxiliary distributions. In our framework, we show that plain HMC with asymmetrical momentum distributions breaks a key self-adjointness requirement. We propose a modified version of HMC, that we call the Alternating Direction HMC (AD-HMC), which overcomes this difficulty. Sufficient conditions are established under which AD-HMC exhibits geometric convergence in Wasserstein distance. The geometric convergence analysis is extended to when the Hamiltonian motion is approximated by the leaprog symplectic integrator, where an additional Metropolis-Hastings rejection step is required. Numerical experiments suggest that AD-HMC can generalize a popular dynamic auxiliary scheme to show improved performance over HMC with Gaussian auxiliaries.
Gosia Lazuka, Andreea Simona Anghel, et al.
SC 2024
Paulo Rodrigo Cavalin, Pedro Henrique Leite Da Silva Pires Domingues, et al.
ACL 2023
Weixin Liang, Girmaw Abebe Tadesse, et al.
Nature Machine Intelligence
Hsi-ai Tsao, Lei Hsiung, et al.
ICLR 2024