Compound Auxiliary Metropolis: Incorporating Auxiliary Variables into Multi-Candidate MCMC

Multiple-try Metropolis (MTM) is a Markov chain Monte Carlo (MCMC) algorithm that improves local transition efficiency by evaluating multiple candidate draws at each iteration. However, for complicated target distributions exhibiting severely non-Gaussian topography or multiple well-separated modes, locally optimal transitions may be insufficient for effective global exploration. In this work, we propose compound auxiliary Metropolis (CAM), a general multi-candidate MCMC method that incorporates both the local state of the chain and auxiliary information into the multi-candidate framework of MTM. Using an auxiliary generating distribution, CAM accommodates a flexible definition of auxiliary information. As examples, we consider three different auxiliary variables: one that promotes state-independent exploration and two that use a reference distribution to improve mixing. These auxiliaries are tested against distributions that present challenging targets for modern MCMC methods. In particular, we focus on the challenges presented by multiple well-separated modes and topography that requires long mixing for local MCMC moves. We find that CAM is able to sample effectively from these distributions, using MTM as a baseline to evaluate the benefit introduced by the auxiliary information. CAM also compares favourably with the No-U-Turn Sampler, showing similar performance for milder test distributions and better performance for the most difficult settings.