Data Efficient Neural Scaling Law via Model Reusing
Peihao Wang, Rameswar Panda, et al.
ICML 2023
We represent the abstract Hamiltonian (Hybrid) Monte Carlo (HMC) algorithm as iterations of an operator on densities in a Hilbert space, and recognize two invariant properties of Hamiltonian motion sufficient for convergence. Under a mild coverage assumption, we present a proof of strong convergence of the algorithm to the target density. The proof relies on the self-adjointness of the operator, and we extend the result to the general case of the motions beyond Hamiltonian ones acting on a finite dimensional space, to the motions acting an abstract space equipped with a reference measure, as long as they satisfy the two sufficient properties. For standard Hamiltonian motion, the convergence is also geometric in the case when the target density satisfies a log-convexity condition.
Peihao Wang, Rameswar Panda, et al.
ICML 2023
Jannis Born, Matteo Manica
Nature Machine Intelligence
Conrad Albrecht, Jannik Schneider, et al.
CVPR 2025
Haohui Wang, Baoyu Jing, et al.
KDD 2024