SCALLOP Speeds Up Generative Model Likelihood Estimation

Flow-based generative models have made strides in producing high-quality samples with minimal computational effort. However, accurately estimating the likelihood of these models, which is crucial for many applications, has remained a challenge. Existing methods either rely on restrictive architectures for exact calculations or use stochastic approximations that introduce significant variance. This new research introduces SCALLOP (SCAlable LikeLihood distillation of flOw maPs), a novel approach designed to overcome these limitations. Building upon the F2D2 model, SCALLOP offers a more scalable likelihood distillation objective that avoids the high variance associated with methods like Hutchinson's trace estimator. Its vectorized formulation contributes to greater efficiency. Empirical evaluations demonstrate SCALLOP's effectiveness as a Boltzmann generator in molecular science and its benefits for image datasets. The method not only reduces training variance and time but also consistently improves performance compared to its predecessor, F2D2. Furthermore, SCALLOP achieves competitive results with state-of-the-art models while offering up to a tenfold increase in inference speed over the fastest existing baselines.