Designing dynamic measure transport for sampling and quantization

Sampling or otherwise summarizing complex probability distributions is a central task in applied mathematics, statistics, and machine learning. Many modern algorithms for this task introduce dynamics in the space of probability measures and design these dynamics to achieve good computational performance. We will discuss several aspects of this broad design endeavor. First is the problem of optimal scheduling of dynamic transport, i.e., with what speed should one proceed along a prescribed path of probability measures? Though many popular methods seek “straight line” trajectories, i.e., trajectories with zero acceleration in a Lagrangian frame, we show how a specific class of “curved” trajectories can improve approximation and learning. We then present extensions of this idea which seek not only schedules but paths that improve spatial regularity of the underlying velocity. Second, we discuss the problem of weighted quantization, i.e., summarizing a complex distribution with a small set of weighted samples. We study this problem from the perspective of minimizing maximum mean discrepancy via gradient flow in the Wasserstein–Fisher–Rao geometry. This perspective motivates a new fixed-point algorithm, called mean shift interacting particles (MSIP), which outperforms state-of-the-art methods for quantization. We describe how MSIP can be used not only to quantize an empirical distribution, but to sample given an unnormalized density.