DC Programming in Wasserstein Space Optimizes MMD and Energy Distance.

Optimizing functions over probability measures is a common task in machine learning, often performed within the Wasserstein space. However, many objective functions of practical interest, such as Maximum Mean Discrepancy (MMD) and Energy Distance (ED), are non-convex when considered along Wasserstein geodesics. This non-convexity makes it difficult to analyze and guarantee the performance of standard first-order optimization methods. This paper addresses this challenge by studying a class of objectives in the Wasserstein space that can be broken down into a "difference-of-convex" (DC) decomposition. It then adapts the well-known convex-concave procedure (CCCP) to this specific setting. Under certain smoothness and strong convexity assumptions on the convex components, the authors prove that their resulting algorithm achieves almost stationarity along its iterations. The primary focus is on MMD and ED functionals, for which the researchers develop explicit Wasserstein DC decompositions. They demonstrate that this scheme achieves local convergence under mild assumptions. Empirical results confirm that using these carefully chosen DC decompositions leads to faster and more stable convergence compared to traditional Wasserstein gradient descent when optimizing MMD objectives. This provides a more robust and efficient way to handle these important statistical distances in machine learning applications.