Talk: Machine learning for scientific computing, Felix Ye, 2/3

Machine Learning for Scientific Computing 

Felix Ye, SUNY Albany

12-1pm EST Monday, Feb. 3, 2025
409 Sondheim Hall, UMBC

 In the first part, I will introduce a nonlinear stochastic model reduction technique for high-dimensional stochastic dynamical systems that have a low-dimensional invariant effective manifold with slow dynamics and high-dimensional, large fast modes. Given only access to a black-box simulator from which short bursts of simulation can be obtained, we design an algorithm that outputs an estimate of the invariant manifold, a process of the effective stochastic dynamics on it, which has averaged out the fast modes, and a simulator thereof. This simulator is efficient in that it exploits of the low dimension of the invariant manifold, and takes time-steps of size dependent on the regularity of the effective process, and therefore typically much larger than that of the original simulator, which had to resolve the fast modes. The algorithm and the estimation can be performed on the fly, leading to efficient exploration of the effective state space, without losing consistency with the underlying dynamics. 

The second part focuses on optimal transport (OT), a powerful framework for comparing probability distributions. Applications such as shuffled regression can be approached by optimizing regularized optimal transport (OT) distances, such as the entropic OT and Sinkhorn distances. A common approach for this optimization is to use a first-order optimizer, which requires the gradient of the OT distance. For faster convergence, one might also resort to a second-order optimizer, which additionally requires the Hessian. The computations of these derivatives are crucial for efficient and accurate optimization. However, they present significant challenges in terms of memory consumption and numerical instability, especially for large datasets and small regularization strengths. We circumvent these issues by analytically computing the gradients for OT distances and the Hessian for the entropic OT distance, which was not previously used due to intricate tensor-wise calculations and the complex dependency on parameters within the bi-level loss function. Through analytical derivation and spectral analysis, we identify and resolve the numerical instability caused by the singularity and ill-posedness of a key linear system. Consequently, we achieve scalable and stable computation of the Hessian, enabling the implementation of the stochastic gradient descent (SGD)-Newton methods. 

Felix Ye is a Assistant Professor in the Department of Mathematics and Statistics at SUNY Albany. His research interest is the intersection of machine learning and dynamical systems and is directed toward data-driven model reduction methods in the context of stochastic dynamical systems. He was a Postdoctoral Fellow at Johns Hopkins University and received a PhD in applied math from the University of Washington, advised by Hong Qian.