Hybrid Method Accelerates MIONet Training

Training MIONets, a type of neural network, can be computationally intensive. This research introduces a novel hybrid optimization approach that combines least squares (LS) and gradient descent (GD) methods to speed up this process. The method extends a similar technique previously applied to DeepONets. The core idea is to view the MIONet as a multilinear function with respect to the final layer parameters of its individual branch networks. This perspective enables the use of an alternating least squares method, where parameters for each branch network are optimized sequentially. To manage the large system matrices that arise in this process, the authors leverage Kronecker and Khatri-Rao products along with tensor permutation matrices. This factorization breaks down large matrices into smaller, more manageable ones, making the optimization more efficient. The proposed method is also flexible enough to accommodate various L2 loss functions with regularization.