PINN Balls: Scaling Second-Order Methods for PINNs with Domain Decomposition and Adaptive Sampling

Andrea Bonfanti (Green 14 AB) · Ismael Medina (Georg-August Universität Göttingen) · Roman List (Technische Hochschule OWL) · Björn Staeves (Technische Universität München) · Roberto Santana (University of the Basque Country) · Marco Ellero (Swansea University)

adversarial adaptive samplingdomain decompositionensemble modelsmixture of experts model sizeparameter-efficiencypartial differential equations physics-informed neural networksscalability propertiesscientific machine learning second-order methods sparse codingtheoretical backgroundtraining enhancement

Recent advances in Scientific Machine Learning have shown that second-order methods can enhance the training of Physics-Informed Neural Networks (PINNs), making them a suitable alternative to traditional numerical methods for Partial Differential Equations (PDEs). However, second-order methods induce large memory requirements, making them scale poorly with the model size. In this paper, we define a local Mixture of Experts (MoE) combining the parameter-efficiency of ensemble models and sparse coding to enable the use of second-order training. Our model -- PINN Balls -- also features a fully learnable domain decomposition structure, achieved through the use of Adversarial Adaptive Sampling (AAS), which adapts the DD to the PDE and its domain. PINN Balls achieves better accuracy than the state-of-the-art in scientific machine learning, while maintaining invaluable scalability properties and drawing from a sound theoretical background.