PirateNets: Physics-informed Deep Learning with Residual Adaptive Networks

Abstraction: Adaptive residual architecture fixing deep PINN training instability

Key points:

• Standard PINNs (physics-informed neural networks) degrade in performance with deeper MLP architectures due to unsuitable initialization causing poor trainability of network derivatives.
• PirateNets introduce adaptive residual connections that initialize as shallow networks and progressively deepen during training, avoiding unstable PDE residual loss minimization.
• The initialization scheme encodes inductive biases specific to a given PDE system directly into the architecture.
• PirateNets achieve state-of-the-art results across multiple PDE benchmarks and gain accuracy from increased depth, unlike standard MLPs.
• Code released at github.com/PredictiveIntelligenceLab/jaxpi; arXiv:2402.00326.

Connections: Physics Informed Neural Networks · Residual Networks · Partial Differential Equations · Deep Learning

Source: https://arxiv.org/abs/2402.00326v1