Return of ChebNet: Understanding and Improving an Overlooked GNN on Long Range Tasks

Alvaro Arroyo (University of Oxford) · Alessio Gravina (University of Pisa) · Xiaowen Dong (Oxford) · Pierre Vandergheynst (EPFL) · Carola-Bibiane Schönlieb (Cambridge University) · Davide Bacciu (University of Pisa) · Ali Hariri (EPFL) · Moshe Eliasof (University of Cambridge) · Kamyar Azizzadenesheli (Purdue University)

chebnetcomputational efficiencycontrollable dynamicseigendecompositionsgraph rewiring graph transformershigh-order polynomialsinformation propagation long-range dependencies message passing neural networks positional encodingsspatial message passingspectral gnnsstable dynamical systemstate-of-the-art performanceunstable regime

ChebNet, one of the earliest spectral GNNs, has largely been overshadowed by Message Passing Neural Networks (MPNNs), which gained popularity for their simplicity and effectiveness in capturing local graph structure. Despite their success, MPNNs are limited in their ability to capture long-range dependencies between nodes. This has led researchers to adapt MPNNs through *rewiring* or make use of *Graph Transformers*, which compromise the computational efficiency that characterized early spatial message passing architectures, and typically disregard the graph structure. Almost a decade after its original introduction, we revisit ChebNet to shed light on its ability to model distant node interactions. We find that out-of-box, ChebNet already shows competitive advantages relative to classical MPNNs and GTs on long-range benchmarks, while maintaining good scalability properties for high-order polynomials. However, we uncover that this polynomial expansion leads ChebNet to an unstable regime during training. To address this limitation, we cast ChebNet as a stable and non-dissipative dynamical system, which we coin Stable-ChebNet. Our Stable-ChebNet model allows for stable information propagation, and has controllable dynamics which do not require the use of eigendecompositions, positional encodings, or graph rewiring. Across several benchmarks, Stable-ChebNet achieves near state-of-the-art performance.