S$^2$M-Former: Spiking Symmetric Mixing Branchformer for Brain Auditory Attention Detection

Siqi Cai (Harbin Institute of Technology) · Jiaqi Wang (Beijing University of Posts and Telecommunications) · Zhengyu Ma (Peng Cheng Lab) · Xiongri Shen (Harbin Institute of Technology) · Chenlin Zhou (Peking University) · Leilei Zhao (Harbin Institute of Technology) · Han Zhang (Shanghai Jiao Tong University) · Yi Zhong (Harbin Institute of Technology) · Zhenxi Song (Harbin Institute of Technology, Shenzhen) · Zhiguo Zhang (Harbin Institute of Technology)
aad benchmarksartificial neural networksauditory attention detectionbiologically plausible token-channel mixersdecoding accuracyelectroencephalographyenergy reductionlightweight 1d token sequenceslow-power solutionsneuro-steered hearing devicesparameter efficiencyspatial and frequency branchesspike-driven architecturespiking neural networksspiking symmetric mixingsynergistic frameworks

Auditory attention detection (AAD) aims to decode listeners' focus in complex auditory environments from electroencephalography (EEG) recordings, which is crucial for developing neuro-steered hearing devices. Despite recent advancements, EEG-based AAD remains hindered by the absence of synergistic frameworks that can fully leverage complementary EEG features under energy-efficiency constraints. We propose ***S$^2$M-Former***, a novel ***s***piking ***s***ymmetric ***m***ixing framework to address this limitation through two key innovations: i) Presenting a spike-driven symmetric architecture composed of parallel spatial and frequency branches with mirrored modular design, leveraging biologically plausible token-channel mixers to enhance complementary learning across branches; ii) Introducing lightweight 1D token sequences to replace conventional 3D operations, reducing parameters by 14.7$\times$. The brain-inspired spiking architecture further reduces power consumption, achieving a 5.8$\times$ energy reduction compared to recent ANN methods, while also surpassing existing SNN baselines in terms of parameter efficiency and performance. Comprehensive experiments on three AAD benchmarks (KUL, DTU and AV-GC-AAD) across three settings (within-trial, cross-trial and cross-subject) demonstrate that S$^2$M-Former achieves comparable state-of-the-art (SOTA) decoding accuracy, making it a promising low-power, high-performance solution for AAD tasks. Code is available at https://github.com/JackieWang9811/S2M-Former.