Revitalizing SVD for Global Covariance Pooling: Halley’s Method to Overcome Over-Flattening

Jiawei Gu (Great Bay University) · Xinming Li (Beijing University of Civil Engineering and Architecture) · Ziyue Qiao (Great Bay University) · Zechao Li (Nanjing University of Science and Techonolgy)

cnnsexcessive compressionglobal covariance poolinggradient explosionshalley's iterationhalley-svdhigh-eigenvalue structurehigh-order iterative methodisqrt-covnewton--schulzover-flattening phenomenonsecond-order statisticsspectral precisionsvd methodstransformer architectures

Global Covariance Pooling (GCP) has garnered increasing attention in visual recognition tasks, where second-order statistics frequently yield stronger representations than first-order approaches. However, two main streams of GCP