Uncovering a Universal Abstract Algorithm for Modular Addition in Neural Networks
abstract algorithmapproximate chinese remainder theoremapproximate cosetsdeep neural networksgeneralizable interpretabilitygroup multiplicationhidden layermodular additionmulti-level analysesmultilayer perceptronsneural network solutionsneuron-level representationstrainable embeddingstransformersuniversality hypothesis
We propose a testable universality hypothesis, asserting that seemingly disparate neural network solutions observed in the simple task of modular addition actually reflect a common abstract algorithm. While prior work interpreted variations in neuron-level representations as evidence for distinct algorithms, we demonstrate