Will Transformers Take Over Artificial Intelligence? | Quanta Magazine

transformersdeep-learningcomputer-visionself-attentionai-architecture

Abstraction: Transformers expanding from NLP to vision, generative, and multimodal AI tasks

Key points:

"Attention Is All You Need" (2017) introduced transformers; self-attention connects every input element to every other simultaneously, unlike CNNs' local-to-global processing
Vision Transformer (ViT), by Dosovitskiy at Google Brain Berlin, tokenizes images into 16x16-pixel patches and achieved >90% accuracy on ImageNet, presented at conference May 2021
By early 2022, ViT variants dominated the ImageNet leaderboard; pure CNNs without transformers barely reached the top 10
Transformers process globally from the first layer; CNNs build up from local features — fundamentally different perception, confirmed by layer-by-layer analysis by Maithra Raghu at Google Brain
AlphaFold uses transformer components; applications also include emotion detection in speech, ECG stress analysis, and robot body-movement recognition
Higher pre-training compute cost than CNNs remains a drawback; future architectures will likely be hybrid (convolutions + self-attention)

Connections: Google · Openai · GPT-3 · Transformers · Self Attention · Computer Vision