Study: Deep neural networks don't see the world the way we do

Abstraction: MIT study reveals deep neural networks build idiosyncratic invariances unlike human perception

Key points:

• MIT neuroscientists (lead: Jenelle Feather; senior: Josh McDermott) published in Nature Neuroscience 2023 showing DNNs develop "idiosyncratic invariances" not shared with human sensory systems
• Using "model metamers" — stimuli generated to produce the same model response as a natural input — researchers found most DNN-generated images and sounds are unrecognizable to humans (random-looking pixels, unintelligible noise)
• Metamers from one model were unrecognizable to other models as well, confirming the invariances are model-specific rather than general perceptual properties
• This distinguishes from the classical view that sensory systems learn to disregard irrelevant variation (lighting, angle) while preserving core identity features
• Adversarial training (training on slightly perturbed examples) improved metamer recognizability by humans, independently of its known benefit against adversarial attacks
• Model metamers proposed as a standard benchmark test for evaluating how closely computational models match biological sensory organization

Connections: Mit · Deep Learning · Adversarial Examples · Computer Vision

Source: https://news.mit.edu/2023/study-deep-neural-networks-perception-1016