A Statistical Theory of Contrastive Learning via Approximate Sufficient Statistics

Song Mei (University of California, Berkeley) · Licong Lin (University of California, Berkeley)

$f$-divergencesapproximate sufficient statisticsclassification contrastive learning contrastive losses data augmentation downstream tasksencoderserror propagation kl-divergencelanguage-image pretrainingmodel adaptation regression simclrsufficiencytheoretical framework

Contrastive learning