Self Supervised Learning

concepts · 4 notes linked

Related: Transformers · Yann LeCun · Reinforcement Learning · Yoshua Bengio · Energy Based Models · Facebook AI · Deep Learning · Google

Notes

• PEGASUS: A State-of-the-Art Model for Abstractive Text Summarization — Google model using gap-sentence pre-training for abstractive summarization
• Self-supervised learning is the future of AI — LeCun's blueprint for self-supervised learning over supervised AI
• Self-supervised learning: The plan to make deep learning data-efficient - TechTalks — Yann LeCun's AAAI 2020 roadmap for data-efficient self-supervised learning
• Test-Time Training (TTT): A New Approach to Sequence Modeling — Hidden state as a learnable model updated at inference