ChatGPT is 'not particularly innovative,' and 'nothing revolutionary', says Meta's chief AI scientist

Abstraction: Yann LeCun argues ChatGPT is solid engineering not scientific breakthrough

Key points:

• Yann LeCun (Meta chief AI scientist, 2019 Turing Award) called ChatGPT "not particularly innovative" and "nothing revolutionary" — well-engineered but not a scientific advance
• ChatGPT builds on Transformer architecture (a Google invention from 2017), self-supervised pretraining, and RLHF (pioneered at DeepMind), none of which originated at OpenAI
• The first large neural net language model was published by Yoshua Bengio roughly 20 years before ChatGPT; the concept of attention also traces back to Bengio's group
• LeCun compared ChatGPT's achievement to IBM Watson at Jeopardy! and Sebastian Thrun's DARPA Grand Challenge win — excellent engineering, not new science
• Google and Meta have similar technology but hesitate to deploy because they "have a lot to lose by putting out systems that make stuff up"
• LeCun predicted Meta's FAIR lab would release generative systems including generative art tools for the 12 million small businesses advertising on Facebook

Connections: Yann LeCun · Meta · Chatgpt · Openai · Large Language Models · Transformers · Reinforcement Learning

Source: https://www.zdnet.com/article/chatgpt-is-not-particularly-innovative-and-nothing-revolutionary-says-metas-chief-ai-scientist/