Why Autoregressive LLMs Are an Off-Ramp on the Road to AGI
Meta's Chief AI Scientist Yann LeCun explains why predicting the next token is an architectural dead end for true intelligence, and why AI needs world models that understand physical cause and effect.
In an incisive debate with Lex Fridman, Yann LeCun dismantled the widespread assumption that scaling autoregressive transformers will inevitably bridge the divide to human-level intelligence.
LeCun argued that autoregressive models inherently accumulate errors: every predicted token contains a non-zero probability of divergence, which compounds exponentially as generation length expands. Because text prediction operates without an internal simulator of physical reality, the model cannot verify whether its statements correspond to genuine physical mechanics.
Instead, LeCun advocates for Joint Embedding Predictive Architectures (JEPA), where systems learn representations of abstract state transitions rather than predicting pixel-level or word-level details. Without this architectural pivot, machines will remain brittle pattern matchers devoid of true common sense.
Where do you stand on Yann LeCun's perspective?
🔍Source Evidence & Verified Citations
More from this conversation
Can A Machine Truly Understand Common Sense Without A Physical Body?
Turing Award winner Yann LeCun explains why an eight-month-old human baby understands more about physical reality than the largest multi-billion parameter language model.
"A cat has a mental model of the world that allows it to pounce on a mouse. No current AI system has the common sense of a house cat."