For a few years now, a conviction has settled into the tech landscape: LLMs have solved almost everything. They write, they code, they reason, they translate. Each new release feels a little closer to the threshold of general intelligence. And yet, a recurring criticism keeps coming up — voiced in particular by Yann LeCun: what if large language models were really good at talking about the world without actually understanding it?
The question isn't trivial. It draws the contours of a fundamental debate about the trajectory of AI — and about the nature of the next breakthrough.
LLMs have won everything… on the surface
Let's be honest: the progress made by LLMs since 2020 is real and spectacular. GPT-4, Claude, Gemini — these models have reached performance levels on complex tasks that few people anticipated. They handle long contexts, reason in steps, generate working code.
But hidden under those impressive performances is a structural limit we tend to underestimate.
An LLM is fundamentally a statistical engine. It learns probabilities: which words follow which other words, in which context, with what frequency. It has no internal representation of the physical world. It can explain why a cup falls off a table, cite Newton's laws, describe the trajectory of an object in free fall. But it does not simulate that fall. It has no causal model of what's happening.
That distinction — describing vs. modeling — sits at the heart of today's debate about the limits of LLMs.
Yann LeCun's critique: understanding versus talking about the world
Yann LeCun, Chief AI Scientist at Meta, has been making this case for years. In his 2022 reference paper, A Path Towards Autonomous Machine Intelligence, he describes what current architectures are missing in his view: a true world model.
His thesis is that LLMs do not develop a causal representation of reality. They learn correlations in text — that is, in descriptions of the world — but not in the world itself. To reach a robust and autonomous form of intelligence, however, a system must be able to simulate the consequences of its actions before acting.
The difference is fundamental:
- An LLM that explains gravity does not understand gravity.
- A two-year-old dropping a toy understands gravity — they have modeled it through direct experience.
This isn't a question of model size or amount of data. It's a question of architecture and learning regime.
World models and JEPA: a different approach to intelligence
In response to that limit, an alternative is emerging: world models. The idea is simple in principle — instead of learning to predict the next token in a text sequence, a world model learns to predict the next state of the world.
Concretely: if you show a world model a car approaching an intersection, it doesn't generate a description of the scene. It simulates the next image: where the car will be in 0.5 seconds, how the lighting will change, what happens if a pedestrian crosses. That requires an understanding of causality, object permanence, and physical laws.
That is exactly what the JEPA architecture (Joint Embedding Predictive Architecture), developed at Meta and championed by LeCun, sets out to build. Instead of predicting raw pixels or tokens, JEPA learns abstract representations of scenes and predicts how those representations evolve.
The early results are striking. VL-JEPA — the vision-language version of this architecture — reaches 65.7% accuracy on the EgoSchema benchmark, which tests temporal reasoning on videos. GPT-4o, with roughly 1.76 trillion parameters, scores 53.3% on the same test. VL-JEPA achieves it with only 1.6 billion parameters — about 1,100 times fewer. And it is 2.85x faster at inference.
That's not a minor detail. It's a demonstration that architectural innovation can outperform brute scale on tasks that require genuine world understanding.
What it changes for autonomous systems
The distinction between LLMs and world models isn't just theoretical. It has direct implications for AI's most ambitious applications.
Robotics can't get by with models that just describe actions. A robot has to simulate what happens before acting — anticipate an object's trajectory, evaluate whether a grasp is stable, plan a motion in real space. LLMs do not have that kind of internal representation.
Autonomous vehicles need to predict the future state of their environment every fraction of a second, not produce a narrative description of it.
A few players are starting to make that vision concrete. DeepMind Genie 3 can generate interactive 3D worlds in real time at 24 frames per second. World Labs, Fei-Fei Li's startup, launched Marble, a first commercial implementation of world generation. These systems are not improved chatbots. They represent a fundamentally different class of architecture.
Scale vs. architecture: the real debate for the next few years
The dominant dynamic in AI since 2020 has followed a simple logic: more data, more compute, more parameters. And it has worked — remarkably well. GPT-3 then GPT-4 confirmed that this recipe produced real, spectacular gains.
But several signals suggest the curve is starting to flatten on certain types of tasks. And, more importantly, some capabilities seem out of reach for scale alone.
Physical causality, spatial reasoning, simulating consequences in the real world — these are structurally different problems from token prediction. Throwing more parameters at them does not solve the underlying architectural constraint.
That is where LeCun's thesis regains relevance: the next breakthrough may not come from a GPT-7 or a Claude 6. It may come from a paradigm shift — a kind of system that learns to model the world rather than talk about it.
That doesn't mean LLMs will disappear. They'll remain powerful tools for everything related to language manipulation, symbolic reasoning, content generation. But to reach a real step-change in autonomy and robustness — what some people call AGI — the current architecture may be hitting its structural limits.
What we collectively underestimate
There is an interesting asymmetry in how we talk about AI right now.
On one side, we may be overestimating what current LLMs represent. Their impressive performance on reasoning benchmarks can mask the absence of a real causal model. A system that predicts text well can give the impression of understanding — without truly understanding in the full sense.
On the other side, we may be underestimating what comes next. World models, JEPA and simulation-oriented architectures are still emerging. The results from VL-JEPA, Genie 3 or Marble don't make front-page news the way a new GPT does. And yet they may be sketching the trajectory of the next decade.
Real breakthroughs aren't always announced with a marketing fanfare. Sometimes they pile up quietly in research papers, in architectures that beat systems 1,000 times larger with a fraction of the resources.
The question raised by the original LinkedIn post still stands: are we still on the right path with giant LLMs? Or will we have to go through world models to clear a real hurdle?
My intuition — and LeCun's — is that the two will coexist for a long time. But the next qualitative leap probably won't look like a bigger LLM. It will look like a system capable of simulating before speaking.
References
- Yann LeCun, A Path Towards Autonomous Machine Intelligence (2022) — yann.lecun.com
- Meta AI, V-JEPA and VL-JEPA — ai.meta.com
- PulseMark, World Models vs Language Models 2026 — pulsemark.ai
- Medium/KAIRI AI, Beyond LLMs and World Models — medium.com
- Medium, World Models: The Next Leap Beyond LLMs — medium.com