LLMs rest on a surprisingly simple principle, predicting the next word, but scaling them up has produced unexpected capabilities. Here is how, step by step.
2.1What is an LLM?
A large language model (LLM) is a neural network trained to predict the next word (more precisely the next token) in a sequence of text. This seemingly simple task, repeated over trillions of words, gives rise to abilities in conversation, writing, translation, summarisation and programming.
2.2The landscape in June 2026
In mid-2026, four "frontier" models dominate the consumer and professional market:
- GPT-5.5 (OpenAI), released in April 2026;
- Claude Opus 4.8 (Anthropic), released on 28 May 2026, top of several reasoning and programming rankings;
- Gemini 3.1 Pro (Google), particularly strong in reasoning and data analysis;
- Grok 4.3 (xAI).
Alongside these proprietary models, an open-weight ecosystem (whose weights can be freely downloaded) is gaining ground: Llama (Meta), Mistral (France), DeepSeek and Qwen (China). These open models give full control over deployment and privacy, often at a much lower cost.
2.3How it works, step by step
1. Tokenisation. The text is split into tokens (fragments of words). Each token is converted into a vector of numbers called an embedding, which encodes its meaning in a mathematical space.
2. The Transformer architecture and attention. The attention mechanism lets the model weigh the importance of each word relative to the others in the context. This is what allows it to "understand" that in "the bank of the river", the word bank does not refer to a financial institution.
3. Pre-training. The model learns by predicting the next token over a gigantic corpus (web, books, code). Through billions of adjustments, it encodes in its parameters (often hundreds of billions) a statistical representation of language and of the world.
4. Alignment (fine-tuning and RLHF). A raw model is powerful but hard to handle. It is then refined so that it follows instructions and remains helpful, honest and harmless. The key technique is RLHF (reinforcement learning from human feedback): humans rank the answers, and the model learns to favour the ones judged better.
5. Inference and reasoning. When you ask a question, the model generates its answer token by token. Recent reasoning models spend more computation "thinking" before answering (chain-of-thought, test-time compute), which clearly improves complex tasks.
Two notions to know
The context window: the amount of text the model can "keep in mind", now often more than a million tokens. Hallucinations: the model can produce false statements with confidence, because it optimises for statistical plausibility, not truth.
2.4World models: another path?
Not all researchers believe that LLMs will lead to human-level intelligence. The best-known sceptic is Yann LeCun, Turing Award winner and former chief AI scientist at Meta. In his view, predicting the next word is not enough: real intelligence must understand how the physical world works, plan and reason.
LeCun advocates an alternative architecture, JEPA (Joint Embedding Predictive Architecture): rather than predicting every pixel or every word, the model learns abstract representations of reality, ignoring irrelevant noise. In November 2025, he left Meta to found AMI Labs (Advanced Machine Intelligence) in Paris, which raised about $1 billion in March 2026, the largest seed round in European history.
A related approach is led by Fei-Fei Li (the pioneer behind ImageNet) with her start-up World Labs, around "spatial intelligence": enabling machines to perceive, reason and act in three-dimensional environments.
Key takeaway
LLMs rest on a simple principle, predicting the next token, but scaling them up has produced unexpected capabilities. The big debate of the moment is about their limits: are they a step towards general intelligence, or a dead end that will need to be complemented by other approaches such as world models?
Sources
Best AI Models, June 2026 (felloai.com); WIRED & TechCrunch on LeCun's departure (2025-2026); AMI Labs raises $1.03B (buildfastwithai.com); vendor documentation (OpenAI, Anthropic, Google, Mistral).
Finished this module?
Your progress is saved on this device.