Emergent Behaviour
Emergent behaviour is a capability a model displays without having been explicitly trained for it, appearing — or appearing to appear — only above a certain scale: arithmetic, translation, code, multi-step reasoning showing up in large language models where smaller ones trained the same way showed none. It matters because a capability nobody designed is one nobody evaluated; and it is contested, because some apparent emergence is an artefact of measurement.
- Also known as
- Emergence, Emergent abilities, Emergent capabilities
- Used at
- the research literature on scaling (the 2022 "Emergent Abilities of Large Language Models" paper catalogued dozens), the labs' evaluation and safety programmes, and the arguments for treating frontier models as different in kind
- Binding
- no rule — but the EU AI Act's approach to general-purpose models with systemic risk, and the labs' scaling policies, exist partly because capabilities can appear that were not anticipated
- Typical examples claimed
- few-shot arithmetic, chain-of-thought reasoning, following instructions in a language the model saw little of, tool use — each absent below a parameter count and present above it
- Typical counter-argument
- a 2023 analysis showed that many "emergent" jumps disappear when the metric is continuous rather than pass/fail — the ability was improving smoothly and the threshold was in the scorer; genuine discontinuities are fewer than the early claims
- Parties
- the labs' researchers, the safety community that uses emergence as a reason for caution, the sceptics who see measurement artefacts, and the companies deploying models whose next version may do something new
How emergent behaviour works in practice
A lab trains a family of models at increasing scale on the same data and objective — predicting the next token. Below some size the models cannot add three-digit numbers; above it, prompted with a few examples, they can. Nothing in the training said "learn arithmetic"; the ability was a by-product of predicting text that contains arithmetic, and it became usable at a scale where the model's internal representations were rich enough. The same pattern was reported for translation, code generation, reasoning chains, and — most relevant to safety — abilities like persuasion and deception that no one would train for on purpose. If capabilities arrive unannounced, the argument runs, a model must be evaluated for what it can do, not for what it was built to do, before it is deployed.
The counter-argument sharpened the claim rather than dismissing it. Measured on a pass/fail exact-match metric, a model that gets 40% of the digits right scores zero until it gets all of them, and the "emergence" is the scorer's threshold; measured continuously, the ability grows smoothly with scale. Some abilities survive the re-measurement and some do not. The practical conclusion is the same either way: evaluate models directly, broadly and repeatedly, because whatever the curve's shape, the next model may cross a line the last one did not.
For a company using models, emergence is the reason a vendor's model upgrade is a change to test and not a free improvement: the new version may do things the old one could not — some of them wanted, some of them not.
Example
A fintech's support bot runs on a vendor model. The vendor releases a larger version; the fintech's engineers swap it in after checking that the old evaluation suite passes. Within a week the bot begins, unprompted, to offer customers detailed comparisons of the fintech's fees with competitors' — accurate, and not something the product team had authorised or the old model had been capable of composing. Compliance halts the feature. The post-mortem finds the new model's stronger reasoning and broader knowledge produced a behaviour the prompt had never forbidden because the old model had never done it. The fintech's evaluation suite gains a section for "things the bot should not volunteer", run on every model change, and its vendor agreement gains a notice period for upgrades.
Common mistakes
- Treating a model upgrade as a drop-in improvement without re-evaluating what the new model will do. - Reading emergence claims without asking how the capability was measured. - Guardrails that forbid what the last model did rather than what the next one might. - Dismissing the safety argument because some emergence was an artefact; the evaluation conclusion holds regardless. - Product teams relying on a capability the vendor's model happens to have this quarter.