Technological Singularity
The technological singularity is a hypothesised point at which artificial intelligence becomes able to improve itself, each generation designing a more capable successor faster than the last, so that progress accelerates beyond human ability to follow or control. The term is from Vernor Vinge (1993), the idea from I. J. Good's "intelligence explosion" (1965), the popular version from Ray Kurzweil; it is an argument about trend lines, not a dated forecast.
- Also known as
- The Singularity, Intelligence explosion, Recursive self-improvement
- Used at
- the philosophy and futurism around AI, the effective-altruism and AI-safety communities' risk arguments, the labs' long-term mission statements, and popular coverage of every model release; not in any engineering roadmap
- Binding
- nothing; the closest policy analogue is the labs' scaling policies and the regulators' compute thresholds, which exist to notice capability jumps before they are irreversible
- Typical argument
- if an AI can do AI research, it can improve itself; improvements compound; the process outruns oversight. Typical objections: intelligence is not one scalar that self-improves; hardware, data and physics limit the loop; the trend lines extrapolated are of compute and benchmarks, not of "intelligence"; and predictions of imminent singularities have a long record of being wrong
- Typical dates offered
- Kurzweil's 2045 (and, later, 2029 for AGI); the labs' leaders have offered various shorter horizons for AGI and none with confidence for anything beyond
- Parties
- futurists and philosophers, the AI-safety field, the labs, the sceptics, and investors for whom the story is part of the valuation
How the idea works in practice
The argument is simple and its inputs are not. Models have improved with compute and data along smooth curves for a decade; the labs now use models to write code, including the code that trains models; if the models become good enough at that work, the loop closes and the curves steepen — the "intelligence explosion". The sceptical reading is that the loop is already partly closed and the curves are still curves: models help engineers, engineers still decide, compute and data are still bought and built, and no self-improvement has been observed that outran the people running it. The safety reading is that the absence of evidence is not comfort, because the transition, if it comes, would be fast — which is why the labs write policies that gate the next training run on evaluations of the last.
For the AI economy the idea sets a frame that both inflates and disciplines. It inflates: a technology that might end history justifies any valuation and any compute purchase. It disciplines: labs that believe it fund safety research, red teams and governance structures that a purely commercial company would not. For the rest of the economy, including fintech, the singularity is a story about the vendors, not the product: the models a company buys are what the labs produced on the way to whatever they think they are building.
For a fintech's board, the useful translation is vendor risk: the companies whose models it depends on are run on a thesis that their next model may be discontinuously better, which means the product, the price and the terms can change faster than a normal supplier's.
Example
A fintech's strategy offsite includes a session on "the singularity and our business". The CTO's slide: the company uses models from two vendors; those vendors' stated missions involve transformative AI; the practical consequences for the fintech are that model capabilities and prices change quarterly, that the vendors' terms of service and safety policies constrain what the fintech's products may do, and that the fintech's competitive position depends on how well it applies models everyone can rent. The session concludes with three actions — a second vendor for every model-dependent feature, an evaluation suite run on every model change, and a data-terms review — and no forecast of the singularity, which the board agrees is the right amount of attention.
Common mistakes
- Building a strategy on a date for an event nobody can predict. - Dismissing the safety practices that the idea motivates because the idea sounds like fiction. - Reading exponential curves of compute as exponential curves of intelligence. - Boards spending the AI session on the singularity and not on vendor concentration. - Confusing the labs' missions with their products' capabilities today.