Chain of Thought
Chain of thought is the technique of having a language model set out intermediate reasoning steps before its final answer — "let's think step by step" — which measurably improves accuracy on arithmetic, logic and multi-step problems the model gets wrong when asked directly. Described in a 2022 Google paper, it became standard prompting practice and then, in the reasoning models of 2024–2025, a built-in behaviour. The trace reads like an explanation but is not always the model's real computation.
- Also known as
- Chain-of-thought prompting, CoT, Reasoning trace, Step-by-step reasoning
- Used at
- any task with several steps — calculations, contract analysis, classification with rules, code, planning; the reasoning models (OpenAI's o-series, Claude's extended thinking, DeepSeek-R1, Gemini's thinking modes) apply it by default and charge for the tokens
- Binding
- no rule; the EU AI Act's explainability obligations for high-risk systems are not satisfied by a chain of thought, which is a generated text, not a faithful account of the model's computation
- Typical gains
- on the 2022 benchmarks, accuracy on grade-school maths problems rose from 18% to 57% for the largest model with chain-of-thought prompting; reasoning models trained to produce long chains extended the gains to competition mathematics and code
- Typical costs
- more output tokens — a chain of thought can be five to fifty times the length of the answer — so more latency and cost per call; a reasoning model's "thinking" is billed as output
- Parties
- the labs that trained reasoning into models, the engineers who prompt for it, and the compliance teams who must not mistake a trace for an audit trail
How chain of thought works in practice
Ask a model "a merchant processes €4.2 million a month at a 1.9% blended rate with 0.8% chargebacks; what does it pay in fees and how many chargebacks is that at a €60 average ticket?" and a 2022 model would produce a plausible-looking wrong number. Ask it to work step by step and it computes: fees €79,800; transactions 70,000; chargebacks 560 — because each step is a small problem it can do and the intermediate results are in the context for the next. The mechanism is that language models generate one token at a time, conditioned on everything before; writing the steps gives the model somewhere to put the intermediate state that a direct answer has to compute in one pass.
The 2024–2025 reasoning models were trained, with reinforcement learning, to produce long chains on their own, checking and revising along the way, and they solve problems the prompted models could not. The practical questions became when to pay for it — a reasoning model is slower and dearer, and a classification task does not need it — and what the trace is worth. It helps engineers debug: a wrong answer with a visible chain shows where the reasoning went astray. It does not certify the answer: research has shown models producing chains that do not reflect the factors that actually drove the output, and a fluent chain to a wrong conclusion is more persuasive than a bare wrong answer.
For a fintech, chain of thought is a tool for accuracy and a risk for governance: use it to get harder tasks right, verify the outputs independently, and never file the trace as the reason a decision was made.
Example
A compliance team uses a model to pre-screen sanctions-screening alerts: for each hit, is it a true match or a false positive? Direct prompting gives 81% agreement with analysts. With chain of thought — the model is asked to compare name, date of birth, nationality and the list entry's aliases step by step before concluding — agreement rises to 93%, and the analysts read the chain to check the model's grounds in a third of the time a full review took. The model-risk committee's rule: the chain is a working note for the analyst, the analyst's own conclusion is the record, and every alert the model dismisses is sampled at 10% for review. When a sampled dismissal shows a chain that ignored a matching date of birth, the prompt is revised and the model re-evaluated on the last 5,000 alerts.
Common mistakes
- Filing a chain of thought as the explanation of a regulated decision. - Paying reasoning-model prices for tasks a small model classifies well. - Trusting a fluent chain more than a bare answer; both need verification. - Reading "step by step" as a guarantee; a chain can be wrong at step one. - Prompting for reasoning and truncating the output before the answer.