Instruction Tuning
Instruction tuning is the supervised fine-tuning of a pre-trained language model on a dataset of instructions paired with good responses — "summarise this contract" with a summary, "classify this ticket" with the class — so that the model, which pre-training taught only to continue text, learns to follow directions and adopt a format. It is the first stage of post-training, usually followed by preference training, and it separates a base model from a chat model.
- Also known as
- Supervised fine-tuning, SFT, Instruction fine-tuning, Post-training
- Used at
- every assistant-style model (the "-instruct" and "-chat" variants of open models, and every commercial chat model), and by companies fine-tuning a model on their own instruction–response pairs for a task, tone or domain
- Binding
- none directly; the EU AI Act's provider obligations cover the trained model, and the instruction data is part of the documentation a provider must keep
- Typical data
- tens of thousands to millions of instruction–response pairs — written by contractors, generated by a stronger model and filtered, or collected from real use — across tasks, formats and refusals; quality matters more than quantity (the 2023 LIMA paper reached strong results with 1,000 careful examples)
- Typical mechanics
- standard supervised fine-tuning — the model is trained to produce the response given the instruction, with the loss computed on the response tokens; a chat template marks turns and roles; a few epochs over the data at a low learning rate
- Typical result
- a base model prompted "What is an earn-out?" might continue with more questions in the same style; the instruction-tuned model answers. It also picks up the dataset's habits — verbosity, hedging, refusals, formatting — which is why the data is curated
- Parties
- the labs and their data contractors, the open-source community producing instruction datasets, and companies tuning models on their own pairs
How instruction tuning works in practice
A base model has read a trillion tokens and can complete anything in the style of its source; asked a question, it may produce a forum thread, a quiz, or the question again. Instruction tuning shows it 100,000 examples of the form it should produce: a question, then a direct answer; an instruction to extract fields, then the fields as JSON; a request it should refuse, then a refusal. After a few epochs the model has learned the pattern "respond to the instruction" as a general behaviour, applying it to instructions it never saw. Preference training afterwards refines which responses are better; but the capability to follow instructions at all comes from this stage.
For a fintech the relevant form is domain instruction tuning. Take an open instruct model; add 3,000 examples of the company's own tasks — its ticket categories, its document schemas, its tone, its refusals ("I can't change account details; here is how to request it") — and fine-tune. The model keeps its general ability and gains the company's specifics, at a cost of hours on one GPU. The care is in the data: each example is a specification the model will generalise from, so an example with a wrong answer teaches a wrong answer, an inconsistent format teaches inconsistency, and a missing refusal teaches compliance with what should be refused. Evaluation on held-out instructions, and a red-team set of things the model must not do, close the loop.
For AI M&A, instruction data is an asset a diligence reads: who wrote it, under what terms, and whether it was generated by another lab's model in breach of that model's terms of use.
Example
A payments company wants a support model that answers merchants in its style, follows its escalation policy and outputs a structured summary for the CRM. A generic instruct model gets the facts right 88% of the time but ignores the escalation policy in 30% of relevant cases and formats the summary inconsistently. The team writes 2,400 instruction–response pairs from real (anonymised) conversations, edited by the support leads to the policy and format, plus 300 refusals for actions the model must route to a human. Fine-tuning takes four hours on one GPU. On a held-out set of 400: factual accuracy 91%, escalation-policy adherence 97%, summary format valid 99.5%. The 2,400 pairs are versioned with the model; a policy change means editing the pairs and retraining.
Common mistakes
- Fine-tuning on unedited transcripts; the model learns the agents' mistakes along with their answers. - Thousands of mediocre examples where a few hundred careful ones would do better. - No refusals in the data, so the model complies with what it should route. - Training on data generated by a model whose terms forbid it. - Skipping the held-out evaluation and red-team set; the model's new habits are unknown until measured.