Transfer Learning
Transfer learning is reusing a model trained on one problem — usually a large, general one — as the starting point for another, related problem, so that the knowledge in its weights (edges and shapes in an image model, grammar and facts in a language model) carries over and the new task needs far less data and compute than training from scratch. Fine-tuning a pre-trained model is its usual form; almost no team trains from nothing.
- Also known as
- Transfer, Pre-trained model reuse, Domain adaptation
- Used at
- image models fine-tuned from ImageNet-trained networks (the 2014–2020 default), language models fine-tuned from pre-trained transformers (BERT onward, and every foundation model since), speech, and tabular models pre-trained on one portfolio and adapted to another
- Binding
- none directly; but the pre-trained model's licence (open weights with conditions, or an API whose terms forbid certain uses) travels with the derived model, and the EU AI Act treats a substantially modified model as the modifier's
- Typical mechanics
- take the pre-trained model; keep its early layers (general features) and retrain the later ones on the new data, or fine-tune all layers at a low learning rate, or add a small task-specific head, or (LoRA) add low-rank adapters and train only those; the new dataset can be hundreds to thousands of examples where training from scratch would need millions
- Typical gains
- an image classifier reaching 95% with 2,000 labelled images by transfer where scratch training needs 100,000; a language classifier at 92% with 3,000 examples; a week of work instead of a quarter
- Typical failures
- negative transfer, when the source and target differ enough that the pre-trained features mislead (a model of consumer transactions transferred to corporate ones); catastrophic forgetting, when fine-tuning erases the general capability; and licence breach, when the source model's terms did not allow the use
- Parties
- the labs and communities that publish pre-trained models, the teams that adapt them, and the lawyers who read the licences
How transfer learning works in practice
A fintech needs a document classifier for 30 types of financial document. Training a vision model from scratch would need hundreds of thousands of labelled images. Instead it takes a network pre-trained on millions of general images, whose early layers already detect edges, textures and layouts, and fine-tunes it on 4,000 labelled documents; accuracy reaches 96% in an afternoon of GPU time. The general visual knowledge transferred; only the mapping from layouts to the company's 30 types was learned. The same pattern with language: a pre-trained transformer knows English, finance vocabulary and document structure; fine-tuning on 2,000 labelled contracts teaches it the company's clause taxonomy, and the result outperforms a model trained from scratch on ten times the data.
The judgement is in the distance between source and target. An ImageNet model transfers well to photographs of documents and less well to X-rays; a general language model transfers to financial text and stumbles on a private code system; a fraud model trained on one country's card transactions may transfer to a neighbour's or may carry patterns that do not hold. Validation on the target data decides. The other judgement is legal: open-weight models come with licences — some permissive, some restricting commercial use or requiring attribution or forbidding use to train competitors — and an API model's terms may forbid using its outputs to train another model, which makes distillation from it a breach.
For a fintech, transfer learning is the default route to any model and the reason a data-science project now starts with "which pre-trained model" rather than "how much data can we collect"; the licence check is part of the start.
Example
A remittance company wants to detect mule accounts. Its own labelled cases number 900 — too few to train from scratch. It takes a transaction-sequence model pre-trained by a vendor on anonymised patterns across many financial institutions and fine-tunes on its 900 cases plus 20,000 negatives; recall at a 2% alert rate rises from 31% (rules) to 64%. Validation by corridor shows the transfer holds for the company's main corridors and fails for one — a corridor the vendor's pre-training data barely covered — where recall is 22%; that corridor keeps the rules until the company has 200 of its own cases there. The vendor's licence permits fine-tuning for internal use and forbids redistribution; the fine-tuned weights stay in the company's own environment, and the licence is filed with the model card.
Common mistakes
- Training from scratch when a pre-trained model would reach the same accuracy with a fraction of the data. - Assuming transfer works without validating on the target's own segments. - Fine-tuning at a high learning rate and erasing the general capability. - Ignoring the source model's licence and terms. - Transferring a model across a distribution gap (country, product, era) without checking.