Forward Propagation
Forward propagation is the computation that passes an input through a neural network's layers in order — each layer multiplying by its weights, adding biases and applying an activation function — to produce the output; it is the operation performed on every prediction and every generated token, and, during training, the first half of each step before backpropagation adjusts the weights. Its cost, multiplied by the number of requests, is the inference bill a fintech pays on every transaction.
- Also known as
- Forward pass, Feedforward, Inference pass
- Used at
- every use of a trained model — a fraud score, a classification, a token from a language model — and every training step, where the forward pass computes the prediction whose error backpropagation corrects
- Binding
- none
- Typical mechanics
- the input vector is multiplied by the first layer's weight matrix, a bias is added, a non-linear activation (ReLU, GELU) is applied, and the result feeds the next layer; in a transformer the layers include attention and feedforward blocks; the final layer produces scores or a probability distribution over outputs; nothing is learned during a forward pass
- Typical cost
- roughly two floating-point operations per parameter per token for a language model — a 70-billion-parameter model spends about 140 billion operations per generated token — which is why model size, quantisation and batching decide inference cost
- Parties
- the engineers who serve models, the hardware makers whose chips are built for matrix multiplication, and the finance teams who see the result as cloud spend
How forward propagation works in practice
A fraud model receives 200 features for a transaction. The first layer, 200 inputs to 512 units, computes 512 weighted sums and applies ReLU; the second, 512 to 256; the third, 256 to 64; the output layer produces one number, passed through a sigmoid into a probability of fraud. The whole pass is a few hundred thousand multiplications — microseconds on a CPU — and it happens for every transaction, a billion times a year. A language model's forward pass is the same idea at another scale: an input of thousands of tokens flows through dozens of transformer layers, each with attention and feedforward sub-layers, to produce the distribution over the next token; that token is appended and the pass runs again, which is why generation cost scales with output length.
The engineering of inference is the engineering of the forward pass. Batching runs many inputs through the weights together so the hardware's matrix units stay busy; quantisation stores weights in 8 or 4 bits so the pass moves less memory; KV caching in transformers avoids recomputing attention over the prompt for every new token; distillation trains a smaller model whose forward pass is cheaper. For a fintech scoring every transaction, the choices are concrete: a 20-millisecond budget per authorisation, a model small enough to meet it on the available hardware, and the larger models reserved for the cases the small one flags.
For AI M&A, the cost of the forward pass at scale is the unit economics of every model-serving business, and the reason inference-optimisation companies were acquired.
Example
A payments company scores every authorisation with a neural network, 1.4 billion transactions a year, on a fleet of CPU servers costing €380,000 a year. An engineer profiles the forward pass: the model is 12 million parameters, in 32-bit floats, scored one transaction at a time. Quantising to 8-bit integers halves the memory traffic; batching authorisations in 5-millisecond windows raises hardware utilisation from 15% to 70%; pruning the model to 7 million parameters loses 0.1 points of AUC. The same fleet now has headroom for a second, larger model on the 6% of transactions the first marks uncertain, and the serving cost per thousand transactions falls by 60% with latency at the 99th percentile down from 18 to 9 milliseconds.
Common mistakes
- Serving one input at a time when batching would multiply throughput. - Ignoring quantisation; most models lose nothing measurable at 8 bits. - Choosing a model by benchmark without measuring its forward pass against the latency budget. - Confusing training cost (rare, large) with inference cost (constant, compounding). - Running the largest model on every request rather than on the ones that need it.