---
title: "AI Bias"
url: "https://mahouse.world/glossary/ai-bias"
type: "glossary term"
topic: "AI governance"
author: "Ihor Vlasov"
reviewed: "2026-09-20"
published: "2026-08-10"
---

# AI Bias

AI bias is the tendency of a model to produce systematically different outcomes for different groups — by sex, race, age, postcode, nationality — not because the group matters to the task but because the training data, the features or the objective encoded the difference. In fintech it is a fair-lending and consumer-protection problem with law behind it: a credit model that declines women more, an identity check that fails darker faces more.

**Also known as:** Algorithmic bias, Model bias, Disparate impact

## Key facts

- **Used at:** credit scoring and pricing, fraud and AML models, identity verification, insurance pricing, hiring, customer-service routing — anywhere a model decides about people
- **Binding:** anti-discrimination and fair-lending law (the US Equal Credit Opportunity Act and disparate-impact doctrine, the EU's equality directives and the AI Act's obligations on high-risk systems, the UK's Equality Act and Consumer Duty); regulators expect testing and mitigation, and enforcement has followed
- **Typical sources:** training data that reflects past discrimination; proxies for protected characteristics (postcode for race, first name for sex, shopping patterns for age); objectives that reward outcomes correlated with a group; and measurement — a model tested on a population unlike the one it serves
- **Typical measures:** outcome rates by group (approval, pricing, false-positive rates), disparate-impact ratios (the four-fifths rule in US practice), equalised odds and calibration across groups; mitigation by removing proxies, reweighting, constrained training, and human review of edge cases
- **Parties:** the model's developers, the business deploying it, the customers affected, the regulators, and the auditors and fairness researchers who test it

## How AI bias works in practice

A lender trains a credit model on ten years of its own lending. The data contains the decisions of human underwriters, who approved fewer applicants from certain postcodes; the model learns that postcode predicts default — partly because it does, and partly because those applicants were given worse terms and defaulted more. The model never sees race; it sees postcode, and postcode is race in much of the country. Deployed, it declines applicants in those postcodes at twice the rate, with a default prediction that is partly true and partly the history it inherited. The lender's fair-lending test — approval rates by protected group, with a disparate-impact ratio below 0.8 — finds it; the regulator, if the lender does not, finds it later.

Mitigation is engineering and judgement. Remove the proxies, and the model finds others; constrain the training so that outcome rates across groups fall within tolerance, and predictive power drops a little; test on the population the model will serve, not the one it was trained on; keep the features the model may use to those with a business justification the lender can defend; and document all of it, because the regulator's question is not "is the model fair" but "how do you know". Identity-verification vendors learned the same lesson from face-matching models trained on lighter faces, and fraud teams from models that flagged foreign names.

For an acquirer of a fintech, the model's fairness testing is diligence with a legal consequence: a biased model is a redress liability and a regulatory finding the buyer inherits.

## Example

A consumer lender's new model lifts approval by 8% at the same loss rate. Its fairness review before deployment: approval rates by sex, age band, and (inferred, for testing only) ethnicity; the model approves applicants under 25 at 62% of the rate of those over 40 at the same score, driven by a "years at current address" feature. Age is a protected characteristic; the feature is a proxy. The team removes it and two related features, retrains, and loses 1.5 points of the lift; the under-25 ratio moves to 0.91. The documentation — the tests, the features removed, the trade-off accepted — goes to the model-risk committee and, a year later, to the regulator's thematic review, which cites the lender as an example of what it expected and fines two peers that had not tested.

## Common mistakes

- Removing the protected characteristic and believing the model cannot discriminate.
- Testing fairness once, at build, and never after retraining on new data.
- Fairness measured on the training population rather than the deployed one.
- Vendors' models deployed with no fairness testing because "the vendor did it".
- Treating a fairness finding as a PR problem when it is a legal one.

## Related terms

- [Explainable AI](https://mahouse.world/glossary/explainable-ai.md)
- [AI Alignment](https://mahouse.world/glossary/ai-alignment.md)
- [Training Data](https://mahouse.world/glossary/training-data.md)
- [Misalignment](https://mahouse.world/glossary/misalignment.md)
- [Identity Verification](https://mahouse.world/glossary/identity-verification.md)
- [ESG Investing](https://mahouse.world/glossary/esg-investing.md)

— Ihor Vlasov, M&A FinTech Expert. Last reviewed 2026-09-20. https://mahouse.world/authors/ihor-vlasov