algorithmic bias
/ AL-guh-rith-mik BY-us /
Algorithmic bias is when a computer system treats some groups of people systematically worse than others, not because anyone wrote a rule saying "be unfair," but because the patterns it learned were skewed to begin with. Picture a hiring tool trained on a decade of a company's past hires. If the company mostly hired men, the tool quietly learns that "looks like a man" correlates with "good hire" — and starts penalizing résumés that mention a women's college or a women's chess club. Nobody typed in that prejudice; it leaked in through the history the machine was fed.
More precisely, algorithmic bias is a measurable difference in how a system performs or decides across groups defined by things like race, sex, age, or disability — differences that are unjustified, harmful, or both. It can creep in from biased training data, from a label that secretly encodes prejudice (e.g. using "who got arrested" as a stand-in for "who committed a crime"), from features that act as proxies for protected traits (a postal code standing in for race), or from optimizing a goal that ignores fairness entirely.
Why it matters: these systems now help decide who gets a loan, a job interview, a medical referral, or a longer prison sentence — at a scale and speed no human committee could match. A single biased model can replicate the same unfair decision millions of times, and because it wears the mask of math, people tend to trust it more than they would a visibly prejudiced human. The honest takeaway is that an algorithm is not automatically neutral; it inherits the world's inequities unless someone deliberately measures and counteracts them.
In 2018 a large retailer scrapped an experimental résumé-screening tool after engineers found it downgraded any application containing the word "women's" (as in "women's soccer captain"). The model had been trained on ten years of mostly-male résumés and had quietly concluded that male candidates were preferable.
The bias was in the historical data, not in any hand-written rule — and it took deliberate auditing to catch it.
"The math is objective, so it can't be biased" is the most dangerous myth here. Math faithfully reproduces whatever is in the data, and historical data is full of human inequity. Bias is the default outcome, not a rare glitch — neutrality is the thing you have to work for.