fairness in vision

Fairness in computer vision is the concern that a model performs unequally across groups of people defined by protected attributes — skin tone, gender, age, ethnicity — and that this disparity can cause real harm. The landmark wake-up call was 'Gender Shades' (Buolamwini and Gebru, 2018): commercial face-analysis systems classified gender almost perfectly on lighter-skinned men (error under 1%) but failed badly on darker-skinned women (error up to ~35%). Same model, same task, wildly different reliability depending on who you are. Because vision systems gate access to phones, buildings, benefits, and policing, such gaps are not academic — they translate into who gets misidentified, denied, or surveilled.

The disparities trace directly back upstream to dataset bias (training sets skewed toward certain demographics, so the model is simply trained less on under-represented groups) and to spurious cues, but fairness is its own evaluation discipline because it asks a question accuracy never does: not 'how good is the model on average?' but 'how is performance distributed across groups, and is that distribution acceptable?'. A model with 95% overall accuracy can hide 99% on one group and 70% on another; aggregate metrics actively conceal the gap.

To make fairness measurable, researchers formalize criteria, and a crucial, mathematically proven fact is that these criteria generally conflict — you usually cannot satisfy all at once. The main families are: demographic parity (the positive-prediction rate is equal across groups, regardless of true labels); equalized odds (true-positive and false-positive rates are equal across groups — equal error rates); equal opportunity (a relaxation requiring only equal true-positive rates); and calibration within groups (a given confidence means the same correctness rate for every group). An impossibility result shows that calibration and equalized odds cannot both hold (except in degenerate cases), so 'fair' is never a single universal definition — it is a value-laden choice of which criterion matters for a given use, and which errors are most harmful to whom.

In practice fairness work means auditing and mitigating. Auditing: disaggregate every metric by subgroup (per-group accuracy, FPR/FNR, calibration), test on demographically balanced benchmarks, and report the gaps, not just the mean — supported by documentation like model cards. Mitigation spans the pipeline: pre-processing (rebalance or augment under-represented groups), in-processing (fairness constraints or adversarial debiasing that removes group-predictive information from features), and post-processing (group-specific thresholds). Hard caveats remain: protected attributes are often unavailable or themselves hard to infer fairly; enforcing one fairness metric can worsen another or reduce overall accuracy; and many harms (mass surveillance, consent, deployment context) are not fixable by any metric and demand governance, not just better optimization. Fairness is therefore treated as a socio-technical problem, where the metric is necessary but never sufficient.