Vision You Can Trust: Bias, Fairness, Interpretability, and Deepfakes

When vision systems harm people

Start with three documented stories. In a widely cited 2018 audit, commercial face-classification systems were nearly perfect on lighter-skinned men but misclassified the gender of darker-skinned women up to roughly one time in three — an error gap of over thirty percentage points between the easiest and hardest group. Around the same time, an automatic soap dispenser went viral because its infrared sensor reliably triggered for light-skinned hands and ignored dark-skinned ones. And in 2015, a photo app auto-tagged a Black couple with the label 'gorillas' — a label the vendor could only 'fix' by deleting that category entirely for years.

These are not just embarrassing bugs. Police forces in several countries have used face-matching to generate suspect leads, and at least a handful of people — disproportionately Black men — have been wrongfully arrested after a face-recognition system returned a confident but wrong match. The model's number was high; the human consequence was a jail cell. The point of this guide is that the harm was real even though, on the vendor's own benchmark, the system was 'accurate'.

So this capstone adds three pillars on top of accuracy and robustness. First, fairness — does the system fail more for some groups than others, and how do we even measure that? Much of the trouble traces back to skewed training data rather than a malicious algorithm, and there is a formal study of fairness in vision to make these questions precise. Second, interpretability — can we see why a decision was made, enough to debug and audit it? Third, integrity — the deepfake problem, where the generative models we learned to evaluate become tools for fraud and disinformation. The tone throughout is calm and engineering-minded: not 'AI is evil', but 'here is where it breaks, here is how to measure it, here is what to do'.

Where bias comes from: the data

It is tempting to imagine a prejudiced line of code somewhere, but that is almost never where vision bias lives. Dataset bias means the training data systematically misrepresents the world the model will act in. A face dataset scraped from one country's celebrity photos will be lighter-skinned, younger, and more conventionally lit than the population a deployed system meets. The network faithfully learns the statistics it was shown — including the gaps. The unfairness is real, but it was inherited from the data and the way the problem was framed, not invented by the optimizer.

This connects directly to Guide 4. Under-representation is literally a distribution shift — but the shift is between who is in the training set and who the system actually serves. If darker-skinned women are 5% of the training images but 25% of real users, then at deployment the model is constantly facing inputs from a region of input space it barely practiced on. Everything you learned about shift hurting accuracy now applies, except the pain is concentrated on a specific group of people rather than spread evenly.

1. Sampling: how examples are collected. Web-scraped, lab-collected, or convenience samples over-represent whoever is easy to capture and under-represent everyone else.
2. Labeling: humans (or older models) assign the ground-truth labels, importing their own assumptions, fatigue, and cultural blind spots into the 'truth' the model is trained against.
3. Proxy variables: the model latches onto a correlated shortcut (skin reflectance, background, clothing) that stands in for the protected attribute even when that attribute is never an explicit input.
4. Feedback loops: biased predictions shape the world, and tomorrow's training data is collected from that biased world — so the bias amplifies itself over successive retrainings.

Measuring fairness in vision

Fairness only becomes engineering when it becomes a number. The trick is simple and reuses the confusion matrix from Guide 1: instead of computing one matrix over everyone, compute a separate confusion matrix per group and compare the metrics. Three formal criteria dominate the field, each a plain-English promise. Demographic parity: the model says 'yes' at the same rate for every group. Equal opportunity: among people who truly are positive, the model catches them at the same rate — that is, equal true-positive rate (recall) across groups, which is exactly the recall idea applied per group. Equalized odds: stronger still — both the true-positive rate and the false-positive rate match across groups. Studying which of these to demand is the heart of fairness in vision.

Two fairness criteria written as conditional probabilities.

Read the symbols carefully. Here ŷ is the model's prediction (1 = positive/'yes'), Y is the true label, and A is the protected attribute — the group, e.g. A=darker-skinned vs A=lighter-skinned. Demographic parity, P(ŷ=1 | A=a), is the selection rate: the chance the model says 'yes' given you are in group a; parity means that chance does not depend on a — same selection rate regardless of group. Equalized odds conditions additionally on the truth Y=y: P(ŷ=1 | Y=1, A=a) is the true-positive rate and P(ŷ=1 | Y=0, A=a) is the false-positive rate; demanding both match across groups means the model is equally good at finding real positives and equally prone to false alarms for everyone. Intuition for why these clash: if the groups have different base rates — say genuine positives are 20% of group A but 10% of group B — then matching the overall 'yes' rate (parity) forces you to flag extra people in the smaller-positive group, who are mostly negatives, which inflates that group's false-positive rate and breaks equalized odds. You cannot, in general, hold the selection rate fixed and the error rates fixed at the same time when the underlying truth differs between groups.

# Per-group fairness audit reusing the Guide-1 confusion matrix.
# Two groups, 1000 people each. Note the DIFFERENT base rates.
#   Group A: 200 truly positive (20% base rate)
#   Group B: 100 truly positive (10% base rate)

groups = {
    # tp, fn, fp, tn
    "A": dict(tp=180, fn=20, fp=40, tn=760),
    "B": dict(tp=70,  fn=30, fp=45, tn=855),
}

for g, c in groups.items():
    n   = c["tp"] + c["fn"] + c["fp"] + c["tn"]
    acc = (c["tp"] + c["tn"]) / n                     # overall accuracy
    tpr = c["tp"] / (c["tp"] + c["fn"])               # recall / equal-opportunity metric
    fpr = c["fp"] / (c["fp"] + c["tn"])               # false-positive rate
    sel = (c["tp"] + c["fp"]) / n                      # selection rate / demographic parity
    print(f"{g}: acc={acc:.3f}  TPR={tpr:.2f}  FPR={fpr:.3f}  sel_rate={sel:.3f}")

# A: acc=0.940  TPR=0.90  FPR=0.050  sel_rate=0.220
# B: acc=0.925  TPR=0.70  FPR=0.050  sel_rate=0.115
# Combined accuracy ~ 0.93  ->  looks great on the headline metric,
# yet recall is 0.90 vs 0.70 (a 20-point equal-opportunity gap),
# and selection rate 0.22 vs 0.115 (demographic parity also fails).

Opening the black box: interpretability

Suppose the audit above shows a recall gap. Your next question is 'why?' — and you cannot answer it from accuracy numbers alone. Model interpretability is the study of making a model's reasoning legible to a human: what in the input drove this output? It splits into two families. Inherently interpretable models (a small decision tree, a linear model) are readable by construction — you can literally see the rule. Post-hoc explanations are applied after training to an already-opaque deep network, reverse-engineering an account of its behavior. Deep vision models are firmly in the second camp: 50 million weights are not a rule a human can read, so we build tools to interrogate them.

The workhorse post-hoc tool for vision is the saliency map, and its most popular form is Grad-CAM. The intuition is a sensitivity question: which regions of the input, if nudged, would most change the score for the predicted class? Highlight those regions and you get a heatmap laid over the image — bright where the network 'looked', dim where it ignored. Grad-CAM computes this efficiently using the gradients flowing into a convolutional layer's feature maps.

Grad-CAM: a gradient-weighted sum of feature maps, passed through ReLU.

Unpack it piece by piece. Pick a convolutional layer; it outputs a stack of feature maps, and A^k is the k-th map — a small 2D grid of activations (say 14×14), where A^k_{ij} is the activation at spatial location (i, j). The term ∂y_c/∂A^k_{ij} is the gradient of the score y_c for the predicted class c with respect to that one activation: how much the class score would rise if that location lit up a little more. We average those gradients over all Z spatial positions (Z = height × width of the map, e.g. 196) to get α_k, a single importance weight for feature map k — 'how much did this whole map matter for class c?'. Then we take the weighted sum Σ_k α_k A^k, combining maps by their importance, and apply ReLU to keep only positive evidence (regions that support class c, not ones that argue against it). The result L is a coarse heatmap, upsampled to image size, that highlights the regions the network actually leaned on. Concretely: if the 'cat' score barely changes no matter how you perturb the map locations over the background but jumps when you perturb the locations over the animal's face, the gradients — and hence the heatmap — concentrate on the face. That is the explanation we wanted.

Synthetic media: deepfakes and their detection

In Guide 2 we learned to evaluate generators with FID — a metric for how realistic a model's samples are. Turn that around and the danger is obvious: a generator that scores a great FID is, by definition, one that produces images people cannot tell from real. The same GANs and diffusion models can fabricate convincing faces of people who do not exist, swap a real person's face into a video, clone a voice from seconds of audio, or stage events that never happened. The harms are concrete: financial fraud via a fake video call from your 'CEO', non-consensual intimate imagery, and political disinformation timed for an election. Deepfake detection is the field that tries to push back.

At its core, deepfake detection is just a binary classifier — real vs synthetic — trained on examples of each. What makes it more than a stock image classifier is the subtle, often invisible cues it learns to exploit. Generators leave fingerprints: frequency-domain artifacts (upsampling layers imprint a faint, regular grid in the image's spectrum that the eye misses but a Fourier transform reveals); inconsistent lighting and reflections (eyes that don't reflect the same light source, shadows that disagree); and broken physiological signals (unnatural or absent eye-blinking, or the absence of the tiny skin-color pulsing that a real beating heart causes).

Here is the uncomfortable structural truth, echoing Guides 3 and 4: detection is an adversarial, moving target. Every cue a detector relies on is a flaw the next generation of generators will be trained to remove — frequency artifacts can be filtered out, blinking can be synthesized. So a detector at 99% accuracy today can collapse against next year's model. Worse, a detector trained on one generator's fakes suffers severe distribution shift when shown fakes from a different architecture it never saw; an unseen generator is effectively an out-of-distribution input, and confident detectors fail silently exactly there. Detection is necessary but never finished.

Deploying responsibly: putting the whole track together

This is the finale not just of the track but of the evaluation-and-ethics arc that runs through all twenty tracks. Everything you learned converges into one discipline: before a vision system touches real people, you measure where it fails, across every axis we have named — average performance, robustness, fairness, interpretability, and integrity. Trustworthiness is not a feature you add at the end; it is the sum of honest measurements and the willingness to act on them.

1. Report several task-appropriate metrics with confidence intervals, not one headline accuracy (Guides 1–2): precision/recall, mAP, IoU, FID — whatever fits the task — each with uncertainty bars.
2. Stress-test under adversarial attack and natural distribution shift (Guides 3–4): report worst-case as well as average-case behavior, and flag out-of-distribution inputs at run time.
3. Audit fairness per group, choosing and justifying a criterion (this guide): publish the per-group confusion matrices, not just the pooled one.
4. Interpret representative decisions (this guide): use saliency/attention to confirm the model leans on the object, not a spurious proxy — and to investigate any fairness gap you found.
5. Document everything and keep watching: ship a model card and a datasheet, define human oversight and recourse, and monitor continuously after launch.

Two artifacts have become standard for that documentation. A model card is a short, honest spec sheet for a trained model: its intended use, the metrics above broken down by group, known limitations, and conditions under which it should not be used. A datasheet for a dataset does the same for the data: how it was collected, who is in it, what it leaves out, and what consent covered it — exactly the upstream facts that, as we saw in Section 2, decide whether dataset bias is even visible. Together they turn 'we tested it' into a record someone else can check.

A word on governance, lightly. Regulation is arriving (the EU AI Act treats remote biometric identification as high-risk or prohibited in some uses), and many vision capabilities are dual-use — the same face model can unlock your phone or surveil a protest. Consent matters: people whose faces, voices, or medical images train your system have a stake in how it is used. You do not need to be a lawyer, but you do need to ask, before building, who could be harmed and whether the task should exist. The combination of fairness auditing, interpretability, and honest documentation is what lets you answer those questions with evidence instead of opinion.