Fairlearn is the library to reach for first when your model lives in the scikit-learn world. It does two jobs: disaggregated measurement through MetricFrame, and mitigation through post-processing and reduction algorithms. The measurement half is the one you will use every week; the mitigation half is the one you should use deliberately and rarely [1].
MetricFrame is the audit table
MetricFrame computes any metric you hand it, sliced by one or more sensitive features. It is the codified version of the discipline from the previous chapter: never a single number, always a per-group table.
from fairlearn.metrics import MetricFrame, selection_rate, count
from sklearn.metrics import recall_score, precision_score
mf = MetricFrame(
metrics={
"selection_rate": selection_rate,
"tpr": recall_score, # equal opportunity compares this row
"precision": precision_score, # predictive parity compares this row
"n": count, # never report a slice without its N
},
y_true=y_test,
y_pred=y_pred,
sensitive_features=X_test[["sex", "race"]],
)
print(mf.by_group) # one row per group (or per intersection)
print(mf.difference()) # max minus min, per metric
print(mf.ratio()) # min over max, per metric (four-fifths style)
Three habits worth hardening. Pass a DataFrame with two columns to sensitive_features and the rows become intersections, which is where audits find what single-axis tables miss. Always include count, because a 12-person slice with a dramatic gap is a sampling artifact until proven otherwise. And keep the raw by_group table in the report; difference() is a headline, not a finding.
For the standard gaps, Fairlearn ships ready-made scalars: demographic_parity_difference, demographic_parity_ratio, equalized_odds_difference, and equal_opportunity_difference. They agree with what you would compute from by_group by hand; use them for dashboards and CI thresholds, and the full table for investigation.
Mitigation: post-processing first
When measurement shows a real gap and the product decision is to close it, Fairlearn gives you three families in increasing order of invasiveness.
ThresholdOptimizer is the Hardt et al. post-processing approach: it leaves the trained model untouched and learns a per-group decision threshold on scores to satisfy a constraint such as "equalized_odds" or "demographic_parity" [2]. It needs the sensitive attribute at prediction time, which is a legal question before it is a technical one; in some domains per-group thresholds are exactly what regulation forbids, in others they are the accepted remedy. It is the cheapest mitigation to try and the easiest to reason about, which is why it comes first.
The reductions family, ExponentiatedGradient and GridSearch, rewrites training instead: it wraps any scikit-learn estimator and turns the fairness constraint into a sequence of reweighted training problems. The result does not need the attribute at prediction time, which resolves the legal issue above at the cost of retraining and a wider search. Constraints are objects (DemographicParity(), EqualizedOdds(), TruePositiveRateParity()), and the difference_bound you set is a product decision to document, not a default to accept.
CorrelationRemover is the preprocessing option: it strips linear correlation between features and the sensitive attribute. It is also the one to be most skeptical of, because nonlinear proxy structure survives it, and a clean correlation matrix is easy to mistake for a clean model.
Figure: Where each Fairlearn mitigation intervenes in a scikit-learn pipeline: CorrelationRemover rewrites features before training, the reductions ExponentiatedGradient and GridSearch retrain the estimator under a fairness constraint, and ThresholdOptimizer learns per-group decision thresholds on the trained model's scores. ThresholdOptimizer is the cheapest to try but needs the sensitive attribute at prediction time; the reductions do not.
Read the trade-off before you ship it
Every mitigation moves accuracy somewhere, and the impossibility results guarantee you cannot satisfy every parity notion at once [3]. The honest workflow is to rerun the full MetricFrame audit on the mitigated model and put before and after side by side: the gap you closed, the gaps you moved, the overall and per-group accuracy you paid. A mitigation that closes the selection-rate gap by refusing qualified applicants from the majority group is a choice someone accountable has to sign, with the table in front of them.
CAUTION
Do not run mitigation to make a bad audit table look better. Mitigation changes the model; it does not answer why the gap existed. If the gap came from label bias or a proxy feature, the reductions will paper over a data problem you still have. Diagnosis first, in the style of error analysis: read the false negatives from the disadvantaged group before deciding anything.
Where Fairlearn stops
Fairlearn assumes a tabular, scikit-learn shaped world with binary decisions at the end. It has no opinion on text generation, no probe sets for LLMs (that is the bias probes chapter), and a narrower metric list than AIF360, which the next chapter covers. For the common case, an sklearn pipeline that ends in a yes-or-no about a person, it is the shortest path from question to defensible table. The fairness audit recipe runs this entire loop on a public dataset with seeded code.