AI Fairness 360 started at IBM and is now an incubation-stage project of the LF AI & Data Foundation, and it is the encyclopedic option: more than 70 metrics and over a dozen mitigation algorithms spanning the whole pipeline [1] [2]. It is a Python library first. The R package is a reticulate wrapper that installs and calls the Python one, and CRAN archived it in June 2023, so R users install it from GitHub [3]. Where Fairlearn optimizes for the common sklearn case, AIF360 optimizes for coverage. That trade shows in the API, and knowing the shape of it in advance saves an afternoon.
The dataset classes are the API
Everything in AIF360 flows through its own dataset containers, chiefly BinaryLabelDataset and StandardDataset. You declare the label, the favorable label value, the protected attributes, and which values count as privileged, and the metrics read that declaration:
from aif360.datasets import StandardDataset
from aif360.metrics import BinaryLabelDatasetMetric, ClassificationMetric
ds = StandardDataset(
df, label_name="approved", favorable_classes=[1],
protected_attribute_names=["sex"], privileged_classes=[[1]],
)
priv, unpriv = [{"sex": 1}], [{"sex": 0}]
# Dataset-level: bias in the labels themselves, before any model
dm = BinaryLabelDatasetMetric(ds, unprivileged_groups=unpriv, privileged_groups=priv)
print(dm.statistical_parity_difference()) # base-rate gap in the data
print(dm.disparate_impact()) # four-fifths style ratio
# Model-level: bias in the predictions
cm = ClassificationMetric(ds_test, ds_pred, unprivileged_groups=unpriv, privileged_groups=priv)
print(cm.equal_opportunity_difference())
print(cm.average_odds_difference())
The wrapping is boilerplate, but it buys a distinction Fairlearn does not foreground: BinaryLabelDatasetMetric measures the labels before any model exists. Running it on your training data answers "how biased is the ground truth", which is the question that decides whether mitigation belongs in the data or in the model. A statistical parity difference in the raw labels means historical process bias baked into what you are about to teach the model to imitate.
Figure: AIF360's two measurement levels. Wrapping a DataFrame in StandardDataset lets BinaryLabelDatasetMetric quantify label bias before any model exists, while ClassificationMetric compares ds_test against ds_pred. Bias in the ground truth points to pre-processing fixes like Reweighing; bias in the predictions points to in- or post-processing mitigation.
The algorithm zoo, by pipeline stage
| Stage | Algorithm | What it does | Reach for it when |
|---|
| Pre | Reweighing | Reweights training examples per group-label cell | You can retrain, want the model untouched otherwise |
| Pre | DisparateImpactRemover | Repairs feature distributions toward group medians | Feature-level repair is defensible in your domain |
| In | AdversarialDebiasing | Trains against an adversary predicting the attribute | Deep model, TensorFlow stack, strong parity target |
| In | PrejudiceRemover | Adds a fairness regularizer to the objective | Logistic-regression class models, tunable trade-off |
| Post | RejectOptionClassification | Flips low-confidence predictions near the boundary | No retraining allowed, score access only |
| Post | CalibratedEqOddsPostprocessing | Equalizes error rates while preserving calibration | Calibration is a hard requirement downstream |
Reweighing is the workhorse: simple, transparent, model-agnostic, and easy to explain to a reviewer. Adversarial debiasing is the strongest in-processing option and the most operationally expensive: AIF360's implementation runs on TensorFlow's tf.compat.v1 session API, so on a PyTorch stack Fairlearn's AdversarialFairnessClassifier, which takes PyTorch or TensorFlow 2 models, is the easier route [2] [4]. The post-processing pair mirrors the trade the metrics chapter warned about: you can equalize odds or preserve calibration, and the calibrated variant is explicit that it relaxes one to keep the other [5].
Honest comparison with Fairlearn
Choose Fairlearn when your model is an sklearn estimator and your need is a disaggregated audit table with one or two standard gaps; MetricFrame is a better audit surface than anything in AIF360, and the API fits pipelines you already have [4]. Choose AIF360 when you need a metric Fairlearn does not ship, dataset-level label bias measurement, or a specific algorithm Fairlearn lacks, such as Reweighing, reject option classification, or calibrated equalized odds. Many teams use both in one audit: AIF360's dataset metrics on the labels, Fairlearn's MetricFrame on the model.
Two cautions from production use. First, alignment errors hide in the labels. ClassificationMetric rejects a ds_pred that differs from ds_test in anything other than labels and scores, but the usual pattern is ds_test.copy(deepcopy=True) followed by overwriting labels with model output, and predictions computed on a reordered DataFrame pass that check and produce plausible nonsense. Assert row order before you assign predictions. Second, the release has gone stale. The newest version on PyPI is still 0.6.1, from April 2024 (fetched as of 2026-10-02), and it sets no upper bound on scikit-learn, while an October 2025 issue reports it failing with an AttributeError on scikit-learn 1.7.2 [6] [7]. Since that release, the main branch has added an intersectional mitigation algorithm, capped scikit-learn below 1.6, and taken June 2026 security fixes for model loading, none of which has shipped [2]. Pin scikit-learn<1.6 yourself. If you run it next to Fairlearn 0.14, which needs scikit-learn 1.5 or later, the shared window is 1.5.x, and Fairlearn's unreleased 0.15 raises its floor to 1.6, which closes that window; give AIF360 its own environment rather than fight the pins [8] [4].
NOTE
Seventy metrics is not a feature for your report. The number you can defend is the number you chose in advance for a written reason. Use the zoo to find the one metric that matches your harm model, then report that metric per group with intervals, and resist decorating the report with every statistic the library can emit.
The fairness audit recipe demonstrates the combined workflow: label-bias measurement in the AIF360 style, model audit and mitigation in Fairlearn, one report at the end.