AI Fairness 360

An open-source Python and R toolkit for measuring bias in datasets and models and for trying out algorithms that reduce it before, during or after training.

What it is¶

AIF360 bundles a large set of group and individual fairness metrics with more than ten bias mitigation algorithms, such as reweighing, adversarial debiasing and reject option classification. You wrap your data in its dataset classes, compute metrics per protected attribute, and then compare mitigation methods at the pre-processing, in-processing or post-processing stage. IBM Research created it in 2018 and moved it to the LF AI & Data Foundation in 2020. Development has slowed (the latest release, v0.6.1, dates from April 2024), but the repository still receives occasional fixes.

In healthcare¶

In clinical AI you can use AIF360 to check whether labels, features or model predictions differ across groups such as sex, age band or ethnicity, and to test whether a mitigation step narrows that gap. Treat its metrics as a starting point: you still have to decide which disparities are clinically meaningful and which reflect real differences in disease prevalence.

Where to use it in The ELSA Way¶

• Data quality and fairness › Bias detection in training data: Its dataset metrics let you compare label rates and outcomes across demographic groups in your training data before you train a model.

• Data quality and fairness › Bias correction techniques: It implements reweighing, resampling-style pre-processing and fairness-penalised training: try several of the listed correction techniques on the same data and compare them.

References¶

• Bellamy RKE, Dey K, Hind M, Hoffman SC, Houde S, Kannan K, et al. AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias. IBM Journal of Research and Development. 2019;63(4/5):4:1-4:15. Bellamy et al. (2019)

Related tools¶

• Fairlearn: An open-source Python library that breaks model performance down by subgroup and offers algorithms to reduce the gaps you find.

• SHAP: A Python library that explains individual predictions by estimating how much each input feature pushed the model’s output up or down, based on Shapley values from game theory.

Also listed in: RAAIT toolbox.