Model Cards

A short, structured document that travels with a trained model and tells readers what it is for, how it was tested and where it performs worse.

What it is¶

A model card is a few pages of standard headings that you fill in for each model you release: details of the model, its intended and out-of-scope uses, the factors that may affect its behaviour, the metrics you chose, the training and evaluation data, results broken down by subgroup, ethical considerations, and caveats. The format was proposed by researchers at Google in 2018 and has since become a common way to publish models, for example on model-sharing platforms. You can write one in plain Markdown; no special software is needed.

In healthcare¶

For a clinical model, the subgroup results and the out-of-scope uses are the parts readers need most: report performance by age, sex, ethnicity, site and device where you can, and say plainly which patients and settings the model was never tested on.

Where to use it in The ELSA Way¶

• Documentation › Model card: The original model card proposal; its headings and worked examples show how to fill in each field this section lists.

References¶

• Mitchell M, Wu S, Zaldivar A, Barnes P, Vasserman L, Hutchinson B, Spitzer E, Raji ID, Gebru T. Model cards for model reporting. In: Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* '19). ACM; 2019. p. 220-229. Mitchell et al. (2019)

Related tools¶

• Datasheets for Datasets: A set of questions you answer about a dataset, from why it was made to how it is maintained, so others can judge whether it suits their purpose.

• TRIPOD+AI: A 27-item checklist for writing up studies that develop or validate a clinical prediction model, whether it uses regression or machine learning.