Datasheets for Datasets

A set of questions you answer about a dataset, from why it was made to how it is maintained, so others can judge whether it suits their purpose.

What it is¶

Datasheets for Datasets is a list of questions grouped by stage of a dataset’s life: motivation, composition, collection, preprocessing and labelling, uses, distribution, and maintenance. You answer them in prose when you create or release a dataset, ideally while the work is still in progress so that the questions also shape your choices. The result is a document that helps people who reuse the data spot gaps, sampling problems and uses the data was never meant for. It was first posted in 2018 and published in Communications of the ACM in 2021.

In healthcare¶

For clinical data, the questions on consent, sensitive attributes, who is missing from the data and how labels were assigned are especially useful; you may want to add fields for coding systems, care setting and the time window of data collection.

Where to use it in The ELSA Way¶

• Documentation › Datasheet for the data: The section’s headings follow this paper, which lists the questions to answer under each one.

References¶

• Gebru T, Morgenstern J, Vecchione B, Vaughan JW, Wallach H, Daumé III H, Crawford K. Datasheets for datasets. Communications of the ACM. 2021;64(12):86-92. Gebru et al. (2021)

Related tools¶

• Model Cards: A short, structured document that travels with a trained model and tells readers what it is for, how it was tested and where it performs worse.