Data collection and management

You planned your data in the design phase (data strategy). This chapter covers carrying out that plan: getting the data, labelling it, keeping track of it and splitting it for training and testing.

Training data requirements¶

Training data should match the patients the system will see (age, sex, disease severity, other conditions, how the disease presents) and the setting: the equipment, measurement protocols and documentation habits of the wards where it will run. Rare outcomes need enough examples for the model to learn them. And the labels must be right, because a model learns inconsistent labels as faithfully as correct ones.

Data collection protocols¶

For retrospective EHR data, the protocol states which patients, period and variables you extract and how you derive labels. In the Netherlands, research on existing records is usually not subject to the WMO (the Medical Research Involving Human Subjects Act) and goes through the hospital’s local review. Health data still needs a lawful basis under GDPR Art. 6 and a condition under Art. 9(2), such as scientific research with safeguards (Art. 9(2)(j), together with Art. 24 of the UAVG, the Dutch GDPR Implementation Act, and Art. 7:458 of the Civil Code, which allows research use without consent under conditions if the patient has not objected) European Parliament and Council (2016).

For prospective collection, also write down and document:

• Inclusion and exclusion criteria

• How data is acquired: standard settings where possible, or recorded variation

• The consent process: what patients agree to and how consent is recorded

• An annotation protocol with written instructions for labelling each case, including unclear cases

Data management¶

Record where each data point came from (site, date, system or equipment, who entered it). Version your datasets so you know which data trained each model, and keep a catalogue of all datasets. The FAIR principles (data that is findable, accessible, interoperable and reusable) give a structure for this Wilkinson et al. (2016), and in the Netherlands, mapping variables to Nictiz’s zibs (standard Dutch health and care information models) makes data easier to combine across hospitals.

Log who accessed which data and when. GDPR Art. 32 requires appropriate security measures without listing specific ones; in Dutch healthcare, NEN 7513 sets out how access to electronic patient records is logged, and NEN 7510 covers information security more broadly.

Data splits¶

Split your data into a training set (to fit the model), a validation set (to compare models and tune settings) and a test set (held back and used once for the final estimate).

Split at patient level, never at the level of single records or scans. Where possible, keep a separate site or time period for external validation. Stratify the split so that outcome prevalence and key patient characteristics are comparable across sets.

Data augmentation¶

Data augmentation creates modified copies of training examples so the model sees more of the variation it will meet in practice. In medical imaging this includes rotation, cropping, changes in brightness and contrast, added noise and simulated differences between scanners or stains. For EHR data it can mean simulating missing measurements or the irregular timing of observations.