MLflow

An open-source platform that records what went into each model run and keeps a versioned registry of trained models, which lets you trace any deployed model back to its code, data and settings.

What it is¶

MLflow Tracking logs the parameters, metrics, code version and output files of every training run, and lets you compare runs in a web interface. The Model Registry stores trained models with version numbers, stages and notes, which records which model is in use and how it was produced. Newer releases add tracing and evaluation features for applications built on large language models. Databricks launched MLflow in 2018, and it is now a Linux Foundation project; you can run it entirely on your own servers.

In healthcare¶

For a clinical AI system, MLflow can provide part of the audit trail that regulators and hospital governance expect: which data version, code commit and hyperparameters produced the model version that is currently deployed. It does not replace Git or proper data versioning, but it links them to each model.

Where to use it in The ELSA Way¶

• Documentation › Version control: Each MLflow run records the code commit and settings used, and the registry numbers model versions, which helps you link every model version to the exact code and data behind it.

• Logging and traceability › What to log: Store the registry’s version identifier with each logged prediction, and every logged AI output can be traced to the model that produced it.

References¶

• Zaharia M, Chen A, Davidson A, Ghodsi A, Hong SA, Konwinski A, et al. Accelerating the Machine Learning Lifecycle with MLflow. IEEE Data Engineering Bulletin. 2018;41(4):39-45. https://people.eecs.berkeley.edu/~matei/papers/2018/ieee_mlflow.pdf

Related tools¶

• Evidently: An open-source Python library for checking data quality, data drift and model performance, which you can run as one-off reports or as an ongoing monitoring dashboard.

• Model Cards: A short, structured document that travels with a trained model and tells readers what it is for, how it was tested and where it performs worse.