Evidently

An open-source Python library for checking data quality, data drift and model performance, which you can run as one-off reports or as an ongoing monitoring dashboard.

What it is¶

Evidently compares a current batch of data or predictions against a reference set, such as your validation data, and produces reports and pass/fail tests. It includes more than 100 metrics, among them over 20 statistical tests for drift in input features and outputs, plus classification and regression metrics once true labels arrive. You can run it in a notebook, add it to a scheduled pipeline, or host its open-source monitoring UI yourself. The company behind it, Evidently AI, also sells a hosted Evidently Cloud with extra features such as alerting; that service is separate from the Apache-2.0 open-source library.

In healthcare¶

After a clinical model goes live, you can use Evidently to spot shifts in patient case mix, coding practice or scanner settings that may degrade performance, often before outcome labels are available. Because it runs self-hosted, you can keep monitoring data inside the hospital’s own infrastructure; check this before sending any data to the cloud version.

Where to use it in The ELSA Way¶

• Monitoring and auditing › Continuous performance monitoring: Its drift tests and performance reports give you a ready-made way to detect changes in input and output distributions and to track metrics over time once your model is in clinical use.

Related tools¶

• MLflow: An open-source platform that records what went into each model run and keeps a versioned registry of trained models, which lets you trace any deployed model back to its code, data and settings.