Model monitoring
H2O MLOps model monitoring observes the performance and behavior of deployed models so they operate effectively and you can identify issues such as model drift.
Key capabilities
- Configure monitoring: Enable monitoring during deployment, select columns to monitor, and provide baseline data for comparison.
- Analyze drift in Superset: Run SQL-based drift queries (TVD, PSI, Z-Score, Hellinger Distance) to detect changes in input data distributions.
- Build dashboards: Save drift results as datasets and create Superset dashboards for ongoing visibility.
- Set up alerts: Get notified when drift metrics exceed thresholds via Slack or Email.
- Export raw data to Kafka: Send raw scoring request and response data to Kafka for custom processing, auditing, or compliance.
Visibility and permissions
Access to monitoring data in Superset is controlled by workspace membership and object ownership:
| Resource | Non-admin users | Admin users |
|---|---|---|
| Schema/data access | Only schemas for workspaces they belong to | All schemas |
| Dashboards, charts, datasets | Only objects they created or co-own | All objects |
| Saved queries | Only their own | All saved queries |
| Alerts and reports | Only their own | All alerts and reports |
tip
If you cannot see a schema, dashboard, or alert, contact your administrator to verify your workspace membership or request co-ownership of the shared resource.
Get started
- Configure monitoring — Enable and configure monitoring during model deployment.
- Analyze drift in Superset — Run drift detection queries on your aggregated scoring data.
- Text column feature monitoring — Understand how H2O MLOps monitors text columns and run SQL drift queries for TEXT features.
- Superset dashboards — Create charts and dashboards to visualize drift over time.
- Monitoring alerts — Configure alerts and scheduled reports for drift thresholds.
- Raw data export to Kafka — Export raw scoring data for downstream processing.
Configure monitoring with the Python client
To configure monitoring programmatically, see Monitoring setup.
To monitor a model that H2O MLOps doesn't deploy, create an external deployment for it. See External deployments.
Feedback
- Submit and view feedback for this page
- Send feedback about H2O MLOps to cloud-feedback@h2o.ai