ML production

Monitor and Measure ROI

Automated strategies to ensure the performance of all production models.

Monitor Any Model No Matter Where it’s Hosted

Understand service health, data drift, and accuracy statistics, and configure monitoring, notification, and retraining settings.

Get Real Time Insights and Alerts

Live health monitoring, alerts, and deep production diagnostics let you see exactly which models are having issues.

Compare and Replace Models with Performance in Mind

DataRobot proactively and automatically suggests challenger models that you can quickly update to prevent production issues.

Maintain ROI of Models in Production

Calculate ROI for complex use cases, then manage and maintain the performance of your deployments over time.

Track Custom Metrics

Enterprises need to be able to directly tie their ML initiatives to top and bottom line impact. DataRobot’s custom inference metrics allow you to build and track business critical metrics and monitor the ROI of each deployment in one central location to ensure they continue to deliver value, even if those models are running outside of the DataRobot AI Platform. Now you can clearly understand business critical metrics like the cost of an error and feed that information into your BI dashboard.

Effortlessly Manage Drift and Accuracy

With a suite of drift and accuracy management capabilities, monitoring and maintaining the performance and health of all your models has never been easier. The speed and depth at which you can analyze a shift means you can take appropriate action before the business is impacted. Easily visualize data drift for a variety of data types including text, then track accuracy for specific batches of predictions and compare them. Drill down into segments to see which specific trends are driving the overall changes in your metrics.

Challenge Your Models

Don’t let your production models get lazy. Analyze performance against your real-world scenarios to identify the best possible model at any given time. Bring your own challenger models with you or allow the DataRobot AI Platform to create them for you. Then generate challenger insights for a deep and intuitive analysis of how well the challenger performs and how it measures up to the champion. Challenger comparisons can be used for Time Series, multiclass, and external models.

Effortless Operational Observability

Our MLOps capabilities give you a 360 degree view of operational activities, alerting and tracking your entire fleet of models. You can graph and set policies around errors and model latency. This helps you maintain service health, uphold your SLAs, and run robust AI-driven applications. To react and respond when deployments start decaying, create multiple alerts based on chosen thresholds and customized model refresh strategies – take action after an event – like a drop in accuracy or when drift occurs – or on a specific schedule.

Monitor for Bias and Fairness

By leveraging five different industry standard fairness metrics used to check for model bias, DataRobot gives you a strong defensive strategy and a guided experience to help you determine which fairness metric will be most meaningful to your use case. After deployment, DataRobot ML Production ensures your model isn’t vulnerable to bias over time, with automated alerts to inform you if your model falls below set thresholds. If bias is detected you can leverage insights to help identify the root cause and quickly address it.