Monitoring and explainability of models in production
The machine learning lifecycle extends beyond the deployment stage. Monitoring deployed models is crucial for continued provision of high quality machine learning enabled services. Key areas include model performance and data monitoring, detecting outliers and data drift using statistical techniques, and providing explanations of historic predictions. We discuss the challenges to successful implementation of solutions in each of these areas with some recent examples of production ready solutions using open source tools.
Code (1)
Tasks
BIG-bench Machine LearningSimilar Papers 제목 키워드 기반
AlerTiger: Deep Learning for AI Model Health Monitoring at LinkedIn
Data-driven companies use AI models extensively to develop products and intelligent business solutions, making the health of these models crucial for business success. Model monitoring and alerting in industries pose uni…
Anomaly DetectionDeep LearningCollaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Process
The introduction of machine learning (ML) components in software projects has created the need for software engineers to collaborate with data scientists and other specialists. While collaboration can always be challengi…
FairnessOpen-Source Reproduction and Explainability Analysis of Corrective Retrieval Augmented Generation
Corrective Retrieval Augmented Generation (CRAG) improves the robustness of RAG systems by evaluating retrieved document quality and triggering corrective actions. However, the original implementation relies on proprieta…
Semantic SimilarityEntity AlignmentCLAIMED, a visual and scalable component library for Trusted AI
Deep Learning models are getting more and more popular but constraints on explainability, adversarial robustness and fairness are often major concerns for production deployment. Although the open source ecosystem is abun…
Adversarial RobustnessFairnessBenchmarking real-time monitoring strategies for ethanol production from lignocellulosic biomass
The goal of this paper is to review and critically assess different methods to monitor key process variables for ethanol production from lignocellulosic biomass. Because cellulose-based biofuels cannot yet compete with n…
Benchmarking