Recommending Root-Cause and Mitigation Steps for Cloud Incidents using Large Language Models
Incident management for cloud services is a complex process involving several steps and has a huge impact on both service health and developer productivity. On-call engineers require significant amount of domain knowledge and manual effort for root causing and mitigation of production incidents. Recent advances in artificial intelligence has resulted in state-of-the-art large language models like GPT-3.x (both GPT-3.0 and GPT-3.5), which have been used to solve a variety of problems ranging from question answering to text summarization. In this work, we do the first large-scale study to evaluate the effectiveness of these models for helping engineers root cause and mitigate production incidents. We do a rigorous study at Microsoft, on more than 40,000 incidents and compare several large language models in zero-shot, fine-tuned and multi-task setting using semantic and lexical metrics. Lastly, our human evaluation with actual incident owners show the efficacy and future potential of using artificial intelligence for resolving cloud incidents.
Code (0)
등록된 구현이 없습니다.
Tasks
ManagementQuestion AnsweringText SummarizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
NetCause: Counterfactual Learning for Root Cause Analysis in Large-Scale Networks
Can a learned model capture how faults propagate through a large-scale network and use this knowledge to causally attribute customer impact to its underlying root cause? Existing root cause analysis techniques often rely…
Self-Supervised LearningOn Root Cause Localization and Anomaly Mitigation through Causal Inference
Due to a wide spectrum of applications in the real world, such as security, financial surveillance, and health risk, various deep anomaly detection models have been proposed and achieved state-of-the-art performance. How…
Anomaly DetectionCausal InferenceDeoxys: A Causal Inference Engine for Unhealthy Node Mitigation in Large-scale Cloud Infrastructure
The presence of unhealthy nodes in cloud infrastructure signals the potential failure of machines, which can significantly impact the availability and reliability of cloud services, resulting in negative customer experie…
Causal InferenceRoot Cause Localization for Microservice Systems in Cloud-edge Collaborative Environments
With the development of cloud-native technologies, microservice-based software systems face challenges in accurately localizing root causes when failures occur. Additionally, the cloud-edge collaborative environment intr…
Graph Neural NetworkWhy Do AI Agents Systematically Fail at Cloud Root Cause Analysis?
Failures in large-scale cloud systems incur substantial financial losses, making automated Root Cause Analysis (RCA) essential for operational stability. Recent efforts leverage Large Language Model (LLM) agents to autom…
Prompt Engineering