Experimental Setup
Evaluation Metrics
- Precision@K (PR@K) \( PR@K = \frac{1}{|\mathbb{A}|}\sum_{a \in \mathbb{A}}\frac{\sum_{i < k}R_a(i)\in V_a}{\min (K, |v_a|)} \), where \(\mathbb{A}\) is the set of system faults, \(a\) is one fault in \(\mathbb{A}\), \(V_a\) is the real root causes of \(a\), \(R_a\) is the predicted root causes of \(a\), and \(i\) is the \(i\)-th predicted cause of \(R_a\)
- Mean Average Precision@K (MAP@K) \( MAP@K = \frac{1}{K|\mathbb{A}|} \sum_{a \in \mathbb{A}} \sum_{i\leq j\leq K} PR@j \)
- Mean Reciprocal Rank (MRR) \( MRR@K = \frac{1}{|\mathbb{A}|}\sum_{a \in \mathbb{A}}\frac{1}{rank_{R_a}}\), where \(rank_{R_a}\) is the rank number of the first correctly predicted root cause for system fault \(a\).
Baseline Methods
| Method | Main Technique | Online/Offline | Time |
|---|---|---|---|
| PC | Constrain-based independence test | Offline | 2003 |
| Circa | Regression-based hypothesis testing | Offline | 2011 |
| Epsilon-Diagnosis | Two-sample test and ϵ-statistics | Both | 2007 |
| RCD | Dependency Graph inference | Both | 2002 |
| Baro | Bayesian change point detection and nonparametric hypothesis testing | Both | 2010 |
For detailed experimental results, please refer to the experiment section in our paper.