Evaluating Explanation Methods for Multivariate Time Series Classification
Multivariate time series classification is an important computational task arising in applications where data is recorded over time and over multiple channels. For example, a smartwatch can record the acceleration and orientation of a person's motion, and these signals are recorded as multivariate time series. We can classify this data to understand and predict human movement and various properties such as fitness levels. In many applications classification alone is not enough, we often need to classify but also understand what the model learns (e.g., why was a prediction given, based on what information in the data). The main focus of this paper is on analysing and evaluating explanation methods tailored to Multivariate Time Series Classification (MTSC). We focus on saliency-based explanation methods that can point out the most relevant channels and time series points for the classification decision. We analyse two popular and accurate multivariate time series classifiers, ROCKET and dResNet, as well as two popular explanation methods, SHAP and dCAM. We study these methods on 3 synthetic datasets and 2 real-world datasets and provide a quantitative and qualitative analysis of the explanations provided. We find that flattening the multivariate datasets by concatenating the channels works as well as using multivariate classifiers directly and adaptations of SHAP for MTSC work quite well. Additionally, we also find that the popular synthetic datasets we used are not suitable for time series analysis.
Code (1)
Tasks
ClassificationTime SeriesTime Series AnalysisTime Series ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Counterfactual Explanation for Multivariate Time Series Forecasting with Exogenous Variables
Currently, machine learning is widely used across various domains, including time series data analysis. However, some machine learning models function as black boxes, making interpretability a critical concern. One appro…
Multivariate Time Series ForecastingXTSC-Bench: Quantitative Benchmarking for Explainers on Time Series Classification
Despite the growing body of work on explainable machine learning in time series classification (TSC), it remains unclear how to evaluate different explainability methods. Resorting to qualitative assessment and user stud…
BenchmarkingTime SeriesTime Series ClassificationM-CELS: Counterfactual Explanation for Multivariate Time Series Data Guided by Learned Saliency Maps
Over the past decade, multivariate time series classification has received great attention. Machine learning (ML) models for multivariate time series classification have made significant strides and achieved impressive s…
ClassificationcounterfactualCounterfactual ExplanationTime Series+1Multi-SpaCE: Multi-Objective Subsequence-based Sparse Counterfactual Explanations for Multivariate Time Series Classification
Deep Learning systems excel in complex tasks but often lack transparency, limiting their use in critical applications. Counterfactual explanations, a core tool within eXplainable Artificial Intelligence (XAI), offer insi…
counterfactualCounterfactual ExplanationExplainable artificial intelligenceExplainable Artificial Intelligence (XAI)+2Improving the Evaluation and Actionability of Explanation Methods for Multivariate Time Series Classification
Explanation for Multivariate Time Series Classification (MTSC) is an important topic that is under explored. There are very few quantitative evaluation methodologies and even fewer examples of actionable explanation, whe…
channel selectionTime SeriesTime Series Classification