Objective Evaluation of Deep Visual Interpretations on Time Series Data
The correct interpretation and understanding of deep learning models is essential in many applications. (Explanatory) visual interpretation approaches for image and natural language processing allow domain experts to validate and understand almost any deep learning model. However, they fall short when generalizing to arbitrary time series data that is less intuitive and more diverse. Whether a visualization explains the true reasoning or captures the real features is more difficult to judge. Hence, instead of blind trust we need an objective evaluation to obtain reliable quality metrics. This paper proposes a framework of six orthogonal quality metrics for gradient- or perturbation-based post-hoc visual interpretation methods designed for time series classification and segmentation tasks. This comprehensive set is either based on "human perception" or on "functional properties". An extensive experimental study includes commonly used neural network architectures for time series and nine visual interpretation methods. We evaluate the visual interpretation methods with diverse datasets from the UCR repository as well another complex real-world dataset. We show that none of the existing methods consistently outperforms any of the others on all metrics while some of them are ahead in either functional or human-based metrics. Our results allow experts to make an informed choice of suitable visualization techniques for the model and task at hand.
Code (0)
등록된 구현이 없습니다.
Tasks
Time SeriesTime Series AnalysisTime Series ClassificationSimilar Papers 제목 키워드 기반
Don't Get Me Wrong: How to Apply Deep Visual Interpretations to Time Series
The correct interpretation and understanding of deep learning models are essential in many applications. Explanatory visual interpretation approaches for image, and natural language processing allow domain experts to val…
Time SeriesTime Series AnalysisTime Series ClassificationvalidST-Tree with Interpretability for Multivariate Time Series Classification
Multivariate time series classification is of great importance in practical applications and is a challenging task. However, deep neural network models such as Transformers exhibit high accuracy in multivariate time seri…
ClassificationDecision MakingRepresentation LearningTime Series+1m-TSNE: A Framework for Visualizing High-Dimensional Multivariate Time Series
Multivariate time series (MTS) have become increasingly common in healthcare domains where human vital signs and laboratory results are collected for predictive diagnosis. Recently, there have been increasing efforts to …
Time SeriesTime Series AnalysisVocal Bursts Intensity PredictionPosition: Empowering Time Series Reasoning with Multimodal LLMs
Understanding time series data is crucial for multiple real-world applications. While large language models (LLMs) show promise in time series tasks, current approaches often rely on numerical data alone, overlooking the…
Decision MakingMultimodal ReasoningPositionTime Series+1How Useful Are the Machine-Generated Interpretations to General Users? A Human Evaluation on Guessing the Incorrectly Predicted Labels
Explaining to users why automated systems make certain mistakes is important and challenging. Researchers have proposed ways to automatically produce interpretations for deep neural network models. However, it is unclear…