Extending Information Bottleneck Attribution to Video Sequences
We introduce VIBA, a novel approach for explainable video classification by adapting Information Bottlenecks for Attribution (IBA) to video sequences. While most traditional explainability methods are designed for image models, our IBA framework addresses the need for explainability in temporal models used for video analysis. To demonstrate its effectiveness, we apply VIBA to video deepfake detection, testing it on two architectures: the Xception model for spatial features and a VGG11-based model for capturing motion dynamics through optical flow. Using a custom dataset that reflects recent deepfake generation techniques, we adapt IBA to create relevance and optical flow maps, visually highlighting manipulated regions and motion inconsistencies. Our results show that VIBA generates temporally and spatially consistent explanations, which align closely with human annotations, thus providing interpretability for video classification and particularly for deepfake detection.
Code (1)
Tasks
DeepFake DetectionFace SwappingOptical Flow EstimationVideo ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Concepts in Motion: Temporal Concept Bottleneck Model for Interpretable Video Classification
Concept Bottleneck Models (CBMs) enable interpretable image classification by structuring predictions around human-understandable concepts, but extending this paradigm to video remains challenging due to the difficulty o…
Video ClassificationImage ClassificationPCBEAR: Pose Concept Bottleneck for Explainable Action Recognition
Human action recognition (HAR) has achieved impressive results with deep learning models, but their decision-making process remains opaque due to their black-box nature. Ensuring interpretability is crucial, especially f…
Action RecognitionAction UnderstandingTemporal Action LocalizationComprehensive Information Bottleneck for Unveiling Universal Attribution to Interpret Vision Transformers
The feature attribution method reveals the contribution of input variables to the decision-making process to provide an attribution map for explanation. Existing methods grounded on the information bottleneck principle c…
Comprehensive Information Bottleneck for Unveiling Universal Attribution to Interpret Vision Transformers
The feature attribution method reveals the contribution of input variables to the decision-making process to provide an attribution map for explanation. Existing methods grounded on the information bottleneck princip…
Decision MakingRestricting the Flow: Information Bottlenecks for Attribution
Attribution methods provide insights into the decision-making of machine learning models like artificial neural networks. For a given input sample, they assign a relevance score to each individual input variable, such as…
Decision Making