The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models
Supervised learning models often make systematic errors on rare subsets of the data. When these subsets correspond to explicit labels in the data (e.g., gender, race) such poor performance can be identified straightforwardly. This paper introduces a method for discovering systematic errors that do not correspond to such explicitly labelled subgroups. The key idea is that similar inputs tend to have similar representations in the final hidden layer of a neural network. We leverage this structure by "shining a spotlight" on this representation space to find contiguous regions where the model performs poorly. We show that the spotlight surfaces semantically meaningful areas of weakness in a wide variety of existing models spanning computer vision, NLP, and recommender systems.
Code (2)
Tasks
Recommendation SystemsSimilar Papers 제목 키워드 기반
Automatic Discovery and Assessment of Interpretable Systematic Errors in Semantic Segmentation
This paper presents a novel method for discovering systematic errors in segmentation models. For instance, a systematic error in the segmentation model can be a sufficiently large number of misclassifications from the mo…
Autonomous DrivingSegmentationSemantic SegmentationSpotlight: Identifying and Localizing Video Generation Errors Using VLMs
Current text-to-video models (T2V) can generate high-quality, temporally coherent, and visually realistic videos. Nonetheless, errors still often occur, and are more nuanced and local compared to the previous generation …
Video GenerationDiscovering and Validating AI Errors With Crowdsourced Failure Reports
AI systems can fail to learn important behaviors, leading to real-world issues like safety concerns and biases. Discovering these systematic failures often requires significant developer attention, from hypothesizing pot…
Identifying Unknown Unknowns in the Open World: Representations and Policies for Guided Exploration
Predictive models deployed in the real world may assign incorrect labels to instances with high confidence. Such errors or unknown unknowns are rooted in model incompleteness, and typically arise because of the mismatch …
Discovering Knowledge Deficiencies of Language Models on Massive Knowledge Base
Large language models (LLMs) possess impressive linguistic capabilities but often fail to faithfully retain factual knowledge, leading to hallucinations and unreliable outputs. Understanding LLMs' knowledge deficiencies …
Semantic SimilaritySemantic Textual SimilarityStochastic Optimization