Deephys: Deep Electrophysiology, Debugging Neural Networks under Distribution Shifts
Deep Neural Networks (DNNs) often fail in out-of-distribution scenarios. In this paper, we introduce a tool to visualize and understand such failures. We draw inspiration from concepts from neural electrophysiology, which are based on inspecting the internal functioning of a neural networks by analyzing the feature tuning and invariances of individual units. Deep Electrophysiology, in short Deephys, provides insights of the DNN's failures in out-of-distribution scenarios by comparative visualization of the neural activity in in-distribution and out-of-distribution datasets. Deephys provides seamless analyses of individual neurons, individual images, and a set of set of images from a category, and it is capable of revealing failures due to the presence of spurious features and novel features. We substantiate the validity of the qualitative visualizations of Deephys thorough quantitative analyses using convolutional and transformers architectures, in several datasets and distribution shifts (namely, colored MNIST, CIFAR-10 and ImageNet).
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
The Debugging Decay Index: Rethinking Debugging Strategies for Code LLMs
The effectiveness of AI debugging follows a predictable exponential decay pattern; most models lose 60-80% of their debugging capability within just 2-3 attempts, despite iterative debugging being a critical capability f…
Code GenerationUnderstanding and Testing Generalization of Deep Networks on Out-of-Distribution Data
Deep network models perform excellently on In-Distribution (ID) data, but can significantly fail on Out-Of-Distribution (OOD) data. While developing methods focus on improving OOD generalization, few attention has been p…
Unlocking datasets by calibrating populations of models to data density: a study in atrial electrophysiology
The understanding of complex physical or biological systems nearly always requires a characterisation of the variability that underpins these processes. In addition, the data used to calibrate such models may also often …
RhythmImproving data sharing and knowledge transfer via the Neuroelectrophysiology Analysis Ontology (NEAO)
Describing the processes involved in analyzing data from electrophysiology experiments to investigate the function of neural systems is inherently challenging. On the one hand, data can be analyzed by distinct methods th…
Knowledge GraphsTransfer LearningA unified framework for geometry-independent operator learning in cardiac electrophysiology simulations
Learning neural operators on heterogeneous and irregular geometries remains a fundamental challenge, as existing approaches typically rely on structured discretisations or explicit mappings to a shared reference domain. …