Analyzing Representations inside Convolutional Neural Networks
How can we discover and succinctly summarize the concepts that a neural network has learned? Such a task is of great importance in applications of networks in areas of inference that involve classification, like medical diagnosis based on fMRI/x-ray etc. In this work, we propose a framework to categorize the concepts a network learns based on the way it clusters a set of input examples, clusters neurons based on the examples they activate for, and input features all in the same latent space. This framework is unsupervised and can work without any labels for input features, it only needs access to internal activations of the network for each input example, thereby making it widely applicable. We extensively evaluate the proposed method and demonstrate that it produces human-understandable and coherent concepts that a ResNet-18 has learned on the CIFAR-100 dataset.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Do Deep Neural Networks for Segmentation Understand Insideness?
Image segmentation aims at grouping pixels that belong to the same object or region. At the heart of image segmentation lies the problem of determining whether a pixel is inside or outside a region, which we denote as th…
Image SegmentationSegmentationSemantic SegmentationDeep learning based supervised semantic segmentation of Electron Cryo-Subtomograms
Cellular Electron Cryo-Tomography (CECT) is a powerful imaging technique for the 3D visualization of cellular structure and organization at submolecular resolution. It enables analyzing the native structures of macromole…
DecoderDeep LearningSegmentationSemantic SegmentationOpening the Black Box of wav2vec Feature Encoder
Self-supervised models, namely, wav2vec and its variants, have shown promising results in various downstream tasks in the speech domain. However, their inner workings are poorly understood, calling for in-depth analyses …
The representation landscape of few-shot learning and fine-tuning in large language models
In-context learning (ICL) and supervised fine-tuning (SFT) are two common strategies for improving the performance of modern large language models (LLMs) on specific tasks. Despite their different natures, these strategi…
Few-Shot LearningIn-Context LearningQuestion AnsweringOn the Universal Statistical Consistency of Expansive Hyperbolic Deep Convolutional Neural Networks
The emergence of Deep Convolutional Neural Networks (DCNNs) has been a pervasive tool for accomplishing widespread applications in computer vision. Despite its potential capability to capture intricate patterns inside th…