On the Performance of Concept Probing: The Influence of the Data (Extended Version)
Concept probing has recently garnered increasing interest as a way to help interpret artificial neural networks, dealing both with their typically large size and their subsymbolic nature, which ultimately renders them unfeasible for direct human interpretation. Concept probing works by training additional classifiers to map the internal representations of a model into human-defined concepts of interest, thus allowing humans to peek inside artificial neural networks. Research on concept probing has mainly focused on the model being probed or the probing model itself, paying limited attention to the data required to train such probing models. In this paper, we address this gap. Focusing on concept probing in the context of image classification tasks, we investigate the effect of the data used to train probing models on their performance. We also make available concept labels for two widely used datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Image ClassificationSimilar Papers 제목 키워드 기반
Concept Probing: Where to Find Human-Defined Concepts (Extended Version)
Concept probing has recently gained popularity as a way for humans to peek into what is encoded within artificial neural networks. In concept probing, additional classifiers are trained to map the internal representation…
Channel Performance Estimations with Extended Channel Probing
We test the concept of extended channel probing in an Optical Spectrum as a Service scenario in coherent optimized flex-grid long-haul and 10Gbit/s OOK optimized 100-GHz fixed-grid dispersion-managed legacy DWDM networks…
In-Context Probing Approximates Influence Function for Data Valuation
Data valuation quantifies the value of training data, and is used for data attribution (i.e., determining the contribution of training data towards model predictions), and data selection; both of which are important for …
Data ValuationBeyond Single Concept Vector: Modeling Concept Subspace in LLMs with Gaussian Distribution
Probing learned concepts in large language models (LLMs) is crucial for understanding how semantic knowledge is encoded internally. Training linear classifiers on probing tasks is a principle approach to denote the vecto…
Text GenerationWhat are They Thinking? Delineation, Probing, and Tracking of Concepts in LLMs
As the influence of LLMs expands, it is imperative to gain insight into their decisions. One way to do that is to develop probes that detect the presence or absence of a broad set of high-level abstract concepts within t…