Targeted Visualization of the Backbone of Encoder LLMs

2024-03-26 · Isaac Roberts, Alexander Schulz, Luca Hermes, Barbara Hammer

Attention based Large Language Models (LLMs) are the state-of-the-art in natural language processing (NLP). The two most common architectures are encoders such as BERT, and decoders like the GPT models. Despite the success of encoder models, on which we focus in this work, they also bear several risks, including issues with bias or their susceptibility for adversarial attacks, signifying the necessity for explainable AI to detect such issues. While there does exist various local explainability methods focusing on the prediction of single inputs, global methods based on dimensionality reduction for classification inspection, which have emerged in other domains and that go further than just using t-SNE in the embedding space, are not widely spread in NLP. To reduce this gap, we investigate the application of DeepView, a method for visualizing a part of the decision function together with a data set in two dimensions, to the NLP domain. While in previous work, DeepView has been used to inspect deep image classification models, we demonstrate how to apply it to BERT-based NLP classifiers and investigate its usability in this domain, including settings with adversarially perturbed input samples and pre-trained, fine-tuned, and multi-task models.

📄 PDF Abstract BibTeX arXiv:2403.18872

Code (1)

LucaHermes/DeepView 공식 구현 pytorch

Tasks

Dimensionality Reduction image-classification Image Classification

Methods 이 논문이 사용한 방법론

Refunds@Expedia|||How do I get a full refund from Expedia? “How do I get a full refund from Expedia? How do I get a full refund from Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Quick Help &…

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Cosine Annealing Cosine Annealing is a type of learning rate schedule that has the effect of starting with a large learning rate that is relatively rapidly decreased to a minimum value before…

Linear Warmup With Cosine Annealing Linear Warmup With Cosine Annealing is a learning rate schedule where we increase the learning rate linearly for $n$ updates and then anneal according to a cosine schedule…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Discriminative Fine-Tuning Discriminative Fine-Tuning is a fine-tuning strategy that is used for ULMFiT type models. Instead of using the same learning rate…

RL makes MLLMs see better than SFT

2025-10-18 · Junha Song, Sangdoo Yun, Dongyoon Han, Jaegul Choo 외 arxiv

A dominant assumption in Multimodal Language Model (MLLM) research is that its performance is largely inherited from the LLM backbone, given its immense parameter scale and remarkable capabilities. This has created a voi…

Reinforcement Learning

Textual Steering Vectors Can Improve Visual Understanding in Multimodal Large Language Models

2025-05-20 · Woody Haosheng Gan, Deqing Fu, Julian Asilis, Ollie Liu 외

Steering methods have emerged as effective and targeted tools for guiding large language models' (LLMs) behavior without modifying their parameters. Multimodal large language models (MLLMs), however, do not currently enj…

Diversity

Visual Exploration of Feature Relationships in Sparse Autoencoders with Curated Concepts

2025-11-08 · Xinyuan Yan, Shusen Liu, Kowshik Thopalli, Bei Wang arxiv

Sparse autoencoders (SAEs) have emerged as a powerful tool for uncovering interpretable features in large language models (LLMs) through the sparse directions they learn. However, the sheer number of extracted directions…

Dimensionality Reduction

Feature Visualization Recovers Known Cortical Selectivity from TRIBE v2

2026-05-13 · Stuart Bladon, Brinnae Bent arxiv

Brain encoder models predict cortical fMRI responses from the internal activations of pretrained vision and language networks, and are typically evaluated by held-out prediction accuracy. This is a useful signal for trai…

Projector Is All You Train

2026-08-20 · Nyx Iskandar, Saathvik Selvan, Slater Victoroff arxiv

The typical training process of a multimodal large language model (MLLM) involves adapting both the language model backbone and the projector between the backbone and a modality-specific encoder. We ask whether fine-tuni…

Spatial Reasoning 3D Classification