Papers Dictionary Learning
“Dictionary Learning” 태그가 달린 논문 823편 · 필터 해제
Joint space-time wind field data extrapolation and uncertainty quantification using nonparametric Bayesian dictionary learning
A methodology is developed, based on nonparametric Bayesian dictionary learning, for joint space-time wind field data extrapolation and estimation of related statistics by relying on limited/incomplete measurements. Spec…
Dictionary LearningUncertainty QuantificationDecomposing MLP Activations into Interpretable Features via Semi-Nonnegative Matrix Factorization
A central goal for mechanistic interpretability has been to identify the right units of analysis in large language models (LLMs) that causally explain their outputs. While early work focused on individual neurons, eviden…
Dictionary LearningEvaluating Sparse Autoencoders: From Shallow Design to Matching Pursuit
Sparse autoencoders (SAEs) have recently become central tools for interpretability, leveraging dictionary learning principles to extract sparse, interpretable features from neural representations whose underlying structu…
Dictionary LearningMechanistic Decomposition of Sentence Representations
Sentence embeddings are central to modern NLP and AI systems, yet little is known about their internal structure. While we can compare these embeddings using measures such as cosine similarity, the contributing features …
Dictionary LearningSentenceSentence EmbeddingSentence-Embedding+1HyperSteer: Activation Steering at Scale with Hypernetworks
Steering language models (LMs) by modifying internal activations is a popular approach for controlling text generation. Unsupervised dictionary learning methods, e.g., sparse autoencoders, can be scaled to produce many s…
Dictionary LearningText GenerationInterpreting Large Text-to-Image Diffusion Models with Dictionary Learning
Sparse autoencoders are a promising new approach for decomposing language model activations for interpretation and control. They have been applied successfully to vision transformer image encoders and to small-scale diff…
Dictionary LearningImage GenerationLanguage ModelingLanguage ModellingTowards Privacy-Preserving Fine-Grained Visual Classification via Hierarchical Learning from Label Proportions
In recent years, Fine-Grained Visual Classification (FGVC) has achieved impressive recognition accuracy, despite minimal inter-class variations. However, existing methods heavily rely on instance-level labels, making the…
ClassificationDictionary LearningFine-Grained Image ClassificationMedical Image Analysis+1DB-KSVD: Scalable Alternating Optimization for Disentangling High-Dimensional Embedding Spaces
Dictionary learning has recently emerged as a promising approach for mechanistic interpretability of large transformer models. Disentangling high-dimensional transformer embeddings, however, requires algorithms that scal…
Dictionary LearningModeling Musical Genre Trajectories through Pathlet Learning
The increasing availability of user data on music streaming platforms opens up new possibilities for analyzing music consumption. However, understanding the evolution of user preferences remains a complex challenge, part…
Dictionary LearningDiversityRecommendation SystemsTowards Understanding the Nature of Attention with Low-Rank Sparse Decomposition
We propose Low-Rank Sparse Attention (Lorsa), a sparse replacement model of Transformer attention layers to disentangle original Multi Head Self Attention (MHSA) into individually comprehensible components. Lorsa is desi…
Dictionary LearningFrom Attention to Atoms: Spectral Dictionary Learning for Fast, Interpretable Language Models
We propose a novel spectral generative modeling framework for natural language processing that jointly learns a global time varying Fourier dictionary and per token mixing coefficients, replacing the ubiquitous self atte…
Dictionary LearningLanguage ModelingLanguage ModellingOptimizing Hard Thresholding for Sparse Model Discovery
Many model selection algorithms rely on sparse dictionary learning to provide interpretable and physics-based governing equations. The optimization algorithms typically use a hard thresholding process to enforce sparse a…
Dictionary LearningmodelModel DiscoveryModel Selection+1Unveiling Hidden Collaboration within Mixture-of-Experts in Large Language Models
Mixture-of-Experts based large language models (MoE LLMs) have shown significant promise in multitask adaptability by dynamically routing inputs to specialized experts. Despite their success, the collaborative mechanisms…
Dictionary LearningMixture-of-ExpertsModel OptimizationRecognition of Geometrical Shapes by Dictionary Learning
Dictionary learning is a versatile method to produce an overcomplete set of vectors, called atoms, to represent a given input with only a few atoms. In the literature, it has been used primarily for tasks that explore it…
Dictionary LearningImage ReconstructionMixture-of-Shape-Experts (MoSE): End-to-End Shape Dictionary Framework to Prompt SAM for Generalizable Medical Segmentation
Single domain generalization (SDG) has recently attracted growing attention in medical image segmentation. One promising strategy for SDG is to leverage consistent semantic shape priors across different imaging protocols…
Dictionary LearningDomain GeneralizationImage SegmentationMedical Image Segmentation+2Koopman-Based Methods for EV Climate Dynamics: Comparing eDMD Approaches
In this paper, data-driven algorithms based on Koopman Operator Theory are applied to identify and predict the nonlinear dynamics of a vapor compression system and cabin temperature in a light-duty electric vehicle. By l…
Dictionary LearningManagementDecentralized Federated Dataset Dictionary Learning for Multi-Source Domain Adaptation
Decentralized Multi-Source Domain Adaptation (DMSDA) is a challenging task that aims to transfer knowledge from multiple related and heterogeneous source domains to an unlabeled target domain within a decentralized frame…
Dictionary LearningDomain AdaptationTopological Dictionary Learning
The aim of this paper is to introduce a novel dictionary learning algorithm for sparse representation of signals defined over combinatorial topological spaces, specifically, regular cell complexes. Leveraging Hodge theor…
Dictionary LearningSparse Dictionary Learning for Image Recovery by Iterative Shrinkage
In this paper we study the sparse coding problem in the context of sparse dictionary learning for image recovery. To this end, we consider and compare several state-of-the-art sparse optimization methods constructed usin…
Computational EfficiencyDenoisingDictionary LearningOnline multidimensional dictionary learning
Dictionary learning is a widely used technique in signal processing and machine learning that aims to represent data as a linear combination of a few elements from an overcomplete dictionary. In this work, we propose a g…
Dictionary Learning