paper-with-me

Papers

LatentExplainer: Explaining Latent Representations in Deep Generative Models with Multimodal Large Language Models

2024-06-21 · Mengdan Zhu, Raasikh Kanjiani, Jiahui Lu, Andrew Choi, Qirui Ye, Liang Zhao

Deep generative models like VAEs and diffusion models have advanced various generation tasks by leveraging latent variables to learn data distributions and generate high-quality samples. Despite the field of explainable AI making strides in interpreting machine learning models, understanding latent variables in generative models remains challenging. This paper introduces \textit{LatentExplainer}, a framework for automatically generating semantically meaningful explanations of latent variables in deep generative models. \textit{LatentExplainer} tackles three main challenges: inferring the meaning of latent variables, aligning explanations with inductive biases, and handling varying degrees of explainability. Our approach perturbs latent variables, interpreting changes in generated data, and uses multi-modal large language models (MLLMs) to produce human-understandable explanations. We evaluate our proposed method on several real-world and synthetic datasets, and the results demonstrate superior performance in generating high-quality explanations for latent variables. The results highlight the effectiveness of incorporating inductive biases and uncertainty quantification, significantly enhancing model interpretability.

📄 PDF Abstract BibTeX arXiv:2406.14862

Code (1)

mengdanzhu/latentexplainer 공식 구현 pytorch

Tasks

Uncertainty Quantification

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Similar Papers 제목 키워드 기반

Explaining latent representations of generative models with large multimodal models

2024-02-02 · Mengdan Zhu, Zhenke Liu, Bo Pan, Abhinav Angirekula 외

Learning interpretable representations of data generative latent factors is an important topic for the development of artificial intelligence. With the rise of the large multimodal model, it can align images with text to…

DisentanglementExplanation Generation

Latent Graph Learning in Generative Models of Neural Signals

2025-08-22 · Nathan X. Kodama, Kenneth A. Loparo arxiv

Inferring temporal interaction graphs and higher-order structure from neural signals is a key problem in building generative models for systems neuroscience. Foundation models for large-scale neural data represent shared…

Graph Learning

Disentangling shared and private latent factors in multimodal Variational Autoencoders

2024-03-10 · Kaspar Märtens, Christopher Yau

Generative models for multimodal data permit the identification of latent factors that may be associated with important determinants of observed data heterogeneity. Common or shared factors could be important for explain…

Disentanglement

CoLiDR: Concept Learning using Aggregated Disentangled Representations

2024-07-27 · Sanchit Sinha, Guangzhi Xiong, Aidong Zhang

Interpretability of Deep Neural Networks using concept-based models offers a promising way to explain model behavior through human-understandable concepts. A parallel line of research focuses on disentangling the data di…

Representation Learning

Uncertainty in latent representations of variational autoencoders optimized for visual tasks

2024-04-23 · Josefina Catoni, Domonkos Martos, Ferenc Csikor, Enzo Ferrante 외

Deep Generative Models (DGMs) can learn flexible latent variable representations of images while avoiding intractable computations, common in Bayesian inference. However, investigating the properties of inference in Vari…

Bayesian InferenceInductive BiasInformativenessOut-of-Distribution Detection