Language-Guided Trajectory Traversal in Disentangled Stable Diffusion Latent Space for Factorized Medical Image Generation
Text-to-image diffusion models have demonstrated a remarkable ability to generate photorealistic images from natural language prompts. These high-resolution, language-guided synthesized images are essential for the explainability of disease or exploring causal relationships. However, their potential for disentangling and controlling latent factors of variation in specialized domains like medical imaging remains under-explored. In this work, we present the first investigation of the power of pre-trained vision-language foundation models, once fine-tuned on medical image datasets, to perform latent disentanglement for factorized medical image generation and interpolation. Through extensive experiments on chest X-ray and skin datasets, we illustrate that fine-tuned, language-guided Stable Diffusion inherently learns to factorize key attributes for image generation, such as the patient's anatomical structures or disease diagnostic features. We devise a framework to identify, isolate, and manipulate key attributes through latent space trajectory traversal of generative models, facilitating precise control over medical image synthesis.
Code (0)
등록된 구현이 없습니다.
Tasks
DiagnosticDisentanglementImage GenerationMedical Image GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Reasonable Motion: A General ASP Foundation for Environment Constrained Movement Trajectory Computation
We present a general answer set programming based hybrid quantitative-qualitative method for computing constrained branching trajectory modes for moving objects in real-world settings. The method performs constrained tra…
Autonomous DrivingInfoGAN-CR: Disentangling Generative Adversarial Networks with Contrastive Regularizers
Standard deep generative models have latent codes that can be arbitrarily rotated, and a specific coordinate has no meaning. For manipulation and exploration of the samples, we seek a disentangled latent code where each …
DisentanglementModel SelectionLatent Traversals in Generative Models as Potential Flows
Despite the significant recent progress in deep generative models, the underlying structure of their latent spaces is still poorly understood, thereby making the task of performing semantically meaningful latent traversa…
DisentanglementInductive BiasVidStyleODE: Disentangled Video Editing via StyleGAN and NeuralODEs
We propose $\textbf{VidStyleODE}$, a spatiotemporally continuous disentangled $\textbf{Vid}$eo representation based upon $\textbf{Style}$GAN and Neural-$\textbf{ODE}$s. Effective traversal of the latent space learned by …
Image AnimationVideo EditingVideo GenerationLearning disentangled representations for explainable chest X-ray classification using Dirichlet VAEs
This study explores the use of the Dirichlet Variational Autoencoder (DirVAE) for learning disentangled latent representations of chest X-ray (CXR) images. Our working hypothesis is that distributional sparsity, as facil…
ClassificationMulti-Label ClassificationMUlTI-LABEL-ClASSIFICATIONX-ray Classification