AudioGenX: Explainability on Text-to-Audio Generative Models
Text-to-audio generation models (TAG) have achieved significant advances in generating audio conditioned on text descriptions. However, a critical challenge lies in the lack of transparency regarding how each textual input impacts the generated audio. To address this issue, we introduce AudioGenX, an Explainable AI (XAI) method that provides explanations for text-to-audio generation models by highlighting the importance of input tokens. AudioGenX optimizes an Explainer by leveraging factual and counterfactual objective functions to provide faithful explanations at the audio token level. This method offers a detailed and comprehensive understanding of the relationship between text inputs and audio outputs, enhancing both the explainability and trustworthiness of TAG models. Extensive experiments demonstrate the effectiveness of AudioGenX in producing faithful explanations, benchmarked against existing methods using novel evaluation metrics specifically designed for audio generation tasks.
Code (1)
Tasks
Audio GenerationcounterfactualTAGSimilar Papers 제목 키워드 기반
Explainability Paths for Sustained Artistic Practice with AI
The development of AI-driven generative audio mirrors broader AI trends, often prioritizing immediate accessibility at the expense of explainability. Consequently, integrating such tools into sustained artistic practice …
Play Me Something Icy: Practical Challenges, Explainability and the Semantic Gap in Generative AI Music
This pictorial aims to critically consider the nature of text-to-audio and text-to-music generative tools in the context of explainable AI. As a group of experimental musicians and researchers, we are enthusiastic about …
mllm-shap: A Shapley Value Explainability Platform for Text-Audio Multimodal Large Language Models
We introduce mllm-shap, an open-source Python framework designed to extend Shapley Value (SV) explainability from text-only Large Language Models to Multimodal LLMs (MLLMs) processing joint text and audio inputs. While t…
Toward Robust Real-World Audio Deepfake Detection: Closing the Explainability Gap
The rapid proliferation of AI-manipulated or generated audio deepfakes poses serious challenges to media integrity and election security. Current AI-driven detection solutions lack explainability and underperform in real…
Audio Deepfake DetectionDeepFake DetectionFace SwappingA Data-Driven Diffusion-based Approach for Audio Deepfake Explanations
Evaluating explainability techniques, such as SHAP and LRP, in the context of audio deepfake detection is challenging due to lack of clear ground truth annotations. In the cases when we are able to obtain the ground trut…
Audio Deepfake DetectionDeepFake DetectionFace Swapping