Enhancing Interpretability in Generative AI Through Search-Based Data Influence Analysis
Generative AI models offer powerful capabilities but often lack transparency, making it difficult to interpret their output. This is critical in cases involving artistic or copyrighted content. This work introduces a search-inspired approach to improve the interpretability of these models by analysing the influence of training data on their outputs. Our method provides observational interpretability by focusing on a model's output rather than on its internal state. We consider both raw data and latent-space embeddings when searching for the influence of data items in generated content. We evaluate our method by retraining models locally and by demonstrating the method's ability to uncover influential subsets in the training data. This work lays the groundwork for future extensions, including user-based evaluations with domain experts, which is expected to improve observational interpretability further.
Code (1)
Similar Papers 제목 키워드 기반
Generative Retrieval with Preference Optimization for E-commerce Search
Generative retrieval introduces a groundbreaking paradigm to document retrieval by directly generating the identifier of a pertinent document in response to a specific query. This paradigm has demonstrated considerable b…
RetrievalFlow Battery Manifold Design with Heterogeneous Inputs Through Generative Adversarial Neural Networks
Generative machine learning has emerged as a powerful tool for design representation and exploration. However, its application is often constrained by the need for large datasets of existing designs and the lack of inter…
XAI meets LLMs: A Survey of the Relation between Explainable AI and Large Language Models
In this survey, we address the key challenges in Large Language Models (LLM) research, focusing on the importance of interpretability. Driven by increasing interest from AI and business sectors, we highlight the need for…
Explainable artificial intelligenceExplainable Artificial Intelligence (XAI)RelationSurveyTowards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements
As generative large model capabilities advance, safety concerns become more pronounced in their outputs. To ensure the sustainable growth of the AI ecosystem, it's imperative to undertake a holistic evaluation and refine…
Adversarial AttackEthicsSurveySHE: Stepwise Hybrid Examination Reinforcement Learning Framework for E-commerce Search Relevance
Query-product relevance prediction is vital for AI-driven e-commerce, yet current LLM-based approaches face a dilemma: SFT and DPO struggle with long-tail generalization due to coarse supervision, while traditional RLVR …
Reinforcement Learning