Interactive Analysis of LLMs using Meaningful Counterfactuals
Counterfactual examples are useful for exploring the decision boundaries of machine learning models and determining feature attributions. How can we apply counterfactual-based methods to analyze and explain LLMs? We identify the following key challenges. First, the generated textual counterfactuals should be meaningful and readable to users and thus can be mentally compared to draw conclusions. Second, to make the solution scalable to long-form text, users should be equipped with tools to create batches of counterfactuals from perturbations at various granularity levels and interactively analyze the results. In this paper, we tackle the above challenges and contribute 1) a novel algorithm for generating batches of complete and meaningful textual counterfactuals by removing and replacing text segments in different granularities, and 2) LLM Analyzer, an interactive visualization tool to help users understand an LLM's behaviors by interactively inspecting and aggregating meaningful counterfactuals. We evaluate the proposed algorithm by the grammatical correctness of its generated counterfactuals using 1,000 samples from medical, legal, finance, education, and news datasets. In our experiments, 97.2% of the counterfactuals are grammatically correct. Through a use case, user studies, and feedback from experts, we demonstrate the usefulness and usability of the proposed interactive visualization tool.
Code (0)
등록된 구현이 없습니다.
Tasks
counterfactualMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
CounteRGAN: Generating Realistic Counterfactuals with Residual Generative Adversarial Nets
The prevalence of machine learning models in various industries has led to growing demands for model interpretability and for the ability to provide meaningful recourse to users. For example, patients hoping to improve t…
counterfactualSCENE: Evaluating Explainable AI Techniques Using Soft Counterfactuals
Explainable Artificial Intelligence (XAI) plays a crucial role in enhancing the transparency and accountability of AI models, particularly in natural language processing (NLP) tasks. However, popular XAI methods such as …
counterfactualExplainable artificial intelligenceExplainable Artificial Intelligence (XAI)text-classification+1Designing Counterfactual Generators using Deep Model Inversion
Explanation techniques that synthesize small, interpretable changes to a given image while producing desired changes in the model prediction have become popular for introspecting black-box models. Commonly referred to as…
counterfactualImage GenerationmodelLarge Language Models as Nondeterministic Causal Models
Recent work by Chatzi et al. and Ravfogel et al. has developed, for the first time, a method for generating counterfactuals of probabilistic Large Language Models. Such counterfactuals tell us what would - or might - hav…
Evaluating and Mitigating Bias in Image Classifiers: A Causal Perspective Using Counterfactuals
Counterfactual examples for an input -- perturbations that change specific features but not others -- have been shown to be useful for evaluating bias of machine learning models, e.g., against specific demographic groups…
BIG-bench Machine LearningcounterfactualFairnessvalid