Comparing zero-shot self-explanations with human rationales in multilingual text classification
Instruction-tuned LLMs are able to provide an explanation about their output to users by generating self-explanations that do not require gradient computations or the application of possibly complex XAI methods. In this paper, we analyse whether this ability results in a good explanation by evaluating self-explanations in the form of input rationales with respect to their plausibility to humans as well as their faithfulness to models. For this, we apply two text classification tasks: sentiment classification and forced labour detection. Next to English, we further include Danish and Italian translations of the sentiment classification task and compare self-explanations to human annotations for all samples. To allow for direct comparisons, we also compute post-hoc feature attribution, i.e., layer-wise relevance propagation (LRP) and apply this pipeline to 4 LLMs (Llama2, Llama3, Mistral and Mixtral). Our results show that self-explanations align more closely with human annotations compared to LRP, while maintaining a comparable level of faithfulness.
Code (0)
등록된 구현이 없습니다.
Tasks
Multilingual text classificationSentiment AnalysisSentiment Classificationtext-classificationText ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
ZARA: Improving Few-Shot Self-Rationalization for Small Language Models
Language models (LMs) that jointly generate end-task answers as well as free-text rationales are known as self-rationalization models. Recent works demonstrate great performance gain for self-rationalization by few-shot …
Natural Language InferenceInterpreting LLMs as Credit Risk Classifiers: Do Their Feature Explanations Align with Classical ML?
Large Language Models (LLMs) are increasingly explored as flexible alternatives to classical machine learning models for classification tasks through zero-shot prompting. However, their suitability for structured tabular…
Feature ImportanceSEZ-HARN: Self-Explainable Zero-shot Human Activity Recognition Network
Human Activity Recognition (HAR), which uses data from Inertial Measurement Unit (IMU) sensors, has many practical applications in healthcare and assisted living environments. However, its use in real-world scenarios has…
Activity RecognitionHuman Activity RecognitionZero-Shot LearningA Framework for Evaluating Zero-Shot Image Generation in Concept-based Explainability
Concept-based Explainable Artificial Intelligence (XAI) interprets deep learning models using human-understandable visual features (e.g., textures or object parts) by linking internal representations to class predictions…
Image GenerationExplain-then-Translate: An Analysis on Improving Program Translation with Self-generated Explanations
This work explores the use of self-generated natural language explanations as an intermediate step for code-to-code translation with language models. Across three types of explanations and 19 programming languages constr…
Code TranslationTranslation