Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement
The behavior of deep neural networks (DNNs) is hard to understand. This makes it necessary to explore post hoc explanation methods. We conduct the first comprehensive evaluation of explanation methods for NLP. To this end, we design two novel evaluation paradigms that cover two important classes of NLP problems: small context and large context problems. Both paradigms require no manual annotation and are therefore broadly applicable. We also introduce LIMSSE, an explanation method inspired by LIME that is designed for NLP. We show empirically that LIMSSE, LRP and DeepLIFT are the most effective explanation methods and recommend them for explaining DNNs in NLP.
Code (1)
Tasks
Sentiment AnalysisMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Evaluating neural network explanation methods using hybrid documents and morphological agreement
The behavior of deep neural networks (DNNs) is hard to understand. This makes it necessary to explore post hoc explanation methods. We conduct the first comprehensive evaluation of explanation methods for NLP. To this en…
Evaluating the Morphosyntactic Well-formedness of Generated Texts
Text generation systems are ubiquitous in natural language processing applications. However, evaluation of these systems remains a challenge, especially in multilingual settings. In this paper, we propose L'AMBRE -- a me…
Machine TranslationText GenerationTranslationWhy Did the Robot Cross the Road? A User Study of Explanation in Human-Robot Interaction
This work documents a pilot user study evaluating the effectiveness of contrastive, causal and example explanations in supporting human understanding of AI in a hypothetical commonplace human robot interaction HRI scenar…
Explainable Artificial Intelligence (XAI)A Survey on Aspect-Based Sentiment Classification
With the constantly growing number of reviews and other sentiment-bearing texts on the Web, the demand for automatic sentiment analysis algorithms continues to expand. Aspect-based sentiment classification (ABSC) allows …
ClassificationSentiment AnalysisSentiment ClassificationSurveyCombining Ontologies and Neural Networks for Analyzing Historical Language Varieties. A Case Study in Middle Low German
In this paper, we describe experiments on the morphosyntactic annotation of historical language varieties for the example of Middle Low German (MLG), the official language of the German Hanse during the Middle Ages and a…
POS