E-VarM: Enhanced Variational Word Masks to Improve the Interpretability of Text Classification Models
Enhancing the interpretability of text classification models can help increase the reliability of these models in real-world applications. Currently, most researchers focus on extracting task-specific words from inputs to improve the interpretability of the model. The competitive approaches exploit the Variational Information Bottleneck (VIB) to improve the performance of word masking at the word embedding layer to obtain task-specific words. However, these approaches ignore the multi-level semantics of the text, which can impair the interpretability of the model, and do not consider the risk of representation overlap caused by the VIB, which can impair the classification performance. In this paper, we propose an enhanced variational word masks approach, named E-VarM, to solve these two issues effectively. The E-VarM combines multi-level semantics from all hidden layers of the model to mask out task-irrelevant words and uses contrastive learning to readjust the distances between representations. Empirical studies on ten benchmark text classification datasets demonstrate that our approach outperforms the SOTA methods in simultaneously improving the interpretability and accuracy of the model.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationContrastive Learningtext-classificationText ClassificationSimilar Papers 제목 키워드 기반
Variationally Mimetic Operator Networks
In recent years operator networks have emerged as promising deep learning tools for approximating the solution to partial differential equations (PDEs). These networks map input functions that describe material propertie…
VARMA-Enhanced Transformer for Time Series Forecasting
Transformer-based models have significantly advanced time series forecasting. Recent work, like the Cross-Attention-only Time Series transformer (CATS), shows that removing self-attention can make the model more accurate…
Time Series ForecastingTime Series AnalysisVarMAE: Pre-training of Variational Masked Autoencoder for Domain-adaptive Language Understanding
Pre-trained language models have achieved promising performance on general benchmarks, but underperform when migrated to a specific domain. Recent works perform pre-training from scratch or continual pre-training on doma…
Citation Intent ClassificationLanguage ModelingLanguage ModellingVariational Structured Semantic Inference for Diverse Image Captioning
Despite the exciting progress in image captioning, generating diverse captions for a given image remains as an open problem. Existing methods typically apply generative models such as Variational Auto-Encoder to diversif…
DecoderDiversityImage CaptioningAn optimal Petrov-Galerkin framework for operator networks
The optimal Petrov-Galerkin formulation to solve partial differential equations (PDEs) recovers the best approximation in a specified finite-dimensional (trial) space with respect to a suitable norm. However, the recover…