LET: Linguistic Knowledge Enhanced Graph Transformer for Chinese Short Text Matching
Chinese short text matching is a fundamental task in natural language processing. Existing approaches usually take Chinese characters or words as input tokens. They have two limitations: 1) Some Chinese words are polysemous, and semantic information is not fully utilized. 2) Some models suffer potential issues caused by word segmentation. Here we introduce HowNet as an external knowledge base and propose a Linguistic knowledge Enhanced graph Transformer (LET) to deal with word ambiguity. Additionally, we adopt the word lattice graph as input to maintain multi-granularity information. Our model is also complementary to pre-trained language models. Experimental results on two Chinese datasets show that our models outperform various typical text matching approaches. Ablation study also indicates that both semantic information and multi-granularity information are important for text matching modeling.
Code (1)
Tasks
Text MatchingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Revisiting and Advancing Chinese Natural Language Understanding with Accelerated Heterogeneous Knowledge Pre-training
Recently, knowledge-enhanced pre-trained language models (KEPLMs) improve context-aware representations via learning from structured relations in knowledge graphs, and/or linguistic knowledge from syntactic or dependency…
GPUKnowledge GraphsLanguage ModelingLanguage Modelling+2Enhancing Chinese Pre-trained Language Model via Heterogeneous Linguistics Graph
Chinese pre-trained language models usually exploit contextual character information to learn representations, while ignoring the linguistics knowledge, e.g., word and sentence information. Hence, we propose a task-free …
Language ModelingLanguage ModellingSentencePredicting Genetic Mutation from Whole Slide Images via Biomedical-Linguistic Knowledge Enhanced Multi-label Classification
Predicting genetic mutations from whole slide images is indispensable for cancer diagnosis. However, existing work training multiple binary classification models faces two challenges: (a) Training multiple binary classif…
Binary ClassificationGraph Representation LearningMulti-Label ClassificationMUlTI-LABEL-ClASSIFICATION+2Linguistic Knowledge in Data Augmentation for Natural Language Processing: An Example on Chinese Question Matching
To investigate the role of linguistic knowledge in data augmentation (DA) for Natural Language Processing (NLP), we designed two adapted DA programs and applied them to LCQMC (a Large-scale Chinese Question Matching Corp…
Data AugmentationLanguage ModellingQuestion SimilaritySemantic Text Matching+1Enhancing Transformer with Sememe Knowledge
While large-scale pretraining has achieved great success in many NLP tasks, it has not been fully studied whether external linguistic knowledge can improve data-driven models. In this work, we introduce sememe knowledge …
Language ModelingLanguage Modelling