A BERT-based Universal Model for Both Within- and Cross-sentence Clinical Temporal Relation Extraction
Classic methods for clinical temporal relation extraction focus on relational candidates within a sentence. On the other hand, break-through Bidirectional Encoder Representations from Transformers (BERT) are trained on large quantities of arbitrary spans of contiguous text instead of sentences. In this study, we aim to build a sentence-agnostic framework for the task of CONTAINS temporal relation extraction. We establish a new state-of-the-art result for the task, 0.684F for in-domain (0.055-point improvement) and 0.565F for cross-domain (0.018-point improvement), by fine-tuning BERT and pre-training domain-specific BERT models on sentence-agnostic temporal relation instances with WordPiece-compatible encodings, and augmenting the labeled data with automatically generated {``}silver{''} instances.
Code (0)
등록된 구현이 없습니다.
Tasks
RelationRelation ExtractionSentenceTemporal Relation ExtractionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders
Pretrained Masked Language Models (MLMs) have revolutionised NLP in recent years. However, previous work has indicated that off-the-shelf MLMs are not effective as universal lexical or sentence encoders without further t…
Contrastive LearningCross-Lingual Semantic Textual SimilarityEntity LinkingSemantic Similarity+4BURT: BERT-inspired Universal Representation from Twin Structure
Pre-trained contextualized language models such as BERT have shown great effectiveness in a wide range of downstream Natural Language Processing (NLP) tasks. However, the effective representations offered by the models t…
Natural Language InferenceSentenceSTStext similarity+1On Learning Universal Representations Across Languages
Recent studies have demonstrated the overwhelming advantage of cross-lingual pre-trained models (PTMs), such as multilingual BERT and XLM, on cross-lingual NLP tasks. However, existing approaches essentially capture the …
Contrastive LearningCross-Lingual Natural Language InferenceLanguage ModelingLanguage Modelling+5Towards Non-task-specific Distillation of BERT via Sentence Representation Approximation
Recently, BERT has become an essential ingredient of various NLP deep models due to its effectiveness and universal-usability. However, the online deployment of BERT is often blocked by its large-scale parameters and hig…
Knowledge DistillationSentenceTransfer LearningModeling Sequential Sentence Relation to Improve Cross-lingual Dense Retrieval
Recently multi-lingual pre-trained language models (PLM) such as mBERT and XLM-R have achieved impressive strides in cross-lingual dense retrieval. Despite its successes, they are general-purpose PLM while the multilingu…
RelationRepresentation LearningRetrievalSentence+1