DSReg: Using Distant Supervision as a Regularizer
In this paper, we aim at tackling a general issue in NLP tasks where some of the negative examples are highly similar to the positive examples, i.e., hard-negative examples. We propose the distant supervision as a regularizer (DSReg) approach to tackle this issue. The original task is converted to a multi-task learning problem, in which distant supervision is used to retrieve hard-negative examples. The obtained hard-negative examples are then used as a regularizer. The original target objective of distinguishing positive examples from negative examples is jointly optimized with the auxiliary task objective of distinguishing softened positive (i.e., hard-negative examples plus positive examples) from easy-negative examples. In the neural context, this can be done by outputting the same representation from the last neural layer to different $softmax$ functions. Using this strategy, we can improve the performance of baseline models in a range of different NLP tasks, including text classification, sequence labeling and reading comprehension.
Code (0)
등록된 구현이 없습니다.
Tasks
Multi-Task LearningReading Comprehensiontext-classificationText ClassificationSimilar Papers 제목 키워드 기반
Joint Representation Learning of Cross-lingual Words and Entities via Attentive Distant Supervision
Joint representation learning of words and entities benefits many NLP tasks, but has not been well explored in cross-lingual settings. In this paper, we propose a novel method for joint representation learning of cross-l…
Cross-Lingual Entity LinkingEntity LinkingRepresentation LearningTranslation+1De-biasing Distantly Supervised Named Entity Recognition via Causal Intervention
Distant supervision tackles the data bottleneck in NER by automatically generating training instances via dictionary matching. Unfortunately, the learning of DS-NER is severely dictionary-biased, which suffers from spuri…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NERDual Supervision Framework for Relation Extraction with Distant Supervision and Human Annotation
Relation extraction (RE) has been extensively studied due to its importance in real-world applications such as knowledge base construction and question answering. Most of the existing works train the models on either dis…
Knowledge Base ConstructionRelationRelation ExtractionSentenceImproving Distant 3D Object Detection Using 2D Box Supervision
Improving the detection of distant 3d objects is an important yet challenging task. For camera-based 3D perception, the annotation of 3d bounding relies heavily on LiDAR for accurate depth information. As such, the dista…
3D Object DetectionDepth EstimationObjectobject-detection+1ANEA: Distant Supervision for Low-Resource Named Entity Recognition
Distant supervision allows obtaining labeled training corpora for low-resource settings where only limited hand-annotated data exists. However, to be used effectively, the distant supervision must be easy to gather. In t…
Low Resource Named Entity Recognitionnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)