paper-with-me

홈 › Papers

Risk Minimization for Zero-shot Sequence Labeling

2021-08-01 · ACL 2021 5 · Zechuan Hu, Yong Jiang, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, Kewei Tu

Zero-shot sequence labeling aims to build a sequence labeler without human-annotated datasets. One straightforward approach is utilizing existing systems (source models) to generate pseudo-labeled datasets and train a target sequence labeler accordingly. However, due to the gap between the source and the target languages/domains, this approach may fail to recover the true labels. In this paper, we propose a novel unified framework for zero-shot sequence labeling with minimum risk training and design a new decomposable risk function that models the relations between the predicted labels from the source models and the true labels. By making the risk function trainable, we draw a connection between minimum risk training and latent variable model learning. We propose a unified learning algorithm based on the expectation maximization (EM) algorithm. We extensively evaluate our proposed approaches on cross-lingual/domain sequence labeling tasks over twenty-one datasets. The results show that our approaches outperform state-of-the-art baseline systems.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Zero-shot Sequence Labeling for Transformer-based Sentence Classifiers

2021-03-26 · ACL (RepL4NLP) 2021 8 · Kamil Bujel, Helen Yannakoudakis, Marek Rei

We investigate how sentence-level transformers can be modified into effective sequence labelers at the token level without any direct supervision. Existing approaches to zero-shot sequence labeling do not perform well wh…

Sentence

Dual Risk Minimization: Towards Next-Level Robustness in Fine-tuning Zero-Shot Models

2024-11-29 · Kaican Li, Weiyan Xie, Yongxiang Huang, Didan Deng 외

Fine-tuning foundation models often compromises their robustness to distribution shifts. To remedy this, most robust fine-tuning methods aim to preserve the pre-trained features. However, not all pre-trained features are…

Self-Training Pre-Trained Language Models for Zero- and Few-Shot Multi-Dialectal Arabic Sequence Labeling

2021-01-12 · EACL 2021 2 · Muhammad Khalifa, Muhammad Abdul-Mageed, Khaled Shaalan

A sufficient amount of annotated data is usually required to fine-tune pre-trained language models for downstream tasks. Unfortunately, attaining labeled data can be costly, especially for multiple language varieties and…

Language ModellingNERPOSPOS Tagging

ToPro: Token-Level Prompt Decomposition for Cross-Lingual Sequence Labeling Tasks

2024-01-29 · Bolei Ma, Ercong Nie, Shuzhou Yuan, Helmut Schmid 외

Prompt-based methods have been successfully applied to multilingual pretrained language models for zero-shot cross-lingual understanding. However, most previous studies primarily focused on sentence-level classification …

BenchmarkingCross-Lingual TransferIn-Context Learningnamed-entity-recognition+8

SLIMER-IT: Zero-Shot NER on Italian Language

2024-09-24 · Andrew Zamai, Leonardo Rigutini, Marco Maggini, Andrea Zugarini

Traditional approaches to Named Entity Recognition (NER) frame the task into a BIO sequence labeling problem. Although these systems often excel in the downstream task at hand, they require extensive annotated data and s…

named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER