Transfer-Free Data-Efficient Multilingual Slot Labeling
Slot labeling (SL) is a core component of task-oriented dialogue (ToD) systems, where slots and corresponding values are usually language-, task- and domain-specific. Therefore, extending the system to any new language-domain-task configuration requires (re)running an expensive and resource-intensive data annotation process. To mitigate the inherent data scarcity issue, current research on multilingual ToD assumes that sufficient English-language annotated data are always available for particular tasks and domains, and thus operates in a standard cross-lingual transfer setup. In this work, we depart from this often unrealistic assumption. We examine challenging scenarios where such transfer-enabling English annotated data cannot be guaranteed, and focus on bootstrapping multilingual data-efficient slot labelers in transfer-free scenarios directly in the target languages without any English-ready data. We propose a two-stage slot labeling approach (termed TWOSL) which transforms standard multilingual sentence encoders into effective slot labelers. In Stage 1, relying on SL-adapted contrastive learning with only a handful of SL-annotated examples, we turn sentence encoders into task-specific span encoders. In Stage 2, we recast SL from a token classification into a simpler, less data-intensive span classification task. Our results on two standard multilingual TOD datasets and across diverse languages confirm the effectiveness and robustness of TWOSL. It is especially effective for the most challenging transfer-free few-shot setups, paving the way for quick and data-efficient bootstrapping of multilingual slot labelers for ToD.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningCross-Lingual TransferSentencetoken-classificationToken ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
End-to-End Slot Alignment and Recognition for Cross-Lingual NLU
Natural language understanding (NLU) in the context of goal-oriented dialog systems typically includes intent classification and slot labeling tasks. Existing methods to expand an NLU system to new languages use machine …
Cross-Lingual TransferGoal-Oriented Dialogintent-classificationIntent Classification+4Analyzing BERT Cross-lingual Transfer Capabilities in Continual Sequence Labeling
Knowledge transfer between neural language models is a widely used technique that has proven to improve performance in a multitude of natural language tasks, in particular with the recent rise of large pre-trained langua…
Continual LearningCross-Lingual Transfernamed-entity-recognitionNamed Entity Recognition+4Improving Zero-Shot Multilingual Text Generation via Iterative Distillation
The demand for multilingual dialogue systems often requires a costly labeling process, where human translators derive utterances in low resource languages from resource rich language annotation. To this end, we explore l…
Knowledge DistillationText GenerationAdaptive Self-training for Neural Sequence Labeling with Few Labels
Neural sequence labeling is an important technique employed for many Natural Language Processing (NLP) tasks, such as Named Entity Recognition (NER), slot tagging for dialog systems and semantic parsing. Large-scale pre-…
Meta-Learningnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+2Source-Critical Reinforcement Learning for Transferring Spoken Language Understanding to a New Language
To deploy a spoken language understanding (SLU) model to a new language, language transferring is desired to avoid the trouble of acquiring and labeling a new big SLU corpus. Translating the original SLU corpus into the …
Cultural Vocal Bursts Intensity Predictiondomain classificationGeneral Classificationreinforcement-learning+5