Hierarchical Meta-Embeddings for Code-Switching Named Entity Recognition

2019-09-18 · IJCNLP 2019 11 · Genta Indra Winata, Zhaojiang Lin, Jamin Shin, Zihan Liu, Pascale Fung

In countries that speak multiple main languages, mixing up different languages within a conversation is commonly called code-switching. Previous works addressing this challenge mainly focused on word-level aspects such as word embeddings. However, in many cases, languages share common subwords, especially for closely related languages, but also for languages that are seemingly irrelevant. Therefore, we propose Hierarchical Meta-Embeddings (HME) that learn to combine multiple monolingual word-level and subword-level embeddings to create language-agnostic lexical representations. On the task of Named Entity Recognition for English-Spanish code-switching data, our model achieves the state-of-the-art performance in the multilingual settings. We also show that, in cross-lingual settings, our model not only leverages closely related languages, but also learns from languages with different roots. Finally, we show that combining different subunits are crucial for capturing code-switching entities.

📄 PDF Abstract BibTeX arXiv:1909.08504

Code (1)

gentaiscool/meta-emb 공식 구현 pytorch

Tasks

named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)Word Embeddings

Learning Multilingual Meta-Embeddings for Code-Switching Named Entity Recognition

2019-08-01 · WS 2019 8 · Genta Indra Winata, Zhaojiang Lin, Pascale Fung

In this paper, we propose Multilingual Meta-Embeddings (MME), an effective method to learn multilingual representations by leveraging monolingual pre-trained embeddings. MME learns to utilize information from these embed…

Language Identification MME named-entity-recognition Named Entity Recognition+1

Are Multilingual Models Effective in Code-Switching?

2021-03-24 · NAACL (CALCS) 2021 6 · Genta Indra Winata, Samuel Cahyawijaya, Zihan Liu, Zhaojiang Lin 외

Multilingual language models have shown decent performance in multilingual and cross-lingual natural language understanding tasks. However, the power of these multilingual models in code-switching tasks has not been full…

named-entity-recognition Named Entity Recognition Named Entity Recognition (NER)Natural Language Understanding+1

Bilingual Character Representation for Efficiently Addressing Out-of-Vocabulary Words in Code-Switching Named Entity Recognition

2018-05-30 · WS 2018 7 · Genta Indra Winata, Chien-Sheng Wu, Andrea Madotto, Pascale Fung

We propose an LSTM-based model with hierarchical architecture on named entity recognition from code-switching Twitter data. Our model uses bilingual character representation and transfer learning to address out-of-vocabu…

named-entity-recognition Named Entity Recognition Named Entity Recognition (NER)Transfer Learning

Multilingual Transfer Learning for Code-Switched Language and Speech Neural Modeling

2021-04-13 · Genta Indra Winata

In this thesis, we address the data scarcity and limitations of linguistic theory by proposing language-agnostic multi-task training methods. First, we introduce a meta-learning-based approach, meta-transfer learning, in…

Data Augmentation Language Modelling Meta-Learning Multi-Task Learning+1

Tabular Embeddings for Tables with Bi-Dimensional Hierarchical Metadata and Nesting

2025-02-20 · Gyanendra Shrestha, Chutain Jiang, Sai Akula, Vivek Yannam 외

Embeddings serve as condensed vector representations for real-world entities, finding applications in Natural Language Processing (NLP), Computer Vision, and Data Management across diverse downstream tasks. Here, we intr…

Management RAG