Joint Speech Translation and Named Entity Recognition

2022-10-21 · Marco Gaido, Sara Papi, Matteo Negri, Marco Turchi

Modern automatic translation systems aim at place the human at the center by providing contextual support and knowledge. In this context, a critical task is enriching the output with information regarding the mentioned entities, which is currently achieved processing the generated translation with named entity recognition (NER) and entity linking systems. In light of the recent promising results shown by direct speech translation (ST) models and the known weaknesses of cascades (error propagation and additional latency), in this paper we propose multitask models that jointly perform ST and NER, and compare them with a cascade baseline. The experimental results show that our models significantly outperform the cascade on the NER task (by 0.4-1.0 F1), without degradation in terms of translation quality, and with the same computational efficiency of a plain direct ST model.

📄 PDF Abstract BibTeX arXiv:2210.11987

Code (1)

hlt-mt/fbk-fairseq 공식 구현 pytorch

Tasks

Computational Efficiency Entity Linkingnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER Translation

Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages

2022-05-02 · Felix Wu, Kwangyoun Kim, Shinji Watanabe, Kyu Han 외

We introduce Wav2Seq, the first self-supervised approach to pre-train both parts of encoder-decoder models for speech data. We induce a pseudo language as a compact discrete representation, and formulate a self-supervise…

Automatic Speech Recognition Automatic Speech Recognition (ASR)Decoder named-entity-recognition+7

WhisperNER: Unified Open Named Entity and Speech Recognition

2024-09-12 · Gil Ayache, Menachem Pirchi, Aviv Navon, Aviv Shamsian 외

Integrating named entity recognition (NER) with automatic speech recognition (ASR) can significantly enhance transcription accuracy and informativeness. In this paper, we introduce WhisperNER, a novel model that allows j…

Automatic Speech Recognition Automatic Speech Recognition (ASR)Informativeness named-entity-recognition+5

CV-18 NER: Augmented Common Voice for Named Entity Recognition from Arabic Speech

2026-04-02 · Youssef Saidi, Haroun Elleuch, Fethi Bougares arxiv

End-to-end speech Named Entity Recognition (NER) aims to directly extract entities from speech. Prior work has shown that end-to-end (E2E) approaches can outperform cascaded pipelines for English, French, and Chinese, bu…

End-to-end named entity extraction from speech

2018-05-30 · Sahar Ghannay, Antoine Caubrière, Yannick Estève, Antoine Laurent 외

Named entity recognition (NER) is among SLU tasks that usually extract semantic information from textual documents. Until now, NER from speech is made through a pipeline process that consists in processing first an autom…

Automatic Speech Recognition Automatic Speech Recognition (ASR)Entity Extraction using GAN named-entity-recognition+5

End-to-end Named Entity Recognition from English Speech

2020-05-22 · Hemant Yadav, Sreyan Ghosh, Yi Yu, Rajiv Ratn Shah

Named entity recognition (NER) from text has been a widely studied problem and usually extracts semantic information from text. Until now, NER from speech is mostly studied in a two-step pipeline process that includes fi…

Automatic Speech Recognition Automatic Speech Recognition (ASR)named-entity-recognition Named Entity Recognition+4