LI-TTA: Language Informed Test-Time Adaptation for Automatic Speech Recognition

2024-08-11 · Eunseop Yoon, Hee Suk Yoon, John Harvill, Mark Hasegawa-Johnson, Chang D. Yoo

Test-Time Adaptation (TTA) has emerged as a crucial solution to the domain shift challenge, wherein the target environment diverges from the original training environment. A prime exemplification is TTA for Automatic Speech Recognition (ASR), which enhances model performance by leveraging output prediction entropy minimization as a self-supervision signal. However, a key limitation of this self-supervision lies in its primary focus on acoustic features, with minimal attention to the linguistic properties of the input. To address this gap, we propose Language Informed Test-Time Adaptation (LI-TTA), which incorporates linguistic insights during TTA for ASR. LI-TTA integrates corrections from an external language model to merge linguistic with acoustic information by minimizing the CTC loss from the correction alongside the standard TTA loss. With extensive experiments, we show that LI-TTA effectively improves the performance of TTA for ASR in various distribution shift situations.

📄 PDF Abstract BibTeX arXiv:2408.05769

Code (1)

esyoon7/litta 공식 구현 pytorch

Tasks

Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Language ModelingLanguage Modelling speech-recognition Speech Recognition Test-time Adaptation

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Attention 설명 없음

CTC Loss 설명 없음

Focus 설명 없음

Groupwise Registration with Physics-Informed Test-Time Adaptation on Multi-parametric Cardiac MRI

2025-10-29 · Xinqi Li, Yi Zhang, Li-Ting Huang, Hsiao-Huang Chang 외 arxiv

Multiparametric mapping MRI has become a viable tool for myocardial tissue characterization. However, misalignment between multiparametric maps makes pixel-wise analysis challenging. To address this challenge, we develop…

Test-time Adaptation Image Registration

PI-MFM: Physics-informed multimodal foundation model for solving partial differential equations

2025-12-28 · Min Zhu, Jingmin Sun, Zecheng Zhang, Hayden Schaeffer 외 arxiv

Partial differential equations (PDEs) govern a wide range of physical systems, and recent multimodal foundation models have shown promise for learning PDE solution operators across diverse equation families. However, exi…

CANDI: Curated Test-Time Adaptation for Multivariate Time-Series Anomaly Detection Under Distribution Shift

2026-04-02 · HyunGi Kim, Jisoo Mok, Hyungyu Lee, Juhyeon Shin 외 arxiv

Multivariate time-series anomaly detection (MTSAD) aims to identify deviations from normality in multivariate time-series and is critical in real-world applications. However, in real-world deployments, distribution shift…

Test-time Adaptation Anomaly Detection

Data-efficient Co-Adaptation of Morphology and Behaviour with Deep Reinforcement Learning

2019-11-15 · Kevin Sebastian Luck, Heni Ben Amor, Roberto Calandra

Humans and animals are capable of quickly learning new behaviours to solve new tasks. Yet, we often forget that they also rely on a highly specialized morphology that co-adapted with motor control throughout thousands of…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

When Audio-Language Models Fail to Leverage Multimodal Context for Dysarthric Speech Recognition

2026-05-04 · Pehuén Moure, Niclas Pokel, Bilal Bounajma, Yingqiang Gao 외 arxiv

Automatic speech recognition (ASR) systems remain brittle on dysarthric and other atypical speech. Recent audio-language models raise the possibility of improving performance by conditioning on additional clinical contex…

Speech Recognition