Audio-Visual Speech Recognition
4개 벤치마크 · 논문 118편 · 이 태스크의 논문 보기 →
Benchmarks
Most implemented
Deep Audio-Visual Speech Recognition
VSRo-200: A Romanian Visual Speech Recognition Dataset for Studying Supervision and Multimodal Robustness
End-to-end Audio-visual Speech Recognition with Conformers
Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels
Discriminative Multi-modality Speech Recognition
Papers
Candor-LR: A Dyadic Conversational Dataset for Audio-Visual Speech Recognition
Current audio-visual speech recognition (AVSR) benchmarks, like LRS3, rely heavily on clean, scripted and rehearsed speech. They fail to reflect the complexity of natural conversation, which involves overlapping speech, …
Audio-Visual Speech RecognitionVSRo-200: A Romanian Visual Speech Recognition Dataset for Studying Supervision and Multimodal Robustness
We introduce VSRo-200, the first large-scale dataset for visual speech recognition (lip reading) in Romanian, comprising 200 hours of real-world podcast videos. All samples are annotated with pseudo-labels generated by a…
Audio-Visual Speech RecognitionDomain GeneralizationLip ReadingVIB-AVSR: Variational Information Bottleneck for Noise-Robust LLM-Based Audio-Visual Speech Recognition
Audio-Visual Speech Recognition takes two input modalities, acoustic and visual streams, where visual information from lip movements aids recognition when audio is noisy. Recently, LLM-based AVSR models have emerged as a…
Audio-Visual Speech RecognitionWhen AVSR Meets Video Conferencing: Dataset, Degradation, and the Hidden Mechanism Behind Performance Collapse
Audio-Visual Speech Recognition (AVSR) has achieved remarkable progress in offline conditions, yet its robustness in real-world video conferencing (VC) remains largely unexplored. This paper presents the first systematic…
Audio-Visual Speech RecognitionSpeech EnhancementDr. SHAP-AV: Decoding Relative Modality Contributions via Shapley Attribution in Audio-Visual Speech Recognition
Audio-Visual Speech Recognition (AVSR) leverages both acoustic and visual information for robust recognition under noise. However, how models balance these modalities remains unclear. We present Dr. SHAP-AV, a framework …
Audio-Visual Speech RecognitionOCR-Enhanced Multimodal ASR Can Read While Listening
Visual information, such as subtitles in a movie, often helps automatic speech recognition. In this paper, we propose Donut-Whisper, an audio-visual ASR model with dual encoder to leverage visual information to improve s…
Audio-Visual Speech RecognitionKnowledge Distillation