Lipreading
8개 벤치마크 · 논문 103편 · 이 태스크의 논문 보기 →
Benchmarks
LRS2
LRS3-TED
Lip Reading in the Wild
CAS-VSR-W1k (LRW-1000)
CMLR
GRID corpus (mixed-speech)
LRW-1000
CAS-VSR-S101
Most implemented
LipNet: End-to-End Sentence-level Lipreading
Deep Audio-Visual Speech Recognition
Combining Residual Networks with LSTMs for Lipreading
End-to-end Audio-visual Speech Recognition with Conformers
Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels
Visual Speech Recognition for Multiple Languages in the Wild
Papers
Learning Speaker-Invariant Visual Features for Lipreading
Lipreading is a challenging cross-modal task that aims to convert visual lip movements into spoken text. Existing lipreading methods often extract visual features that include speaker-specific lip attributes (e.g., shape…
DisentanglementLipreadingSpeaker RecognitionUniCUE: Unified Recognition and Generation Framework for Chinese Cued Speech Video-to-Speech Generation
Cued Speech (CS) enhances lipreading through hand coding, providing precise speech perception support for the hearing-impaired. CS Video-to-Speech generation (CSV2S) task aims to convert the CS visual expressions (CS vid…
cross-modal alignmentLipreadingtext-to-speechText to SpeechOXSeg: Multidimensional attention UNet-based lip segmentation using semi-supervised lip contours
Lip segmentation plays a crucial role in various domains, such as lip synchronization, lipreading, and diagnostics. However, the effectiveness of supervised lip segmentation is constrained by the availability of lip cont…
Generative Adversarial NetworkLipreadingSegmentationAudio-Visual Representation Learning via Knowledge Distillation from Speech Foundation Models
Audio-visual representation learning is crucial for advancing multimodal speech processing tasks, such as lipreading and audio-visual speech recognition. Recently, speech foundation models (SFMs) have shown remarkable ge…
Audio-Visual Speech RecognitionAutomatic Speech RecognitionAutomatic Speech Recognition (ASR)Knowledge Distillation+5Target Speaker Lipreading by Audio-Visual Self-Distillation Pretraining and Speaker Adaptation
Lipreading is an important technique for facilitating human-computer interaction in noisy environments. Our previously developed self-supervised learning method, AV2vec, which leverages multimodal self-distillation, has …
Cross-Lingual TransferLipreadingSelf-Supervised LearningTransfer LearningEvaluation of End-to-End Continuous Spanish Lipreading in Different Data Conditions
Visual speech recognition remains an open research problem where different challenges must be considered by dispensing with the auditory sense, such as visual ambiguities, the inter-personal variability among speakers, a…
Lipreadingspeech-recognitionSpeech RecognitionVisual Speech Recognition