Lip-reading with Densely Connected Temporal Convolutional Networks
In this work, we present the Densely Connected Temporal Convolutional Network (DC-TCN) for lip-reading of isolated words. Although Temporal Convolutional Networks (TCN) have recently demonstrated great potential in many vision tasks, its receptive fields are not dense enough to model the complex temporal dynamics in lip-reading scenarios. To address this problem, we introduce dense connections into the network to capture more robust temporal features. Moreover, our approach utilises the Squeeze-and-Excitation block, a light-weight attention mechanism, to further enhance the model's classification power. Without bells and whistles, our DC-TCN method has achieved 88.36% accuracy on the Lip Reading in the Wild (LRW) dataset and 43.65% on the LRW-1000 dataset, which has surpassed all the baseline methods and is the new state-of-the-art on both datasets.
Code (1)
Tasks
Lip ReadingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Training Strategies for Improved Lip-reading
Several training strategies and temporal models have been recently proposed for isolated word lip-reading in a series of independent works. However, the potential of combining the best strategies and investigating the im…
Data AugmentationLipreadingLip ReadingDensely Connected Attention Propagation for Reading Comprehension
We propose DecaProp (Densely Connected Attention Propagation), a new densely connected neural architecture for reading comprehension (RC). There are two distinct characteristics of our model. Firstly, our model densely c…
AllOpen-Domain Question AnsweringQuestion AnsweringReading ComprehensionReading Scene Text with Attention Convolutional Sequence Modeling
Reading text in the wild is a challenging task in the field of computer vision. Existing approaches mainly adopted Connectionist Temporal Classification (CTC) or Attention models based on Recurrent Neural Network (RNN), …
Scene Text RecognitionActive Deep Densely Connected Convolutional Network for Hyperspectral Image Classification
Deep learning based methods have seen a massive rise in popularity for hyperspectral image classification over the past few years. However, the success of deep learning is attributed greatly to numerous labeled samples. …
Active LearningClassificationDeep LearningGeneral Classification+3Assessment of Breast Cancer Histology using Densely Connected Convolutional Networks
Breast cancer is the most frequently diagnosed cancer and leading cause of cancer-related death among females worldwide. In this article, we investigate the applicability of densely connected convolutional neural network…
General Classificationimage-classificationImage ClassificationImage Segmentation+2