Decoding visemes: improving machine lipreading
To undertake machine lip-reading, we try to recognise speech from a visual signal. Current work often uses viseme classification supported by language models with varying degrees of success. A few recent works suggest phoneme classification, in the right circumstances, can outperform viseme classification. In this work we present a novel two-pass method of training phoneme classifiers which uses previously trained visemes in the first pass. With our new training algorithm, we show classification performance which significantly improves on previous lip-reading results.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationGeneral ClassificationLipreadingLip ReadingSimilar Papers 제목 키워드 기반
Decoding visemes: improving machine lipreading
Machine lipreading (MLR) is speech recognition from visual cues and a niche research problem in speech processing & computer vision. Current challenges fall into two groups: the content of the video, such as rate of spee…
ClusteringGeneral ClassificationLipreadingRobust classification+2Understanding the visual speech signal
For machines to lipread, or understand speech from lip movement, they decode lip-motions (known as visemes) into the spoken sounds. We investigate the visual speech channel to further our understanding of visemes. This h…
LipreadingComparing phonemes and visemes with DNN-based lipreading
There is debate if phoneme or viseme units are the most effective for a lipreading system. Some studies use phoneme units even though phonemes describe unique short sounds; other studies tried to improve lipreading accur…
DecoderLipreadingAlternative Visual Units for an Optimized Phoneme-Based Lipreading System
Lipreading is understanding speech from observed lip movements. An observed series of lip motions is an ordered sequence of visual lip gestures. These gestures are commonly known, but as yet are not formally defined, as …
LipreadingManagementVisual gesture variability between talkers in continuous visual speech
Recent adoption of deep learning methods to the field of machine lipreading research gives us two options to pursue to improve system performance. Either, we develop end-to-end systems holistically or, we experiment to f…
Lipreading