FaraPy: An Augmented Reality Feedback System for Facial Paralysis using Action Unit Intensity Estimation
Facial paralysis is the most common facial nerve disorder. It causes functional and aesthetic defcitsthat often lead to emotional distress and afect psychosocial well-being. One form of treatment is mirror therapy, which has shown potential but has several mirror-related drawbacks that limit its efectiveness. We propose FaraPy, the frst mobile augmented reality mirror therapy system for facial paralysis that provides real-time feedback and tracks user progress over time. We developed an efcient convolutional neural network to detect muscle activations and intensities as users perform facial exercises in front of a mobile device camera. Our model outperformsthe stateof-the-art model on benchmark data for detecting facial action unit intensity. Our user study (n=20) shows high user satisfaction and greater preference for our interactive system over traditional mirror therapy
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Has the Virtualization of the Face Changed Facial Perception? A Study of the Impact of Photo Editing and Augmented Reality on Facial Perception
Augmented reality and other photo editing filters are popular methods used to modify faces online. Considering the important role of facial perception in communication, how do we perceive this increasing number of modifi…
FaceEraser: Removing Facial Parts for Augmented Reality
Our task is to remove all facial parts (e.g., eyebrows, eyes, mouth and nose), and then impose visual elements onto the ``blank'' face for augmented reality. Conventional object removal methods rely on image inpainting t…
Image InpaintingFaceAtlasAR: Atlas of Facial Acupuncture Points in Augmented Reality
Acupuncture is a technique in which practitioners stimulate specific points on the body. Those points, called acupuncture points (or acupoints), anatomically define areas on the skin relative to specific landmarks on the…
Face AlignmentAEGIS: A real-time multimodal augmented reality computer vision based system to assist facial expression recognition for individuals with autism spectrum disorder
The ability to interpret social cues comes naturally for most people, but for those living with Autism Spectrum Disorder (ASD), some experience a deficiency in this area. This paper presents the development of a multimod…
Facial Expression RecognitionFacial Expression Recognition (FER)Pedagogical Reflections on the Holistic Cognitive Development (HCD) Framework and AI-Augmented Learning in Creative Computing
This paper presents an expanded account of the Holistic Cognitive Development (HCD) framework for reflective and creative learning in computing education. The HCD framework integrates design thinking, experiential learni…