Sonification of Facial Actions for Musical Expression
The central role of the face in social interaction and non-verbal communication suggests we explore facial action as a means of musical expression. This paper presents the design, implementation, and preliminary studies of a novel system utilizing face detection and optic flow algorithms to associate facial movements with sound synthesis in a topographically specific fashion. We report on our experience with various gesture-to-sound mappings and applications, and describe our preliminary experiments at musical performance using the system.
Code (0)
등록된 구현이 없습니다.
Tasks
Face DetectionSimilar Papers 제목 키워드 기반
Music and musical sonification for the rehabilitation of Parkinsonian dysgraphia: Conceptual framework
Music has been shown to enhance motor control in patients with Parkinson's disease (PD). Notably, musical rhythm is perceived as an external auditory cue that helps PD patients to better control movements. The rationale …
RhythmFacial gesture interfaces for expression and communication
Considerable effort has been devoted to the automatic extraction of information about action of the face from image sequences. Within the context of human-computer interaction (HCI) we may distinguish systems that allow …
A Minimalist Brain-Computer Musical Interface for Real-Time Emotion-Driven Sonification: System Design and Preliminary Evaluation
This paper presents a minimalist brain-computer Musical Interface (BCMI) that functions as a real-time affective sonification system, translating prefrontal EEG activity into adaptive music. Emotional valence is estimate…
Music GenerationMugeetion: Musical Interface Using Facial Gesture and Emotion
People feel emotions when listening to music. However, emotions are not tangible objects that can be exploited in the music composition process as they are difficult to capture and quantify in algorithms. We present a no…
Tokyo Kion-On: Query-Based Generative Sonification of Atmospheric Data
Amid growing environmental concerns, interactive displays of data constitute an important tool for exploring and understanding the impact of climate change on the planet's ecosystemic integrity. This paper presents Tokyo…