Multi-person Spatial Interaction in a Large Immersive Display Using Smartphones as Touchpads
In this paper, we present a multi-user interaction interface for a large immersive space that supports simultaneous screen interactions by combining (1) user input via personal smartphones and Bluetooth microphones, (2) spatial tracking via an overhead array of Kinect sensors, and (3) WebSocket interfaces to a webpage running on the large screen. Users are automatically, dynamically assigned personal and shared screen sub-spaces based on their tracked location with respect to the screen, and use a webpage on their personal smartphone for touchpad-type input. We report user experiments using our interaction framework that involve image selection and placement tasks, with the ultimate goal of realizing display-wall environments as viable, interactive workspaces with natural multimodal interfaces.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
OmniCharacter: Towards Immersive Role-Playing Agents with Seamless Speech-Language Personality Interaction
Role-Playing Agents (RPAs), benefiting from large language models, is an emerging interactive AI system that simulates roles or characters with diverse personalities. However, existing methods primarily focus on mimickin…
Multi-Source Spatial Knowledge Understanding for Immersive Visual Text-to-Speech
Visual Text-to-Speech (VTTS) aims to take the environmental image as the prompt to synthesize reverberant speech for the spoken content. Previous works focus on the RGB modality for global environmental modeling, overloo…
object-detectionObject DetectionPositiontext-to-speech+1Realizing Immersive Volumetric Video: A Multimodal Framework for 6-DoF VR Engagement
Fully immersive experiences that tightly integrate 6-DoF visual and auditory interaction are essential for virtual and augmented reality. While such experiences can be achieved through computer-generated content, constru…
Closing the Loop: Unified 3D Scene Generation and Immersive Interaction via LLM-RL Coupling
Recent advances in large language models (LLMs) have significantly improved language-driven 3D content generation, but most existing approaches still treat scene generation and user interaction as separate processes, lim…
Reinforcement LearningScene GenerationTyping Reinvented: Towards Hands-Free Input via sEMG
We explore surface electromyography (sEMG) as a non-invasive input modality for mapping muscle activity to keyboard inputs, targeting immersive typing in next-generation human-computer interaction (HCI). This is especial…