SpaceBlender: Creating Context-Rich Collaborative Spaces Through Generative 3D Scene Blending
There is increased interest in using generative AI to create 3D spaces for Virtual Reality (VR) applications. However, today's models produce artificial environments, falling short of supporting collaborative tasks that benefit from incorporating the user's physical context. To generate environments that support VR telepresence, we introduce SpaceBlender, a novel pipeline that utilizes generative AI techniques to blend users' physical surroundings into unified virtual spaces. This pipeline transforms user-provided 2D images into context-rich 3D environments through an iterative process consisting of depth estimation, mesh alignment, and diffusion-based space completion guided by geometric priors and adaptive text prompts. In a preliminary within-subjects study, where 20 participants performed a collaborative VR affinity diagramming task in pairs, we compared SpaceBlender with a generic virtual environment and a state-of-the-art scene generation framework, evaluating its ability to create virtual spaces suitable for collaboration. Participants appreciated the enhanced familiarity and context provided by SpaceBlender but also noted complexities in the generative environments that could detract from task focus. Drawing on participant feedback, we propose directions for improving the pipeline and discuss the value and design of blended spaces for different scenarios.
Code (0)
등록된 구현이 없습니다.
Tasks
Depth EstimationScene GenerationSimilar Papers 제목 키워드 기반
Cross-lingual alignments of ELMo contextual embeddings
Building machine learning prediction models for a specific NLP task requires sufficient training data, which can be difficult to obtain for less-resourced languages. Cross-lingual embeddings map word embeddings from a le…
Dependency Parsingnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+3Enhancing Collaborative Semantics of Language Model-Driven Recommendations via Graph-Aware Learning
Large Language Models (LLMs) are increasingly prominent in the recommendation systems domain. Existing studies usually utilize in-context learning or supervised fine-tuning on task-specific data to align LLMs into recomm…
In-Context LearningLanguage ModelingLanguage ModellingRecommendation SystemsCollaborative Semantic Alignment in Recommendation Systems
Traditional recommender systems primarily leverage identity-based (ID) representations for users and items, while the advent of pre-trained language models (PLMs) has introduced rich semantic modeling of item description…
Collaborative FilteringLanguage ModellingRecommendation SystemsMaskMA: Towards Zero-Shot Multi-Agent Decision Making with Mask-Based Collaborative Learning
Building a single generalist agent with strong zero-shot capability has recently sparked significant advancements. However, extending this capability to multi-agent decision making scenarios presents challenges. Most cur…
Decision MakingSMACSMAC+Harnessing Generalist Agents for Contextualized Time Series
Time series are often embedded in rich contexts that are essential for holistic modeling. Moreover, real-world practitioners often require end-to-end workflows for analyzing temporal dynamics, where widely studied tasks …