Transcending Dimensions using Generative AI: Real-Time 3D Model Generation in Augmented Reality

2025-04-27 · Majid Behravan, Maryam Haghani, Denis Gracanin

Traditional 3D modeling requires technical expertise, specialized software, and time-intensive processes, making it inaccessible for many users. Our research aims to lower these barriers by combining generative AI and augmented reality (AR) into a cohesive system that allows users to easily generate, manipulate, and interact with 3D models in real time, directly within AR environments. Utilizing cutting-edge AI models like Shap-E, we address the complex challenges of transforming 2D images into 3D representations in AR environments. Key challenges such as object isolation, handling intricate backgrounds, and achieving seamless user interaction are tackled through advanced object detection methods, such as Mask R-CNN. Evaluation results from 35 participants reveal an overall System Usability Scale (SUS) score of 69.64, with participants who engaged with AR/VR technologies more frequently rating the system significantly higher, at 80.71. This research is particularly relevant for applications in gaming, education, and AR-based e-commerce, offering intuitive, model creation for users without specialized skills.

📄 PDF Abstract BibTeX arXiv:2504.21033

Code (0)

등록된 구현이 없습니다.

Tasks

object-detection Object Detection

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

RoIAlign Region of Interest Align, or RoIAlign, is an operation for extracting a small feature map from each RoI in detection and segmentation based tasks. It removes the harsh…

RPN A Region Proposal Network, or RPN, is a fully convolutional network that simultaneously predicts object bounds and objectness scores at each position. The RPN is trained…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Mask R-CNN Mask R-CNN extends Faster R-CNN to solve instance segmentation tasks. It achieves this by adding a branch for predicting an…

Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AI

2025-10-06 · Kun Xiang, Terry Jingchen Zhang, Yinya Huang, Jixi He 외 arxiv

The rapid advancement of embodied intelligence and world models has intensified efforts to integrate physical laws into AI systems, yet physical perception and symbolic physics reasoning have developed along separate tra…

Transcending Controlled Environments Assessing the Transferability of ASRRobust NLU Models to Real-World Applications

2024-01-12 · Hania Khan, Aleena Fatima Khalid, Zaryab Hassan

This research investigates the transferability of Automatic Speech Recognition (ASR)-robust Natural Language Understanding (NLU) models from controlled experimental conditions to practical, real-world applications. Focus…

Automatic Speech Recognition Automatic Speech Recognition (ASR)Domain Adaptation Natural Language Understanding+3

Intelligent Canvas: Enabling Design-Like Exploratory Visual Data Analysis with Generative AI through Rapid Prototyping, Iteration and Curation

2024-02-13 · Zijian Ding, Joel Chan

Complex data analysis inherently seeks unexpected insights through exploratory visual analysis methods, transcending logical, step-by-step processing. However, existing interfaces such as notebooks and dashboards have li…

Management

ComboStoc: Combinatorial Stochasticity for Diffusion Generative Models

2024-05-22 · Rui Xu, Jiepeng Wang, Hao Pan, Yang Liu 외

In this paper, we study an under-explored but important factor of diffusion generative models, i.e., the combinatorial complexity. Data samples are generally high-dimensional, and for various structured generation tasks,…

Simulating the Real World: A Unified Survey of Multimodal Generative Models

2025-03-06 · Yuqi Hu, Longguang Wang, Xian Liu, Ling-Hao Chen 외

Understanding and replicating the real world is a critical challenge in Artificial General Intelligence (AGI) research. To achieve this, many existing approaches, such as world models, aim to capture the fundamental prin…

3D Generation Survey