Real-Time Onboard Object Detection for Augmented Reality: Enhancing Head-Mounted Display with YOLOv8
This paper introduces a software architecture for real-time object detection using machine learning (ML) in an augmented reality (AR) environment. Our approach uses the recent state-of-the-art YOLOv8 network that runs onboard on the Microsoft HoloLens 2 head-mounted display (HMD). The primary motivation behind this research is to enable the application of advanced ML models for enhanced perception and situational awareness with a wearable, hands-free AR platform. We show the image processing pipeline for the YOLOv8 model and the techniques used to make it real-time on the resource-limited edge computing platform of the headset. The experimental results demonstrate that our solution achieves real-time processing without needing offloading tasks to the cloud or any other external servers while retaining satisfactory accuracy regarding the usual mAP metric and measured qualitative performance
Code (0)
등록된 구현이 없습니다.
Tasks
Edge-computingobject-detectionObject DetectionReal-Time Object DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Complementing Onboard Sensors with Satellite Map: A New Perspective for HD Map Construction
High-definition (HD) maps play a crucial role in autonomous driving systems. Recent methods have attempted to construct HD maps in real-time using vehicle onboard sensors. Due to the inherent limitations of onboard senso…
Autonomous DrivingSemantic SegmentationReal-Time Multi-Modal Semantic Fusion on Unmanned Aerial Vehicles
Unmanned aerial vehicles (UAVs) equipped with multiple complementary sensors have tremendous potential for fast autonomous or remote-controlled semantic scene analysis, e.g., for disaster examination. In this work, we pr…
Image Segmentationobject-detectionObject DetectionSegmentation+1PillarGrid: Deep Learning-based Cooperative Perception for 3D Object Detection from Onboard-Roadside LiDAR
3D object detection plays a fundamental role in enabling autonomous driving, which is regarded as the significant key to unlocking the bottleneck of contemporary transportation systems from the perspectives of safety, mo…
3D Object DetectionAutonomous DrivingObjectobject-detection+1WildLive: Near Real-time Visual Wildlife Tracking onboard UAVs
Live tracking of wildlife via high-resolution video processing directly onboard drones is widely unexplored and most existing solutions rely on streaming video to ground stations to support navigation. Yet, both autonomo…
4kObject TrackingOptical Flow EstimationSqueezed Edge YOLO: Onboard Object Detection on Edge Devices
Demand for efficient onboard object detection is increasing due to its key role in autonomous navigation. However, deploying object detection models such as YOLO on resource constrained edge devices is challenging due to…
Autonomous NavigationObjectobject-detectionObject Detection