STARC: See-Through-Wall Augmented Reality Framework for Human-Robot Collaboration in Emergency Response
In emergency response missions, first responders must navigate cluttered indoor environments where occlusions block direct line-of-sight, concealing both life-threatening hazards and victims in need of rescue. We present STARC, a see-through AR framework for human-robot collaboration that fuses mobile-robot mapping with responder-mounted LiDAR sensing. A ground robot running LiDAR-inertial odometry performs large-area exploration and 3D human detection, while helmet- or handheld-mounted LiDAR on the responder is registered to the robot's global map via relative pose estimation. This cross-LiDAR alignment enables consistent first-person projection of detected humans and their point clouds - rendered in AR with low latency - into the responder's view. By providing real-time visualization of hidden occupants and hazards, STARC enhances situational awareness and reduces operator risk. Experiments in simulation, lab setups, and tactical field trials confirm robust pose alignment, reliable detections, and stable overlays, underscoring the potential of our system for fire-fighting, disaster relief, and other safety-critical operations. Code and design will be open-sourced upon acceptance.
Code (0)
등록된 구현이 없습니다.
Tasks
Pose EstimationPoint CloudsSimilar Papers 제목 키워드 기반
See Behind Walls in Real-time Using Aerial Drones and Augmented Reality
This work presents ARD2, a framework that enables real-time through-wall surveillance using two aerial drones and an augmented reality (AR) device. ARD2 consists of two main steps: target direction estimation and contour…
Gaze-Vergence-Controlled See-Through Vision in Augmented Reality
Augmented Reality (AR) see-through vision is an interesting research topic since it enables users to see through a wall and see the occluded objects. Most existing research focuses on the visual effects of see-through vi…
Depth EstimationFollow the Saliency: Supervised Saliency for Retrieval-augmented Dense Video Captioning
Existing retrieval-augmented approaches for Dense Video Captioning (DVC) often fail to achieve accurate temporal segmentation aligned with true event boundaries, as they rely on heuristic strategies that overlook ground …
Dense Video CaptioningHighlight DetectionImplementing a Wall-In Building Placement in StarCraft with Declarative Programming
In real-time strategy games like StarCraft, skilled players often block the entrance to their base with buildings to prevent the opponent's units from getting inside. This technique, called "walling-in", is a vital part …
Real-Time Strategy GamesStarcraftDeepPerimeter: Indoor Boundary Estimation from Posed Monocular Sequences
We present DeepPerimeter, a deep learning based pipeline for inferring a full indoor perimeter (i.e. exterior boundary map) from a sequence of posed RGB images. Our method relies on robust deep methods for depth estimati…
ClusteringDepth Estimation