VizFlyt: Perception-centric Pedagogical Framework For Autonomous Aerial Robots
Autonomous aerial robots are becoming commonplace in our lives. Hands-on aerial robotics courses are pivotal in training the next-generation workforce to meet the growing market demands. Such an efficient and compelling course depends on a reliable testbed. In this paper, we present VizFlyt, an open-source perception-centric Hardware-In-The-Loop (HITL) photorealistic testing framework for aerial robotics courses. We utilize pose from an external localization system to hallucinate real-time and photorealistic visual sensors using 3D Gaussian Splatting. This enables stress-free testing of autonomy algorithms on aerial robots without the risk of crashing into obstacles. We achieve over 100Hz of system update rate. Lastly, we build upon our past experiences of offering hands-on aerial robotics courses and propose a new open-source and open-hardware curriculum based on VizFlyt for the future. We test our framework on various course projects in real-world HITL experiments and present the results showing the efficacy of such a system and its large potential use cases. Code, datasets, hardware guides and demo videos are available at https://pear.wpi.edu/research/vizflyt.html
Code (1)
Similar Papers 제목 키워드 기반
Grid-Centric Traffic Scenario Perception for Autonomous Driving: A Comprehensive Review
Grid-centric perception is a crucial field for mobile robot perception and navigation. Nonetheless, grid-centric perception is less prevalent than object-centric perception as autonomous vehicles need to accurately perce…
Autonomous DrivingAutonomous VehiclesSensor FusionUniVision: A Unified Framework for Vision-Centric 3D Perception
The past few years have witnessed the rapid development of vision-centric 3D perception in autonomous driving. Although the 3D perception models share many structural and conceptual similarities, there still exist gaps i…
Autonomous DrivingData Augmentationobject-detectionObject DetectionUniDrive: Towards Universal Driving Perception Across Camera Configurations
Vision-centric autonomous driving has demonstrated excellent performance with economical sensors. As the fundamental step, 3D perception aims to infer 3D information from 2D images based on 3D-2D projection. This makes d…
Autonomous DrivingAre We Ready for Vision-Centric Driving Streaming Perception? The ASAP Benchmark
In recent years, vision-centric perception has flourished in various autonomous driving tasks, including 3D detection, semantic map construction, motion forecasting, and depth estimation. Nevertheless, the latency of vis…
Autonomous DrivingDepth EstimationMotion ForecastingLeveraging Large Language Models for Enhancing Autonomous Vehicle Perception
Autonomous vehicles (AVs) rely on sophisticated perception systems to interpret their surroundings, a cornerstone for safe navigation and decision-making. The integration of Large Language Models (LLMs) into AV perceptio…
Autonomous DrivingAutonomous VehiclesDecision MakingSensor Fusion