VGF-Net: Visual-Geometric Fusion Learning for Simultaneous Drone Navigation and Height Mapping
The drone navigation requires the comprehensive understanding of both visual and geometric information in the 3D world. In this paper, we present a Visual-Geometric Fusion Network(VGF-Net), a deep network for the fusion analysis of visual/geometric data and the construction of 2.5D height maps for simultaneous drone navigation in novel environments. Given an initial rough height map and a sequence of RGB images, our VGF-Net extracts the visual information of the scene, along with a sparse set of 3D keypoints that capture the geometric relationship between objects in the scene. Driven by the data, VGF-Net adaptively fuses visual and geometric information, forming a unified Visual-Geometric Representation. This representation is fed to a new Directional Attention Model(DAM), which helps enhance the visual-geometric object relationship and propagates the informative data to dynamically refine the height map and the corresponding keypoints. An entire end-to-end information fusion and mapping system is formed, demonstrating remarkable robustness and high accuracy on the autonomous drone navigation across complex indoor and large-scale outdoor scenes. The dataset can be found in http://vcc.szu.edu.cn/research/2021/VGFNet.
Code (0)
등록된 구현이 없습니다.
Tasks
Drone navigationSimilar Papers 제목 키워드 기반
A3D: Adaptive, Accurate, and Autonomous Navigation for Edge-Assisted Drones
Accurate navigation is of paramount importance to ensure flight safety and efficiency for autonomous drones. Recent research starts to use Deep Neural Networks to enhance drone navigation given their remarkable predictiv…
Autonomous NavigationDeep Reinforcement LearningDrone navigationImage CompressionSimultaneous Localization and 3D-Semi Dense Mapping for Micro Drones Using Monocular Camera and Inertial Sensors
Monocular simultaneous localization and mapping (SLAM) algorithms estimate drone poses and build a 3D map using a single camera. Current algorithms include sparse methods that lack detailed geometry, while learning-drive…
Pose EstimationEdge DetectionVLN-Pilot: Large Vision-Language Model as an Autonomous Indoor Drone Operator
This paper introduces VLN-Pilot, a novel framework in which a large Vision-and-Language Model (VLLM) assumes the role of a human pilot for indoor drone navigation. By leveraging the multimodal reasoning abilities of VLLM…
Multimodal ReasoningDrone navigationUrban Drone Navigation: Autoencoder Learning Fusion for Aerodynamics
Drones are vital for urban emergency search and rescue (SAR) due to the challenges of navigating dynamic environments with obstacles like buildings and wind. This paper presents a method that combines multi-objective rei…
Drone navigationMulti-Objective Reinforcement Learningreinforcement-learningTowards Perception-based Collision Avoidance for UAVs when Guiding the Visually Impaired
Autonomous navigation by drones using onboard sensors combined with machine learning and computer vision algorithms is impacting a number of domains, including agriculture, logistics, and disaster management. In this pap…
Autonomous NavigationCollision AvoidanceManagement