ATLASv2: LLM-Guided Adaptive Landmark Acquisition and Navigation on the Edge
Autonomous systems deployed on edge devices face significant challenges, including resource constraints, real-time processing demands, and adapting to dynamic environments. This work introduces ATLASv2, a novel system that integrates a fine-tuned TinyLLM, real-time object detection, and efficient path planning to enable hierarchical, multi-task navigation and manipulation all on the edge device, Jetson Nano. ATLASv2 dynamically expands its navigable landmarks by detecting and localizing objects in the environment which are saved to its internal knowledge base to be used for future task execution. We evaluate ATLASv2 in real-world environments, including a handcrafted home and office setting constructed with diverse objects and landmarks. Results show that ATLASv2 effectively interprets natural language instructions, decomposes them into low-level actions, and executes tasks with high success rates. By leveraging generative AI in a fully on-board framework, ATLASv2 achieves optimized resource utilization with minimal prompting latency and power consumption, bridging the gap between simulated environments and real-world applications.
Code (0)
등록된 구현이 없습니다.
Tasks
object-detectionObject DetectionReal-Time Object DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
AtlasVA: Self-Evolving Visual Skill Memory for Teacher-Free VLM Agents
Vision-language model (VLM) agents increasingly rely on memory-augmented reinforcement learning to reuse experience across long-horizon tasks, yet most existing frameworks store memory as text and depend on proprietary t…
Reinforcement LearningDecision MakingAnatomical Landmark-Guided Deep Reinforcement Learning for Autonomous Gastric Navigation
Wireless capsule endoscopy (WCE) enables painless visualization of the gastrointestinal tract, but its diagnostic potential is limited by incomplete mucosal coverage and poor transferability of existing navigation method…
Reinforcement LearningLandmark-Guided Knowledge for Vision-and-Language Navigation
Vision-and-language navigation is one of the core tasks in embodied intelligence, requiring an agent to autonomously navigate in an unfamiliar environment based on natural language instructions. However, existing methods…
AtlasVLA: Persistent World-Ego State Modeling for Vision-Language-Action Models
While Vision-Language-Action (VLA) models have advanced embodied AI, their fundamentally reactive paradigm severely limits performance in partially observable and long-horizon tasks. When restricted to a single wrist-mou…
Spatial ReasoningGrounded Entity-Landmark Adaptive Pre-training for Vision-and-Language Navigation
Cross-modal alignment is one key challenge for Vision-and-Language Navigation (VLN). Most existing studies concentrate on mapping the global instruction or single sub-instruction to the corresponding trajectory. However,…
cross-modal alignmentDescriptiveVision and Language Navigation