Visual Odometry with Neuromorphic Resonator Networks
Visual Odometry (VO) is a method to estimate self-motion of a mobile robot using visual sensors. Unlike odometry based on integrating differential measurements that can accumulate errors, such as inertial sensors or wheel encoders, visual odometry is not compromised by drift. However, image-based VO is computationally demanding, limiting its application in use cases with low-latency, -memory, and -energy requirements. Neuromorphic hardware offers low-power solutions to many vision and AI problems, but designing such solutions is complicated and often has to be assembled from scratch. Here we propose to use Vector Symbolic Architecture (VSA) as an abstraction layer to design algorithms compatible with neuromorphic hardware. Building from a VSA model for scene analysis, described in our companion paper, we present a modular neuromorphic algorithm that achieves state-of-the-art performance on two-dimensional VO tasks. Specifically, the proposed algorithm stores and updates a working memory of the presented visual environment. Based on this working memory, a resonator network estimates the changing location and orientation of the camera. We experimentally validate the neuromorphic VSA-based approach to VO with two benchmarks: one based on an event camera dataset and the other in a dynamic scene with a robotic task.
Code (0)
등록된 구현이 없습니다.
Tasks
Motion EstimationNavigateSimultaneous Localization and MappingVisual OdometrySimilar Papers 제목 키워드 기반
Neuromorphic Visual Scene Understanding with Resonator Networks
Analyzing a visual scene by inferring the configuration of a generative model is widely considered the most flexible and generalizable approach to scene understanding. Yet, one major problem is the computational challeng…
Scene UnderstandingTranslationRange and Angle Estimation with Spiking Neural Resonators for FMCW Radar
Automotive radar systems face the challenge of managing high sampling rates and large data bandwidth while complying with stringent real-time and energy efficiency requirements. The growing complexity of autonomous vehic…
Autonomous VehiclesPCA Event-Based Optical Flow for Visual Odometry
With the advent of neuromorphic vision sensors such as event-based cameras, a paradigm shift is required for most computer vision algorithms. Among these algorithms, optical flow estimation is a prime candidate for this …
Event-based Optical FlowOptical Flow EstimationVisual OdometrySqueezeLight: Towards Scalable Optical Neural Networks with Multi-Operand Ring Resonators
Optical neural networks (ONNs) have demonstrated promising potentials for next-generation artificial intelligence acceleration with ultra-low latency, high bandwidth, and low energy consumption. However, due to high area…
Interfacing spiking VCSEL-neurons with silicon photonics weight banks towards integrated neuromorphic photonic systems
Spiking neurons and neural networks constitute a fundamental building block for brain-inspired computing, which is posed to benefit significantly from photonic hardware implementations. In this work, we experimentally in…