The FlEye camera: Sampling the joint distribution of natural scenes and motion
To make efficient use of limited physical resources, the brain must match its coding and computational strategies to the statistical structure of input signals. An attractive testing ground for these principles is the problem of motion estimation in the fly visual system: we understand the optics of the compound eye, have a quantitative description of input signals and noise from the retina, and can record from output neurons that encode estimates of different velocity components. Furthermore, recent work provides a nearly complete wiring diagram of the intervening circuitry. What is missing is a characterization of the visual signals and motions that flies encounter in a natural context. We attack this directly with the development of a specialized camera that matches the high temporal resolution, optical properties, and spectral sensitivity of the fly's eye; inertial motion sensors provide ground truth about rotations and translations through the world. We describe the design, construction, and performance characteristics of this FlEye camera. To illustrate the opportunities created by this instrument we use data on movies and motion to construct optimal local motion estimators that can be compared with the responses of the fly's motion sensitive neurons.
Code (0)
등록된 구현이 없습니다.
Tasks
Motion EstimationSimilar Papers 제목 키워드 기반
Pulp Motion: Framing-aware multimodal camera and human motion generation
Treating human motion and camera trajectory generation separately overlooks a core principle of cinematography: the tight interplay between actor performance and camera work in the screen space. In this paper, we are the…
TD-NeRF: Novel Truncated Depth Prior for Joint Camera Pose and Neural Radiance Field Optimization
The reliance on accurate camera poses is a significant barrier to the widespread deployment of Neural Radiance Fields (NeRF) models for 3D reconstruction and SLAM tasks. The existing method introduces monocular depth pri…
3D ReconstructionNeRFPose EstimationFocalPose++: Focal Length and Object Pose Estimation via Render and Compare
We introduce FocalPose++, a neural render-and-compare method for jointly estimating the camera-object 6D pose and camera focal length given a single RGB input image depicting a known object. The contributions of this wor…
ObjectPose EstimationNo Conditional Models for me: Training Joint EBMs on Mixed Continuous and Discrete Data
We propose energy-based models of the joint distribution of data and supervision. While challenging to work with, this approach gives flexibility when designing energy functions and easy parameterization for structured s…
Camera Pose Matters: Improving Depth Prediction by Mitigating Pose Distribution Bias
Monocular depth predictors are typically trained on large-scale training sets which are naturally biased w.r.t the distribution of camera poses. As a result, trained predictors fail to make reliable depth predictions for…
Data AugmentationDepth EstimationDepth Prediction