EgoPet: Egomotion and Interaction Data from an Animal's Perspective
Animals perceive the world to plan their actions and interact with other agents to accomplish complex tasks, demonstrating capabilities that are still unmatched by AI systems. To advance our understanding and reduce the gap between the capabilities of animals and AI systems, we introduce a dataset of pet egomotion imagery with diverse examples of simultaneous egomotion and multi-agent interaction. Current video datasets separately contain egomotion and interaction examples, but rarely both at the same time. In addition, EgoPet offers a radically distinct perspective from existing egocentric datasets of humans or vehicles. We define two in-domain benchmark tasks that capture animal behavior, and a third benchmark to assess the utility of EgoPet as a pretraining resource to robotic quadruped locomotion, showing that models trained from EgoPet outperform those trained from prior datasets.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
LegoPET: Hierarchical Feature Guided Conditional Diffusion for PET Image Reconstruction
Positron emission tomography (PET) is widely utilized for cancer detection due to its ability to visualize functional and biological processes in vivo. PET images are usually reconstructed from histogrammed raw data (sin…
Image ReconstructionSSIMEgoPrune: Efficient Token Pruning for Egomotion Video Reasoning in Embodied Agent
Egomotion videos are first-person recordings where the view changes continuously due to the agent's movement. As they serve as the primary visual input for embodied AI agents, making egomotion video reasoning more effici…
Multimodal ReasoningFast, Robust, Continuous Monocular Egomotion Computation
We propose robust methods for estimating camera egomotion in noisy, real-world monocular image sequences in the general case of unknown observer rotation and translation with two views and a small baseline. This is a dif…
counterfactualMotion EstimationOptical Flow EstimationTranslationUniEgoMotion: A Unified Model for Egocentric Motion Reconstruction, Forecasting, and Generation
Egocentric human motion generation and forecasting with scene-context is crucial for enhancing AR/VR experiences, improving human-robot interaction, advancing assistive technologies, and enabling adaptive healthcare solu…
Motion ForecastingMotion SynthesisOn the Coupling of Depth and Egomotion Networks for Self-Supervised Structure from Motion
Structure from motion (SfM) has recently been formulated as a self-supervised learning problem, where neural network models of depth and egomotion are learned jointly through view synthesis. Herein, we address the open p…
Self-Supervised Learning