Animal Kingdom: A Large and Diverse Dataset for Animal Behavior Understanding
Understanding animals' behaviors is significant for a wide range of applications. However, existing animal behavior datasets have limitations in multiple aspects, including limited numbers of animal classes, data samples and provided tasks, and also limited variations in environmental conditions and viewpoints. To address these limitations, we create a large and diverse dataset, Animal Kingdom, that provides multiple annotated tasks to enable a more thorough understanding of natural animal behaviors. The wild animal footages used in our dataset record different times of the day in extensive range of environments containing variations in backgrounds, viewpoints, illumination and weather conditions. More specifically, our dataset contains 50 hours of annotated videos to localize relevant animal behavior segments in long videos for the video grounding task, 30K video sequences for the fine-grained multi-label action recognition task, and 33K frames for the pose estimation task, which correspond to a diverse range of animals with 850 species across 6 major animal classes. Such a challenging and comprehensive dataset shall be able to facilitate the community to develop, adapt, and evaluate various types of advanced methods for animal behavior analysis. Moreover, we propose a Collaborative Action Recognition (CARe) model that learns general and specific features for action recognition with unseen new animals. This method achieves promising performance in our experiments. Our dataset can be found at https://sutdcv.github.io/Animal-Kingdom.
Code (1)
Tasks
Action RecognitionAnimal Action RecognitionAnimal Pose EstimationVideo GroundingSimilar Papers 제목 키워드 기반
SAM 3D Animal: Promptable Animal 3D Reconstruction from Images in the Wild
3D animal reconstruction in the wild remains challenging due to large species variation, frequent occlusions, and the prevalence of multi-animal scenes, while existing methods predominantly focus on single-animal setting…
3D ReconstructionOn the Order of Gene Distribution on Chromosomes Across the Animal Kingdom
Background. The large-scale pattern of distribution of genes on the chromosomes in the known animal genomes is not well characterized. We hypothesized that individual genes will be distributed on chromosomes in a mathema…
AniMer: Animal Pose and Shape Estimation Using Family Aware Transformer
Quantitative analysis of animal behavior and biomechanics requires accurate animal pose and shape estimation across species, and is important for animal welfare and biological research. However, the small network capacit…
Conditional Image GenerationContrastive LearningImage GenerationMARINE: A Computer Vision Model for Detecting Rare Predator-Prey Interactions in Animal Videos
Encounters between predator and prey play an essential role in ecosystems, but their rarity makes them difficult to detect in video recordings. Although advances in action recognition (AR) and temporal action detection (…
Action DetectionAction RecognitionPRIMA: Boosting Animal Mesh Recovery with Biological Priors and Test-Time Adaptation
We present PRIMA (*PRI*ors for *M*esh *A*daptation), a framework for robust 3D quadruped mesh recovery under severe species and pose imbalance. Existing animal reconstruction methods often regress toward mean shapes and …
Test-time Adaptation