Who Let The Dogs Out? Modeling Dog Behavior From Visual Data
We introduce the task of directly modeling a visually intelligent agent. Computer vision typically focuses on solving various subtasks related to visual intelligence. We depart from this standard approach to computer vision; instead we directly model a visually intelligent agent. Our model takes visual information as input and directly predicts the actions of the agent. Toward this end we introduce DECADE, a large-scale dataset of ego-centric videos from a dog's perspective as well as her corresponding movements. Using this data we model how the dog acts and how the dog plans her movements. We show under a variety of metrics that given just visual input we can successfully model this intelligent agent in many situations. Moreover, the representation learned by our model encodes distinct information compared to representations trained on image classification, and our learned representation can generalize to other domains. In particular, we show strong results on the task of walkable surface estimation by using this dog modeling task as representation learning.
Code (1)
Tasks
image-classificationRepresentation LearningSimilar Papers 제목 키워드 기반
Categorizing the Visual Environment and Analyzing the Visual Attention of Dogs
Dogs have a unique evolutionary relationship with humans and serve many important roles e.g. search and rescue, blind assistance, emotional support. However, few datasets exist to categorize visual features and objects a…
Fair game: Urban free-ranging dogs balance resource use and risk aversion at seasonal fairs
Seasonal fairs, bustling with human activity, provide a unique environment for exploring the intricate interplay between humans and free-ranging dogs. This study investigated these interactions during seasonal fairs in N…
Identifying Individual Dogs in Social Media Images
We present the results of an initial study focused on developing a visual AI solution able to recognize individual dogs in unconstrained (wild) images occurring on social media. The work described here is part of joint p…
object-detectionObject DetectionTransfer LearningDigitally-Enhanced Dog Behavioral Testing: Getting Help from the Machine
The assessment of behavioral traits in dogs is a well-studied challenge due to its many practical applications such as selection for breeding, prediction of working aptitude, chances of being adopted, etc. Most methods f…
ClusteringLearning Hierarchical Visual Representations in Deep Neural Networks Using Hierarchical Linguistic Labels
Modern convolutional neural networks (CNNs) are able to achieve human-level object classification accuracy on specific tasks, and currently outperform competing models in explaining complex human visual representations. …