Recognizing Scenes from Novel Viewpoints
Humans can perceive scenes in 3D from a handful of 2D views. For AI agents, the ability to recognize a scene from any viewpoint given only a few images enables them to efficiently interact with the scene and its objects. In this work, we attempt to endow machines with this ability. We propose a model which takes as input a few RGB images of a new scene and recognizes the scene from novel viewpoints by segmenting it into semantic categories. All this without access to the RGB images from those views. We pair 2D scene recognition with an implicit 3D representation and learn from multi-view 2D annotations of hundreds of scenes without any 3D supervision beyond camera poses. We experiment on challenging datasets and demonstrate our model's ability to jointly capture semantics and geometry of novel scenes with diverse layouts, object types and shapes.
Code (0)
등록된 구현이 없습니다.
Tasks
Scene RecognitionSimilar Papers 제목 키워드 기반
Recognizing Actions in Videos from Unseen Viewpoints
Standard methods for video recognition use large CNNs designed to capture spatio-temporal data. However, training these models requires a large amount of labeled training data, containing a wide variety of actions, scene…
Action ClassificationAction RecognitionVideo RecognitionDeep Learning for Single-View Instance Recognition
Deep learning methods have typically been trained on large datasets in which many training examples are available. However, many real-world product datasets have only a small number of images available for each product. …
Deep LearningImproving Viewpoint-Independent Object-Centric Representations through Active Viewpoint Selection
Given the complexities inherent in visual scenes, such as object occlusion, a comprehensive understanding often requires observation from multiple viewpoints. Existing multi-viewpoint object-centric learning methods typi…
ObjectImproving Person Re-Identification via Pose-Aware Multi-Shot Matching
Person re-identification is the problem of recognizing people across images or videos from non-overlapping views. Although there has been much progress in person re-identification for the last decade, it still remains a …
Person Re-IdentificationPaMM: Pose-aware Multi-shot Matching for Improving Person Re-identification
Person re-identification is the problem of recognizing people across different images or videos with non-overlapping views. Although there has been much progress in person re-identification over the last decade, it remai…
Person Re-Identification