SceneNet RGB-D: 5M Photorealistic Images of Synthetic Indoor Trajectories with Ground Truth

2016-12-15 · John McCormac, Ankur Handa, Stefan Leutenegger, Andrew J. Davison

We introduce SceneNet RGB-D, expanding the previous work of SceneNet to enable large scale photorealistic rendering of indoor scene trajectories. It provides pixel-perfect ground truth for scene understanding problems such as semantic segmentation, instance segmentation, and object detection, and also for geometric computer vision problems such as optical flow, depth estimation, camera pose estimation, and 3D reconstruction. Random sampling permits virtually unlimited scene configurations, and here we provide a set of 5M rendered RGB-D images from over 15K trajectories in synthetic layouts with random but physically simulated object poses. Each layout also has random lighting, camera trajectories, and textures. The scale of this dataset is well suited for pre-training data-driven computer vision techniques from scratch with RGB-D inputs, which previously has been limited by relatively small labelled datasets in NYUv2 and SUN RGB-D. It also provides a basis for investigating 3D scene labelling tasks by providing perfect camera poses and depth data as proxy for a SLAM system. We host the dataset at http://robotvault.bitbucket.io/scenenet-rgbd.html

📄 PDF Abstract BibTeX arXiv:1612.05079

Code (1)

fzi-forschungszentrum-informatik/mrf

Tasks

3D Reconstruction Camera Pose Estimation Depth Estimation Instance Segmentation object-detection Object Detection Optical Flow Estimation Pose Estimation Scene Understanding Semantic Segmentation

SceneNet RGB-D: Can 5M Synthetic Images Beat Generic ImageNet Pre-Training on Indoor Segmentation?

2017-10-01 · ICCV 2017 10 · John McCormac, Ankur Handa, Stefan Leutenegger, Andrew J. Davison

We introduce SceneNet RGB-D, a dataset providing pixel-perfect ground truth for scene understanding problems such as semantic segmentation, instance segmentation, and object detection. It also provides perfect camera pos…

16k Camera Pose Estimation Instance Segmentation object-detection+6

SegmATRon: Embodied Adaptive Semantic Segmentation for Indoor Environment

2023-10-18 · Tatiana Zemskova, Margarita Kichik, Dmitry Yudin, Aleksei Staroverov 외

This paper presents an adaptive transformer model named SegmATRon for embodied image semantic segmentation. Its distinctive feature is the adaptation of model weights during inference on several images using a hybrid mul…

Segmentation Semantic Segmentation

OpenRooms: An Open Framework for Photorealistic Indoor Scene Datasets

2021-06-19 · CVPR 2021 1 · Zhengqin Li, Ting-Wei Yu, Shen Sang, Sarah Wang 외

We propose a novel framework for creating large-scale photorealistic datasets of indoor scenes, with ground truth geometry, material, lighting and semantics. Our goal is to make the dataset creation process widely ac…

Friction Inverse Rendering Lighting Estimation Scene Understanding

PerspectiveNet: A Scene-consistent Image Generator for New View Synthesis in Real Indoor Environments

2019-12-01 · NeurIPS 2019 12 · Ben Graham, David Novotny, Jeremy Reizenstein

Given a set of a reference RGBD views of an indoor environment, and a new viewpoint, our goal is to predict the view from that location. Prior work on new-view generation has predominantly focused on significantly constr…

Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding

2020-11-04 · ICCV 2021 10 · Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar 외

For many fundamental scene understanding tasks, it is difficult or impossible to obtain per-pixel ground truth labels from real images. We address this challenge by introducing Hypersim, a photorealistic synthetic datase…

Multi-Task Learning Scene Understanding Semantic Segmentation