OmniFlow: Human Omnidirectional Optical Flow
Optical flow is the motion of a pixel between at least two consecutive video frames and can be estimated through an end-to-end trainable convolutional neural network. To this end, large training datasets are required to improve the accuracy of optical flow estimation. Our paper presents OmniFlow: a new synthetic omnidirectional human optical flow dataset. Based on a rendering engine we create a naturalistic 3D indoor environment with textured rooms, characters, actions, objects, illumination and motion blur where all components of the environment are shuffled during the data capturing process. The simulation has as output rendered images of household activities and the corresponding forward and backward optical flow. To verify the data for training volumetric correspondence networks for optical flow estimation we train different subsets of the data and test on OmniFlow with and without Test-Time-Augmentation. As a result we have generated 23,653 image pairs and corresponding forward and backward optical flow. Our dataset can be downloaded from: https://mytuc.org/byfs
Code (0)
등록된 구현이 없습니다.
Tasks
Optical Flow EstimationSimilar Papers 제목 키워드 기반
PanoFlow: Learning 360° Optical Flow for Surrounding Temporal Understanding
Optical flow estimation is a basic task in self-driving and robotics systems, which enables to temporally interpret traffic scenes. Autonomous vehicles clearly benefit from the ultra-wide Field of View (FoV) offered by 3…
Autonomous VehiclesOptical Flow EstimationLearning Omnidirectional Flow in 360-degree Video via Siamese Representation
Optical flow estimation in omnidirectional videos faces two significant issues: the lack of benchmark datasets and the challenge of adapting perspective video-based methods to accommodate the omnidirectional nature. This…
DiversityOptical Flow EstimationRepresentation LearningOptical Flow Estimation in 360$^\circ$ Videos: Dataset, Model and Application
Optical flow estimation has been a long-lasting and fundamental problem in the computer vision community. However, despite the advances of optical flow estimation in perspective videos, the 360$^\circ$ videos counterpart…
Activity RecognitionEgocentric Activity RecognitionOptical Flow EstimationRepresentation LearningOMNIFLOW: A Physics-Grounded Multimodal Agent for Generalized Scientific Reasoning
Large Language Models (LLMs) have demonstrated exceptional logical reasoning capabilities but frequently struggle with the continuous spatiotemporal dynamics governed by Partial Differential Equations (PDEs), often resul…
Zero-shot GeneralizationDomain GeneralizationWeather ForecastingLogical ReasoningOmniFlow: Any-to-Any Generation with Multi-Modal Rectified Flows
We introduce OmniFlow, a novel generative model designed for any-to-any generation tasks such as text-to-image, text-to-audio, and audio-to-image synthesis. OmniFlow advances the rectified flow (RF) framework used in tex…
Audio SynthesisImage GenerationText Generation