DyLiN: Making Light Field Networks Dynamic
Light Field Networks, the re-formulations of radiance fields to oriented rays, are magnitudes faster than their coordinate network counterparts, and provide higher fidelity with respect to representing 3D structures from 2D observations. They would be well suited for generic scene representation and manipulation, but suffer from one problem: they are limited to holistic and static scenes. In this paper, we propose the Dynamic Light Field Network (DyLiN) method that can handle non-rigid deformations, including topological changes. We learn a deformation field from input rays to canonical rays, and lift them into a higher dimensional space to handle discontinuities. We further introduce CoDyLiN, which augments DyLiN with controllable attribute inputs. We train both models via knowledge distillation from pretrained dynamic radiance fields. We evaluated DyLiN using both synthetic and real world datasets that include various non-rigid deformations. DyLiN qualitatively outperformed and quantitatively matched state-of-the-art methods in terms of visual fidelity, while being 25 - 71x computationally faster. We also tested CoDyLiN on attribute annotated data and it surpassed its teacher model. Project page: https://dylin2023.github.io .
Code (0)
등록된 구현이 없습니다.
Tasks
AttributeKnowledge DistillationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Beyond Static Vision: Scene Dynamic Field Unlocks Intuitive Physics Understanding in Multi-modal Large Language Models
While Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in image and video understanding, their ability to comprehend the physical world has become an increasingly important research focu…
Advecting Scaffolds: Controlling The Remodelling Of Actomyosin With Anillin
We propose and analyse an active hydrodynamic theory that characterises the effects of the scaffold protein anillin. Anillin is found at major sites of cortical activity, such as adherens junctions and the cytokinetic fu…
Temporal receptive field in dynamic graph learning: A comprehensive analysis
Dynamic link prediction is a critical task in the analysis of evolving networks, with applications ranging from recommender systems to economic exchanges. However, the concept of the temporal receptive field, which refer…
BenchmarkingDynamic Link PredictionGraph LearningLink Prediction+1Investigation of HIV-1 Gag binding with RNAs and Lipids using Atomic Force Microscopy
Atomic Force Microscopy was utilized to study the morphology of Gag, {\Psi}RNA, and their binding complexes with lipids in a solution environment with 0.1{\AA} vertical and 1nm lateral resolution. TARpolyA RNA was used a…
An Integrated Representation & Compression Scheme Based on Convolutional Autoencoders with 4D DCT Perceptual Encoding for High Dynamic Range Light Fields
The emerging and existing light field displays are highly capable of realistic presentation of 3D scenes on auto-stereoscopic glasses-free platforms. The light field size is a major drawback while utilising 3D displays a…