HyperNVD: Accelerating Neural Video Decomposition via Hypernetworks
Decomposing a video into a layer-based representation is crucial for easy video editing for the creative industries, as it enables independent editing of specific layers. Existing video-layer decomposition models rely on implicit neural representations (INRs) trained independently for each video, making the process time-consuming when applied to new videos. Noticing this limitation, we propose a meta-learning strategy to learn a generic video decomposition model to speed up the training on new videos. Our model is based on a hypernetwork architecture which, given a video-encoder embedding, generates the parameters for a compact INR-based neural video decomposition model. Our strategy mitigates the problem of single-video overfitting and, importantly, shortens the convergence of video decomposition on new, unseen videos. Our code is available at: https://hypernvd.github.io/
Code (0)
등록된 구현이 없습니다.
Tasks
Meta-LearningVideo EditingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
HyperAdaLoRA: Accelerating LoRA Rank Allocation During Training via Hypernetworks without Sacrificing Performance
Parameter-Efficient Fine-Tuning (PEFT), especially Low-Rank Adaptation (LoRA), has emerged as a promising approach to fine-tuning large language models(LLMs) while reducing computational and memory overhead. However, LoR…
parameter-efficient fine-tuningHyperE2VID: Improving Event-Based Video Reconstruction via Hypernetworks
Event-based cameras are becoming increasingly popular for their ability to capture high-speed motion with low latency and high dynamic range. However, generating videos from events remains challenging due to the highly s…
Event-Based Video ReconstructionVideo ReconstructionHyperNetworks
This work explores hypernetworks: an approach of using a one network, also known as a hypernetwork, to generate the weights for another network. Hypernetworks provide an abstraction that is similar to what is found in na…
Handwriting generationLanguage ModellingMachine TranslationTranslationLatent-INR: A Flexible Framework for Implicit Representations of Videos with Discriminative Semantics
Implicit Neural Networks (INRs) have emerged as powerful representations to encode all forms of data, including images, videos, audios, and scenes. With video, many INRs for video have been proposed for the compression t…
RetrievalVideo RetrievalEffect of Optimizer, Initializer, and Architecture of Hypernetworks on Continual Learning from Demonstration
In continual learning from demonstration (CLfD), a robot learns a sequence of real-world motion skills continually from human demonstrations. Recently, hypernetworks have been successful in solving this problem. In this …
Continual Learning