AICL: Action In-Context Learning for Video Diffusion Model
The open-domain video generation models are constrained by the scale of the training video datasets, and some less common actions still cannot be generated. Some researchers explore video editing methods and achieve action generation by editing the spatial information of the same action video. However, this method mechanically generates identical actions without understanding, which does not align with the characteristics of open-domain scenarios. In this paper, we propose AICL, which empowers the generative model with the ability to understand action information in reference videos, similar to how humans do, through in-context learning. Extensive experiments demonstrate that AICL effectively captures the action and achieves state-of-the-art generation performance across three typical video diffusion models on five metrics when using randomly selected categories from non-training datasets.
Code (1)
Tasks
Action GenerationIn-Context LearningVideo EditingVideo GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
SAICL: Student Modelling with Interaction-level Auxiliary Contrastive Tasks for Knowledge Tracing and Dropout Prediction
Knowledge tracing and dropout prediction are crucial for online education to estimate students' knowledge states or to prevent dropout rates. While traditional systems interacting with students suffered from data sparsit…
Contrastive LearningData AugmentationKnowledge TracingMetaICL: Learning to Learn In Context
We introduce MetaICL (Meta-training for In-Context Learning), a new meta-training framework for few-shot learning where a pretrained language model is tuned to do in-context learning on a large set of training tasks. Thi…
Few-Shot LearningIn-Context LearningLanguage ModellingMulti-Task Learning+2Scaling In-Context Demonstrations with Structured Attention
The recent surge of large language models (LLMs) highlights their ability to perform in-context learning, i.e., "learning" to perform a task from a few demonstrations in the context without any parameter updates. However…
DecoderIn-Context LearningSentenceParaICL: Towards Parallel In-Context Learning
Large language models (LLMs) have become the norm in natural language processing (NLP), excelling in few-shot in-context learning (ICL) with their remarkable abilities. Nonetheless, the success of ICL largely hinges on t…
In-Context LearningSemantic SimilaritySemantic Textual SimilarityMetaICL: Learning to Learn In Context
We introduce MetaICL (Meta-training for In-Context Learning), a new meta-training framework for few-shot learning where a pretrained language model is tuned to do in-context learning on a large set of training tasks. Thi…
Few-Shot LearningIn-Context LearningLanguage ModelingLanguage Modelling+3