Implementing Adaptations for Vision AutoRegressive Model
Vision AutoRegressive model (VAR) was recently introduced as an alternative to Diffusion Models (DMs) in image generation domain. In this work we focus on its adaptations, which aim to fine-tune pre-trained models to perform specific downstream tasks, like medical data generation. While for DMs there exist many techniques, adaptations for VAR remain underexplored. Similarly, differentially private (DP) adaptations-ones that aim to preserve privacy of the adaptation data-have been extensively studied for DMs, while VAR lacks such solutions. In our work, we implement and benchmark many strategies for VAR, and compare them to state-of-the-art DM adaptation strategies. We observe that VAR outperforms DMs for non-DP adaptations, however, the performance of DP suffers, which necessitates further research in private adaptations for VAR. Code is available at https://github.com/sprintml/finetuning_var_dp.
Code (1)
Tasks
Image GenerationmodelMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Reward-Forcing: Autoregressive Video Generation with Reward Feedback
While most prior work in video generation relies on bidirectional architectures, recent efforts have sought to adapt these models into autoregressive variants to support near real-time generation. However, such adaptatio…
Video GenerationEstimation methods of Matrix-valued AR model
This article proposes novel estimation methods for the Matrix Autoregressive (MAR) model, specifically adaptations of the Yule-Walker equations and Burg's method, addressing limitations in existing techniques. The MAR mo…
modelTime SeriesFast-dVLM: Efficient Block-Diffusion VLM via Direct Conversion from Autoregressive VLM
Vision-language models (VLMs) predominantly rely on autoregressive decoding, which generates tokens one at a time and fundamentally limits inference throughput. This limitation is especially acute in physical AI scenario…
Autonomous DrivingText GenerationImproving image classifiers for small datasets by learning rate adaptations
Our paper introduces an efficient combination of established techniques to improve classifier performance, in terms of accuracy and training time. We achieve two-fold to ten-fold speedup in nearing state of the art accur…
DiagnosticXLM: A Python package for non-autoregressive language models
In recent years, there has been a resurgence of interest in non-autoregressive text generation in the context of general language modeling. Unlike the well-established autoregressive language modeling paradigm, which has…
Text Generation