Adaptor: Objective-Centric Adaptation Framework for Language Models
Progress in natural language processing research is catalyzed by the possibilities given by the widespread software frameworks. This paper introduces Adaptor library that transposes the traditional model-centric approach composed of pre-training + fine-tuning steps to objective-centric approach, composing the training process by applications of selected objectives. We survey research directions that can benefit from enhanced objective-centric experimentation in multitask training, custom objectives development, dynamic training curricula, or domain adaptation. Adaptor aims to ease reproducibility of these research directions in practice. Finally, we demonstrate the practical applicability of Adaptor in selected unsupervised domain adaptation scenarios.
Code (1)
Tasks
Domain AdaptationUnsupervised Domain AdaptationSimilar Papers 제목 키워드 기반
All-in-One Transferring Image Compression from Human Perception to Multi-Machine Perception
Efficiently transferring Learned Image Compression (LIC) model from human perception to machine perception is an emerging challenge in vision-centric representation learning. Existing approaches typically adapt LIC to do…
AllImage CompressionRepresentation LearningSkillAdaptor: Self-Adapting Skills for LLM Agents from Trajectories
Large language model (LLM) agents increasingly rely on reusable external skills to solve long-horizon interactive tasks. Existing training-free skill adaptation pipelines usually update skills from full trajectories or s…
Self-Domain Adaptation for Face Anti-Spoofing
Although current face anti-spoofing methods achieve promising results under intra-dataset testing, they suffer from poor generalization to unseen attacks. Most existing works adopt domain adaptation (DA) or domain genera…
Domain AdaptationDomain GeneralizationFace Anti-SpoofingMeta-LearningSODA: Robust Training of Test-Time Data Adaptors
Adapting models deployed to test distributions can mitigate the performance degradation caused by distribution shifts. However, privacy concerns may render model parameters inaccessible. One promising approach involves u…
Collaborative and Efficient Personalization with Mixtures of Adaptors
Non-iid data is prevalent in real-world federated learning problems. Data heterogeneity can come in different types in terms of distribution shifts. In this work, we are interested in the heterogeneity that comes from co…
Federated LearningMulti-Task Learning