Child-Tuning

2000년 도입 · 논문 1편에서 사용

Child-Tuning is a fine-tuning technique that updates a subset of parameters (called child network) of large pretrained models via strategically masking out the gradients of the non-child network during the backward process. It decreases the hypothesis space of the model via a task-specific mask applied to the full gradients, helping to effectively adapt the large-scale pretrained model to various tasks and meanwhile aiming to maintain its original generalization ability.

출처: Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning

소개 논문: Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning