MixLoRA

2000년 도입 · 논문 1편에서 사용

MixLoRA is a type of PEFT method which construct a resource-efficient sparse MoE model based on LoRA.

출처: MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts

소개 논문: MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts