Cascaded Multi-task Adaptive Learning Based on Neural Architecture Search
Cascading multiple pre-trained models is an effective way to compose an end-to-end system. However, fine-tuning the full cascaded model is parameter and memory inefficient and our observations reveal that only applying adapter modules on cascaded model can not achieve considerable performance as fine-tuning. We propose an automatic and effective adaptive learning method to optimize end-to-end cascaded multi-task models based on Neural Architecture Search (NAS) framework. The candidate adaptive operations on each specific module consist of frozen, inserting an adapter and fine-tuning. We further add a penalty item on the loss to limit the learned structure which takes the amount of trainable parameters into account. The penalty item successfully restrict the searched architecture and the proposed approach is able to search similar tuning scheme with hand-craft, compressing the optimizing parameters to 8.7% corresponding to full fine-tuning on SLURP with an even better performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Neural Architecture SearchMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
SpeechCLIP+: Self-supervised multi-task representation learning for speech via CLIP and speech-image data
The recently proposed visually grounded speech model SpeechCLIP is an innovative framework that bridges speech and text through images via CLIP without relying on text transcription. On this basis, this paper introduces …
Keyword ExtractionMulti-Task LearningRepresentation LearningRetrievalFace alignment by coarse-to-fine shape searching
We present a novel face alignment framework based on coarse-to-fine shape searching. Unlike the conventional cascaded regression approaches that start with an initial shape and refine the shape in a cascaded manner, our …
Face AlignmentregressionMultiview Stereo with Cascaded Epipolar RAFT
We address multiview stereo (MVS), an important 3D vision task that reconstructs a 3D model such as a dense point cloud from multiple calibrated images. We propose CER-MVS (Cascaded Epipolar RAFT Multiview Stereo), a new…
Optical Flow EstimationSliding Mode Control of Active Magnetic Bearings -- A Cascaded Architecture
Accurate and robust positioning of rotor axle is essential for efficient and safe operation of high-speed rotational machines with active magnetic bearings. This study presents a cascaded nonlinear control strategy for v…
PositionCascaded Semantic and Positional Self-Attention Network for Document Classification
Transformers have shown great success in learning representations for language modelling. However, an open challenge still remains on how to systematically aggregate semantic information (word embedding) with positional …
ClassificationDocument ClassificationGeneral ClassificationLanguage Modelling