Exploring the parameter reusability of CNN
In recent times, using small data to train networks has become a hot topic in the field of deep learning. Reusing pre-trained parameters is one of the most important strategies to address the issue of semi-supervised and transfer learning. However, the fundamental reason for the success of these methods is still unclear. In this paper, we propose a solution that can not only judge whether a given network is reusable or not based on the performance of reusing convolution kernels but also judge which layers' parameters of the given network can be reused, based on the performance of reusing corresponding parameters and, ultimately, judge whether those parameters are reusable or not in a target task based on the root mean square error (RMSE) of the corresponding convolution kernels. Specifically, we define that the success of a CNN's parameter reuse depends upon two conditions: first, the network is a reusable network; and second, the RMSE between the convolution kernels from the source domain and target domain is small enough. The experimental results demonstrate that the performance of reused parameters applied to target tasks, when these conditions are met, is significantly improved.
Code (0)
등록된 구현이 없습니다.
Tasks
Semantic SegmentationTransfer LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
An Intermediate Data-driven Methodology for Scientific Workflow Management System to Support Reusability
In this thesis first we propose an intermediate data management scheme for a SWfMS. In our second attempt, we explored the possibilities and introduced an automatic recommendation technique for a SWfMS from real-world wo…
ManagementUsing UNet and PSPNet to explore the reusability principle of CNN parameters
How to reduce the requirement on training dataset size is a hot topic in deep learning community. One straightforward way is to reuse some pre-trained parameters. Some previous work like Deep transfer learning reuse the …
Transfer LearningOn the reusability of samples in active learning
An interesting but not extensively studied question in active learning is that of sample reusability: to what extent can samples selected for one learner be reused by another? This paper explains why sample reusability i…
Active LearningReusability and Transferability of Macro Actions for Reinforcement Learning
Conventional reinforcement learning (RL) typically determines an appropriate primitive action at each timestep. However, by using a proper macro action, defined as a sequence of primitive actions, an agent is able to byp…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Evaluating Chain-of-Thought Reasoning through Reusability and Verifiability
In multi-agent IR pipelines for tasks such as search and ranking, LLM-based agents exchange intermediate reasoning in terms of Chain-of-Thought (CoT) with each other. Current CoT evaluation narrowly focuses on target tas…