IC Networks: Remodeling the Basic Unit for Convolutional Neural Networks
Convolutional neural network (CNN) is a class of artificial neural networks widely used in computer vision tasks. Most CNNs achieve excellent performance by stacking certain types of basic units. In addition to increasing the depth and width of the network, designing more effective basic units has become an important research topic. Inspired by the elastic collision model in physics, we present a general structure which can be integrated into the existing CNNs to improve their performance. We term it the "Inter-layer Collision" (IC) structure. Compared to the traditional convolution structure, the IC structure introduces nonlinearity and feature recalibration in the linear convolution operation, which can capture more fine-grained features. In addition, a new training method, namely weak logit distillation (WLD), is proposed to speed up the training of IC networks by extracting knowledge from pre-trained basic models. In the ImageNet experiment, we integrate the IC structure into ResNet-50 and reduce the top-1 error from 22.38% to 21.75%, which also catches up the top-1 error of ResNet-100 (21.75%) with nearly half of FLOPs.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Learning Interpretable Anatomical Features Through Deep Generative Models: Application to Cardiac Remodeling
Alterations in the geometry and function of the heart define well-established causes of cardiovascular disease. However, current approaches to the diagnosis of cardiovascular diseases often rely on subjective human asses…
DiagnosticGeneral ClassificationConvTimeNet: A Deep Hierarchical Fully Convolutional Model for Multivariate Time Series Analysis
Designing effective models for learning time series representations is foundational for time series analysis. Many previous works have explored time series representation modeling approaches and have made progress in thi…
Time SeriesTime Series AnalysisTime Series ForecastingIC-Network: Efficient Structure for Convolutional Neural Networks
Neural networks have been widely used, and most networks achieve excellent performance by stacking certain types of basic units. Compared to increasing the depth and width of the network, designing more effective basic u…
General ClassificationImage ClassificationAnalyzing ribosome remodeling in health and disease
regulation largely unexplored, in part due to methodological limitations. Indeed, we review evidence demonstrating that commonly used methods, such as transcriptomics, are inadequate because the variability in mRNAs codi…
A role for ATP-dependent chromatin remodeling in the hierarchical cooperativity between noninteracting transcription factors
Chromatin remodeling machineries are abundant and diverse in eukaryotic cells. They have been involved in a variety of situations such as histone exchange and DNA repair, but their importance in gene expression remains u…
Position