Flattened Convolutional Neural Networks for Feedforward Acceleration
We present flattened convolutional neural networks that are designed for fast feedforward execution. The redundancy of the parameters, especially weights of the convolutional filters in convolutional neural networks has been extensively studied and different heuristics have been proposed to construct a low rank basis of the filters after training. In this work, we train flattened networks that consist of consecutive sequence of one-dimensional filters across all directions in 3D space to obtain comparable performance as conventional convolutional networks. We tested flattened model on different datasets and found that the flattened layer can effectively substitute for the 3D filters without loss of accuracy. The flattened convolution pipelines provide around two times speed-up during feedforward pass compared to the baseline model due to the significant reduction of learning parameters. Furthermore, the proposed method does not require efforts in manual tuning or post processing once the model is trained.
Code (2)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Physics-Guided Neural Networks for Inversion-based Feedforward Control applied to Linear Motors
Ever-increasing throughput specifications in semiconductor manufacturing require operating high-precision mechatronics, such as linear motors, at higher accelerations. In turn this creates higher nonlinear parasitic forc…
A Kernel-Based Identification Approach to LPV Feedforward: With Application to Motion Systems
The increasing demands for motion control result in a situation where Linear Parameter-Varying (LPV) dynamics have to be taken into account. Inverse-model feedforward control for LPV motion systems is challenging, since …
SchedulingFlattened Graph Convolutional Networks For Recommendation
Graph Convolutional Networks (GCNs) and their variants have achieved significant performances on various recommendation tasks. However, many existing GCN models tend to perform recursive aggregations among all related no…
Exponentially Faster Language Modelling
Language models only really need to use an exponential fraction of their neurons for individual inferences. As proof, we present UltraFastBERT, a BERT variant that uses 0.3% of its neurons during inference while performi…
BenchmarkingCPULanguage ModellingFeedback U-net for Cell Image Segmentation
Human brain is a layered structure, and performs not only a feedforward process from a lower layer to an upper layer but also a feedback process from an upper layer to a lower layer. The layer is a collection of neurons,…
Image SegmentationSemantic Segmentation