Deep Networks with Fast Retraining

2020-08-13 · Wandong Zhang, Yimin Yang, Jonathan Wu

Recent work [1] has utilized Moore-Penrose (MP) inverse in deep convolutional neural network (DCNN) learning, which achieves better generalization performance over the DCNN with a stochastic gradient descent (SGD) pipeline. However, Yang's work has not gained much popularity in practice due to its high sensitivity of hyper-parameters and stringent demands of computational resources. To enhance its applicability, this paper proposes a novel MP inverse-based fast retraining strategy. In each training epoch, a random learning strategy that controls the number of convolutional layers trained in the backward pass is first utilized. Then, an MP inverse-based batch-by-batch learning strategy, which enables the network to be implemented without access to industrial-scale computational resources, is developed to refine the dense layer parameters. Experimental results empirically demonstrate that fast retraining is a unified strategy that can be used for all DCNNs. Compared to other learning strategies, the proposed learning pipeline has robustness against the hyper-parameters, and the requirement of computational resources is significantly reduced. [1] Y. Yang, J. Wu, X. Feng, and A. Thangarajah, "Recomputation of dense layers for the perfor-238mance improvement of dcnn," IEEE Trans. Pattern Anal. Mach. Intell., 2019.

📄 PDF Abstract BibTeX arXiv:2008.07387

Code (0)

등록된 구현이 없습니다.

Tasks

Image Classification

Methods 이 논문이 사용한 방법론

DCNN Diffusion-convolutional neural networks (DCNN) is a model for graph-structured data. Through the introduction of a diffusion-convolution operation, diffusion-based representations…

SGD Stochastic Gradient Descent is an iterative optimization technique that uses minibatches of data to form an expectation of the gradient, rather than the full gradient using…

Fast On-the-fly Retraining-free Sparsification of Convolutional Neural Networks

2018-11-10 · NIPS Workshop CDNNRIA 2018 · Amir H. Ashouri, Tarek S. Abdelrahman, Alwyn Dos Remedios

Modern Convolutional Neural Networks (CNNs) are complex, encompassing millions of parameters. Their deployment exerts computational, storage and energy demands, particularly on embedded platforms. Existing approaches to …

Fast and Automatic Object Registration for Human-Robot Collaboration in Industrial Manufacturing

2022-04-01 · Manuela Geiß, Martin Baresch, Georgios Chasparis, Edwin Schweiger 외

We present an end-to-end framework for fast retraining of object detection models in human-robot-collaboration. Our Faster R-CNN based setup covers the whole workflow of automatic image generation and labeling, model ret…

Image Generation Object object-detection Object Detection

Fast-FedUL: A Training-Free Federated Unlearning with Provable Skew Resilience

2024-05-28 · Thanh Trung Huynh, Trong Bang Nguyen, Phi Le Nguyen, Thanh Tam Nguyen 외

Federated learning (FL) has recently emerged as a compelling machine learning paradigm, prioritizing the protection of privacy for training data. The increasing demand to address issues such as ``the right to be forgotte…

Backdoor Attack Data Poisoning Federated Learning

Effective End-to-End Vision Language Pretraining with Semantic Visual Loss

2023-01-18 · Xiaofeng Yang, Fayao Liu, Guosheng Lin

Current vision language pretraining models are dominated by methods using region visual features extracted from object detectors. Given their good performance, the extract-then-process pipeline significantly restricts th…

GPU

TabICLv2: A better, faster, scalable, and open tabular foundation model

2026-02-11 · Jingang Qu, David Holzmüller, Gaël Varoquaux, Marine Le Morvan arxiv

Tabular foundation models, such as TabPFNv2 and TabICL, have recently dethroned gradient-boosted trees at the top of predictive benchmarks, demonstrating the value of in-context learning for tabular data. We introduce Ta…

Synthetic Data Generation