KRNet: Towards Efficient Knowledge Replay
The knowledge replay technique has been widely used in many tasks such as continual learning and continuous domain adaptation. The key lies in how to effectively encode the knowledge extracted from previous data and replay them during current training procedure. A simple yet effective model to achieve knowledge replay is autoencoder. However, the number of stored latent codes in autoencoder increases linearly with the scale of data and the trained encoder is redundant for the replaying stage. In this paper, we propose a novel and efficient knowledge recording network (KRNet) which directly maps an arbitrary sample identity number to the corresponding datum. Compared with autoencoder, our KRNet requires significantly ($400\times$) less storage cost for the latent codes and can be trained without the encoder sub-network. Extensive experiments validate the efficiency of KRNet, and as a showcase, it is successfully applied in the task of continual learning.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningDomain AdaptationSimilar Papers 제목 키워드 기반
Bounded KRnet and its applications to density estimation and approximation
In this paper, we develop an invertible mapping, called B-KRnet, on a bounded domain and apply it to density estimation/approximation for data or the solutions of PDEs such as the Fokker-Planck equation and the Keller-Se…
Density EstimationVAE-KRnet and its applications to variational Bayes
In this work, we have proposed a generative model, called VAE-KRnet, for density estimation or approximation, which combines the canonical variational autoencoder (VAE) with our recently developed flow-based generative m…
Density EstimationDimensionality ReductionKnowledge Rectification for Camouflaged Object Detection: Unlocking Insights from Low-Quality Data
Low-quality data often suffer from insufficient image details, introducing an extra implicit aspect of camouflage that complicates camouflaged object detection (COD). Existing COD methods focus primarily on high-quality …
Diversityobject-detectionObject DetectionSuper-ResolutionAugmented KRnet for density estimation and approximation
In this work, we have proposed augmented KRnets including both discrete and continuous models. One difficulty in flow-based generative modeling is to maintain the invertibility of the transport map, which is often a trad…
Density EstimationLearning Knowledge Representation with Meta Knowledge Distillation for Single Image Super-Resolution
Knowledge distillation (KD), which can efficiently transfer knowledge from a cumbersome network (teacher) to a compact network (student), has demonstrated its advantages in some computer vision applications. The represen…
Image Super-ResolutionKnowledge DistillationMeta-LearningSuper-Resolution