paper-with-me

Papers

TeachAugment: Data Augmentation Optimization Using Teacher Knowledge

2022-02-25 · CVPR 2022 1 · Teppei Suzuki

Optimization of image transformation functions for the purpose of data augmentation has been intensively studied. In particular, adversarial data augmentation strategies, which search augmentation maximizing task loss, show significant improvement in the model generalization for many tasks. However, the existing methods require careful parameter tuning to avoid excessively strong deformations that take away image features critical for acquiring generalization. In this paper, we propose a data augmentation optimization method based on the adversarial strategy called TeachAugment, which can produce informative transformed images to the model without requiring careful tuning by leveraging a teacher model. Specifically, the augmentation is searched so that augmented images are adversarial for the target model and recognizable for the teacher model. We also propose data augmentation using neural networks, which simplifies the search space design and allows for updating of the data augmentation using the gradient method. We show that TeachAugment outperforms existing methods in experiments of image classification, semantic segmentation, and unsupervised representation learning tasks.

📄 PDF Abstract BibTeX arXiv:2202.12513

Code (1)

DensoITLab/TeachAugment 공식 구현 pytorch

Tasks

Data Augmentationimage-classificationImage ClassificationRepresentation LearningSemantic Segmentation

Similar Papers 제목 키워드 기반

Role-Wise Data Augmentation for Knowledge Distillation

2020-04-19 · ICLR 2020 1 · Jie Fu, Xue Geng, Zhijian Duan, Bohan Zhuang 외

Knowledge Distillation (KD) is a common method for transferring the ``knowledge'' learned by one machine learning model (the \textit{teacher}) into another model (the \textit{student}), where typically, the teacher has a…

Data AugmentationKnowledge Distillation

HARD: Hard Augmentations for Robust Distillation

2023-05-24 · Arne F. Nix, Max F. Burg, Fabian H. Sinz

Knowledge distillation (KD) is a simple and successful method to transfer knowledge from a teacher to a student model solely based on functional activity. However, current KD has a few shortcomings: it has recently been …

Data AugmentationDomain GeneralizationKnowledge Distillation

Exploring Inconsistent Knowledge Distillation for Object Detection with Data Augmentation

2022-09-20 · Jiawei Liang, Siyuan Liang, Aishan Liu, Ke Ma 외

Knowledge Distillation (KD) for object detection aims to train a compact detector by transferring knowledge from a teacher model. Since the teacher model perceives data in a way different from humans, existing KD methods…

Data AugmentationKnowledge Distillationobject-detectionObject Detection

Single-Teacher View Augmentation: Boosting Knowledge Distillation via Angular Diversity

2025-10-26 · Seonghoon Yu, Dongjun Nam, Dina Katabi, Jeany Son arxiv

Knowledge Distillation (KD) aims to train a lightweight student model by transferring knowledge from a large, high-capacity teacher. Recent studies have shown that leveraging diverse teacher perspectives can significantl…

Knowledge Distillation

Refine Myself by Teaching Myself: Feature Refinement via Self-Knowledge Distillation

2021-03-15 · CVPR 2021 1 · Mingi Ji, Seungjae Shin, Seunghyun Hwang, Gibeom Park 외

Knowledge distillation is a method of transferring the knowledge from a pretrained complex teacher model to a student model, so a smaller network can replace a large teacher network at the deployment stage. To reduce the…

Data AugmentationKnowledge Distillationobject-detectionObject Detection+3