Visual Transformer Meets CutMix for Improved Accuracy, Communication Efficiency, and Data Privacy in Split Learning

2022-07-01 · Sihun Baek, Jihong Park, Praneeth Vepakomma, Ramesh Raskar, Mehdi Bennis, Seong-Lyun Kim

This article seeks for a distributed learning solution for the visual transformer (ViT) architectures. Compared to convolutional neural network (CNN) architectures, ViTs often have larger model sizes, and are computationally expensive, making federated learning (FL) ill-suited. Split learning (SL) can detour this problem by splitting a model and communicating the hidden representations at the split-layer, also known as smashed data. Notwithstanding, the smashed data of ViT are as large as and as similar as the input data, negating the communication efficiency of SL while violating data privacy. To resolve these issues, we propose a new form of CutSmashed data by randomly punching and compressing the original smashed data. Leveraging this, we develop a novel SL framework for ViT, coined CutMixSL, communicating CutSmashed data. CutMixSL not only reduces communication costs and privacy leakage, but also inherently involves the CutMix data augmentation, improving accuracy and scalability. Simulations corroborate that CutMixSL outperforms baselines such as parallelized SL and SplitFed that integrates FL with SL.

📄 PDF Abstract BibTeX arXiv:2207.00234

Code (0)

등록된 구현이 없습니다.

Tasks

Data Augmentation Federated Learning

Methods 이 논문이 사용한 방법론

CutMix CutMix is an image data augmentation strategy. Instead of simply removing pixels as in Cutout, we replace the removed regions with…

Privacy-Preserving Split Learning with Vision Transformers using Patch-Wise Random and Noisy CutMix

2024-08-02 · Seungeun Oh, Sihun Baek, Jihong Park, Hyelin Nam 외

In computer vision, the vision transformer (ViT) has increasingly superseded the convolutional neural network (CNN) for improved accuracy and robustness. However, ViT's large model sizes and high sample complexity make i…

Privacy Preserving

Differentially Private CutMix for Split Learning with Vision Transformer

2022-10-28 · Seungeun Oh, Jihong Park, Sihun Baek, Hyelin Nam 외

Recently, vision transformer (ViT) has started to outpace the conventional CNN in computer vision tasks. Considering privacy-preserving distributed learning with ViT, federated learning (FL) communicates models, which be…

Federated Learning Privacy Preserving

The Impact of CutMix on Reliability and Robustness in Semantic Segmentation

2026-08-19 · Steven Landgraf, Markus Ulrich arxiv

Ensuring not only high accuracy but also reliable and robust predictions is critical for the deployment of semantic segmentation models in safety-critical applications such as autonomous driving. Despite the widespread u…

Semantic Segmentation Autonomous Driving Data Augmentation

Use the Detection Transformer as a Data Augmenter

2023-04-10 · Luping Wang, Bin Liu

Detection Transformer (DETR) is a Transformer architecture based object detection model. In this paper, we demonstrate that it can also be used as a data augmenter. We term our approach as DETR assisted CutMix, or DeMix …

Data Augmentation image-classification Image Classification object-detection+1

Inducing Spatial Locality in Vision Transformers through the Training Protocol

2026-05-11 · Eduardo Santiago Toledo, Asael Fabian Martínez arxiv

We investigate whether the training protocol can induce spatial locality in the early layers of a Vision Transformer (ViT) trained from scratch, without large-scale pretraining. Keeping the architecture and optimization …