Slimmable ConvNeXt: Width-Adaptive Inference for Efficient Multi-Device Deployment

2026-05-21 · Janek Haberer, Jon Eike Wilhelm, Olaf Landsiedel arxiv

Deploying vision models across devices with varying resource constraints, or even on a single device where available compute fluctuates due to battery state, thermal throttling, or latency deadlines, typically requires training and maintaining separate models. Width-adaptive inference addresses this by training a single set of shared weights containing multiple nested subnetworks of increasing capacity, but prior CNN-based approaches required switchable batch normalization, while recent scalable methods have focused on Vision Transformers. We present Slimmable ConvNeXt, which shows that ConvNeXt's modern design, specifically LayerNorm and inverted bottlenecks, makes it particularly suited for channel-width slimming, eliminating the normalization overhead of classical slimmable networks and producing a simpler training pipeline than both prior CNN and ViT approaches. On ImageNet-1k, Slimmable ConvNeXt-T with 3 subnetworks achieves 80.8% top-1 accuracy at 4.5 GMACs and 77.4% at 1.2 GMACs, trained from scratch for 600 epochs. At comparable compute, this exceeds HydraViT's 6-head subnetwork (78.4% at 4.6 GMACs) by 2.4 percentage points and its 3-head configuration (73.0% at 1.3 GMACs) by 4.4 percentage points, while also outperforming MatFormer-S (78.6%) and SortedNet-S (78.2%) at the same GMACs. Scaling to Slimmable ConvNeXt-B further improves maximum accuracy to 82.8% at 15.35 GMACs.

📄 PDF Abstract BibTeX arXiv:2605.22677

Code (0)

등록된 구현이 없습니다.

MoSE: Mixture of Slimmable Experts for Efficient and Adaptive Language Models

2026-02-05 · Nurbek Tastan, Stefanos Laskaridis, Karthik Nandakumar, Samuel Horvath arxiv

Mixture-of-Experts (MoE) models scale large language models efficiently by sparsely activating experts, but once an expert is selected, it is executed fully. Hence, the trade-off between accuracy and computation in an Mo…

Universally Slimmable Networks and Improved Training Techniques

2019-03-12 · ICCV 2019 10 · Jiahui Yu, Thomas Huang

Slimmable networks are a family of neural networks that can instantly adjust the runtime width. The width can be chosen from a predefined widths set to adaptively optimize accuracy-efficiency trade-offs at runtime. In th…

Deep Reinforcement Learning Image Super-Resolution Reinforcement Learning Super-Resolution

Slimmable Neural Networks

2018-12-21 · ICLR 2019 5 · Jiahui Yu, Linjie Yang, Ning Xu, Jianchao Yang 외

We present a simple and general method to train a single neural network executable at different widths (number of channels in a layer), permitting instant and adaptive accuracy-efficiency trade-offs at runtime. Instead o…

Instance Segmentation Keypoint Detection object-detection Object Detection+1

Joslim: Joint Widths and Weights Optimization for Slimmable Neural Networks

2020-07-23 · Ting-Wu Chin, Ari S. Morcos, Diana Marculescu

Slimmable neural networks provide a flexible trade-off front between prediction error and computational requirement (such as the number of floating-point operations or FLOPs) with the same storage requirement as a single…

PareCO: Pareto-aware Channel Optimization for Slimmable Neural Networks

2020-09-28 · Rudy Chin, Ari S. Morcos, Diana Marculescu

Slimmable neural networks provide a flexible trade-off front between prediction error and computational cost (such as the number of floating-point operations or FLOPs) with the same storage cost as a single model. They h…