MobiSR: Efficient On-Device Super-Resolution through Heterogeneous Mobile Processors
In recent years, convolutional networks have demonstrated unprecedented performance in the image restoration task of super-resolution (SR). SR entails the upscaling of a single low-resolution image in order to meet application-specific image quality demands and plays a key role in mobile devices. To comply with privacy regulations and reduce the overhead of cloud computing, executing SR models locally on-device constitutes a key alternative approach. Nevertheless, the excessive compute and memory requirements of SR workloads pose a challenge in mapping SR networks on resource-constrained mobile platforms. This work presents MobiSR, a novel framework for performing efficient super-resolution on-device. Given a target mobile platform, the proposed framework considers popular model compression techniques and traverses the design space to reach the highest performing trade-off between image quality and processing speed. At run time, a novel scheduler dispatches incoming image patches to the appropriate model-engine pair based on the patch's estimated upscaling difficulty in order to meet the required image quality with minimum processing latency. Quantitative evaluation shows that the proposed framework yields on-device SR designs that achieve an average speedup of 2.13x over highly-optimized parallel difficulty-unaware mappings and 4.79x over highly-optimized single compute engine implementations.
Code (0)
등록된 구현이 없습니다.
Tasks
Cloud ComputingImage RestorationModel CompressionSuper-ResolutionSimilar Papers 제목 키워드 기반
NAWQ-SR: A Hybrid-Precision NPU Engine for Efficient On-Device Super-Resolution
In recent years, image and video delivery systems have begun integrating deep learning super-resolution (SR) approaches, leveraging their unprecedented visual enhancement capabilities while reducing reliance on networkin…
CPUQuantizationSuper-ResolutionIndividualized Dynamic Latent Factor Model for Multi-resolutional Data with Application to Mobile Health
Mobile health has emerged as a major success for tracking individual health status, due to the popularity and power of smartphones and wearable devices. This has also brought great challenges in handling heterogeneous, m…
Irregular Time SeriesTime SeriesXCAT -- Lightweight Quantized Single Image Super-Resolution using Heterogeneous Group Convolutions and Cross Concatenation
We propose a lightweight, single image super-resolution network for mobile devices, named XCAT. XCAT introduces Heterogeneous Group Convolution Blocks with Cross Concatenations (HXBlock). The heterogeneous split of the i…
Data AugmentationGPUImage Super-ResolutionQuantization+1NEMO: enabling neural-enhanced video streaming on commodity mobile devices
The demand for mobile video streaming has experienced tremendous growth over the last decade. However, existing methods of video delivery fall short of delivering high-quality video. Recent advances in neural super-resol…
Super-ResolutionVideo Super-ResolutionTrimming Down Large Spiking Vision Transformers via Heterogeneous Quantization Search
Spiking Neural Networks (SNNs) are amenable to deployment on edge devices and neuromorphic hardware due to their lower dissipation. Recently, SNN-based transformers have garnered significant interest, incorporating atten…
Model CompressionQuantization