ESSR: An 8K@30FPS Super-Resolution Accelerator With Edge Selective Network
Deep learning-based super-resolution (SR) is challenging to implement in resource-constrained edge devices for resolutions beyond full HD due to its high computational complexity and memory bandwidth requirements. This paper introduces an 8K@30FPS SR accelerator with edge-selective dynamic input processing. Dynamic processing chooses the appropriate subnets for different patches based on simple input edge criteria, achieving a 50\% MAC reduction with only a 0.1dB PSNR decrease. The quality of reconstruction images is guaranteed and maximized its potential with \textit{resource adaptive model switching} even under resource constraints. In conjunction with hardware-specific refinements, the model size is reduced by 84\% to 51K, but with a decrease of less than 0.6dB PSNR. Additionally, to support dynamic processing with high utilization, this design incorporates a \textit{configurable group of layer mapping} that synergizes with the \textit{structure-friendly fusion block}, resulting in 77\% hardware utilization and up to 79\% reduction in feature SRAM access. The implementation, using the TSMC 28nm process, can achieve 8K@30FPS throughput at 800MHz with a gate count of 2749K, 0.2075W power consumption, and 4797Mpixels/J energy efficiency, exceeding previous work.
Code (0)
등록된 구현이 없습니다.
Tasks
8kSuper-ResolutionSimilar Papers 제목 키워드 기반
ResSR: A Computationally Efficient Residual Approach to Super-Resolving Multispectral Images
Multispectral imaging sensors typically have wavelength-dependent resolution, which limits downstream processing. Consequently, researchers have proposed multispectral image super-resolution (MSI-SR) methods which upsamp…
Image Super-ResolutionMultispectral Image Super-resolutionSuper-ResolutionTextlessRAG: End-to-End Visual Document RAG by Speech Without Text
Document images encapsulate a wealth of knowledge, while the portability of spoken queries enables broader and flexible application scenarios. Yet, no prior work has explored knowledge base question answering over visual…
Knowledge Base Question AnsweringA High-Performance Accelerator for Super-Resolution Processing on Embedded GPU
Recent years have witnessed impressive progress in super-resolution (SR) processing. However, its real-time inference requirement sets a challenge not only for the model design but also for the on-chip implementation. In…
Dictionary LearningGPUSuper-ResolutionVocal Bursts Intensity PredictionMondrian: On-Device High-Performance Video Analytics with Compressive Packed Inference
In this paper, we present Mondrian, an edge system that enables high-performance object detection on high-resolution video streams. Many lightweight models and system optimization techniques have been proposed for resour…
GPUobject-detectionObject DetectionBSRA: Block-based Super Resolution Accelerator with Hardware Efficient Pixel Attention
Increasingly, convolution neural network (CNN) based super resolution models have been proposed for better reconstruction results, but their large model size and complicated structure inhibit their real-time hardware imp…
Image ReconstructionSuper-Resolution