Edge-Cloud Cooperation for DNN Inference via Reinforcement Learning and Supervised Learning

2022-10-11 · Tinghao Zhang, Zhijun Li, Yongrui Chen, Kwok-Yan Lam, Jun Zhao

Deep Neural Networks (DNNs) have been widely applied in Internet of Things (IoT) systems for various tasks such as image classification and object detection. However, heavyweight DNN models can hardly be deployed on edge devices due to limited computational resources. In this paper, an edge-cloud cooperation framework is proposed to improve inference accuracy while maintaining low inference latency. To this end, we deploy a lightweight model on the edge and a heavyweight model on the cloud. A reinforcement learning (RL)-based DNN compression approach is used to generate the lightweight model suitable for the edge from the heavyweight model. Moreover, a supervised learning (SL)-based offloading strategy is applied to determine whether the sample should be processed on the edge or on the cloud. Our method is implemented on real hardware and tested on multiple datasets. The experimental results show that (1) The sizes of the lightweight models obtained by RL-based DNN compression are up to 87.6% smaller than those obtained by the baseline method; (2) SL-based offloading strategy makes correct offloading decisions in most cases; (3) Our method reduces up to 78.8% inference latency and achieves higher accuracy compared with the cloud-only strategy.

📄 PDF Abstract BibTeX arXiv:2210.05182

Code (0)

등록된 구현이 없습니다.

Tasks

image-classification Image Classification object-detection Object Detection reinforcement-learning Reinforcement Learning (RL)

A Multi-intersection Vehicular Cooperative Control based on End-Edge-Cloud Computing

2020-12-01 · Mingzhi Jiang, Tianhao Wu, Zhe Wang, Yi Gong 외

Cooperative Intelligent Transportation Systems (C-ITS) will change the modes of road safety and traffic management, especially at intersections without traffic lights, namely unsignalized intersections. Existing research…

Cloud Computing Management

Learning Optimal Fronthauling and Decentralized Edge Computation in Fog Radio Access Networks

2021-03-21 · Hoon Lee, Junbeom Kim, Seok-Hwan Park

Fog radio access networks (F-RANs), which consist of a cloud and multiple edge nodes (ENs) connected via fronthaul links, have been regarded as promising network architectures. The F-RAN entails a joint optimization of c…

Edge-computing

Smaller, Smarter, Closer: The Edge of Collaborative Generative AI

2025-05-22 · Roberto Morabito, SiYoung Jang

The rapid adoption of generative AI (GenAI), particularly Large Language Models (LLMs), has exposed critical limitations of cloud-centric deployments, including latency, cost, and privacy concerns. Meanwhile, Small Langu…

Collaborative Inference

A Cooperation Graph Approach for Multiagent Sparse Reward Reinforcement Learning

2022-08-05 · Qingxu Fu, Tenghai Qiu, Zhiqiang Pu, Jianqiang Yi 외

Multiagent reinforcement learning (MARL) can solve complex cooperative tasks. However, the efficiency of existing MARL methods relies heavily on well-defined reward functions. Multiagent tasks with sparse reward feedback…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

DVFO: Learning-Based DVFS for Energy-Efficient Edge-Cloud Collaborative Inference

2023-06-02 · Ziyang Zhang, Yang Zhao, Huan Li, Changyao Lin 외

Due to limited resources on edge and different characteristics of deep neural network (DNN) models, it is a big challenge to optimize DNN inference performance in terms of energy consumption and end-to-end latency on edg…

Collaborative Inference CPU Deep Reinforcement Learning GPU