paper-with-me

Papers

Dynamic Semantic Compression for CNN Inference in Multi-access Edge Computing: A Graph Reinforcement Learning-based Autoencoder

2024-01-19 · Nan Li, Alexandros Iosifidis, Qi Zhang

This paper studies the computational offloading of CNN inference in dynamic multi-access edge computing (MEC) networks. To address the uncertainties in communication time and computation resource availability, we propose a novel semantic compression method, autoencoder-based CNN architecture (AECNN), for effective semantic extraction and compression in partial offloading. In the semantic encoder, we introduce a feature compression module based on the channel attention mechanism in CNNs, to compress intermediate data by selecting the most informative features. In the semantic decoder, we design a lightweight decoder to reconstruct the intermediate data through learning from the received compressed data to improve accuracy. To effectively trade-off communication, computation, and inference accuracy, we design a reward function and formulate the offloading problem of CNN inference as a maximization problem with the goal of maximizing the average inference accuracy and throughput over the long term. To address this maximization problem, we propose a graph reinforcement learning-based AECNN (GRL-AECNN) method, which outperforms existing works DROO-AECNN, GRL-BottleNet++ and GRL-DeepJSCC under different dynamic scenarios. This highlights the advantages of GRL-AECNN in offloading decision-making in dynamic MEC.

📄 PDF Abstract BibTeX arXiv:2401.12167

Code (0)

등록된 구현이 없습니다.

Tasks

Decision MakingDecoderEdge-computingFeature CompressionSemantic Compression

Similar Papers 제목 키워드 기반

DynSplit-KV: Dynamic Semantic Splitting for KVCache Compression in Efficient Long-Context LLM Inference

2026-02-03 · Jiancai Ye, Jun Liu, Qingchen Li, Tianlang Zhao 외 arxiv

Although Key-Value (KV) Cache is essential for efficient large language models (LLMs) inference, its growing memory footprint in long-context scenarios poses a significant bottleneck, making KVCache compression crucial. …

Semantic Feature Division Multiple Access for Digital Semantic Broadcast Channels

2025-02-06 · Shuai Ma, Zhiye Sun, Bin Shen, Youlong Wu 외

In this paper, we propose a digital semantic feature division multiple access (SFDMA) paradigm in multi-user broadcast (BC) networks for the inference and the image reconstruction tasks. In this SFDMA scheme, the multi-u…

Data CompressionImage Reconstruction

Understanding the Physics of Key-Value Cache Compression for LLMs through Attention Dynamics

2026-03-02 · Samhruth Ananthanarayanan, Ayan Sengupta, Tanmoy Chakraborty arxiv

As context windows in LLMs scale to 100K+ tokens, the key-value (KV) cache becomes the dominant memory bottleneck, with recent methods claiming 80-90% savings and minimal benchmark degradation. We argue these evaluations…

T-REX: A 68-567 μs/token, 0.41-3.95 μJ/token Transformer Accelerator with Reduced External Memory Access and Enhanced Hardware Utilization in 16nm FinFET

2025-03-01 · Seunghyun Moon, Mao Li, Gregory Chen, Phil Knag 외

This work introduces novel training and post-training compression schemes to reduce external memory access during transformer model inference. Additionally, a new control flow mechanism, called dynamic batching, and a no…

Task-Aware Structured Memory for Dynamic Multi-modal In-Context Learning

2026-06-10 · Zhirui Chen, Ziwei Chen, Ling Shao arxiv

Multi-modal large language models (MLLMs) depend on in-context learning (ICL) for rapid task adaptation, but their scalability is severely limited by finite context windows and the growing cost of key-value (KV) caches i…

Graph Matching