Distilling Large Language Models for Network Active Queue Management
The growing complexity of network traffic and demand for ultra-low latency communication require smarter packet traffic management. Existing Deep Learning-based queuing approaches struggle with dynamic network scenarios and demand high engineering effort. We propose AQM-LLM, distilling Large Language Models (LLMs) with few-shot learning, contextual understanding, and pattern recognition to improve Active Queue Management (AQM) [RFC 9330] with minimal manual effort. We consider a specific case where AQM is Low Latency, Low Loss, and Scalable Throughput (L4S) and our design of AQM-LLM builds on speculative decoding and reinforcement-based distilling of LLM by tackling congestion prevention in the L4S architecture using Explicit Congestion Notification (ECN) [RFC 9331] and periodic packet dropping. We develop a new open-source experimental platform by executing L4S-AQM on FreeBSD-14, providing interoperable modules to support LLM integration and facilitate IETF recognition through wider testing. Our extensive evaluations show L4S-LLM enhances queue management, prevents congestion, reduces latency, and boosts network performance, showcasing LLMs' adaptability and efficiency in uplifting AQM systems.
Code (0)
등록된 구현이 없습니다.
Tasks
Few-Shot LearningManagementSimilar Papers 제목 키워드 기반
Queue management for slo-oriented large language model serving
Large language model (LLM) serving is becoming an increasingly critical workload for cloud providers. Existing LLM serving systems focus on interactive requests, such as chatbots and coding assistants, with tight latency…
BlockingGPULanguage ModelingLanguage Modelling+2Fast Distributed Inference Serving for Large Language Models
Large language models (LLMs) power a new generation of interactive AI applications exemplified by ChatGPT. The interactive nature of these applications demands low latency for LLM inference. Existing LLM serving systems …
BlockingGPUManagementSchedulingAirport Terminal Passenger Queue Forecasting for Departure Gates and Security Checkpoints
Accurate passenger queue forecasting in airport terminals is essential for efficient departure operations, as it enables proactive congestion management. However, time-varying passenger demand and heterogeneous facility …
SAM: Support Vector Machine Based Active Queue Management
Recent years have seen an increasing interest in the design of AQM (Active Queue Management) controllers. The purpose of these controllers is to manage the network congestion under varying loads, link delays and bandwidt…
ManagementActive queue management: First steps toward a new control-theoretic viewpoint
Active Queue Management (AQM) for mitigating Internet congestion has been addressed via various feedback control syntheses, among which P, PI, and PID regulators are quite popular and often associated to a Smith predicto…
ManagementTime SeriesTime Series Analysis