MoCA: Memory-Centric, Adaptive Execution for Multi-Tenant Deep Neural Networks

2023-05-10 · Seah Kim, Hasan Genc, Vadim Vadimovich Nikiforov, Krste Asanović, Borivoje Nikolić, Yakun Sophia Shao

Driven by the wide adoption of deep neural networks (DNNs) across different application domains, multi-tenancy execution, where multiple DNNs are deployed simultaneously on the same hardware, has been proposed to satisfy the latency requirements of different applications while improving the overall system utilization. However, multi-tenancy execution could lead to undesired system-level resource contention, causing quality-of-service (QoS) degradation for latency-critical applications. To address this challenge, we propose MoCA, an adaptive multi-tenancy system for DNN accelerators. Unlike existing solutions that focus on compute resource partition, MoCA dynamically manages shared memory resources of co-located applications to meet their QoS targets. Specifically, MoCA leverages the regularities in both DNN operators and accelerators to dynamically modulate memory access rates based on their latency targets and user-defined priorities so that co-located applications get the resources they demand without significantly starving their co-runners. We demonstrate that MoCA improves the satisfaction rate of the service level agreement (SLA) up to 3.9x (1.8x average), system throughput by 2.3x (1.7x average), and fairness by 1.3x (1.2x average), compared to prior work.

📄 PDF Abstract BibTeX arXiv:2305.05843

Code (1)

ucb-bar/moca 공식 구현

Tasks

Fairness

Methods 이 논문이 사용한 방법론

Golden Queue Managers 설명 없음

FreeMOCA: Memory-Free Continual Learning for Malicious Code Analysis

2026-05-10 · Zahra Asadi, Haeseung Jeon, Sohyun Han, Md Mahmuduzzaman Kamol 외 arxiv

As over 200 million new malware samples are identified each year, antivirus systems must continuously adapt to the evolving threat landscape. However, retraining solely on new samples leads to catastrophic forgetting and…

Continual Learning Malware Detection

Kalypso: Relational LLM Serving

2026-07-26 · Hojae Son, Md Ashraful Islam, Huy Gia Cao, Hui Guan 외 arxiv

Large language models are increasingly used as semantic operators for filtering, extracting, ranking, joining, and transforming unstructured data. Existing semantic query processing systems invoke request-centric LLM ser…

EgoExoMoCap: Distributed Ego-Exo Human Motion Capture

2026-07-17 · Jiaxi Jiang, Bharat Lal Bhatnagar, Nan Yang, Lingni Ma 외 arxiv

Human motion capture from head-mounted devices (HMDs) offers a scalable way to acquire real-world human motion and interaction data, which is crucial for applications in embodied AI and VR/AR. Existing approaches focus o…

Horizon-LM: A RAM-Centric Architecture for LLM Training

2026-02-04 · Zhengqing Yuan, Lichao Sun, Yanfang Ye arxiv

The rapid growth of large language models (LLMs) has outpaced the evolution of single-GPU hardware, making model scale increasingly constrained by memory capacity rather than computation. While modern training systems ex…

Domain Adaptation

PoseStreamer: A Multi-modal Framework for 3D Tracking of Unseen Moving Objects

2025-12-28 · Huiming Yang, Linglin Liao, Fei Ding, Sibo Wang 외 arxiv

Six degree of freedom (6DoF) pose estimation for novel objects is a critical task in computer vision, yet it faces significant challenges in high-speed and low-light scenarios where standard RGB cameras suffer from motio…

Pose Estimation