Pinpointing the Memory Behaviors of DNN Training
The training of deep neural networks (DNNs) is usually memory-hungry due to the limited device memory capacity of DNN accelerators. Characterizing the memory behaviors of DNN training is critical to optimize the device memory pressures. In this work, we pinpoint the memory behaviors of each device memory block of GPU during training by instrumenting the memory allocators of the runtime system. Our results show that the memory access patterns of device memory blocks are stable and follow an iterative fashion. These observations are useful for the future optimization of memory-efficient training from the perspective of raw memory access patterns.
Code (0)
등록된 구현이 없습니다.
Tasks
GPUSimilar Papers 제목 키워드 기반
Axiom Pinpointing
Axiom pinpointing refers to the task of finding the specific axioms in an ontology which are responsible for a consequence to follow. This task has been studied, under different names, in many research areas, leading to …
Symmetry's Edge in Cortical Dynamics: Multiscale Dynamics of Ensemble Excitation and Inhibition
Creating a quantitative theory for the cortex presents challenges and raises questions. What are the significant scales--micro, meso, or macroscopic? What are the interactions--pairwise, higher order, or mean-field? And …
Multimodal Industrial Anomaly Detection by Crossmodal Feature Mapping
The paper explores the industrial multimodal Anomaly Detection (AD) task, which exploits point clouds and RGB images to localize anomalies. We introduce a novel light and fast framework that learns to map features from o…
Anomaly DetectionDenoising and Prompt-Tuning for Multi-Behavior Recommendation
In practical recommendation scenarios, users often interact with items under multi-typed behaviors (e.g., click, add-to-cart, and purchase). Traditional collaborative filtering techniques typically assume that users only…
Collaborative FilteringDenoisingInterpreting Agentic Systems: Beyond Model Explanations to System-Level Accountability
Agentic systems have transformed how Large Language Models (LLMs) can be leveraged to create autonomous systems with goal-directed behaviors, consisting of multi-step planning and the ability to interact with different e…