CEEMS: A Resource Manager Agnostic Energy and Emissions Monitoring Stack
With the rapid acceleration of ML/AI research in the last couple of years, the energy consumption of the Information and Communication Technology (ICT) domain has rapidly increased. As a major part of this energy consumption is due to users' workloads, it is evident that users need to be aware of the energy footprint of their applications. Compute Energy and Emissions Monitoring Stack (CEEMS) has been designed to address this issue. CEEMS can report energy consumption and equivalent emissions of user workloads in real time for HPC and cloud platforms alike. Besides CPU energy usage, it supports reporting energy usage of workloads on NVIDIA and AMD GPU accelerators. CEEMS has been built around the prominent open-source tools in the observability eco-system like Prometheus and Grafana. CEEMS has been designed to be extensible and it allows the Data Center (DC) operators to easily define the energy estimation rules of user workloads based on the underlying hardware. This paper explains the architectural overview of CEEMS, data sources that are used to measure energy usage and estimate equivalent emissions and potential use cases of CEEMS from operator and user perspectives. Finally, the paper will conclude by describing how CEEMS deployment on the Jean-Zay supercomputing platform is capable of monitoring more than 1400 nodes that have a daily job churn rate of around 20k jobs.
Code (0)
등록된 구현이 없습니다.
Tasks
CPUGPUMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Logarithmic Mean Divisia Index Decomposition of CO$_2$ Emissions from Energy Use in Romania
Carbon emissions have become a specific alarming indicators and intricate challenges that lead an extended argue about climate change. The growing trend in the utilization of fossil fuels for the economic progress and si…
The Hidden Environmental Cost of Poor Coding Practices in TensorFlow and Keras Applications: A Study on Resource Leaks and Carbon Emissions
Efficiency and sustainability are critical considerations in the development and deployment of machine learning (ML) applications. Among the factors influencing sustainability, resource leaks in ML code can introduce hid…
Leveraging policy instruments and financial incentives to reduce embodied carbon in energy retrofits
The existing buildings and building construction sectors together are responsible for over one-third of the total global energy consumption and nearly 40% of total greenhouse gas (GHG) emissions. GHG emissions from the b…
Nowcasting day-ahead marginal emissions using multi-headed CNNs and deep generative models
Nowcasting day-ahead marginal emissions factors is increasingly important for power systems with high flexibility and penetration of distributed energy resources. With a significant share of firm generation from natural …
The energy return on investment of whole energy systems: application to Belgium
Planning the defossilization of energy systems by facilitating high penetration of renewables and maintaining access to abundant and affordable primary energy resources is a nontrivial multi-objective problem. However, s…