Unveiling Environmental Impacts of Large Language Model Serving: A Functional Unit View
Large language models (LLMs) offer powerful capabilities but come with significant environmental impact, particularly in carbon emissions. Existing studies benchmark carbon emissions but lack a standardized basis for comparison across different model configurations. To address this, we introduce the concept of functional unit (FU) as a standardized basis and develop FUEL, the first FU-based framework for evaluating LLM serving's environmental impact. Through three case studies, we uncover key insights and trade-offs in reducing carbon emissions by optimizing model size, quantization strategy, and hardware choice, paving the way for more sustainable LLM serving. The code is available at https://github.com/jojacola/FUEL.
Code (1)
Tasks
Language ModelingLanguage ModellingLarge Language ModelModel SelectionQuantizationSimilar Papers 제목 키워드 기반
BIRDS: Characterizing and Understanding Biodiversity Impact of Large Language Model Serving
Large language model (LLM) serving creates environmental impacts beyond carbon and water, including ecosystem damage through biodiversity-related pathways. We present BIRDS, a framework for Biodiversity Impact of Request…
Crosswashing in Sustainable Investing: Unveiling Strategic Practices Impacting ESG Scores
This paper introduces and defines a novel concept in sustainable investing, termed crosswashing, and explore its impact on ESG (Environmental, Social, and Governance) ratings through quantitative analysis using a Multi-C…
Decision MakingThe Environmental Impacts of Machine Learning Training Keep Rising Evidencing Rebound Effect
Recent Machine Learning (ML) approaches have shown increased performance on benchmarks but at the cost of escalating computational demands. Hardware, algorithmic and carbon optimizations have been proposed to curb energy…
How Green Can AI Be? A Study of Trends in Machine Learning Environmental Impacts
The compute requirements associated with training Artificial Intelligence (AI) models have increased exponentially over time. Optimisation strategies aim to reduce the energy consumption and environmental impacts associa…
Measuring the environmental impact of delivering AI at Google Scale
The transformative power of AI is undeniable - but as user adoption accelerates, so does the need to understand and mitigate the environmental impact of AI serving. However, no studies have measured AI serving environmen…