Inference economics of language models
We develop a theoretical model that addresses the economic trade-off between cost per token versus serial token generation speed when deploying LLMs for inference at scale. Our model takes into account arithmetic, memory bandwidth, network bandwidth and latency constraints; and optimizes over different parallelism setups and batch sizes to find the ones that optimize serial inference speed at a given cost per token. We use the model to compute Pareto frontiers of serial speed versus cost per token for popular language models.
Code (1)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Beyond Benchmarks: The Economics of AI Inference
The inference cost of Large Language Models (LLMs) has become a critical factor in determining their commercial viability and widespread adoption. This paper introduces a quantitative ``economics of inference'' framework…
Causal Inference in Network Economics
Network economics is the study of a rich class of equilibrium problems that occur in the real world, from traffic management to supply chains and two-sided online marketplaces. In this paper we explore causal inference i…
Causal InferenceManagementComputational Challenges in Token Economics: Bridging Economic Theory and AI System Design
Token economics has emerged as a useful lens for understanding resource allocation, value creation, and pricing in large language model systems. While recent work has increasingly treated tokens as economic primitives, t…
Causal Inference with Large Language Model: A Survey
Causal inference has been a pivotal challenge across diverse domains such as medicine and economics, demanding a complicated integration of human knowledge, mathematical reasoning, and data mining capabilities. Recent ad…
Causal InferenceLanguage ModelingLanguage ModellingLarge Language Model+2The Rise of Health Economics: Transforming the Landscape of Economic Research
This paper explores the evolving role of health economics within economic research and publishing over the past 30 years. Historically largely a niche field, health economics has become increasingly prominent, with the s…