Cost-Effective Hyperparameter Optimization for Large Language Model Generation Inference
Large Language Models (LLMs) have sparked significant interest in their generative capabilities, leading to the development of various commercial applications. The high cost of using the models drives application builders to maximize the value of generation under a limited inference budget. This paper presents a study of optimizing inference hyperparameters such as the number of responses, temperature and max tokens, which significantly affects the utility/cost of text generation. We design a framework named EcoOptiGen which leverages economical hyperparameter optimization and cost-based pruning. Experiments with the GPT-3.5/GPT-4 models on a variety of tasks verify its effectiveness. EcoOptiGen is implemented in the `autogen' package of the FLAML library: \url{https://aka.ms/autogen}.
Code (3)
Tasks
Hyperparameter OptimizationLanguage ModelingLanguage ModellingLarge Language ModelText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
ACE: Adaptive Constraint-aware Early Stopping in Hyperparameter Optimization
Deploying machine learning models requires high model quality and needs to comply with application constraints. That motivates hyperparameter optimization (HPO) to tune model configurations under deployment constraints. …
FairnessHyperparameter OptimizationStart Small, Think Big: On Hyperparameter Optimization for Large-Scale Knowledge Graph Embeddings
Knowledge graph embedding (KGE) models are an effective and popular approach to represent and reason with multi-relational data. Prior studies have shown that KGE models are sensitive to hyperparameter settings, however,…
Graph EmbeddingHyperparameter OptimizationKnowledge Graph EmbeddingKnowledge Graph Embeddings+2Tune As You Scale: Hyperparameter Optimization For Compute Efficient Training
Hyperparameter tuning of deep learning models can lead to order-of-magnitude performance gains for the same amount of compute. Despite this, systematic tuning is uncommon, particularly for large models, which are expensi…
Bayesian OptimizationHyperparameter OptimizationFrugal Optimization for Cost-related Hyperparameters
The increasing demand for democratizing machine learning algorithms calls for hyperparameter optimization (HPO) solutions at low cost. Many machine learning algorithms have hyperparameters which can cause a large variati…
AutoMLBIG-bench Machine LearningHyperparameter OptimizationIntrinsic Mutual Information as a Modulator for Preference Optimization
Offline preference optimization methods, such as Direct Preference Optimization (DPO), offer significant advantages in aligning Large Language Models (LLMs) with human values. However, achieving optimal performance with …