Papers Inference Optimization
“Inference Optimization” 태그가 달린 논문 56편 · 필터 해제
Sub-MoE: Efficient Mixture-of-Expert LLMs Compression via Subspace Expert Merging
Mixture of Experts (MoE) LLMs face significant obstacles due to their massive parameter scale, which imposes memory, storage, and deployment challenges. Although recent expert merging methods promise greater efficiency b…
Inference OptimizationMixture-of-ExpertsThe Foundation Cracks: A Comprehensive Study on Bugs and Testing Practices in LLM Libraries
Large Language Model (LLM) libraries have emerged as the foundational infrastructure powering today's AI revolution, serving as the backbone for LLM deployment, inference optimization, fine-tuning, and production serving…
Bug fixingInference OptimizationLarge Language ModelBrevity is the soul of sustainability: Characterizing LLM response lengths
A significant portion of the energy consumed by Large Language Models (LLMs) arises from their inference processes; hence developing energy-efficient methods for inference is crucial. While several techniques exist for i…
DecoderInference OptimizationPrompt EngineeringDSMentor: Enhancing Data Science Agents with Curriculum Learning and Online Knowledge Accumulation
Large language model (LLM) agents have shown promising performance in generating code for solving complex data science problems. Recent studies primarily focus on enhancing in-context learning through improved search, sa…
In-Context LearningInference OptimizationLarge Language ModelFaster MoE LLM Inference for Extremely Large Models
Sparse Mixture of Experts (MoE) large language models (LLMs) are gradually becoming the mainstream approach for ultra-large-scale models. Existing optimization efforts for MoE models have focused primarily on coarse-grai…
Inference OptimizationMixture-of-ExpertsOptimizing LLM Inference: Fluid-Guided Online Scheduling with Memory Constraints
Large Language Models (LLMs) are indispensable in today's applications, but their inference procedure -- generating responses by processing text in segments and using a memory-heavy Key-Value (KV) cache -- demands signif…
GPUInference OptimizationSchedulingSimpleAR: Pushing the Frontier of Autoregressive Visual Generation through Pretraining, SFT, and RL
This work presents SimpleAR, a vanilla autoregressive visual generation framework without complex architecure modifications. Through careful exploration of training and inference optimization, we demonstrate that: 1) wit…
Inference OptimizationThe 1st Solution for 4th PVUW MeViS Challenge: Unleashing the Potential of Large Multimodal Models for Referring Video Segmentation
Motion expression video segmentation is designed to segment objects in accordance with the input motion expressions. In contrast to the conventional Referring Video Object Segmentation (RVOS), it places emphasis on motio…
Inference OptimizationReferring Video Object SegmentationSegmentationSemantic Segmentation+3Energy-Efficient Transformer Inference: Optimization Strategies for Time Series Classification
The increasing computational demands of transformer models in time series classification necessitate effective optimization strategies for energy-efficient deployment. Our study presents a systematic investigation of opt…
ClassificationInference OptimizationQuantizationTime Series+1Hybrid Offline-online Scheduling Method for Large Language Model Inference Optimization
With the development of large language models (LLMs), it has become increasingly important to optimize hardware usage and improve throughput. In this paper, we study the inference optimization of the serving system that …
GSM8KInference OptimizationLanguage ModelingLanguage Modelling+2DVFS-Aware DNN Inference on GPUs: Latency Modeling and Performance Analysis
The rapid development of deep neural networks (DNNs) is inherently accompanied by the problem of high computational costs. To tackle this challenge, dynamic voltage frequency scaling (DVFS) is emerging as a promising tec…
CPUInference OptimizationHellinger-Kantorovich Gradient Flows: Global Exponential Decay of Entropy Functionals
We investigate a family of gradient flows of positive and probability measures, focusing on the Hellinger-Kantorovich (HK) geometry, which unifies transport mechanism of Otto-Wasserstein, and the birth-death mechanism of…
Inference OptimizationA Survey on Inference Optimization Techniques for Mixture of Experts Models
The emergence of large-scale Mixture of Experts (MoE) models represents a significant advancement in artificial intelligence, offering enhanced model capacity and computational efficiency through conditional computation.…
Computational EfficiencyDistributed ComputingInference OptimizationKnowledge Distillation+4FluidML: Fast and Memory Efficient Inference Optimization
Machine learning models deployed on edge devices have enabled numerous exciting new applications, such as humanoid robots, AR glasses, and autonomous vehicles. However, the computing resources available on these edge dev…
Autonomous VehiclesInference OptimizationManagementA Temporal Linear Network for Time Series Forecasting
Recent research has challenged the necessity of complex deep learning architectures for time series forecasting, demonstrating that simple linear models can often outperform sophisticated approaches. Building upon this i…
Computational EfficiencyInference OptimizationTime SeriesTime Series ForecastingLLM-Rank: A Graph Theoretical Approach to Pruning Large Language Models
The evolving capabilities of large language models are accompanied by growing sizes and deployment costs, necessitating effective inference optimisation techniques. We propose a novel pruning method utilising centrality …
Inference OptimizationNetwork PruningEdgeRL: Reinforcement Learning-driven Deep Learning Model Inference Optimization at Edge
Balancing mutually diverging performance metrics, such as, processing latency, outcome accuracy, and end device energy consumption is a challenging undertaking for deep learning model inference in ad-hoc edge environment…
Deep LearningInference OptimizationReinforcement Learning (RL)CycleBNN: Cyclic Precision Training in Binary Neural Networks
This paper works on Binary Neural Networks (BNNs), a promising avenue for efficient deep learning, offering significant reductions in computational overhead and memory footprint to full precision networks. However, the c…
Inference OptimizationRevisiting SMoE Language Models by Evaluating Inefficiencies with Task Specific Expert Pruning
Sparse Mixture of Expert (SMoE) models have emerged as a scalable alternative to dense models in language modeling. These models use conditionally activated feedforward subnetworks in transformer blocks, allowing for a s…
Inference OptimizationLanguage ModelingLanguage ModellingThe Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities
This report examines the fine-tuning of Large Language Models (LLMs), integrating theoretical insights with practical applications. It outlines the historical evolution of LLMs from traditional Natural Language Processin…
Computational EfficiencyInference OptimizationMixture-of-Experts