Papers Inference Optimization
“Inference Optimization” 태그가 달린 논문 56편 · 필터 해제
An approach to optimize inference of the DIART speaker diarization pipeline
Speaker diarization answers the question "who spoke when" for an audio file. In some diarization scenarios, low latency is required for transcription. Speaker diarization with low latency is referred to as online speaker…
Inference OptimizationKnowledge DistillationQuantizationspeaker-diarization+1LLaSA: Large Language and E-Commerce Shopping Assistant
The e-commerce platform has evolved rapidly due to its widespread popularity and convenience. Developing an e-commerce shopping assistant for customers is crucial to aiding them in quickly finding desired products and re…
Inference OptimizationSpecificityPatched MOA: optimizing inference for diverse software development tasks
This paper introduces Patched MOA (Mixture of Agents), an inference optimization technique that significantly enhances the performance of large language models (LLMs) across diverse software development tasks. We evaluat…
Inference OptimizationInference Optimization of Foundation Models on AI Accelerators
Powerful foundation models, including large language models (LLMs), with Transformer architectures have ushered in a new era of Generative AI across various industries. Industry and research community have witnessed a la…
Inference OptimizationModel CompressionVideo GenerationInference Performance Optimization for Large Language Models on CPUs
Large language models (LLMs) have shown exceptional performance and vast potential across diverse tasks. However, the deployment of LLMs with high performance in low-resource environments has garnered significant attenti…
CPUGPUInference OptimizationRobust Zero-Shot Text-to-Speech Synthesis with Reverse Inference Optimization
In this paper, we propose reverse inference optimization (RIO), a simple and effective method designed to enhance the robustness of autoregressive-model-based zero-shot text-to-speech (TTS) systems using reinforcement le…
Inference OptimizationSpeech Synthesistext-to-speechText to Speech+1Scaling the Vocabulary of Non-autoregressive Models for Efficient Generative Retrieval
Generative Retrieval introduces a new approach to Information Retrieval by reframing it as a constrained generation task, leveraging recent advancements in Autoregressive (AR) language models. However, AR-based Generativ…
Inference OptimizationInformation RetrievalNatural QuestionsRetrievalEfficiency optimization of large-scale language models based on deep learning in natural language processing tasks
The internal structure and operation mechanism of large-scale language models are analyzed theoretically, especially how Transformer and its derivative architectures can restrict computing efficiency while capturing long…
Inference OptimizationKnowledge DistillationModel CompressionAdvances and Open Challenges in Federated Foundation Models
The integration of Foundation Models (FMs) with Federated Learning (FL) presents a transformative paradigm in Artificial Intelligence (AI). This integration offers enhanced capabilities, while addressing concerns of priv…
Computational EfficiencyFederated LearningInference OptimizationSurveyFederated Learning While Providing Model as a Service: Joint Training and Inference Optimization
While providing machine learning model as a service to process users' inference requests, online applications can periodically upgrade the model utilizing newly collected data. Federated learning (FL) is beneficial for e…
Federated LearningInference OptimizationIntegrating Pre-Trained Speech and Language Models for End-to-End Speech Recognition
Advances in machine learning have made it possible to perform various text and speech processing tasks, such as automatic speech recognition (ASR), in an end-to-end (E2E) manner. E2E approaches utilizing pre-trained mode…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Domain AdaptationInference Optimization+5SySMOL: Co-designing Algorithms and Hardware for Neural Networks with Heterogeneous Precisions
Recent quantization techniques have enabled heterogeneous precisions at very fine granularity, e.g., each parameter/activation can take on a different precision, resulting in compact neural networks without sacrificing a…
CPUGPUInference OptimizationQuantizationBayesian Active Learning in the Presence of Nuisance Parameters
In many settings, such as scientific inference, optimization, and transfer learning, the learner has a well-defined objective, which can be treated as estimation of a target parameter, and no intrinsic interest in charac…
Active LearningExperimental DesignInference OptimizationMeta-Learning+1Representing Edge Flows on Graphs via Sparse Cell Complexes
Obtaining sparse, interpretable representations of observable data is crucial in many machine learning and signal processing tasks. For data representing flows along the edges of a graph, an intuitively interpretable way…
Inference OptimizationRepresentation LearningPainterly Image Harmonization using Diffusion Model
Painterly image harmonization aims to insert photographic objects into paintings and obtain artistically coherent composite images. Previous methods for this task mainly rely on inference optimization or generative adver…
Generative Adversarial NetworkImage HarmonizationInference OptimizationmodelResidual-Based Error Corrector Operator to Enhance Accuracy and Reliability of Neural Operator Surrogates of Nonlinear Variational Boundary-Value Problems
This work focuses on developing methods for approximating the solution operators of a class of parametric partial differential equations via neural operators. Neural operators have several challenges, including the issue…
Inference OptimizationAdaptive Deep Neural Network Inference Optimization with EENet
Well-trained deep neural networks (DNNs) treat all test samples equally during prediction. Adaptive DNN inference with early exiting leverages the observation that some test examples can be easier to predict than others.…
Inference OptimizationSchedulingSST-2Networked Signal and Information Processing
The article reviews significant advances in networked signal and information processing, which have enabled in the last 25 years extending decision making and inference, optimization, control, and learning to the increas…
Decision MakingInference OptimizationEnhanced graph-learning schemes driven by similar distributions of motifs
This paper looks at the task of network topology inference, where the goal is to learn an unknown graph from nodal observations. One of the novelties of the approach put forth is the consideration of prior information ab…
Graph LearningInference OptimizationSelf-Constrained Inference Optimization on Structural Groups for Human Pose Estimation
We observe that human poses exhibit strong group-wise structural correlation and spatial coupling between keypoints due to the biological constraints of different body parts. This group-wise structural correlation can be…
Inference OptimizationMulti-Person Pose EstimationPose EstimationPose Prediction+1