When Large Language Models Meet Speech: A Survey on Integration Approaches
Recent advancements in large language models (LLMs) have spurred interest in expanding their application beyond text-based tasks. A large number of studies have explored integrating other modalities with LLMs, notably speech modality, which is naturally related to text. This paper surveys the integration of speech with LLMs, categorizing the methodologies into three primary approaches: text-based, latent-representation-based, and audio-token-based integration. We also demonstrate how these methods are applied across various speech-related applications and highlight the challenges in this field to offer inspiration for
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Deep Learning for Pathological Speech: A Survey
Advancements in spoken language technologies for neurodegenerative speech disorders are crucial for meeting both clinical and technological needs. This overview paper is vital for advancing the field, as it presents a co…
Automatic Speech RecognitionData AugmentationDeep Learningspeech-recognition+2Improving Meeting Inclusiveness using Speech Interruption Analysis
Meetings are a pervasive method of communication within all types of companies and organizations, and using remote collaboration systems to conduct meetings has increased dramatically since the COVID-19 pandemic. However…
Telephone Surveys Meet Conversational AI: Evaluating a LLM-Based Telephone Survey System at Scale
Telephone surveys remain a valuable tool for gathering insights but typically require substantial resources in training and coordinating human interviewers. This work presents an AI-driven telephone survey system integra…
AI AgentLarge Language ModelSpeech-to-TextSurvey+2Summarizing Speech: A Comprehensive Survey
Speech summarization has become an essential tool for efficiently managing and accessing the growing volume of spoken and audiovisual content. However, despite its increasing importance, speech summarization remains loos…
Meeting Summarizationspeech-recognitionSpeech RecognitionSurvey+1A Survey of Serverless Machine Learning Model Inference
Recent developments in Generative AI, Computer Vision, and Natural Language Processing have led to an increased integration of AI models into various products. This widespread adoption of AI requires significant efforts …
GPUmodelSurvey