AutoStyle-TTS: Retrieval-Augmented Generation based Automatic Style Matching Text-to-Speech Synthesis
With the advancement of speech synthesis technology, users have higher expectations for the naturalness and expressiveness of synthesized speech. But previous research ignores the importance of prompt selection. This study proposes a text-to-speech (TTS) framework based on Retrieval-Augmented Generation (RAG) technology, which can dynamically adjust the speech style according to the text content to achieve more natural and vivid communication effects. We have constructed a speech style knowledge database containing high-quality speech samples in various contexts and developed a style matching scheme. This scheme uses embeddings, extracted by Llama, PER-LLM-Embedder,and Moka, to match with samples in the knowledge database, selecting the most appropriate speech style for synthesis. Furthermore, our empirical research validates the effectiveness of the proposed method. Our demo can be viewed at: https://thuhcsi.github.io/icme2025-AutoStyle-TTS
Code (0)
등록된 구현이 없습니다.
Tasks
RAGRetrieval-augmented GenerationSpeech Synthesistext-to-speechText to SpeechText-To-Speech SynthesisSimilar Papers 제목 키워드 기반
From Pixels to Posts: Retrieval-Augmented Fashion Captioning and Hashtag Generation
This paper introduces the retrieval-augmented framework for automatic fashion caption and hashtag generation, combining multi-garment detection, attribute reasoning, and Large Language Model (LLM) prompting. The system a…
Domain GeneralizationFrom Query to Explanation: Uni-RAG for Multi-Modal Retrieval-Augmented Learning in STEM
In AI-facilitated teaching, leveraging various query styles to interpret abstract educational content is crucial for delivering effective and accessible learning experiences. However, existing retrieval systems predomina…
Image MatchingPersonalized Text Generation with Contrastive Activation Steering
Personalized text generation aims to infer users' writing style preferences from their historical texts and generate outputs that faithfully reflect these stylistic characteristics. Existing solutions primarily adopt two…
parameter-efficient fine-tuningRAGRetrievalRetrieval-augmented Generation+1Diversify Question Generation with Retrieval-Augmented Style Transfer
Given a textual passage and an answer, humans are able to ask questions with various expressions, but this ability is still challenging for most question generation (QG) systems. Existing solutions mainly focus on the in…
DiversityQuestion AnsweringQuestion GenerationQuestion-Generation+3ALoFTRAG: Automatic Local Fine Tuning for Retrieval Augmented Generation
Retrieval Augmented Generation (RAG) systems have been shown to improve the accuracy of Large Language Model (LLM) outputs. However, these models can often achieve low accuracy when applied to new data domains. We introd…
Language ModelingLanguage ModellingLarge Language ModelRAG+2