Papers Audio Synthesis
“Audio Synthesis” 태그가 달린 논문 127편 · 필터 해제
MIDI-VALLE: Improving Expressive Piano Performance Synthesis Through Neural Codec Language Modelling
Generating expressive audio performances from music scores requires models to capture both instrument acoustics and human interpretation. Traditional music performance synthesis pipelines follow a two-stage approach, fir…
Audio SynthesisLanguage Modellingtext-to-speechText to SpeechStep-by-Step Video-to-Audio Synthesis via Negative Audio Guidance
We propose a novel step-by-step video-to-audio generation method that sequentially produces individual audio tracks, each corresponding to a specific sound event in the video. Our approach mirrors traditional Foley workf…
Audio GenerationAudio SynthesisNegationDiffusion-Based Symbolic Regression
Diffusion has emerged as a powerful framework for generative modeling, achieving remarkable success in applications such as image and audio synthesis. Enlightened by this progress, we propose a novel diffusion-based appr…
Audio SynthesisDenoisingregressionSymbolic RegressionSpecMaskFoley: Steering Pretrained Spectral Masked Generative Transformer Toward Synchronized Video-to-audio Synthesis via ControlNet
Foley synthesis aims to synthesize high-quality audio that is both semantically and temporally aligned with video frames. Given its broad application in creative industries, the task has gained increasing attention in th…
Audio SynthesisCommunication-Efficient Diffusion Denoising Parallelization via Reuse-then-Predict Mechanism
Diffusion models have emerged as a powerful class of generative models across various modalities, including image, video, and audio synthesis. However, their deployment is often limited by significant inference latency, …
Audio SynthesisDenoisingDPN-GAN: Inducing Periodic Activations in Generative Adversarial Networks for High-Fidelity Audio Synthesis
In recent years, generative adversarial networks (GANs) have made significant progress in generating audio sequences. However, these models typically rely on bandwidth-limited mel-spectrograms, which constrain the resolu…
Audio GenerationAudio SynthesisMusic GenerationSpeech SynthesisFast Differentiable Modal Simulation of Non-linear Strings, Membranes, and Plates
Modal methods for simulating vibrations of strings, membranes, and plates are widely used in acoustics and physically informed audio synthesis. However, traditional implementations, particularly for non-linear models lik…
Audio SynthesisCPUGPUSupervising 3D Talking Head Avatars with Analysis-by-Audio-Synthesis
In order to be widely applicable, speech-driven 3D head avatars must articulate their lips in accordance with speech, while also conveying the appropriate emotions with dynamically changing facial expressions. The key pr…
Audio SynthesisTARO: Timestep-Adaptive Representation Alignment with Onset-Aware Conditioning for Synchronized Video-to-Audio Synthesis
This paper introduces Timestep-Adaptive Representation Alignment with Onset-Aware Conditioning (TARO), a novel framework for high-fidelity and temporally coherent video-to-audio synthesis. Built upon flow-based transform…
Audio SynthesisFADDesigning Neural Synthesizers for Low-Latency Interaction
Neural Audio Synthesis (NAS) models offer interactive musical control over high-quality, expressive audio generators. While these models can operate in real-time, they often suffer from high latency, making them unsuitab…
Audio SynthesisLong-Video Audio Synthesis with Multi-Agent Collaboration
Video-to-audio synthesis, which generates synchronized audio for visual content, critically enhances viewer immersion and narrative coherence in film and interactive media. However, video-to-audio dubbing for long-form c…
Audio SynthesisScene SegmentationScript GenerationNexus: An Omni-Perceptive And -Interactive Model for Language, Audio, And Vision
This work proposes an industry-level omni-modal large language model (LLM) pipeline that integrates auditory, visual, and linguistic modalities to overcome challenges such as limited tri-modal datasets, high computationa…
Audio SynthesisAutomatic Speech RecognitionAutomatic Speech Recognition (ASR)Language Modeling+9XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
The rapid proliferation of generative audio synthesis and editing technologies has raised significant concerns about copyright infringement, data provenance, and the spread of misinformation through deepfake audio. Water…
Audio SynthesisFace SwappingMisinformationGenerative diffusion model with inverse renormalization group flows
Diffusion models represent a class of generative models that produce data by denoising a sample corrupted by white noise. Despite the success of diffusion models in computer vision, audio synthesis, and point cloud gener…
Audio SynthesisDenoisingImage Generationmodel+2Customized Condition Controllable Generation for Video Soundtrack
Recent advances in latent diffusion models (LDMs) have enabled data-driven paradigms for video soundtrack generation, improving multimodal alignment capabilities. However, current two-stage frameworks--which separate…
Audio SynthesisTri-Ergon: Fine-grained Video-to-Audio Generation with Multi-modal Conditions and LUFS Control
Video-to-audio (V2A) generation utilizes visual-only video features to produce realistic sounds that correspond to the scene. However, current V2A models often lack fine-grained control over the generated audio, especial…
Audio GenerationAudio SynthesisMMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis
We propose to synthesize high-quality and synchronized audio, given video and optional text conditions, using a novel multimodal joint training framework MMAudio. In contrast to single-modality training conditioned on (l…
Audio GenerationAudio SynthesisAudio-Visual SynchronizationVideo-to-Sound GenerationCSSinger: End-to-End Chunkwise Streaming Singing Voice Synthesis System Based on Conditional Variational Autoencoder
Singing Voice Synthesis (SVS) aims to generate singing voices of high fidelity and expressiveness. Conventional SVS systems usually utilize an acoustic model to transform a music score into acoustic features, followed by…
Audio SynthesisSinging Voice Synthesistext-to-speechText to SpeechZero-Shot Mono-to-Binaural Speech Synthesis
We present ZeroBAS, a neural method to synthesize binaural audio from monaural audio recordings and positional information without training on any binaural data. To our knowledge, this is the first published zero-shot ne…
Audio SynthesisDenoisingSpeech SynthesisZero-Shot LearningGeneralized Diffusion Model with Adjusted Offset Noise
Diffusion models have become fundamental tools for modeling data distributions in machine learning and have applications in image generation, drug discovery, and audio synthesis. Despite their success, these models face …
Audio SynthesisDrug DiscoveryImage Generationmodel