Papers Audio Generation
“Audio Generation” 태그가 달린 논문 392편 · 필터 해제
StepAudio 3 Gen Technical Report
We introduce StepAudio 3 Gen, a general-purpose audio generation model that supports zero-shot text-to-speech (TTS), voice design, vocal generation, sound effects, music, vibe speech, and mixtures of multiple audio types…
Audio GenerationPRISM-Bench: An Audio-Centric Diagnostic Benchmark for Text-to-Audio-Video Generation
Text-to-audio-video (T2AV) generation has advanced rapidly, but its evaluation still underestimates the audio modality. Existing benchmarks either treat audio as an auxiliary component of video quality or assess it in is…
Audio GenerationVideo GenerationThe Attention Triangle in Audio-Video Models
Audio-video diffusion models rely on cross-modal attention to coordinate text, sound, and visual content, yet this same mechanism can introduce subtle and systematic semantic leakage. We study these models by probing and…
Audio GenerationVideo GenerationThe Missing Temporal Link: Temporal Context Routing for Script-Driven Audio-Video Generation
Joint audio-video generation models have made substantial progress in visual quality and audio-visual synchronization. However, they still provide limited control over when shot transitions occur and dialogue is spoken. …
Audio GenerationVideo GenerationTurboT2VA: Fast Large-Scale Text-to-Video-Audio Generation via Score-Regularized Consistency Distillation
Joint text-to-video-audio generation produces synchronized visual and acoustic content, but the long sampling trajectories and heterogeneous multimodal computation of large models make inference prohibitively expensive. …
Audio GenerationGeometric Iterative Retrieval for Neural Audio Codec Resynthesis
Neural audio codecs based on Residual Vector Quantization (RVQ) have become the dominant discrete representation for token-based general audio generation, yet resynthesizing high-quality audio from coarse codec tokens re…
Audio GenerationVA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generation
Using reinforcement learning to post-train joint video-audio generation models requires a reward signal. Existing methods construct this reward by combining metrics for individual quality dimensions, including audio qual…
Reinforcement LearningVideo GenerationAudio GenerationMusicLayout: Explicit Structural Planning for Controllable Text-to-Music Generation
Text-to-music generation has advanced rapidly, but current systems still rely primarily on global text prompts, leaving the structural organization of generated music implicit and difficult to inspect, control, or revise…
Text-to-Music GenerationAudio GenerationSwanTale: Unified Multi-Speaker Speech and Audio Generation for Instruct and Zero-Shot Tasks
Speech and audio generation is often needed in animation dubbing, audio drama, movies, advertising, games, podcasts, and short-video production. In these scenarios, creators may need to design voices without reference re…
Audio GenerationScene2Sound: Auditory-Grounded Soundscape Generation for 3D Gaussian Worlds
3D Gaussian Splatting (3DGS) turns captured or generated imagery into photorealistic 3D world simulations that users can freely explore, yet these worlds remain silent. Because existing audio generation methods condition…
Audio GenerationFlashDiff: Efficient Regional Execution and Scheduling for Diffusion Model Serving
Diffusion models have become the central backbone for modern image, video, and audio generation, but their efficient service remains a challenge. Unlike autoregressive decoding, diffusion inference repeatedly updates hig…
Audio GenerationA Quantized Native Runtime for On-Device Semantic Audio Generation
Semantic audio applications increasingly require controllable generation on commodity and embedded hardware rather than through framework-heavy datacenter stacks. We present aria, a dependency-free native runtime that ru…
Audio GenerationUnified Audio Intelligence Without Regressing on Text Intelligence
Audio intelligence involves understanding, reasoning about, and generating both audio and speech. In this work, we introduce Nemotron-Labs-Audex-30B-A3B (Audex), a unified audio-text LLM built on Nemotron-Cascade-2-30B-A…
multimodal generationSpeech RecognitionAudio GenerationSynthetic Audio Generation Framework for Air Traffic Control Speech Recognition
Automatic Speech Recognition (ASR) systems, despite achieving remarkable accuracy in general-purpose domains with native speech (L1), struggle in domains like Air Traffic Control (ATC) due to strong channel noise, a pres…
Synthetic Data GenerationSpeech RecognitionAudio GenerationVoice ConversionFoleyGenEx: Unified Video-to-Audio Generation with Multi-Modal Control, Temporal Alignment, and Semantic Precision
We present FoleyGenEx, a unified video-to-audio (VTA) framework integrating multi-modal control, frame-level temporal alignment, and fine-grained semantics, enabling synchronized, versatile audio synthesis for diverse ta…
Data AugmentationAudio GenerationAudioX-Turbo: A Unified Framework for Efficient Anything-to-Audio Generation
Audio and music generation based on flexible multimodal control signals is a widely applicable topic, with the following key challenges: 1) a unified multimodal modeling framework, 2) large-scale, high-quality training d…
Text-to-Music GenerationAudio GenerationAnyAudio-Judge: A Dynamic Rubric-Based Benchmark and Evaluator for Audio Instruction Following
The rapid advancement of instruction-guided audio generation has highlighted the critical need for robust alignment evaluation. Current automated evaluation methods heavily rely on holistic scoring from general-purpose l…
Reinforcement LearningInstruction FollowingAudio GenerationSketchSong: Hierarchical Song Generation with Sketch Planning and Fine-Grained Multi-Track Modeling
Recent song generation systems can synthesize realistic audio, yet generating complete songs remains challenging for two reasons. First, explicit song-level arrangement planning remains limited in existing methods, so mo…
Audio GenerationJenBridge: Adaptive Long-Form Video Soundtracking across Scene Transitions
We address the challenge of generating high-fidelity, long-form soundtracks that remain coherent across scene transitions. Existing AI music systems are mainly designed for short, isolated clips and lack mechanisms to en…
Audio GenerationImmersiveTTS: Environment-Aware Text-to-Speech with Multimodal Diffusion Transformer and Domain-Specific Representation Alignment
Recent advancements in text-guided audio generation have yielded promising results in diverse domains, including sound effects, speech, and music. However, jointly generating speech with environmental audio remains chall…
Audio Generation