Papers Audio Generation
“Audio Generation” 태그가 달린 논문 396편 · 필터 해제
Kandinsky 6.0 Video: Foundation Models for Synchronized Video and Audio Generation
We present Kandinsky 6.0 Video, a family of foundation diffusion models for synchronized text-to-audio-video generation, comprising Kandinsky 6.0 Video Lite (3B parameters) and Kandinsky 6.0 Video Pro (29B parameters). B…
Audio GenerationVideo GenerationPrism: Dynamic Sparse Attention for Native 2K Joint Video-Audio Generation Model Training
Natively training joint video-audio generation models at higher resolutions empowers them to learn richer visual details and sharper motion dynamics. However, full attention incurs quadratic cost and, as resolution incre…
Audio GenerationJoint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomy
Video and audio are perceived together, yet most generative models treat them in isolation. We examine methods that model the two modalities jointly, generate one from the other, or edit them in a coupled manner, organiz…
Audio GenerationAll modalities are equal, but video is more equal: Closing the Cross-Attention Gap in Joint Video Generation
Video is a rich representation of a physical event, capturing appearance, geometry, motion, and temporal evolution. Other modalities, such as 3D body motion or audio, encode narrower aspects of the same event. We find th…
Audio GenerationVideo GenerationStepAudio 3 Gen Technical Report
We introduce StepAudio 3 Gen, a general-purpose audio generation model that supports zero-shot text-to-speech (TTS), voice design, vocal generation, sound effects, music, vibe speech, and mixtures of multiple audio types…
Audio GenerationPRISM-Bench: An Audio-Centric Diagnostic Benchmark for Text-to-Audio-Video Generation
Text-to-audio-video (T2AV) generation has advanced rapidly, but its evaluation still underestimates the audio modality. Existing benchmarks either treat audio as an auxiliary component of video quality or assess it in is…
Audio GenerationVideo GenerationThe Attention Triangle in Audio-Video Models
Audio-video diffusion models rely on cross-modal attention to coordinate text, sound, and visual content, yet this same mechanism can introduce subtle and systematic semantic leakage. We study these models by probing and…
Audio GenerationVideo GenerationThe Missing Temporal Link: Temporal Context Routing for Script-Driven Audio-Video Generation
Joint audio-video generation models have made substantial progress in visual quality and audio-visual synchronization. However, they still provide limited control over when shot transitions occur and dialogue is spoken. …
Audio GenerationVideo GenerationTurboT2VA: Fast Large-Scale Text-to-Video-Audio Generation via Score-Regularized Consistency Distillation
Joint text-to-video-audio generation produces synchronized visual and acoustic content, but the long sampling trajectories and heterogeneous multimodal computation of large models make inference prohibitively expensive. …
Audio GenerationGeometric Iterative Retrieval for Neural Audio Codec Resynthesis
Neural audio codecs based on Residual Vector Quantization (RVQ) have become the dominant discrete representation for token-based general audio generation, yet resynthesizing high-quality audio from coarse codec tokens re…
Audio GenerationVA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generation
Using reinforcement learning to post-train joint video-audio generation models requires a reward signal. Existing methods construct this reward by combining metrics for individual quality dimensions, including audio qual…
Reinforcement LearningVideo GenerationAudio GenerationMusicLayout: Explicit Structural Planning for Controllable Text-to-Music Generation
Text-to-music generation has advanced rapidly, but current systems still rely primarily on global text prompts, leaving the structural organization of generated music implicit and difficult to inspect, control, or revise…
Text-to-Music GenerationAudio GenerationSwanTale: Unified Multi-Speaker Speech and Audio Generation for Instruct and Zero-Shot Tasks
Speech and audio generation is often needed in animation dubbing, audio drama, movies, advertising, games, podcasts, and short-video production. In these scenarios, creators may need to design voices without reference re…
Audio GenerationScene2Sound: Auditory-Grounded Soundscape Generation for 3D Gaussian Worlds
3D Gaussian Splatting (3DGS) turns captured or generated imagery into photorealistic 3D world simulations that users can freely explore, yet these worlds remain silent. Because existing audio generation methods condition…
Audio GenerationFlashDiff: Efficient Regional Execution and Scheduling for Diffusion Model Serving
Diffusion models have become the central backbone for modern image, video, and audio generation, but their efficient service remains a challenge. Unlike autoregressive decoding, diffusion inference repeatedly updates hig…
Audio GenerationA Quantized Native Runtime for On-Device Semantic Audio Generation
Semantic audio applications increasingly require controllable generation on commodity and embedded hardware rather than through framework-heavy datacenter stacks. We present aria, a dependency-free native runtime that ru…
Audio GenerationUnified Audio Intelligence Without Regressing on Text Intelligence
Audio intelligence involves understanding, reasoning about, and generating both audio and speech. In this work, we introduce Nemotron-Labs-Audex-30B-A3B (Audex), a unified audio-text LLM built on Nemotron-Cascade-2-30B-A…
multimodal generationSpeech RecognitionAudio GenerationSynthetic Audio Generation Framework for Air Traffic Control Speech Recognition
Automatic Speech Recognition (ASR) systems, despite achieving remarkable accuracy in general-purpose domains with native speech (L1), struggle in domains like Air Traffic Control (ATC) due to strong channel noise, a pres…
Synthetic Data GenerationSpeech RecognitionAudio GenerationVoice ConversionFoleyGenEx: Unified Video-to-Audio Generation with Multi-Modal Control, Temporal Alignment, and Semantic Precision
We present FoleyGenEx, a unified video-to-audio (VTA) framework integrating multi-modal control, frame-level temporal alignment, and fine-grained semantics, enabling synchronized, versatile audio synthesis for diverse ta…
Data AugmentationAudio GenerationAudioX-Turbo: A Unified Framework for Efficient Anything-to-Audio Generation
Audio and music generation based on flexible multimodal control signals is a widely applicable topic, with the following key challenges: 1) a unified multimodal modeling framework, 2) large-scale, high-quality training d…
Text-to-Music GenerationAudio Generation