Papers Music Captioning
“Music Captioning” 태그가 달린 논문 14편 · 필터 해제
MusiScene: Leveraging MU-LLaMA for Scene Imagination and Enhanced Video Background Music Generation
Humans can imagine various atmospheres and settings when listening to music, envisioning movie scenes that complement each piece. For example, slow, melancholic music might evoke scenes of heartbreak, while upbeat melodi…
Language ModelingLanguage ModellingMusic CaptioningMusic GenerationSonicVerse: Multi-Task Learning for Music Feature-Informed Captioning
Detailed captions that accurately reflect the characteristics of a music piece can enrich music databases and drive forward research in music AI. This paper introduces a multi-task music captioning model, SonicVerse, tha…
Caption GenerationDescriptiveKey DetectionLarge Language Model+2SLEEPING-DISCO 9M: A large-scale pre-training dataset for generative music modeling
We present Sleeping-DISCO 9M, a large-scale pre-training dataset for music and song. To the best of our knowledge, there are no open-source high-quality dataset representing popular and well-known songs for generative mu…
Music CaptioningMusic ModelingSinging Voice SynthesisCMI-Bench: A Comprehensive Benchmark for Evaluating Music Instruction Following
Recent advances in audio-text large language models (LLMs) have opened new possibilities for music understanding and generation. However, existing benchmarks are limited in scope, often relying on simplified tasks or mul…
Beat TrackingGenre classificationInformation RetrievalInstruction Following+5Do Captioning Metrics Reflect Music Semantic Alignment?
Music captioning has emerged as a promising task, fueled by the advent of advanced language generation models. However, the evaluation of music captioning relies heavily on traditional metrics such as BLEU, METEOR, and R…
Music CaptioningText GenerationEvaluation of pretrained language models on music understanding
Music-text multimodal systems have enabled new approaches to Music Information Research (MIR) applications such as audio-to-text and text-to-audio retrieval, text-based song generation, and music captioning. Despite the …
Music CaptioningNegationSensitivityText to Audio Retrieval+1Futga: Towards Fine-grained Music Understanding through Temporally-enhanced Generative Augmentation
Existing music captioning methods are limited to generating concise global descriptions of short music clips, which fail to capture fine-grained musical characteristics and time-aware musical changes. To address these li…
Music CaptioningMusic GenerationThe Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluation
We introduce the Song Describer dataset (SDD), a new crowdsourced corpus of high-quality audio-caption pairs, designed for the evaluation of music-and-language models. The dataset consists of 1.1k human-written natural l…
Music CaptioningMusic GenerationRetrievalText to Audio Retrieval+1MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Response
Large Language Models (LLMs) have shown immense potential in multimodal applications, yet the convergence of textual and musical domains remains not well-explored. To address this gap, we present MusiLingo, a novel syste…
Caption GenerationLanguage ModellingMusic CaptioningMusic Understanding LLaMA: Advancing Text-to-Music Generation with Question Answering and Captioning
Text-to-music generation (T2M-Gen) faces a major obstacle due to the scarcity of large-scale publicly available music datasets with natural language captions. To address this, we propose the Music Understanding LLaMA (MU…
Caption GenerationLarge Language ModelMultimodal Music GenerationMusic Captioning+2LP-MusicCaps: LLM-Based Pseudo Music Captioning
Automatic music captioning, which generates natural language descriptions for given music tracks, holds significant potential for enhancing the understanding and organization of large volumes of musical data. Despite its…
Language ModelingLanguage ModellingLarge Language ModelMusic Captioning+2ALCAP: Alignment-Augmented Music Captioner
Music captioning has gained significant attention in the wake of the rising prominence of streaming media platforms. Traditional approaches often prioritize either the audio or lyrics aspect of the music, inadvertently i…
Contrastive LearningMusic CaptioningRetrievalMusCaps: Generating Captions for Music Audio
Content-based music information retrieval has seen rapid progress with the adoption of deep learning. Current approaches to high-level music description typically make use of classification models, such as in auto-taggin…
Audio captioningClassificationCross-Modal RetrievalDecoder+3Towards Music Captioning: Generating Music Playlist Descriptions
Descriptions are often provided along with recommendations to help users' discovery. Recommending automatically generated music playlists (e.g. personalised playlists) introduces the problem of generating descriptions. I…
Music Captioning