paper-with-me

홈 › Papers

Bob's Confetti: Phonetic Memorization Attacks in Music and Video Generation

2025-07-23 · Jaechul Roh, Zachary Novack, Yuefeng Peng, Niloofar Mireshghallah, Taylor Berg-Kirkpatrick, Amir Houmansadr arxiv

Generative AI systems for music and video commonly use text-based filters to prevent regurgitation of copyrighted material. We expose a significant vulnerability in this approach by introducing Adversarial PhoneTic Prompting (APT), a novel attack that bypasses these safeguards by exploiting phonetic memorization--the tendency of models to bind sub-lexical acoustic patterns (phonemes, rhyme, stress, cadence) to memorized copyrighted content. APT replaces iconic lyrics with homophonic but semantically unrelated alternatives (e.g., "mom's spaghetti" becomes "Bob's confetti"), preserving phonetic structure while evading lexical filters. We evaluate APT on leading lyrics-to-song models (Suno, YuE) across English and Korean songs spanning rap, pop, and K-pop. APT achieves 91% average similarity to copyrighted originals, versus 13.7% for random lyrics and 42.2% for semantic paraphrases. Embedding analysis confirms the mechanism: YuE's text encoder treats APT-modified lyrics as near-identical to originals (cosine similarity 0.90) while Sentence-BERT semantic similarity drops to 0.71, showing the model encodes phonetic structure over meaning. This vulnerability extends cross-modally--Veo 3 reconstructs visual scenes from original music videos when prompted with APT lyrics alone, despite no visual cues in the prompt. We further show that phonetic-semantic defense signatures fail, as APT prompts exhibit higher semantic similarity than benign paraphrases. Our findings reveal that sub-lexical acoustic structure acts as a cross-modal retrieval key, rendering current copyright filters systematically vulnerable. Demo examples are available at https://jrohsc.github.io/music_attack/.

📄 PDF Abstract BibTeX arXiv:2507.17937

Code (0)

등록된 구현이 없습니다.

Tasks

Cross-Modal RetrievalSemantic SimilarityVideo Generation

Similar Papers 제목 키워드 기반

Counterfactual Explainable AI (XAI) Method for Deep Learning-Based Multivariate Time Series Classification

2025-11-17 · Alan G. Paredes Cetina, Kaouther Benguessoum, Raoni Lourenço, Sylvain Kubler arxiv

Recent advances in deep learning have improved multivariate time series (MTS) classification and regression by capturing complex patterns, but their lack of transparency hinders decision-making. Explainable AI (XAI) meth…

Time Series Classification

VidLeaks: Membership Inference Attacks Against Text-to-Video Models

2026-01-16 · Li Wang, Wenyu Chen, Ning Yu, Zheng Li 외 arxiv

The proliferation of powerful Text-to-Video (T2V) models, trained on massive web-scale datasets, raises urgent concerns about copyright and privacy violations. Membership inference attacks (MIAs) provide a principled too…

Video Generation

Surface-Form Neural Sparse Retrieval: Robust Fuzzy Matching for Industrial Music Search

2026-05-18 · Paul Greyson, Zhichao Geng, Wei Zhang, Yang Yang arxiv

Music search at the scale of Amazon Music presents a unique challenge: queries frequently deviate from indexed metadata due to misspellings, transpositions, and phonetic variations, yet the retrieval system must operate …

MM-ALT: A Multimodal Automatic Lyric Transcription System

2022-07-13 · Xiangming Gu, Longshen Ou, Danielle Ong, Ye Wang

Automatic lyric transcription (ALT) is a nascent field of study attracting increasing interest from both the speech and music information retrieval communities, given its significant application potential. However, ALT w…

Action DetectionActivity DetectionAutomatic Speech RecognitionAutomatic Speech Recognition (ASR)+5

Memorization in NLP Fine-tuning Methods

2022-05-25 · FatemehSadat Mireshghallah, Archit Uniyal, Tianhao Wang, David Evans 외

Large language models are shown to present privacy risks through memorization of training data, and several recent works have studied such risks for the pre-training phase. Little attention, however, has been given to th…

Memorization