Exploring Softly Masked Language Modelling for Controllable Symbolic Music Generation
This document presents some early explorations of applying Softly Masked Language Modelling (SMLM) to symbolic music generation. SMLM can be seen as a generalisation of masked language modelling (MLM), where instead of each element of the input set being either known or unknown, each element can be known, unknown or partly known. We demonstrate some results of applying SMLM to constrained symbolic music generation using a transformer encoder architecture. Several audio examples are available at https://erl-j.github.io/smlm-web-supplement/
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModellingMusic GenerationSimilar Papers 제목 키워드 기반
Unsupervised Aspect-Level Sentiment Controllable Style Transfer
Unsupervised style transfer in text has previously been explored through the sentiment transfer task. The task entails inverting the overall sentiment polarity in a given input sentence, while preserving its content. Fro…
Aspect-Based Sentiment AnalysisAspect-Based Sentiment Analysis (ABSA)DecoderLanguage Modelling+3Mini Minds: Exploring Bebeshka and Zlata Baby Models
In this paper, we describe the University of Lyon 2 submission to the Strict-Small track of the BabyLM competition. The shared task is created with an emphasis on small-scale language modelling from scratch on limited-si…
DecoderLanguage AcquisitionLanguage ModellingControllableGPT: A Ground-Up Designed Controllable GPT for Molecule Optimization
Large Language Models (LLMs) employ three popular training approaches: Masked Language Models (MLM), Causal Language Models (CLM), and Sequence-to-Sequence Models (seq2seq). However, each approach has its strengths and l…
Discriminative protein sequence modelling with Latent Space Diffusion
We explore a framework for protein sequence representation learning that decomposes the task between manifold learning and distributional modelling. Specifically we present a Latent Space Diffusion architecture which com…
DenoisingLanguage ModelingLanguage ModellingProperty Prediction+1EvoFlows: Evolutionary Edit-Based Flow-Matching for Protein Engineering
We introduce EvoFlows, a variable-length protein sequence-to-sequence modeling approach designed for protein engineering. Existing protein language models are poorly suited for optimization tasks: autoregressive models r…