ConchShell: A Generative Adversarial Networks that Turns Pictures into Piano Music
We present ConchShell, a multi-modal generative adversarial framework that takes pictures as input to the network and generates piano music samples that match the picture context. Inspired by I3D, we introduce a novel image feature representation method: time-convolutional neural network (TCNN), which is used to forge features for images in the temporal dimension. Although our image data consists of only six categories, our proposed framework will be innovative and commercially meaningful. The project will provide technical ideas for work such as 3D game voice overs, short-video soundtracks, and real-time generation of metaverse background music.We have also released a new dataset, the Beach-Ocean-Piano Dataset (BOPD) 1, which contains more than 3,000 images and more than 1,500 piano pieces. This dataset will support multimodal image-to-music research.
Code (1)
Similar Papers 제목 키워드 기반
Multiple Generative Adversarial Networks Analysis for Predicting Photographers' Retouching
Anyone can take a photo, but not everybody has the ability to retouch their pictures and obtain result close to professional. Since it is not possible to ask experts to retouch thousands of pictures, we thought about tea…
The power of pictures: using ML assisted image generation to engage the crowd in complex socioscientific problems
Human-computer image generation using Generative Adversarial Networks (GANs) is becoming a well-established methodology for casual entertainment and open artistic exploration. Here, we take the interaction a step further…
Image GenerationCorrGAN: Sampling Realistic Financial Correlation Matrices Using Generative Adversarial Networks
We propose a novel approach for sampling realistic financial correlation matrices. This approach is based on generative adversarial networks. Experiments demonstrate that generative adversarial networks are able to recov…
Diffuse Map Guiding Unsupervised Generative Adversarial Network for SVBRDF Estimation
Reconstructing materials in the real world has always been a difficult problem in computer graphics. Accurately reconstructing the material in the real world is critical in the field of realistic rendering. Traditionally…
Generative Adversarial NetworkSVBRDF EstimationEffects of Dataset properties on the training of GANs
Generative Adversarial Networks are a new family of generative models, frequently used for generating photorealistic images. The theory promises for the GAN to eventually reach an equilibrium where generator produces pic…