Real-time interactive sequence generation and control with Recurrent Neural Network ensembles
Recurrent Neural Networks (RNN), particularly Long Short Term Memory (LSTM) RNNs, are a popular and very successful method for learning and generating sequences. However, current generative RNN techniques do not allow real-time interactive control of the sequence generation process, thus aren't well suited for live creative expression. We propose a method of real-time continuous control and 'steering' of sequence generation using an ensemble of RNNs and dynamically altering the mixture weights of the models. We demonstrate the method using character based LSTM networks and a gestural interface allowing users to 'conduct' the generation of text.
Code (0)
등록된 구현이 없습니다.
Tasks
continuous-controlContinuous ControlMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Interactive Music Generation with Positional Constraints using Anticipation-RNNs
Recurrent Neural Networks (RNNS) are now widely used on sequence generation tasks due to their ability to learn long-range dependencies and to generate sequences of arbitrary length. However, their left-to-right generati…
Music GenerationRealCam: Real-Time Novel-View Video Generation with Interactive Camera Control
Camera-controlled video-to-video (V2V) generation enables dynamic viewpoint synthesis from monocular footage, holding immense potential for interactive filmmaking and live broadcasting. However, existing implicit synthes…
Data AugmentationVideo GenerationHunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Condition
Recent advances in diffusion-based and controllable video generation have enabled high-quality and temporally coherent video synthesis, laying the groundwork for immersive interactive gaming experiences. However, current…
Temporal SequencesVideo GenerationControllable Long-term Motion Generation with Extended Joint Targets
Generating stable and controllable character motion in real-time is a key challenge in computer animation. Existing methods often fail to provide fine-grained control or suffer from motion degradation over long sequences…
Style TransferGameGen-X: Interactive Open-world Game Video Generation
We introduce GameGen-X, the first diffusion transformer model specifically designed for both generating and interactively controlling open-world game videos. This model facilitates high-quality, open-domain generation by…
Text-to-Video GenerationVideo Generation