Deep Generative Networks For Sequence Prediction
This thesis investigates unsupervised time series representation learning for sequence prediction problems, i.e. generating nice-looking input samples given a previous history, for high dimensional input sequences by decoupling the static input representation from the recurrent sequence representation. We introduce three models based on Generative Stochastic Networks (GSN) for unsupervised sequence learning and prediction. Experimental results for these three models are presented on pixels of sequential handwritten digit (MNIST) data, videos of low-resolution bouncing balls, and motion capture data. The main contribution of this thesis is to provide evidence that GSNs are a viable framework to learn useful representations of complex sequential input data, and to suggest a new framework for deep generative models to learn complex sequences by decoupling static input representations from dynamic time dependency representations.
Code (1)
Tasks
PredictionRepresentation LearningTime SeriesTime Series AnalysisSimilar Papers 제목 키워드 기반
RITA: a Study on Scaling Up Generative Protein Sequence Models
In this work we introduce RITA: a suite of autoregressive generative models for protein sequences, with up to 1.2 billion parameters, trained on over 280 million protein sequences belonging to the UniRef-100 database. Su…
PredictionProtein DesignGenerative Temporal Link Prediction via Self-tokenized Sequence Modeling
We formalize networks with evolving structures as temporal networks and propose a generative link prediction model, Generative Link Sequence Modeling (GLSM), to predict future links for temporal networks. GLSM captures t…
Link PredictionPredictionHow Does Beam Search improve Span-Level Confidence Estimation in Generative Sequence Labeling?
Sequence labeling is a core task in text understanding for IE/IR systems. Text generation models have increasingly become the go-to solution for such tasks (e.g., entity extraction and dialog slot filling). While most re…
slot-fillingSlot FillingText GenerationGenerative Bridging Network in Neural Sequence Prediction
In order to alleviate data sparsity and overfitting problems in maximum likelihood estimation (MLE) for sequence prediction tasks, we propose the Generative Bridging Network (GBN), in which a novel bridge module is intro…
Abstractive Text SummarizationLanguage ModelingLanguage ModellingMachine Translation+3Generative Bridging Network for Neural Sequence Prediction
In order to alleviate data sparsity and overfitting problems in maximum likelihood estimation (MLE) for sequence prediction tasks, we propose the Generative Bridging Network (GBN), in which a novel bridge module is intro…
Abstractive Text SummarizationImage CaptioningLanguage ModelingLanguage Modelling+6