Exploring Story Generation with Multi-task Objectives in Variational Autoencoders

2021-11-15 · ALTA 2021 12 · Zhuohan Xie, Trevor Cohn, Jey Han Lau

GPT-2 has been frequently adapted in story generation models as it provides powerful generative capability. However, it still fails to generate consistent stories and lacks diversity. Current story generation models leverage additional information such as plots or commonsense into GPT-2 to guide the generation process. These approaches focus on improving generation quality of stories while our work look at both quality and diversity. We explore combining BERT and GPT-2 to build a variational autoencoder (VAE), and extend it by adding additional objectives to learn global features such as story topic and discourse relations. Our evaluations show our enhanced VAE can provide better quality and diversity trade off, generate less repetitive story content and learn a more informative latent variable.

📄 PDF Abstract BibTeX arXiv:2111.08133

Code (0)

등록된 구현이 없습니다.

Tasks

DiversityStory Generation

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

WordPiece 설명 없음

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

Adam 설명 없음

BERT BERT, or Bidirectional Encoder Representations from Transformers, improves upon standard Transformers by removing the…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

LSDSem 2017: Exploring Data Generation Methods for the Story Cloze Test

2017-04-01 · WS 2017 4 · Michael Bugert, Yevgeniy Puzikov, Andreas R{\"u}ckl{\'e}, Judith Eckle-Kohler 외

The Story Cloze test is a recent effort in providing a common test scenario for text understanding systems. As part of the LSDSem 2017 shared task, we present a system based on a deep learning architecture combined with …

Cloze Test

Robust and Versatile Bipedal Jumping Control through Reinforcement Learning

2023-02-19 · Zhongyu Li, Xue Bin Peng, Pieter Abbeel, Sergey Levine 외

This work aims to push the limits of agility for bipedal robots by enabling a torque-controlled bipedal robot to perform robust and versatile dynamic jumps in the real world. We present a reinforcement learning framework…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Storyfier: Exploring Vocabulary Learning Support with Text Generation Models

2023-08-07 · Zhenhui Peng, Xingbo Wang, Qiushi Han, Junkai Zhu 외

Vocabulary learning support tools have widely exploited existing materials, e.g., stories or video clips, as contexts to help users memorize each target word. However, these tools could not provide a coherent context for…

Cloze Test Text Generation

Incorporating Textual Evidence in Visual Storytelling

2019-11-21 · WS 2019 11 · Tianyi Li, Sujian Li

Previous work on visual storytelling mainly focused on exploring image sequence as evidence for storytelling and neglected textual evidence for guiding story generation. Motivated by human storytelling process which reca…

Object Recognition Story Generation Visual Storytelling

Knowledge-enriched Attention Network with Group-wise Semantic for Visual Storytelling

2022-03-10 · Tengpeng Li, Hanli Wang, Bin He, Chang Wen Chen

As a technically challenging topic, visual storytelling aims at generating an imaginary and coherent story with narrative multi-sentences from a group of relevant images. Existing methods often generate direct and rigid …

Decoder Story Generation Visual Storytelling