Automatic Generation of German Drama Texts Using Fine Tuned GPT-2 Models

2023-01-08 · Mariam Bangura, Kristina Barabashova, Anna Karnysheva, Sarah Semczuk, Yifan Wang

This study is devoted to the automatic generation of German drama texts. We suggest an approach consisting of two key steps: fine-tuning a GPT-2 model (the outline model) to generate outlines of scenes based on keywords and fine-tuning a second model (the generation model) to generate scenes from the scene outline. The input for the neural model comprises two datasets: the German Drama Corpus (GerDraCor) and German Text Archive (Deutsches Textarchiv or DTA). In order to estimate the effectiveness of the proposed method, our models are compared with baseline GPT-2 models. Our models perform well according to automatic quantitative evaluation, but, conversely, manual qualitative analysis reveals a poor quality of generated texts. This may be due to the quality of the dataset or training inputs.

📄 PDF Abstract BibTeX arXiv:2301.03119

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

Residual Connection 설명 없음

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Cosine Annealing Cosine Annealing is a type of learning rate schedule that has the effect of starting with a large learning rate that is relatively rapidly decreased to a minimum value before…

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

GerDraCor-Coref: A Coreference Corpus for Dramatic Texts in German

2020-05-01 · LREC 2020 5 · Janis Pagel, Nils Reiter

Dramatic texts are a highly structured literary text type. Their quantitative analysis so far has relied on analysing structural properties (e.g., in the form of networks). Resolving coreferences is crucial for an analys…

coreference-resolution Coreference Resolution

German Text Simplification: Finetuning Large Language Models with Semi-Synthetic Data

2024-02-16 · Lars Klöser, Mika Beele, Jan-Niklas Schagen, Bodo Kraft

This study pioneers the use of synthetically generated data for training generative models in document-level text simplification of German texts. We demonstrate the effectiveness of our approach with real-world online te…

Text Simplification

Fine-grained German Sentiment Analysis on Social Media

2012-05-01 · LREC 2012 5 · Saeedeh Momtazi

Expressing opinions and emotions on social media becomes a frequent activity in daily life. People express their opinions about various targets via social media and they are also interested to know about other opinions o…

Sentiment Analysis

Exploring Automatic Text Simplification of German Narrative Documents

2023-12-15 · Thorben Schomacker, Tillmann Dönicke, Marina Tropmann-Frick

In this paper, we apply transformer-based Natural Language Generation (NLG) techniques to the problem of text simplification. Currently, there are only a few German datasets available for text simplification, even fewer …

Text Generation Text Simplification

TINY_SCHILLER: A Drop-In German Drama Corpus for Small Language Models

2026-07-22 · Mark Schutera arxiv

tiny_schiller closes the small-language-model prototyping, fine-tuning, education, and research gap for German literary text, providing a single-file, drop-in counterpart to Karpathy's tiny_shakespeare. The available Ger…