The Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Task
A writer's style depends not just on personal traits but also on her intent and mental state. In this paper, we show how variants of the same writing task can lead to measurable differences in writing style. We present a case study based on the story cloze task (Mostafazadeh et al., 2016a), where annotators were assigned similar writing tasks with different constraints: (1) writing an entire story, (2) adding a story ending for a given story context, and (3) adding an incoherent ending to a story. We show that a simple linear classifier informed by stylistic features is able to successfully distinguish among the three cases, without even looking at the story context. In addition, combining our stylistic features with language model predictions reaches state of the art performance on the story cloze challenge. Our results demonstrate that different task framings can dramatically affect the way people write.
Code (1)
Tasks
Language ModelingLanguage ModellingSimilar Papers 제목 키워드 기반
Mode Effects' Challenge to Authorship Attribution
The success of authorship attribution relies on the presence of linguistic features specific to individual authors. There is, however, limited research assessing to what extent authorial style remains constant when indiv…
Authorship AttributionFeature EngineeringSentenceTowards Improved Model Design for Authorship Identification: A Survey on Writing Style Understanding
Authorship identification tasks, which rely heavily on linguistic styles, have always been an important part of Natural Language Understanding (NLU) research. While other tasks based on linguistic style understanding ben…
BIG-bench Machine LearningNatural Language UnderstandingSurveyHow to Write High-quality News on Social Network? Predicting News Quality by Mining Writing Style
Rapid development of Internet technologies promotes traditional newspapers to report news on social networks. However, people on social networks may have different needs which naturally arises the question: whether can w…
Persona-Augmented Benchmarking: Evaluating LLMs Across Diverse Writing Styles
Current benchmarks for evaluating Large Language Models (LLMs) often do not exhibit enough writing style diversity, with many adhering primarily to standardized conventions. Such benchmarks do not fully capture the rich …
Investigating writing style as a contributor to gender gaps in science and technology
A growing stream of research finds that scientific contributions are evaluated differently depending on the gender of the author. In this article, we consider whether gender differences in writing styles - how men and wo…