Interacting with next-phrase suggestions: How suggestion systems aid and influence the cognitive processes of writing
Writing with next-phrase suggestions powered by large language models is becoming more pervasive by the day. However, research to understand writers' interaction and decision-making processes while engaging with such systems is still emerging. We conducted a qualitative study to shed light on writers' cognitive processes while writing with next-phrase suggestion systems. To do so, we recruited 14 amateur writers to write two reviews each, one without suggestions and one with suggestions. Additionally, we also positively and negatively biased the suggestion system to get a diverse range of instances where writers' opinions and the bias in the language model align or misalign to varying degrees. We found that writers interact with next-phrase suggestions in various complex ways: Writers abstracted and extracted multiple parts of the suggestions and incorporated them within their writing, even when they disagreed with the suggestion as a whole; along with evaluating the suggestions on various criteria. The suggestion system also had various effects on the writing process, such as altering the writer's usual writing plans, leading to higher levels of distraction etc. Based on our qualitative analysis using the cognitive process model of writing by Hayes as a lens, we propose a theoretical model of 'writer-suggestion interaction' for writing with GPT-2 (and causal language models in general) for a movie review writing task, followed by directions for future research and design.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingLanguage ModellingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers
We present an in-depth analysis of the impact of multi-word suggestion choices from a neural language model on user behaviour regarding input and text composition in email writing. Our study for the first time compares d…
Language ModelingLanguage ModellingGenerating Query Suggestions to Support Task-Based Search
We address the problem of generating query suggestions to support users in completing their underlying tasks (which motivated them to search in the first place). Given an initial query, these query suggestions should pro…
How do people interact with biased text prediction models while writing?
Recent studies have shown that a bias in thetext suggestions system can percolate in theuser’s writing. In this pilot study, we ask thequestion: How do people interact with text pre-diction models, in an inline next phra…
Smart Word Suggestions for Writing Assistance
Enhancing word usage is a desired feature for writing assistance. To further advance research in this area, this paper introduces "Smart Word Suggestions" (SWS) task and benchmark. Unlike other works, SWS emphasizes end-…
Counterfactual Language Model Adaptation for Suggesting Phrases
Mobile devices use language models to suggest words and phrases for use in text entry. Traditional language models are based on contextual word frequency in a static corpus of text. However, certain types of phrases, whe…
counterfactualLanguage ModelingLanguage Modellingmodel