Embedding Style Beyond Topics: Analyzing Dispersion Effects Across Different Language Models
This paper analyzes how writing style affects the dispersion of embedding vectors across multiple, state-of-the-art language models. While early transformer models primarily aligned with topic modeling, this study examines the role of writing style in shaping embedding spaces. Using a literary corpus that alternates between topics and styles, we compare the sensitivity of language models across French and English. By analyzing the particular impact of style on embedding dispersion, we aim to better understand how language models process stylistic information, contributing to their overall interpretability.
Code (1)
Tasks
SensitivitySimilar Papers 제목 키워드 기반
Measuring Embedding Sensitivity to Authorial Style in French: Comparing Literary Texts with Language Model Rewritings
Large language models (LLMs) can convincingly imitate human writing styles, yet it remains unclear how much stylistic information is encoded in embeddings from any language model and retained after LLM rewriting. We inve…
Representing Mixtures of Word Embeddings with Mixtures of Topic Embeddings
A topic model is often formulated as a generative model that explains how each word of a document is generated given a set of topics and document-specific topic proportions. It is focused on capturing the word co-occurre…
parameter estimationWord EmbeddingsAn Image is Worth $K$ Topics: A Visual Structural Topic Model with Pretrained Image Embeddings
Political scientists are increasingly interested in analyzing visual content at scale. However, the existing computational toolbox is still in need of methods and models attuned to the specific challenges and goals of so…
Comparing Writing Styles using Word Embedding and Dynamic Time Warping
The development of plot or story in novels is reflected in the content and the words used. The flow of sentiments, which is one aspect of writing style, can be quantified by analyzing the flow of words. This study explor…
Dynamic Time WarpingBeats of Bias: Analyzing Lyrics with Topic Modeling and Gender Bias Measurements
This paper uses topic modeling and bias measurement techniques to analyze and determine gender bias in English song lyrics. We utilize BERTopic to cluster 537,553 English songs into distinct topics and chart their develo…
Word Embeddings