When do Words Matter? Understanding the Impact of Lexical Choice on Audience Perception using Individual Treatment Effect Estimation
Studies across many disciplines have shown that lexical choice can affect audience perception. For example, how users describe themselves in a social media profile can affect their perceived socio-economic status. However, we lack general methods for estimating the causal effect of lexical choice on the perception of a specific sentence. While randomized controlled trials may provide good estimates, they do not scale to the potentially millions of comparisons necessary to consider all lexical choices. Instead, in this paper, we first offer two classes of methods to estimate the effect on perception of changing one word to another in a given sentence. The first class of algorithms builds upon quasi-experimental designs to estimate individual treatment effects from observational data. The second class treats treatment effect estimation as a classification problem. We conduct experiments with three data sources (Yelp, Twitter, and Airbnb), finding that the algorithmic estimates align well with those produced by randomized-control trials. Additionally, we find that it is possible to transfer treatment effect classifiers across domains and still maintain high accuracy.
Code (0)
등록된 구현이 없습니다.
Tasks
SentenceSimilar Papers 제목 키워드 기반
When classifying grammatical role, BERT doesn't care about word order... except when it matters
Because meaning can often be inferred from lexical semantics alone, word order is often a redundant cue in natural language. For example, the words cut, chef, and onion are more likely used to convey "The chef cut the on…
When classifying grammatical role, BERT doesn't care about word order... except when it matters
Because meaning can often be inferred from lexical semantics alone, word order is often a redundant cue in natural language. For example, the words chopped, chef, and onion are more likely used to convey "The chef choppe…
When classifying grammatical role, BERT doesn’t care about word order... except when it matters
Because meaning can often be inferred from lexical semantics alone, word order is often a redundant cue in natural language. For example, the words chopped, chef, and onion are more likely used to convey “The chef choppe…
Local Structure Matters Most in Most Languages
Many recent perturbation studies have found unintuitive results on what does and does not matter when performing Natural Language Understanding (NLU) tasks in English. Coding properties, such as the order of words, can o…
Natural Language UnderstandingWords that Matter: The Impact of Negative Words on News Sentiment and Stock Market Index
This study investigates the impact of negative words on sentiment analysis and its effect on the South Korean stock market index, KOSPI200. The research analyzes a dataset of 45,723 South Korean daily economic news artic…
ArticlesSentiment Analysis