Can Demographic Factors Improve Text Classification? Revisiting Demographic Adaptation in the Age of Transformers

2022-10-13 · Chia-Chien Hung, Anne Lauscher, Dirk Hovy, Simone Paolo Ponzetto, Goran Glavaš

Demographic factors (e.g., gender or age) shape our language. Previous work showed that incorporating demographic factors can consistently improve performance for various NLP tasks with traditional NLP models. In this work, we investigate whether these previous findings still hold with state-of-the-art pretrained Transformer-based language models (PLMs). We use three common specialization methods proven effective for incorporating external knowledge into pretrained Transformers (e.g., domain-specific or geographic knowledge). We adapt the language representations for the demographic dimensions of gender and age, using continuous language modeling and dynamic multi-task learning for adaptation, where we couple language modeling objectives with the prediction of demographic classes. Our results, when employing a multilingual PLM, show substantial gains in task performance across four languages (English, German, French, and Danish), which is consistent with the results of previous work. However, controlling for confounding factors - primarily domain and language proficiency of Transformer-based PLMs - shows that downstream performance gains from our demographic adaptation do not actually stem from demographic knowledge. Our results indicate that demographic specialization of PLMs, while holding promise for positive societal impact, still represents an unsolved problem for (modern) NLP.

📄 PDF Abstract BibTeX arXiv:2210.07362

Code (1)

umanlp/socioadapt 공식 구현 pytorch

Tasks

Language ModelingLanguage Modelling Multi-Task Learning text-classification Text Classification

Neural User Factor Adaptation for Text Classification: Learning to Generalize Across Author Demographics

2019-06-01 · SEMEVAL 2019 6 · Xiaolei Huang, Michael J. Paul

Language use varies across different demographic factors, such as gender, age, and geographic location. However, most existing document classification methods ignore demographic variability. In this study, we examine emp…

Classification Document Classification General Classification text-classification+1

Demographic Factors Improve Classification Performance

2015-07-01 · IJCNLP 2015 7 · Dirk Hovy

Classification Domain Adaptation General Classification Text Classification

Zero-shot Fairness with Invisible Demographics

2021-01-01 · Thomas Kehrenberg, Viktoriia Sharmanska, Myles Scott Bartlett, Novi Quadrianto

In a statistical notion of algorithmic fairness, we partition individuals into groups based on some key demographic factors such as race and gender, and require that some statistics of a classifier be approximately equal…

Disentanglement Fairness

Deep Learning feature selection to unhide demographic recommender systems factors

2020-06-17 · Jesús Bobadilla, Ángel González-Prieto, Fernando Ortega, Raúl Lara-Cabrera

Extracting demographic features from hidden factors is an innovative concept that provides multiple and relevant applications. The matrix factorization model generates factors which do not incorporate semantic knowledge.…

Collaborative Filtering Deep Learning Fairness feature selection+2

Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognition

2020-02-24 · LREC 2020 5 · Xiaolei Huang, Linzi Xing, Franck Dernoncourt, Michael J. Paul

Existing research on fairness evaluation of document classification models mainly uses synthetic monolingual data without ground truth for author demographic attributes. In this work, we assemble and publish a multilingu…

Document Classification Fairness Hate Speech Detection speech-recognition+1