raceBERT -- A Transformer-based Model for Predicting Race and Ethnicity from Names

2021-12-07 · Prasanna Parasurama

This paper presents raceBERT -- a transformer-based model for predicting race and ethnicity from character sequences in names, and an accompanying python package. Using a transformer-based model trained on a U.S. Florida voter registration dataset, the model predicts the likelihood of a name belonging to 5 U.S. census race categories (White, Black, Hispanic, Asian & Pacific Islander, American Indian & Alaskan Native). I build on Sood and Laohaprapanon (2018) by replacing their LSTM model with transformer-based models (pre-trained BERT model, and a roBERTa model trained from scratch), and compare the results. To the best of my knowledge, raceBERT achieves state-of-the-art results in race prediction using names, with an average f1-score of 0.86 -- a 4.1% improvement over the previous state-of-the-art, and improvements between 15-17% for non-white names.

📄 PDF Abstract BibTeX arXiv:2112.03807

Code (1)

parasurama/raceBERT 공식 구현 pytorch

Methods 이 논문이 사용한 방법론

American 설명 없음

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Tanh Activation 설명 없음

Sigmoid Activation 설명 없음

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Predicting Race and Ethnicity From the Sequence of Characters in a Name

2018-05-05 · Rajashekar Chintalapati, Suriyan Laohaprapanon, Gaurav Sood

To answer questions about racial inequality and fairness, we often need a way to infer race and ethnicity from names. One way to infer race and ethnicity from names is by relying on the Census Bureau's list of popular la…

Fairness

Rethnicity: Predicting Ethnicity from Names

2021-09-19 · Fangzhou Xie

In this study, a new R package, \texttt{rethnicity} is provided for predicting ethnicity based on names. The Bidirectional LSTM and Florida Voter Registration were used as the model and training data, respectively. Speci…

STraceBERT: Source Code Retrieval using Semantic Application Traces

2023-12-07 · Claudio Spiess

Software reverse engineering is an essential task in software engineering and security, but it can be a challenging process, especially for adversarial artifacts. To address this challenge, we present STraceBERT, a novel…

Retrieval

Race and ethnicity data for first, middle, and last names

2022-08-26 · Evan T. R. Rosenman, Santiago Olivella, Kosuke Imai

We provide the largest compiled publicly available dictionaries of first, middle, and last names for the purpose of imputing race and ethnicity using, for example, Bayesian Improved Surname Geocoding (BISG). The dictiona…

Imputation

Do Large Language Models Discriminate in Hiring Decisions on the Basis of Race, Ethnicity, and Gender?

2024-06-15 · Haozhe An, Christabel Acquaye, Colin Wang, Zongxia Li 외

We examine whether large language models (LLMs) exhibit race- and gender-based name discrimination in hiring decisions, similar to classic findings in the social sciences (Bertrand and Mullainathan, 2004). We design a se…