paper-with-me

Papers

Gender Bias in Transformer Models: A comprehensive survey

2023-06-18 · Praneeth Nemani, Yericherla Deepak Joel, Palla Vijay, Farhana Ferdousi Liza

Gender bias in artificial intelligence (AI) has emerged as a pressing concern with profound implications for individuals' lives. This paper presents a comprehensive survey that explores gender bias in Transformer models from a linguistic perspective. While the existence of gender bias in language models has been acknowledged in previous studies, there remains a lack of consensus on how to effectively measure and evaluate this bias. Our survey critically examines the existing literature on gender bias in Transformers, shedding light on the diverse methodologies and metrics employed to assess bias. Several limitations in current approaches to measuring gender bias in Transformers are identified, encompassing the utilization of incomplete or flawed metrics, inadequate dataset sizes, and a dearth of standardization in evaluation methods. Furthermore, our survey delves into the potential ramifications of gender bias in Transformers for downstream applications, including dialogue systems and machine translation. We underscore the importance of fostering equity and fairness in these systems by emphasizing the need for heightened awareness and accountability in developing and deploying language technologies. This paper serves as a comprehensive overview of gender bias in Transformer models, providing novel insights and offering valuable directions for future research in this critical domain.

📄 PDF Abstract BibTeX arXiv:2306.10530

Code (0)

등록된 구현이 없습니다.

Tasks

FairnessMachine TranslationSurvey

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음
Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Position-Wise Feed-Forward Layer 설명 없음
Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
Adam 설명 없음
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Similar Papers 제목 키워드 기반

A Survey on Gender Bias in Natural Language Processing

2021-12-28 · Karolina Stanczak, Isabelle Augenstein

Language can be used as a means of reproducing and enforcing harmful stereotypes and biases and has been analysed as such in numerous research. In this paper, we present a survey of 304 papers on gender bias in natural l…

Survey

Theories of "Gender" in NLP Bias Research

2022-05-05 · Hannah Devinney, Jenny Björklund, Henrik Björklund

The rise of concern around Natural Language Processing (NLP) technologies containing and perpetuating social biases has led to a rich and rapidly growing area of research. Gender bias is one of the central biases being a…

Articles

Survey of Bias In Text-to-Image Generation: Definition, Evaluation, and Mitigation

2024-04-01 · Yixin Wan, Arjun Subramonian, Anaelia Ovalle, Zongyu Lin 외

The recent advancement of large and powerful models with Text-to-Image (T2I) generation abilities -- such as OpenAI's DALLE-3 and Google's Gemini -- enables users to generate high-quality images from textual prompts. How…

Image GenerationText to Image GenerationText-to-Image Generation

Gender Bias in Machine Translation and The Era of Large Language Models

2024-01-18 · Eva Vanmassenhove

This chapter examines the role of Machine Translation in perpetuating gender bias, highlighting the challenges posed by cross-linguistic settings and statistical dependencies. A comprehensive overview of relevant existin…

FairnessMachine TranslationTranslation

Towards a Comprehensive Understanding and Accurate Evaluation of Societal Biases in Pre-Trained Transformers

2021-06-01 · NAACL 2021 4 · Andrew Silva, Pradyumna Tambwekar, Matthew Gombolay

The ease of access to pre-trained transformers has enabled developers to leverage large-scale language models to build exciting applications for their users. While such pre-trained models offer convenient starting points…