홈 › Papers

HBert + BiasCorp -- Fighting Racism on the Web

2021-04-06 · Olawale Onabola, Zhuang Ma, Yang Xie, Benjamin Akera, Abdulrahman Ibraheem, Jia Xue, Dianbo Liu, Yoshua Bengio

Subtle and overt racism is still present both in physical and online communities today and has impacted many lives in different segments of the society. In this short piece of work, we present how we're tackling this societal issue with Natural Language Processing. We are releasing BiasCorp, a dataset containing 139,090 comments and news segment from three specific sources - Fox News, BreitbartNews and YouTube. The first batch (45,000 manually annotated) is ready for publication. We are currently in the final phase of manually labeling the remaining dataset using Amazon Mechanical Turk. BERT has been used widely in several downstream tasks. In this work, we present hBERT, where we modify certain layers of the pretrained BERT model with the new Hopfield Layer. hBert generalizes well across different distributions with the added advantage of a reduced model complexity. We are also releasing a JavaScript library and a Chrome Extension Application, to help developers make use of our trained model in web applications (say chat application) and for users to identify and report racially biased contents on the web respectively.

📄 PDF Abstract BibTeX arXiv:2104.02242

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Residual Connection 설명 없음

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Adam 설명 없음

Multi-Head Attention 설명 없음

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

hBERT + BiasCorp - Fighting Racism on the Web

2021-04-01 · EACL (LTEDI) 2021 4 · Olawale Onabola, Zhuang Ma, Xie Yang, Benjamin Akera 외

Fighting the COVID-19 Infodemic in Social Media: A Holistic Perspective and a Call to Arms

2020-07-15 · Firoj Alam, Fahim Dalvi, Shaden Shaar, Nadir Durrani 외

With the outbreak of the COVID-19 pandemic, people turned to social media to read and to share timely information including statistics, warnings, advice, and inspirational stories. Unfortunately, alongside all this usefu…

Misinformation

MathBERT: A Pre-trained Language Model for General NLP Tasks in Mathematics Education

2021-06-02 · Jia Tracy Shen, Michiharu Yamashita, Ethan Prihar, Neil Heffernan 외

Since the introduction of the original BERT (i.e., BASE BERT), researchers have developed various customized BERT models with improved performance for specific domains and tasks by exploiting the benefits of transfer lea…

Knowledge Tracing Language Modeling Language Modelling Math+1

TurkishBERTweet: Fast and Reliable Large Language Model for Social Media Analysis

2023-11-29 · Ali Najafi, Onur Varol

Turkish is one of the most popular languages in the world. Wide us of this language on social media platforms such as Twitter, Instagram, or Tiktok and strategic position of the country in the world politics makes it app…

Hate Speech Detection Language Modeling Language Modelling Large Language Model+4

RoChBert: Towards Robust BERT Fine-tuning for Chinese

2022-10-28 · Zihan Zhang, Jinfeng Li, Ning Shi, Bo Yuan 외

Despite of the superb performance on a wide range of tasks, pre-trained language models (e.g., BERT) have been proved vulnerable to adversarial texts. In this paper, we present RoChBERT, a framework to build more Robust …

Data Augmentation Language Modeling Language Modelling