Divide and Conquer: An Ensemble Approach for Hostile Post Detection in Hindi

2021-01-20 · Varad Bhatnagar, Prince Kumar, Sairam Moghili, Pushpak Bhattacharyya

Recently the NLP community has started showing interest towards the challenging task of Hostile Post Detection. This paper present our system for Shared Task at Constraint2021 on "Hostile Post Detection in Hindi". The data for this shared task is provided in Hindi Devanagari script which was collected from Twitter and Facebook. It is a multi-label multi-class classification problem where each data instance is annotated into one or more of the five classes: fake, hate, offensive, defamation, and non-hostile. We propose a two level architecture which is made up of BERT based classifiers and statistical classifiers to solve this problem. Our team 'Albatross', scored 0.9709 Coarse grained hostility F1 score measure on Hostile Post Detection in Hindi subtask and secured 2nd rank out of 45 teams for the task. Our submission is ranked 2nd and 3rd out of a total of 156 submissions with Coarse grained hostility F1 score of 0.9709 and 0.9703 respectively. Our fine grained scores are also very encouraging and can be improved with further finetuning. The code is publicly available.

📄 PDF Abstract BibTeX arXiv:2101.07973

Code (1)

varadhbhatnagar/Hostile-Post-Detection-in-Hindi 공식 구현

Tasks

Multi-class Classification Multi-Label Text Classification

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Residual Connection 설명 없음

Adam 설명 없음

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Multi-Head Attention 설명 없음

Walk in Wild: An Ensemble Approach for Hostility Detection in Hindi Posts

2021-01-15 · Chander Shekhar, Bhavya Bagla, Kaushal Kumar Maurya, Maunendra Sankar Desarkar

As the reach of the internet increases, pejorative terms started flooding over social media platforms. This leads to the necessity of identifying hostile content on social media platforms. Identification of hostile conte…

Binary Classification Classification General Classification Multi-class Classification

Hostility Detection Dataset in Hindi

2020-11-06 · Mohit Bhardwaj, Md Shad Akhtar, Asif Ekbal, Amitava Das 외

In this paper, we present a novel hostility detection dataset in Hindi language. We collect and manually annotate ~8200 online posts. The annotated dataset covers four hostility dimensions: fake news, hate speech, offens…

Enhancing Representation Learning on High-Dimensional, Small-Size Tabular Data: A Divide and Conquer Method with Ensembled VAEs

2023-06-27 · Navindu Leelarathna, Andrei Margeloiu, Mateja Jamnik, Nikola Simidjievski

Variational Autoencoders and their many variants have displayed impressive ability to perform dimensionality reduction, often achieving state-of-the-art performance. Many current methods however, struggle to learn good r…

Data Augmentation Dimensionality Reduction Disentanglement Representation Learning

Hostility Detection in Hindi leveraging Pre-Trained Language Models

2021-01-14 · Ojasv Kamal, Adarsh Kumar, Tejas Vaidhya

Hostile content on social platforms is ever increasing. This has led to the need for proper detection of hostile posts so that appropriate action can be taken to tackle them. Though a lot of work has been done recently i…

Fake News Detection Hate Speech Detection Transfer Learning

Evaluation of Deep Learning Models for Hostility Detection in Hindi Text

2021-01-11 · Ramchandra Joshi, Rushabh Karnavat, Kaustubh Jirapure, Raviraj Joshi

The social media platform is a convenient medium to express personal thoughts and share useful information. It is fast, concise, and has the ability to reach millions. It is an effective place to archive thoughts, share …

Multi-Label Classification MUlTI-LABEL-ClASSIFICATION Text Detection Word Embeddings