Fine-grained domain classification using Transformers
The introduction of transformers in 2017 and successively BERT in 2018 brought about a revolution in the field of natural language processing. Such models are pretrained on vast amounts of data, and are easily extensible to be used for a wide variety of tasks through transfer learning. Continual work on transformer based architectures has led to a variety of new models with state of the art results. RoBERTa(CITATION) is one such model, which brings about a series of changes to the BERT architecture and is capable of producing better quality embeddings at an expense of functionality. In this paper, we attempt to solve the well known text classification task of fine-grained domain classification using BERT and RoBERTa and perform a comparative analysis of the same. We also attempt to evaluate the impact of data preprocessing specially in the context of fine-grained domain classification. The results obtained outperformed all the other models at the ICON TechDOfication 2020 (subtask-2a) Fine-grained domain classification task and ranked first. This proves the effectiveness of our approach.
Code (0)
등록된 구현이 없습니다.
Tasks
Classificationdomain classificationtext-classificationText ClassificationTransfer LearningSimilar Papers 제목 키워드 기반
Performance of computer vision algorithms for fine-grained classification using crowdsourced insect images
With fine-grained classification, we identify unique characteristics to distinguish among classes of the same super-class. We are focusing on species recognition in Insecta, as they are critical for biodiversity monitori…
Transfer Learning for Fine-grained Classification Using Semi-supervised Learning and Visual Transformers
Fine-grained classification is a challenging task that involves identifying subtle differences between objects within the same category. This task is particularly challenging in scenarios where data is scarce. Visual tra…
Classificationimage-classificationImage ClassificationTransfer LearningAre Pre-trained Transformers Robust in Intent Classification? A Missing Ingredient in Evaluation of Out-of-Scope Intent Detection
Pre-trained Transformer-based models were reported to be robust in intent classification. In this work, we first point out the importance of in-domain out-of-scope detection in few-shot intent recognition tasks and then …
intent-classificationIntent ClassificationIntent DetectionIntent RecognitionDynamic Grained Encoder for Vision Transformers
Transformers, the de-facto standard for language modeling, have been recently applied for vision tasks. This paper introduces sparse queries for vision transformers to exploit the intrinsic spatial redundancy of natural …
image-classificationImage ClassificationLanguage ModelingLanguage Modelling+2Attention at SemEval-2023 Task 10: Explainable Detection of Online Sexism (EDOS)
In this paper, we have worked on interpretability, trust, and understanding of the decisions made by models in the form of classification tasks. The task is divided into 3 subtasks. The first task consists of determining…
Classification