Application of Mix-Up Method in Document Classification Task Using BERT
The mix-up method (Zhang et al., 2017), one of the methods for data augmentation, is known to be easy to implement and highly effective. Although the mix-up method is intended for image identification, it can also be applied to natural language processing. In this paper, we attempt to apply the mix-up method to a document classification task using bidirectional encoder representations from transformers (BERT) (Devlin et al., 2018). Since BERT allows for two-sentence input, we concatenated word sequences from two documents with different labels and used the multi-class output as the supervised data with a one-hot vector. In an experiment using the livedoor news corpus, which is Japanese, we compared the accuracy of document classification using two methods for selecting documents to be concatenated with that of ordinary document classification. As a result, we found that the proposed method is better than the normal classification when the documents with labels shortages are mixed preferentially. This indicates that how to choose documents for mix-up has a significant impact on the results.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationData AugmentationDocument ClassificationSentenceSimilar Papers 제목 키워드 기반
DocBERT: BERT for Document Classification
We present, to our knowledge, the first application of BERT to document classification. A few characteristics of the task might lead one to think that BERT is not the most appropriate model: syntactic structures matter l…
ClassificationDocument ClassificationGeneral ClassificationSentiment Analysis+1Effectively Leveraging BERT for Legal Document Classification
Bidirectional Encoder Representations from Transformers (BERT) has achieved state-of-the-art performances on several text classification tasks, such as GLUE and sentiment analysis. Recent work in the legal domain started…
Binary ClassificationClassificationDocument ClassificationSentiment Analysis+2A Sentence-level Hierarchical BERT Model for Document Classification with Limited Labelled Data
Training deep learning models with limited labelled data is an attractive scenario for many NLP tasks, including document classification. While with the recent emergence of BERT, deep learning language models can achieve…
ClassificationDocument ClassificationSentenceHierarchical BERT for Medical Document Understanding
Medical document understanding has gained much attention recently. One representative task is the International Classification of Disease (ICD) diagnosis code assignment. Existing work adopts either RNN or CNN as the bac…
document understandingSentenceAspect-based Document Similarity for Research Papers
Traditional document similarity measures provide a coarse-grained distinction between similar and dissimilar documents. Typically, they do not consider in what aspects two documents are similar. This limits the granulari…
Document ClassificationRecommendation Systems