Fine-tuning Pre-trained Contextual Embeddings for Citation Content Analysis in Scholarly Publication
Citation function and citation sentiment are two essential aspects of citation content analysis (CCA), which are useful for influence analysis, the recommendation of scientific publications. However, existing studies are mostly traditional machine learning methods, although deep learning techniques have also been explored, the improvement of the performance seems not significant due to insufficient training data, which brings difficulties to applications. In this paper, we propose to fine-tune pre-trained contextual embeddings ULMFiT, BERT, and XLNet for the task. Experiments on three public datasets show that our strategy outperforms all the baselines in terms of the F1 score. For citation function identification, the XLNet model achieves 87.2%, 86.90%, and 81.6% on DFKI, UMICH, and TKDE2019 datasets respectively, while it achieves 91.72% and 91.56% on DFKI and UMICH in term of citation sentiment identification. Our method can be used to enhance the influence analysis of scholars and scholarly publications.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Contextualized Embeddings based Transformer Encoder for Sentence Similarity Modeling in Answer Selection Task
Word embeddings that consider context have attracted great attention for various natural language processing tasks in recent years. In this paper, we utilize contextualized word embeddings with the transformer encoder fo…
Answer SelectionSentenceSentence SimilarityWord EmbeddingsAdapting Pretrained Language Models for Citation Classification via Self-Supervised Contrastive Learning
Citation classification, which identifies the intention behind academic citations, is pivotal for scholarly analysis. Previous works suggest fine-tuning pretrained language models (PLMs) on citation classification datase…
Contrastive LearningDecoderSentenceContextual Squeeze-and-Excitation for Efficient Few-Shot Image Classification
Recent years have seen a growth in user-centric applications that require effective knowledge transfer across tasks in the low-data regime. An example is personalization, where a pretrained system is adapted by learning …
Few-Shot Image ClassificationFew-Shot Learningimage-classificationImage Classification+1Improved Biomedical Word Embeddings in the Transformer Era
Biomedical word embeddings are usually pre-trained on free text corpora with neural methods that capture local and global distributional properties. They are leveraged in downstream tasks using various neural architectur…
SentenceWord EmbeddingsWord SimilarityDebiasing Pre-trained Contextualised Embeddings
In comparison to the numerous debiasing methods proposed for the static non-contextualised word embeddings, the discriminative biases in contextualised embeddings have received relatively little attention. We propose a f…
SentenceWord Embeddings