paper-with-me

홈 › Papers

Pairwise Multi-Class Document Classification for Semantic Relations between Wikipedia Articles

2020-03-22 · Malte Ostendorff, Terry Ruas, Moritz Schubotz, Georg Rehm, Bela Gipp

Many digital libraries recommend literature to their users considering the similarity between a query document and their repository. However, they often fail to distinguish what is the relationship that makes two documents alike. In this paper, we model the problem of finding the relationship between two documents as a pairwise document classification task. To find the semantic relation between documents, we apply a series of techniques, such as GloVe, Paragraph-Vectors, BERT, and XLNet under different configurations (e.g., sequence length, vector concatenation scheme), including a Siamese architecture for the Transformer-based systems. We perform our experiments on a newly proposed dataset of 32,168 Wikipedia article pairs and Wikidata properties that define the semantic document relations. Our results show vanilla BERT as the best performing system with an F1-score of 0.93, which we manually examine to better understand its applicability to other domains. Our findings suggest that classifying semantic relations between documents is a solvable task and motivates the development of recommender systems based on the evaluated techniques. The discussions in this paper serve as first steps in the exploration of documents through SPARQL-like queries such that one could find documents that are similar in one aspect but dissimilar in another.

📄 PDF Abstract BibTeX arXiv:2003.09881

Code (4)

malteos/semantic-document-relations 공식 구현 pytorch
masukislam/DS8008_NLP pytorch
wimurad1981/DS8008-NLP pytorch
wimurad1981/DS8008-NLP-Project pytorch

Tasks

ArticlesDocument ClassificationGeneral ClassificationRecommendation Systems

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
GloVe GloVe Embeddings are a type of word embedding that encode the co-occurrence probability ratio between two words as vector differences. GloVe uses a weighted least squares…
Residual Connection 설명 없음
Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…
Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…
Weight Decay 설명 없음
SentencePiece 설명 없음

Similar Papers 제목 키워드 기반

Specialized Document Embeddings for Aspect-based Similarity of Research Papers

2022-03-28 · Malte Ostendorff, Till Blume, Terry Ruas, Bela Gipp 외

Document embeddings and similarity measures underpin content-based recommender systems, whereby a document is commonly represented as a single generic embedding. However, similarity computed on single vector representati…

Document ClassificationRecommendation SystemsSegmentation

Pairwise Instance Relation Augmentation for Long-tailed Multi-label Text Classification

2022-11-19 · Lin Xiao, Pengyu Xu, Liping Jing, Xiangliang Zhang

Multi-label text classification (MLTC) is one of the key tasks in natural language processing. It aims to assign multiple target labels to one document. Due to the uneven popularity of labels, the number of documents per…

DiversityMulti Label Text ClassificationMulti-Label Text ClassificationRelation+2

Unsupervised Semantic Hashing with Pairwise Reconstruction

2020-07-01 · Casper Hansen, Christian Hansen, Jakob Grue Simonsen, Stephen Alstrup 외

Semantic Hashing is a popular family of methods for efficient similarity search in large-scale datasets. In Semantic Hashing, documents are encoded as short binary vectors (i.e., hash codes), such that semantic similarit…

DecoderSemantic SimilaritySemantic Textual Similarity

Semantic Matching of Documents from Heterogeneous Collections: A Simple and Transparent Method for Practical Applications

2019-04-29 · Mark-Christoph Müller

We present a very simple, unsupervised method for the pairwise matching of documents from heterogeneous collections. We demonstrate our method with the Concept-Project matching task, which is a binary classification task…

Binary ClassificationGeneral Classification

Semantic Matching of Documents from Heterogeneous Collections: A Simple and Transparent Method for Practical Applications

2019-05-01 · WS 2019 5 · Mark-Christoph Mueller

We present a very simple, unsupervised method for the pairwise matching of documents from heterogeneous collections. We demonstrate our method with the Concept-Project matching task, which is a binary classification task…

Binary Classification