BART-TL: Weakly-Supervised Topic Label Generation

2021-04-01 · EACL 2021 2 · Cristian Popa, Traian Rebedea

We propose a novel solution for assigning labels to topic models by using multiple weak labelers. The method leverages generative transformers to learn accurate representations of the most important topic terms and candidate labels. This is achieved by fine-tuning pre-trained BART models on a large number of potential labels generated by state of the art non-neural models for topic labeling, enriched with different techniques. The proposed BART-TL model is able to generate valuable and novel labels in a weakly-supervised manner and can be improved by adding other weak labelers or distant supervision on similar tasks.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Topic Models

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Learning to Selectively Learn for Weakly-supervised Paraphrase Generation

2021-09-25 · EMNLP 2021 11 · Kaize Ding, Dingcheng Li, Alexander Hanbo Li, Xing Fan 외

Paraphrase generation is a longstanding NLP task that has diverse applications for downstream NLP tasks. However, the effectiveness of existing efforts predominantly relies on large amounts of golden labeled data. Though…

Language Modeling Language Modelling Meta-Learning Paraphrase Generation+1

A Bayesian Topic Model for Human-Evaluated Interpretability

2022-06-01 · LREC 2022 6 · Justin Wood, Corey Arnold, Wei Wang

One desiderata of topic modeling is to produce interpretable topics. Given a cluster of document-tokens comprising a topic, we can order the topic by counting each word. It is natural to think that each topic could easil…

Topic Models

Topics and Label Propagation: Best of Both Worlds for Weakly Supervised Text Classification

2017-12-04 · Sachin Pawar, Nitin Ramrakhiyani, Swapnil Hingmire, Girish K. Palshikar

We propose a Label Propagation based algorithm for weakly supervised text classification. We construct a graph where each document is represented by a node and edge weights represent similarities among the documents. Add…

General Classification text-classification Text Classification

BERT-Flow-VAE: A Weakly-supervised Model for Multi-Label Text Classification

2022-10-27 · COLING 2022 10 · Ziwen Liu, Josep Grau-Bove, Scott Allan Orr

Multi-label Text Classification (MLTC) is the task of categorizing documents into one or more topics. Considering the large volumes of data and varying domains of such tasks, fully supervised learning requires manually f…

Multi Label Text Classification Multi-Label Text Classification Natural Language Inference Sentence+3

A Weakly Supervised Approach for Classifying Stance in Twitter Replies

2021-03-12 · Sumeet Kumar, Ramon Villa Cox, Matthew Babcock, Kathleen M. Carley

Conversations on social media (SM) are increasingly being used to investigate social issues on the web, such as online harassment and rumor spread. For such issues, a common thread of research uses adversarial reactions,…