Transformers to Learn Hierarchical Contexts in Multiparty Dialogue for Span-based Question Answering

2020-04-07 · ACL 2020 6 · Changmao Li, Jinho D. Choi

We introduce a novel approach to transformers that learns hierarchical representations in multiparty dialogue. First, three language modeling tasks are used to pre-train the transformers, token- and utterance-level language modeling and utterance order prediction, that learn both token and utterance embeddings for better understanding in dialogue contexts. Then, multi-task learning between the utterance prediction and the token span prediction is applied to fine-tune for span-based question answering (QA). Our approach is evaluated on the FriendsQA dataset and shows improvements of 3.8% and 1.4% over the two state-of-the-art transformer models, BERT and RoBERTa, respectively.

📄 PDF Abstract BibTeX arXiv:2004.03561

Code (1)

emorynlp/friendsqa pytorch

Tasks

Language ModelingLanguage Modelling Multi-Task Learning Prediction Question Answering

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Position-Wise Feed-Forward Layer 설명 없음

RoBERTa 설명 없음

Residual Connection 설명 없음

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Weight Decay 설명 없음

DADgraph: A Discourse-aware Dialogue Graph Neural Network for Multiparty Dialogue Machine Reading Comprehension

2021-04-26 · Jiaqi Li, Ming Liu, Zihao Zheng, Heng Zhang 외

Multiparty Dialogue Machine Reading Comprehension (MRC) differs from traditional MRC as models must handle the complex dialogue discourse structure, previously unconsidered in traditional MRC. To fully exploit such disco…

Graph Neural Network Machine Reading Comprehension Question Answering Reading Comprehension

Discourse Structure and Dialogue Acts in Multiparty Dialogue: the STAC Corpus

2016-05-01 · LREC 2016 5 · Nicholas Asher, Julie Hunter, Mathieu Morey, Benamara Farah 외

This paper describes the STAC resource, a corpus of multi-party chats annotated for discourse structure in the style of SDRT (Asher and Lascarides, 2003; Lascarides and Asher, 2009). The main goal of the STAC project is …

Multilingual Coreference Resolution in Multiparty Dialogue

2022-08-02 · Boyuan Zheng, Patrick Xia, Mahsa Yarmohammadi, Benjamin Van Durme

Existing multiparty dialogue datasets for entity coreference resolution are nascent, and many challenges are still unaddressed. We create a large-scale dataset, Multilingual Multiparty Coref (MMC), for this task based on…

coreference-resolution Coreference Resolution Data Augmentation

FriendsQA: Open-Domain Question Answering on TV Show Transcripts

2019-09-01 · WS 2019 9 · Zhengzhe Yang, Jinho D. Choi

This paper presents FriendsQA, a challenging question answering dataset that contains 1,222 dialogues and 10,610 open-domain questions, to tackle machine comprehension on everyday conversations. Each dialogue, involving …

Open-Domain Question Answering Question Answering Reading Comprehension

The Tutorbot Corpus --- A Corpus for Studying Tutoring Behaviour in Multiparty Face-to-Face Spoken Dialogue

2014-05-01 · LREC 2014 5 · Maria Koutsombogera, Samer Al Moubayed, Bajibabu Bollepalli, Ahmed Hussen Abdelaziz 외

This paper describes a novel experimental setup exploiting state-of-the-art capture equipment to collect a multimodally rich game-solving collaborative multiparty dialogue corpus. The corpus is targeted and designed towa…

Spoken Dialogue Systems