SMRT Chatbots: Improving Non-Task-Oriented Dialog with Simulated Multiple Reference Training

2020-11-01 · Findings of the Association for Computational Linguistics 2020 · Huda Khayrallah, João Sedoc

Non-task-oriented dialog models suffer from poor quality and non-diverse responses. To overcome limited conversational data, we apply Simulated Multiple Reference Training (SMRT; Khayrallah et al., 2020), and use a paraphraser to simulate multiple responses per training prompt. We find SMRT improves over a strong Transformer baseline as measured by human and automatic quality scores and lexical diversity. We also find SMRT is comparable to pretraining in human evaluation quality, and outperforms pretraining on automatic quality and lexical diversity, without requiring related-domain dialog data.

📄 PDF Abstract BibTeX arXiv:2011.00547

Code (0)

등록된 구현이 없습니다.

Tasks

Diversity

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Position-Wise Feed-Forward Layer 설명 없음

Adam 설명 없음

Residual Connection 설명 없음

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Multi-Head Attention 설명 없음

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Simulated Multiple Reference Training Improves Low-Resource Machine Translation

2020-04-30 · EMNLP 2020 11 · Huda Khayrallah, Brian Thompson, Matt Post, Philipp Koehn

Many valid translations exist for a given sentence, yet machine translation (MT) is trained with a single reference translation, exacerbating data sparsity in low-resource settings. We introduce Simulated Multiple Refere…

Machine Translation Sentence Translation valid

Task-Oriented Dialogue with In-Context Learning

2024-02-19 · Tom Bocklisch, Thomas Werkmeister, Daksh Varshneya, Alan Nichol

We describe a system for building task-oriented dialogue systems combining the in-context learning abilities of large language models (LLMs) with the deterministic execution of business logic. LLMs are used to translate …

In-Context Learning Navigate Task-Oriented Dialogue Systems

Self-Attentional Models Application in Task-Oriented Dialogue Generation Systems

2019-09-11 · RANLP 2019 9 · Mansour Saffar Mehrjardi, Amine Trabelsi, Osmar R. Zaiane

Self-attentional models are a new paradigm for sequence modelling tasks which differ from common sequence modelling methods, such as recurrence-based and convolution-based sequence learning, in the way that their archite…

Dialogue Generation Machine Translation Translation

GoChat: Goal-oriented Chatbots with Hierarchical Reinforcement Learning

2020-05-24 · Jianfeng Liu, Feiyang Pan, Ling Luo

A chatbot that converses like a human should be goal-oriented (i.e., be purposeful in conversation), which is beyond language generation. However, existing dialogue systems often heavily rely on cumbersome hand-crafted r…

Chatbot Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning+3

Beyond Ontology in Dialogue State Tracking for Goal-Oriented Chatbot

2024-10-30 · Sejin Lee, Dongha Kim, Min Song

Goal-oriented chatbots are essential for automating user tasks, such as booking flights or making restaurant reservations. A key component of these systems is Dialogue State Tracking (DST), which interprets user intent a…

Chatbot Dialogue State Tracking Hallucination Language Modeling+2