Impact of Decoding Methods on Human Alignment of Conversational LLMs
To be included into chatbot systems, Large language models (LLMs) must be aligned with human conversational conventions. However, being trained mainly on web-scraped data gives existing LLMs a voice closer to informational text than actual human speech. In this paper, we examine the effect of decoding methods on the alignment between LLM-generated and human conversations, including Beam Search, Top K Sampling, and Nucleus Sampling. We present new measures of alignment in substance, style, and psychometric orientation, and experiment with two conversation datasets. Our results provide subtle insights: better alignment is attributed to fewer beams in Beam Search and lower values of P in Nucleus Sampling. We also find that task-oriented and open-ended datasets perform differently in terms of alignment, indicating the significance of taking into account the context of the interaction.
Code (0)
등록된 구현이 없습니다.
Tasks
ChatbotSimilar Papers 제목 키워드 기반
Decoding Workload and Agreement From EEG During Spoken Dialogue With Conversational AI
Passive brain-computer interfaces offer a potential source of implicit feedback for alignment of large language models, but most mental state decoding has been done in controlled tasks. This paper investigates whether es…
Sentence CompletionFine-grained Conversational Decoding via Isotropic and Proximal Search
General-purpose text decoding approaches are usually adopted for dialogue response generation. Although the quality of the generated responses can be improved with dialogue-specific encoding methods, conversational decod…
InformativenessResponse GenerationWell Begun is Half Done: Low-resource Preference Alignment by Weak-to-Strong Decoding
Large Language Models (LLMs) require alignment with human preferences to avoid generating offensive, false, or meaningless content. Recently, low-resource methods for LLM alignment have been popular, while still facing c…
Steering Output Style and Topic in Neural Response Generation
We propose simple and flexible training and decoding methods for influencing output style and topic in neural encoder-decoder based language generation. This capability is desirable in a variety of applications, includin…
DecoderResponse GenerationText GenerationPlug-and-Play Conversational Models
There has been considerable progress made towards conversational models that generate coherent and fluent responses; however, this often involves training large language models on large dialogue datasets, such as Reddit.…
AttributeLanguage ModelingLanguage ModellingLarge Language Model+1