TalkTive: A Conversational Agent Using Backchannels to Engage Older Adults in Neurocognitive Disorders Screening

2022-02-16 · Zijian Ding, Jiawen Kang, Tinky Oi Ting HO, Ka Ho Wong, Helene H. Fung, Helen Meng, Xiaojuan Ma

Conversational agents (CAs) have the great potential in mitigating the clinicians' burden in screening for neurocognitive disorders among older adults. It is important, therefore, to develop CAs that can be engaging, to elicit conversational speech input from older adult participants for supporting assessment of cognitive abilities. As an initial step, this paper presents research in developing the backchanneling ability in CAs in the form of a verbal response to engage the speaker. We analyzed 246 conversations of cognitive assessments between older adults and human assessors, and derived the categories of reactive backchannels (e.g. "hmm") and proactive backchannels (e.g. "please keep going"). This is used in the development of TalkTive, a CA which can predict both timing and form of backchanneling during cognitive assessments. The study then invited 36 older adult participants to evaluate the backchanneling feature. Results show that proactive backchanneling is more appreciated by participants than reactive backchanneling.

📄 PDF Abstract BibTeX arXiv:2202.08216

Code (0)

등록된 구현이 없습니다.

Batch Recurrent Q-Learning for Backchannel Generation Towards Engaging Agents

2019-08-06 · Nusrah Hussain, Engin Erzin, T. Metin Sezgin, Yucel Yemez

The ability to generate appropriate verbal and non-verbal backchannels by an agent during human-robot interaction greatly enhances the interaction experience. Backchannels are particularly important in applications like …

Imitation Learning Q-Learning reinforcement-learning Reinforcement Learning+1

Exploring Semi-Supervised Learning for Predicting Listener Backchannels

2021-01-06 · Vidit Jain, Maitree Leekha, Rajiv Ratn Shah, Jainendra Shukla

Developing human-like conversational agents is a prime area in HCI research and subsumes many tasks. Predicting listener backchannels is one such actively-researched task. While many studies have used different approache…

Speech Driven Backchannel Generation using Deep Q-Network for Enhancing Engagement in Human-Robot Interaction

2019-08-05 · Nusrah Hussain, Engin Erzin, T. Metin Sezgin, Yucel Yemez

We present a novel method for training a social robot to generate backchannels during human-robot interaction. We address the problem within an off-policy reinforcement learning framework, and show how a robot may learn …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Yeah, Un, Oh: Continuous and Real-time Backchannel Prediction with Fine-tuning of Voice Activity Projection

2024-10-21 · Koji Inoue, Divesh Lala, Gabriel Skantze, Tatsuya Kawahara

In human conversations, short backchannel utterances such as "yeah" and "oh" play a crucial role in facilitating smooth and engaging dialogue. These backchannels signal attentiveness and understanding without interruptin…

Prediction Type prediction

Investigating the Representation of Backchannels and Fillers in Fine-tuned Language Models

2025-09-24 · Yu Wang, Leyi Lao, Langchu Huang, Gabriel Skantze 외 arxiv

Backchannels and fillers are important linguistic expressions in dialogue, but often treated as 'noise' to be bypassed in modern transformer-based language models (LMs). Here, we study how they are represented in LMs usi…