EmotionQueen: A Benchmark for Evaluating Empathy of Large Language Models
Emotional intelligence in large language models (LLMs) is of great importance in Natural Language Processing. However, the previous research mainly focus on basic sentiment analysis tasks, such as emotion recognition, which is not enough to evaluate LLMs' overall emotional intelligence. Therefore, this paper presents a novel framework named EmotionQueen for evaluating the emotional intelligence of LLMs. The framework includes four distinctive tasks: Key Event Recognition, Mixed Event Recognition, Implicit Emotional Recognition, and Intention Recognition. LLMs are requested to recognize important event or implicit emotions and generate empathetic response. We also design two metrics to evaluate LLMs' capabilities in recognition and response for emotion-related statements. Experiments yield significant conclusions about LLMs' capabilities and limitations in emotion intelligence.
Code (0)
등록된 구현이 없습니다.
Tasks
Emotional IntelligenceEmotion RecognitionIntent DetectionSentiment AnalysisMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
PERM: Psychology-grounded Empathetic Reward Modeling for Large Language Models
Large Language Models (LLMs) are increasingly deployed in human-centric applications, yet they often fail to provide substantive emotional support. While Reinforcement Learning (RL) has been utilized to enhance empathy o…
Reinforcement LearningEmotional IntelligenceAEQ-Bench: Measuring Empathy of Omni-Modal Large Models
While the automatic evaluation of omni-modal large models (OLMs) is essential, assessing empathy remains a significant challenge due to its inherent affectivity. To investigate this challenge, we introduce AEQ-Bench (Aud…
EmpathyAgent: Can Embodied Agents Conduct Empathetic Actions?
Empathy is fundamental to human interactions, yet it remains unclear whether embodied agents can provide human-like empathetic support. Existing works have studied agents' tasks solving and social interactions abilities,…
From Measurement to Expertise: Empathetic Expert Adapters for Context-Based Empathy in Conversational AI Agents
Empathy is a critical factor in fostering positive user experiences in conversational AI. While models can display empathy, it is often generic rather than tailored to specific tasks and contexts. In this work, we introd…
Can Machines Resonate with Humans? Evaluating the Emotional and Empathic Comprehension of LMs
Empathy plays a pivotal role in fostering prosocial behavior, often triggered by the sharing of personal experiences through narratives. However, modeling empathy using NLP approaches remains challenging due to its deep …
Contrastive Learning