Are Generative Language Models Multicultural? A Study on Hausa Culture and Emotions using ChatGPT
Large Language Models (LLMs), such as ChatGPT, are widely used to generate content for various purposes and audiences. However, these models may not reflect the cultural and emotional diversity of their users, especially for low-resource languages. In this paper, we investigate how ChatGPT represents Hausa's culture and emotions. We compare responses generated by ChatGPT with those provided by native Hausa speakers on 37 culturally relevant questions. We conducted experiments using emotion analysis and applied two similarity metrics to measure the alignment between human and ChatGPT responses. We also collected human participants ratings and feedback on ChatGPT responses. Our results show that ChatGPT has some level of similarity to human responses, but also exhibits some gaps and biases in its knowledge and awareness of the Hausa culture and emotions. We discuss the implications and limitations of our methodology and analysis and suggest ways to improve the performance and evaluation of LLMs for low-resource languages.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityEmotion RecognitionSimilar Papers 제목 키워드 기반
Multilingual Language Models are not Multicultural: A Case Study in Emotion
Emotions are experienced and expressed differently across the world. In order to use Large Language Models (LMs) for multilingual tasks that require emotional sensitivity, LMs must reflect this cultural variation in emot…
SensitivityFrom Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Models
Despite recent advancements in vision-language models, their performance remains suboptimal on images from non-western cultures due to underrepresentation in training datasets. Various benchmarks have been proposed to te…
DiversityRetrievalVisual GroundingBeyond Alignment: Value Diversity as a Collective Property in Multicultural Agent Systems
Multicultural multi-agent systems are increasingly deployed in globally diverse settings, where different agents are grounded in different cultural backgrounds. Existing cultural evaluation focuses on value alignment: ho…
A Deep Convolutional Neural Network-based Model for Aspect and Polarity Classification in Hausa Movie Reviews
Aspect-based Sentiment Analysis (ABSA) is crucial for understanding sentiment nuances in text, especially across diverse languages and cultures. This paper introduces a novel Deep Convolutional Neural Network (CNN)-based…
Aspect-Based Sentiment AnalysisAspect-Based Sentiment Analysis (ABSA)Sentiment AnalysisTerm ExtractionThe Many Faces of Anger: A Multicultural Video Dataset of Negative Emotions in the Wild (MFA-Wild)
The portrayal of negative emotions such as anger can vary widely between cultures and contexts, depending on the acceptability of expressing full-blown emotions rather than suppression to maintain harmony. The majority o…
Cultural Vocal Bursts Intensity PredictionEmotion ClassificationEmotion Recognition