FairEval: Evaluating Fairness in LLM-Based Recommendations with Personality Awareness
Recent advances in Large Language Models (LLMs) have enabled their application to recommender systems (RecLLMs), yet concerns remain regarding fairness across demographic and psychological user dimensions. We introduce FairEval, a novel evaluation framework to systematically assess fairness in LLM-based recommendations. FairEval integrates personality traits with eight sensitive demographic attributes,including gender, race, and age, enabling a comprehensive assessment of user-level bias. We evaluate models, including ChatGPT 4o and Gemini 1.5 Flash, on music and movie recommendations. FairEval's fairness metric, PAFS, achieves scores up to 0.9969 for ChatGPT 4o and 0.9997 for Gemini 1.5 Flash, with disparities reaching 34.79 percent. These results highlight the importance of robustness in prompt sensitivity and support more inclusive recommendation systems.
Code (0)
등록된 구현이 없습니다.
Tasks
FairnessRecommendation SystemsSensitivitySimilar Papers 제목 키워드 기반
Uncertainty and Fairness Awareness in LLM-Based Recommendation Systems
Large language models (LLMs) enable powerful zero-shot recommendations by leveraging broad contextual knowledge, yet predictive uncertainty and embedded biases threaten reliability and fairness. This paper studies how un…
Recommendation SystemsPerFairX: Is There a Balance Between Fairness and Personality in Large Language Model Recommendations?
The integration of Large Language Models (LLMs) into recommender systems has enabled zero-shot, personality-based personalization through prompt-based interactions, offering a new paradigm for user-centric recommendation…
Continual LearningUnmasking Gender Bias in Recommendation Systems and Enhancing Category-Aware Fairness
Recommendation systems are now an integral part of our daily lives. We rely on them for tasks such as discovering new movies, finding friends on social media, and connecting job seekers with relevant opportunities. Given…
FairnessRecommendation SystemsMeasuring Fairness Under Unawareness of Sensitive Attributes: A Quantification-Based Approach
Algorithms and models are increasingly deployed to inform decisions about people, inevitably affecting their lives. As a consequence, those in charge of developing these models must carefully evaluate their impact on dif…
FairnessThe Better Angels of Machine Personality: How Personality Relates to LLM Safety
Personality psychologists have analyzed the relationship between personality and safety behaviors in human society. Although Large Language Models (LLMs) demonstrate personality traits, the relationship between personali…
FairnessSafety Alignment