"Even explanations will not help in trusting [this] fundamentally biased system": A Predictive Policing Case-Study
In today's society, where Artificial Intelligence (AI) has gained a vital role, concerns regarding user's trust have garnered significant attention. The use of AI systems in high-risk domains have often led users to either under-trust it, potentially causing inadequate reliance or over-trust it, resulting in over-compliance. Therefore, users must maintain an appropriate level of trust. Past research has indicated that explanations provided by AI systems can enhance user understanding of when to trust or not trust the system. However, the utility of presentation of different explanations forms still remains to be explored especially in high-risk domains. Therefore, this study explores the impact of different explanation types (text, visual, and hybrid) and user expertise (retired police officers and lay users) on establishing appropriate trust in AI-based predictive policing. While we observed that the hybrid form of explanations increased the subjective trust in AI for expert users, it did not led to better decision-making. Furthermore, no form of explanations helped build appropriate trust. The findings of our study emphasize the importance of re-evaluating the use of explanations to build [appropriate] trust in AI based systems especially when the system's use is questionable. Finally, we synthesize potential challenges and policy recommendations based on our results to design for appropriate trust in high-risk based AI-based systems.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Impact of Explanation on Trust of a Novel Mobile Robot
One challenge with introducing robots into novel environments is misalignment between supervisor expectations and reality, which can greatly affect a user's trust and continued use of the robot. We performed an experimen…
Feature Attributions and Counterfactual Explanations Can Be Manipulated
As machine learning models are increasingly used in critical decision-making settings (e.g., healthcare, finance), there has been a growing emphasis on developing methods to explain model predictions. Such \textit{explan…
BIG-bench Machine LearningcounterfactualDecision MakingGeneralizing Logic-based Explanations for Machine Learning Classifiers via Optimization
Machine learning models support decision-making, yet the reasons behind their predictions are opaque. Clear and reliable explanations help users make informed decisions and avoid blindly trusting model outputs. However, …
Trusting the Explainers: Teacher Validation of Explainable Artificial Intelligence for Course Design
Deep learning models for learning analytics have become increasingly popular over the last few years; however, these approaches are still not widely adopted in real-world settings, likely due to a lack of trust and trans…
Explainable artificial intelligenceThe Effect of Biased Communications On Both Trusting and Suspicious Voters
In recent studies of political decision-making, apparently anomalous behavior has been observed on the part of voters, in which negative information about a candidate strengthens, rather than weakens, a prior positive op…
Decision Making