Toward a Human-Centered Evaluation Framework for Trustworthy LLM-Powered GUI Agents
The rise of Large Language Models (LLMs) has revolutionized Graphical User Interface (GUI) automation through LLM-powered GUI agents, yet their ability to process sensitive data with limited human oversight raises significant privacy and security risks. This position paper identifies three key risks of GUI agents and examines how they differ from traditional GUI automation and general autonomous agents. Despite these risks, existing evaluations focus primarily on performance, leaving privacy and security assessments largely unexplored. We review current evaluation metrics for both GUI and general LLM agents and outline five key challenges in integrating human evaluators for GUI agent assessments. To address these gaps, we advocate for a human-centered evaluation framework that incorporates risk assessments, enhances user awareness through in-context consent, and embeds privacy and security considerations into GUI agent design and evaluation.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
HELM: A Human-Centered Evaluation Framework for LLM-Powered Recommender Systems
The integration of Large Language Models (LLMs) into recommendation systems has introduced unprecedented capabilities for natural language understanding, explanation generation, and conversational interactions. However, …
Natural Language UnderstandingCollaborative FilteringExplanation GenerationRecommendation SystemsToward Human-Centered Readability Evaluation
Text simplification is essential for making public health information accessible to diverse populations, including those with limited health literacy. However, commonly used evaluation metrics in Natural Language Process…
Text SimplificationThe Challenges and Opportunities of Human-Centered AI for Trustworthy Robots and Autonomous Systems
The trustworthiness of Robots and Autonomous Systems (RAS) has gained a prominent position on many research agendas towards fully autonomous systems. This research systematically explores, for the first time, the key fac…
EthicsHuman-Centered Artificial Intelligence: Reliable, Safe & Trustworthy
Well-designed technologies that offer high levels of human control and high levels of computer automation can increase human performance, leading to wider adoption. The Human-Centered Artificial Intelligence (HCAI) frame…
Explanation User Interfaces: A Systematic Literature Review
Artificial Intelligence (AI) is one of the major technological advancements of this century, bearing incredible potential for users through AI-powered applications and tools in numerous domains. Being often black-box (i.…
Decision MakingExplainable artificial intelligenceExplainable Artificial Intelligence (XAI)Systematic Literature Review