Insight: Enhancing Mobile Accessibility for Blind and Visually Impaired Users with LLMs
This research paper addresses the limitations of current mobile accessibility services like TalkBack, which provide manual gesture-based sequential feedback to BVI users. Motivated by the promise of large language models (LLMs), this paper introduces Insight, an Android accessibility service that provides natural language interaction and real-time summarization of the screen. The paper performs a within-subject experimental study with users to compare Insight and TalkBack on usability factors. Results show Insight reduced mental effort and task time, and was preferred because of its dialogue interface, but users felt the need for interruption management. Results show LLM-based interfaces can significantly improve mobile accessibility, and describe the potential of hybrid solutions combining gesture and dialogue modalities towards more inclusive design.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
How Users Who are Blind or Low Vision Play Mobile Games: Perceptions, Challenges, and Strategies
As blind and low-vision (BLV) players engage more deeply with games, accessibility features have become essential. While some research has explored tools and strategies to enhance game accessibility, the specific experie…
Game DesignReal-Time Pill Identification for the Visually Impaired Using Deep Learning
The prevalence of mobile technology offers unique opportunities for addressing healthcare challenges, especially for individuals with visual impairments. This paper explores the development and implementation of a deep l…
Deep LearningManagementtext-to-speechText to SpeechACCSAMS: Automatic Conversion of Exam Documents to Accessible Learning Material for Blind and Visually Impaired
Exam documents are essential educational materials for exam preparation. However, they pose a significant academic barrier for blind and visually impaired students, as they are often created without accessibility conside…
Improve accessibility for Low Vision and Blind people using Machine Learning and Computer Vision
With the ever-growing expansion of mobile technology worldwide, there is an increasing need for accommodation for those who are disabled. This project explores how machine learning and computer vision could be utilized t…
Money Recognition for the Visually Impaired: A Case Study on Sri Lankan Banknotes
Currency note recognition is a critical accessibility need for blind individuals, as identifying banknotes accurately can impact their independence and security in financial transactions. Several traditional and technolo…