Video Question Answering for People with Visual Impairments Using an Egocentric 360-Degree Camera
This paper addresses the daily challenges encountered by visually impaired individuals, such as limited access to information, navigation difficulties, and barriers to social interaction. To alleviate these challenges, we introduce a novel visual question answering dataset. Our dataset offers two significant advancements over previous datasets: Firstly, it features videos captured using a 360-degree egocentric wearable camera, enabling observation of the entire surroundings, departing from the static image-centric nature of prior datasets. Secondly, unlike datasets centered on singular challenges, ours addresses multiple real-life obstacles simultaneously through an innovative visual-question answering framework. We validate our dataset using various state-of-the-art VideoQA methods and diverse metrics. Results indicate that while progress has been made, satisfactory performance levels for AI-powered assistive services remain elusive for visually impaired individuals. Additionally, our evaluation highlights the distinctive features of the proposed dataset, featuring ego-motion in videos captured via 360-degree cameras across varied scenarios.
Code (0)
등록된 구현이 없습니다.
Tasks
Question AnsweringVideo Question AnsweringVisual Question AnsweringSimilar Papers 제목 키워드 기반
Grounding Answers for Visual Questions Asked by Visually Impaired People
Visual question answering is the task of answering questions about images. We introduce the VizWiz-VQAGrounding dataset, the first dataset that visually grounds answers to visual questions asked by people with visual imp…
Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)Grounding Answers for Visual Questions Asked by Visually Impaired People
Visual question answering is the task of answering questions about images. We introduce the VizWiz-VQA-Grounding dataset, the first dataset that visually grounds answers to visual questions asked by people with visual im…
Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)TVQA+: Spatio-Temporal Grounding for Video Question Answering
We present the task of Spatio-Temporal Video Question Answering, which requires intelligent systems to simultaneously retrieve relevant moments and detect referenced visual concepts (people and objects) to answer natural…
Question AnsweringVideo Question AnsweringVIAssist: Adapting Multi-modal Large Language Models for Users with Visual Impairments
Individuals with visual impairments, encompassing both partial and total difficulties in visual perception, are referred to as visually impaired (VI) people. An estimated 2.2 billion individuals worldwide are affected by…
Improving the Accessibility of Dating Websites for Individuals with Visual Impairments
People now frequently meet and develop relationships through online dating. Yet, due to their limited accessibility, utilizing dating services can be difficult and irritating for people with visual impairments. The signi…