Saliency Prediction for Mobile User Interfaces
We introduce models for saliency prediction for mobile user interfaces. A mobile interface may include elements like buttons, text, etc. in addition to natural images which enable performing a variety of tasks. Saliency in natural images is a well studied area. However, given the difference in what constitutes a mobile interface, and the usage context of these devices, we postulate that saliency prediction for mobile interface images requires a fresh approach. Mobile interface design involves operating on elements, the building blocks of the interface. We first collected eye-gaze data from mobile devices for free viewing task. Using this data, we develop a novel autoencoder based multi-scale deep learning model that provides saliency prediction at the mobile interface element level. Compared to saliency prediction approaches developed for natural images, we show that our approach performs significantly better on a range of established metrics.
Code (0)
등록된 구현이 없습니다.
Tasks
PredictionSaliency PredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Understanding Visual Saliency in Mobile User Interfaces
For graphical user interface (UI) design, it is important to understand what attracts visual attention. While previous work on saliency has focused on desktop and web-based UIs, mobile app UIs differ from these in severa…
A Deep Learning Framework for Visual Attention Prediction and Analysis of News Interfaces
News outlets' competition for attention in news interfaces has highlighted the need for demographically-aware saliency prediction models. Despite recent advancements in saliency detection applied to user interfaces (UI),…
Saliency DetectionSaliency PredictionSaliency RankingPredicting Visual Importance Across Graphic Design Types
This paper introduces a Unified Model of Saliency and Importance (UMSI), which learns to predict visual importance in input graphic designs, and saliency in natural images, along with a new dataset and applications. Prev…
Predicting the usability of mobile applications using AI tools: the rise of large user interface models, opportunities, and challenges
This article proposes the so-called large user interface models (LUIMs) to enable the generation of user interfaces and prediction of usability using artificial intelligence in the context of mobile applications.
Predicting and Explaining Mobile UI Tappability with Vision Modeling and Saliency Analysis
We use a deep learning based approach to predict whether a selected element in a mobile UI screenshot will be perceived by users as tappable, based on pixels only instead of view hierarchies required by previous work. To…