Applied Awareness: Test-Driven GUI Development using Computer Vision and Cryptography
Graphical user interface testing is significantly challenging, and automating it even more so. Test-driven development is impractical: it generally requires an initial implementation of the GUI to generate golden images or to construct interactive test scenarios, and subsequent maintenance is costly. While computer vision has been applied to several aspects of GUI testing, we demonstrate a novel and immediately applicable approach of interpreting GUI presentation in terms of backend communications, modeling "awareness" in the fashion employed by cryptographic proofs of security. This focus on backend communication circumvents deficiencies in typical testing methodologies that rely on platform-dependent UI affordances or accessibility features. Our interdisciplinary work is ready for off-the-shelf practice: we report self-contained, practical implementation with both online and offline validation, using simple designer specifications at the outset and specifically avoiding any requirements for a bootstrap implementation or golden images. In addition to practical implementation, ties to formal verification methods in cryptography are explored and explained, providing fertile perspectives on assurance in UI and interpretability in AI.
Code (0)
등록된 구현이 없습니다.
Tasks
test driven developmentMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Self-awareness in intelligent vehicles: Feature based dynamic Bayesian models for abnormality detection
The evolution of Intelligent Transportation Systems in recent times necessitates the development of self-awareness in agents. Before the intensive use of Machine Learning, the detection of abnormalities was manually prog…
Anomaly DetectionAutonomous VehiclesTime SeriesTime Series AnalysisTowards Situation Awareness and Attention Guidance in a Multiplayer Environment using Augmented Reality and Carcassonne
Augmented reality (AR) games are a rich environment for researching and testing computational systems that provide subtle user guidance and training. In particular computer systems that aim to augment a user's situation …
Energy Usage Reports: Environmental awareness as part of algorithmic accountability
The carbon footprint of algorithms must be measured and transparently reported so computer scientists can take an honest and active role in environmental sustainability. In this paper, we take analyses usually applied at…
Rethinking AI Literacy Education in Higher Education: Bridging Risk Perception and Responsible Adoption
As AI becomes increasingly embedded across societal domains, understanding how future AI practitioners, particularly technology students, perceive its risks is essential for responsible development and adoption. This stu…
Data-Driven AI Model Signal-Awareness Enhancement and Introspection
AI modeling for source code understanding tasks has been making significant progress, and is being adopted in production development pipelines. However, reliability concerns, especially whether the models are actually le…