GUITester: Enabling GUI Agents for Exploratory Defect Discovery

2026-01-08 · Yifei Gao, Jiang Wu, Xiaoyi Chen, Yifan Yang, Zhe Cui, Tianyi Ma, Jiaming Zhang, Jitao Sang arxiv

Exploratory GUI testing is essential for software quality but suffers from high manual costs. While Multi-modal Large Language Model (MLLM) agents excel in navigation, they fail to autonomously discover defects due to two core challenges: \textit{Goal-Oriented Masking}, where agents prioritize task completion over reporting anomalies, and \textit{Execution-Bias Attribution}, where system defects are misidentified as agent errors. To address these, we first introduce \textbf{GUITestBench}, the first interactive benchmark for this task, featuring 143 tasks across 26 defects. We then propose \textbf{GUITester}, a multi-agent framework that decouples navigation from verification via two modules: (i) a \textit{Planning-Execution Module (PEM)} that proactively probes for defects via embedded testing intents, and (ii) a \textit{Hierarchical Reflection Module (HRM)} that resolves attribution ambiguity through interaction history analysis. GUITester achieves an F1-score of 48.90\% (Pass@3) on GUITestBench, outperforming state-of-the-art baselines (33.35\%). Our work demonstrates the feasibility of autonomous exploratory testing and provides a robust foundation for future GUI quality assurance~\footnote{Our code is now available in~\href{https://github.com/ADaM-BJTU/GUITestBench}{https://github.com/ADaM-BJTU/GUITestBench}}.

📄 PDF Abstract BibTeX arXiv:2601.04500

Code (0)

등록된 구현이 없습니다.

GUITestScape: Towards Open-set Evaluation on Exploratory GUI Testing

2026-05-28 · Xiaoyi Chen, Yifei Gao, Yang Xu, Xingxing Song 외 arxiv

Exploratory GUI testing is a particularly demanding setting for MLLM agents: without predefined test scripts, an agent must autonomously navigate an application and discover defects through its own interaction. However, …

Enhancing Agentic Autonomous Scientific Discovery with Vision-Language Model Capabilities

2025-11-18 · Kahaan Gandhi, Boris Bolliet, Inigo Zubeldia arxiv

We show that multi-agent systems guided by vision-language models (VLMs) improve end-to-end autonomous scientific discovery. By treating plots as verifiable checkpoints, a VLM-as-a-judge evaluates figures against dynamic…

ComSD: Balancing Behavioral Quality and Diversity in Unsupervised Skill Discovery

2023-09-29 · Xin Liu, Yaran Chen, Dongbin Zhao

This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. Unsupervised skill discovery seeks to acquire differen…

Contrastive Learning Diversity Reinforcement Learning (RL)Unsupervised Reinforcement Learning

InsightEval: An Expert-Curated Benchmark for Assessing Insight Discovery in LLM-Driven Data Agents

2025-11-28 · Zhenghao Zhu, Yuanfeng Song, Xin Chen, Chengzhong Liu 외 arxiv

Data analysis has become an indispensable part of scientific research. To discover the latent knowledge and insights hidden within massive datasets, we need to perform deep exploratory analysis to realize their full valu…

Hypothesis Hunting with Evolving Networks of Autonomous Scientific Agents

2025-10-08 · Tennison Liu, Silas Ruhrberg Estévez, David L. Bentley, Mihaela van der Schaar arxiv

Large-scale scientific datasets -- spanning health biobanks, cell atlases, Earth reanalyses, and more -- create opportunities for exploratory discovery unconstrained by specific research questions. We term this process h…