paper-with-me

Papers

MR-Coupler: Automated Metamorphic Test Generation via Functional Coupling Analysis

2026-04-11 · Congying Xu, Hengcheng Zhu, Songqiang Chen, Jiarong Wu, Valerio Terragni, Shing-Chi Cheung arxiv

Metamorphic testing (MT) is a widely recognized technique for alleviating the oracle problem in software testing. However, its adoption is hindered by the difficulty of constructing effective metamorphic relations (MRs), which often require domain-specific or hard-to-obtain knowledge. In this work, we propose a novel approach that leverages the functional coupling between methods, which is readily available in source code, to automatically construct MRs and generate metamorphic test cases (MTCs). Our technique, MR-Coupler, identifies functionally coupled method pairs, employs large language models to generate candidate MTCs, and validates them through test amplification and mutation analysis. In particular, we leverage three functional coupling features to avoid expensive enumeration of possible method pairs, and a novel validation mechanism to reduce false alarms. Our evaluation of MR-Coupler on 100 human-written MTCs and 50 real-world bugs shows that it generates valid MTCs for over 90% of tasks, improves valid MTC generation by 64.90%, and reduces false alarms by 36.56% compared to baselines. Furthermore, the MTCs generated by MR-Coupler detect 44% of the real bugs. Our results highlight the effectiveness of leveraging functional coupling for automated MR construction and the potential of MR-Coupler to facilitate the adoption of MT in practice. We also released the tool and experimental data to support future research.

📄 PDF Abstract BibTeX arXiv:2604.10126

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Automated identification of metamorphic test scenarios for an ocean-modeling application

2020-09-03 · Dilip J. Hiremath, Martin Claus, Wilhelm Hasselbring, Willi Rath

Metamorphic testing seeks to validate software in the absence of test oracles. Our application domain is ocean modeling, where test oracles often do not exist, but where symmetries of the simulated physical systems are k…

BIG-bench Machine LearningRelation

Multi-Agent Specification-based Metamorphic Testing of FMU-Based Simulations

2026-05-24 · Ashir Kulshreshtha, Abdullah Mughees, Gaadha Sudheerbabu, Tanwir Ahmad 외 arxiv

In many industrial domains, the Functional Mock-up Interface (FMI) is used to exchange simulation models as Functional Mock-up Units (FMUs) across different partners using various modelling tools. This opens up the possi…

MORTAR: Multi-turn Metamorphic Testing for LLM-based Dialogue Systems

2024-12-20 · Guoxiang Guo, Aldeida Aleti, Neelofar Neelofar, Chakkrit Tantithamthavorn 외

With the widespread application of LLM-based dialogue systems in daily life, quality assurance has become more important than ever. Recent research has successfully introduced methods to identify unexpected behaviour in …

From Untestable to Testable: Metamorphic Testing in the Age of LLMs

2026-03-25 · Valerio Terragni arxiv

This article discusses the challenges of testing software systems with increasingly integrated AI and LLM functionalities. LLMs are powerful but unreliable, and labeled ground truth for testing rarely scales. Metamorphic…

LLMORPH: Automated Metamorphic Testing of Large Language Models

2026-03-24 · Steven Cho, Stefano Ruberto, Valerio Terragni arxiv

Automated testing is essential for evaluating and improving the reliability of Large Language Models (LLMs), yet the lack of automated oracles for verifying output correctness remains a key challenge. We present LLMORPH,…