Test2Vec: An Execution Trace Embedding for Test Case Prioritization

2022-06-28 · Emad Jabbar, Soheila Zangeneh, Hadi Hemmati, Robert Feldt

Most automated software testing tasks can benefit from the abstract representation of test cases. Traditionally, this is done by encoding test cases based on their code coverage. Specification-level criteria can replace code coverage to better represent test cases' behavior, but they are often not cost-effective. In this paper, we hypothesize that execution traces of the test cases can be a good alternative to abstract their behavior for automated testing tasks. We propose a novel embedding approach, Test2Vec, that maps test execution traces to a latent space. We evaluate this representation in the test case prioritization (TP) task. Our default TP method is based on the similarity of the embedded vectors to historical failing test vectors. We also study an alternative based on the diversity of test vectors. Finally, we propose a method to decide which TP to choose, for a given test suite. The experiment is based on several real and seeded faults with over a million execution traces. Results show that our proposed TP improves best alternatives by 41.80% in terms of the median normalized rank of the first failing test case (FFR). It outperforms traditional code coverage-based approaches by 25.05% and 59.25% in terms of median APFD and median normalized FFR.

📄 PDF Abstract BibTeX arXiv:2206.15428

Code (0)

등록된 구현이 없습니다.

Tasks

software testing

Methods 이 논문이 사용한 방법론

Test 설명 없음

Learning to Encode and Classify Test Executions

2020-01-08 · Foivos Tsimpourlas, Ajitha Rajan, Miltiadis Allamanis

The challenge of automatically determining the correctness of test executions is referred to as the test oracle problem and is one of the key remaining issues for automated testing. The goal in this paper is to solve the…

General Classification Specificity

Localizing Bugs in Program Executions with Graphical Models

2009-12-01 · NeurIPS 2009 12 · Laura Dietz, Valentin Dallmeier, Andreas Zeller, Tobias Scheffer

We devise a graphical model that supports the process of debugging software by guiding developers to code that is likely to contain defects. The model is trained using execution traces of passing test runs; it reflects t…

Bayesian Inference

Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration

2018-11-09 · Helge Spieker, Arnaud Gotlieb, Dusica Marijan, Morten Mossige

Testing in Continuous Integration (CI) involves test case prioritization, selection, and execution at each cycle. Selecting the most promising test cases to detect bugs is hard if there are uncertainties on the impact of…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

SolidCoder: Bridging the Mental-Reality Gap in LLM Code Generation through Concrete Execution

2026-04-20 · Woojin Lee, Jin-Xia Huang arxiv

State-of-the-art code generation frameworks rely on mental simulation, where LLMs internally trace execution to verify correctness. We expose a fundamental limitation: the Mental-Reality Gap -- where models hallucinate e…

Code Generation

One for All and All for One: GNN-based Control-Flow Attestation for Embedded Devices

2024-03-12 · Marco Chilese, Richard Mitev, Meni Orenbach, Robert Thorburn 외

Control-Flow Attestation (CFA) is a security service that allows an entity (verifier) to verify the integrity of code execution on a remote computer system (prover). Existing CFA schemes suffer from impractical assumptio…

All