Towards Explainable Test Case Prioritisation with Learning-to-Rank Models
Test case prioritisation (TCP) is a critical task in regression testing to ensure quality as software evolves. Machine learning has become a common way to achieve it. In particular, learning-to-rank (LTR) algorithms provide an effective method of ordering and prioritising test cases. However, their use poses a challenge in terms of explainability, both globally at the model level and locally for particular results. Here, we present and discuss scenarios that require different explanations and how the particularities of TCP (multiple builds over time, test case and test suite variations, etc.) could influence them. We include a preliminary experiment to analyse the similarity of explanations, showing that they do not only vary depending on test case-specific predictions, but also on the relative ranks.
Code (0)
등록된 구현이 없습니다.
Tasks
Learning-To-RankSimilar Papers 제목 키워드 기반
Neural Rankers for Effective Screening Prioritisation in Medical Systematic Review Literature Search
Medical systematic reviews typically require assessing all the documents retrieved by a search. The reason is two-fold: the task aims for ``total recall''; and documents retrieved using Boolean search are an unordered se…
Document RankingTowards Explainable Stakeholder-Aware Requirements Prioritisation in Aged-Care Digital Health
Requirements engineering for aged-care digital health must account for human aspects, because requirement priorities are shaped not only by technical functionality but also by stakeholders' health conditions, socioeconom…
GARL: Game-Theoretic Reinforcement Learning for Multi-Agent Strategic Prioritisation
LLM-based multi-agent systems are increasingly used for strategic decision-making tasks. In such settings, performance depends not only on individual model capabilities, but also on the policies by which agents interact …
Multi-agent Reinforcement LearningDense Retrieval with Continuous Explicit Feedback for Systematic Review Screening Prioritisation
The goal of screening prioritisation in systematic reviews is to identify relevant documents with high recall and rank them in early positions for review. This saves reviewing effort if paired with a stopping criterion, …
RetrievalTARA Taxonomy of Information Attributes for Test Case Prioritisation: Applicability, Machine Learning
Most software companies have extensive test suites and re-run parts of them continuously to ensure recent changes have no adverse effects. Since test suites are costly to execute, industry needs methods for test case pri…
AttributeBIG-bench Machine Learning