Exploring the Potential of Conversational Test Suite Based Program Repair on SWE-bench
Automatic program repair at project level may open yet to be seen opportunities in various fields of human activity. Since the SWE-Bench challenge was presented, we have seen numerous of solutions. Patch generation is a part of program repair, and test suite-based conversational patch generation has proven its effectiveness. However, the potential of conversational patch generation has not yet specifically estimated on SWE-Bench. This study reports experimental results aimed at evaluating the individual effectiveness of conversational patch generation on problems from SWE-Bench. The experiments show that a simple conversational pipeline based on LLaMA 3.1 70B can generate valid patches in 47\% of cases, which is comparable to the state-of-the-art in program repair on SWE-Bench.
Code (0)
등록된 구현이 없습니다.
Tasks
Program RepairvalidMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MutaBot: A Mutation Testing Approach for Chatbots
Mutation testing is a technique aimed at assessing the effectiveness of test suites by seeding artificial faults into programs. Although available for many platforms and languages, no mutation testing tool is currently a…
Evaluating LLMs for Hardware Design and Test
Large Language Models (LLMs) have demonstrated capabilities for producing code in Hardware Description Languages (HDLs). However, most of the focus remains on their abilities to write functional code, not test code. The …
Automating Autograding: Large Language Models as Test Suite Generators for Introductory Programming
Automatically graded programming assignments provide instant feedback to students and significantly reduce manual grading time for instructors. However, creating comprehensive suites of test cases for programming problem…
Development of Conversational AI for Sleep Coaching Programme
Almost 30{\%} of the adult population in the world is experiencing or has experience insomnia. Cognitive Behaviour Therapy for insomnia (CBT-I) is one of the most effective treatment, but it has limitations on accessibil…
Exploring Plausible Patches Using Source Code Embeddings in JavaScript
Despite the immense popularity of the Automated Program Repair (APR) field, the question of patch validation is still open. Most of the present-day approaches follow the so-called Generate-and-Validate approach, where fi…
Program Repair