PerfAgent: Profiler-Guided Iterative Refinement for Repository-Level Code Optimization
Large language model (LLM) agents now perform well on correctness-oriented repository-level tasks, including SWE-Bench issue resolution and feature implementation in real codebases. However, they still struggle with repository-level code optimization, which requires preserving behavior while improving runtime performance. Passing tests is not enough in this setting; a patch must preserve behavior, implement code optimization, and approach expert speedups. Current agents often miss bottlenecks hidden behind abstraction layers and native extensions, stop after shallow speedups, or insufficiently test the code patches that thus may silently break edge cases. We present PerfAgent, a profiler-guided, verifier-in-the-loop workflow that gives an off-the-shelf coding agent the feedback needed to find real hotspots, improve beyond the first passing patch, and use profiler evidence rather than timing alone to decide what to optimize next. On two challenging optimization benchmarks, GSO and SWE-fficiency-Lite, PerfAgent more than doubles the rate of expert-matching patches over OpenHands with GPT-5.1, improving from 19.6% to 39.2% on GSO and from 26% to 74% on SWE-fficiency-Lite. It also surpasses an oracle best-of-five baseline at substantially lower cost, showing that the gains come from better feedback rather than additional test-time sampling.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
CellProfiler plugins -- an easy image analysis platform integration for containers and Python tools
CellProfiler is a widely used software for creating reproducible, reusable image analysis workflows without needing to code. In addition to the >90 modules that make up the main CellProfiler program, CellProfiler has a p…
IOHprofiler: A Benchmarking and Profiling Tool for Iterative Optimization Heuristics
IOHprofiler is a new tool for analyzing and comparing iterative optimization heuristics. Given as input algorithms and problems written in C or Python, it provides as output a statistical evaluation of the algorithms' pe…
BenchmarkingBenchmarking Discrete Optimization Heuristics with IOHprofiler
Automated benchmarking environments aim to support researchers in understanding how different algorithms perform on different types of optimization problems. Such comparisons provide insights into the strengths and weakn…
BenchmarkingBenchmarking Algorithms for Submodular Optimization Problems Using IOHProfiler
Submodular functions play a key role in the area of optimization as they allow to model many real-world problems that face diminishing returns. Evolutionary algorithms have been shown to obtain strong theoretical perform…
BenchmarkingEvolutionary AlgorithmsA Review of Repository Level Prompting for LLMs
As coding challenges become more complex, recent advancements in Large Language Models (LLMs) have led to notable successes, such as achieving a 94.6\% solve rate on the HumanEval benchmark. Concurrently, there is an inc…
Code CompletionCode GenerationHumanEvalRetrieval