AutoCodeRover: Autonomous Program Improvement
This program is tentative and subject to change.
Researchers have made significant progress in automating the software development process in the past decades. Automated techniques for issue summarisation, bug reproduction, fault localisation, and program repair have been built to ease the workload of developers. Recent progress in Large Language Models (LLMs) has significantly impacted the development process, where developers can use LLM-based programming assistants to achieve automated coding. Nevertheless software engineering involves the process of program improvement apart from coding, specifically to enable software maintenance (e.g. program repair to fix bugs) and software evolution (e.g. feature additions). In this paper, we propose an automated approach for solving Github issues to autonomously achieve program improvement. In our approach called AutoCodeRover, LLMs are combined with sophisticated code search capabilities, ultimately leading to a program modification or patch. In contrast to recent LLM agent approaches from AI researchers and practitioners, our outlook is more software engineering oriented. We work on a program representation (abstract syntax tree) as opposed to viewing a software project as a mere collection of files. Our code search exploits the program structure in the form of classes/methods to enhance LLM’s understanding of the issue’s root cause, and effectively retrieves a context via iterative search. The use of spectrum based fault localisation using tests, further sharpens the context, as long as a test-suite is available. We consider the recently proposed SWE-bench-lite which consists of 300 real-life Github issues involving bug fixing and feature additions. Experiments on SWE-bench-lite show increased efficacy in solving Github issues (22-23% on SWE-bench-lite), as compared to recent efforts from the AI community. Interestingly, our approach resolved 67 GitHub issues in less than ten minutes each, whereas developers spent more than 2.77 days on average. On the full SWE-bench consisting of 2294 GitHub issues, AutoCodeRover solved 15-16% of issues, which is higher than the efficacy of the recently reported AI software engineer Devin from Cognition Labs. We posit that our workflow enables autonomous software engineering, where, in future, auto-generated code from LLMs can be autonomously improved.
This program is tentative and subject to change.
Thu 19 SepDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
10:30 - 11:50 | |||
10:30 20mTalk | Automating Zero-Shot Patch Porting for Hard Forks Technical Papers Shengyi Pan Zhejiang University, You Wang Zhejiang University, Zhongxin Liu Zhejiang University, Xing Hu Zhejiang University, Xin Xia Huawei Technologies, Shanping Li Zhejiang University DOI Pre-print | ||
10:50 20mTalk | Benchmarking Automated Program Repair: An Extensive Study on Both Real-World and Artificial Bugs Technical Papers Yicheng Ouyang University of Illinois at Urbana-Champaign, Jun Yang UIUC, Lingming Zhang University of Illinois Urbana-Champaign DOI | ||
11:10 20mTalk | Neurosymbolic Repair of Test Flakiness Technical Papers Yang Chen University of Illinois at Urbana-Champaign, Reyhaneh Jabbarvand University of Illinois at Urbana-Champaign | ||
11:30 20mTalk | AutoCodeRover: Autonomous Program Improvement Technical Papers Yuntong Zhang National University of Singapore, Haifeng Ruan National University of Singapore, Zhiyu Fan National University of Singapore, Abhik Roychoudhury National University of Singapore |