Researchers have made significant progress in automating the software development process in the past decades. Automated techniques for issue summarisation, bug reproduction, fault localisation, and program repair have been built to ease the workload of developers. Recent progress in Large Language Models (LLMs) has significantly impacted the development process, where developers can use LLM-based programming assistants to achieve automated coding. Nevertheless software engineering involves the process of program improvement apart from coding, specifically to enable software maintenance (e.g. program repair to fix bugs) and software evolution (e.g. feature additions). In this paper, we propose an automated approach for solving Github issues to autonomously achieve program improvement. In our approach called AutoCodeRover, LLMs are combined with sophisticated code search capabilities, ultimately leading to a program modification or patch. In contrast to recent LLM agent approaches from AI researchers and practitioners, our outlook is more software engineering oriented. We work on a program representation (abstract syntax tree) as opposed to viewing a software project as a mere collection of files. Our code search exploits the program structure in the form of classes/methods to enhance LLM’s understanding of the issue’s root cause, and effectively retrieves a context via iterative search. The use of spectrum based fault localisation using tests, further sharpens the context, as long as a test-suite is available. We consider the recently proposed SWE-bench-lite which consists of 300 real-life Github issues involving bug fixing and feature additions. Experiments on SWE-bench-lite show increased efficacy in solving Github issues (22-23% on SWE-bench-lite), as compared to recent efforts from the AI community. Interestingly, our approach resolved 67 GitHub issues in less than ten minutes each, whereas developers spent more than 2.77 days on average. On the full SWE-bench consisting of 2294 GitHub issues, AutoCodeRover solved 15-16% of issues, which is higher than the efficacy of the recently reported AI software engineer Devin from Cognition Labs. We posit that our workflow enables autonomous software engineering, where, in future, auto-generated code from LLMs can be autonomously improved.