AI Coders Are among Us: Rethinking Programming Language Grammar towards Efficient Code GenerationACM SIGSOFT Distinguished Paper Award
Artificial Intelligence (AI) models have emerged as another important audience for programming languages alongside humans and machines, as we enter the era of large language models (LLMs). LLMs can now perform well in coding competitions and even write programs like developers to solve various tasks, including mathematical problems. However, the grammar and layout of current programs are designed to cater the needs of human developers – with many grammar tokens and formatting tokens being used to make the code easier for humans to read. While this is helpful, such a design adds unnecessary computational work for LLMs, as each token they either use or produce consumes computational resources.
To improve inference efficiency and reduce computational costs, we propose the concept of AI-oriented grammar.This aims to represent code in a way that better suits the working mechanism of AI models. Code written with AI-oriented grammar discards formats and uses a minimum number of tokens to convey code semantics effectively. To demonstrate the feasibility of this concept, we explore and implement the first AI-oriented grammar for Python, named Simple Python (SimPy). SimPy is crafted by revising the original Python grammar through a series of heuristic rules. Programs written in SimPy maintain identical Abstract Syntax Tree (AST) structures to those in standard Python. This allows for not only execution via a modified AST parser, but also seamless transformation between programs written in Python and SimPy, enabling human developers and LLMs to use Python and SimPy, respectively, when they need to collaborate. We also look into methods to help existing LLMs understand and use SimPy effectively. In the experiments, compared with Python, SimPy enables a reduction in token usage by 13.5% and 10.4% for CodeLlama and GPT-4, respectively, when completing the same set of code-related tasks. Additionally, these models can maintain or even improve their performance when using SimPy instead of Python for these tasks. With these promising results, we call for further contributions to the development of AI-oriented program grammar within our community.
Wed 18 SepDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
10:30 - 11:50 | |||
10:30 20mTalk | AI Coders Are among Us: Rethinking Programming Language Grammar towards Efficient Code GenerationACM SIGSOFT Distinguished Paper Award Technical Papers Zhensu Sun Singapore Management University, Xiaoning Du Monash University, Zhou Yang Singapore Management University, Li Li Beihang University, David Lo Singapore Management University DOI Pre-print | ||
10:50 20mTalk | When to Stop? Towards Efficient Code Generation in LLMs with Excess Token PreventionACM SIGSOFT Distinguished Paper Award Technical Papers Lianghong Guo Sun Yat-sen University, Yanlin Wang Sun Yat-sen University, Ensheng Shi Xi’an Jiaotong University, Wanjun Zhong Sun Yat-sen University, Hongyu Zhang Chongqing University, Jiachi Chen Sun Yat-sen University, Ruikai Zhang Huawei Cloud Computing Technologies, Yuchi Ma Huawei Cloud Computing Technologies, Zibin Zheng Sun Yat-sen University DOI | ||
11:10 20mTalk | FT2Ra: A Fine-Tuning-Inspired Approach to Retrieval-Augmented Code Completion Technical Papers Qi Guo Tianjin University, Xiaohong Li Tianjin University, Xiaofei Xie Singapore Management University, Shangqing Liu Nanyang Technological University, Ze Tang Nanjing University, Ruitao Feng Singapore Management University, Junjie Wang Tianjin University, Jidong Ge Nanjing University, Lei Bu Nanjing University DOI | ||
11:30 20mTalk | Calico: Automated Knowledge Calibration and Diagnosis for Elevating AI Mastery in Code Tasks Technical Papers Yuxin Qiu University of California at Riverside, Jie Hu University of California at Riverside, Qian Zhang University of California at Riverside, Heng Yin University of California at Riverside DOI | ||