UniTSyn: A Large-Scale Dataset Capable of Enhancing the Prowess of Large Language Models for Program Testing
The remarkable capability of large language models (LLMs) in
generating high-quality code has drawn increasing attention
in the software testing community.
However, existing code LLMs often demonstrate unsatisfactory capabilities in generating accurate, complete tests
since they were trained on code snippets collected without
differentiating between code for testing and for other purposes.
In this paper, we present a large-scale dataset, UniTSyn, which can enhance LLMs for Unit Test Synthesis.
Associating tests with the tested functions is crucial for LLMs to infer the expected behavior and the logic paths to be verified.
By leveraging Language Server Protocol, UniTSyn achieves the challenging goal of collecting focal-test pairs without per-project execution setups or per-language heuristics, which tend to be fragile and difficult to scale.
Containing 2.7 million focal-test pairs across five mainstream programming languages, it can enhance the test generation ability of LLMs.
Our experiments demonstrate that,
by building an autoregressive LLM based on UniTSyn,
we can achieve significant benefits in learning and understanding unit test representations,
resulting in improved generation accuracy and code coverage
across all the evaluated programming languages.
Wed 18 SepDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
13:30 - 14:30 | Test GenerationTechnical Papers at EI 9 Hlawka Chair(s): Alessio Gambi Austrian Institute of Technology (AIT) | ||
13:30 20mTalk | Domain Adaptation for Code Model-Based Unit Test Case Generation Technical Papers Jiho Shin York University, Sepehr Hashtroudi University of Calgary, Hadi Hemmati York University, Song Wang York University DOI | ||
13:50 20mTalk | Practitioners’ Expectations on Automated Test Generation Technical Papers Xiao Yu Huawei, Lei Liu Xi’an Jiaotong University, Xing Hu Zhejiang University, Jacky Keung City University of Hong Kong, Xin Xia Huawei, David Lo Singapore Management University DOI | ||
14:10 20mTalk | UniTSyn: A Large-Scale Dataset Capable of Enhancing the Prowess of Large Language Models for Program Testing Technical Papers Yifeng He University of California at Davis, Jiabo Huang Tencent, Yuyang Rong University of California at Davis, Yiwen Guo Unaffiliated, Ethan Wang University of California at Davis, Hao Chen University of California at Davis DOI | ||