LLM4Fin: Fully Automating LLM-Powered Test Case Generation for FinTech Software Acceptance Testing
This program is tentative and subject to change.
FinTech software, crucial for both safety and timely market deployment, presents a compelling case for automated acceptance testing against regulatory business rules. However, the inherent challenges of comprehending unstructured natural language descriptions of these rules and crafting comprehensive test cases demand human intelligence. The emergence of Large Language Models (LLMs) holds promise for automated test case generation, leveraging their natural language processing capabilities. Yet, their dependence on human intervention for effective prompting hampers efficiency.
In response, we introduce a groundbreaking, fully automated approach for generating high-coverage test cases from natural language business rules. Our methodology seamlessly integrates the versatility of LLMs with the predictability of algorithmic methods. We fine-tune pre-trained LLMs for improved information extraction accuracy and algorithmically generate comprehensive testable scenarios for the extracted business rules. Our prototype, LLM4Fin, is designed for testing real-world stock-trading software. Experimental results demonstrate LLM4Fin’s superiority over both state-of-the-art LLM, such as ChatGPT, and a skilled testing engineer. Our approach achieves remarkable performance, with up to 98.18% business scenario coverage in about 7 seconds on generating test cases. Furthermore, LLM4Fin reduces time costs from 4 hours to a mere 17 seconds. These results provide robust evidence of the framework’s practical applicability and efficiency, marking a significant advancement in FinTech software testing.
This program is tentative and subject to change.
Wed 18 SepDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
13:30 - 14:50 | |||
13:30 20mTalk | LLM4Fin: Fully Automating LLM-Powered Test Case Generation for FinTech Software Acceptance Testing Technical Papers Zhiyi Xue East China Normal University, Liangguo East China Normal University, Senyue Tian East China Normal University, Xiaohong Chen ECNU, Pingping Li Guotai Junan Securities Co., Ltd, Liangyu Chen East China Normal University, Tingting Jiang Guotai Junan Securities Co., Ltd, Min Zhang East China Normal University | ||
13:50 20mTalk | Domain Adaptation for Code Model-based Unit Test Case Generation Technical Papers Jiho Shin York University, Sepehr Hashtroudi , Hadi Hemmati York University, Song Wang York University | ||
14:10 20mTalk | Practitioners’ Expectations on Automated Test Generation Technical Papers | ||
14:30 20mTalk | UniTSyn: A Large-Scale Dataset Capable of Enhancing the Prowess of Large Language Models for Program Testing Technical Papers Yifeng He University of California, Davis, Jiabo Huang Tencent, Yuyang Rong University of California, Davis, Yiwen Guo Unaffiliated, Ethan Wang University of California, Davis, Hao Chen University of California, Davis |