ISSTA 2024
Mon 16 - Fri 20 September 2024 Vienna, Austria
co-located with ISSTA/ECOOP 2024

This program is tentative and subject to change.

Thu 19 Sep 2024 11:10 - 11:30 at EI 7 - Program Repair 2

Test flakiness, a non-deterministic behavior of builds irrelevant to code changes, is a major and continuing impediment to delivering reliable software. The very few techniques for the automated repair of test flakiness are specifically crafted to repair either Order-Dependent (OD) or Implementation-Dependent (ID) flakiness. They are also all symbolic approaches, i.e., leverage program analysis to detect and repair known test flakiness patterns and root causes, failing to generalize. To bridge the gap, we propose FlakyDoctor, a neuro-symbolic technique that combines the power of LLMs—generalizability—and program analysis—soundness—to fix different types of test flakiness. Our extensive evaluation using 873 confirmed flaky tests (332 OD and 541 ID) from 243 real-world projects demonstrates the ability of FlakyDoctor in repairing flakiness, achieving 57% (OD) and 59% (ID) success rate. Comparing to three alternative flakiness repair approaches, FlakyDoctor can repair 8% more ID tests than DexFix, 12% more OD flaky tests than ODRepair, and 17% more OD flaky tests than iFixFlakies. Regardless of underlying LLM, the non-LLM components of FlakyDoctor contribute to 12–31 % of the overall performance, i.e., while part of the FlakyDoctor power is from using LLMs, they are not good enough to repair flaky tests in real-world projects alone. What makes the proposed technique superior to related research on test flakiness mitigation specifically and program repair, in general, is repairing 79 previously unfixed flaky tests in real-world projects. We opened pull requests for all cases with corresponding patches; 19 of them were accepted and merged at the time of submission.

This program is tentative and subject to change.

Thu 19 Sep

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

10:30 - 11:50
Program Repair 2Technical Papers at EI 7
10:30
20m
Talk
Automating Zero-Shot Patch Porting for Hard Forks
Technical Papers
Shengyi Pan Zhejiang University, You Wang Zhejiang University, Zhongxin Liu Zhejiang University, Xing Hu Zhejiang University, Xin Xia Huawei Technologies, Shanping Li Zhejiang University
DOI Pre-print
10:50
20m
Talk
Benchmarking Automated Program Repair: An Extensive Study on Both Real-World and Artificial Bugs
Technical Papers
Yicheng Ouyang University of Illinois at Urbana-Champaign, Jun Yang UIUC, Lingming Zhang University of Illinois Urbana-Champaign
DOI
11:10
20m
Talk
Neurosymbolic Repair of Test Flakiness
Technical Papers
Yang Chen University of Illinois at Urbana-Champaign, Reyhaneh Jabbarvand University of Illinois at Urbana-Champaign
11:30
20m
Talk
AutoCodeRover: Autonomous Program Improvement
Technical Papers
Yuntong Zhang National University of Singapore, Haifeng Ruan National University of Singapore, Zhiyu Fan National University of Singapore, Abhik Roychoudhury National University of Singapore

Information for Participants
Thu 19 Sep 2024 10:30 - 11:50 at EI 7 - Program Repair 2
Info for room EI 7:

Map: https://tuw-maps.tuwien.ac.at/?q=CDEG13

Room tech: https://raumkatalog.tiss.tuwien.ac.at/room/15417