ISSTA 2024
Mon 16 - Fri 20 September 2024 Vienna, Austria
co-located with ISSTA/ECOOP 2024
Wed 18 Sep 2024 10:50 - 11:10 at EI 3 Sahulka - UI-Level Testing Chair(s): Chunyang Chen

Flaky tests hinder the development process by exhibiting uncertain behavior in regression testing. A flaky test may pass in some runs and fail in others while running on the same code version. The non-deterministic outcome frequently misleads the developers into debugging non-existent faults in the code. To effectively debug the flaky tests, developers need to reproduce them. The industry de facto to reproduce flaky tests is to rerun them multiple times. However, rerunning a flaky test numerous times is time and resource-consuming.

This work presents a technique for rapidly and reliably reproducing timing-dependent GUI flaky tests, acknowledged as the most common type of flaky tests in Android apps. Our insight is that flakiness in such tests often stems from event racing on GUI data. Given stack traces of a failure, our technique employs dynamic analysis to infer event races likely leading to the failure and reproduces it by selectively delaying only \textit{relevant} events involved in these races. Thus, our technique can efficiently reproduce a failure within minimal test runs. The experiments conducted on 80 timing-dependent flaky tests collected from 22 widely-used Android apps show our technique is efficient in flaky test failure reproduction. Out of the 80 flaky tests, our technique could successfully reproduce 73 within 1.71 test runs on average. Notably, it exhibited extremely high reliability by consistently reproducing the failure for 20 runs.

Wed 18 Sep

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

10:30 - 11:50
UI-Level TestingTechnical Papers at EI 3 Sahulka
Chair(s): Chunyang Chen TU Munich
10:30
20m
Talk
Toward the Automated Localization of Buggy Mobile App UIs from Bug Descriptions
Technical Papers
Antu Saha William & Mary, Yang Song William & Mary, Junayed Mahmud University of Central Florida, Ying Zhou George Mason University, Kevin Moran University of Central Florida, Oscar Chaparro William & Mary
DOI
10:50
20m
Talk
Reproducing Timing-Dependent GUI Flaky Tests in Android Apps via a Single Event Delay
Technical Papers
Xiaobao Cai Fudan University, Zhen Dong Fudan University, Yongjiang Wang Fudan University, Abhishek Tiwari University of Passau, Xin Peng Fudan University
DOI
11:10
20m
Talk
Semantic Constraint Inference for Web Form Test Generation
Technical Papers
Parsa Alian University of British Columbia, Noor Nashid University of British Columbia, Mobina Shahbandeh University of British Columbia, Ali Mesbah University of British Columbia
DOI
11:30
20m
Talk
Guardian: A Runtime Framework for LLM-Based UI Exploration
Technical Papers
Dezhi Ran Peking University, Hao Wang Peking University, Zihe Song University of Texas at Dallas, Mengzhou Wu Peking University, Yuan Cao Peking University, Ying Zhang Peking University, Wei Yang University of Texas at Dallas, Tao Xie Peking University
DOI

Information for Participants
Wed 18 Sep 2024 10:30 - 11:50 at EI 3 Sahulka - UI-Level Testing Chair(s): Chunyang Chen
Info for room EI 3 Sahulka:

Map: https://tuw-maps.tuwien.ac.at/?q=CF0205

Room tech: https://raumkatalog.tiss.tuwien.ac.at/room/15663