Beyond Pairwise Testing: Advancing 3-wise Combinatorial Interaction Testing for Highly Configurable Systems
To meet the rising demand for software customization, highly configurable software systems play key roles in practice. Combinatorial interaction testing (CIT) is recognized as an effective approach for testing such systems. For CIT, the most important problem is constrained covering array generation (CCAG), which aims to construct a minimum-sized t-wise covering array (CA), where t denotes testing strength. Compared to pairwise testing (i.e., 2-wise CIT) that is a widely-used CIT technique, 3-wise CIT can discover more faults and bring more benefit in real-world applications. However, current state-of-the-art CCAG algorithms suffer from the severe scalability challenge for 3-wise CIT, which renders them ineffective in building 3-wise CAs for highly configurable systems. In this work, we perform an empirical study on various practical, highly configurable systems to present that it is promising to build 3-wise CA through extending 2-wise CA. Inspired by this, we propose ScalableCA, a novel and scalable algorithm that can effectively alleviate the scalability challenge for 3-wise CIT. Further, ScalableCA introduces three new and effective techniques, including fast invalidity detection, uncovering-guided sampling, and remainder-aware local search, to enhance its performance. Our experiments on extensive real-world, highly configurable systems show that, compared to current state-of-the-art algorithms, ScalableCA requires one to two orders of magnitude less running time to build 3-wise CA of 38.9% smaller size in average for large-scale instances. Our results indicate that ScalableCA greatly advances the state of the art in 3-wise CIT.
Fri 20 SepDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
13:30 - 14:50 | |||
13:30 20mTalk | Segment-Based Test Case Prioritization: A Multi-objective Approach Technical Papers Hieu Huynh Katalon, Nhu Pham Katalon; Ho Chi Minh City University of Science, Tien N. Nguyen University of Texas at Dallas, Vu Nguyen Katalon; Ho Chi Minh City University of Science; Vietnam National University DOI | ||
13:50 20mTalk | Beyond Pairwise Testing: Advancing 3-wise Combinatorial Interaction Testing for Highly Configurable Systems Technical Papers Chuan Luo Beihang University, Shuangyu Lyu Beihang University, Qiyuan Zhao National University of Singapore, Wei Wu Central South University; Xiangjiang Laboratory, Hongyu Zhang Chongqing University, Chunming Hu Beihang University DOI | ||
14:10 20mTalk | Synthesis-Based Enhancement for GUI Test Case Migration Technical Papers Yakun Zhang Peking University, Qihao Zhu DeepSeek-AI, Jiwei Yan Institute of Software at Chinese Academy of Sciences, Chen Liu Peking University, Wenjie Zhang Peking University, Yifan Zhao Peking University, Dan Hao Peking University, Lu Zhang Peking University DOI | ||
14:30 20mTalk | Revisiting Test-Case Prioritization on Long-Running Test Suites Technical Papers Runxiang Cheng University of Illinois at Urbana-Champaign, Shuai Wang University of Illinois at Urbana-Champaign, Reyhaneh Jabbarvand University of Illinois at Urbana-Champaign, Darko Marinov University of Illinois at Urbana-Champaign DOI |