Towards More Complete Constraints for Deep Learning Library Testing via Complementary Set Guided Refinement
Deep learning library is important in AI systems. Recently, many works have been proposed to ensure its reliability. They often model inputs of tensor operations as constraints to guide the generation of test cases. However, these constraints may narrow the search space, resulting in incomplete testing. This paper introduces a complementary set-guided refinement that can enhance the completeness of constraints. The basic idea is to see if the complementary set of constraints yields valid test cases. If so, the original constraint is incomplete and needs refinement. Based on this idea, we design an automatic constraint refinement tool, DeepConstr, which adopts a genetic algorithm to refine constraints for better completeness. We evaluated it on two DL libraries, PyTorch and TensorFlow. DeepConstr discovered 84 unknown bugs, out of which 72 were confirmed, with 51 fixed. Compared to state-of-the-art fuzzers, DeepConstr increased coverage for 43.44% of operators supported by NNSmith, and 59.16% of operators supported by NeuRI.
Wed 18 SepDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
13:30 - 14:50 | Domain-Specific TestingTechnical Papers at EI 10 Fritz Paschke Chair(s): Marcelo d'Amorim North Carolina State University | ||
13:30 20mTalk | UPBEAT: Test Input Checks of Q# Quantum Libraries Technical Papers Tianmin Hu Northwest University, Guixin Ye Northwest University, Zhanyong Tang Northwest University, Shin Hwei Tan Concordia University, Huanting Wang University of Leeds, Meng Li Hefei University of Technology, Zheng Wang University of Leeds DOI | ||
13:50 20mTalk | Towards More Complete Constraints for Deep Learning Library Testing via Complementary Set Guided Refinement Technical Papers Gwihwan Go Tsinghua University, Chijin Zhou Tsinghua University, Quan Zhang Tsinghua University, Xiazijian Zou Central South University, Heyuan Shi Central South University, Yu Jiang Tsinghua University DOI | ||
14:10 20mTalk | AsFuzzer: Differential Testing of Assemblers with Error-Driven Grammar InferenceACM SIGSOFT Distinguished Paper Award Technical Papers DOI | ||
14:30 20mTalk | Ma11y: A Mutation Framework for Web Accessibility Testing Technical Papers Mahan Tafreshipour University of California at Irvine, Anmol Vilas Deshpande University of California at Irvine, Forough Mehralian University of California at Irvine, Iftekhar Ahmed University of California at Irvine, Sam Malek University of California at Irvine DOI | ||