Distance-Aware Test Input Selection for Deep Neural Networks
Deep Neural Network (DNN) testing is one of the common practices to guarantee the quality of DNNs.
However, DNN testing in general requires a significant amount of test inputs with oracle information (labels), which can be challenging and resource-intensive to obtain. To relieve this problem, we propose DATIS, a distance-aware test input selection approach for DNNs. Specifically, DATIS adopts a two-step approach for selecting test inputs. In the first step, it selects test inputs based on improved uncertainty scores derived from the distances between the test inputs and their nearest neighbor training samples. In the second step, it further eliminates test inputs that may cover the same faults by examining the distances among the selected test inputs. To evaluate DATIS, we conduct extensive experiments on 8 diverse subjects, taking into account different domains of test inputs, varied DNN structures, and diverse types of test inputs. Evaluation results show that DATIS significantly outperforms 15 baseline approaches in both selecting test inputs with high fault-revealing power and guiding the selection of data for DNN enhancement.
Thu 19 SepDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
10:30 - 11:50 | |||
10:30 20mTalk | Distance-Aware Test Input Selection for Deep Neural Networks Technical Papers Zhong Li Nanjing University, Zhengfeng Xu Nanjing University, Ruihua Ji Nanjing University, Minxue Pan Nanjing University, Tian Zhang Nanjing University, Linzhang Wang Nanjing University, Xuandong Li Nanjing University DOI | ||
10:50 20mTalk | Test Selection for Deep Neural Networks using Meta-Models with Uncertainty Metrics Technical Papers Demet Demir Middle East Technical University, Aysu Betin Can Middle East Technical University, Elif Surer Middle East Technical University DOI | ||
11:10 20mTalk | Datactive: Data Fault Localization for Object Detection Systems Technical Papers Yining Yin Nanjing University, Yang Feng Nanjing University, Shihao Weng Nanjing University, Yuan Yao Nanjing University, Jia Liu Nanjing University, Zhihong Zhao Nanjing University DOI | ||
11:30 20mTalk | TeDA: A Testing Framework for Data Usage Auditing in Deep Learning Model Development Technical Papers Xiangshan Gao Zhejiang University; Huawei Technology, Jialuo Chen Zhejiang University, Jingyi Wang Zhejiang University, Jie Shi Huawei International, Peng Cheng Zhejiang University, Jiming Chen Zhejiang University; Hangzhou Dianzi University DOI | ||