Datactive: Data Fault Localization for Object Detection Systems
This program is tentative and subject to change.
Object detection models are seamlessly integrated into numerous intelligent software systems, playing a crucial role in various tasks. These models are typically constructed upon human-annotated datasets, whose quality may significantly influence the model performance and reliability. Erroneous and inadequate annotated datasets can induce classification/localization inaccuracies during deployment, precipitating security breaches or traffic accidents that inflict property damage or even loss of life. Therefore, ensuring and improving dataset quality is a crucial issue for the reliability of the object detection system.
In this paper, we propose Datactive, a data fault localization technique for object detection systems. Datactive is designed to locate various types of data faults including mis-localization and missing objects, without utilizing the prediction of object detection models trained on dirty datasets. To achieve this, we first construct foreground-only and background-included datasets via data disassembling strategies. We then train robust classifiers using these disassembled datasets. Based on the predictions made by these classifiers, Datactive produces a unified suspiciousness score for both foreground annotations and image backgrounds. It allows testers to easily identify and correct faulty or missing annotations with minimal effort. We conducted experiments on three application datasets using six baselines, showing the effectiveness of Datactive. We also explored Datactive’s ability to find natural data faults and its application in both training and evaluation scenarios.
This program is tentative and subject to change.
Thu 19 SepDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
10:30 - 11:50 | |||
10:30 20mTalk | Distance-Aware Test Input Selection for Deep Neural Networks Technical Papers Zhong Li , Zhengfeng Xu Nanjing University, Ruihua Ji Nanjing University, Minxue Pan Nanjing University, Tian Zhang Nanjing University, Linzhang Wang Nanjing University, Xuandong Li Nanjing University DOI | ||
10:50 20mTalk | Test Selection for Deep Neural Networks using Meta-Models with Uncertainty Metrics Technical Papers Demet Demir Department of Information Systems, Graduate School of Informatics, Middle East Technical University, Ankara, Türkiye, Aysu Betin Can Department of Information Systems, Graduate School of Informatics, Middle East Technical University, Ankara, Türkiye, Elif Surer Department of Modeling and Simulation, Graduate School of Informatics, Middle East Technical University, Ankara, Türkiye | ||
11:10 20mTalk | Datactive: Data Fault Localization for Object Detection Systems Technical Papers Yining Yin Nanjing University, China, Yang Feng Nanjing University, Shihao Weng Nanjing University, Yuan Yao Nanjing University, Jia Liu Nanjing University, Zhihong Zhao | ||
11:30 20mTalk | TeDA: A Testing Framework for Data Usage Auditing in Deep Learning Model Development Technical Papers Xiangshan Gao Zhejiang University and Huawei Technology, Jialuo Chen Zhejiang University, Jingyi Wang Zhejiang University, Jie Shi Huawei International, Peng Cheng Zhejiang University, Jiming Chen Zhejiang University |