Enhancing Multi-Agent System Testing with Diversity-Guided Exploration and Adaptive Critical State Exploitation
This program is tentative and subject to change.
Multi-agent systems (MASs) have achieved remarkable success in multi-robot control, intelligent transportation, and multiplayer games, etc. Thorough testing for MAS is urgently needed to ensure its robustness in the face of constantly changing and unexpected scenarios. Existing methods mainly focus on single-agent system testing and cannot be directly applied to MAS testing due to the complexity of MAS. To our best knowledge, there are fewer studies on MAS testing. While several studies have focused on adversarial attacks on MASs they primarily target failure detection from an attack perspective, i.e., discovering failure scenarios, while ignoring the diversity of scenarios. In this paper, to highlight a typical balance between exploration (diversifying behaviors) and exploitation (detecting failures), we propose an advanced testing framework for MAS called MASTest with diversity-guided exploration and adaptive critical state exploitation. It incorporates both individual diversity and team diversity, and designs an adaptive perturbation mechanism to perturb the action at the critical states, so as to trigger more and more diverse failure scenarios of the system. We evaluate MASTest on two popular MAS simulation environments: Coop Navi and StarCraft II. Results show that the average distance of the resulting failure scenarios is increased by 29.55%-103.57% and 74.07% - 370.00% on two environments compared to the baselines. Also, the failure patterns found by MASTest are improved by 71.44%-300.00% and 50%-500.00% on two experimental environments compared to the baselines.
This program is tentative and subject to change.
Fri 20 SepDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
15:30 - 16:30 | |||
15:30 20mTalk | Enhancing Multi-Agent System Testing with Diversity-Guided Exploration and Adaptive Critical State Exploitation Technical Papers Ma Xuyan University of Chinese Academy of Sciences, Beijing, China; Laboratory for Internet Software Technologies, Institute of Software Chinese Academy of Sciences, Beijing, China;, Yawen Wang Institute of Software, Chinese Academy of Sciences, Junjie Wang Institute of Software, Chinese Academy of Sciences, Xiaofei Xie Singapore Management University, Boyu Wu University of Chinese Academy of Sciences, Beijing, China, Shoubin Li Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences, Fanjiang Xu Institute of Software, Chinese Academy of Sciences, Qing Wang Institute of Software, Chinese Academy of Sciences | ||
15:50 20mTalk | Commit Artifact Preserving Build Prediction Technical Papers Guoqing Wang Peking University, Zeyu Sun Institute of Software, Chinese Academy of Sciences, Yizhou Chen Peking University, Yifan Zhao Peking University, Qingyuan Liang Peking University, Dan Hao Peking University | ||
16:10 20mTalk | Detecting Build Dependency Errors in Incremental Builds Technical Papers Jun Lyu Nanjing University, Shanshan Li Software Institute, Nanjing University, He Zhang Nanjing University, Yang Zhang Nanjing University, Guoping Rong Nanjing University, Manuel Rigger National University of Singapore DOI |