An Empirical Study of Static Analysis Tools for Secure Code Review
Early identification of security issues in software development is vital to minimize their unanticipated impacts. Code review is a widely used manual analysis method that aims to uncover security issues along with other coding issues in software projects. While some studies suggest that automated static application security testing tools (SASTs) could enhance security issue identification, there is limited understanding of SAST’s practical effectiveness in supporting secure code review. Moreover, most SAST studies rely on synthetic or fully vulnerable versions of the subject program, which may not accurately represent real-world code changes in the
code review process.
To address this gap, we study C/C++ SASTs using a dataset of actual code changes that contributed to exploitable vulnerabilities. Beyond SAST’s effectiveness, we quantify potential benefits when changed functions are prioritized by SAST warnings. Our dataset comprises 319 real-world vulnerabilities from 815 vulnerability-contributing commits (VCCs) in 92 C and C++ projects. The result reveals that a single SAST can produce warnings in vulnerable functions of 52% of VCCs. Prioritizing changed functions with SAST warnings can improve accuracy (i.e., 12% of precision and
5.6% of recall) and reduce Initial False Alarm (lines of code in non-vulnerable functions inspected until the first vulnerable function) by 13%. Nevertheless, at least 76% of the warnings in vulnerable functions are irrelevant to the VCCs, and 22% of VCCs remain undetected due to limitations of SAST rules. Our findings highlight the benefits and the remaining gaps of SAST-supported secure code reviews and challenges that should be addressed in future work.
Thu 19 SepDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
15:30 - 17:10 | Vulnerabilities and MalwareTechnical Papers at EI 10 Fritz Paschke Chair(s): Wei You Renmin University of China | ||
15:30 20mTalk | Silent Taint-Style Vulnerability Fixes Identification Technical Papers Zhongzhen Wen Nanjing University, Jiayuan Zhou Huawei, Minxue Pan Nanjing University, Shaohua Wang Central University of Finance and Economics, Xing Hu Zhejiang University, Tongtong Xu Huawei, Tian Zhang Nanjing University, Xuandong Li Nanjing University DOI | ||
15:50 20mTalk | FortifyPatch: Towards Tamper-Resistant Live Patching in Linux-Based Hypervisor Technical Papers Zhenyu Ye Hunan University, Lei Zhou National University of Defense Technology, Fengwei Zhang Southern University of Science and Technology, Wenqiang Jin Hunan University, Zhenyu Ning Hunan University; Xinchuang Haihe Laboratory, Yupeng Hu Hunan University, Zheng Qin Hunan University DOI | ||
16:10 20mTalk | Maltracker: A Fine-Grained NPM Malware Tracker Copiloted by LLM-Enhanced Dataset Technical Papers Zeliang Yu Huazhong University of Science and Technology, Ming Wen Huazhong University of Science and Technology, Xiaochen Guo Huazhong University of Science and Technology, Hai Jin Huazhong University of Science and Technology DOI | ||
16:30 20mTalk | PatchFinder: A Two-Phase Approach to Security Patch Tracing for Disclosed Vulnerabilities in Open-Source Software Technical Papers Kaixuan Li East China Normal University; Nanyang Technological University, Jian Zhang Nanyang Technological University, Sen Chen Tianjin University, Han Liu East China Normal University, Yang Liu Nanyang Technological University, Yixiang Chen East China Normal University DOI Pre-print | ||
16:50 20mTalk | An Empirical Study of Static Analysis Tools for Secure Code Review Technical Papers Wachiraphan (Ping) Charoenwet University of Melbourne, Patanamon Thongtanunam University of Melbourne, Thuan Pham University of Melbourne, Christoph Treude Singapore Management University DOI | ||