Face It Yourselves: An LLM-Based Two-Stage Strategy to Localize Configuration Errors via Logs
Configurable software systems are prone to configuration errors, resulting in significant losses to companies. However, diagnosing these errors is challenging due to the vast and complex configuration space. These errors pose significant challenges for both experienced maintainers and new end-users, particularly those without access to the source code of the software systems. Given that logs are easily accessible to most end-users, we conduct a preliminary study to outline the challenges and opportunities of utilizing logs in localizing configuration errors. Based on the insights gained from the preliminary study, we propose an LLM-based two-stage strategy for end-users to localize the root-cause configuration properties based on logs. We further implement a tool, LogConfigLocalizer, aligned with the design of the aforementioned strategy, hoping to assist end-users in coping with configuration errors through log analysis.
To the best of our knowledge, this is the first work to localize the root-cause configuration properties for end-users based on Large Language Models (LLMs) and logs. We evaluate the proposed strategy on Hadoop by LogConfigLocalizer and prove its efficiency with an average accuracy as high as $99.91%$. Additionally, we also demonstrate the effectiveness and necessity of different phases of the methodology by comparing it with two other variants and a baseline tool. Moreover, we validate the proposed methodology through a practical case study to demonstrate its effectiveness and feasibility.
Thu 19 SepDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
10:30 - 11:50 | Logging and Field BugsTechnical Papers at EI 10 Fritz Paschke Chair(s): Willem Visser Amazon Web Services | ||
10:30 20mResearch paper | A Large-Scale Evaluation for Log Parsing Techniques: How Far Are We? Technical Papers Zhihan Jiang Chinese University of Hong Kong, Jinyang Liu Chinese University of Hong Kong, Junjie Huang Chinese University of Hong Kong, Yichen LI Chinese University of Hong Kong, Yintong Huo Chinese University of Hong Kong, Jiazhen Gu Chinese University of Hong Kong, Zhuangbin Chen Sun Yat-sen University, Jieming Zhu Huawei Noah’s Ark Lab, Michael Lyu Chinese University of Hong Kong DOI Pre-print | ||
10:50 20mTalk | FastLog: An End-to-End Method to Efficiently Generate and Insert Logging Statements Technical Papers Xiaoyuan Xie Wuhan University, Zhipeng Cai Wuhan University, Songqiang Chen The Hong Kong University of Science and Technology, Jifeng Xuan Wuhan University DOI | ||
11:10 20mTalk | Face It Yourselves: An LLM-Based Two-Stage Strategy to Localize Configuration Errors via Logs Technical Papers Shiwen Shan Sun Yat-sen University, Yintong Huo Chinese University of Hong Kong, Yuxin Su Sun Yat-sen University, Yichen LI Chinese University of Hong Kong, Dan Li Sun Yat-sen University, Zibin Zheng Sun Yat-sen University DOI | ||
11:30 20mTalk | Foliage: Nourishing Evolving Software by Characterizing and Clustering Field Bugs Technical Papers Zhanyao Lei Shanghai Jiao Tong University, Yixiong Chen Shanghai Jiao Tong University, Mingyuan Xia AppetizerIO, Zhengwei Qi Shanghai Jiao Tong University DOI |