The ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) is the leading research symposium on software testing and analysis, bringing together academics, industrial researchers, and practitioners to exchange new ideas, problems, and experience on how to analyze and test software systems.
ISSTA 2024 will feature two submission deadlines. You can choose to submit at either deadline, but papers submitted to the first deadline may receive a chance to submit a major revision of the initial submission to the second deadline. Papers submitted to the second deadline will be either accepted or rejected, i.e., there is no option for a major revision.
Accepted Papers
Call for Papers
ISSTA invites three kinds of submissions. The majority of submissions is expected to be “Research Papers”, but submissions that best fit the description of “Experience Papers” or “Replicability Studies” should be submitted as such.
Research Papers
Authors are invited to submit research papers describing original contributions in testing or analysis of computer software. Papers describing original theoretical or empirical research, new techniques, methods for emerging systems, in-depth case studies, infrastructures of testing and analysis, or tools are welcome.
Experience Papers
Authors are invited to submit experience papers describing a significant experience in applying software testing and analysis methods or tools and should carefully identify and discuss important lessons learned so that other researchers and/or practitioners can benefit from the experience. Of special interest are experience papers that report on industrial applications of software testing and analysis methods or tools.
Replicability Studies
ISSTA would like to encourage researchers to replicate results from previous papers. A replicability study must go beyond simply re-implementing an algorithm and/or re-running the artifacts provided by the original paper. It should at the very least apply the approach to new, significantly broadened inputs. Particularly, replicability studies are encouraged to target techniques that previously were evaluated only on proprietary subject programs or inputs. A replicability study should clearly report on results that the authors were able to replicate as well as on aspects of the work that were not replicable. In the latter case, authors are encouraged to make an effort to communicate or collaborate with the original paper’s authors to determine the cause for any observed discrepancies and, if possible, address them (e.g., through minor implementation changes). We explicitly encourage authors to not focus on a single paper/artifact only, but instead to perform a comparative experiment of multiple related approaches.
In particular, replicability studies should follow the ACM guidelines on replicability (different team, different experimental setup): The measurement can be obtained with stated precision by a different team, a different measuring system, in a different location on multiple trials. For computational experiments, this means that an independent group can obtain the same result using artifacts which they develop completely independently. This means that it is also insufficient to focus on reproducibility (i.e., different team, same experimental setup) alone. Replicability Studies will be evaluated according to the following standards:
- Depth and breadth of experiments
- Clarity of writing
- Appropriateness of conclusions
- Amount of useful, actionable insights
- Availability of artifacts
We expect replicability studies to clearly point out the artifacts the study is built on, and to submit those artifacts to the artifact evaluation. Artifacts evaluated positively will be eligible to obtain the prestigious Results Reproduced badge.
Two Submission Deadlines and Major Revisions
ISSTA 2024 features two submission deadlines. The instructions in this call apply to both deadlines. You can choose to submit at either deadline. Only papers submitted to the first deadline may receive a chance to submit a major revision of the initial submission to the second deadline. Papers submitted to the second deadline will be either accepted or rejected, i.e., there is no option for a major revision.
Papers that are rejected during the first round may not be resubmitted to the second round. Authors who try to bypass this rule (e.g., by changing the paper title without significantly changing paper content, or by making small changes to the paper content) will have their papers desk-rejected without further consideration. Papers rejected from the first or second submission round can, of course, be submitted to ISSTA 2025 without any restrictions.
Submission Guidelines
Submissions must be original and should not have been published previously or be under consideration for publication while being evaluated for ISSTA. By submitting an article to an ACM Publication, authors are acknowledging that that all co-authors are subject to all ACM Publications Policies, including ACM’s new Publications Policy on Research Involving Human Participants and Subjects. Alleged violations of this policy or any ACM Publications Policy will be investigated by ACM and may result in a full retraction of the paper, in addition to other potential penalties, as per ACM Publications Policy.
Research Papers, Experience Papers, and Replicability Studies should be at most 10 pages in length, with at most 2 additional pages for references only. The page limit is strict, i.e., papers that take more than 10 pages for anything apart from references (including any section, figure, text, or appendix), will be desk-rejected. Experience papers and replicability studies should clearly specify their category in the paper title upon submission, e.g., “XXX (Experience Paper)”. All authors should use the official “ACM Master article template”, which can be obtained from the ACM Proceedings Template pages. Latex users should use the “sigconf” option, as well as the “review” (to produce line numbers for easy reference by the reviewers) and “anonymous” (omitting author names) options. To that end, the following Latex code can be placed at the start of the Latex document:
\documentclass[sigconf,review, anonymous]{acmart}
\acmConference[ISSTA 2024]{ACM SIGSOFT International Symposium on Software Testing and Analysis}{16-20 September, 2024}{Vienna, Austria}
Submit your papers via the HotCRP ISSTA 2024 submission website.
Each submission will be reviewed by at least three members of the program committee. Authors will have an opportunity to respond to reviews during a rebuttal period. Submissions will be evaluated on the basis of originality, importance of contribution, soundness, evaluation, quality of presentation, appropriate comparison to related work, and verifiability/transparency of the work. Some papers may have more than three reviews, as the PC chair may solicit additional reviews based on factors such as reviewer expertise and strong disagreement between reviewers. The authors will have a chance to read the additional reviews and respond to them during the additional short response period. The program committee as a whole will make final decisions about which submissions to accept for presentation at the conference.
Double-blind Reviewing
ISSTA 2024 will conduct double-blind reviewing. Submissions should not reveal the identity of the authors in any way. Authors should leave out author names and affiliations from the body of their submission. They should also ensure that any citations to related work by themselves are written in third person, that is, “the prior work of XYZ” as opposed to “our prior work”.
Authors have the right to upload preprints on arXiv or similar sites, but they must avoid specifying that the paper was submitted to ISSTA.
Authors with further questions on double-blind reviewing are encouraged to contact the Program Chair by email.
Open Science Policy and “Data Availability” Section
ISSTA has adopted an open science policy. Openness in science is key to fostering scientific progress via transparency, reproducibility, and replicability. The steering principle is that all research results should be accessible to the public, if possible, and that empirical studies should be reproducible. In particular, we actively support the adoption of open data and open source principles, and encourage all contributing authors to disclose data to increase reproducibility and replicability.
Upon submission, authors are asked to make their code, data, etc. available to the program committee, or to comment on why this is not possible or desirable. Data must be shared in an anonymized way (e.g., no information on authors/affiliations in the code) via a site that reveals neither the authors’ nor the reviewers’ identities (e.g., not via Google Drive). At least one of the reviewers will check the provided data. While sharing the data is not mandatory for submission or acceptance, it will inform the program committee’s decision. Furthermore, we ask authors to provide a supporting statement on the data availability (or lack thereof) in their submitted papers in a section named “Data Availability” after the Conclusion section.
Publication Date
The official publication date is the date the proceedings are made available in the ACM Digital Library. This date may be up to two weeks prior to the first day of your conference. The official publication date affects the deadline for any patent filings related to published work.
| Wed 31 Jul 2024 Camera-ready papers (Round 2 and Major Revisions) |
| Wed 3 Jul 2024 Notification (Round 2 and Major Revisions) |
| Tue 11 - Thu 13 Jun 2024 Author response (Round 2 and Major Revisions) |
| Fri 12 Apr 2024 Submission deadline (Round 2 and Major Revisions) |
| Sun 31 Mar 2024 Camera-ready papers (Round 1) |
| Sat 2 Mar 2024 Notification (Round 1) |
| Wed 7 - Fri 9 Feb 2024 Author response (Round 1) |
| Fri 15 Dec 2023 Submission deadline (Round 1) |
Michael PradelPC Chair
University of Stuttgart
Germany
Nazareno Aguirre
University of Rio Cuarto and CONICET
Argentina
Saba Alimadadi
Simon Fraser University
Canada
Paolo Arcaini
National Institute of Informatics
Japan
Steven Arzt
Fraunhofer SIT; ATHENE
Germany
Hamid Bagheri
University of Nebraska-Lincoln
United States
Guangdong Bai
University of Queensland
Australia
Jonathan Bell
Northeastern University
United States
Tegawendé F. Bissyandé
University of Luxembourg
Luxembourg
Tegan Brennan
Stevens Institute of Technology
United States
Tevfik Bultan
University of California at Santa Barbara
United States
Marcel Böhme
MPI-SP, Bochum
Germany
Joanna C. S. Santos
University of Notre Dame
United States
Cristian Cadar
Imperial College London
United Kingdom
Haipeng Cai
Washington State University
United States
Sang Kil Cha
KAIST
South Korea
Bihuan Chen
Fudan University
China
Chunyang Chen
Monash University
Australia
Jiachi Chen
Sun Yat-sen University
China
Jieshan Chen
CSIRO's Data61
Australia
Junjie Chen
Tianjin University
China
Kai Chen
Institute of Information Engineering at Chinese Academy of Sciences
China
Sen Chen
Tianjin University
China
Tao Chen
University of Birmingham
United Kingdom
Yuqi Chen
ShanghaiTech University, China
China
Zhenbang Chen
College of Computer, National University of Defense Technology
China
Zhenyu Chen
Nanjing University
China
Shing-Chi Cheung
Hong Kong University of Science and Technology
China
Myra Cohen
Iowa State University
United States
James C. Davis
Purdue University
United States
Jin Song Dong
National University of Singapore
Singapore
Zhen Dong
Fudan University, China
China
Wensheng Dou
Institute of Software Chinese Academy of Sciences
China
Xiaoning Du
Monash University, Australia
Gregory J. Duck
National University of Singapore
Singapore
Lingling Fan
Nankai University
China
Chunrong Fang
Nanjing University
China
Mattia Fazzini
University of Minnesota
United States
Yang Feng
Nanjing University
China
Marcelo F. Frias
Dept. of Software Engineering Instituto Tecnológico de Buenos Aires
United States
Alessio Gambi
Austrian Institute of Technology (AIT)
Austria
Cuiyun Gao
Harbin Institute of Technology
China
Xiang Gao
Beihang University
China
Joshua Garcia
University of California, Irvine
United States
Ali Ghanbari
Auburn University
United States
Milos Gligoric
The University of Texas at Austin
United States
Rahul Gopinath
University of Sydney
Australia
Mark Harman
Meta Platforms, Inc. and UCL
United Kingdom
Dongjie He
Chongqing University, China
China
Pinjia He
Chinese University of Hong Kong, Shenzhen
China
Shilin He
Microsoft Research
n.n.
Dominik Helm
TU Darmstadt | ATHENE - National Research Center for Applied Cybersecurity, Darmstadt
Germany
Martin Hirzel
IBM Research
United States
Xing Hu
Zhejiang University
China
Jeff Huang
Texas A&M University
United States
Yu Huang
Vanderbilt University
United States
Reyhaneh Jabbarvand
University of Illinois at Urbana-Champaign
United States
Gunel Jahangirova
King's College London
United Kingdom
Jiajun Jiang
Tianjin University
China
Yu Jiang
Tsinghua University
China
Hai Jin
Huazhong University of Science and Technology
China
Maria Kechagia
University College London
United Kingdom
Martin Kellogg
New Jersey Institute of Technology
United States
Jacques Klein
University of Luxembourg
Luxembourg
Anil Koyuncu
Bilkent University
Turkey
Wing Lam
George Mason University
United States
Wei Le
Iowa State University
United States
Seongmin Lee
Max Planck Institute for Security and Privacy (MPI-SP)
Germany
Sungho Lee
Chungnam National University, Korea
South Korea
Daniel Lehmann
Google, Germany
Germany
Li Li
Beihang University
China
Lian Li
Institute of Computing Technology at Chinese Academy of Sciences; University of Chinese Academy of Sciences; Zhongguancun Laboratory
China
Shanshan Li
National University of Defense Technology
China
Yi Li
Nanyang Technological University
Singapore
Yuekang Li
The University of New South Wales
Australia
Yun Lin
Shanghai Jiao Tong University
China
Mario Linares-Vásquez
Universidad de los Andes
Colombia
Hui Liu
Beijing Institute of Technology
China
Kui Liu
Huawei
China
Ting Liu
Xi'an Jiaotong University
Yang Liu
Nanyang Technological University
Singapore
Jian-Guang Lou
Microsoft Research
China
Yiling Lou
Fudan University
China
Xiapu Luo
The Hong Kong Polytechnic University
China
Thibaud Lutellier
University of Alberta
Canada
Michael Lyu
Chinese University of Hong Kong
China
Jie M. Zhang
King's College London
United Kingdom
Lei Ma
The University of Tokyo & University of Alberta
Japan
Shiqing Ma
University of Massachusetts, Amherst
United States
Wei Ma
Nanyang Technological University, Singapore
Fernanda Madeiral
Vrije Universiteit Amsterdam
Netherlands
Leonardo Mariani
University of Milano-Bicocca
Italy
Anastasia Mavridou
KBR / NASA Ames Research Center
Ana Milanova
Rensselaer Polytechnic Institute
Kevin Moran
University of Central Florida
United States
Shiva Nejati
University of Ottawa
Canada
Tien N. Nguyen
University of Texas at Dallas
United States
Pengyu Nie
University of Waterloo
Canada
Yannic Noller
Ruhr University Bochum
Germany
Hakjoo Oh
Korea University
South Korea
Alessandro Orso
Georgia Institute of Technology
United States
Rohan Padhye
Carnegie Mellon University
United States
Mike Papadakis
University of Luxembourg
Luxembourg
Kexin Pei
Columbia University; University of Chicago
United States
Xin Peng
Fudan University
China
Thuan Pham
The University of Melbourne
Alexander Pretschner
TU Munich
Germany
Geguang Pu
East China Normal University
Germán Regis
University of Rio Cuarto and CONICET
Argentina
Vincenzo Riccio
University of Udine
Italy
Oliviero Riganelli
University of Milano - Bicocca
Italy
Subhajit Roy
IIT Kanpur
India
Guido Salvaneschi
University of St. Gallen
Switzerland
Elena Sherman
Boise State University
United States
August Shi
The University of Texas at Austin
United States
Jeongju Sohn
University of Luxembourg
Luxembourg
Fu Song
State Key Laboratory of Computer Science and Institute of Software, Chinese Academy of Sciences.
China
Wei Song
Nanjing University of Science and Technology
China
Ezekiel Soremekun
Royal Holloway, University of London
United Kingdom
Cristian-Alexandru Staicu
CISPA Helmholtz Center for Information Security
Germany
Dominic Steinhöfel
CISPA Helmholtz Center for Information Security
Germany
Yuxin Su
Sun Yat-sen University
Yulei Sui
UNSW
Australia
Chengnian Sun
University of Waterloo
Canada
Jun Sun
School of Information Systems, Singapore Management University, Singapore
Singapore
Zeyu Sun
Institute of Software, Chinese Academy of Sciences
China
Chungha Sung
Amazon Web Services, USA
United States
Shin Hwei Tan
Concordia University
Canada
Valerio Terragni
University of Auckland
New Zealand
Frank Tip
Northeastern University
United States
Saeid Tizpaz-Niari
University of Texas at El Paso
United States
Flavio Toffalini
EPFL
Paolo Tonella
USI Lugano
Switzerland
Omer Tripp
Amazon Web Services
Alexi Turcotte
Northeastern University
United States
Yao Wan
Huazhong University of Science and Technology
China
Haoyu Wang
Huazhong University of Science and Technology
China
Ji Wang
School of Computer, National University of Defense Technology, China
Jingbo Wang
University of Southern California
United States
Jingyi Wang
Zhejiang University
China
Shaohua Wang
Central University of Finance and Economics
China
Shuai Wang
The Hong Kong University of Science and Technology
China
Song Wang
York University
Canada
Weihang Wang
University of Southern California
Yanlin Wang
Sun Yat-sen University
China
Zheng Wang
University of Leeds
Heike Wehrheim
University of Oldenburg
Lili Wei
McGill University
Canada
Shiyi Wei
University of Texas at Dallas
United States
Christian Wimmer
Oracle Labs
United States
Jiajing Wu
Sun Yat-sen University
China
Rongxin Wu
School of Informatics, Xiamen University
China
Yueming Wu
Nanyang Technological University
Singapore
Xin Xia
Huawei Technologies
China
Xusheng Xiao
Arizona State University
United States
Tao Xie
Peking University
China
Xiaofei Xie
Singapore Management University
Singapore
Xiaoyuan Xie
School of Computer Science, Wuhan University, China
China
Qi Xin
Wuhan University
China
Zhenchang Xing
CSIRO’s Data61; Australian National University
Australia
Jingling Xue
UNSW Sydney
Australia
Jiwei Yan
Institute of Software at Chinese Academy of Sciences, China
China
Meng Yan
Chongqing University
China
Guowei Yang
University of Queensland
Australia
Jinqiu Yang
Concordia University
Canada
Wei Yang
University of Texas at Dallas
United States
Yibiao Yang
Nanjing University
China
Peisen Yao
Zhejiang University
China
Guixin Ye
Northwest University
China
Shin Yoo
Korea Advanced Institute of Science and Technology
South Korea
Wei You
Renmin University of China
China
Tingting Yu
University of Connecticut
United States
Tao Yue
Beihang University
China
Andreas Zeller
CISPA Helmholtz Center for Information Security
Germany
Juan Zhai
University of Massachusetts, Amherst
United States
Chao Zhang
Tsinghua University
China
Charles Zhang
The Hong Kong University of Science and Technology
China
Fengwei Zhang
Southern University of Science and Technology
China
Hongyu Zhang
Chongqing University
China
Jian Zhang
Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences
China
Lingming Zhang
University of Illinois Urbana-Champaign
United States
Man Zhang
Beihang University, China
Mingxue Zhang
Chinese University of Hong Kong
China
Qian Zhang
University of California, Riverside
United States
Xiangyu Zhang
Purdue University
United States
Yuqun Zhang
Southern University of Science and Technology
China
Zhuo Zhang
Purdue University
United States
Jianjun Zhao
Kyushu University
Japan
Yanjie Zhao
Huazhong University of Science and Technology
China
Hao Zhou
Hong Kong Polytechnic University
China
Marcelo d'Amorim
North Carolina State University
United States