Towards Automatic Oracle Prediction for AR Testing: Assessing Virtual Object Placement Quality under Real-World Scenes
Augmented Reality (AR) technology opens up exciting possibilities in various fields, such as education, work guidance, shopping, communication, and gaming. However, users often encounter usability and user experience issues in current AR apps, often due to the imprecise placement of virtual objects. Detecting these inaccuracies is crucial for AR app testing, but automating the process is challenging due to its reliance on human perception and validation. This paper introduces VOPA (Virtual Object Placement Assessment), a novel approach that automatically identifies imprecise virtual object placements in real-world AR apps. VOPA involves instrumenting real-world AR apps to collect screenshots representing various object placement scenarios and their corresponding metadata under real-world scenes. The collected data are then labeled through crowdsourcing and used to train a hybrid neural network that identifies object placement errors. VOPA aims to enhance AR app testing by automating the assessment of virtual object placement quality and detecting imprecise instances. In our evaluation of a test set of 304 screenshots, VOPA achieved an accuracy of 99.34%, precision of 96.92% and recall of 100%. Furthermore, VOPA successfully identified 38 real-world object placement errors, including instances where objects were hovering between two surfaces or appearing embedded in the wall.
Fri 20 SepDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
10:30 - 11:50 | |||
10:30 20mTalk | Atlas: Automating Cross-Language Fuzzing on Android Closed-Source Libraries Technical Papers Hao Xiong Zhejiang University; ZJU-Hangzhou Global Scientific and Technological Innovation Center, Qinming Dai Zhejiang University; ZJU-Hangzhou Global Scientific and Technological Innovation Center, Rui Chang Zhejiang University, Mingran Qiu Zhejiang University; ZJU-Hangzhou Global Scientific and Technological Innovation Center, Renxiang Wang Zhejiang University; ZJU-Hangzhou Global Scientific and Technological Innovation Center, Wenbo Shen Zhejiang University, Yajin Zhou Zhejiang University; ZJU-Hangzhou Global Scientific and Technological Innovation Center DOI | ||
10:50 20mTalk | Feedback-Driven Automated Whole Bug Report Reproduction for Android Apps Technical Papers Dingbang Wang University of Connecticut, Yu Zhao University of Cincinnati, Sidong Feng Monash University, Zhaoxu Zhang University of Southern California, William G.J. Halfond University of Southern California, Chunyang Chen TU Munich, Xiaoxia Sun China Mobile (Suzhou) Software Technology, Jiangfan Shi Zhejiang University, Tingting Yu University of Connecticut DOI | ||
11:10 20mTalk | NativeSummary: Summarizing Native Binary Code for Inter-language Static Analysis of Android Apps Technical Papers Jikai Wang Huazhong University of Science and Technology, Haoyu Wang Huazhong University of Science and Technology DOI | ||
11:30 20mTalk | Towards Automatic Oracle Prediction for AR Testing: Assessing Virtual Object Placement Quality under Real-World Scenes Technical Papers Xiaoyi Yang Rochester Institute of Technology, Yuxing Wang Rochester Institute of Technology, Tahmid Rafi University of Texas at San Antonio, Dongfang Liu Rochester Institute of Technology, Xiaoyin Wang University of Texas at San Antonio, Xueling Zhang Rochester Institute of Technology DOI |