How Does Regression Test Prioritization Perform in Real-World Software Evolution? | |
Lu, Yafeng ; Lou, Yiling ; Cheng, Shiyang ; Zhang, Lingming ; Hao, Dan ; Zhou, Yangfan ; Zhang, Lu | |
2016 | |
关键词 | TEST-SUITE REDUCTION COVERAGE |
英文摘要 | In recent years, researchers have intensively investigated various topics in test prioritization, which aims to re-order tests to increase the rate of fault detection during regression testing. While the main research focus in test prioritization is on proposing novel prioritization techniques and evaluating on more and larger subject systems, little effort has been put on investigating the threats to validity in existing work on test prioritization. One main threat to validity is that existing work mainly evaluates prioritization techniques based on simple artificial changes on the source code and tests. For example, the changes in the source code usually include only seeded program faults, whereas the test suite is usually not augmented at all. On the contrary, in real-world software development, software systems usually undergo various changes on the source code and test suite augmentation. Therefore, it is not clear whether the conclusions drawn by existing work in test prioritization from the artificial changes are still valid for real-world software evolution. In this paper, we present the first empirical study to investigate this important threat to validity in test prioritization. We reimplemented 24 variant techniques of both the traditional and time-aware test prioritization, and investigated the impacts of software evolution on those techniques based on the version history of 8 real-world Java programs from GitHub. The results show that for both traditional and time-aware test prioritization, test suite augmentation significantly hampers their effectiveness, whereas source code changes alone do not influence their effectiveness much.; faculty startup funds from the University of Texas at Dallas; National Basic Research Program of China (973) [2014CB347701]; National Natural Science Foundation of China [61421091, 91318301, 61225007, 61522201]; EI; CPCI-S(ISTP); 535-546; 14-22-May-2016 |
语种 | 英语 |
出处 | 38th IEEE/ACM International Conference on Software Engineering (ICSE) |
DOI标识 | 10.1145/2884781.2884874 |
内容类型 | 其他 |
源URL | [http://ir.pku.edu.cn/handle/20.500.11897/436174] ![]() |
专题 | 信息科学技术学院 |
推荐引用方式 GB/T 7714 | Lu, Yafeng,Lou, Yiling,Cheng, Shiyang,et al. How Does Regression Test Prioritization Perform in Real-World Software Evolution?. 2016-01-01. |
个性服务 |
查看访问统计 |
相关权益政策 |
暂无数据 |
收藏/分享 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论