A Decision Support System for Integration Test Selection

by Sahar Tahvili

Software testing generally suffers from time and budget limitations. Indiscriminately executing all available test cases leads to sub-optimal exploitation of testing resources. Selecting too few test cases for execution on the other hand might leave a large number of faults undiscovered. Test case selection and prioritization techniques can lead to more efficient usage of testing resources and also, early detection of faults. Test case selection addresses the problem of selecting a subset of an existing set of test cases, typically by discarding test cases that do not improve the quality of the system under test. Test case prioritization schedules test cases for execution in order to increase their effectiveness at achieving some performance goals such as: earlier fault detection, optimal allocation of testing resources and reducing overall testing effort. In practice, prioritized selection of test cases requires the evaluation of different test case criteria. Therefore, this problem can be formulated as a multi-criteria decision making problem. As the number of decision criteria grows, application of a systematic decision-making solution becomes a necessity. In this thesis, we propose a tool-supported framework using a decision support system, for prioritizing and selecting integration test cases in embedded system development. This framework provides a complete loop for selecting the best candidate test case for execution based on a finite set of criteria. The results of multiple case studies, done on a train control management subsystem, from Bombardier Transportation AB in Sweden, demonstrate how our approach helps to select test cases in a systematic way. This can lead to early detection of faults while respecting various criteria. Also, we have evaluated a customized return on investment metric to quantify the economic benefits in optimizing system integration testing using our framework.

Hardcover

Published October 1, 2016

About the author

Sahar Tahvili is a researcher at the software testing laboratory at Mälardalen University, who holds a Ph. D. in Software Engineering since 2018. Her doctoral thesis entitled " Multi-Criteria Optimization of System Integration Testing " was named one of the best new Software Integration Testing books by BookAuthority.
Sahar’s research focuses on artificial intelligence (AI), advanced methods for testing complex software-intensive systems, and designing decision support systems (DSS). Previously she worked as a senior researcher at the Research Institutes of Sweden (RISE) with close industrial research collaboration with Bombardier transportation in Sweden.