JUCS - Journal of Universal Computer Science 20(5): 615-639, doi: 10.3217/jucs-020-05-0615
Controlled Experiments Comparing Black-box Testing Strategies for Software Product Lines
expand article infoPaola Accioly, Paulo Borba, Rodrigo Bonifacio§
‡ Federal University of Pernambuco, Recife, Brazil§ University of Brasília, Brasília, Brazil
Open Access
Abstract
SPL testing has been considered a challenging task, mainly due to the diversity of products that might be generated from an SPL. To deal with this problem, techniques for specifying and deriving product specific functional test cases have been proposed. However, there is little empirical evidence of the benefits and drawbacks of these techniques. To provide this kind of evidence, in a previous work we conducted an empirical study that compared two design techniques for black-box manual testing, a generic technique that we have observed in an industrial test execution environment, and a product specific technique whose functional test cases could be derived using any SPL approach that considers variations in functional tests. Besides revisiting the first study, here we present a second study that reinforce our findings and brings new insights to our investigation. Both studies indicate that specific test cases improve test execution productivity and quality.
Keywords
black-box testing, software product lines, empirical software engineering