Objective
The present study evaluates the total and additional WC techniques aimed at prioritizing test cases for multiple aggregate changes, in order to assess their prioritization results and efficiency as compared with other change-based approach and a baseline (random prioritization), from the point of view of software test engineers.
Research Questions and Metrics
RQ1. Are the test suites prioritized with the WC techniques more efficient in identifying faults in mutants of the System Under Test (SUT) with multiple modifications?
For this measurement, we apply well-known metrics Average Percentage Faults Detected (APFD) [Elbaum et al.] and F-measure [Zhou].
RQ2. What is the answer of RQ1, taking into account mutants with a single modification?
Also using APFD and F-Measure.
RQ3. Do the test suites prioritized with the WC techniques maintain spreading of failed test cases as presented by the Changed Blocks technique?
For this answer, we apply metric F-Spreading [Alves et al.] and proposed two metrics for assessing previously-undetected defect anticipation and grouping, Group-measure and Group-spreading.
RQ4. What is the answer of RQ3, considering the spreading of test cases for a single modification?
Also using F-Spreading, Group-spreading and Group-Measure.
RQ5. What is the difference in execution time for the prioritization algorithms?
Execution time as measured in exact clone environments.
Setup
Systems and mutants
Data
Main conclusions