Transcript .ppt
Research Paper Analysis Written By: Qing Xie and Atif Memon ACM Transactions on Software Engineering and Methodology, vol. 18, issue 2 November 2008 Presented By: Matt Catron EEL6883 Spring 2009 University of Central Florida BACKGROUND & RELATED WORK OBJECTIVE KEY CONCEPTS PILOT STUDY ANALYSIS AND RESULTS EVENT INTERACTION GRAPH APPLICATION OF CONCEPTS THREATS TO VALIDITY FUTURE RESEARCH SUMMARY AND CONCLUSIONS Graphical User Interfaces (GUIs) are one of the most widely used concepts in software development Testing of GUI functionality is understudied “Did you really test EVERY possible scenario?” Concept of Event Flow Graph (EFG), which can model all possible sequences of events EFG’s refined into Event Interaction Graph (EIG), which also portrays the interaction between events Definition of EIG’s through informal GUI code examination “to systematically generate potentially problematic sequences by empirically understanding event sequences that lead to successful fault detection” A GUI’s can be modeled as a set of objects and and a set of properties of those objects. An event causes a state transition Events can be strung together into sequences The set of all possible event sequences becomes the EFG SIMPLE GUI EVENT FLOW GRAPH A fault F happens when the expected and actual states differ immediately after an event e occurs Effective Event Context (EEC) is any sequence of events that will lead to a fault Hence, the MEEC is the shortest EEC to produce a given fault F Use TerpOffice suite as test subjects Applications include TerpWord, TerpCalc, TerpPaint, and TerpSpreadSheet Fault seeding was used to place known deterministic errors into each application 200 versions of each application were created, with one fault seeded in each ALGORITHM FOR TEST CASE GENERATION USING EFG TEST CASE LENGTH FREQUENCY DISTRIBUTION Many events occurred in multiple test cases For a specific event ei: In some cases it would cause a fault In other cases it would not This means that the event itself is not the cause of the fault EEC must occur for ei to cause the fault Event context must be satisfied for the fault to occur EEC has an effect 163 mutants killed out of possible 800 Average MEEC for the 4 applications: TerpCalc – 2.21 TerpPaint – 3.57 TerpSpreadSheet – 4.62 TerpWord – 3.86 Three Types of Events: Reachability Events (R) Termination Events (T) System Interaction Events (S) R and T events manipulate the structure of the GUI EIG represents an executable sequence of events Derived from EFG by removing all R events and adjusting edges on a per-node basis Less nodes than an EFG, but number of paths may be more Test cases can be easily generated from the EIG EVENT INTERACTION GRAPH FOR SIMPLE GUI Selected 4 applications from SourceForge.net Open Source Robust Extensive bug reports Significant QA provisions Millions of downloads each GUITAR tool (http://guitar.cs.umd.edu) Automated EIG and test case creation Fully automatic end-to-end GUI testing NUMBER OF TEST CASES GENERATED FOR EACH VERSION OF EACH APPLICATION NUMBER OF FAULTS DETECTED FOR EACH VERSION OF EACH APPLICATION 69 faults were found in well-maintained and highly used open source software Some bugs counted in multiple versions of the software Exceptions include: NullPointerException FileNotFoundException NumberFormatException SocketException Crashes were the only failures set up to be detected Fairly simple back-end to all case study applications Fairly undeveloped EIG model may lack robustness Should study functionality failures beyond crashes Evolve EIG modeling techniques by maturing the MEEC concept Apply MEEC concept to regression testing Vary system configuration Test GUI at the component level Can this be applied to Non-GUI applications? GOOD BAD Definition of MEEC Produced actual results on actual applications (SourceForge bug #1245216) With some effort, commercialization is possible Concepts are new and not yet robust Selection of test cases focused on non-complex GUI’s Make it robust Commercialize it Apply to a wider array of applications