Transcript Software Test Automation

Software Test Automation
SQuAD
February 2010
Scott M. Allman
[email protected]
http://xkcd.com/627
Classi
c
2
Outline
•This is an interactive session
•What I am NOT covering today
•Stages of software testing and tools for that
stage
•General observations on the tools
•Your stories and questions about software test
automation tools
3
Not Covered Today
•Commercial tools
•Art of downloading and installing
•Comparisons of different tools that play similar
roles
•People problems - technical problems
4
Universal and Easy Tools
•OpenOffice Freeware substitute for MS Office
– http://www.openoffice.org/
•Yahoo Widgets
– ScreenShot
• JEdit - Language sensitive editor
– http://www.jedit.org/
•XAMPP
– http://www.apachefriends.org/en/xampp-windows.html
5
6
7
8
Often Heard Reasons To Not
Automate Software Testing
• I am not a programmer
• Test automation tools do not apply to our
system
• If I automate the testing I might lose my job
• It takes too much time and not worth the
investment
• Can you think of other reasons?
9
Stages of Software Testing
1.Design the experiments
2.Create the data and inputs
3.Set up the lab
4.Execute tests
5.Analyze results
6.Report results
7.Retrospect
10
Design the Experiments
•Few tools to help with experimental designs
•Familiar: ECP and Boundary Value Analysis
•FitNesse – a universal test runner
–User inputs to tests
– Worksheet inputs
11
FitNesse
•Installation
– http://www.fitnesse.org/
•Key features
– Wiki hierarchy of tests
– History feature
– Run Fitnesse
•Your stories?
12
Create the Data
•Generating data – XML
•Generating loads
•Mock objects for tough scenarios
•Your Stories?
13
Set up the Lab
•Ensure tests run the same
–Ant
–Scripting languages
14
Execute the Tests
•Unit Testing - JUnit
•Ant, again
•Your stories ?
15
Analyze Results
•Visual capture of the screen and save the
image
•Chainsaw - Logs, custom logs, logs of all types
16
Chainsaw example
17
Chainsaw Log Analyzer
• Installation
– http://logging.apache.org/chainsaw/index.html
• Replaces scripts – tail, grep
• Originally designed for log4j but will work with
any well structured log
• Run the Chainsaw
18
Report Results
•Installation
– http://blog.testlink.org/
•Testlink
– Reports are available anytime, anywhere
– Archived results
– Also has other test manager features
•Run TestLink
19
Retrospect
•How good are the tests ?
•Coverage analyzers offer some data
•Installation
– http://emma.sourceforge.net/
•Emma Coverage Report
20
Summarize
•All tests need designs – Worksheets, TestLink
•Data and inputs – simulators, mock object
•Set up the lab – ant, TestLink
•Execute the tests – Nunit, scripting languages
•Analyze results – Chainsaw, screen capture
•Report results – TestLink
•Retrospect – coverage analyzers
21
Algorithm: Open Source Tools
Here is a simple approach:
1.Find it
2.Download and install it
3.Learn it
4.Adapt it
5.Adopt it
6.Try getting with FindBugs (for Java)
22
Your Stories
•Tools
•Recommended tools?
•Failures and disappointments ?
•Success stories ?
23
Sources for the Tools
–
–
–
–
–
–
–
–
http://xkcd.com/627
http://www.openoffice.org/
http://www.jedit.org/
http://www.apachefriends.org/en/xampp-windows.html
http://www.fitnesse.org/
http://logging.apache.org/chainsaw/index.html
http://blog.testlink.org/
http://emma.sourceforge.net/
•My thanks to SQuAD for the opportunity to present today
•[email protected]
24
Questions?
25