Transcript Document

Cem Kaner, J.D., Ph.D.
Director
Mike Andrews, Ph.D.
Pat Bond, Ph.D.
James Whittaker, Ph.D.
Scott Tilley, Ph.D.
MISSION: Create effective, grounded, timely materials to
support the teaching and self-study of software testing,
software reliability, and quality-related software metrics.
1
2
Testing?
• Testing is often taught as a routine activity.
– This misses the point
• Testing is a technical investigation of a product
– Complete testing would require infinite time
– Instead, we have to apply judgment & skill
• to develop the optimal set of tests and test documentation
• within the project’s time, cost and reporting constraints
• THIS is a much more challenging task,
– to learn
– to teach, and
– to do.
3
Testing Techniques?
• Testing courses are often focused on
– a miscellaneous collection of test techniques, plus
– rules for doing the paperwork
• Any technique addresses one or a few of these six issues:
–
–
–
–
–
–
Risk (what bugs we’re looking for)
Activity (how we do the testing)
Tester (who does the testing)
Coverage (how much of what gets tested)
Evaluation (how to tell whether the program passed the test)
Reporting requirements (needed outputs)
• Our task is to apply the right techniques to
serve the specific needs of the project at hand
• For that, we need to understand
– techniques
– context (the circumstances surrounding the project)
– project requirements (how testing serves the project)
4
5
Black Box Testing Course
• Professional version in development since 1992
• Taught at wide range of companies, including Hewlett-Packard,
Microsoft, Intel, Symantec, Postalsoft, Avid, BEA Systems,
Cognos, California Dept of Transportation, Iomega, Peoplesoft,
Rational, Cigital, Software AG, Safeco, many others.
• Techniques taught / practiced in context of a sample application
• Customizable
– At your site (1-5 days)
• Core learning objectives
• Selection of topics and examples
• Balance of lab vs lecture
• (co-taught with your technical manager)
– At Florida Tech (4.5 days typical)
• Selection of topics
• To lesser extent, lab vs lecture
6
Black Box Testing Course Materials
• Oracles
• Impossibility of complete
testing
• Techniques
– Function testing
– Domain testing
– Risk-based testing
• Bug taxonomies
• Attacks
– Scenario testing
– Specification-based
– User testing
– Stress testing
– Multivariable tests
– Regression testing
– Stochastic testing & highvolume automation
• Exploratory testing
–
–
–
–
Scripting problems
Questioning strategies
Ad hoc models
paired testing
• Testability
• Bug reporting
– effective advocacy
– quality/cost analysis
• Test documentation
• Role of test groups /
mission of testing
• Adapting testing to
different lifecycles
• Test-related metrics and
measurement theory
• Recruiting
• Career planning
• Evaluating staff
performance
– doc components
– analyzing requirements
for testdocs
• Test planning
• Outsourcing
• Legal issues
7
Possibilities for the Future
We’re glad to organize these if there is enthusiasm:
• Homebrew test automation
Available NOW
– Bret Pettichord
– Using / customizing the new generation of free software test tools
• Test-driven development course
Potentially now
– project-driven, paired programming
– extensive personal practice, feedback
• Testing boot camp
–
–
–
–
Summer 2005? Interest?
3 weeks
High teacher : student ratio
Mix of academic and practitioner teachers
Extensive personal practice/coaching
• Lecture (a.m)
• Coached practice (p.m.)
• Group assignments (evening)
8
9