Transcript Commercial Test Automation Tools and their real

Prashant Lambat
Sr. Manager SQA Engineering
Symantec Corporation,
Pune
Date: 29 th January 2011
 Save Manual Testing effort
 Better Test coverage across platforms
 Quick, Reliable, Comprehensive, Reusable
 Things which are difficult to do manually (Simulate
load)
 Find Defects early in the cycle
 Build Validation Testing
 Regression Testing
 Load Testing
 Performance Testing
 New Feature Testing
 In-house developed tools
 Open source tools
 Commercial automation tools
 Functional Testing
 QTP , Test Complete, Silk Test, IBM Rational Robot
 Performance Testing
 Load Runner
 Features suitable to Application under test
 Cost per license
 Execution License policy
 Stability of the tool
 Technical Support
 Technical Community
 Forums/Blogs
 Ease of use –Commercial automation tools are much simpler
and easy to use.
 Technical Support: Commercial automation tools provides
support to the corporate world
 Cost: Looks high however looking at the skills/investment to
develop similar kind of tool and support we get its worth.
 Most commercial tools matches the pace of technologies
changes, they usually support Web/GUI technologies(.Net,
MFC, WPF, etc)
 Commercial tools usually have a large community of
users, which translates into better availability of
qualified resources
 Requires not so advanced programming
 Commercial tools are integrated or can be integrated
test case management tools or automation
infrastructure.
“Test Automation is simple, that every tester can do it”
Automation process:
 Record the script
 Enhance the script
Execution
 Run the scripts
 Report results
Under the influence of this myth the QA manager can proudly say:
All our testers are developing test automation.
*Note –This still works if planned properly. In most cases it fails, ~60
of automation projects fails because of poor planning and
execution
Test automation is a software development task
 Automation should be designed, developed and
Tested
 You need to have some kind of a programming
background to implement test automation.
 Test Automation is not as complex as system
programming.
 Test automation standards should be developed
 Automated test components are assets that should be
treated like application source code
Manual test cycle takes 8 hours to execute one cycle
To Automate the same, automation takes 40 hours
In a release if there are 5 test cycles to be done then
we hit a break even.
If there are more cycles to be done across releases
for e.g. 10
then we have 40 hours savings in terms of testing
efforts.
 More testing gets done faster, increasing the odds of
finding defects
 Defects found early have better chances of being fixed
 Manual Testers can concentrate on exploratory testing
to find more defects than spending time on repeating
the same testing.
 Hotfix or service pack testing can be done overnight
with good test coverage in various product areas. This
is not possible via manual testing in a short span of
time
 Structured Automation
 Throw away code automation by Manual Test Engineer
 Automate Test Execution
 Make automation execution uninterrupted
 Build An Infrastructure to track maintain automation
efforts
 Infrastructure to keep track of all the execution results
 Keep Test case management and Defect Tracking system different from
Automation infrastructure
 Keep links to Test case management system and Defect Tracking
system
 Automate new features once the feature is code complete.
 Use Version Control system which is used for product development
(CVS, Perforce etc)
 Set the expectations right in the beginning…you may not get returns in
the first release it self.
 Test Automation Let’s talk business by -Igor
Gershovich