Download Report

Transcript Agile_Practice_Benchmark_Case_Study_OpCord

Agile Practices (Testing)
Benchmarking Case Study
About Organization - OpCord
General Organizational Information
Captive / Non Captive
Non Captive
About Business
Products / Services
Major Verticals
Testing (Automation), Mobile Apps,
Number of Employees
30 - 35
Quality/Process Models
Agile Methodologies
Others 1
Others 2
Contact for this Presentation
[email protected]
Agile Penetration @ Organization
Agile Related Information
Type of Project
(please elaborate)
Manual Testing , Test Automation, Mobile
Apps Development
Supply Chain Mgmt, Logistics, Web
Applications, Portals
Testing Technologies
Open Source tools, like, AutoIT, Selenium,
Sahi, Jmeter, Jbehave, Sikuli, Junit etc and
commercial tools, like, QTP, HPQC etc
Number of Projects &
% of Projects
Agile Institutionalized
Inception, last 2.5 yrs
Agile Practices in Project
Project Name Desktop Application Test Automation (DATA)
Project Size /
4 test programmers at OpCord and 6 Developers at Client location
(India & US)
Project Type
Migration to newer technologies
Skype was used for voice calls / screen sharing with US/India folks;
TeamViewer was also used for screen sharing;
Client visited OpCord team for initial 2 weeks to explain the
product/domain and set expectations
Daily meeting with client and clarifications
Agile Approach
Scrum; TDD & CI from XP
Agile Practices
sprint/iteration, Standup meeting, continuous integration,
Automated testing, code refactoring
Agile Metrics
% Automated Test Cases
Testing tool, AutoIT; CI tool, Hudson; VM Player, Google Docs, SVN
Why Agile in this
Customer was very particular on ATDD and Test Automation;
OpCord’s rich experience in Agile helped scale faster on agile testing
Testing Objectives
Comprehensive testing of the application
Test automation using open source tool from end user’s and developer’s
Test Automation required for different platforms, like, Windows XP,
Windows Vista (64 bit & 32 Bit), Windows 7 (64 bit & 32 Bit) etc
Measure Performance Parameters
CPU & Memory utilization for each test case
Time taken to execute every activity, every scenario and for the complete test suite
Testing of Installer
Agile Testing Strategy –
Automation Tool - AutoIT
AutoIT v3.2 (recent version, 2011) is a freeware BASIC-like scripting language
designed for automating the Windows GUI and general scripting.
Uses a combination of simulated keystrokes, mouse movement and window/control
manipulation in order to automate tasks in a way not possible or reliable with other
languages (e.g. VBScript and SendKeys).
Compatible with all versions of Windows, 2000 / XP / 2003 / 2008. Works with
Windows Vista’s and Windows 7 User Account Control (UAC) also
Has abundant built-in commands
Interact with all standard windows controls, like, move, hide, show, resize, activate,
close feature etc. Windows can be referenced by title, text on the window, size,
position, class and even internal Win32 API handles.
Detailed helpfile and large community-based support forums
Comes with SciTe editor/compiler, which allows AutoComplete AutoIt3 commands,
Intellisense (Show a ToolTip), Code folding for easy code viewing, Auto indentation
while typing and dynamic help facility!
Agile Testing –
Test Environment
Test scripts to be made compatible on 5 windows OSs
Client provided VMs (Virtual Machines) for all required windows licenses. It
saved number of machines
In total, 4 machines were allocated with min 3 GB RAM. Min 3 GB RAM is
needed to run VMs and automated test suite
We faced issues with Vista OS and had to increase RAM more
Later stage of project, more machines were provided because each test cycle
use to take 4-5 hours to complete
Agile Testing Strategy –
Framework -1
At first, test automation is targeted for Windows XP OS
All common functions were identified, like, main menus, delete, etc and
scripts were written
Scripts and Data Separation
Single code base was designed to run on all agreed operating systems
Project Folder structure was decided like any other development project
Agile Testing Strategy –
Framework -2
Coding guidelines, Naming Conventions, Commenting guidelines were agreed
upon and evolved over time
Automation was done at 2 levels
Scenario based (Developer’s perspective)
Each scenario had 5 to 20 test cases and In total, about 100 major scenarios (work-flow)
were identified with the help of client and automated
Each test scenario (or workflow) was automated as separate function
One scenario takes 5 to 7 minutes to complete the testing and provide test results
Each scenario can be run independently so that developer can test particular functionality
Total Flow (End user’s perspective)
Later, All scenarios were integrated to run as a single test suite
Whole testing was made possible by single command
If a scenario fails than it records it as FAIL and continue to run next test scenario
Agile Testing Approach –
Client shared the base software and visited OpCord to explain the
Team first wrote the test cases and shared with client developers
Developers developed (or migrated) the desktop application using newer
technologies and meanwhile testing team did the automation of test cases
Daily meetings were held with client to clarify the requirements as new
application had to resolve issues in earlier version
Skype and TeamViewer were used extensively to do reviews and pairprogramming when needed
It was must for everyone to always remain available on skype to have
seamless discussions
Skype discussions were saved and sent to all team members, if required
Google docs were used to share test cases and other documents
Agile Testing Execution
Every week team released automated scenarios / scripts to developers
Developers used these scripts to fine tune the code and check whether
functionality is correctly built or not
Defects were reported in Excel (using Google Docs) or basically failed test
cases were considered as defects
After stabilizing scripts on Windows XP, Test environment was set for other
OSs and scripts were fine tuned for other OS
Single code base was maintained to ensure compantibility
Build process was automated using Hudson and whole test suite was
integrated with build process on cloud
Any time, any one can check the health of the product by just clicking a
Sometimes, AutoIT is not able to detect diagrams in the applications
Scripts are sensitive to key input, while running of scripts, no other
applications or internet should run on the system.
While scripting, TAB orders were made based on the Control Focus on the
Present Window. This situation may not occur during UAT.
Time out & Sleep time issues varies with the environment
If testing involves data creation than data should be deleted once script is
over to bring application to same old state
Integration is elaborative – Script editing of a single file requires compilation
of other dependency files along with the edited file
Recommendations & Significant
Use freewares like Skype, TeamViewer, Google Docs, BugZilla to collaborate
with distributed teams
Test suite automation is like developing application hence similar approach
needs to be followed
Agile testing and test automation is a must for products however need to keep
maintenance aspect in mind
No. of testers reduced 5 times because scripts were compatible with 5
different operating systems
Total regression testing took only 5 hours per OS and can be scheduled in
Modular approach to test automation suite allows for reusability of scripts
Thank You