Transcript Adam Guida Introduction Adam Guida - Senior QA Automation Developer Work History: - 2011 – Present: - Deutsche Bank Global Technologies Cary, NC - 2006-2011: - lulu.com - Raleigh, NC Email: [email protected].

Adam Guida
Introduction
Adam Guida
- Senior QA Automation
Developer
Work History:
- 2011 – Present:
-
Deutsche Bank Global Technologies
Cary, NC
- 2006-2011:
-
lulu.com - Raleigh, NC
Email: [email protected]
2
Automated Testing is a Liability
Unless you have
Stable
 Test Environments
 Deployment Automation
Strong
 Communication
 Collaboration
 Technical Expertise
The Truth About Automated Testing
Automated Testing:
 When done correctly, it can significantly increase the repeatability and
effectiveness of testing efforts
 Augments classical test methodology but does NOT replace manual
testing
 Increases regression consistency
 Reduces tedium
 Allows human testers a greater focus on new and creative ways of breaking
things
However:
 Automated testing is ineffective without an approach that considers all
facets of the development automation ecosystem
Purpose of this lecture
Lessons Learned
 Poorly written test
automation isn’t sustainable
 Writing good tests alone
isn’t enough
 Continuous Automation in QA
 Test Strategies
 Best Practices
 Environments
 Tools & Technologies
 Communication
Continuous Automation
Practices
Continuous Integration (CI)
 A practice where code is automatically compiled and
unit tested on every developer code check-in
Continuous Deployment
 If CI phase passes, application(s) are automatically
deployed to one or more pre-production
environments, triggering additional layers of
automated testing
Continuous Automation
public class Feature {
Unit test failed! I’ll take a
private int 1;
look
at
broke
and
Latest
build
ontry
public
My
void
bad!doSomething(){
I’ll what
check-in
ais
fixup
for
that
if (a
right
== away.
b){
again
CI
for testing!
return true;
}
Sweet! All tests
passed!
DEV & QA
Let me take a look. I see! You’re
Feature: Test A and B are True
This
one is my
fault.
LooksEquals
like I B
Scenario:
Hey,
can youVerify
takeLet’s
athat
lookAat
the last
Awesome!
forgot
to adjust
some“Feature”
testsgrab
after
Given
I open
run?
Looks
like a the
regression test
that
last
build.
I’ll
check-in
a fix &
When Ilunch!
click on “A” and “B”
some
failed.
restart the automation
Then the result is “…….”
Dude, I’m having a problem
getting this test working. The
web service is returning a 404
Test SCM
the wrong resource. I’ll IM
App calling
SCM
you
the correct name
Automation Pipeline
Build +
Unit Test
Test Environment
Deploy
Automated
Testing
Report
Continuous Automation
Collaboration
 Establish partnerships & comradery with your
development team
 Treat QA & Developers as a single team
 Close low-level knowledge gaps in functional testing
 Develop workflows for how DEV & QA address test failures as
a team
 Foster a culture that values cross-team learning, exchange of
knowledge and professional growth
Continuous Automation
Best Practices Demo
Continuous Automation
Best Practices
Automated Test Suite should be
run as often as possible
 Automated tests should run……………………...automatically
 On every successful automated deployment
 Automation executes using the fewest clicks possible
 Start Small and Simple – It won’t all happen overnight
Automation Test Team:
Strategies
 Delegation - Separate the technical from test responsibilities for a given task

Increases efficiency and focus by reducing context switching
 Pair-testing - Pair technical & non-technical testers together for greater
impact
Increases technical experience and promotes mentorship of non-technical testers
 Increases usability & readability of tests

 Reporting - Test Lead/Manager - keep a close eye on automated test reporting.
If tests start failing, seek to understand the underlying causes, then try to assist test team by
removing roadblocks
 Understanding daily/weekly trends in test results helps guide risk assessments

Continuous Automation
Best Practices
Treat Your Test Suite like an Enterprise Application
Automated testing *IS* a form of programming and bound by the same laws as
software development
 Strive to be the best programmer you can & adhere to software development best practices
 Use a Source Control Management tool
 Unit Test your test automation (Yes, that means testing your tests)
 Practice Code reviews
 Seek architectural guidance and input from your development team
 Defect tracking (automated tests apps get bugs too!)
Automated Test Strategies
The Perfect Automated Test
 Fast – Speed impacts every aspect of a quality automation suite
 Reusable – Reuse reduces code complexity and increases simplicity of a test suite
 Relevant – Test indicate a problem we care about and is not redundant
 Independent – Is self sufficient and does not depend on the ordering of other tests
 Focused – Tests a specific aspect of the application without being too broad
 Clear – Determines a pass or failure result automatically, ideally narrowing down the
exact cause of the failure
 Understandable – Is written in such a way that documents behavior other stake-
holders can understand
 Flexible – Change is the only constant – write tests in ways that can handle changes
within reason
Automated Test Strategies
Managing the Complexity Versus Coverage Tradeoff
 System Integration / End-to-End –Provides the
most comprehensive test coverage at the cost of speed
and complexity.
 Unit – Isolated testing of individual classes, methods
and functions
 Component – Tests are isolated to the component
itself. Inputs/Outputs are simulated through test
adapters or mock
Automated Test Strategies
Example of Component Testing
Data Transformer Testing
MOCK DATABASE
H2
End-2-End
TEST 1
User
Database
Product
Database
Marketing
Aggregator
WEB SERVICE CALL
Web Server
Browser
Order
Database
TEST 2
Web Service Testing
TEST 3
MOCK
AGGREGATOR
UI Testing
Automated Test Strategies
Managing the Complexity Versus Coverage Tradeoff
 Diversify your test portfolio using a combination of Unit, Component
and System testing
 Promote Unit Testing w/ Devs - allows you to start trusting some basic
elements of the system, so you can focus on more challenging test scenarios
 Minimize System/End-to-End tests - to fewer high-level smoke tests based
on areas of importance
 Maximize your Component Testing - focus on lower level functionality, API’s
and data model interactions
 Use Test Adapters – for both greater control & reliability of the application
input/outputs
Choosing the right
Tools & Technologies
 Don’t Reinvent the Wheel – someone has already solved your problem
 Carefully Evaluate All Technologies – Require experience prior to adoption.
Invest time in evaluating the pros & cons of a variety of different solutions. Build
POC’s and give stakeholders (developers) a role in decision making
 Straightforward Integration – Utilize tools which accept open formats for
read/write – ensure each tool can fully integrate with others as necessary
 Active Community of Users – Favor technologies where an active online support
community exists over managed (paid) support solutions
Technologies
Programming Language
Rule of thumb: Test Automation should be based on the same
programming language of your application
 Enables sharing of development resources, support and
contribution
 Shared libraries and other resources between testing & dev
 Should be capable of utilizing a highly functional IDE’s such as
Eclipse, Intellij, Visual Studio, etc
Tools: IDEs (Integrated Development Environment)
Intellij IDEA / RubyMines
 Fully integrated support for Cucumber
 Syntax highlighting
 Code Editing/Refactoring
 Supports many languages
 Build-in database browser
 Debugger
Tools: Continuous Integration Server
 CI (Continous Integration Server)
More than just a build agent
 Manages the end-to-end flow of your
automation pipelines and reporting
 Is both the glue and front-end that
binds the continuous deployment,
testing and reporting processes
Tools: VMs - Virtual Machines
 VirtualBox
 Virtualized OS’s

Linux

Windows

Android

OS X
 Contained sandbox
environments
 Backup/Restore, share and
take snapshots of your VM
environments
Tools:
 Create and configure
lightweight, reproducible,
and portable development
environments
 Easy to setup/tear down
and deploy to a
standardize test
environment
 Works with VirtualBox
and other virtual
environment products
 Support integrated into
Intellij IDEA via plug-in
Tools: Deployment Orchestration
 Help automate aspects of environment provisioning &
deployment
 Install & Configure Servers & other resources
 Helps facilitate continuous deployment of your
application
 Many great frameworks to choose from - each has it’s
own set of strength & weakness
Tools & Technology:
Database Management
•Treats SQL like version-able code
• XML, JSON, YAML and generic SQL
syntax supports multiple database
technologies
• Flexible runtime: Maven Plugin and
standalone support
24
Final Thoughts
 Automation is an investment, that when managed effectively, provides a
particular set of benefits which increase efficiency and repeatability of
testing.
AUTOMATION IS NOT A BENEFIT IN ITSELF
 Treat automation as a 1st class citizen of your project
 If you’d like a copy of this presentation, please send me an email at:
[email protected]