Transcript TRANSITIONING FROM MANUAL TO AUTOMATED TESTING TRANSITIONING FROM MANUAL TO AUTOMATED TESTING Our Story UI Test Automation Data ETL Test Automation Q&A Resources \

TRANSITIONING FROM MANUAL TO AUTOMATED
TESTING
TRANSITIONING FROM MANUAL TO
AUTOMATED TESTING
1
Our Story
2
UI Test Automation
3
Data ETL Test Automation
4
Q&A
5
Resources
\
2
OUR STORY
IN THE BEGINNING…
Testing as a Service
What do Developers need?
What are End Users looking for?
“Management” Implications
Existing Manual Test Team
Is the transition really possible?
\
3
LEARNING FROM PAST EXPERIENCE
Maintenance costs far more than the initial effort
Hours of production code changes should not lead to days of test
code changes
Sales tax example
Can make it harder for a non-technical person to follow
\
4
SPLITTING TEST INTENTION FROM TEST
IMPLEMENTATION
Cucumber
Gherkin syntax
Easy entry point for manual testers or other staff
Easy future transition to true BDD or acceptance test first approach
\
5
PROGRAMMING LANGUAGE SELECTION
Ruby
First project was Ruby on Rails
Had experience
Comfortable coaching
Good community
\
6
THE TRANSITION
CAN IT BE DONE?
Early Success
All tests automated in first sprint of our first project
Continuous Integration in second sprint
\
7
CONTINUOUS INTEGRATION
Travis CI
Hosted service, more difficult to customize
Jenkins
Self-hosted, more customizable
\
8
TESTING THE API LAYER
First Restful API, Java production code
Ruby backed Cucumber inserted in to the Maven Build cycle
Fun with Jacoco code coverage
How much of the code is covered by unit tests? 45%
How much by the Cucumber tests? 78%
Both? 86%
JUnit
Cucumber
Fun with Static Analyzers
\
9
TEAM BUILDING
Difficult to recruit excellent test engineers
Transitioned the existing team
Investment in test training
Lead by example
Be willing to share knowledge
Patience
Both with the trainee and with the coach
Individual training plans
Self–led training
\
10
SIX MONTHS IN
Those transitioning still require direction
Duplication between UI and API tests
Back to Sales Tax, where would it be better to test all 50 states?
Started increased focus on API layer
Ideally, even further down in the domain layer
\
11
TODAY
Independent work nearly all the time
Few of the teams are comfortable testing at all levels of the stack
Still favoring Ruby
More emphasis on refactoring
\
12
IF I HAD A TIME MACHINE…
Written in the same language as the developer
Started training plans earlier
Begin with less emphasis on UI
Shared responsibility with developers more
\
13
TOMORROW
Move from spending most of the time testing ourselves to spending
most of our time helping developers test themselves
More emphasis on building generic test solutions
Out of the box testing
\
14
AUTOMATION OF UI TESTS IN
SIMULMEDIA
\
15
TOOLS THAT WE USE FOR UI AUTOMATION
To write tests:
To run tests:
Cucumber
Jenkins
Selenium WebDriver, Watir
WebDriver + Page Object gem
Headless gem
Ruby
Docker
Browsers: Firefox, Chrome
Rspec
\
16
TEST DESIGN PROBLEM
Code example:
@driver.navigate.to ‘http://simulmedia.net’
@driver.find_element(:id, 'Email').send_keys email
@driver.find_element(:id, 'Passwd').send_keys password
@driver.find_element(:id, 'signIn').click
Problems with this approach:
No separation between the test method and locators
Locators would be spread in multiple tests
Hard to read
Hard to maintain
\
17
SOLUTION
Page Object Model:
A page object contains the representation of the page, and the services
the page provides via methods
No code related to what is being tested should be within the page object
Page objects themselves should never make verifications or assertions
Web Page
Page Class
Test
Base Page
Class
\
18
CREATING PAGE OBJECT
Creating Login page class with page-object gem:
class LoginPage
include PageObject
text_field(:email, :id => 'Email')
text_field(:password, :id => 'Password')
button(:sign_in, :id => 'signIn')
def login_with(email, password)
self.email = email
self.password = password
sign_in
end
end
Usage of the page would be:
login_page.login_with '[email protected]', ’password'
\
19
BEFORE / AFTER
email = '[email protected]'
password = 'my super secret password’
Before:
@driver.navigate.to ‘http://simulmedia.net’
@driver.find_element(:id, 'Email').send_keys email
@driver.find_element(:id, 'Passwd').send_keys password
@driver.find_element(:id, 'signIn').click
After:
login_page.login_with email, password
\
20
TEST DATA DESIGN
Do not
change test
data of other
tests
Running well
on all
environments
Tests are
independent
Run in
parallel!
\
21
SUMMARY
Cucumber + Watir/Selenium + Page Object = work great together!
Reduce code duplication
Enhance test maintenance
Design your test data in a smart way
Links:
https://github.com/cheezy/page-object
http://watirwebdriver.com
Nice book from Jeff Morgan:
“Cucumber and Cheese”
\
22
DATA ETL TEST AUTOMATION
\
23
DATA ETL TEST AUTOMATION
ETL (Extract Transform Load)
The process of taking data from the source system, transforming the data,
and loading the data into a target database
ETL_Overview
\
24
DATA ETL TEST AUTOMATION
ETL at Simulmedia
Consume and process millions set top box data daily for a number of data
providers
Pull data from the data providers
Extract/Unpack data
Transform data
Load data into the Data Warehouse
Consume schedule data and cost data and align them with all the events
from set top box data
\
25
TOOLS FOR ETL TEST AUTOMATION
PYTHON TOOLSET
Python: The language of choice for ETL
Lettuce: Because it is Cucumber-like
\
26
PUTTING THE TOOLS INTO PRACTICE
Quick wins
Work well and easy to maintain
Run locally and on Jenkins
Developers share responsibility for maintaining tests
\
27
PYTHON AND LETTUCE LESSONS
LEARNED
PROS:
Able to write test code purely in Python
CONS:
Steep learning curve
Required ETL to be completed, not iterative
ETL uses cloud services, which are difficult to replicate locally
\
28
TOOLS FOR ETL TEST AUTOMATION
SCALA TOOLSET
New ETL using Scala
Processes millions of files daily
Biggest part of the ETL was the Transformation state, LOTS of logic
So we also automated our tests in Scala
Used Specs2
Unit testing framework
Good enough for testing the Transformation stage
Used for learning
But probably use Cucumber-JVM next time
\
29
SCALABILITY IN ETL TEST AUTOMATION
Not easy to scale between projects
Trying out new roles:
Test Engineers define test guidelines
Developers automate the tests for specific projects
Test Engineers review pull requests and sign off
Allowing Test Engineers more time to focus on:
Gathering test requirements and identifying edge cases
Testing frameworks that can be used on all ETLs
\
30
SUMMARY
Share responsibility
Set the bar high
Build momentum
\
31
Q&A
RESOURCES
Cucumber - https://cucumber.io/
Selenium - http://docs.seleniumhq.org/docs/03_webdriver.jsp
Watir - http://watirwebdriver.com
Page Object gem - https://github.com/cheezy/page-object
Learn Ruby the Hard Way - http://learncodethehardway.org/ruby/
RubyMine - https://www.jetbrains.com/ruby/
Lettuce - http://lettuce.it/
Scala - https://github.com/cucumber/cucumber-jvm/tree/master/scala
Specs2 - https://etorreborre.github.io/specs2/
Jacoco - http://www.eclemma.org/jacoco/trunk/doc/
\
33
SIMULMEDIA RESOURCES
Simulmedia Engineering Blog
http://simulmedia.com/insights/blog/categories/engineering/
Building an ETL framework with Akka and Scaldi
Stop adding features! A lesson for start-up engineers
Modifying Redis for Fun and Profit
Careers - http://simulmedia.com/about/careers/
Data Engineer
Senior UI/UX Engineer
\
34
THANK YOU
JOHN SINGH
@Simulmedia
visit us at: www.simulmedia.com
KATERYNA SEMENOVA
SCOTT VANHO