Transcript here - TAMU Computer Science Faculty Pages

CSCE 181
Test Automation at Schlumberger
Glen Accardo, HTC Test Automation Lead
John Wanke, HQ Test Automation Engineer
Agenda

Introduction

Overview of Schlumberger

A footnote about your degree program

What is automated testing, and why do it?

Challenges in test automation
Glen Accardo

Started at Schlumberger in 2005

Tested drilling, real-time, and production applications

Currently I manage the test automation function at
the Houston Technology Center
Glen Accardo



Studied at University of Southwestern Louisiana
– Computer Science, Mathematics, English
Held the following positions before Schlumberger
– Technical writer
– Web master/application developer
– Developer/Tester for database performance
monitoring/tuning, and backup/recovery
http://www.linkedin.com/pub/glen-accardo/0/905/904
Agenda

Introduction

Overview of Schlumberger

A footnote about your degree program

What is automated testing, and why do it?

Challenges in test automation
What is Schlumberger?




The leading provider of
technology and project
management to the upstream
oil and gas industry for over 80
years
Over 110,000 employees, 140
nationalities in 80 countries
25 research & engineering
facilities worldwide-2010
invested $919 million in R&D
The only true multinational
corporation in the composition
of its workforce, management
team and Board of Directors
Agenda

Introduction

Overview of Schlumberger

A footnote about your degree program

What is automated testing, and why do it?

Challenges in test automation
Anyone Recognize This Phrase?
Program Overview
…a sound preparation in science, mathematics, English,
The computer science curriculum is designed to prepare students to enter the rapidly expanding
statistics
computing.
computer
field.and
It is based
upon the Institute of Electrical and Electronic Engineer Computer Society
and the Association for Computing Machinery recommendations for curricula and courses. The fouryear undergraduate curriculum in computer science includes a sound preparation in science,
mathematics, English, statistics and computing. Students majoring in computer science at Texas
A&M will take a 12-hour area of concentration in another department. Electives are available in the
areas of: algorithms, languages and computability, software systems, networking, computer systems
and architecture, artificial intelligence and cognitive modeling, vision, graphics and robotics and
computational science and engineering. Advanced undergraduates may take graduate courses in
these areas.
Why is English in the List?


Nearly all jobs require “Excellent communications skills”
Employers will probably see your writing before your
programming—laziness or ineptitude in one will be applied to
the other

Not just for technical writers

Not just for a job interview

Clear writing will distinguish you from people with equal
technical skills
Agenda

Introduction

Overview of Schlumberger

A footnote about your degree program

What is automated testing, and why do it?

Challenges in test automation
What is Testing?
“Testing is an infinite process of comparing the invisible
to the ambiguous in order to avoid the unthinkable
happening to the anonymous.”
- James Bach
Problem Number 1
Development Effort
Incremental
Which Leads To:
- Risks for customers
- Risks for product team
- Increased support cost
Testing Effort
Cumulative
Take short cuts:
- Don’t test features
- Don’t test configurations
- Don’t do non-functional tests
Debate the finish criteria:
- Remove features
- Extend deadlines
- Add resources
Solution: Add Test Automation
Development Effort
Manual Test Effort
Test Automation Effort
What if we eliminate repetitive tests?
We can refocus manual test efforts
on important, new testing.
What Must Automated Tests Do?



Mimic behavior of a user
or application in an
environment
Validate the results of
this behavior
Maintain or improve
upon standards of other
forms of testing.
What is Test Automation Not?


Not record and playback
Not a cheap alternative
to other methods of
testing
Example
The TestComplete version of “Hello World.”
Impact?

Application exists (is installed?)

Application starts without error

Application accepts input without error

Application saves file without error

Application exits without error
Basic Metrics
Metric
Value
Time to Develop
15 minutes
Time to run
16 seconds
Time to run manually
2 minutes
Metric
Value
Performance Improvement
86%
Can run 7.5x as often
Cost
Cost of computer for 16 seconds vs.
Cost of person for 2 minutes
Expandability
Can run simultaneously on any number of
configurations
Repeatability
Near perfect
Problem Number 2
Question: “Should add feature X?”
Answer: “Hmmm.”
Question: “Should we support configuration X?”
Answer: “Yes.”
Testing Effort Grows on Multiple Dimensions
Features
Configurations
Data Sets
Two Part Solution


All Pairs Testing
http://en.wikipedia.org/wiki/All-pairs_testing
Run automated tests in parallel
WebCam of HTC Automation
Impact
Product
Automated Tests
Equivalent Manual Tests
Studio
20 hours * 4 platforms
8 person weeks
Matira
5 hours * 3 platforms
4 person weeks
Tupai
3 hours * 9 platforms
5 person weeks
PIPESIM
40 hours * 5 platforms
?????
Seabed
4 platforms * 1 hour
1 person week
OFM
TBD
TBD
Merak
TBD
TBD
MEP
TBD
TBD
IAM
TBD
TBD
Problem Number 3
Agenda

Introduction

Overview of Schlumberger

A footnote about your degree program

What is automated testing, and why do it?

Challenges in test automation
John Wanke



Started at Schlumberger in 2010
Tested economics, asset management, and enterprise planning
applications
Currently I lead the test automation effort for the Production
Solutions group (services-based products)
John Wanke



Studied at Wichita State University
– Electrical Engineering, Mathematics
Held the following positions before Schlumberger
– Lead Systems Analyst / Software Engineer
– Lab Systems Administrator
– Web master/application developer
– I/T Manager
– Software Quality Assurance Engineer/Developer for
message queueing administration and monitoring
http://www.linkedin.com/in/johnwanke
Management, Marketing, and Machiavelli

Well-known

Alien and foreign

Established

Newcomer

Easily understood

Complex

Simple operation

Requires skill to use
Challenges of Test Automation:
Manage Expectations
Metric
Value
Time to Develop
15 minutes
Time to run
16 seconds
Time to run manually
2 minutes
Reality
Meaning
Development Skills
Much more than “hello world”
I/T skills
Computers don’t magically appear and operate
Application Knowledge
Not all applications are == notepad
Automation tool skills
The application doesn’t run itself, does it?
Testing skills
What can be tested? How?
Collaboration
Dev aware of how to test. Tester must show results.
Merak Enterprise Planning (MEP) Solution Overview
Integrate
Merak Peep
databases
OFM
Excel
DB
Intelligent oil & gas dimensions
Asia B.U.
N. America
B.U.
VOLTS
Quality
Control
Staging
area
Europe
B.U.
Data
Warehouse
Calc
Node
External Systems
Secure rolebased web or
desktop
access
Provide
Visualize
Calc
Node
Calc
Node
Calc
Node
Synchronize
Generate
Merak
Cube
Calc Node
Calculate
Collaborate
MEP Product Architecture
The Technical Challenge
“The challenge you face is gaining automated / deterministic
control of your test environment.” -- Jonathan Lee
The Technical Challenge, Part 1:
Known Test Platforms
Merak Enterprise Planning Test Environment
The Technical Challenge, Part 2:
Known Testware
Development Effort
Manual Test Effort
Test Automation Effort

Software written to test software

Subject to same controls as any other software asset
–
Versioned to be in sync with app
The Technical Challenge, Part 3:
Find a Tester or Make One
Q&A