Transcript www.uwplatt.edu

Automated Testing
Group 4 – Evan Feller, Lawrence Green, Alexander Eirmann,
David Duffrin, Ryan Shohoney
Introduction
What is Automated Testing
-
Wikipedia Definition : “In software testing, test automation is the use of
special software (separate from the software being tested) to control the
execution of tests and the comparison of actual outcomes with predicted
outcomes. Test automation can automate some repetitive but necessary
tasks in a formalized testing process already in place, or add additonal testing
that would be difficult to perform manually.”
Why use Automated Testing
-
Some software testing can be tedious and time wasting to do manually
Manual approaches might not always find all the errors in software
- Automated Testing offers a quick and efficient method of handling these
2 issues
Benefits of Automated Testing
-
-
Once automated tests are planned and developed they can be reused quickly
and efficiently
- Good for projects with a long life span/maintenance life
- Good for making sure that little changes do not affect other features
throughout the development of the software
Being a more efficient way of catching problems than manual testing
- Will save you money to find these errors in an earlier stage rather than
during deployment
- Can be repeated quicker than manual testing can
- Once made it should not need adjusting
- Will not underestimate effort of test cases leading to more cost in the
future of project
Cons of Automated Testing
-
At first it is more expensive than manual testing
-
Because of this fact it is used for long life projects since after multiple
reuse it will be more cost effective than manually testing throughout the
whole project.
-
It takes more time for the initial development of a feature
-
Requires a higher skill level of your team
-
Complex Analysis required when a failed test is encountered
-
-
Is the test now obsolete due to a change
-
Or is it that the test is non-obsolete and we just made a mistake
If you change your code and your test does not take it into account, a passed
test might not be the correct answer
-
Gives you a fall sense of security
If so many cons then why use automated testing
-
Automated Testing in the long run is a much better way to test your project
-
The cons can all be avoided if automated testing is used correctly and people
stay up to date on their changes
-
-
not making undocumented changes or breaking the program
If used with the right project it will save you vast amounts of time and money
Code-Driven Testing
Code-Driven Testing (Test-driven)
•A software development process which uses a variety of inputs to test a software
public interfaces.
•Use Test cases
•Describe tests that need to be run to verify that the program runs as
expected.
•Use Unit tests
•Define the functionality of the code before it is written.
•Determine if various sections of the code is acting as expected under
different circumstances
More reliable, costs less, & safer
•Better over all code coverage
•Every line of code will be tested at least once
•Runs constantly during development phase
•besides just the end
•Discover defects upon making changes
•The earlier you catch and fix a bug the cheaper it is
•Less likely to create new defects
•Breaks down the code into simpler form
•Less code duplication
•Allows the developer to focus only on what is currently important
Code-Driven Testing process
1.Add a test
2.Run all tests to see if the new one fails
3.Write some code
4.Run test
5.Refactor code
6.Repeat steps 1-5
Graphical User Interface Testing
GUI testing
-
The purpose of GUI testing is to ensure the correct functionality of the
application and that it conforms to the specified requirements
-
In addition to the pure functionality, GUI testing also covers non-functional
aspects such as:
-
layout
-
captions
-
colors
-
buttons
-
fonts
-
lists
-
font sizes
-
icons
-
labels
-
links
-
text boxes
-
content
-
text formatting
GUI testing issues
-
There are a lot of GUI operations! A simple program like WordPad already
has 325 different GUI operation
-
The GUI often changes (e.g. redesign), even if the underlying functionality
stays the same.
=> Regression testing is a big issue e.g. path through the GUI might fail
because a button, menu item or dialog have changed
GUI testing issues
-
A lot of GUI operations require a certain sequence of operations before they
can be performed e.g. to open a file
=> Click “open file icon” -> browse to folder - > select file - > click open
-
Increasing the number of possible operations increases the sequencing
problem exponentially
=> Human couldn’t handle that -> create them automatically
GUI testing solutions
-
Capture & playback:
-
Record clicks and keyboard inputs and replay them automatically
E.g. Testing Anywhere or Selenium
-
Advantage: Very simple
-
Drawback: Test might not work anymore after a GUI change
=> Big maintenance efforts (re-record the previous “test-cases”)
-
https://www.youtube.com/watch?v=JwJ7LfZiKGQ
GUI testing solutions
-
Scripted
-
Test is implemented by using the API of a framework e.g. Selenium
WebDriver API
-
WebDriver invokes methods from the various GUI elements to create
the desired test case e.g. open browser, clicks or even playing videos
-
Resulting test is similar to a unit test
-
Advantage: Independent from layout changes
-
https://www.youtube.com/watch?v=V1z0vmRrphg
GUI testing solutions
-
Planning system
-
Approach from artificial intelligence area
-
Creates a kind of finite state automata which represents the different
states and transitions of the GUI
-
Some tools can use such a model to automatically create test cases
which cover most of the GUI’s sequences and functionality
GUI testing solutions
API Driven Testing
What is API Testing?
-White-Box Testing Method
-API (Application Programming Interface)
-The testing of an application through the use of public and
private APIs
-Becoming more common due to GUI complications
How is it done?
-Scripting or programming language for scripts
-Scripts call exposed interfaces
-Scripts can be executed by:
-Programming Languages
-Automation frameworks
-Test results compared with expected behavior
What to test for?
-Several things are tested:
-Return value or lack thereof
-Proper event triggering
-Event
-API Call
-Data structure updates
-internal data
-external data (databases)
Typical API Targets
Some commonly tested APIs:
-REST
-JSON
-SOAP
-HTTP
-COM
-Custom built APIs as well
Framework Approach In Testing
What is the Framework Approach in Testing
-
Wikipedia Definition: “A test automation framework is an integrated system
that sets the rules of automation of a specific product. This system integrates
the function libraries, test data sources, object details and various reusable
modules. These components act as small building blocks which need to be
assembled to represent a business process. The framework provides the
basis of test automation and simplifies the automation effort.”
Benefits of the Framework Approach in Testing
-
The main advantage is the low cost for maintenance
Only the test case file needs to be updated in case of a change to any test
case
No need to update the scripts in case of changes to the application
Quality Costs of Framework Approach in Testing
-
Main costs due to development and maintenance
Cost may vary based on the following:
- The framework/scripting technique chosen
- The approach of scripting used during test automation
Various Techniques of Framework Approach in Testing
-
Linear
-
Structured
-
Data-driven
-
Keyword-driven
-
Hybrid
-
Agile automation framework
Responsibilities of Framework Approach in Testing
1) Defining the format in which to express expectations
2) Creating a mechanism to hook into or drive the application under test
3) Executing the tests
4) Reporting results
Test Automation Interface Model
Test Automation Interface
-
-
Interface Engine
-
Built on top of Interface Environment
-
Consists of:
The parser is present to parse the object files coming from the object
repository into the test specific scripting language
-
The test runner executes the test scripts using a test harness
Interface Environment
-
-
-
Consists of:
-
Framework Library have modules related with the overall test suite
-
Product/Project Library have modules specific to the application under
test
Object Repository
Test Automation Interface
-
Test automation interface are platforms that provide a single workspace for
incorporating multiple testing tools and frameworks for System/Integration
testing
-
Goal: simplify the process of mapping tests to business criteria without
coding coming in the way of the process
-
Expected to improve the efficiency and flexibility of maintaining test scripts