Transcript Document

Achieving 99.99% Quality
in SOA Applications –
The “Headless” Dilemma
John Michelsen
StickyMinds Webinar
November 7, 2006
© 2006, iTKO, Inc. | All rights reserved.
Agenda
SOA development and QA trends
Strategy/best practices - the Three C’s
“Headless” Testing Fundamentals
LISA Intro, and LISA WS-Testing Offer
Who are we?
•
Entered the market to deliver an
SOA testing solution, LISA
•
Current version 3.5 (2006)
•
Key value propositions:
•
•
•
Increase Test Coverage
Lower Cost per Test
Faster Discovery & Resolution
Everyone should own quality™.
2006
2004
2003
2001
1999
1999 .
3.5
– LISA 2.0
– LISA 1.0 Released to Market
– LISA under development – (4) Patents Pending
– iTKO Incorporates (Privately Held)
2001.
2003
.
2004
.
2005 .
2006
.
- LISA 3 &
Platform Evolution Causes Testing Evolution
3270/CICS data
stream testing
User-Interface
oriented testing
Mainframe
Client Based
Component,
heterogeneous,
distributed testing
SOA/
Composite Apps
1980 . . . 1985 . . . 1990 . . . 1995 . . . 2000 . . . 2005 . . . 2010 . . . 2015
The Promise of SOA
• Reduce integration cost
• Through loosely-coupled forms of integration and industry standards
• Increase asset reuse
• Build new business workflows from existing Services to form
composite applications
• Increase business agility
• Better control business process definition and management to meet
customer needs
• Reduce business risk
• Governance, compliance, and risk reduction through increased
business visibility and agility
…before SOA
Standard
enterprise apps
Browser
UIs
Client
UIs
Server
(database)
You had a standard delivery platform.
And a standard infrastructure.
SOA promises a composite workflow
Division 1
Transaction service
Transaction
provider
Partner
Reseller
Partner
Your extended
enterprise
Business rules
Reseller
Order management
BI tools
SOA was supposed to simplify extending workflows.
Core app
Outsourced
supplier
Data
warehouse
Division 2
Customer
buyer
Customer buyer
But heterogeneous technology is complex
Transaction service
Your Company
Division 1
Channel Partner
Financials
Standards
and components are still evolving
Mainframe
Legacy Data
Legacy App
.NET Ordering
Service
SOAP objects
Legacy App
Content
Database
Messaging
service
Web interface
Composite apps have multiple owners
Your App
Workflow
Customer company
BI tools
Outsourced firm
Business
Rules
ESB
It’s Continuously
Changing
Data
File System
Division2
warehouse
Web Services
RMI objects
CRM Web App
Why A New Strategy?
Database/
mainframe
= 1 change
• Few changes at the
database level, but multiplies
issues downstream
Middle-tier
level
= 100x
• For every middle tier
change, there are hundreds
of front-end impacts possible
at the UI layer
• Limitless front end options
UI level
=1000x
• Any client, multiple customers
• Mix of technologies –
XML/RPC, Swing, AWT, JS,
DOM, now AJAX
• Services consume business
logic
Best Practices: “Three C’s” of SOA Testing
Complete Testing
• Every heterogeneous layer of
architecture
• Test UI verify in system of record
• Reuse functional test for
performance testing
Complete >
single component  all technologies
Collaborative >
Collaborative Testing
Dev/QA test silos  everyone tests, every phase
Continuous >
Phased tests  constant testing
• Test early before UIs are created
• Not just dev, business analysts and
QA should verify processes
Continuously Test
• Regress on existing functionality
• Add testing of new services to
existing testing workflows
Complete: SOA Testing across every layer
Web UI
Swing
ASP.NET
Presentation layer
BPM
CORBA
J2EE
.NET WS
Process and
services layer
ESB
RMI
MQ
SOAP/XML
Legacy App
Database
File System
Custom API
Integration layer
Data / Applications
Typical Trading System Example
Database
RMI
CORBA App
Ordering
Swing App
1
App Server
EJB, WS,
POJO
ESB
Business
Process
App
A
9
Processing
Web App
Business
Process
App
B
Additional
Additional
Apps
Now
Additional
Apps
Now
& Future
Additional
Apps
Now
& Future
Apps Now
& Future
& Future
DB
Data is correlated
in here.
Collaborative: SOA Requires Agile
Business Analysts
outline a business
process and ‘not
implemented’ test
cases
Developers unit test and
jump-start QA
Complete Test Coverage
Lower Cost per Test
Faster Discovery & Resolution
Support rapidly debug issues and
communicates appropriately
QA expands dev tests and
create functional and load
tests
Production test their
implementation and report
issues to support
Collaborative Testing in an SOA environment
BPM Validation
Check Credit
Check Inv
Order Acquisition
Place Order
Orchestration Tests
Service 2
Place Order
Service 1
Service 3
QE
Service Validation
QA
Dev/QE
Service 3
Component Tests
Continuous SOA Test Platform
QA/BA
Continuous: SOA is never done
• Project-based development had a “test phase”
Database
Create
components
Unit test
Integration
Create UI
Acceptance
Test
Deploy
App
• SOA development is constantly evolving
Database
Service
retired
Create UI
Create UI
New service
Service
changed
Create UI
New service
Database
New service
Continuous: Minimize Unintended Consequences
Registry
MDM
?
Transactions
?
?
APP 1
?
ESB
APP 2
Database
?
Internal
Service
NEW
Legacy
App
?
Partner
Service
?
?
Continuous: Minimize Unintended Consequences
Registry
MDM
Transactions
Continuous
Testing
A data service is
currently failing
due to a change in
the internal
service
APP 1
ESB
APP 2
Database
Internal
Service
Legacy
App
Partner
Service
Continuous: SOA at 5-9’s
?????%
functional
integrity
Integrity
100%
SOA-aware Dev/QA Process
Comprehensive Testing Tools
Continuous Testing Service
Business
Applications
Scalability
100%
App Servers
Transaction Monitors
Clustered Services
99.999%
application
availability
Application
Infrastructure
Availability
100%
Fault Tolerant HW
Diesel Power
Redundant Bandwidth
99.999%
system
uptime
Physical
Infrastructure
Business Investment
Availability to the business is the product of all three
Continuous Levels: Build & Deploy
Production
Deployment
Level
Pre-production
Staging
test bed
QA/QE
Build
Level
Development
Component/
Service Teams
test bed
Dev build
product
component
Dev build
product
component
Continuous SOA Test Platform
Production
So how do we defeat the “Headless” dilemma?
Requirements are King -- On-line Music Scenario
• Use Case: (UC1) Customer with existing account does a search for
‘Rush’, selects the Grace Under Pressure tour DVD and uses the 1click purchase feature to buy.
• Business: (BR1) We need a flexible mechanism to change these
criteria at our whim. After discussion: for now partners get 20% of
net proceeds on their product sold on the first $100,000 / mo, 10%
thereafter.
• PSR: (PSR1) Search can’t take any longer than 3 seconds for a
user on a typical broadband connection.
• Policy: Punt for now. This is security, interop standards, corp dev
standards, etc.
• Functional: (FR1) Search must include results from CDs, DVDs,
Books, and Memorabilia, with CDs on the top (UC1).
• Technical: (TR1) Create a search service that aggregates CD, DVD,
Book, and Mem data (FR2). (TR2) Search service execution must be
under 2 seconds (PSR1).
Service Level Guidelines
• UC & BR: Validation mechanism
• PSR: Load/performance test at the service level
• FR: Likely 80% of your requirements can be nailed
right here
• TR: Dev test at the component level
Service Validation
Dev/QE
Service 3
Component Tests
UC=Use Case, BR=Business Requirements, FR=Functional Requirements, TR=Technical Requirements
Orchestration Level Testing Guidelines
• UC & BR: Not usually
• PSR: Often the best way to prove PSR
• FR: Invoke a behavior, validate outcomes
• TR: Simulate unstable/unavailable services
Orchestration Tests
Service 2
Place Order
Service 1
Service 3
QE
QA
UC=Use Case, BR=Business Requirements, FR=Functional Requirements, TR=Technical Requirements
Composite/BPM Level Testing Guidelines
• UC: Primary focus, performs a cause that we validate
at the service/component level
• BR: Many times same as UC, sometimes not relevant
• PSR: A few PSR scenarios proved at this level, but
continuous testing is the key here
• FR & TR: Hopefully very few
BPM Validation
Check Credit
Check Inv
Order Acquisition
QA/BA
Place Order
UC=Use Case, BR=Business Requirements, FR=Functional Requirements, TR=Technical Requirements
Typical SOA Test Map -- Dramatically Simplified
UC1: Search & Buy
UC1
Presentation
Layer
Content
Mgmnt
Commission
Rules Svc
Search
CRM
Facade
Content
CD
Books
Offers &
Specials
Order
Mgmnt
UC1
DVD
Mem
AP
Media Delivery
UC1
Batch
Feeds
Media Repo
CRM System
Billing
Typical SOA Test Map -- Dramatically Simplified
BR1: Commissions
Presentation
Layer
Content
Mgmnt
Commission
Rules Svc
Search
CRM
Facade
Content
CD
Books
Offers &
Specials
Order
Mgmnt
BR1*
DVD
Mem
AP
Media Delivery
Batch
Feeds
Media Repo
CRM System
Billing
BR1
Typical SOA Test Map -- Dramatically Simplified
PSR1: 3 sec search
PSR1
Presentation
Layer
Content
Mgmnt
Commission
Rules Svc
Search
TR2*
Content
CD
Books
CRM
Facade
Offers &
Specials
Order
Mgmnt
DVD
Mem
AP
Media Delivery
Batch
Feeds
Media Repo
CRM System
Billing
Typical SOA Test Map -- Dramatically Simplified
FR1: Search is aggregate,
CDs top results
Presentation
Layer
Content
Mgmnt
Commission
Rules Svc
Search
FR1
Content
CD
Books
CRM
Facade
Offers &
Specials
Order
Mgmnt
DVD
Mem
AP
Media Delivery
Batch
Feeds
Media Repo
CRM System
Billing
Typical SOA Test Map -- Dramatically Simplified
TR1: Create search on 4 repos
TR2: 2 sec search
Presentation
Layer
Content
Mgmnt
Commission
Rules Svc
Search
TR1 & 2
Content
CD
Books
CRM
Facade
Offers &
Specials
Order
Mgmnt
DVD
Mem
AP
Media Delivery
TR1
Batch
Feeds
Media Repo
CRM System
Billing
The Three Dimensions of Quality maturity
Complete >
single component  all technologies
Collaborative >
Dev/QA test silos  everyone tests, every phase
Continuous >
Phased tests  constant testing
LISA meets SOA Testing Challenges
• LISA is Complete - Test heterogeneous SOA
• LISA can test every tier of an application, regardless of location
• LISA can test every service technology with one tool, one test
• LISA’s framework approach creates bridges to legacy services
that means ‘you can get there from here’
• LISA is Collaborative – Everyone owns quality
• LISA shines at testing all components before UI even exists, as
well as integrating with the team’s ALM lifecycle and process
• Unit, regression, system, load, and monitoring in one tool
• Leverages diverse skill sets in Dev, QA/QE, and BA roles
• LISA is Continuous – Enables Agile SOA
• Tests constantly evolving, interdependent systems during and
after deployment
• Traps for the “what-if” unintended system-wide consequences of
making changes or corrections to components.
LISA WS-Testing $5,000 Off for next 50k users
• LISA WS-Testing Edition is THE tool for Web Services testing
• All our web services goodies are in there:
•
•
•
•
WSDL-based, point and click
WS-I Interop testing
Functional testing and service simulation
UI and Command Line execution
Thank you for your interest!
• iTKO Web Site Resources
•
•
•
•
URL: http://www.itko.com or http://www.ws-testing.com
White papers
Support FAQs, Tech Notes, and Forums (once registered)
Download WS-Testing or start a larger evaluation
• More from me
• See above
• Blog: http://itko.blogspot.com
• Email: Start with [email protected]; I’m [email protected]
?
?
?