Transcript Slide 1
Packaged Test Automation for SAP Mar 20, 2008 Danis Yadegar (President & CEO Presenter www.arsin.com Agenda SAP Vs Custom Applications Testing needs of SAP Applications Summary Current Approach – Challenges & Limitations Solution Q&A CONFIDENTIAL © 2008 Arsin Corporation. All Rights Reserved 2 SAP Vs Custom-built Applications CONFIDENTIAL © 2008 2006 Arsin Corporation. All Rights Reserved 3 SAP Vs Custom Application Testing SAP Custom • Transaction Focused Testing • GUI/API Focused Testing • Configurable and Customizable Execution • Reusable GUI/API Execution Components • Screen based Edit Rules & Referential Integrity Components • Configurable and Customizable Business Rule checks • Limited and Controlled set of Test Data Validation • Complex and Large set of Test Data • Easy and Full Control over QA Environment, • Data in QA Environment is Hard to Refresh Easy to reset baseline test data CONFIDENTIAL © 2008 2006 Arsin Corporation. All Rights Reserved 4 Change Drivers SAP Within Company Support Packs Business Process Changes Sarbanes-Oxley Partner & Customer Integration New Modules Additions Mergers and Acquisitions FDA System Validation Upgrades Business Unit Changes Enhancements/Fixes Regulatory Compliance System Environment Consolidations Disaster Recovery Readiness Recovery From Crash Hardware and OS Changes Data Base System Changes International Regulatory Compliance Requirements Other Applications Packaged Applications Complimentary Software Custom Applications SAP New Dimension Products Integration Platforms 5 Summary Testing SAP requires a different set of tools, in addition to the commercially available GUI testing tools Testing SAP requires a new testing methodology and process Testing SAP requires people with different knowledge areas – business process, SAP configuration & development CONFIDENTIAL © 2008 2006 Arsin Corporation. All Rights Reserved 6 Current Approach - Limitations SAP Test Automation Manual and screen based validations Input and expected data is managed in spread sheets, which is difficult to maintain Managing changes – Inability to rapidly change test artifacts during changes in the systems 7 Test Automation Challenges - An Example Order To Cash Scenario Sales Order Display Sales Order Display Customer Display Condition Stock Overview Delivery Goods Issue Display Out Bound Delivery Display Material Document Stock Overview Balance Display Display Acct. Document Cust. Acct. Balance Display Billing Display Sales Order Display Acct. Document Balance Display Cust. Acct. Balance Display Incoming Payment Display Acct. Document Cust Acct. Balance Display Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Execution Validation Data Data Execution Validation Data Data Execution Validation Data Data Execution Validation Data Data Execution Data Validation Data Validation Data Validation Data Validation Data Validation Data Validation Data Validation Data Validation Data LEGEND Execution Validation Validation Data Validation Data Validation Data Validation Data CONFIDENTIAL © 2008 Arsin Corporation. All Rights Reserved 8 Problem One: Data Management Order To Cash Sales Order Delivery Goods Issue Billing Incoming Payment Sales Order Test Script Sales Order VA01 Display Sales Order VA03 Execution Display Customer XD03 Display Condition VK13 Stock Overview MMBE Validation Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Execution Validation Data Data Execution Validation Data Data Execution Validation Data Data Execution Validation Data Data Execution Data Validation Data Validation Data Validation Data Validation Data Validation Data Validation Data Validation Data Validation Data Validation Data Validation Data Validation Data Validation Data 9 Problem Two: Validation Sales Order Test Script Sales Order VA01 Display Sales Order VA03 Execution Display Customer XD03 Display Condition VK13 Stock Overview MMBE Validation 1:3 Ratio – Execution to Validation Screen Based Access - No Access to Internals High Degree of Test Automation Skill Required 3X as Hard to Maintain The Only Way to Reduce the Functionals’ Workload 10 Problem Three: Managing Change Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Execution Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Data Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Validation Validation Validation Validation Execution Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Data Data Data Data Data Validation Validation Validation Validation Execution Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Data Data Data Data Data Validation Validation Validation Validation Execution Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Data Data Data Data Data Validation Validation Validation Validation Execution Display Sales Display Display Stock Spreadsheet Goods Spreadsheet Spreadsheet Spreadsheet Spreadsheet Display Delivery Data Data Data Data Data Validation Validation Validation Validation Execution Sales Order Sales Customer Display Condition Display Overview Stock Issue Goods Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet Execution Order Sales Display Delivery Data Data Data Data Data Validation Validation Validation Validation Execution Order Sales Customer Display Condition Display Overview Stock Issue Goods Data Validation Validation Validation Validation Execution Order Sales Display Delivery Data Data Data Data Data Validation Validation Validation Validation Execution Order Sales Customer Display Condition Display Overview Stock Issue Goods Data Data Data Data Data Validation Validation Validation Validation Execution Order Sales Delivery Data Data Data Data Data OrderData Customer Condition Overview Issue Data Data Data Data Validation Validation Validation Execution OrderValidation Display Display Display Display Cust. Acct. Validation Data Data Data Data Data Validation Validation Validation Execution Sales Display Display Stock Goods Stock Balance Sales Display Out Display Bound Material Display Acct. Display Cust. Acct.Delivery Data Data Data Balance DataBilling Data Validation Validation Validation Validation Execution Order Sales Display Condition Display Overview Stock Issue Goods Overview Stock Customer Display Balance Order Sales Display Delivery Out Display Bound Document Material Display Acct. Display Display Balance Cust. Acct.Delivery Data Data Document Data DataBilling Data Validation Validation Validation Validation Execution Order Sales Display Condition Display Overview Stock Issue Goods Overview Stock Customer Display Balance Order Sales Data Delivery Out Display Bound Document Material Display Acct. DisplayData Display Balance Cust. Acct.Delivery Data Document DataBilling Data Order To Cash Scenario 11 Order To Cash Scenario Order OrderTo ToCash CashScenario Scenario11 Order To Cash Scenario 11 Order To Cash Scenario Order To Cash Scenario 1 Order Overview Issue Overview Stock Customer Display BalanceCondition Delivery Out Bound Document Material Order Document Acct. Display Balance Billing Overview Display Document Document DisplayDelivery Display Cust. Acct. DisplayDisplay Acct. Display Display Cust.Cust Acct. Balance Incoming Display Stock Balance Sales DisplayOut Display Acct. Display Balance Cust. Acct. Acct. Display Balance Cust Acct. Billing Bound Material Display Display Balance Cust. Acct. Display BalanceOverview Payment Incoming Acct. Stock Display Balance Order Sales Display Document Acct. Display Display Balance Cust. Acct. Document Acct. Display Display Balance Cust Acct. Billing Display Display Display Cust. Acct. Delivery Out Display Bound Document Material Document Acct. Balance Display BalanceOverview Payment Incoming Stock Display Balance Order Sales Display Document Acct. Display Display Balance Cust. Acct. Document Acct. Display Display Balance Cust Acct. Billing Out Bound Material Acct. Balance Delivery Document Document Display Display BalanceOverview Payment Incoming Order Sales Delivery Document Acct. Document Display Balance Display Document Acct. Display Balance Display Display PaymentDocument Order Document Display Document Display Display Display Cust. Acct. Display Cust Acct. Balance Incoming Sales Display Acct. Display Balance Cust. Acct. Acct. Display Balance Cust Acct. Display Balance Payment Incoming Order Sales Acct. Display Balance Document Acct. Display Balance Display Document Display Cust. Acct. Display Cust Acct. Display Payment Balance Incoming Order Document Display Document Display Sales Acct. Balance Acct. Balance Display Payment Order Document Display Document Display New billing field 11 Current Approach - Limitations Knowledge Management SAP testing is spearheaded by functional experts who map the Business Process to SAP modules During the initial implementations there is more focus on testing than on capturing knowledge residing with functional & technical consultants As a result, the regression library is incomplete for future support packs and upgrades Methodology & Process Formal Testing is addressed as one of the last activities after completion of development and unit testing ASAP methodology guidelines for testing are very high level and does not focus on detailed activities to be performed by the QA team during various phases implementation As a Result testing gets pushed till the end and quality is compromised due to lack of early engagement and preparation 12 Solution Technology Methodology & Process Knowledge Management CONFIDENTIAL © 2008 Arsin Corporation. All Rights Reserved 13 Technology Solution Flexible multi-platform GUI test automation tool – RFT, QTP, TP Scoping Workbench Test Automation Framework for SAP Landscape Configurable and Customizable Test Components Built-in Data and Business Rule Validations for: Standard and Custom Transactions Custom Processing Programs Inbound/Outbound Interfaces Sophisticated Test Data Management Pre and Post Change Comparators for Documents and Tables CONFIDENTIAL © 2008 Arsin Corporation. All Rights Reserved 14 Test Automation Process Flow QA Mapper™ RFT Log 7 Test Requirements Test Cases Test Procedures Meta Data of ECs Definition of VC Data Sets System Under Test 4 Runs EC Extracts Keys Pass it to Effecta for Validation GUI Automation Tool 5 SAP GUI Core Business Processes SAP ERP Database Effecta™ Validation Engine 2 I n t e r f a c e s 6 1 Wrapper Extracts Test Case Info EC Meta Data VC Name Data Set Info From QA Mapper DB Wrapper on the Shared Drive Validation Reports 3 EC Scripts on Shared Drive 15 Test Automation Solution for Interfaces QA Mapper/GUI Tool CONFIDENTIAL © 2008 Arsin Corporation. All Rights Reserved 16 Methodology & Process Needs Compatible with SAP implementation Lifecycle Process to Test Configuration & Modifications Conversions Interfaces Connectivity Security Archiving Production Fixes and Enhancements Support Packs & Upgrades Include Processes to handle compliance requirements CONFIDENTIAL © 2008 Arsin Corporation. All Rights Reserved 17 SAP QA Process Page: 1 / 1 Revision date: May 16, 2007 File: VisioDocument QA Activities - SDLC Phase Mapping Define Design Build / Test Prod. Readiness Delivery Warr. Support QA Environment Readiness Review Conversions and Master Data Setup Complete detailed sizing QA Resource Planning Inputs: High Level QA Test Plan Outputs: QA/Partner Resources Requirements/ Assignments QA lead completes phase checlist Outputs: High Level QA Test Plan Develop detailed test plan, test requirements, test cases, test procedures Prioritize QA test requirements -business SME & designer sign-off Data Mining Connectivity Testing Identify / Create baseline data QA lead completes phase checlist Outputs: High Level Resource and Time Estimates Test Execution Knowledge Transfer Review High Level Sizing Inputs: Project Charter SCOPE Document Project Plan Initial Requirements Review Documents Technical Specs Development Test Plan Change Records/ Development list Test Post Go Live Changes Turnover to QA Production Support QA Metric – Open Defects Captured QA PreCertification Meeting QA Certification Document/ Risk assessment document -business sign-off -project team review Quality Metrics Defect Detection Effectiveness Test Emergency Requests Post Prod Readiness Audit QA lead completes phase checlist QA Test Planning Inputs: “To Be” System Design “To Be” Business Process Flows and BPPs Functional Specs QA lead completes phase checlist QA Early Engagement Assign QA resource for Early Engagement Review BR, High Level Sizing, Timeline QA Kick-off meeting Root Cause Analysis Process Metrics Schedule Variance Effort Variance Cost Variance Final Audit Lessons Learned CMs and change records approved Conduct Survey Test Automation Requirements Project Plan Pre-Production Readiness Audit Regression Library Updates PM or Designee to review Test Automation Updates CONFIDENTIAL © 2008 Arsin Corporation. All Rights Reserved 18 SAP QA Knowledge Management QA Process Retains Knowledge through Detailed Documentation of regression library including validation rules Automated Execution Automated Validation of Business Rules Training Program SAP Professionals on QA Processes Functional & Technical Training for Non-ERP IT Professionals Test Automation Frameworks CONFIDENTIAL © 2008 Arsin Corporation. All Rights Reserved 19 Ideal Testing Landscape QA Analyst Basis Administrator ABAP Developer Application X Developer SAP Functional Analyst Business Process Analyst Compliance Auditor Test Management TEST REQUIREMENTS TEST CASES Test Component Library Business Process Test Library Interfaces FI SAP MM SD EDI PP PS FS Batch SRM CRM APO EAI Non-SAP Applications Client Server AS400 Web Other Sell-Side Processes EXECUTE Manufacturing Processes Buy Side Processes PROVIDE BUILDING BLOCKS Financial Processes BUILD MAINTAIN BUILD MAINTAIN QA Automation Engineer TEST EXECUTION & METRICS & DEFECTS TEST PROCEDURE EffectaTM SAP Validation Engine QA Analyst SME GUI Test Automation Tool 20 IBM-Arsin’s Test Automation Solution QA Mapper™ System Under Test Web Browser Test Req SAP GUI Test Cases Test Procedures Execution Engine Test Data Test Scripts Web Database Rational Functional Tester Core Business Processes SAP ERP Database Effecta™ Validation Engine I n t e r f a c e s 21 For White Paper Contact: John Hudson [email protected] 408-532-5579 www.arsin.com 22 Q&A CONFIDENTIAL © 2006 Arsin Corporation. All Rights Reserved 23