Transcript Folie 1 - UBS Hainer GmbH

UBS HAINER
TEST DATA MANAGEMENT
TEST DATA MANAGEMENT
CONCEPTS & METHODS FOR DB2
1 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
Agenda
2 | 54
• Test data – Where and how it's used
03 - 04
• Scheduled Cloning
05 - 13
• Ideal Test Environments
14 - 18
• Providing Bulk Data – Fast & Efficient
19 - 21
• Consistency, Masking and Subsets
22 - 34
• Using fast Copiers
36 - 37
• Helpful Features
38 - 40
• Developer's Tool
42 - 45
• Where UBS Hainer can help
47 - 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
TEST DATA – WHERE AND HOW IT'S USED
Test Data Agreement (SLA)
Service Level
Production
Test Areas
• Release
• Regression
• Acceptance • Perfomance
Programs/
Structures
3 | 54
Test Data
© UBS Hainer GmbH, the IBM Business Partner
Development
• Unit
• Segment
Programs/
Structures
UBS HAINER
TEST DATA MANAGEMENT
TEST DATA – WHERE AND HOW IT'S USED
Decoupling: Data Security & Consistency
Production
Preproduction
DB2, IMS, etc.
Copies
DB2, IMS, etc.
U/L
Copy
Clone
4 | 54
© UBS Hainer GmbH, the IBM Business Partner
Test Environments
UBS HAINER
TEST DATA MANAGEMENT
SCHEDULED CLONING
Preproduction Clone for DB2, IMS, etc.
Cloning
5 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
SCHEDULED CLONING
How does a clone become consistent?
POINT
NIGHT
SHIFT
IN
TIME
Time
6 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
SCHEDULED CLONING
How does a clone become consistent?
FlashCopy1
SnapShot
EMC TimeFinder
IBM PPRC
HDS ShadowImage
Softek TDMF
Fujitsu Equivalent Copy
Time
7 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
SCHEDULED CLONING
Transactions
Time
8 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
SCHEDULED CLONING
Prepare & (Re)Start
Current Status Rebuild
Time
Point of
Flash/Split/Snap
9 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
SCHEDULED CLONING
Implementation of a Cloning Process
• Contingency Plan
• Data Warehouse
• System Level Backup
• Flashcopy (1)
Production
DB2, IMS, etc.
10 | 54
© UBS Hainer GmbH, the IBM Business Partner
Clone
Preproduction
DB2, IMS, etc.
UBS HAINER
TEST DATA MANAGEMENT
SCHEDULED CLONING
Preproduction Renewal by Cloning
•Goal
• Automated, Scheduler-driven
• Minimal / No impact on Production
• Consistency, Data Integrity, Operational Readiness
Achievements
• Security: Decoupling
• Operational Flexibility
• Robust Procedure, Clear Agreements
11 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
SCHEDULED CLONING
Justifiable Expenditure
• 2500/3000 €/$ per Terabyte/Month
• Clones require no Maintenance
12 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
SCHEDULED CLONING
What is a Preproduction used for?
• Data Corpus for Test Data Provision
Preproduction
Clone
• Test of Release Propagation
• Acceptance Test
• DBMS Upgrades
• General Test environment?
13 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
IDEAL TEST ENVIRONMENTS
Test Environments
• Integration, Regression,
Performance Tests
> Bulk Data
Modified
Masked
14 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
IDEAL TEST ENVIRONMENTS
Test Environments
• Integration, Regression,
Performance Tests
> Bulk Data
• Unit Tests
> Some Parts, still Bulk Data
Modified
Masked
15 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
IDEAL TEST ENVIRONMENTS
Test Environments
• Integration, Regression,
Performance Tests
> Bulk Data
• Unit Tests
> Some Parts, still Bulk Data
• Developer Tests
> Some rows that fit together
Modified
Masked
16 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
IDEAL TEST ENVIRONMENTS
Test Environments
• Integration, Regression,
Performance Tests
> Bulk Data
• Unit Tests
> Some Parts, still Bulk Data
Test Case Data
• Developer Tests
> Some rows that fit together
17 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
IDEAL TEST ENVIRONMENTS
Test Environments
• Integration, Regression,
Performance Tests
> Bulk Data
Efficiency, low cost, scheduler driven
integrated function, fast, complete,
ready to use results
• Unit Tests
> Some Parts, still Bulk Data
• Developer Tests
> Some rows that fit together
18 | 54
© UBS Hainer GmbH, the IBM Business Partner
Operator convenience, flexibility,
great functionality, cross-platform
UBS HAINER
TEST DATA MANAGEMENT
PROVIDING BULK DATA – FAST & EFFICIENT
Providing Bulk Data - Fast & Efficient
• DSN1COPY
Ten times faster than Unload/Load
One tenth of CPU
• Further acceleration through • • •
• Parallel Processing
• Feasible:
1TB/hour, 300-500 MB/sec*
*) without hardware-assisted copy functions like Flashcopy or the like
19 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
PROVIDING BULK DATA – FAST & EFFICIENT
• DDL, JCL
• Indices
• Runstats, Views, Trigger, Grants, Synonyms
• Adaptation, Conversion
• Masking
• Reduction
Wish List
DB2P
CATALOG
DB2T
CATALOG
COMPARE, CREATE, MAPPING, Runstats, Views, Trigger, Synon
IX
IX
TS
TS
TS
COPY
TS
Indexes, Adaptation, Storage, Conversion,, Masking, Reduction
PROD
DB2 SUBSYS
20 | 54
© UBS Hainer GmbH, the IBM Business Partner
ANY TEST ENVIRONMENT
DB2 SUBSYS
UBS HAINER
TEST DATA MANAGEMENT
PROVIDING BULK DATA – FAST & EFFICIENT
Relief through Automation
Definition of Tasks
+ Approval
• Define a copy task
• Select, Rename
• Reduction / Subsets
• Task Options
Separation of Tasks
• Masking
Separation - functional Responsibility
• Efficient execution
• Scheduler-driven
Task Execution
21 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
CONSISTENCY, MASKING AND SUBSETS
Change of Table Content
Reduction
Masking
23 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
CONSISTENCY, MASKING AND SUBSETS
UNLOAD & LOAD
UNLOAD
LOAD
PROD
ANY TEST ENVIRONMENT
DB2 SUBSYS
DB2 SUBSYS
Masking Program
25 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
CONSISTENCY, MASKING AND SUBSETS
Integrated Copy + Masking + Select Subsets
DB2P
CATALOG
COMPARE, CREATE
DB2T
CATALOG
IX
IX
TS
TS
TS
TS
COPY
MASKING
REDUCTION
PROD
DB2 SUBSYS
26 | 54
© UBS Hainer GmbH, the IBM Business Partner
ANY TEST ENVIRONMENT
DB2 SUBSYS
UBS HAINER
TEST DATA MANAGEMENT
CONSISTENCY, MASKING AND SUBSETS
Integrated Copy + Consistency
DB2P
CATALOG
COMPARE, CREATE
DB2T
CATALOG
IX
IX
TS
TS
TS
COPY
TS
LOG APPLY
PROD
DB2 SUBSYS
27 | 54
© UBS Hainer GmbH, the IBM Business Partner
ANY TEST ENVIRONMENT
DB2 SUBSYS
UBS HAINER
28 | 54
TEST DATA MANAGEMENT
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
29 | 54
TEST DATA MANAGEMENT
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
30 | 54
TEST DATA MANAGEMENT
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
31 | 54
TEST DATA MANAGEMENT
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
32 | 54
TEST DATA MANAGEMENT
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
33 | 54
TEST DATA MANAGEMENT
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
CONSISTENCY, MASKING AND SUBSETS
Integrated Copy + Consistency + Masking
DB2P
CATALOG
DB2A
CATALOG
IX
CATALOG
IX
TS
TS
DB2X
IX
TS
TS
COPY
TS
COPY
TS
MASKING
REDUCTION
LOG APPLY
PROD
PROD
34 | 54
REPOSITORY
PRE-PROD, ACCEPTANCE
© UBS Hainer GmbH, the IBM Business Partner
ANY TEST ENVIRONMENT
TEST
UBS HAINER
TEST DATA MANAGEMENT
GENERATIONS OF TEST DATA
Freeze & Defreeze
DB2P
CATALOG
SHIFT
SHIFT
META-DATA
IX
CATALOG
IX
TS
TS
DB2X
IX
TS
TS
COPY
TS
COPY
TS
LOG APPLY
PROD
STORAGE REPOSITORY
DB2 SUBSYS
35 | 54
© UBS Hainer GmbH, the IBM Business Partner
(No SUBSYS / INSTANCE)
ANY TEST ENVIRONMENT
DB2 SUBSYS
UBS HAINER
TEST DATA MANAGEMENT
USING FAST COPIERS
What about Flashcopy and the like?
• Deployment is limited to rare cases
• OBIDs
• Log RBA
• Version
• SEGSIZE, PGSIZE, ENCODING_SCHEME, DSSIZE
• & PARTITIONS
• Structures non-equal
• XML
• Data sharing – non DS
36 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
USING FAST COPIERS
How Flashcopy fits
DB2P
CATALOG
SHIFT
SHIFT
META-DATA
IX
CATALOG
IX
TS
TS
DB2X
IX
TS
FLASHCOPY
TS
TS
COPY
TS
LOG APPLY
PROD
DB2 SUBSYS
37 | 54
© UBS Hainer GmbH, the IBM Business Partner
REPOSITORY
(No SUBSYS / INSTANCE)
ANY TEST ENVIRONMENT
DB2 SUBSYS
UBS HAINER
TEST DATA MANAGEMENT
HELPFUL FEATURES
Control Language / BI
CREATE COPY TASK "PRCLONE"
-- Source and target specification
FROM "DB8G" ON "PROD"
TO "DB8G" ON "DEV"
-- The processing options of the task
COPY_STATISTICS "N"
DROP_OPTION "N"
-- The selection rules of the task
SELECTION (DATABASE "DSN8%")
-- Rename the database name
MAPPING (OBJECT ("D")
FIELD ("N")
TARGET "HUGO%")
-- Rename the creator of all objects
MAPPING (OBJECT ("%")
FIELD ("C")
TARGET "HUGO")
38 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
HELPFUL FEATURES
Control Language / BI
CREATE COPY TASK "PRCLONE"
-- Source and target specification
FROM "DB8G" ON "PROD"
TO "DB8G" ON "DEV"
-- The processing options of the task
COPY_STATISTICS "N"
DROP_OPTION "N"
-- The selection rules of the task
SELECTION (DATABASE "DSN8%")
-- Rename the database name
MAPPING (OBJECT ("D")
FIELD ("N")
TARGET "HUGO%")
-- Rename the creator of all objects
MAPPING (OBJECT ("%")
FIELD ("C")
TARGET "HUGO")
39 | 54
© UBS Hainer GmbH, the IBM Business Partner
• In Batch
• From Workstation
• By Application
• Mass Change
UBS HAINER
TEST DATA MANAGEMENT
HELPFUL FEATURES
Access difficult? TCP/IP is powerful
TCP/IP
Plex A
Group A
40 | 54
© UBS Hainer GmbH, the IBM Business Partner
Plex B
Group B
Plex C
Group C
UBS HAINER
TEST DATA MANAGEMENT
BULK DATA SUMMARY
Benefits of an apt approach?
• Efficient copy method saves 90% CPU
• and shortens provision time
• Automatic handling of: • • • • • • • • • • •
• JCL, DDL, Mapping• • • • • • • • • • • • • •
• Structure Adaption• • • • • • • • • • • • • •
• Integrity Check• • • • • • • • • • • • • • • • •
• Freeze & Restore• • • • • • • • • • • • • • •
• Command Interface
• Saves man-days
41 | 54
© UBS Hainer GmbH, the IBM Business Partner
Development
Unit
Integration
Test
Environments
UBS HAINER
TEST DATA MANAGEMENT
DEVELOPER'S TOOL
Providing "Test Cases"
• Select / fetch data
• Add / alter / delete columns
Development
• Edit rows
• Add / delete rows
Unit
• Save it / reproduce it
Integration
Test
Environments
42 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
DEVELOPER'S TOOL
43 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
DEVELOPER'S TOOL
44 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
DEVELOPER'S TOOL
Demands on a Workstation
• Import / interface to …
• Graphic representation
Development
• Reduce / edit
• Add / modify
Unit
• Support impact analysis
Integration
Test
Environments
45 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
CUSTOMER INSIGHT
46 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
WHERE UBS HAINER CAN HELP
huge manual effort
time consuming
high CPU consumption
Where to start?
costly
detect bugs
no copy opportunities
copy window too short
A sustainable concept doesn't have to be realized in a day.
Implementations can vary and be executed step-by-step.
47 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
WHERE UBS HAINER CAN HELP
What can UBS Hainer do
for your company?
48 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
WHERE UBS HAINER CAN HELP
BCV4™
Full DB2
Subsystem
Clones in minutes versus days
The
BCVn
Product
Suite
BCV5™ Save 90% CPU & Elapsed time with each DB2 copy
BCV6™ Log enhanced copies for 24x7 DB2 refresh/migration
BPA4DB2™ Premier advisor for DB2 buffer pool optimization
TUC™ RTS & Policy driven automation of DB2 data maintenance
XM4DB2™ Pro-active surveillance for a greater DB2 availability
ULT4DB2™ Easily identify & restore unwanted changes of DB2 data
49 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
WHERE UBS HAINER CAN HELP
BCV4
• Quickly clones DB2 & IMS systems in an• • • • •
• automated process
• Allows homogeneous system copies in a • • • •
• non-stop 24x7 environment
• Creates environments for preproduction • • • • •
• and release changes
50 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
WHERE UBS HAINER CAN HELP
BCV5 & Components
• Saves 90% of CPU usage, and is ten • • • •
• times faster than common
• Provides consistent test data within • • • • • •
• regular batch window
• BCV6: Consitent copies at anytime for • • •
• 24x7 shops
• Icebox: Archive data & populate different• •
• environments
• Language Interface
51 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
WHERE UBS HAINER CAN HELP
You are keen to see what you
can accomplish with BCVn
in your installation?
52 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
WHERE UBS HAINER CAN HELP
Schedule your private
company presentation.
We would be pleased to discuss
your specific interests,
Your UBS Hainer Team
53 | 54
© UBS Hainer GmbH, the IBM Business Partner
UBS HAINER
TEST DATA MANAGEMENT
WHERE UBS HAINER CAN HELP
Thank you for your attention!
54 | 54
© UBS Hainer GmbH, the IBM Business Partner