Transcript tcVISION – DB2 CDC - Treehouse Software

A Global Leader in Data Migration,
Replication, and Integration Solutions
tcVISION: Enterprise Data Replication Without Compromise
1
Agenda
• tcVISION
• Overview
• Architecture
• Metadata & Modeling
• Replication
• tcVISION Demo
• Q&A
tcVISION: Enterprise Data Replication Without Compromise
2
Mainframe Data Replication and Integration
without Compromise
tcVISION: Enterprise Data Replication Without Compromise
3
Data Replication
The Problem:
•
•
•
•
•
The Solution:
• Moving/replicating data...
• as much as needed
• as little...
• as transparent...
• as flexible...
• as secure...
• ...AS POSSIBLE
Different data formats
Different data models
Large data volumes
Limited batch window
Requirement for up-todate information
...with
tcVISION: Enterprise Data Replication Without Compromise
4
tcVISION Overview
• Innovative software solution to process changed data
• Automatically identifies changes and files
• Continuously propagates the changed data to target
systems in real-time or time intervals
• Offers a rich set of processing and control mechanisms
• Contains powerful bulk processors for initial load of mass
data or the cyclic exchange of larger data volumes
• Efficient, fast and reliable
• Supports multiple datasources – Oracle, SQL Server, DB/2,
IMS/DB, VSAM, DL/I, ADABAS, IDMS, DATACOM, Flat File
tcVISION: Enterprise Data Replication Without Compromise
5
tcVISION Architecture
tcVISION: Enterprise Data Replication Without Compromise
6
Control Board
•
•
•
•
•
MS Windows GUI Interface
Single and central point of
administration
• View manager and
transaction logs
Built-in Wizards for mapping
and modeling metadata
Script Generation for bulk
transfer and replication
Monitoring
• Replication status
• Script output
• DBMS extensions
tcVISION: Enterprise Data Replication Without Compromise
7
Wizard-Assisted Metadata Import
Copy Books
• VSAM file layout
• IMS/DLI Segment layout
• PS/PDS/SAM file layout
IMS DBD, PSB
• Segment hierarchy
• Key information
Dictionaries
• ADABAS PREDICT
• CA-DATACOM
Schema definitions
• CA-IDMS
tcVISION: Enterprise Data Replication Without Compromise
8
Mapping & Modeling
• Metadata import supported for ADABAS,
DATACOM, DB2, Copybooks, IDMS, MySQL,
ODBC data sources, Oracle, SQL Server, and
tcACCESS data sources
• Automated generation of normalized RDBMS
schemata with explicit source column to target
column mapping
• Robust modeling and mapping via wizards –
normalize, denormalize, substring, concatenate,
filter, table lookup, user-exits, operators (+, -, *, /),
date/time
• All mapping information stored in a RDBMS-based
metadata repository
tcVISION: Enterprise Data Replication Without Compromise
9
Intuitive Mapping & Modeling
tcVISION: Enterprise Data Replication Without Compromise
10
Designed for Bi-Directional Replication
• Replicate to non-RDBMS databases’ native schema, no
schema changes required for target mainframe databases
(ADABAS, DATACOM, DB/2, IDMS, IMS & VSAM)
• Sample table replication scenarios:
• 1:1 Replication
• 1:n Replication with no dependencies
• 1:n Replication with dependencies (OCCURS and MU/PE)
• n:1 Replication with no dependencies
• n:1 Adjunction replication (source Parent & Child table(s)
• n:1 Combination replication
• n:n Replications
tcVISION: Enterprise Data Replication Without Compromise
11
Support for Complex Replication Scenarios
1:1
n:1
1:n
n:n
Also special cases:
• Data Warehouse Replication
• Journal Replication
tcVISION: Enterprise Data Replication Without Compromise
12
Wizard assisted column mapping
tcVISION: Enterprise Data Replication Without Compromise
13
Automatically apply target schema
tcVISION: Enterprise Data Replication Without Compromise
14
Multiple CDC Methods
Data Latency
Data Transfer Volume
tcVISION: Enterprise Data Replication Without Compromise
15
Multiple CDC Methods
•
•
•
•
•
•
•
•
•
•
Efficient transfer of entire
databases
Analysis for data consistency
(verification)
Initial load (ETL) and periodic
mass data transfer
One-step data transfer
IMS/DB, DL/I, VSAM, DB2,
Oracle, SQL Server, ADABAS,
CA-IDMS, CA-DATACOM,
sequential files, ODBC
Comparison of data snapshots
using checksums
Efficient transfer of changed data
since last processing
Flexible processing options (SORT
etc.)
Automatic creation of deltas by
tcVISION
IMS/DB, DL/I, VSAM, DB2, Oracle,
SQL Server, ADABAS, CA-IDMS,
CA-DATACOM, sequential files,
ODBC
tcVISION: Enterprise Data Replication Without Compromise
•
•
•
•
•
•
•
•
Transfer of changed data in
scheduled time frame
Generally batch-type
process
Generally run immediately
after archive log creation
IMS/DB, VSAM, DB2, DL/I,
Oracle, SQL Server,
ADABAS, IDMS, CADATACOM, CICS logs
Real-time capture of
changed data directly from
the DBMS
Secure data storage even
across DBMS restart
Flexible propagation
methods
CA-IDMS, IMS/DB, VSAM,
DB2, CA-DATACOM,
ADABAS, Oracle, SQL
Server
16
Staged Processing
Stage 3: Data in
DML or Loader
format
Exit points
available at
every stage
Stage 2:
Data normalized with
structure definition
Stage 1:
Data in tcVISION format
(before and after images)
Stage 0:
Data in internal raw
format
tcVISION: Enterprise Data Replication Without Compromise
17
tcVISION - Flexibility
•
•
•
•
User defines on WHICH platform
stage processing should be done
Normal implementation: Do as little as
possible on the mainframe
Mainframe: Capture data; Send data
(internal format) to target (Stage 0)
Receiving machine: Process data up to
Stage 3
tcVISION: Enterprise Data Replication Without Compromise
18
Cost Savings
• Use Staged Processing to reduce mainframe resource consumption
• Minimum processing on the mainframe – only what is essential
• Process all DBMS Journal and Log datasets on Open System
• Data Transformation is carried out on target platform – Open
System; e.g.,
• Data converted from mainframe data types to relational data
types
• Generation of SQL / SQL Loader files
• Date-time formatting
• EBCDIC-ASCII translation
tcVISION: Enterprise Data Replication Without Compromise
19
tcVISION – ADABAS CDC
• Real-time CDC – captures the Adabas CLOG using
Adabas UEX4 & UEX8. tcVISION uses a Data Collector
& Data Pool to collect & temporarily store the CLOG
records until they are ready to be processed
• Batch CDC – uses the Adabas PLOG
• Efficient – tcVISION reads the CLOG & PLOG and only
transfers transactions for Adabas files defined to
tcVISION. All other transactions are ignored.
• Using Staged Processing, transformation of CLOG &
PLOG data is performed on open systems, which saves
mainframe CPU & DASD
tcVISION: Enterprise Data Replication Without Compromise
20
tcVISION – Oracle CDC
• Uses ORACLE REDO Logs
• 'Single Instance Mode' supported for
10.1, 10.2 and 11.1
• Log processing may not be on the same
processor where the database is running
• may not be on the same archticeture
(Little/Big Endian machines)
• may not be on the same operating system
(Windows, UNIX, LINUX)
tcVISION: Enterprise Data Replication Without Compromise
35
tcVISION – Oracle CDC
SCN – System Change Number
• This is a sequential counter, identifying precisely a
moment in the database. This is the most accurate way
to identify a moment in time.
• tcVISION Control Board can be used to display current
SCN
• PROLOG script can be used query current SCN from
ORACLE and pass to tcSCRIPT
• Using RESTART FILES a "running SCN" is
automatically maintained
tcVISION: Enterprise Data Replication Without Compromise
36
Summary
tcVISION: Enterprise Data Replication Without Compromise
52