PPRC Overview
Download
Report
Transcript PPRC Overview
IBM Systems & Technology Group
Global Copy, Metro Mirror, Global Mirror, and
Metro/Global Mirror Overview
Charlie Burger
Storage Systems Advanced Technical Support
© 2004 IBM Corporation
Advanced Technical Support, Americas
Table of Contents
Background
– Consistency and dependent writes
Peer-to-Peer Remote Copy - PPRC
PPRC Considerations
Establish Path Considerations
FCP vs ESCON Links
Metro Mirror
Global Copy
Global Mirror
Metro/Global Mirror
Failover
Failback
Addendum
2
© 2005 IBM Corporation
Advanced Technical Support, Americas
Background
Dependent writes
– The start of one write operation is dependent upon the completion of a
previous write to a disk in either the same subsystem frame or a different
subsystem frame
– Basis for providing consistent data for copy operations
Consistency
– Preserves the order of dependent writes
• For databases, consistent data provides the capability to perform a data
base restart rather than a data base recovery
– Restart can be measured in minutes while recovery could be hours or
even days
Asynchronous processing
– The separation of data transmission from the signaling of I/O complete
• Distance between primary and secondary has little impact upon the
response time of the primary volume
• Helps minimize impact to application performance
3
© 2005 IBM Corporation
Advanced Technical Support, Americas
Database RESTART or RECOVER?
Operating
System
Applications
Other
UNIX/NT
AS/400's
RS/6000
S/390
Operations Staff
Other
UNIX/NT
AS/400's
Network Staff
RS/6000
Data
Applications Staff
Management Control
Telecom Network
Physical Facilities
Consistency provides:
"DB Restart" - To start a DataBase application following an outage without having to
restore the database. This is a process measured in minutes.
and avoid: "DB Recover" - Restore last set of DataBase Image Copy tapes and apply
log changes to bring database up to point of failure. This is a process measured in
hours or even days
4
© 2005 IBM Corporation
Advanced Technical Support, Americas
Dependent Writes
1. Log Update
OK
B
C
OK
L
M
X
Y
B
C
L
M
DataBase
Application
OK
3. Mark Log Complete
2. Update database
X
Y
Many examples where start of one write operation is time
dependent on the completion of a previous write on a different
disk group or even different disk frame
Data base & log for example
Synchronous copy insures data integrity
5
© 2005 IBM Corporation
Advanced Technical Support, Americas
Dependent Write Consistency (1)
1. Log Update
OK
DataBase
Application
B
C
L
M
X
Y
B
C
L
X
X
M
Y
Scenario: 'Update Database' doesn't get propagated to 2nd site
Synchronous copy and 'dependant writes' means 'Mark Log Complete' will
never be issued by application
Result:
Database is consistent
6
© 2005 IBM Corporation
Advanced Technical Support, Americas
Dependent Write Consistency (2)
1. Log Update
OK
L
DataBase
Application
OK
C
B
X
M
X
Y
B
C
L
M
X
Y
2. Update database
Scenario: 'Mark Log Complete' doesn't get propagated to 2nd site
Result:
Secondary site logs say update was not completed
Backout of valid data will be done upon restart at secondary site
But database is consistent
7
© 2005 IBM Corporation
Advanced Technical Support, Americas
Consistency
To achieve consistency at a remote mirror location you must
maintain the order of dependent writes
– You cannot have a write to one volume mirrored and then have a
dependent write to another volume not mirrored
The remote mirror functions each maintain consistency in
their own way
– Metro Mirror uses ELB (Extended Long Busy) for CKD volumes and
I/O Queue Full for FB LUNs
– Global Mirror holds write I/Os while building an alternate bit map prior
before draining the OOS (out of sync) bit map when creating a
consistency group
– z/OS Global Mirror (XRC) uses timestamps to create consistency
groups
– Global Copy requires procedures to create consistency
8
© 2005 IBM Corporation
Advanced Technical Support, Americas
Peer-to-Peer Remote Copy - PPRC
Metro Mirror – Synchronous PPRC
– Synchronous mirroring with consistency at the remote site
• RPO of 0
Global Copy – PPRC Extended Distance (XD)
– Asynchronous mirroring without consistency at the remote site
• Consistency manually created by user
– RPO determined by how often user is willing to create consistent data at the remote
Global Mirror
– Asynchronous mirroring with consistency at the remote site
• RPO can be somewhere between 3-5 seconds
Metro/Global Mirror
– Three site mirroring solution using Metro Mirror between site 1 and site 2 and Global
Mirror between site 2 and site 3
• Consistency maintained at sites 2 and 3
– RPO at site 2 near 0
– RPO at site 3 near 0 if site 1 is lost
– RPO at site 3 somewhere between 3-5 seconds if site 2 is lost
9
© 2005 IBM Corporation
Advanced Technical Support, Americas
PPRC Considerations
PPRC secondary volume must be off-line to all attached systems
One-to-one PPRC volume relationship
Only IBM (DS8000/DS6000/ESS) to IBM is supported
•Logical paths have to be established between Logical Subsystems
•FCP links are bidirectional
•FCP links can also be used for server data transfer
•Up to 8 links per LSS
More than 8 links per physical subsystem is allowed
•Up to 4 LSS secondaries can be connected to a primary LSS
A secondary LSS can be connected to as many primary LSS systems
as links are available
Distance
ESCON links - 103 KM
FCP links - 300 KM
10
© 2005 IBM Corporation
Advanced Technical Support, Americas
Establish Path Considerations
If paths have been established, issuing another path establish will
overlay the existing established path
For example:
– 2 Paths are established using this command
mkpprcpath -dev IBM.2107-75FA120 -remotedev IBM.2107-75FA150 -srclss 01 -tgtlss 01
–remotewwnn 12341234000A000F I1A10:I2A20 I1A11:I2A21
– I wish to add another path so this command is issued
mkpprcpath -dev IBM.2107-75FA120 -remotedev IBM.2107-75FA150 -srclss 01 -tgtlss 01
–remotewwnn 12341234000A000F I0100:I0100
– The result will be the loss of the 2 previously established paths and only having
the new path established
– To add the path, the following should be issued
mkpprcpath -dev IBM.2107-75FA120 -remotedev IBM.2107-75FA150 -srclss 01 -tgtlss 01
–remotewwnn 12341234000A000F I0100:I0100 I1A10:I2A20 I1A11:I2A21
11
© 2005 IBM Corporation
Advanced Technical Support, Americas
FCP vs ESCON
•ESCON links run at 17. MB/sec with a sustained rate of approximately 12-14 MB/sec
•Fibre links can run 100-400 MB/sec depending upon the adapter with a sustained rate of approximately 80-320 MB/sec
•DS8000 and DS6000 only support PPRC fcp links
•ESS 800 supports both PPRC ESCON and fcp links
12
© 2005 IBM Corporation
Advanced Technical Support, Americas
What is Metro Mirror?
Disaster protection for all IBM supported platforms
Other potential uses:
–Data migration/movement between devices
–Data workload migration to alternate site
Hardware and LIC solution
Synchronous copy, mirroring (RAID 1) to another
DS8000/DS6000/ESS
Application independent
Some performance impact on application I/Os
Established at a disk level
A 2 site solution
13
© 2005 IBM Corporation
Advanced Technical Support, Americas
Profile of a PPRC Synchronous Write
Synchronous write
1 Write (channel end)
2 Write to secondary
3 Write acknowledged by secondary
4 Acknowledgement (Device end -- I/O
complete
1.
4.
3.
2.
14
© 2005 IBM Corporation
Advanced Technical Support, Americas
Maintaining Consistency – Metro Mirror
Consistency group on the establish path commands
– Loss of communication between primary and secondary sites will
cause an extended long busy (ELB) for zSeries and I/O queue full for
open systems to be returned to any write issued to a volume in the
LSS that lost communication
Automation can issue FREEZE commands to all of the LSSs
that have dependent data with the LSS that lost
communication
– Returning ELB or I/O queue full causes the next dependent write to
NOT be issued maintaining the order of the dependent writes
– After all of the FREEZE commands have been issued, RUN
commands can be issued to the LSSs to resume, otherwise the ELB
or I/O queue full will be returned for a default of 2 minutes
Automation is required when dependent data spans across
multiple physical disk subsystems and is HIGHLY
recommended when all primaries are within a single
subsystem
15
© 2005 IBM Corporation
Advanced Technical Support, Americas
Consistency – Metro Mirror One Physical Subsystem
No automation
1
2
3
4
16
x
All paths lost means that
no updates are xmitted to
secondaries and consistency
is maintained
© 2005 IBM Corporation
Advanced Technical Support, Americas
Consistency – Metro Mirror One Physical Subsystem
No automation
1
x
2
3
4
One pair suspends,
others still mirror,
lose consistency
17
© 2005 IBM Corporation
Advanced Technical Support, Americas
Consistency – Metro Mirror Multiple Physical Subsystems
No Automation
1
2
x
3
4
Without automation,
order of dependent writes not
maintained
and consistency is lost
18
© 2005 IBM Corporation
Advanced Technical Support, Americas
Consistency – Metro Mirror Multiple Physical Subsystems
1
Automation
2
3
x
4
Automation insures
that the order of dependent
writes is maintained
19
© 2005 IBM Corporation
Advanced Technical Support, Americas
When to use Metro Mirror
Recovery system required to be current with the primary application
system
Can accept some performance impact to application write I/O
operations at the primary location
Recovery is on a disk-by-disk basis
Distance within maximum limits
103 KM for ESCON links and 300 KM for fcp links
RPQ for greater disances
20
© 2005 IBM Corporation
Advanced Technical Support, Americas
Metro Mirror Sequential Write Data Rate – Turbo R2
21
© 2005 IBM Corporation
Advanced Technical Support, Americas
Pre-Turbo DS8000 Metro Mirror 4 KB Write Service
Time Comparisons
22
© 2005 IBM Corporation
Advanced Technical Support, Americas
Pre-Turbo DS8100 Metro Mirror Sequential Write
Throughput
23
© 2005 IBM Corporation
Advanced Technical Support, Americas
Pre-Turbo DS8100 Metro Mirror Sequential Write
Throughput
24
© 2005 IBM Corporation
Advanced Technical Support, Americas
What is Global Copy?
Global Copy uses an additional PPRC mode designed for high
performance data copy at long distances
– TSP OPTION(XD) Extended Distance or ds cli –type gcp
– Disk level option
Asynchronous transfer of application primary writes to secondary
allows mirroring over long distances with minimal impact to host
performance
– Writes to primary disk receive immediate completion status while in XD mode
Writes can be out of sequence on secondary disk
– Develop procedures to create a point in time consistency
A 2 site solution
25
© 2005 IBM Corporation
Advanced Technical Support, Americas
Profile of an Asynchronous Write
Asynchronous write
Synchronous write
1. Write
Write (channel end)
Write to secondary
2. Write acknowledgement
(channel end / device end)
Write acknowledged by secondary
Acknowledgement (Device end -- I/O
complete
1.
1.
2.
4.
3. Write to secondary
3.
4. Write acknowledged by
secondary
2.
4.
3.
26
© 2005 IBM Corporation
Advanced Technical Support, Americas
Global Copy – How it works (1)
Synchronous PPRC establish (initial copy) is done in two phases:
– Phase I - Copy all tracks in the volume starting at zero and going to the end of
the volume. Use a bitmap to keep track of which tracks need to be copied. Do
not transfer any host updates - just set the bit in the bitmap for new host
updates.
– Phase II - Go back through the bitmap to copy any host updates received
while in Phase I. Any host updates received during this phase, and for the
remainder of the PPRC pair life, will be sent synchronously to the remote
volume.
Extended Distance PPRC
– Stay in Phase I forever
– No impact to host write response time
– Copy at remote site is "fuzzy" - updates are not sent in order or in time
consistent groups
27
© 2005 IBM Corporation
Advanced Technical Support, Americas
Global Copy – How it works (2)
Establish PPRC pairs with Extended Distance option
– Writes to primary receive immediate completion status
Primary records updated tracks in a bitmap
Incremental copy of changed tracks or records periodically
sent to secondary
To create a point in time consistency:
– Transition to PPRC synchronous until full duplex state is reached
• Usually a matter of seconds
– Alternatively, quiesce of I/O and flushing of buffers on primary host will
result in consistent secondary disk
28
© 2005 IBM Corporation
Advanced Technical Support, Americas
Global Copy – How it works (4)
Agents process a
volume using the
Out-of-Sync (OOS)
bit map to determine
which tracks to xmit
Primary
Secondary
Primary
Secondary
Primary
Secondary
Primary
Secondary
Not all volumes are
processed at the
same time
As the volume is
processed, tracks
updated behind the
active track being
xmitted is recorded
in the OOS and will
be processed the
next time
29
© 2005 IBM Corporation
Advanced Technical Support, Americas
PPRC State Changes
Transition to simplex means PPRC is withdrawn
XD is established at the volume/LUN level
Simplex
Start
Syncronous
Extended
Distance
Pending
Duplex
Fuzzy
Suspended
30
© 2005 IBM Corporation
Advanced Technical Support, Americas
Volume State Transitioning
To transition from…..
To…..
Use the following command…..
SYNC
SIMPLEX
CDELPAIR
SIMPLEX
SYNC
CESTPAIR OPTION(SYNC)
SYNC
SUSP
CSUSPEND
SUSP
SYNC
CESTPAIR MODE(RESYNC)
XD
SUSP
CSUSPEND
SUSP
XD
CESTPAIR MODE(RESYNC) OPTION(XD)
XD
SIMPLEX
CDELPAIR
SIMPLEX
XD
CESTPAIR OPTION(XD)
SUSP
SIMPLEX
CDELPAIR
XD
SYNC
CESTPAIR OPTION(SYNC)
31
© 2005 IBM Corporation
Advanced Technical Support, Americas
Maintaining Consistency – Global Copy
Consistency group is NOT specified on the establish path
command
– Data on Global Copy secondaries is not consistent so there is no need
to maintain the order of dependent writes
Consistent data is created by the user
– Quiesce I/O
– Suspend the pairs
• FREEZE can be used and ELB will not be returned to the server since
consistency group was NOT specified on the establish path
– FlashCopy secondary to tertiary
• Tertiary will have consistent data
– Reestablish paths (if necessary)
– RESYNC (resumepprc) Global Copy
32
© 2005 IBM Corporation
Advanced Technical Support, Americas
When to use Global Copy
Recovery system does not need to be current with the
primary application system
– RPO in the range of hours or days
– User creates consistent copy of recovery data
Minor impact to application write I/O operations at the
primary location
Recovery uses copies of data created by the user on tertiary
volumes
Distance beyond ESCON or fcp limits
– 103 KM for ESCON links and 300 KM for fcp links
• RPQ for greater disances
A great tool for migrating data
33
© 2005 IBM Corporation
Advanced Technical Support, Americas
What is Global Mirror?
A Disaster Recovery (DR) data replication solution
– Reduced (less than peak bandwidth) network bandwidth requirements (duplicate
writes not sent)
A 2 site solution
Asynchronous data transfer
– No impact to the production write I/Os
Peer-to-peer (no, outside the box, server MIPS)
– Microcode controlled
– Peer-to-peer data copy mechanism is Global Copy
– Consistency Group formation mechanism is FlashCopy
3 copies (A B C)
– Or 4 copies (if test/practice copy (D copy) & DR is to be continued)
Unlimited distance
Very little data loss (Recovery Point Objective (RPO))
– Single digit seconds (goal was/is 3-5 seconds)
Scalable
– Up to 8 primary and secondary physical subsystems
• More with an RPQ
34
© 2005 IBM Corporation
Advanced Technical Support, Americas
Global Mirror: Basic concept
Concept
Asynchronous long distance copy (Global Copy), i.e., little to no impact to application writes
Momentarily pause application writes (fraction of millisecond to few milliseconds)
Create point in time consistency group across all primary subsystems (in OOS bitmap)
New updates saved in Change Recording bitmap
Restart application writes and complete write (drain) of point in time consistent data to remote site
Stop drain of data from primary (after all consistent data has been copied to secondary)
Logically FlashCopy all data (i.e., 2ndary is consistent, now make tertiary look like 2 ndary)
Restart Global Copy writes from primary
Automatic repeat of sequence every few seconds to minutes to hours (selectable and can be
immediate)
Intended benefit
Long distance, no application impact (adjusts to peak workloads automatically), small RPO,
remote copy solution for zSeries and Open Systems data, and consistency across multiple
Global Copy (PPRC-XD)
subsystems
over long distance
Host
I/O
Primar
y
Local
Site
35
Could require channel
extenders
FlashCopy
FCP links only
(record, nocopy, persistent, inhibit target
write)
Secondary
Tertiary
(Asynchronous)
Global Copy
Remot
e Site
© 2005 IBM Corporation
Advanced Technical Support, Americas
Consistency Group Formation
Coordinate
local units
Drain Time
FlashCopy Relationships
being established
CG Interval Time
...
...
...
...
Coordination
Time
Let CG data drain to remote
Record new writes in bitmaps
but do not copy to remote
All FlashCopy Relationships
established
Global Copy continually cycles
through volume bitmaps
copying changed data to remote
mirror volumes
36
© 2005 IBM Corporation
Advanced Technical Support, Americas
Tuneables (input parameters)
Maximum Coordination Time
– Maximum allowed pause of production write updates for the Consistency Group coordination
action
•
I.e., when the Master coordinates the formation of the Consistency Group with all Subordinates
– When coordination is completed, writes are allowed to continue
– Default = 50 milliseconds {Range: 0 to 65535 ms (65+ seconds)}
– If the ‘coordination time” is exceeded, coordination is stopped and all writes are allowed to
continue
– Design point is 2-3 ms
Maximum Drain Time
– Maximum CG drain time in seconds before failing (terminating) current drain activity
– Default = 30 seconds {Range: 0 to 65535 (just over 18 hours)}
– After 5 failures, drain time is infinite, i.e., until a consistency group is form, i.e., completely drained
Consistency Group Interval Time
– Time to wait before again starting the next consistency group formation process
– Default = 0 seconds {Range = 0 to 65535 seconds (just over 18 hours)}
37
© 2005 IBM Corporation
Advanced Technical Support, Americas
Typical Global Mirror Configuration
Multiple primary to multiple secondary subsystems
Consistency across all primary subsystems
LS
S
LS
S
Host I/O
When forming a Consistency Group,
PPRC-XD continues transmitting /
draining the consistent data to the
secondary site. Once the consistency
group is formed, the new update data
will be transmitted as in a PPRC-XD
environment without Asynchronous
PPRC.
LS
S
Once the A volumes have
been drained to the B
volumes, the B volumes
will be FlashCopied to the
C volumes.
LS
S
Local Site
LS
S
One Master,
multiple
Subordinates
Subordinate
Subordinate
LS
S
Remote Site
LS
S
LS
S
LS
S
LS
S
Master
Master communicates with
Subordinates to form consistency
groups
Note: The Master performs the same operations on volumes in the consistency group in its box when it directs the
Subordinates to perform operations.
38
© 2005 IBM Corporation
Advanced Technical Support, Americas
Global Mirror Initialization Process
4. Define Global Mirror session and
add volumes to the session
1. Establish Global Copy paths
Subordinate
2. Establish Global Copy pairs
FlashCop
y
Subordinate
SAN
.
.
FlashCop
y
FlashCop
y
Master
Wait until Global
Copy pairs have
completed 1st pass
copy then establish
FlashCopy pairs
3. Establish FlashCopy pairs
6. Start Global Mirror with Start
command sent to Mast
5. Establish control paths between
Master and Subordinates
Note: These paths could be created earlier
39
© 2005 IBM Corporation
Advanced Technical Support, Americas
Maintaining Consistency – Global Mirror
Consistency group is NOT specified on the establish path
command
– Loss of communication will NOT cause ELB to be returned for writes
– FREEZE command can be used to suspend pairs after Global Mirror
session is paused but ELB will NOT be returned for writes to LSSs
Consistency is maintained by not returning CE/DE or I/O
complete during the coordination phase when forming a
Consistency Group
– Not returning CE/DE or I/O complete causes the next dependent write
to NOT be issued maintaining the order of the dependent writes
40
© 2005 IBM Corporation
Advanced Technical Support, Americas
When to use Global Mirror?
RPO can be greater than 0 but still needs to be very current
– In the single digit second range
Limited impact to application write I/O operations at the
primary location
– Asynchronous data transfer
Recovery is on a disk-by-disk basis
Distance exceeds maximum limits for synchronous data
transfer
– 300 KM for fcp links
• Global Mirror only supports fcp links
41
© 2005 IBM Corporation
Advanced Technical Support, Americas
Global Mirror at 1000 mi DS8300 vs ESS 800 (both w/
128 x 10k RPM disk)
42
© 2005 IBM Corporation
Advanced Technical Support, Americas
What is Metro/Global Mirror
A 3 site Disaster Recovery (DR) data replication solution
– Metro Mirror from local (A) to intermediate (B) and Global Mirror from
intermediate to remote (C)
– The Metro Mirror secondary is cascaded to the remote site
4 copies of data (A B C D)
– C and D are Global Mirror secondary and FlashCopy volumes
– Or 5 copies (if test/practice copy (D copy) & DR is to be continued)
Unlimited distance between intermediate site and remote
RPO of 0 for “A” site failure
– Zero RPO implies automation to ensure no production updates if
mirroring stops
Potential RPO of 3-5 seconds for “A” and “B” twin site
failure
– Depends on workload and bandwidth between B and C
43
© 2005 IBM Corporation
Advanced Technical Support, Americas
When to use Metro/Global Mirror
When two recovery sites are required
44
© 2005 IBM Corporation
Advanced Technical Support, Americas
Remote Mirror Comparisons
RPO
Distance
45
Global Mirror
for zSeries
(XRC)
Metro Mirror
(PPRC)
Global Copy
(PPRC-XD)
Global Mirror
> 0 (3-5 sec)
0
> 0 (hours)
> 0 (3-5 sec)
Unlimited
300 KM (fcp)
Unlimited
Unlimited
CKD
Yes
Yes
Yes
Yes
FB
Yes
Yes
Yes
Yes
CKD & FB
No
Yes
Yes
Yes
© 2005 IBM Corporation
Advanced Technical Support, Americas
Failover Processing (1)
The secondary volume to which the command was issued
becomes a suspended PPRC primary
The targeted volume gets a Change Recording bitmap
– Used to track changes that make it different from its partner
Establishes a new relationship between the volume the
command was issued to and its PPRC primary volume
Valid for both Metro Mirror and Global Copy
Failover
Secondary
Primary
GC or MM
Before
46
Primary
SUS
?
CR
SUS
or
active
SUS
or
active
Primary
After
0
1
0
1
0
0
0
1
0
0
0
0
1
0
© 2005 IBM Corporation
Advanced Technical Support, Americas
Failover Processing (2)
No communication occurs between the two volumes
Typically, failover is used when the relationship between the
volumes is suspended
– Consider a path failure – Primary goes suspended, Secondary does
not know anything is wrong, is not suspended
If the relationship is NOT suspended when the command is
issued:
– The secondary volume WILL become a suspended primary
– The primary volume will BECOME a suspended primary when host I/O
is targeted to the volume – or a suspend command is issued to the
primary volume
– If neither I/O nor suspend occur, problems may arise during failback
47
© 2005 IBM Corporation
Advanced Technical Support, Americas
Failback Processing (1)
The primary volume to which this command is issued has
it’s PPRC partner converted, if necessary, to a PPRC
secondary
A path(s) must exist between the pairs
The volume to which the command was issued
– Combines the partner bitmaps to get total “difference”
– begins to resync to its partner which becomes a PPRC secondary
volume – data begins to transfer
Example 1: Failback original Primary
Primary
48
0
1
0
1
0
0
0
1
0
0
0
0
1
0
Before
0
1
0
1
0
0
0
1
0
0
0
0
1
0
SUS
0
1
0
1
0
0
0
1
0
0
0
0
1
0
OOS
CR
SUS
Secondary
Primary
CR
Primary
CR
Failback
0
1
0
1
0
0
0
1
0
0
0
0
1
0
Secondary
Primary
GC
GC or MM
During
After
© 2005 IBM Corporation
Advanced Technical Support, Americas
Failback Processing (2)
Similar to all PPRC establish operations, the resync begins
processing in “Global Copy” mode (First Pass) keeping
track of updates received during the resync (CR bitmap)
The pairs will return to their original mode (Global Copy or
Metro Mirror) at the conclusion of the resync operation
Failover and Failback are applicable to all PPRC
relationships, not just Global Mirror, as we will see in later
lectures and labs
Example 2: Failback original Target
Failback
49
0
1
0
1
0
0
0
1
0
0
0
0
1
0
Before
CR
CR
SUS
0
1
0
1
0
0
0
1
0
0
0
0
1
0
SUS
Primary
Secondary
GC
During
0
1
0
1
0
0
0
1
0
0
0
0
1
0
0
1
0
1
0
0
0
1
0
0
0
0
1
0
Secondary
Primary
GC or MM
After
OOS
CR
Primary
Primary
© 2005 IBM Corporation
Advanced Technical Support, Americas
Failover and Failback Command Parameters
For both failover and failback, the Primary and Secondary
parameters must reflect the “new direction” of the copy
operation
Serial
85551
A
Serial
ABC2A
Serial
85551
Serial
ABC2A
B
A
B
Secondary
Primary
Primary
Primary
To issue FAILOVER to B:
To issue FAILBACK to A:
CESTPAIR FO DEVN(B) PRI(ABC2A) SEC(85551)
failoverpprc –dev abc2a –remotedev 85551 b:a
CESTPAIR FB DEVN(A) PRI(85551) SEC(ABC2A)
failbackpprc –dev 85551 –remotedev abc2a a:b
50
© 2005 IBM Corporation
Advanced Technical Support, Americas
Addendum
Management Tools
Establish Paths
Establish Pairs
FREEZE/RUN
Resynchronizing Pairs
Path and Pair Status
Line speeds
References
51
© 2005 IBM Corporation
Advanced Technical Support, Americas
Management Tools
Runs on
z/OS
Runs on
Open
Server
Manages CKD
Manages FB
TSO
Yes
No
Yes
Yes (1)
API
Yes
No
Yes
Yes (1)
ICKDSF
Yes
No
Yes
No
DS CLI
No
Yes
Yes
Yes
TPC for R
No
Yes
Yes
Yes
GDPS
Yes
No
Yes
Yes (1)
Note:
1. A CKD unit address (and host UCB) must be defined in the same DS8000/DS6000
server against which host I/O may be issued to manage FB LUNs.
52
© 2005 IBM Corporation
Advanced Technical Support, Americas
Establish Paths
DS CLI
mkpprcpath –dev storage_image_id –remotedev storage_image_id –remotewwnn wwnn
–srclss source_LSS_ID –tgtlss target_LSS_ID source_port_ID:target_port_ID
ICKDSF
PPRCOPY ESTPATH UNIT(ccuu) FCPPATHS(X’aaaabbbb’) PRIMARY(ssid,ser#)
SECONDARY(ssid,ser#) LSS(X’pp’,X’ss’) WWNN(pwwnn,swwnn)
TSO
CESTPATH DEVN(device_number) PRIM(ssid wwnn lss) SEC(ssid wwnn lss)
LINK(aaaabbbb)
53
© 2005 IBM Corporation
Advanced Technical Support, Americas
Establish Pairs
DS CLI
mkpprc –dev storage_image_ID –remotedev storage_image_ID –type gcp –mode
full SourceVolumeId:TargetVolumeId
ICKDSF
PPRCOPY ESTPAIR DDNAME(dname) PRIMARY(ssid,ser#,cca)
SECONDARY(ssid,ser#,cca) LSS(X’pp’,X’ss’) MODE(COPY) OPTION(XD)
TSO
CESTPAIR DEVN(device_number) PRIM(ssid serialno cca lss) SEC(ssid serialno
cca lss) MODE(COPY) OPTION(XD)
54
© 2005 IBM Corporation
Advanced Technical Support, Americas
FREEZE
DS CLI
freezepprc –dev storage_image_ID –remotedev storage_image_ID
Source_LSS_ID:Target_LSS_ID
ICKDSF
PPRCOPY FREEZE DDNAME(dname) PRIMARY(ssid,ser#) SECONDARY(ssid,ser#)
LSS(X’pp’,X’ss’)
TSO
CGROUP DEVN(device_number) PRIM(ssid serialno lss)
SEC(ssid serialno lss) FREEZE
55
© 2005 IBM Corporation
Advanced Technical Support, Americas
RUN
DS CLI
unfreezepprc –dev storage_image_ID –remotedev storage_image_ID
Source_LSS_ID:Target_LSS_ID
ICKDSF
PPRCOPY RUN DDNAME(dname) PRIMARY(ssid,ser#) SECONDARY(ssid,ser#)
LSS(X’pp’,X’ss’)
TSO
CGROUP DEVN(device_number) PRIM(ssid serialno lss)
SEC(ssid serialno lss) RUN
56
© 2005 IBM Corporation
Advanced Technical Support, Americas
Resynchronizing Pairs
DS CLI
resumepprc –dev storage_image_ID –remotedev storage_image_ID –type gcp
SourceVolumeId:TargetVolumeId
ICKDSF
PPRCOPY ESTPAIR DDNAME(dname) PRIMARY(ssid,ser#,cca)
SECONDARY(ssid,ser#,cca) LSS(X’pp’,X’ss’) MODE(RESYNC) OPTION(XD)
TSO
CESTPAIR DEVN(device_number) PRIM(ssid serialno cca lss) SEC(ssid
serialno cca lss) MODE(RESYNC) OPTION(XD)
57
© 2005 IBM Corporation
Advanced Technical Support, Americas
Path Status DS CLI
lspprcpath –dev storage_image_ID Source_LSS_ID
58
© 2005 IBM Corporation
Advanced Technical Support, Americas
Pair Status DS CLI
lspprc –dev storage_image_ID –remotedev storage_image_ID –l
SourceVolumeId:TargetVolumeId
59
© 2005 IBM Corporation
Advanced Technical Support, Americas
Path Status ICKDSF
PPRCOPY QUERY DDNAME(dname) PATHS
ICK00700I DEVICE INFORMATION FOR 5C11 IS CURRENTLY AS FOLLOWS:
PHYSICAL DEVICE = 3390
STORAGE CONTROLLER = 3990
STORAGE CONTROL DESCRIPTOR = E9 DEVICE DESCRIPTOR = 0A
ADDITIONAL DEVICE INFORMATION = 4A001B35
TRKS/CYL = 15, # PRIMARY CYLS = 150
ICK04030I DEVICE IS A PEER TO PEER REMOTE COPY VOLUME
QUERY REMOTE COPY – PATHS
PRIMARY CONTROL UNIT INFORMATION
SERIAL
WORLD WIDE
NUMBER SSID LSS NODE NAME
------- ---- --- ---------------14940
9400 00
5005076300C01F4C
SECONDARY CONTROL UNIT INFORMATION
SERIAL
WORLD WIDE
NUMBER SSID LSS NODE NAME
------- ---- --- ---------------14940
9401 01
PATHS:
SERIAL
WORLD WIDE
NUMBER SSID LSS NODE NAME
PATH SAID DEST S*
------- ---- --- ---------------- ---- ---- ---- -1ST:
60
14940
9401
01 5005076300C01F4C
1
002C 00AC 13
2
00AC 002C 13
2ND: ....... .... ... ................
.... ....
00
3RD: ....... .... ... ................
.... ....
00
4TH: ....... .... ... ................
.... ....
00
© 2005 IBM Corporation
Advanced Technical Support, Americas
Pair Status ICKDSF
PPRCOPY QUERY UNIT(ccuu)
ICK00700I DEVICE INFORMATION FOR 5C11 IS CURRENTLY AS FOLLOWS:
PHYSICAL DEVICE = 3390
STORAGE CONTROLLER = 3990
STORAGE CONTROL DESCRIPTOR = E9
DEVICE DESCRIPTOR = 0A
ADDITIONAL DEVICE INFORMATION = 4A001B35 TRKS/CYL = 15, # PRIMARY CYLS = 150
ICK04030I DEVICE IS A PEER TO PEER REMOTE COPY VOLUME
QUERY REMOTE COPY - VOLUME
(PRIMARY) (SECONDARY)
SSID CCA
DEVICE LEVEL
STATE
PATH STATUS
------
---------
-------------- ----------- -
5C11
PRIMARY DUPLEX
ACTIVE
SSID CCA
SER # LSS SER # LSS
----------
-----------
9400 11
9401 11
14940 00
14940 01
PATHS SAID/DEST STATUS DESCRIPTION
----- --------- ------ ---------------2 002C 00AC 13 ESTABLISHED FIBRE CHANNEL PATH
00AC 002C 13 ESTABLISHED FIBRE CHANNEL PATH
ICK02206I PPRCOPY QUERY FUNCTION COMPLETED SUCCESSFULLY
ICK00001I FUNCTION COMPLETED, HIGHEST CONDITION CODE WAS 0
61
© 2005 IBM Corporation
Advanced Technical Support, Americas
Path Status TSO
CQUERY DEVN(device_number) PATHS
62
© 2005 IBM Corporation
Advanced Technical Support, Americas
Pair Status TSO
CQUERY DEVN(device_number)
63
© 2005 IBM Corporation
Advanced Technical Support, Americas
Pair Status ISMF
To Further Limit the Generated List, Specify a Single Value or List
of Values
in any of the following:
Rel Op
Value
Value
Value
Value
------ --------- --------- ---------
--------Cache Fast Write Status
. .
CF Volume Status . . . . . .
Dasd Fast Write Status . . .
Duplex Status
. . . . . . .
Index Status . . . . . . . .
Physical Status
Read Cache Status
. . . . . .
Shared Dasd
. . . . .
. . . . . . . .
Use Attributes . . . . . . .
64
© 2005 IBM Corporation
Advanced Technical Support, Americas
Pair Status ISMF
Use ENTER to continue, END to exit Help.
HELP--------------------
DUPLEX STATUS (Page 2 of 2)
-------------------HELP
COMMAND ===>
65
PPRIMARY
The volume is primary of a PPRC pair.
PSECNDRY
The volume is secondary of a PPRC pair.
PPRI-PEN
The volume is primary of a PPRC pair in the process of being established.
PSEC-PEN
The volume is secondary of a PPRC pair in the process of being established.
PPRI-SUS
The volume is primary of a PPRC pair that is suspended.
PSEC-SUS
The volume is secondary of a PPRC pair that is suspended.
PPRI-FAI
The volume is primary of a PPRC pair in fail status.
PSEC-FAI
The volume is secondary of a PPRC pair in fail status.
© 2005 IBM Corporation
Advanced Technical Support, Americas
Line Speeds
66
Mbps
ApproximateMB/Sec
Equivalent T1
lines
T1
1.544
.1544
1
T3
44.746
4.4746
28
OC3
155
15.5
100
OC12
622
62.2
400
OC48
2488
248.8
1600
© 2005 IBM Corporation
Advanced Technical Support, Americas
IBM Copy Services Technologies – DS6K/DS8K
FlashCopy
Point in time copy
Available on:
Synchronous mirroring
Asynchronous mirroring
Available on:
Available on:
DS8000, DS6000, ESS
SAN Volume Controller
DS4000
N Series
Within
Storage
System
Global Mirror
Metro Mirror
DS8000, DS6000, ESS
SAN Volume Controller
DS4000
N Series
DS8000, DS6000, ESS
SAN Volume Controller
DS4000
N Series
Metro / Global Mirror
Three site synchronous and
asynchronous mirroring
Available on:
DS8000, ESS
N Series
Primary
Metro distance
Primary
Primary
Metro
Site A
<300km
Site A
Site A
Site B
Site B
Out of
67
Region
Out of
Site
Region
B
Site C
© 2005 IBM Corporation
Advanced Technical Support, Americas
Copy Services Matrix
Device
Is
GMz10
GMz10
(XRC)
Primary
(XRC)
Secondary
No
Metro Mirror
or Global
Copy Primary
Metro Mirror
or Global
Copy
Secondary
Global
Mirror
Primary
Global
Mirror
Secondary
FlashCopy
Source
FlashCopy
Target
Incremental
FLC Source
Incremental
FLC Target
Concurrent
Copy
Source
Yes11
Yes
No
Yes
No
Yes
Yes
No
No
Yes
Yes11
No
Yes
No5
Yes
No5
Yes
No5
Yes
No5
Yes
Metro Mirror
or Global
Copy
Primary
Yes
Yes
No
Yes 1
No6
Yes1
Yes
Yes
Yes
No6
Yes
Metro Mirror
or Global
Copy
Secondary
No
No5
Yes 1
No
Yes1
No
Yes
Yes8
Yes
Yes
No
Global
Mirror
Primary
Yes
Yes
No6
Yes1
No
No
Yes
Yes
Yes
Yes
Yes
Global
Mirror
Secondary
No
No5
Yes1
No
No
No
Yes
Yes8
Yes9
No
No
FlashCopy
Source
Yes
Yes
Yes
Yes
Yes
Yes
Yes 3,4
Yes 4
Yes3
No
Yes
FlashCopy
Target
No
No5
Yes 2
No
No
No
Yes 4
Yes4
No
No
No
Incremental
FLC Source
No7
Yes
Yes
Yes
Yes
Yes9
Yes
No
No
No
Yes
Incremental
FLC Target
No7
No5
Yes 2
No
No
No
No
No
No
No
Yes
Concurrent
Copy
Source
Yes
Yes
Yes
No
Yes
No
Yes
Yes
Yes
Yes
Yes
May
Become
GMz10
(XRC)
Primary
GMz10
(XRC)
Secondary
68
© 2005 IBM Corporation
Advanced Technical Support, Americas
Notes:
1.
Only in a Metro/Global Copy (supported on ESS) or a Metro/Global Mirror Environment (supported
on ESS and DS8000).
2.
FlashCopy V2 at LIC 2.4.0 and higher on ESS800 (DS6000 and DS8000 utilize FlashCopy V2 by
default).
–
–
–
You must specify the proper parameter to perform this
Metro Mirror primary will go from full duplex to copy pending until all of the flashed data is transmitted to remote
Global Mirror primary cannot be a FlashCopy target
3.
FlashCopy V2 Multiple Relationship.
4.
FlashCopy V2 Data Set FlashCopy (only available for z/OS volumes).
5.
The Storage Controller will not enforce this restriction, but it is not recommended.
6.
A volume may be converted between the states Global Mirror primary, Metro Mirror primary and
Global Copy primary via commands, but it two relations cannot exist at the same time (i.e. multitarget).
7.
GMz (XRC) Primary, Global Mirror Secondary, Incremental FlashCopy Source and Incremental
FlashCopy Target all use the Change Recording Function. For a particular volume only one of these
relationships may exist.
8.
Updates to the affected extents will result in the implicit removal of the FlashCopy relationship, if
the relationship is not persistent.
9.
This relationship must be the FlashCopy relationship associated with Global Mirror – i.e. there may
not be a separate Incremental FlashCopy relationship.
10. Global Mirror for zOS (GMz) is supported on ESS and DS8000
11. In order to ensure Data Consistency, the XRC Journal volumes must also be copied.
69
© 2005 IBM Corporation
Advanced Technical Support, Americas
References
SC26-7916 DS8000 Command-Line Interface User’s Guide
GC26-7922 DS6000 Command-Line Interface User’s Guide
SC35-0428 DFSMS Advanced Copy Services
SG24-6788 IBM System Storage DS8000 Series: Copy Services in Open Environments
SG24-6787 IBM System Storage DS8000 Series: Copy Services with IBM System z
SG24-6783 IBM System Storage DS6000 Series: Copy Services in Open Environments
SG24-6782 DS6000 Series: Copy Services with IBM System z Servers
Performance White Paper
– http://w31.ibm.com/sales/systems/portal/_s.155/254?navID=f320s260&geoID=All&prodID=System%20Storage&do
cID=ditlDS8000PerfWPPower5
DS8000/DS6000 Copy Services: Getting Started
– WP100905
– http://www03.ibm.com/support/techdocs/atsmastr.nsf/WebDocs/?Search&Query=[HTMLDocumentName=WM*]+AN
D+(burger)&Start=1&Count=50&SearchOrder=1&SearchMax=10000
70
© 2005 IBM Corporation
Advanced Technical Support, Americas
Trademarks
The following terms are trademarks or registered trademarks of the IBM Corporation in either the United States, other countries or both.
AIX
AIX 5L
BladeCenter
Chipkill
DB2
DB2 Universal Database
DFSMSdss
DFSMShsm
DFSMSrmm
Domino
e-business logo
Enterprise Storage Server
ESCON
eServer
FICON
FlashCopy
GDPS
Geographically Dispersed
Parallel Sysplex
HiperSockets
i5/OS
IBM
IBM eServer
IBM logo
iSeries
Lotus
ON (button device)
On demand business
OnForever
OpenPower
OS/390
OS/400
Parallel Sysplex
POWER
POWER5
Predictive Failure Analysis
pSeries
S/390
Seascape
ServerProven
System z9
System p5
System Storage
Tivoli
TotalStorage
TotalStorage Proven
TPF
Virtualization Engine
X-Architecture
xSeries
z/OS
z/VM
zSeries
Linear Tape-Open, LTO, LTO Logo, Ultrium logo, Ultrium 2 Logo and Ultrium 3 logo are trademarks in the United States and other countries of
Certance, Hewlett-Packard, and IBM.
Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States and/or other countries.
Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States and/or other countries.
Intel, Intel Inside (logos), MMX and Pentium are trademarks of Intel Corporation in the United States and/or other countries.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Linux is a trademark of Linus Torvalds in the United States and other countries.
Other company, product, or service names may be trademarks or service marks of others.
71
© 2005 IBM Corporation