Transcript Before You Begin: Assign Information Classification

Informix Enterprise
Replication in 5 easy
Steps
WAIUG Forum
December 8-9, 2006
Nancy Balsbaugh
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
1
What are we going to cover?
Overview
 Types of Enterprise Replication
 Requirements – Linux, 10.00.UC5
 How to set it up
 How to verify/check replication
 Troubleshooting
 Pitfalls and Gotchas
 Magic Cookies
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
2
Types of Enterprise Replication
 Primary – Target (tree)
Simple
No Conflict
Works or Not
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
3
Types of Enterprise Replication cont’d
 Update Anywhere
Complex
Conflict resolution
May Partially Work
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
4
Requirements
 Connectivity
Sqlhosts
g_hdr group g_server1
-
group -
server1 onsoctcp
g_server2
i=1
i=2
hostname1
group -
server2 onsoctcp
-
-
server1 g=g_server1
i=3
hostname2
server2 g=g_server2
.rhosts
/etc/services
/etc/hosts
Template File <filename>
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
5
Requirements Cont’d
 Onconfig
Dbserveralias – used in the sqlhosts file
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
6
How To Set It Up
 Step 1 - Define the Servers
 Step 2 - Define the replicates
 Step 3 - Change the state of the replicate (start
replicating)
 Step 4 - Synchronize the Data (copy pre-replication
data)
 Step 5 - Verify
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
7
Step 1 - Define the servers
 Participants, master, conflict resolution rules and scope, frequency,
error logging, full rows or changed columns, triggers, etc.
 Primary
cdr define server --connect=server1 --idle=0 --ats=/errorlog/ats -ris=/errorlog/ris --init g_server1
 Secondary
cdr define server -c server1 --idle=0 --init --leaf --sync=g_server1
server2
 ATS – aborted transaction spooling, RIS – row information
spooling, Init - option initializes the server, Idle – idle timeout, Leaf
means tree hierarchy – leave that out if you want update anywhere
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
8
After the first server is defined
cdr list server
SERVER
ID STATE
STATUS
QUEUE
CONNECTION CHANGED
-----------------------------------------------------------------------
g_server1
3 Active
Local
0
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
9
After the 2nd server is defined
cdr list server
SERVER
ID STATE
STATUS
QUEUE
CONNECTION CHANGED
-----------------------------------------------------------------------
g_server1
3 Active
Local
0
g_server2
7 Active
Connected
0 Nov 14 13:47:09
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
10
Cdr list of tree hierarchy
Listing g_server1
SERVER
ID STATE
STATUS
QUEUE CONNECTION CHANGED
----------------------------------------------------------------------g_server2
4 Active
Connected
0 Nov 14 13:46:16
g_server3
5 Active
Connected
0 Nov 14 13:47:29
g_server1
3 Active
Local
0
**********
Listing g_server2
SERVER
ID STATE
STATUS
QUEUE CONNECTION CHANGED
----------------------------------------------------------------------g_server2
4 Active
Local
0
g_server1
3 Active
Connected
0 Nov 14 13:46:17
**********
Listing g_server3
SERVER
ID STATE
STATUS
QUEUE CONNECTION CHANGED
----------------------------------------------------------------------g_server3
5 Active
Local
0
g_server1
3 Active
Connected
0 Nov 14 13:47:28
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
11
Update Anywhere cdr list
Listing g_server1
SERVER
ID STATE
STATUS
QUEUE CONNECTION CHANGED
----------------------------------------------------------------------g_server2
4 Active
Connected
0 Nov 14 13:46:16
g_server3
5 Active
Connected
0 Nov 14 13:47:29
g_server1
3 Active
Local
0
**********
Listing g_server2
SERVER
ID STATE
STATUS
QUEUE CONNECTION CHANGED
----------------------------------------------------------------------g_server2
4 Active
Local
0
g_server1
3 Active
Connected
0 Nov 14 13:46:17
g_server3
5 Active
Connected
0 Nov 14 13:47:29
**********
Listing g_server3
SERVER
ID STATE
STATUS
QUEUE CONNECTION CHANGED
----------------------------------------------------------------------g_server3
5 Active
Local
0
g_server1
3 Active
Connected
0 Nov 14 13:47:28
g_server2
4 Active
Connected
0 Nov 14 13:46:16
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
12
Step 2 – Option 1 - Define the replicates
cdr define replicate --conflict=timestamp --scope=row --ats --ris -master=g_server1 <replicate_name>
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
13
Step 2 – Option 1 - Define the replicates
cdr list repl
CURRENTLY DEFINED REPLICATES
------------------------------REPLICATE:
r_test1
STATE:
Active ON:g_a1
CONFLICT:
Always Apply
FREQUENCY:
immediate
QUEUE SIZE:
0
PARTICIPANT:
testdb:informix.test1
OPTIONS:
transaction,ris,ats,fullrow
REPLID:
65537 / 0x10001
REPLMODE:
PRIMARY
APPLY-AS:
INFORMIX ON:g_a1
REPLTYPE:
Master
REPLICATE:
r_test2
STATE:
Active ON:g_a1
CONFLICT:
Always Apply
FREQUENCY:
immediate
QUEUE SIZE:
0
PARTICIPANT:
testdb:informix.test2
OPTIONS:
transaction,ris,ats,fullrow
REPLID:
65538 / 0x10002
REPLMODE:
PRIMARY
APPLY-AS:
INFORMIX ON:g_a1
REPLTYPE:
Master
ON:g_a1
ON:g_a1
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
14
Step 2 - Option 2 – Define a Template
 cdr define template <templatename>
--conflict=timestamp
--scope=row --ats --ris --firetrigger
--master=g_server1
--database=dbname --file=<filename>
 The file contains a list of tables to be replicated –
easiest way.
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
15
Step 2 - Option 2 – Define a Template
cdr list template
TEMPLATE
DATABASE
TABLES
==============================================
Template_name
dbname
informix.table1
dbname
informix.table2
dbname
informix.table3
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
16
Step 3 - Option 1 – Change the replicate
cdr start replicate <replicate_name> g_server1 g_server2
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
17
Step 3 - Option 2 - Realize the template
 cdr realize template <templatename> g_server1
 cdr realize template <templatename> g_server2 g_server3
 This copies the replicate to the other servers and
changes it’s state – “starts it”
 Do the primary first!
 Then broadcast the rest
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
18
Synchronize the data
 cdr sync replset -m g_server1 -s <templatename>
--firetrigger g_server2 g_server3
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
19
How to Verify/Check Replication
 cdr check replicateset -c server1 -m g_server1 -s
<templatename> -v -a
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
20
Troubleshooting
 To Repair Replication
cdr check replicateset -c server1 -m g_server1 -s
<templatename> -v –a –R
 Cdr list of all the servers (with the connect option)
 Cdr err of all the servers (with the connect option)
 Check all the servers message logs
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
21
cdr err –a Without Errors
SERVER:SEQNO
REVIEW TIME
ERROR
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
22
cdr err –a With Errors
SERVER:SEQNO
REVIEW
TIME
ERROR
g_server2:164
N
2006-11-02 19:39:01
31
GC operation participate replicate
'sync_17797_131103_1162517923' failed: undefined replicate
g_server3:165
N
2006-11-02 19:39:01
31
GC operation participate replicate
'sync_17797_131103_1162517923' failed: undefined replicate
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
23
Online Log when replication is being set-up
12:10:39
12:10:40
12:10:42
12:10:42
Building 'syscdr' database ...
'syscdr' database built successfully.
CDR queuer initialization complete
CDR NIF listening on asf://g_server1
12:13:10 Checkpoint Completed: duration was 1 seconds.
12:13:10 Checkpoint loguniq 298, logpos 0x132a7018, timestamp:
0xf5f9165
.
12:20:22 ER checkpoint started
.
12:20:25 DDR Log Snooping - Snooping started in log 299
12:25:41 Checkpoint Completed: duration was 0 seconds.
12:25:41 Checkpoint loguniq 299, logpos 0x3bb9018, timestamp:
0xf63be8a
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
24
Pitfalls and Gotchas
 cdr err will mark errors and you’ll need to make sure
you use cdr err –a to see all errors that haven’t been
zapped.
 If Update-Anywhere, then use the –c option to connect
to the other servers and look from their perspective.
 Use the –c (connect) option to make control codes
come from one central server.
 Define can only be done serially – same with all the
commands. Do not run them in parallel. To speed
things up – Use broadcast mode.
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
25
Pitfalls and Gotchas cont’d
You must list all servers for cdr list otherwise 2 servers
can have problems between them that doesn’t affect
the primary. If you are looking only at the primary, you
won’t see it.
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
26
Magic Cookies
 Check the return code
 Check the output of cdr err –a
 Use cdr err --zap to clear out errors
 Always check the list of all the servers and the errors
on all the servers.
 Use cdr finderr to lookup errors
 Scripts Included! Very Basic! Need improvement.
Mostly so you can see the cdr commands. Feel free to
improve upon them and send them back to me.
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
27
cdr finderr
12:11:10
CDR GC synchronization error 35
finderr 35
-35
Operation would block.
An operating-system error code with the
meaning shown was unexpectedly returned
to the database server. If the error
recurs, note all circumstances and
contact Technical Support at
[email protected].
WAIUG Forum December 8-9, 2006 Introduction to IDS
Enterprise Replication
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
28
cdr finderr cont’d
cdr finderr 35
35
server not defined in sqlhosts
Sqlhosts on leaf
g_hdr
group
-
g_server1
group
server1
-
-
i=2
onsoctcp
SERV1
server1
g_server2
group
-
i=3
server2
onsoctcp
SERV2
server2
Presentation_ID
-
i=1
-
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
g=g_server1
g=g_server2
29
Q&A
Any Questions?
Source: Placeholder for Notes is 14 points
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
30
Thank You
Nancy Balsbaugh
[email protected]
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
31
To list all the replication servers (cdr list)







#!/bin/ksh
echo "Building custom scripts for this Cluster"
echo "#!/bin/ksh" > cdrlist.new
chmod 700 cdrlist.new
echo "echo \"***********************\"" >> cdrlist.new
echo "date" >> cdrlist.new
echo "echo \"Starting Listing Servers\"" >> cdrlist.new










for i in `cat pub`
do
echo "Adding in the publisher $i"
echo "echo \"Listing g_$i\"" >> cdrlist.new
echo "cdr list server -c $i" >> cdrlist.new
echo "echo \"Return value:\$?\"" >> cdrlist.new
echo "echo \"\"" >> cdrlist.new
echo "echo \"\"" >> cdrlist.new
echo "echo \"**********\"" >> cdrlist.new
done


export pub=$i
echo "pub = $pub"










for i in `cat servers`
do
echo "Adding in the subscriber $i"
echo "echo \"Listing g_$i\"" >> cdrlist.new
echo "cdr list server -c $i" >> cdrlist.new
echo "echo \"Return value:\$?\"" >> cdrlist.new
echo "echo \"\"" >> cdrlist.new
echo "echo \"\"" >> cdrlist.new
echo "echo \"**********\"" >> cdrlist.new
done
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
32
To see errors (cdr err)





echo "#!/bin/ksh" > cdrerr.new
chmod 700 cdrerr.new
echo "echo \"***********************\"" >> cdrerr.new
echo "date" >> cdrerr.new
echo "echo \"Starting Listing Servers\"" >> cdrerr.new










for i in `cat pub`
do
echo "Adding in the publisher $i"
echo "echo \"Showing Errors from g_$i\"" >> cdrerr.new
echo "cdr err -a -c $i" >> cdrerr.new
echo "echo \"Return value:\$?\"" >> cdrerr.new
echo "echo \"\"" >> cdrerr.new
echo "echo \"\"" >> cdrerr.new
echo "echo \"**********\"" >> cdrerr.new
done


export pub=$i
echo "pub = $pub"










for i in `cat servers`
do
echo "Adding in the subscriber $i"
echo "echo \"Showing Errors from g_$i\"" >> cdrerr.new
echo "cdr err -a -c $i" >> cdrerr.new
echo "echo \"Return value:\$?\"" >> cdrerr.new
echo "echo \"\"" >> cdrerr.new
echo "echo \"\"" >> cdrerr.new
echo "echo \"**********\"" >> cdrerr.new
done
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
33
To clear errors in cdr err





echo "#!/bin/ksh" > cdrzap.new
chmod 700 cdrzap.new
echo "echo \"***********************\"" >> cdrzap.new
echo "date" >> cdrzap.new
echo "echo \"Starting Listing Servers\"" >> cdrzap.new










for i in `cat pub`
do
echo "Adding in the publisher $i"
echo "echo \"Showing Errors from g_$i\"" >> cdrzap.new
echo "cdr err -z -c $i" >> cdrzap.new
echo "echo \"Return value:\$?\"" >> cdrzap.new
echo "echo \"\"" >> cdrzap.new
echo "echo \"\"" >> cdrzap.new
echo "echo \"**********\"" >> cdrzap.new
done


export pub=$i
echo "pub = $pub"










for i in `cat servers`
do
echo "Adding in the subscriber $i"
echo "echo \"Showing Errors from g_$i\"" >> cdrzap.new
echo "cdr err -z -c $i" >> cdrzap.new
echo "echo \"Return value:\$?\"" >> cdrzap.new
echo "echo \"\"" >> cdrzap.new
echo "echo \"\"" >> cdrzap.new
echo "echo \"**********\"" >> cdrzap.new
done
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
34
To Run Cdr Check

echo "#!/bin/ksh" > cdrcheck.new



chmod 700 cdrcheck.new
echo "date" >> cdrcheck.new
echo "echo \"Running cdr check for status of Cluster\"" >> cdrcheck.new

echo "time cdr check replicateset -c $pub -m g_$pub -s <templatename> -v -a >>
/home/informix/cdr_check_output_all_\`date +%Y%m%d%H%M\` 2>&1" >> cdrcheck.new







echo "echo \"\"" >> cdrcheck.new
echo "echo \"Showing Errors from g_$i\"" >> cdrcheck.new
for i in `cat servers`
do
echo "Adding in the subscriber $i"
echo "$i" >> cdrcheck.new
done

echo "-v -a >> /home/informix/cdr_check_output_all_\`date +%Y%m%d%H%M\` 2>&1" >> cdrcheck.new
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
35
To Delete a server


echo "#!/bin/ksh" > deleteserv.new
chmod 700 deleteserv.new

echo "date" >> deleteserv.new


for i in `cat servers`
do


echo "Adding in $i"
echo "echo \"Deleting g_$i\"" >> deleteserv.new


echo "cdr delete server g_$i --connect=$i" >> deleteserv.new
echo "echo \"Return value:\$?\"" >> deleteserv.new

done


for i in `cat pub`
do


echo "Adding in the publisher $i"
echo "echo \"Deleting g_$i\"" >> deleteserv.new


echo "cdr delete server g_$i" >> deleteserv.new
echo "echo \"Return value:\$?\"" >> deleteserv.new

done
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
36
To Define a server




echo "#!/bin/ksh" > defserv.new
chmod 700 defserv.new
echo "date" >> defserv.new
echo "echo \"Starting Define Servers\"" >> defserv.new







for i in `cat pub`
do
echo "Adding in the publisher $i"
echo "echo \"Defining g_$i\"" >> defserv.new
echo "cdr define server --connect=$i --idle=0 --init g_$i" >> defserv.new
echo "echo \"Return value:\$?\"" >> defserv.new
done


export pub=$i
echo "pub = $pub"







for i in `cat servers`
do
echo "Adding in the subscriber $i"
echo "echo \"Defining g_$i\"" >> defserv.new
echo "cdr define server -c $i --idle=0 --init --leaf --sync=g_$pub g_$i" >> defserv.new
echo "echo \"Return value:\$?\"" >> defserv.new
done
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
37
To Define a template

echo "#!/bin/ksh" > deftemp.new



chmod 700 deftemp.new
echo "date" >> deftemp.new
echo "echo \"Starting Define Template\"" >> deftemp.new


export pub=`cat pub`
echo "pub = $pub"


export database=`cat database`
echo "database = $database"

echo "cdr define template <templatename> --conflict=always --scope=row --master=g_$pub -database=$database --file=<filename>" >> deftemp.new
echo "echo \"Return value:\$?\"" >> deftemp.new

Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
38
To Realize the template (copy the template to
the other servers and turn on replication)




echo "#!/bin/ksh" > realizetemp.new
chmod 700 realizetemp.new
echo "date" >> realizetemp.new
echo "echo \"Starting Realize Template\"" >> realizetemp.new







for i in `cat pub`
do
echo "Adding in the publisher $i"
echo "echo \"Realizing template on g_$i\"" >> realizetemp.new
echo "cdr realize template <templatename> g_$i" >> realizetemp.new
echo "echo \"Return value:\$?\"" >> realizetemp.new
done


export pub=$i
echo "pub is $pub"









echo "Adding in the subscribers"
echo "echo \"Realizing template on subscribers\"" >> realizetemp.new
echo "cdr realize template <templatename>" >> realizetemp.new
for i in `cat servers`
do
echo " g_$i" >> realizetemp.new
done
echo "echo \"Return value:\$?\"" >> realizetemp.new
echo "YOU NEED TO MODIFY REALIZETEMP"
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
39
To Synchronize the servers

echo "#!/bin/ksh" > syncsubs.new

chmod 700 syncsubs.new

echo "date" >> syncsubs.new

echo "echo \"Starting to Sync\"" >> syncsubs.new

for i in `cat pub`

do

echo "cdr sync replset -m g_$i -s <templatename> --firetrigger" >> syncsubs.new

for i in `cat servers`

do

echo g_$i >> syncsubs.new

done

echo "echo \"Return value:\$?\"" >> syncsubs.new

Done

echo "YOU NEED TO MODIFY SYNCSUBS"
Presentation_ID
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Public
40