Transcript e-Business Suite on Linux RAC (Platform Transition through Upgrade)

e-Business Suite on Linux RAC
(Platform Transition through
Upgrade)
Chakri Reddy
Director of Delivery
Zensar Technologies
San Jose, CA.
March 2005
Ram Laxmanan
Director, PMO Office
Zensar Technologies
San Jose, CA.
Agenda
•
•
•
•
•
•
•
•
•
Who we are
Description of project Goals
Hardware Architecture & Scaling Plan
11i Linux / RAC Architecture
Upgrade Process Overview
Upgrade Takeaways
Infrastructure Issues
Global Team Tactics
Questions & Answers
Who we are
• Zensar Technologies (www.zensar.com)
– Global information technology organization
– Decade of experience of providing IT and IT enabled service
– Gained over 7500+ person years of experience and executed over 1250+
engagements
– Leading software exporter in India, over 2000 skilled resources across six
delivery centers
– 23 offices worldwide
• Zensar’s established Oracle Applications Practice
– Enterprise Applications Group (EAG) is a focus area in packaged
solutions
– Dedicated Oracle competency center, with over 600 skilled resources
– Engagements include implementations, upgrades, customizations, 3rd
party integration and on-going support
– Customers include Cisco, Logitech, Fujitsu, Fluke, PO Ned Lloyd and
Sprint
About Electronic Arts, our customer
• The #1 video game publisher in the US
• Over a hundred popular titles such as NBA Street, Madden
NFL 2004, SimCity 4, Need for Speed, and Medal of
Honor.
• Has also published extremely successful games based on
Hollywood franchises, including The Lord of the Rings,
Harry Potter and James Bond
• Total revenue for year 2003-04 : $3 billion
• Has offices throughout the Asia-pacific, Europe and North
America – EU territories contributed to over $1 billion of
revenue last year
Project Goals (Business)
• Move all business applications from RISC platform to
Linux
• Plan for future growth and scalability
• Cost savings in terms of licensing and hardware
• Rapid deployment
• Save $$$’s
• OM BPR
• Review and update customisations
Project Goals (Technical)
• Standardize the installation of Oracle Applications 11.5.9
across 4 global installations in Electronic Arts
• Upgrade to Oracle 11.5.9 e-business suite of products
• Transition to e-business suite running on Linux RAC
– Along with the upgrade
• Produce a common and extensible hardware environment
– That meets EA’s user and performance criteria, currently and into
the future
• Introduce a hardware and software architecture that would
permit both scaling of capacity and continuous operations
• The upgrade and platform transition, including user testing
must be accomplished inside of 130 hours
Fact sheet
•
•
•
•
•
Organizations, 1 in US, 21 in EU
Languages, 1 in US, 4 in EU
Database size, 200Gb in US, 350 in EU
Users, 175 in US, 350 in EU
Transaction volumes
– 9000 order lines booked per day
– 30000 picking line details per day
– 10000 concurrent requests per day
• Boundary systems, 15 in US, 25 in EU
• Customizations, 1000 each in US and EU
• Dual on-shore / off-shore delivery model
Hardware Architecture & Scaling Plan
• Dell was selected as the vendor of choice at EA for world wide
deployment of Oracle 11.5.9
• RedHat 2.1 was the operating system at all tiers of the
architecture
• “Multi-node” & “Multi-tier”
• Combination of both local storage and SAN storage
– Intelligently deployed to get the highest performance and greatest
economy from Oracle Applications
• Hardware load balanced Oracle Applications servers
– Easily configured and robustly implemented.
• Dual concurrent process servers
– Assuring rapid process completion
– Confident & continuous operations, deployed with PCP (parallel
concurrent processing)
• Dual RAC nodes
11i Linux / RAC Architecture
Linux Production Hardware Diagram
Application Tier
User Tier
Applications Server 01
Database Tier
DB Server 01
IBM Compatible
Concurrent Server 01
IBM Compatible
SAN Storage Device
Load Balancer
Concurrent Server 02
IBM Compatible
DB Server 02
IBM Compatible
Applications Server 02
EU Production Configuration
2 Applications Servers = Dell 2 processors 4 gig memory
2 Concurrent Mgr Servers = Dell 2 processors 4 gig memory
2 Database Servers
= Dell 2 processors 4 gig memory
11i Linux / RAC Architecture
L4 Switch
Load
Balancer
App 2
App 1
RAC
CM 1
CM 2
DB Node1
DB Node2
Database
Datafiles
Storage
Upgrade Process - Overview
• Source
– Oracle Applications Release 11.03 with 8i DB running on HPUX
– Single Node Configuration
– 8 Processors at 575 MHz Speed
• Target
–
–
–
–
Oracle e-Business Suite Release 11.5.9
2 Application Servers (H/W Load Balance)
2 Concurrent Manager Servers with PCP Configuration
2 RAC Load Balanced DB Nodes
• Upgrade Starts With
–
–
–
–
Single Node RISC Server Running 11.03
One Intel Linux Application Server
One Linux CM Server
One Linux DB Server
Finishing Steps and
Testing
App SVR 02
Phase 3
6 hr *
18 hr
5 hr
6 hr *
CM SVR 02
Oracle 11.0.3
8.1.7.4 DB
App SVR 02
CM SVR 02
Upgrade = 188 Linux 11.5.9 App SVR
Backup = 29
Total = 217
Linux CM Servers
Linux 9.2.0.4 RACDB
Oracle 11.5.9
9.2.0.4
15 hr
Index Create
Linux 11.5.9 App SVR
Linux CM Servers
cv
Estimate times in italics e.g. Additional
Setup
6 hr *
2 hr
Final Backup
Multi-node + RAC
App SVR 02
10 hr 11.5.9.2 Cumulative Patch
Oracle 11.0.3
8.1.7.3 DB
Post Import
13 hr
5 hr *
5 hr
4 hr
App SVR 01
4 hr
CM SVR 01
Pre AutoUpgrade
First Backup
DB Upgrade 8.1.7.4
CAT 3
CAT 1 & 2
Transfer to Virginia
EA EU CRP-2 Linux Upgrade Model
1 hr
Validate Oracle
9.2.0.4 & RAC
RAPID Install Apps
Prepare for Functional and
DBA Category Steps
Linux RAC Software Install
and Configuration
Linux App SVR and CM
SVR Configuration
eucr1db1
6 hr
App SVR 01
Linux 11.5.9 APPS
Parallel Export/Import DB
CM SVR 01
22 hr
eucr1db1
Pre Export
Phase 2
2 hr
Fourth Backup
CAT 4, 5, 6 Functional
CAT 4, 5, 6 DBA + NLS
Post Autopatch
Third Backup
Oracle 11.0.3
8.1.7.3 DB
Auto - Patch
Second Backup
eucr1db1
Gain Approval to
? hr Release the Instance
Oracle 11.0.3
8.1.7.4 DB
Auto - Upgrade
Linux 11.5.9 APPS
23 hr
App SVR 01
6 hr *
46 hr
CM SVR 01
Conduct
Handover
Activities
Receive Single Node Surce
DB & App From EA
Receive and Validate Linux
Hardware Palatform and
HP DB server
Phase 1
? hr
Roll Out
Connectivity
Information
Internal User
Testing
eucr1db1
? hr
? hr
8 hr Additional Setup
4 hr
Upgrade Process - Overview
Downtime begins
CM SVR 02
Downtime
ends
Linux 9.2.0.4 RACDB
Oracle 11.5.9
9.2.0.4
Notes:
* Indicates actual backup time. Expectation
is ~15 min backup time using mirror/split
Upgrade Process - Overview
Auto
Patch
11.5.9
CM2
Cat 4-6
App 1
CM 1
Oracle 11.0.3
8.1.7.3
HP Server
Single-node
Linux 11.5.9
Oracle Apps 11i – Linux Apps/CM, HP DB
Cat 1-3
Auto
Upgrade
Export
Import
Index
Build
App 1
RAC (DB2)
App2
CM2
App 2
CM 1
Export &
Import
CM 2
DB Node1 DB Node2
Oracle 11.5.9
9.2.0.4
Linux 11.5.9
Upgrade Process - Overview
• Customized Oracle Upgrade Approach
– Heterogeneous upgrade approach
• Admin node running on Linux and DB node running on RISC server
– Pre upgrade steps
• Rapid install (Linux application server & CM server)
• 9i DB Installation (Linux DB server)
– Category 1, 2 and 3 steps
• Perform all relevant category steps on RISC server running 11.03
• Gather schema statistics (30%)
• Backup applications
– Auto Upgrade
• Admin node running on Linux and DB node running on RISC server
• Batch size of 10000 & max workers of 20
• Backup applications
– Auto Patch
• Gather schema statistics (30%)
• Backup applications
Upgrade Process - Overview
• Customized Oracle Upgrade Approach
– Patch the applications
• Apply consolidated update patch 2
• Apply one off patches
– Category 4, 5 and 6 steps
• Perform all relevant category steps
• Apply customizations (if any)
• Apply conversions (if any)
– e-Business suite functioning with
• Linux application server
• Linux CM server
• RISC DB server
Upgrade Process - Overview
• Customized Oracle Upgrade Approach
– Export source DB
• Apply applications consolidated export import patch (2981815 )
for 9i on SOURCE admin node.
• Direct path full DB export
– Import DB
•
•
•
•
Apply patch # 2873456
Point your APPL_TOP to the 9i DB
Full system import or
Two phased import process where structure is imported first
and data is subsequently imported.
• Index builds are always last with either of the import choice
• Post import activities
– Gather schema statistics
– Modify TNS names on Admin and CM nodes to point to 9i DB
Upgrade Process - Overview
• Customized Oracle Upgrade Approach
– Converting to RAC DB
• Modify ORACM configuration file
• Follow Metalink documentation
– Adding Second Application Server
• Change the application server name to load balancer
• Review H/W load balancer setup
• TAR and UNTAR the APPL_TOP onto second node
– Adding Second CM Server
• Create second CM node by cloning from first CM
• Setup Internal Monitor (ICM)
• Define Primary & Secondary nodes for all Concurrent
Managers
• Define the transaction managers for all the instances in RAC
cluster
Upgrade Takeaways
• Total Production downtime of 98 hours
• Perform as many iterations as possible
• 12 practice iterations were performed
• Impact of hardware
• Import process needs lot of attention
• Impact of Orgs and Languages
The instance
Upgradeplanning
– Category
0 will need them!
• Be generous with your
– you
• Refine backup methodology, to provide total database backups as
rapidly as technically achievable
• Perform Validate APPS Schema script on production regularly and fix
the issues
• Make time for high quality testing of both standard and custom Oracle
Applications code
– 20 One off patches applied on consolidated patch 2
– Perform load test and regression testing
Infrastructure Issues
• Instance Planning
– Differences from Traditional Unix to Linux platforms
– Impact of server numbers
– Don’t forget the disk space!
• Important considerations
–
–
–
–
Data Archiving
Getting it all to work! e.g. load balancing, DNS etc.
Log/out files, UTL, FTP
Individual server memory footprint
Global Team Tactics
• Global Project technical, functional and support teams
• Managing communication
• Coping with time zones
– Make it work to your advantage
• Sharing of knowledge
– Client/Vendor sharing
• Examples of success
Agenda Revisited
•
•
•
•
•
•
•
•
•
Who we are
Description of the Goals
Hardware Architecture & Scaling Plan
11i Linux / RAC Architecture
Upgrade Process Overview
Upgrade Takeaways
Infrastructure Issues
Global Team Tactics
Questions & Answers
Q&A
Questions?
Zensar Technologies
Chakri Reddy
[email protected]
Ram Laxmanan
[email protected]