Transcript Lessons Learned With Oracle9i on Linux for S/390 • Denny Dutcavich • [email protected].

Lessons Learned With
Oracle9i on Linux for S/390
• Denny Dutcavich
• [email protected]
Agenda
• Introduction
– Objectives
– Overview
• Lessons learned
–
–
–
–
–
CPU
Memory
Paging and Swap Space
VM and Linux Guest Setup
Horizontal and Vertical Scaling
–
–
–
–
Async I/O
Storage (disk) considerations
Backup Options
Measurement Tools
– Oracle
• Summary of Key Factors
• New Workshop
Introduction
• The objective of this presentation is to highlight our
experiences with monitoring and performance tuning
gained from:
– The development team at Oracle in preparing for GA
– Early adopter customers
– Testing done at Oracle and IBM Boeblingen
• These topics are covered in the following Redbooks
– Experiences with Oracle9i for Linux for S/390 SG24-6552
– Linux on IBM eServer zSeries Performance Measurement
and Tuning SG24-6926
CPU
• Plan for sufficient CPU resource
– How many Linux guests can run on an IFL?
• Options
– Linux guest under z/VM
– Linux native LPAR, IFL or general CPUs
• CPU Allocation
– Virtual CPUs per constrained guests should equal Real
CPUs
– Never have Virtual CPUs per guest greater than Real
CPUs
– Use QUICKDISP for constrained quests
• Speed Matters
– Similar clock speeds will exhibit similar performance
• Insure proper expectations when consolidating
servers
– Spectrum of performance runs from CPU intensive to I/O
intensive
Operational Choices
z/VM 4.3
MVS01 MVS02 MVS03
MVS04
MVS01
Linux
guests
z/VM 4.3
IFLs
Linux guests
LPARs
Memory
• Plan size of the Linux Guest
– Linux uses all memory available to it
• Allow only for Oracle (or application) and Linux
– Size the Oracle System Global Area (memory
footprint)
• 31-bit Oracle has maximum 1.5GB SGA for both 31bit
and 64bit Linux on zSeries
– Need genksms from 9.2.0.2
• Don’t expect to move a very large SGA into a 256MB
Virtual Machine
– Linux may appear hung if it detects memory
shortage
• KILLMEM process shuts down processes to free memory
– Paging will occur if memory is over-committed
• not a good thing
Paging and Swap Space
• VM paging space
– Configure expanded storage for VM to use as first
paging device
• Configure 25% of physical memory for
expanded storage
• Alter as performance needs are better
understood
• Separate paging space from guest data
– Monitor VM paging
• VM likes contiguous space on paging DASD
Keep paging space at apprx < 50%
Paging and Swap Space
• Linux Swap
– Linux swaps pages when it needs
memory
• Could be double paging
– Configure VDISKS
• Swap space on VDISKs not utilized until
Linux pages
Double Paging
z/VM Expanded Storage
B
3
• Linux completes P.O.
–
Moves page to
swap device
A
B
C
D
3
A
B
C
D
Linux Paging
1
2
•1 Page B paged out by z/VM
–
–
B
.
.
.
•2 Page B paged in by z/VM
–
–
Linux Guest
z/VM Main Memory
Available for P.I.
Linux clueless
Linux needs to page
Page fault accessing Page B
VM and Linux Guest Setup
• Decisions - z/VM or standalone in an LPAR
– Value proposition is to consolidate many servers using the
virtualization capabilities of z/VM
– Linux can be run standalone in an LPAR
• Nothing is free - z/VM has its overhead
– VM overhead 0 - 12% on our tests
– Minidisk I/O uses more VM processing
 Minidisk caching may improve throughput on data reuse
• Think through all the options
VM and Linux Guest Setup
• DASD - sample setup
201
202
203
204
205
206
root
swap
swap
home
tmp
Usr
200-600 CYL
VDISK (2GB)
VDISK (2GB)
200 – 1000 CYL
2700-3000 CYL
Linux Guest Priorities
• Idle Linux guest machine takes resources
– Logoff VM after shutting down Linux
– Timer patch
• Popup blocker for Linux
• Stops Linux from waking up to check for work
– Remove unneeded cron tasks
– Stop unnecessary daemons
• LPAR Weights determine CPU for guests in that LPAR
• Great tuning tips at:
– http://www.vm.ibm.com/perf/tips/#Tune
– http://www.vm.ibm.com/perf/tips/linuxper.html
Scheduling
Dispatch List
Eligible List
Resource Available
or E0
VM Definition
Block D1
VM Definition
Block E1
Time Slice
Expires
Resource Limit
Exceeded
VM Definition
Block Dn
Idle
VM Machine
waiting
VM Machine
ready
VM Definition
Block En
Work To Do
Dormant List
Logoff
Logon
Scaling
• Horizontal scaling
– Add new instances of Linux – a Penguin
colony
– Have sufficient resource
• Vertical scaling
– Increase LPAR weight
– Increase number of real CPUs or IFLs
– Don’t forget to retune Oracle
Async I/O
• Oracle 9i 31bit for Linux for S/390 was built
without support for Asynchronous I/O
• You must put DBWR_IO_SLAVES=4 in
init.ora
– Number could be higher than 4 for very
intensive I/O workloads
• Should not be an issue in future releases of
Oracle
Storage (Disk) Considerations
– ESS 800 should not be treated as a black
box
• Layout of files for database can have huge
effect on performance
• Our tests showed 3x to 4x improvement in
throughput
Storage Configuration
• Best performance will be achieved using:
–
–
–
–
multiple host adapters,
both caches,
more channels,
more CHPIDs and more paths
• Spread data across multiple arrays (ranks)
• FICON Express is better than ESCON
Creating LVMs
• Use Logical Volume Manager to create file systems
for best performance
• Striping critical to get more than one channel used for
the data transfers
– Performance gated by number of CHIPIDs
• File systems in TBytes can be created
– Change extent size
– Default value is 256M
• RedHat 7.2 does not have LVM – use software raid
Backup
• Backup SW used to manage backup sets created with
Oracle RMAN
 FDR Upstream
 Tivioli
 CA
• PPRC can be used for backup (ESS remote mirroring)
or Oracle Advanced Replication feature
Monitoring Performance
• VM has the only accurate view of what
resources each guest is consuming, so any
measurements need to start there.
• Goal of VM is to provide each guest with the
illusion that it's got an entire machine all to
itself.
• A result is that no guest can possibly tell what
real resources it is using, unless it asks VM for
the information.
Performance Monitoring - Tools
• Used VM measurement tools such as
– CP Indicate command (IND)
• Ind user vmlinux2 exp
– CP Monitor facility
– VM Real Time Monitor (RTM)
• Used Linux facilities such as
–
–
–
–
VMSTAT
NETSTAT
IOSTAT and SAR (not on SuSE)
TOP
• Total Storage Expert for ESS
• CA Unicenter
• Velocity Software (ESALPS)
Performance Monitoring – CP Cmds
• INDICATE - Broad overview of how system resources are being
used
• LOCK - Lock in real storage selected pages
• SET SHARE - Control percentage of system resources a guest
receives
• SET QUICKDSP - Designate guests that don't wait in the
eligible list
• SET RESERVED - Set number of pages resident in real
storage
• DEDICATE - Allocate a processor to a guest
Oracle is Oracle is Oracle
• Most Oracle performance problems are
application issues
– Remember to use StatsPack
• Oracle looks the same on Linux for S/390 as it
does on other platforms
• Check init.ora parameter and SGA size
Key Success Factors
• Memory for Linux guest – less is better or
Linux spends it’s time managing memory
• Swap space and expanded storage are
mandatory
• I/O bottlenecks can be avoided by
Using multiple arrays in ESS
LVM with striping
Oracle Database 10g Workshop
A new offering - Dennis Wunder from the Poughkeepsie eServer Briefing Center has scheduled 2 new
workshops in Poughkeepsie, New York. This workshop is intended to help customers gain an understanding
of how to install and manage an Oracle 10g database on Linux on zSeries and z/VM, by providing the basic
skills necessary to launch a Linux guest with Oracle on zSeries.
The workshop is designed with minimal lecture and maximum hands-on, practicing a learning by doing
methodology. Experimentation is encouraged.
Attendees will learn:
Overview of VM and Linux
Overview of Oracle on Linux on zSeries
Installing a Linux guest
Installing Oracle 10g Binaries
*
*
*
*
Measuring the Oracle workload
Etc
The Workshop will be held at the eServer Executive Briefing Center located in Poughkeepsie, N.Y..
The dates selected for the Workshop are as follows:
- June 22 -24
-August 10 - 12
Acknowledgements
• Thanks to the following contributors
– Oracle – Barry Perkins, Andres Lubomirsky, Thomas
Niewel
– IBM BOE- Martin Kammerer, Juergen Doelle
– IBM Storage – Bill Worthington, Pat Blaney, Sandy Abu
– IBM/Oracle team- Bruce Frank, Tom Russell, Kathryn
Arrell, Denny Dutcavich