Transcript PIA Load Balancing, JVMs and Garbage Collection

PIA Load Balancing, JVMs and
Garbage Collection
David Kurtz
Go-Faster Consultancy Ltd.
[email protected]
www.go-faster.co.uk
Who am I?
• DBA
• Book
– Independent consultant
• System Performance
tuning
– www.psftdba.com
– PeopleSoft ERP
– Oracle RDBMS
• Training Course
• UK Oracle User Group
– Unix SIG
• Oak Table
– www.oaktable.net
PSFT Tech PUG 5.9.05
www.go-faster.co.uk
2
Resources
• If you can’t hear me say so now.
• Please feel free to ask questions as we go
along.
• The presentation will be available from
– www.go-faster.co.uk
PSFT Tech PUG 5.9.05
www.go-faster.co.uk
3
A War Story
• Two forms of load
balancing
WEB1
50.0.0.50:443
• PSFT Recommendations:
PIA1
Windows Network
Load Balancing
– Windows Network
Load Balancing
– IIS Proxy
– Red Paper HA & Clustering
IIS
50.0.0.7:7001
50.0.0.7:443
PIA2
50.0.0.7:17001
WEB2
PIA1
IIS
50.0.0.8:443
50.0.0.8:7001
PIA2
50.0.0.8:17001
PSFT Tech PUG 5.9.05
www.go-faster.co.uk
4
Problems
• Strange Pauses
– Mostly to do with garbage collection
• IIS Proxy wouldn’t load balance
– All of load on one PIA or the other
• Pauses cause IIS Proxy failover
• On Failover users loose their sessions
PSFT Tech PUG 5.9.05
www.go-faster.co.uk
5
IIS Load Balancing
• iisproxy.ini
WebLogicCluster=50.0.0.8:7001,50.0.0.8:17001
DynamicServerList=OFF
…
PSFT Tech PUG 5.9.05
www.go-faster.co.uk
6
IIS Load Balancing
• Uneven loading of PIAs caused overload of
one PIA.
• Duration of garbage collection with 100%
of load on single PIA much higher
PSFT Tech PUG 5.9.05
www.go-faster.co.uk
7
How do I know the load is
balanced?
• Web server access log
• PeopleSoft Performance Monitor
• PT>=8.44
– Event 150: JVM Status
– Event 152: Web Site Status
– Event 153: Web Servlet Status
PSFT Tech PUG 5.9.05
www.go-faster.co.uk
8
How do I know the load is
balanced?
PIA Activity
hrmsweb001 17001:17002
hrmsweb001 7001:7002
hrmsweb002 17001:17002
hrmsweb002 7001:7002
300
Session in Web-App
250
200
150
100
50
0
Wed 29.06.05 12:00
Wed 29.06.05 13:00
Wed 29.06.05 14:00
Wed 29.06.05 15:00
Time
PSFT Tech PUG 5.9.05
www.go-faster.co.uk
9
JVM Garbage Collection
• Java Command Line Options
– verbosegc
• Writes to weblogic.log
– Xloggc:<filename>
0.000: [GC 46591K->3553K(518464K), 0.1262603 secs]
19.604: [GC 50145K->6662K(518464K), 0.3389070 secs]
70.052: [Full_GC 46565K->10901K(518464K), 0.9507536 secs]
90.967: [Full_GC 55273K->16571K(518464K), 1.1008306 secs]
PSFT Tech PUG 5.9.05
www.go-faster.co.uk
10
Garbage Collector Log
90.967: [Full_GC 55273K->16571K(518464K), 1.1008306 secs]
–
–
–
–
–
Elapsed time since JVM started
Collection type
Size of live objects before and after GC
Total space available, less permanent objects
Duration of collection
PSFT Tech PUG 5.9.05
www.go-faster.co.uk
11
Timestamps on GC log
-XX:+PrintGCTimeStamps
• But I couldn’t get it to work!
PSFT Tech PUG 5.9.05
www.go-faster.co.uk
12
Other Garbage Collection
Options
• -XX:+UseConcMarkSweepGC
• -XX:+UseParNewGC
• -XX:+CMSParallelRemarkEnabled
•
•
Caveat: These options are not presented as being any kind of panacea, but
something to consider. You need to do your own testing.
BTW: I got the idea for this from a UKOUG Oracle Application Server SIG!
PSFT Tech PUG 5.9.05
www.go-faster.co.uk
13
-XX:+UseConcMarkSweepGC
• Concurrent Low Pause Collector
• GC is, as far as possible, concurrent with
the execution of the application.
– Uses a separate thread
• App is paused for shorter periods during GC
• Only effective on multi-processor machines
PSFT Tech PUG 5.9.05
www.go-faster.co.uk
14
-XX:+UseParNewGC
• Parallel young generation GC with
concurrent GC
– for multiprocessor machines,
– enables multi threaded young generation
collection
– Application is not paused
PSFT Tech PUG 5.9.05
www.go-faster.co.uk
15
-XX:+CMSParallelRemarkEnabled
• Reduces remark pauses
PSFT Tech PUG 5.9.05
www.go-faster.co.uk
16
Default Garbage Collector
• Individual GC duration
GC Duration
DURATION
10
1
Duration (s)
0
20000
40000
60000
80000
100000
120000
0.1
0.01
0.001
Offset Seconds
PSFT Tech PUG 5.9.05
www.go-faster.co.uk
17
Parallel Garbage Collector
• Individual GC duration
GC Duration
DURATION
10
1
Duration (s)
0
20000
40000
60000
80000
100000
120000
0.1
0.01
0.001
Offset Seconds
PSFT Tech PUG 5.9.05
www.go-faster.co.uk
18
Default Garbage Collector
• Cumulative GC duration / minute
GC Duration
DURATION
GC Duration seconds/minute
10
1
0
20000
40000
60000
80000
100000
120000
0.1
0.01
Offset Seconds
PSFT Tech PUG 5.9.05
www.go-faster.co.uk
19
Parallel Garbage Collection
• Cumulative GC duration / minute
GC Duration
DURATION
10
GC Duration seconds/minute
1
0
20000
40000
60000
80000
100000
120000
0.1
0.01
0.001
Offset Seconds
PSFT Tech PUG 5.9.05
www.go-faster.co.uk
20
Conclusion
• Improved GC performance does not have a
great effect on Component Performance
• It does reduce the ‘strange pause’ giving the
user a better experience
• Parallel GC seems to slowly leak memory.
– Possible bug in a particular JVM
– I have only tested on Windows
PSFT Tech PUG 5.9.05
www.go-faster.co.uk
21
Questions?
PSFT Tech PUG 5.9.05
www.go-faster.co.uk
22
PIA Load Balancing, JVMs and
Garbage Collection
David Kurtz
Go-Faster Consultancy Ltd.
[email protected]
www.go-faster.co.uk