Transcript Slide 1

<Insert Picture Here>
Garbage Collection Mythbusters
Simon Ritter
Java Technology Evangelist
The Goal
Cover the strengths and weaknesses of
garbage collection
What GC does well
And what not so well
33
Tracing GC: A Refresher Course
• Tracing-based garbage collectors:
• Discover the live objects
• All objects transitively reachable from a set of “roots”
(“roots” - known live references that exist outside the
heap, e.g., thread stacks, virtual machine data)
• Deduce that the rest are dead
• Reclaim them
• An “indirect” GC technique
• Examples
• Mark & Sweep
• Mark & Compact
• Copying
44
Tracing GC Example
Runtime
Stack
Hea
p
A
B
C
G
K
D
H
E
I
L
F
J
M
55
Tracing GC Example
Runtime
Stack
Hea
p
A
B
C
G
K
D
H
E
I
L
F
J
M
The Runtime Stack is considered live by default. We start
tracing transitively from it and mark objects we reach.
66
Tracing GC Example
Runtime
Stack
Heap
A
B
C
G
K
D
H
E
I
L
F
J
M
77
Tracing GC Example
Runtime
Stack
Heap
A
B
C
G
K
D
H
E
I
L
F
J
M
88
Tracing GC Example
Runtime
Stack
Heap
A
B
C
G
K
D
H
E
I
L
F
J
M
We identified all reachable objects. We can deduce that
the rest are unreachable and, therefore, dead.
99
Alternative 1: In-place Deallocation
Runtime
Stack
Heap
A
C
K
H
B
E
I
L
Keep track of the free space explicit (using free lists,
a buddy system, a bitmap, etc.).
10
10
Alternative 2: Sliding Compaction
Runtime
Stack
Heap
A
C
K
H
B
E
I
L
Slide all live objects to one end of the Heap. All free space
is located at the other end of the heap.
11
11
Copying GC Example
Heap (From-Space)
Runtime
Stack
A
C
G
D
H
E
I
B
K
L
Heap (To-Space)
12
12
Copying GC Example
Heap (From-Space)
Runtime
Stack
A
B
C
G
D
H
E
I
K
L
Heap (To-Space)
A
C
13
13
Copying GC Example
Heap (From-Space)
Runtime
Stack
A
B
C
G
K
D
H
E
I
L
H
K
Heap (To-Space)
A
C
14
14
Copying GC Example
Heap (From-Space)
Runtime
Stack
A
B
C
G
K
D
H
E
I
L
H
K
Heap (To-Space)
A
C
L
B
15
15
Copying GC Example
Heap (From-Space)
Runtime
Stack
A
B
C
G
D
H
E
I
L
K
Heap (To-Space)
A
C
H
K
L
B
E
I
16
16
Copying GC Example
Heap (From-Space)
Runtime
Stack
Heap (To-Space)
A
C
H
K
L
B
E
I
17
17
Myth 1:
malloc/free always perform better than
GC.
18
18
Object Relocation
• GC enables object relocation, which in turn enables
• Compaction: eliminates fragmentation
• Generational GC: decreases GC overhead
• Linear Allocation: best allocation performance
• Fast path: ~10 native instructions, inlined, no sync
top
new top
en
d
new object
used space
free space
19
19
Generational GC is Fast!
• Compare costs (first-order approximation)
• malloc/free:
• all_objects * costmalloc + freed_objects * costfree
• Generational GC with copying young generation:
• all_objects * costlinear_alloc + surviving_objects * costcopy
• Consider:
• costlinear_alloc much less than costmalloc
• surviving_objects often 5% or less of all_objects
20
20
GC vs. malloc Study
• Recent publication shows
• When space is tight
• malloc/free outperform GC
• When space is ample
• GC can match (or better) malloc/free
• GC just as fast
• if given “breathing room”
• Matthew Hertz and Emery Berger
• Quantifying the Performance of Garbage Collection vs.
Explicit Memory Management, In Proceedings of OOPSLA
2005, October 2005
21
21
Object Relocation: Other Benefits
• Compaction: can improve page locality
• Fewer TLB misses
• Cluster objects to improve locality
• Important on NUMA architectures
• Relocation ordering: can improve cache locality
• Fewer cache misses
• The important points:
• Allocation and reclamation are fast
• Relocation can boost application performance
22
22
Myth 1:
malloc/free always perform better than
GC.
23
23
Myth 1:
malloc/free always perform better than
GC.
24
24
Myth 2:
Reference counting would solve all my
GC problems.
25
25
Reference Counting
• Each object holds a count
• How many references point to it
• Increment it when a new reference points to the object
• Decrement it when a reference to the object is dropped
• When reference count reaches 0
• Object is unreachable
• It can be reclaimed
• A “direct” GC technique
26
26
Reference Counting Example
Runtime
Stack
Hea
p
1
A
2
1
C
1
1
B
1
1
E
F
1
1
K
H
I
J
2
1
L
M
27
27
Reference Counting Example
Runtime
Stack
Hea
p
1
A
Delete reference,
Decrease H's RC
2
1
C
1
1
B
1
1
E
F
1
1
K
H
I
J
2
1
L
M
28
28
Reference Counting Example
Runtime
Stack
Hea
p
1
A
2
1
C
0
1
B
1
1
E
F
1
1
K
H
I
J
2
1
L
M
29
29
Reference Counting Example
Runtime
Stack
Hea
p
1
A
Decrease K's & L's RCs,
Reclaim H
2
1
C
0
1
B
1
1
E
F
1
1
K
H
I
J
2
1
L
M
30
30
Reference Counting Example
Runtime
Stack
Hea
p
1
A
1
B
2
1
1
0
C
E
F
1
1
I
J
1
1
K
L
M
31
31
Reference Counting Example
Reclaim K
Runtime
Stack
Hea
p
1
A
1
B
2
1
1
0
C
E
F
1
1
I
J
1
1
K
L
M
32
32
Reference Counting Example
Runtime
Stack
Hea
p
1
A
1
B
2
1
1
C
E
F
1
1
I
J
1
1
L
M
33
33
Traditional Reference Counting
• Extra space overhead
• One reference count per object
• Extra time overhead
• Up to two reference count updates per reference field update
• Very expensive in a multi-threaded environment
• Non-moving
• Fragmentation
• Not always incremental or prompt
• Garbage cycles
• Counts never reach 0
• Cannot be reclaimed
34
34
Reference Counting Example
Runtime
Stack
Heap
1
1
A
B
2
1
1
C
E
F
1
1
I
J
1
1
L
M
Objects F and J form a garbage cycle and also retain M too.
35
35
Advanced Reference Counting
• Two-bit reference counts
• Most objects pointed to by one or two references
• When max count (3) is reached
• Object becomes “sticky”
• Buffer reference updates
• Apply them in bulk
• Combine with copying GC
• Use a backup GC algorithm
• Handle cyclic garbage
• Deal with “sticky” objects
• Typically, the cyclic GC is a tracing GC
• Complex, and still non-moving
36
36
Myth 2:
Reference counting would solve all my
GC problems.
37
37
Myth 2:
Reference counting would solve all my
GC problems.
38
38
Myth 3:
GC with explicit deallocation would
drastically improve performance.
39
39
GC with Explicit Deallocation?
• Philosophically
• Would compromise safety
• Practically
• Not all GC algorithms can support it
• Mark-Compact & Copying GCs
• Do not maintain free lists
• Reclaim space by moving live objects
• Overwrite reclaimed objects
• No way to reuse space from a single object
• Unless the object is at the end of the heap
40
40
GC with Explicit Deallocation? (ii)
• Explicit deallocation is incompatible with this model
• Would compromise the very fast allocation path
• GCs have a different reclamation pattern
• Reclaim objects in bulk
• Free-space management is optimized for that
• Also applies to static analysis techniques
• They can prove that an object can be safely deallocated
• …but there is no mechanism to do the deallocation!
41
41
How to deallocate?
• How can we deallocate the dead object when we only
maintain top?
top
en
d
dead object
used space
free space
42
42
Myth 3:
GC with explicit deallocation would
drastically improve performance.
43
43
Myth 3:
GC with explicit deallocation would
drastically improve performance.
44
44
Myth 4:
Finalizers can (and should) be called as
soon as objects become unreachable.
45
45
Finalizers
• Typical use of Finalizers:
• Reclaim external resources associated with objects in heap
• e.g., native GUI components (windows, color maps, etc.)
• Finalizers are called on objects that GC has found to
be garbage
• Tracing GC
• Does not always have liveness information for every object
• Liveness information up to date only at certain points
• Immediately after a tracing cycle
• Must finish a tracing cycle to find finalizable objects
46
46
Finalization Reality Check
• Finalizers are not like C++ destructors
• No guarantees
•
•
•
•
When they will be run
Which thread will run them
Which will run first, second, … last
Or that they will be run at all!
• If you want prompt external resource reclamation
• Don't rely on finalizers
• Dispose explicity instead
• Use finalization as a safety net
47
47
Myth 4:
Finalizers can (and should) be called as
soon as objects become unreachable.
49
49
Myth 4:
Finalizers can (and should) be called as
soon as objects become unreachable.
50
50
Myth 5:
Garbage collection eliminates all
memory leaks
51
51
Unused Reachable Objects
• Consider the following code:
class ImageMap {
private Map<File, Image> map;
public void add(File file, Image img) {
map.put(file, img);
}
public Image get(File file){ return map.get(file); }
public void remove(File file) { map.remove(file); }
}
static ImageMap imageMap;
…
File f = new File(imageFileName);
Image img = readImage(f);
imageMap.add(f, img);
f = null;
52
52
GC and Memory Leaks
• Consider the (f, img) tuple in the previous example
• After we null f, the tuple is unused
• We cannot retrieve it (don't have the key any more)
• We cannot remove it (don't have the key any more)
• So (f, img) will take up space while imageMap is alive
• … without the application being able to access it
• GC reclaims unreachable objects
• But not unused objects that are reachable
• And it cannot know when a reachable object is unused
53
53
GC and Memory Leaks (ii)
• Effort required to track down such leaks
• Can't override malloc anymore
• Tools are needed to help
• Heap population statistics (what is being retained?)
• Reachability information (why is it being retained?)
54
54
Myth 5:
Garbage collection eliminates all
memory leaks.
55
55
Myth 5:
Garbage collection eliminates all
memory leaks.
56
56
Myth 6:
I can get a GC that delivers very high
throughput and very low latency.
57
57
Throughput vs. Latency
• For most applications, GC overhead is small
• 2% – 5%
• Throughput GCs
• Move most work to GC pauses
• Application threads do as little as possible
• Least overall GC overhead
• Low-latency GCs
• Move work out of GC pauses
• Application threads do more work
• Bookkeeping for GC more expensive
• More overall GC overhead
58
58
Throughput vs. Latency (ii)
• Goals are conflicting
• GCs are architected differently
• One GC does not rule them all
• Must choose the best GC for the job
• Also consider another dimension:
• Footprint
• Why can't the VM choose the right GC?
• Impossible to know application priorities
• Hints may help
• …but for now, human must decide
59
59
Myth 6:
I can get a GC that delivers very high
throughput and very low latency.
60
60
Myth 6:
I can get a GC that delivers very high
throughput and very low latency.
61
61
Myth 7:
I need to disable GC in critical
sections of my code.
62
62
Why Disable GC?
• Application has a critical deadline
• Display a video frame
• Complete a stock trade
• Adjust nuclear reactor control rods
• GC pause may cause deadline to be missed
• Jittery video, missed profit, boom!
• So simply ...
• Disable GC
• Run without interruptions
• Viola! Meet your deadline
63
63
Coding Without GC
• GC typically occurs because heap is full / nearly full
• No GC → no allocation
• Code in critical section cannot allocate safely
• Possible solution: Allocate in advance
• Only access pre-allocated objects
• Prohibit allocations during the critical section
• How? Throw exception? Code analysis?
• Must know exactly what data is needed
• Before entering critical section
• Must audit every change to critical section
64
64
Using Libraries
• Libraries freely allocate objects
• And with good reason
• Clear programming model
• Lack of side-effects good for concurrency
• Can you always avoid using them in critical sections?
•
•
•
•
Concatenate two strings
Use the concurrency libraries
Add a new element to a collection
etc.
65
65
A Few More Problems
• Other threads
• Cannot allocate either
• Stop them all?
• What if they hold a lock you need? → deadlock
• Overlapping critical sections
• Can never do GC!!!
• Abuse
• Long / unpredictable critical sections
• blocking I/O,waiting for a lock, etc.
• Libraries that have critical sections
• We have seen many libraries that call System.gc()
• -XX:+DisableCriticalSections?
66
66
The Bottom Line
• It might work in very few, limited cases
• Not a general-purpose solution
• Too many ways to shoot yourself in the foot
• You should really consider looking at the RTSJ
• Real-Time Specification for Java
• But it also has a lot of the same problems we just described
67
67
Myth 7:
I need to disable GC in critical
sections of my code.
68
68
Myth 7:
I need to disable GC in critical
sections of my code.
69
69
Myth 8:
GC settings that worked for my last
app will also work for my next app.
70
70
What Affects GC Performance?
• Application behavior
• Allocation rate
• Higher allocation rate → more frequent GCs
• Live data size
• More live data → longer tracing cycles
• Mutation rate
• Higher mutation rate → more load on the write barriers,
hence more load on incremental GCs
• Hardware
• Number of cores, clock rate, total RAM, cache sizes
• GC tuning parameters
71
71
What Affects GC Performance? (ii)
• Primary factors
• App behavior, hardware, tuning parameters
• Keep those factors constant
• GC should have consistent performance
• Change any of them …
• Examples:
• Increase object lifetimes → increase GC time and/or
frequency
• Increase average object size → increase copying costs
• Move to faster hardware → allocate more objects
• GC performance will change, too
72
72
GC Tuning Parameters
• Yet, we often see ...
• Customers move tuning parameters from one app to another
• Transferring parameters
• Mixed results at best
• Sometimes it works
• Maybe when defaults are really bad!
• Usually it doesn't
• Mostly luck
• Performance often left on the table
• If applications are very similar
• e.g., version 2 of the same app
• Use previous tuning parameters only as a starting point
73
73
Guaranteed GC Performance?
• In theory: yes
• Real-Time GCs are available
• But they require strict bounds on application characteristics
• Modern applications
• Very large, very complex, very dynamic
• Virtually impossible to analyze them
• At least, to get realistic bounds
• At best, approximations based on testing
• Realistically: no hard real-time guarantees
• For non-trivial applications
• Soft real-time at best…
74
74
Myth 8:
GC settings that worked for my last
app will also work for my next app.
75
75
Myth 8:
GC settings that worked for my last
app will also work for my next app.
76
76
Myth 9:
77
77
Myth 9:
This talk is over.
78
78
Myth 9:
This talk is over.
79
79
The preceding is intended to outline our general
product direction. It is intended for information
purposes only, and may not be incorporated into any
contract. It is not a commitment to deliver any
material, code, or functionality, and should not be
relied upon in making purchasing decisions.
The development, release, and timing of any
features or functionality described for Oracle’s
products remains at the sole discretion of Oracle.
80
80