Dueling Segmented LRU Replacement Algorithm

Hongliang Gao Chris Wilkerson

The Basic Ideas

• Auxiliary Directory: – Evaluates “dueling” replacement algorithms.

• Segmented LRU list: – Reference bit protects lines with good locality.

– Aging/ Random Promotion.

• Adaptive Bypass: – Protect cache contents by bypassing the cache completely.

Dueling Replacement Algos

Auxiliary Directory

Set0 Set1 Set2 Set3 Set4 Set5 Set6 Set7

Tag Array Saturating Counter

• 32 sets sampled (static) • 2 policies evaluated in each sampled set.

• 16-bit mini-tags • Counter updated when policies differ.

Review of Segmented LRU

SLRU: Reference Bit 4 LRU bits per line track LRU position Tag

•

Reference bit is marked when a line is referenced.

•

Replace any non-referenced lines first.

•

Replace global LRU if all lines are referenced.

SLRU Features

• •

Random Promotion

–

Reference bit is marked when referenced or when randomly promoted.

–

Eg: 1/32 newly allocated lines may randomly be selected for promotion.

Aging

– –

Reference bits can be cleared as well as set. Line allocations cause the reference bit of the LRU line to be cleared.

Adaptive Bypass

•

Misses result in allocation or bypass.

Data Structure Cache Thrashing on 4 th w/o Bypass way No Thrashing w/ Bypass

•

Bypass based on a random probability.

–

Eg: 1, 1/2, 1/4, … 1/4096.

–

Probability is doubled/halved according to the success of previous bypasses.

SLRU w/ Adaptive Bypassing

SLRU: Reference Bit 1 1

• De-allocated line tracked by partial tag.

• Allocated line tracked by 4 bit pointer.

• Valid Bit • Virtual Bypass Bit 16 bit partial tag for “out-of-cache” competitor 4 bit pointer for “in-cache competitor”

Frequency of Bypass

100.00% 80.00% 60.00% 40.00% 20.00% 0.00% 401.bzip2

403.gcc429.mcf

434.zeusmp

400.perlbench

P0_BYPASSED P1_BYPASSED 473.astar

471.omnetpp

481.wrf

482.sphinx3

483.xala

ncb Benchmark mk average

DSB impact on MPKI vs TLRU 70 60 50 40 MPKI for true LRU % reduction MPKI w/ DSB 30 20 10 0 -10 mk et pp x3 429.mcf

450.soplex

471.omn

483.xalancb

482.sphin

447.dealII

434.zeu

sm p 401.bz

ip 2 403.gcc

481.wrf

464.h264ref

400.perlbench

er gm ean 70 60 50 40 30 20 10 0 -10

Speedup 1.8

1.6

0.6

0.4

0.2

0 1.4

1.2

1 0.8

429.mcf

450.soplex

ncb mk 483.xala

447.dealII

482.sphinx3

401.bzip2

473.astar

481.wrf

434.zeusmp

464.h264ref

400.perlbench

DSB NoAge NoByp gm ean

BACKUP

SLRU w/ Adaptive Bypassing

SLRU: Reference Bit

• Bypass • Bypassed line tracked by partial tag.

1 0

• Incumbent line tracked by 4 bit pointer.

• Subsequent reference to bypass line reduces bypass probability.

16 bit partial tag for “out-of-cache” competitor • Subsequent reference 4 bit pointer for “in-cache competitor” to incumbent increases bypass probability.

Enable bypassing for policy0 Enable bypassing for policy1 Random promotion probability for policy0 Random promotion probability for policy1 Aging for policy0 Aging for policy1 Virtual bypassing probability Initial bypassing probability Second minimum bypassing probability (minimum is 0) 0 0 CONFIG 1 True False 0 1 16 64 1/256 CONFIG 2 True True 0 0 0 1 8 64 1/4096 CONFIG 3 True True 0 16 0 1 8 8 1/4096 Config2: 2 Policies

Set0 Set1 Set2 Set3 Set4 Set5 Set6 auxiliary directory collects statistics replacement policy performance and updates a policy selector counter.

SLRU

1-reference bit indicates whether each line is in the reference or non-reference list.

4 LRU bits per line track LRU position Set7

Tag Array

valid bits Tracking bypass 4 bit pointer for “in-cache competitor”