Transcript Lecture 1: Introduction

Lecture 3: Dynamic ILP
CS 505: Computer Architecture
Spring 2005
Thu D. Nguyen
Recall from Pipelining Lecture
Pipeline CPI = Ideal pipeline CPI + Structural Stalls +
Data Hazard Stalls + Control Stalls
Ideal pipeline CPI: measure of the maximum performance
attainable by the implementation
Structural hazards: HW cannot support this combination of
instructions
Data hazards: Instruction depends on result of prior
instruction still in the pipeline
Control hazards: Caused by delay between the fetching of
instructions and decisions about changes in control flow
(branches and jumps)
Computer Science, Rutgers
2
CS 505: Computer Structures
Data Dependence and Hazards
Dependences are a property of programs
Presence of dependence indicates potential for a
hazard, but actual hazard and length of any stall is a
property of the pipeline
Importance of the data dependencies
1) indicates the possibility of a hazard
2) determines order in which results must be calculated
3) sets an upper bound on how much parallelism can possibly be
exploited
Today looking at HW schemes to avoid hazard
Computer Science, Rutgers
3
CS 505: Computer Structures
Dealing with Hazards
In-order issue, in-order execution
Stall
Forwarding
Avoid WAR and WAW by only write in last stage
In-order issue, out-of-order execution
Why?
Dynamic scheduling with Scoreboarding
Computer Science, Rutgers
4
CS 505: Computer Structures
Ideas to Reduce Stalls
Chapter 3
Chapter 4
Technique
Dynamic scheduling
Dynamic branch
prediction
Issuing multiple
instructions per cycle
Speculation
Dynamic memory
disambiguation
Loop unrolling
Basic compiler pipeline
scheduling
Compiler dependence
analysis
Software pipelining and
trace scheduling
Compiler speculation
Computer Science, Rutgers
5
Reduces
Data hazard stalls
Control stalls
Ideal CPI
Data and control stalls
Data hazard stalls involving
memory
Control hazard stalls
Data hazard stalls
Ideal CPI and data hazard stalls
Ideal CPI and data hazard stalls
Ideal CPI, data and control stalls
CS 505: Computer Structures
Instruction-Level Parallelism (ILP)
Basic Block (BB) ILP is quite small
BB: a straight-line code sequence with no branches in except to the
entry and no branches out except at the exit
average dynamic branch frequency 15% to 25%
=> 4 to 7 instructions execute between a pair of branches
Plus instructions in BB likely to depend on each other
To obtain substantial performance enhancements, we must exploit
ILP across multiple basic blocks
Simplest: loop-level parallelism to exploit parallelism among
iterations of a loop
Vector is one way
If not vector, then either dynamic via branch prediction or static via
loop unrolling by compiler
We will cover branch prediction but not until next time
Computer Science, Rutgers
6
CS 505: Computer Structures
ILP and Data Hazards
Program order: order instructions would execute in if executed
sequentially 1 at a time as determined by original source program
HW/SW goal: exploit parallelism while preserving appearance of
program order
modify order in manner than cannot be observed by program
must not affect the outcome of the program
Ex: Instructions involved in a name dependence can execute
simultaneously if name used in instructions is changed so
instructions do not conflict
Register renaming resolves name dependence for regs
Either by compiler or by HW
add r1, r2, r3
sub r2, r4,r5
and r3, r2, 1
Computer Science, Rutgers
7
CS 505: Computer Structures
Control Dependencies
Every instruction is control dependent on some set of
branches, and, in general, these control dependencies
must be preserved to preserve program order
if p1 {
S1;
};
if p2 {
S2;
}
S1 is control dependent on p1, and S2 is control
dependent on p2 but not on p1.
Computer Science, Rutgers
8
CS 505: Computer Structures
Control Dependence Ignored
Control dependence need not always be preserved
Willing to execute instructions that should not have been
executed, thereby violating the control dependences, if can do
so without affecting correctness of the program
Instead, 2 properties critical to program correctness
are exception behavior and data flow
Computer Science, Rutgers
9
CS 505: Computer Structures
Exception Behavior
Preserving exception behavior => any changes in
instruction execution order must not change how
exceptions are raised in program (=> no new exceptions)
Example:
DADDU
BEQZ
LW
L1:
R2,R3,R4
R2,L1
R1,0(R2)
Problem with moving LW before BEQZ?
Computer Science, Rutgers
10
CS 505: Computer Structures
Data Flow
Data flow: actual flow of data values among instructions
that produce results and those that consume them
branches make flow dynamic, determine which instruction is
supplier of data
Example:
L:
DADDUR1,R2,R3
BEQZ R4,L
DSUBUR1,R5,R6
…
OR
R7,R1,R8
OR depends on DADDU or DSUBU?
Must preserve data flow on execution
Computer Science, Rutgers
11
CS 505: Computer Structures
Advantages of Dynamic Scheduling
Handles cases when dependences unknown at compile
time
(e.g., because they may involve a memory reference)
It simplifies the compiler
Allows code that compiled for one pipeline to run
efficiently on a different pipeline
Hardware speculation, a technique with significant
performance advantages, that builds on dynamic
scheduling
Computer Science, Rutgers
12
CS 505: Computer Structures
HW Schemes: Instruction Parallelism
Key idea: Allow instructions behind stall to proceed
DIVD
ADDD
SUBD
F0,F2,F4
F10,F0,F8
F12,F8,F14
Enables out-of-order execution
and allows out-of-order completion
Will distinguish when an instruction begins execution
and when it completes execution; between 2 times, the
instruction is in execution
In a dynamically scheduled pipeline, all instructions pass
through issue stage in order (in-order issue)
Computer Science, Rutgers
13
CS 505: Computer Structures
Dynamic Scheduling Step 1
Simple pipeline had 1 stage to check both structural and
data hazards: Instruction Decode (ID), also called
Instruction Issue
Split the ID pipe stage of simple 5-stage pipeline into 2
stages:
Issue—Decode instructions, check for structural
hazards
Read operands—Wait until no data hazards, then read
operands
Computer Science, Rutgers
14
CS 505: Computer Structures
A Dynamic Algorithm: Tomasulo’s
Algorithm
For IBM 360/91 (before caches!)
Goal: High Performance without special compilers
Small number of floating point registers (4 in 360)
prevented interesting compiler scheduling of operations
This led Tomasulo to try to figure out how to get more
effective registers — renaming in hardware!
Why Study 1966 Computer?
The descendants of this have flourished!
Alpha 21264, HP 8000, MIPS 10000, Pentium III, PowerPC
604, …
Computer Science, Rutgers
15
CS 505: Computer Structures
Tomasulo Algorithm
Control & buffers distributed with Function Units (FU)
FU buffers called “reservation stations”; have pending
operands
Registers in instructions replaced by values or pointers to
reservation stations (RS);
form of register renaming ;
avoids WAR, WAW hazards
More reservation stations than registers, so can do
optimizations compilers can’t
Results to FU from RS, not through registers, over Common Data
Bus that broadcasts results to all FUs
Load and Stores treated as FUs with RSs as well
Computer Science, Rutgers
16
CS 505: Computer Structures
Tomasulo Organization
FP Registers
From Mem
FP Op
Queue
Load Buffers
Load1
Load2
Load3
Load4
Load5
Load6
Store
Buffers
Add1
Add2
Add3
Mult1
Mult2
FP adders
Computer Science, Rutgers
Reservation
Stations
To Mem
FP multipliers
Common Data Bus (CDB)
17
CS 505: Computer Structures
Reservation Station Components
Op:
Operation to perform in the unit (e.g., + or –)
Vj, Vk: Value of Source operands
Store buffers has V field, result to be stored
Qj, Qk: Reservation stations producing source registers (value to
be written)
Note: Qj,Qk=0 => ready
Store buffers only have Qi for RS producing result
Busy:
Indicates reservation station or FU is busy
Register result status—Indicates which functional unit will write
each register, if one exists. Blank when no pending instructions
that will write that register.
Computer Science, Rutgers
18
CS 505: Computer Structures
Three Stages of Tomasulo Algorithm
1.Issue—get instruction from FP Op Queue
If reservation station free (no structural hazard),
control issues instr & sends operands (renames registers).
2.Execute—operate on operands (EX)
When both operands ready then execute;
if not ready, watch Common Data Bus for result
3.Write result—finish execution (WB)
Write on Common Data Bus to all awaiting units;
mark reservation station available
Common data bus: data + source (“come from” bus)
64 bits of data + 4 bits of Functional Unit source address
Write if matches expected Functional Unit (produces result)
Does the broadcast
Example speed: 3 clocks for Fl .pt. +,-; 10 for * ; 40 clks for /
Computer Science, Rutgers
19
CS 505: Computer Structures
Tomasulo Example
Instruction stream
Instruction status:
Instruction
LD
F6
LD
F2
MULTD F0
SUBD
F8
DIVD
F10
ADDD
F6
j
34+
45+
F2
F6
F0
F8
k
R2
R3
F4
F2
F6
F2
Exec Write
Issue Comp Result
Load1
Load2
Load3
Register result status:
Clock
0
Clock cycle
counter
No
No
No
3 Load/Buffers
Reservation Stations:
Time Name Busy
Add1
No
Add2
No
FU count
Add3
No
down
Mult1 No
Mult2 No
Busy Address
Op
S1
Vj
S2
Vk
RS
Qj
RS
Qk
3 FP Adder R.S.
2 FP Mult R.S.
F0
F2
F4
F6
F8
F10
F12
...
FU
Computer Science, Rutgers
20
CS 505: Computer Structures
F30
Tomasulo Example Cycle 1
Instruction status:
Instruction
LD
F6
LD
F2
MULTD F0
SUBD
F8
DIVD
F10
ADDD
F6
j
34+
45+
F2
F6
F0
F8
k
R2
R3
F4
F2
F6
F2
Exec Write
Issue Comp Result
1
Reservation Stations:
Time Name Busy
Add1
No
Add2
No
Add3
No
Mult1 No
Mult2 No
Register result status:
Clock
1
Load1
Load2
Load3
Op
S1
Vj
S2
Vk
RS
Qj
RS
Qk
F0
F2
F4
F6
F8
FU
Computer Science, Rutgers
Busy Address
Yes
No
No
34+R2
F10
F12
...
Load1
21
CS 505: Computer Structures
F30
Tomasulo Example Cycle 2
Instruction status:
Instruction
LD
F6
LD
F2
MULTD F0
SUBD
F8
DIVD
F10
ADDD
F6
j
34+
45+
F2
F6
F0
F8
k
R2
R3
F4
F2
F6
F2
Exec Write
Issue Comp Result
1
2
Reservation Stations:
Time Name Busy
Add1
No
Add2
No
Add3
No
Mult1 No
Mult2 No
Register result status:
Clock
2
FU
Busy Address
Load1
Load2
Load3
Op
S1
Vj
S2
Vk
RS
Qj
RS
Qk
F0
F2
F4
F6
F8
Load2
Yes
Yes
No
34+R2
45+R3
F10
F12
...
Load1
Note: Can have multiple loads outstanding
Computer Science, Rutgers
22
CS 505: Computer Structures
F30
Tomasulo Example Cycle 3
Instruction status:
Instruction
LD
F6
LD
F2
MULTD F0
SUBD
F8
DIVD
F10
ADDD
F6
j
34+
45+
F2
F6
F0
F8
k
R2
R3
F4
F2
F6
F2
Exec Write
Issue Comp Result
1
2
3
Reservation Stations:
Time Name Busy Op
Add1
No
Add2
No
Add3
No
Mult1 Yes MULTD
Mult2 No
Register result status:
Clock
3
FU
Computer Science, Rutgers
F0
Busy Address
3
S1
Vj
Load1
Load2
Load3
S2
Vk
RS
Qj
Yes
Yes
No
34+R2
45+R3
F10
F12
RS
Qk
R(F4) Load2
F2
F4
Mult1 Load2
F6
F8
...
Load1
23
CS 505: Computer Structures
F30
Tomasulo Example Cycle 3
Note: registers names are removed (“renamed”) in
Reservation Stations; MULT issued
Load1 completing; what is waiting for Load1?
Computer Science, Rutgers
24
CS 505: Computer Structures
Tomasulo Example Cycle 4
Instruction status:
Instruction
LD
F6
LD
F2
MULTD F0
SUBD
F8
DIVD
F10
ADDD
F6
j
34+
45+
F2
F6
F0
F8
k
R2
R3
F4
F2
F6
F2
Exec Write
Issue Comp Result
1
2
3
4
Reservation Stations:
Busy Address
3
4
4
Load1
Load2
Load3
S1
Vj
S2
Vk
RS
Qj
RS
Qk
F2
F4
F6
F8
No
Yes
No
45+R3
F10
F12
Time Name Busy Op
Add1 Yes SUBD M(A1)
Load2
Add2
No
Add3
No
Mult1 Yes MULTD
R(F4) Load2
Mult2 No
Register result status:
Clock
4
FU
F0
Mult1 Load2
...
M(A1) Add1
• Load2 completing; what is waiting for Load2?
Computer Science, Rutgers
25
CS 505: Computer Structures
F30
Tomasulo Example Cycle 5
Instruction status:
Instruction
LD
F6
LD
F2
MULTD F0
SUBD
F8
DIVD
F10
ADDD
F6
j
34+
45+
F2
F6
F0
F8
k
R2
R3
F4
F2
F6
F2
Exec Write
Issue Comp Result
1
2
3
4
5
Reservation Stations:
Busy Address
3
4
4
5
Load1
Load2
Load3
S1
Vj
S2
Vk
RS
Qj
RS
Qk
F2
F4
F6
F8
Time Name Busy Op
2 Add1 Yes SUBD M(A1) M(A2)
Add2
No
Add3
No
10 Mult1 Yes MULTD M(A2) R(F4)
Mult2 Yes DIVD
M(A1) Mult1
Register result status:
Clock
5
FU
F0
Mult1 M(A2)
No
No
No
F10
F12
...
M(A1) Add1 Mult2
• Timer starts down for Add1, Mult1
Computer Science, Rutgers
26
CS 505: Computer Structures
F30
Tomasulo Example Cycle 6
Instruction status:
Instruction
LD
F6
LD
F2
MULTD F0
SUBD
F8
DIVD
F10
ADDD
F6
j
34+
45+
F2
F6
F0
F8
k
R2
R3
F4
F2
F6
F2
Exec Write
Issue Comp Result
1
2
3
4
5
6
Reservation Stations:
Busy Address
3
4
4
5
Load1
Load2
Load3
S1
Vj
S2
Vk
RS
Qj
RS
Qk
F2
F4
F6
F8
Time Name Busy Op
1 Add1 Yes SUBD M(A1) M(A2)
Add2 Yes ADDD
M(A2) Add1
Add3
No
9 Mult1 Yes MULTD M(A2) R(F4)
Mult2 Yes DIVD
M(A1) Mult1
Register result status:
Clock
6
FU
F0
Mult1 M(A2)
Add2
No
No
No
F10
F12
...
Add1 Mult2
• Issue ADDD here despite name dependency on F6?
Computer Science, Rutgers
27
CS 505: Computer Structures
F30
Tomasulo Example Cycle 7
Instruction status:
Instruction
LD
F6
LD
F2
MULTD F0
SUBD
F8
DIVD
F10
ADDD
F6
j
34+
45+
F2
F6
F0
F8
k
R2
R3
F4
F2
F6
F2
Exec Write
Issue Comp Result
1
2
3
4
5
6
Reservation Stations:
3
4
Busy Address
4
5
Load1
Load2
Load3
7
S1
Vj
S2
Vk
RS
Qj
RS
Qk
F2
F4
F6
F8
Time Name Busy Op
0 Add1 Yes SUBD M(A1) M(A2)
Add2 Yes ADDD
M(A2) Add1
Add3
No
8 Mult1 Yes MULTD M(A2) R(F4)
Mult2 Yes DIVD
M(A1) Mult1
Register result status:
Clock
7
FU
F0
No
No
No
Mult1 M(A2)
Add2
F10
F12
...
Add1 Mult2
• Add1 (SUBD) completing; what is waiting for it?
Computer Science, Rutgers
28
CS 505: Computer Structures
F30
Tomasulo Example Cycle 8
Instruction status:
Instruction
LD
F6
LD
F2
MULTD F0
SUBD
F8
DIVD
F10
ADDD
F6
j
34+
45+
F2
F6
F0
F8
k
R2
R3
F4
F2
F6
F2
Exec Write
Issue Comp Result
1
2
3
4
5
6
Reservation Stations:
Busy Address
3
4
4
5
Load1
Load2
Load3
7
8
S1
Vj
S2
Vk
RS
Qj
RS
Qk
F2
F4
F6
F8
Time Name Busy Op
Add1
No
2 Add2 Yes ADDD (M-M) M(A2)
Add3
No
7 Mult1 Yes MULTD M(A2) R(F4)
Mult2 Yes DIVD
M(A1) Mult1
Register result status:
Clock
8
FU
Computer Science, Rutgers
F0
Mult1 M(A2)
No
No
No
F10
F12
...
Add2 (M-M) Mult2
29
CS 505: Computer Structures
F30
Tomasulo Example Cycle 9
Instruction status:
Instruction
LD
F6
LD
F2
MULTD F0
SUBD
F8
DIVD
F10
ADDD
F6
j
34+
45+
F2
F6
F0
F8
k
R2
R3
F4
F2
F6
F2
Exec Write
Issue Comp Result
1
2
3
4
5
6
Reservation Stations:
Busy Address
3
4
4
5
Load1
Load2
Load3
7
8
S1
Vj
S2
Vk
RS
Qj
RS
Qk
F2
F4
F6
F8
Time Name Busy Op
Add1
No
1 Add2 Yes ADDD (M-M) M(A2)
Add3
No
6 Mult1 Yes MULTD M(A2) R(F4)
Mult2 Yes DIVD
M(A1) Mult1
Register result status:
Clock
9
FU
Computer Science, Rutgers
F0
Mult1 M(A2)
No
No
No
F10
F12
...
Add2 (M-M) Mult2
30
CS 505: Computer Structures
F30
Tomasulo Example Cycle 10
Instruction status:
Instruction
LD
F6
LD
F2
MULTD F0
SUBD
F8
DIVD
F10
ADDD
F6
j
34+
45+
F2
F6
F0
F8
k
R2
R3
F4
F2
F6
F2
Exec Write
Issue Comp Result
1
2
3
4
5
6
Reservation Stations:
3
4
4
5
7
8
Busy Address
Load1
Load2
Load3
10
S1
Vj
S2
Vk
RS
Qj
RS
Qk
F2
F4
F6
F8
Time Name Busy Op
Add1
No
0 Add2 Yes ADDD (M-M) M(A2)
Add3
No
5 Mult1 Yes MULTD M(A2) R(F4)
Mult2 Yes DIVD
M(A1) Mult1
Register result status:
Clock
10
FU
F0
No
No
No
Mult1 M(A2)
F10
F12
...
Add2 (M-M) Mult2
• Add2 (ADDD) completing; what is waiting for it?
Computer Science, Rutgers
31
CS 505: Computer Structures
F30
Tomasulo Example Cycle 11
Instruction status:
Instruction
LD
F6
LD
F2
MULTD F0
SUBD
F8
DIVD
F10
ADDD
F6
j
34+
45+
F2
F6
F0
F8
k
R2
R3
F4
F2
F6
F2
Exec Write
Issue Comp Result
1
2
3
4
5
6
Reservation Stations:
Busy Address
3
4
4
5
Load1
Load2
Load3
7
8
10
11
S1
Vj
S2
Vk
RS
Qj
RS
Qk
F2
F4
F6
F8
Time Name Busy Op
Add1
No
Add2
No
Add3
No
4 Mult1 Yes MULTD M(A2) R(F4)
Mult2 Yes DIVD
M(A1) Mult1
Register result status:
Clock
11
FU
F0
Mult1 M(A2)
No
No
No
F10
F12
...
(M-M+M)(M-M) Mult2
• Write result of ADDD here?
• All quick instructions complete in this cycle!
Computer Science, Rutgers
32
CS 505: Computer Structures
F30
Tomasulo Example Cycle 12
Instruction status:
Instruction
LD
F6
LD
F2
MULTD F0
SUBD
F8
DIVD
F10
ADDD
F6
j
34+
45+
F2
F6
F0
F8
k
R2
R3
F4
F2
F6
F2
Exec Write
Issue Comp Result
1
2
3
4
5
6
Reservation Stations:
Busy Address
3
4
4
5
Load1
Load2
Load3
7
8
10
11
S1
Vj
S2
Vk
RS
Qj
RS
Qk
F2
F4
F6
F8
Time Name Busy Op
Add1
No
Add2
No
Add3
No
3 Mult1 Yes MULTD M(A2) R(F4)
Mult2 Yes DIVD
M(A1) Mult1
Register result status:
Clock
12
FU
Computer Science, Rutgers
F0
Mult1 M(A2)
No
No
No
F10
F12
...
(M-M+M)(M-M) Mult2
33
CS 505: Computer Structures
F30
Tomasulo Example Cycle 13
Instruction status:
Instruction
LD
F6
LD
F2
MULTD F0
SUBD
F8
DIVD
F10
ADDD
F6
j
34+
45+
F2
F6
F0
F8
k
R2
R3
F4
F2
F6
F2
Exec Write
Issue Comp Result
1
2
3
4
5
6
Reservation Stations:
Busy Address
3
4
4
5
Load1
Load2
Load3
7
8
10
11
S1
Vj
S2
Vk
RS
Qj
RS
Qk
F2
F4
F6
F8
Time Name Busy Op
Add1
No
Add2
No
Add3
No
2 Mult1 Yes MULTD M(A2) R(F4)
Mult2 Yes DIVD
M(A1) Mult1
Register result status:
Clock
13
FU
Computer Science, Rutgers
F0
Mult1 M(A2)
No
No
No
F10
F12
...
(M-M+M)(M-M) Mult2
34
CS 505: Computer Structures
F30
Tomasulo Example Cycle 14
Instruction status:
Instruction
LD
F6
LD
F2
MULTD F0
SUBD
F8
DIVD
F10
ADDD
F6
j
34+
45+
F2
F6
F0
F8
k
R2
R3
F4
F2
F6
F2
Exec Write
Issue Comp Result
1
2
3
4
5
6
Reservation Stations:
Busy Address
3
4
4
5
Load1
Load2
Load3
7
8
10
11
S1
Vj
S2
Vk
RS
Qj
RS
Qk
F2
F4
F6
F8
Time Name Busy Op
Add1
No
Add2
No
Add3
No
1 Mult1 Yes MULTD M(A2) R(F4)
Mult2 Yes DIVD
M(A1) Mult1
Register result status:
Clock
14
FU
Computer Science, Rutgers
F0
Mult1 M(A2)
No
No
No
F10
F12
...
(M-M+M)(M-M) Mult2
35
CS 505: Computer Structures
F30
Tomasulo Example Cycle 15
Instruction status:
Instruction
LD
F6
LD
F2
MULTD F0
SUBD
F8
DIVD
F10
ADDD
F6
j
34+
45+
F2
F6
F0
F8
k
R2
R3
F4
F2
F6
F2
Exec Write
Issue Comp Result
1
2
3
4
5
6
Reservation Stations:
Busy Address
3
4
15
7
4
5
Load1
Load2
Load3
10
11
S1
Vj
S2
Vk
RS
Qj
RS
Qk
F2
F4
F6
F8
8
Time Name Busy Op
Add1
No
Add2
No
Add3
No
0 Mult1 Yes MULTD M(A2) R(F4)
Mult2 Yes DIVD
M(A1) Mult1
Register result status:
Clock
15
FU
F0
No
No
No
Mult1 M(A2)
F10
F12
...
(M-M+M)(M-M) Mult2
• Mult1 (MULTD) completing; what is waiting for it?
Computer Science, Rutgers
36
CS 505: Computer Structures
F30
Tomasulo Example Cycle 16
Instruction status:
Instruction
LD
F6
LD
F2
MULTD F0
SUBD
F8
DIVD
F10
ADDD
F6
j
34+
45+
F2
F6
F0
F8
k
R2
R3
F4
F2
F6
F2
Exec Write
Issue Comp Result
1
2
3
4
5
6
Reservation Stations:
3
4
15
7
4
5
16
8
Load1
Load2
Load3
10
11
S1
Vj
S2
Vk
RS
Qj
RS
Qk
F2
F4
F6
F8
Time Name Busy Op
Add1
No
Add2
No
Add3
No
Mult1 No
40 Mult2 Yes DIVD M*F4 M(A1)
Register result status:
Clock
16
FU
F0
Busy Address
M*F4 M(A2)
No
No
No
F10
F12
...
(M-M+M)(M-M) Mult2
• Just waiting for Mult2 (DIVD) to complete
Computer Science, Rutgers
37
CS 505: Computer Structures
F30
Faster than light computation
(skip a couple of cycles)
Computer Science, Rutgers
38
CS 505: Computer Structures
Tomasulo Example Cycle 55
Instruction status:
Instruction
LD
F6
LD
F2
MULTD F0
SUBD
F8
DIVD
F10
ADDD
F6
j
34+
45+
F2
F6
F0
F8
k
R2
R3
F4
F2
F6
F2
Exec Write
Issue Comp Result
1
2
3
4
5
6
Reservation Stations:
3
4
15
7
4
5
16
8
Load1
Load2
Load3
10
11
S1
Vj
S2
Vk
RS
Qj
RS
Qk
F2
F4
F6
F8
Time Name Busy Op
Add1
No
Add2
No
Add3
No
Mult1 No
1 Mult2 Yes DIVD M*F4 M(A1)
Register result status:
Clock
55
FU
Computer Science, Rutgers
F0
Busy Address
M*F4 M(A2)
No
No
No
F10
F12
...
(M-M+M)(M-M) Mult2
39
CS 505: Computer Structures
F30
Tomasulo Example Cycle 56
Instruction status:
Instruction
LD
F6
LD
F2
MULTD F0
SUBD
F8
DIVD
F10
ADDD
F6
j
34+
45+
F2
F6
F0
F8
k
R2
R3
F4
F2
F6
F2
Exec Write
Issue Comp Result
1
2
3
4
5
6
Reservation Stations:
3
4
15
7
56
10
4
5
16
8
Load1
Load2
Load3
S1
Vj
S2
Vk
RS
Qj
RS
Qk
56
FU
F0
F2
F4
F6
F8
No
No
No
11
Time Name Busy Op
Add1
No
Add2
No
Add3
No
Mult1 No
0 Mult2 Yes DIVD M*F4 M(A1)
Register result status:
Clock
Busy Address
M*F4 M(A2)
F10
F12
...
(M-M+M)(M-M) Mult2
• Mult2 (DIVD) is completing; what is waiting for it?
Computer Science, Rutgers
40
CS 505: Computer Structures
F30
Tomasulo Example Cycle 57
Instruction status:
Instruction
LD
F6
LD
F2
MULTD F0
SUBD
F8
DIVD
F10
ADDD
F6
j
34+
45+
F2
F6
F0
F8
k
R2
R3
F4
F2
F6
F2
Exec Write
Issue Comp Result
1
2
3
4
5
6
Reservation Stations:
3
4
15
7
56
10
4
5
16
8
57
11
Load1
Load2
Load3
S1
Vj
S2
Vk
RS
Qj
RS
Qk
F2
F4
F6
F8
Time Name Busy Op
Add1
No
Add2
No
Add3
No
Mult1 No
Mult2 Yes DIVD M*F4 M(A1)
Register result status:
Clock
56
FU
F0
Busy Address
M*F4 M(A2)
No
No
No
F10
F12
...
(M-M+M)(M-M) Result
• Once again: In-order issue, out-of-order execution
and out-of-order completion.
Computer Science, Rutgers
41
CS 505: Computer Structures
F30
Tomasulo Drawbacks
Complexity
Many associative stores (CDB) at high speed
Performance limited by Common Data Bus
Each CDB must go to multiple functional units
high capacitance, high wiring density
Number of functional units that can complete per cycle limited
to one!
Multiple CDBs  more FU logic for parallel assoc stores
Non-precise interrupts!
We will address this later
Computer Science, Rutgers
42
CS 505: Computer Structures
Tomasulo Loop Example
Loop:
LD
F0
0
R1
MULTD
F4
F0
F2
SD
F4
0
R1
SUBI
R1
R1
#8
BNEZ
R1
Loop
This time assume Multiply takes 4 clocks
Assume 1st load takes 8 clocks
(L1 cache miss), 2nd load takes 1 clock (hit)
To be clear, will show clocks for SUBI, BNEZ
Show 2 iterations
Computer Science, Rutgers
43
CS 505: Computer Structures
Loop Example
Instruction status:
ITER Instruction
1
1
1
Iter2
ation 2
Count 2
LD
MULTD
SD
LD
MULTD
SD
F0
F4
F4
F0
F4
F4
j
k
0
F0
0
0
F0
0
R1
F2
R1
R1
F2
R1
Reservation Stations:
Time
Name Busy
Add1
No
Add2
No
Add3
No
Mult1 No
Mult2 No
Op
Vj
Exec Write
Issue CompResult
Busy Addr
Load1
Load2
Load3
Store1
Store2
Store3
S1
Vk
S2
Qj
RS
Qk
Code:
LD
MULTD
SD
SUBI
BNEZ
No
No
No
No
No
No
Added Store Buffers
F0
F4
F4
R1
R1
Register result status
Clock
0
F0
R1
80
F2
F4
F6
F8
Fu
F10 F12
0
F0
0
R1
Loop
R1
F2
R1
#8
...
F30
Instruction Loop
Fu
Value of Register used for address, iteration control
Computer Science, Rutgers
44
CS 505: Computer Structures
Loop Example Cycle 1
Instruction status:
ITER Instruction
1
LD
F0
j
k
0
R1
1
Vj
S1
Vk
Reservation Stations:
Time
Name Busy
Add1
No
Add2
No
Add3
No
Mult1 No
Mult2 No
Exec Write
Issue CompResult
Op
S2
Qj
RS
Qk
Busy Addr
Fu
Load1
Load2
Load3
Store1
Store2
Store3
Yes
No
No
No
No
No
80
Code:
LD
MULTD
SD
SUBI
BNEZ
F0
F4
F4
R1
R1
0
F0
0
R1
Loop
R1
F2
R1
#8
...
F30
Register result status
Clock
1
F0
R1
80
F2
F4
F6
F8
F10 F12
Fu Load1
Computer Science, Rutgers
45
CS 505: Computer Structures
Loop Example Cycle 2
Instruction status:
ITER Instruction
1
1
LD
MULTD
F0
F4
j
k
0
F0
R1
F2
1
2
Vj
S1
Vk
Reservation Stations:
Time
Exec Write
Issue CompResult
Name Busy Op
Add1
No
Add2
No
Add3
No
Mult1 Yes Multd
Mult2 No
S2
Qj
RS
Qk
R(F2) Load1
Busy Addr
Fu
Load1
Load2
Load3
Store1
Store2
Store3
Yes
No
No
No
No
No
80
Code:
LD
MULTD
SD
SUBI
BNEZ
F0
F4
F4
R1
R1
0
F0
0
R1
Loop
R1
F2
R1
#8
...
F30
Register result status
Clock
2
F0
R1
80
Fu Load1
Computer Science, Rutgers
F2
F4
F6
F8
F10 F12
Mult1
46
CS 505: Computer Structures
Loop Example Cycle 3
Instruction status:
ITER Instruction
1
1
1
LD
MULTD
SD
F0
F4
F4
j
k
0
F0
0
R1
F2
R1
Reservation Stations:
Time
Name Busy Op
Add1
No
Add2
No
Add3
No
Mult1 Yes Multd
Mult2 No
Vj
Exec Write
Issue CompResult
1
2
3
S1
Vk
S2
Qj
RS
Qk
R(F2) Load1
Busy Addr
Fu
Load1
Load2
Load3
Store1
Store2
Store3
Yes
No
No
Yes
No
No
80
80
Mult1
Code:
LD
MULTD
SD
SUBI
BNEZ
F0
F4
F4
R1
R1
0
F0
0
R1
Loop
R1
F2
R1
#8
...
F30
Register result status
Clock
3
R1
80
F0
Fu Load1
F2
F4
F6
F8
F10 F12
Mult1
Implicit
renaming sets up data
flow graph
Computer Science, Rutgers
47
CS 505: Computer Structures
Loop Example Cycle 4
Instruction status:
ITER Instruction
1
1
1
LD
MULTD
SD
F0
F4
F4
j
k
0
F0
0
R1
F2
R1
Reservation Stations:
Time
Name Busy Op
Add1
No
Add2
No
Add3
No
Mult1 Yes Multd
Mult2 No
Vj
Exec Write
Issue CompResult
1
2
3
S1
Vk
S2
Qj
RS
Qk
R(F2) Load1
Busy Addr
Fu
Load1
Load2
Load3
Store1
Store2
Store3
Yes
No
No
Yes
No
No
80
80
Mult1
Code:
LD
MULTD
SD
SUBI
BNEZ
F0
F4
F4
R1
R1
0
F0
0
R1
Loop
R1
F2
R1
#8
...
F30
Register result status
Clock
4
F0
R1
80
Fu Load1
F2
F4
F6
F8
F10 F12
Mult1
Dispatching SUBI Instruction (not in FP queue)
Computer Science, Rutgers
48
CS 505: Computer Structures
Loop Example Cycle 5
Instruction status:
ITER Instruction
1
1
1
LD
MULTD
SD
F0
F4
F4
j
k
0
F0
0
R1
F2
R1
Reservation Stations:
Time
Name Busy Op
Add1
No
Add2
No
Add3
No
Mult1 Yes Multd
Mult2 No
Vj
Exec Write
Issue CompResult
1
2
3
S1
Vk
S2
Qj
RS
Qk
R(F2) Load1
Busy Addr
Fu
Load1
Load2
Load3
Store1
Store2
Store3
Yes
No
No
Yes
No
No
80
80
Mult1
Code:
LD
MULTD
SD
SUBI
BNEZ
F0
F4
F4
R1
R1
0
F0
0
R1
Loop
R1
F2
R1
#8
...
F30
Register result status
Clock
5
F0
R1
72
Fu Load1
F2
F4
F6
F8
F10 F12
Mult1
And, BNEZ instruction (not in FP queue)
Computer Science, Rutgers
49
CS 505: Computer Structures
Loop Example Cycle 6
Instruction status:
ITER Instruction
1
1
1
2
LD
MULTD
SD
LD
F0
F4
F4
F0
j
k
0
F0
0
0
R1
F2
R1
R1
1
2
3
6
Vj
S1
Vk
Reservation Stations:
Time
Exec Write
Issue CompResult
Name Busy Op
Add1
No
Add2
No
Add3
No
Mult1 Yes Multd
Mult2 No
S2
Qj
RS
Qk
R(F2) Load1
Busy Addr
Fu
Load1
Load2
Load3
Store1
Store2
Store3
Yes
Yes
No
Yes
No
No
80
72
80
Mult1
Code:
LD
MULTD
SD
SUBI
BNEZ
F0
F4
F4
R1
R1
0
F0
0
R1
Loop
R1
F2
R1
#8
...
F30
Register result status
Clock
6
F0
R1
72
Fu Load2
F2
F4
F6
F8
F10 F12
Mult1
Notice that F0 never sees Load from location 80
Computer Science, Rutgers
50
CS 505: Computer Structures
Loop Example Cycle 7
Instruction status:
ITER Instruction
1
1
1
2
2
LD
MULTD
SD
LD
MULTD
F0
F4
F4
F0
F4
j
k
0
F0
0
0
F0
R1
F2
R1
R1
F2
1
2
3
6
7
Vj
S1
Vk
Reservation Stations:
Time
Exec Write
Issue CompResult
Name Busy Op
Add1
No
Add2
No
Add3
No
Mult1 Yes Multd
Mult2 Yes Multd
S2
Qj
RS
Qk
R(F2) Load1
R(F2) Load2
Busy Addr
Fu
Load1
Load2
Load3
Store1
Store2
Store3
Yes
Yes
No
Yes
No
No
80
72
80
Mult1
Code:
LD
MULTD
SD
SUBI
BNEZ
F0
F4
F4
R1
R1
0
F0
0
R1
Loop
R1
F2
R1
#8
...
F30
Register result status
Clock
7
F0
R1
72
Fu Load2
Computer Science, Rutgers
F2
F4
F6
F8
F10 F12
Mult2
51
CS 505: Computer Structures
Loop Example Cycle 7
Register file completely detached from computation
First and Second iteration completely overlapped
Computer Science, Rutgers
52
CS 505: Computer Structures
Loop Example Cycle 8
Instruction status:
ITER Instruction
1
1
1
2
2
2
LD
MULTD
SD
LD
MULTD
SD
F0
F4
F4
F0
F4
F4
j
k
0
F0
0
0
F0
0
R1
F2
R1
R1
F2
R1
1
2
3
6
7
8
Vj
S1
Vk
Reservation Stations:
Time
Exec Write
Issue CompResult
Name Busy Op
Add1
No
Add2
No
Add3
No
Mult1 Yes Multd
Mult2 Yes Multd
S2
Qj
RS
Qk
R(F2) Load1
R(F2) Load2
Busy Addr
Fu
Load1
Load2
Load3
Store1
Store2
Store3
Yes
Yes
No
Yes
Yes
No
80
72
80
72
Mult1
Mult2
Code:
LD
MULTD
SD
SUBI
BNEZ
F0
F4
F4
R1
R1
0
F0
0
R1
Loop
R1
F2
R1
#8
...
F30
Register result status
Clock
8
F0
R1
72
Fu Load2
Computer Science, Rutgers
F2
F4
F6
F8
F10 F12
Mult2
53
CS 505: Computer Structures
Loop Example Cycle 9
Instruction status:
ITER Instruction
1
1
1
2
2
2
LD
MULTD
SD
LD
MULTD
SD
F0
F4
F4
F0
F4
F4
j
k
0
F0
0
0
F0
0
R1
F2
R1
R1
F2
R1
1
2
3
6
7
8
9
Vj
S1
Vk
S2
Qj
Reservation Stations:
Time
Exec Write
Issue CompResult
Name Busy Op
Add1
No
Add2
No
Add3
No
Mult1 Yes Multd
Mult2 Yes Multd
RS
Qk
R(F2) Load1
R(F2) Load2
Busy Addr
Fu
Load1
Load2
Load3
Store1
Store2
Store3
Yes
Yes
No
Yes
Yes
No
80
72
80
72
Mult1
Mult2
Code:
LD
MULTD
SD
SUBI
BNEZ
F0
F4
F4
R1
R1
0
F0
0
R1
Loop
R1
F2
R1
#8
...
F30
Register result status
Clock
9
F0
R1
72
Fu Load2
F2
F4
F6
F8
F10 F12
Mult2
Load1 completing: who is waiting? Note: Dispatching SUBI
Computer Science, Rutgers
54
CS 505: Computer Structures
Loop Example Cycle 10
Instruction status:
ITER Instruction
1
1
1
2
2
2
LD
MULTD
SD
LD
MULTD
SD
F0
F4
F4
F0
F4
F4
j
k
0
F0
0
0
F0
0
R1
F2
R1
R1
F2
R1
Reservation Stations:
Time
4
Exec Write
Issue CompResult
1
2
3
6
7
8
S1
Vk
9
10
10
S2
Qj
Name Busy Op
Vj
Add1
No
Add2
No
Add3
No
Mult1 Yes Multd M[80] R(F2)
Mult2 Yes Multd
R(F2) Load2
RS
Qk
Busy Addr
Load1
Load2
Load3
Store1
Store2
Store3
No
Yes
No
Yes
Yes
No
Code:
LD
MULTD
SD
SUBI
BNEZ
F0
F4
F4
R1
R1
Fu
72
80
72
Mult1
Mult2
0
F0
0
R1
Loop
R1
F2
R1
#8
...
F30
Register result status
Clock
10
F0
R1
64
Fu Load2
F2
F4
F6
F8
F10 F12
Mult2
Load2 completing: who is waiting? Note: Dispatching BNEZ
Computer Science, Rutgers
55
CS 505: Computer Structures
Loop Example Cycle 11
Instruction status:
ITER Instruction
1
1
1
2
2
2
LD
MULTD
SD
LD
MULTD
SD
F0
F4
F4
F0
F4
F4
j
k
0
F0
0
0
F0
0
R1
F2
R1
R1
F2
R1
Reservation Stations:
Time
3
4
Exec Write
Issue CompResult
1
2
3
6
7
8
S1
Vk
Name Busy Op
Vj
Add1
No
Add2
No
Add3
No
Mult1 Yes Multd M[80] R(F2)
Mult2 Yes Multd M[72] R(F2)
9
10
10
11
S2
Qj
RS
Qk
Busy Addr
Load1
Load2
Load3
Store1
Store2
Store3
No
No
Yes
Yes
Yes
No
Code:
LD
MULTD
SD
SUBI
BNEZ
F0
F4
F4
R1
R1
64
80
72
Fu
Mult1
Mult2
0
F0
0
R1
Loop
R1
F2
R1
#8
...
F30
Register result status
Clock
11
F0
R1
64
F2
Fu Load3
F4
F6
F8
F10 F12
Mult2
Next load in sequence
Computer Science, Rutgers
56
CS 505: Computer Structures
Loop Example Cycle 12
Instruction status:
ITER Instruction
1
1
1
2
2
2
LD
MULTD
SD
LD
MULTD
SD
F0
F4
F4
F0
F4
F4
j
k
0
F0
0
0
F0
0
R1
F2
R1
R1
F2
R1
Reservation Stations:
Time
2
3
Exec Write
Issue CompResult
1
2
3
6
7
8
S1
Vk
Name Busy Op
Vj
Add1
No
Add2
No
Add3
No
Mult1 Yes Multd M[80] R(F2)
Mult2 Yes Multd M[72] R(F2)
9
10
10
11
S2
Qj
RS
Qk
Busy Addr
Load1
Load2
Load3
Store1
Store2
Store3
No
No
Yes
Yes
Yes
No
Code:
LD
MULTD
SD
SUBI
BNEZ
F0
F4
F4
R1
R1
64
80
72
Fu
Mult1
Mult2
0
F0
0
R1
Loop
R1
F2
R1
#8
...
F30
Register result status
Clock
12
F0
R1
64
Fu Load3
F2
F4
F6
F8
F10 F12
Mult2
Why not issue third multiply?
Computer Science, Rutgers
57
CS 505: Computer Structures
Loop Example Cycle 13
Instruction status:
ITER Instruction
1
1
1
2
2
2
LD
MULTD
SD
LD
MULTD
SD
F0
F4
F4
F0
F4
F4
j
k
0
F0
0
0
F0
0
R1
F2
R1
R1
F2
R1
Reservation Stations:
Time
1
2
Exec Write
Issue CompResult
1
2
3
6
7
8
S1
Vk
Name Busy Op
Vj
Add1
No
Add2
No
Add3
No
Mult1 Yes Multd M[80] R(F2)
Mult2 Yes Multd M[72] R(F2)
9
10
10
11
S2
Qj
RS
Qk
Busy Addr
Load1
Load2
Load3
Store1
Store2
Store3
No
No
Yes
Yes
Yes
No
Code:
LD
MULTD
SD
SUBI
BNEZ
F0
F4
F4
R1
R1
64
80
72
Fu
Mult1
Mult2
0
F0
0
R1
Loop
R1
F2
R1
#8
...
F30
Register result status
Clock
13
F0
R1
64
Fu Load3
F2
F4
F6
F8
F10 F12
Mult2
Why not issue third store?
Computer Science, Rutgers
58
CS 505: Computer Structures
Loop Example Cycle 14
Instruction status:
ITER Instruction
1
1
1
2
2
2
LD
MULTD
SD
LD
MULTD
SD
F0
F4
F4
F0
F4
F4
j
k
0
F0
0
0
F0
0
R1
F2
R1
R1
F2
R1
Reservation Stations:
Time
0
1
Exec Write
Issue CompResult
1
2
3
6
7
8
9
14
10
11
S1
Vk
S2
Qj
RS
Qk
Name Busy Op
Vj
Add1
No
Add2
No
Add3
No
Mult1 Yes Multd M[80] R(F2)
Mult2 Yes Multd M[72] R(F2)
10
Busy Addr
Load1
Load2
Load3
Store1
Store2
Store3
No
No
Yes
Yes
Yes
No
Code:
LD
MULTD
SD
SUBI
BNEZ
F0
F4
F4
R1
R1
64
80
72
Fu
Mult1
Mult2
0
F0
0
R1
Loop
R1
F2
R1
#8
...
F30
Register result status
Clock
14
F0
R1
64
Fu Load3
F2
F4
F6
F8
F10 F12
Mult2
Mult1 completing. Who is waiting?
Computer Science, Rutgers
59
CS 505: Computer Structures
Loop Example Cycle 15
Instruction status:
ITER Instruction
1
1
1
2
2
2
LD
MULTD
SD
LD
MULTD
SD
F0
F4
F4
F0
F4
F4
j
k
0
F0
0
0
F0
0
R1
F2
R1
R1
F2
R1
Reservation Stations:
Time
0
Exec Write
Issue CompResult
1
2
3
6
7
8
9
14
10
15
11
S1
Vk
S2
Qj
RS
Qk
Name Busy Op
Vj
Add1
No
Add2
No
Add3
No
Mult1 No
Mult2 Yes Multd M[72] R(F2)
10
15
Busy Addr
Load1
Load2
Load3
Store1
Store2
Store3
No
No
Yes
Yes
Yes
No
Code:
LD
MULTD
SD
SUBI
BNEZ
F0
F4
F4
R1
R1
64
80
72
Fu
[80]*R2
Mult2
0
F0
0
R1
Loop
R1
F2
R1
#8
...
F30
Register result status
Clock
15
F0
R1
64
Fu Load3
F2
F4
F6
F8
F10 F12
Mult2
Mult2 completing. Who is waiting?
Computer Science, Rutgers
60
CS 505: Computer Structures
Loop Example Cycle 16
Instruction status:
ITER Instruction
1
1
1
2
2
2
LD
MULTD
SD
LD
MULTD
SD
F0
F4
F4
F0
F4
F4
j
k
0
F0
0
0
F0
0
R1
F2
R1
R1
F2
R1
1
2
3
6
7
8
9
14
10
15
11
16
Vj
S1
Vk
S2
Qj
RS
Qk
Reservation Stations:
Time
4
Exec Write
Issue CompResult
Name Busy Op
Add1
No
Add2
No
Add3
No
Mult1 Yes Multd
Mult2 No
10
15
R(F2) Load3
Busy Addr
Load1
Load2
Load3
Store1
Store2
Store3
No
No
Yes
Yes
Yes
No
Code:
LD
MULTD
SD
SUBI
BNEZ
F0
F4
F4
R1
R1
64
80
72
Fu
[80]*R2
[72]*R2
0
F0
0
R1
Loop
R1
F2
R1
#8
...
F30
Register result status
Clock
16
F0
R1
64
Fu Load3
Computer Science, Rutgers
F2
F4
F6
F8
F10 F12
Mult1
61
CS 505: Computer Structures
Loop Example Cycle 17
Instruction status:
ITER Instruction
1
1
1
2
2
2
LD
MULTD
SD
LD
MULTD
SD
F0
F4
F4
F0
F4
F4
j
k
0
F0
0
0
F0
0
R1
F2
R1
R1
F2
R1
1
2
3
6
7
8
9
14
10
15
11
16
Vj
S1
Vk
S2
Qj
RS
Qk
Reservation Stations:
Time
Exec Write
Issue CompResult
Name Busy Op
Add1
No
Add2
No
Add3
No
Mult1 Yes Multd
Mult2 No
10
15
R(F2) Load3
Busy Addr
Fu
Load1
Load2
Load3
Store1
Store2
Store3
No
No
Yes
Yes
Yes
Yes
64
80
72
64
Code:
LD
MULTD
SD
SUBI
BNEZ
F0
F4
F4
R1
R1
0
F0
0
R1
Loop
R1
F2
R1
#8
...
F30
[80]*R2
[72]*R2
Mult1
Register result status
Clock
17
F0
R1
64
Fu Load3
Computer Science, Rutgers
F2
F4
F6
F8
F10 F12
Mult1
62
CS 505: Computer Structures
Loop Example Cycle 18
Instruction status:
ITER Instruction
1
1
1
2
2
2
LD
MULTD
SD
LD
MULTD
SD
F0
F4
F4
F0
F4
F4
j
k
0
F0
0
0
F0
0
R1
F2
R1
R1
F2
R1
1
2
3
6
7
8
9
14
18
10
15
10
15
Vj
S1
Vk
S2
Qj
RS
Qk
Reservation Stations:
Time
Exec Write
Issue CompResult
Name Busy Op
Add1
No
Add2
No
Add3
No
Mult1 Yes Multd
Mult2 No
11
16
R(F2) Load3
Busy Addr
Fu
Load1
Load2
Load3
Store1
Store2
Store3
No
No
Yes
Yes
Yes
Yes
64
80
72
64
Code:
LD
MULTD
SD
SUBI
BNEZ
F0
F4
F4
R1
R1
0
F0
0
R1
Loop
R1
F2
R1
#8
...
F30
[80]*R2
[72]*R2
Mult1
Register result status
Clock
18
F0
R1
64
Fu Load3
Computer Science, Rutgers
F2
F4
F6
F8
F10 F12
Mult1
63
CS 505: Computer Structures
Loop Example Cycle 19
Instruction status:
ITER Instruction
1
1
1
2
2
2
LD
MULTD
SD
LD
MULTD
SD
F0
F4
F4
F0
F4
F4
j
k
0
F0
0
0
F0
0
R1
F2
R1
R1
F2
R1
1
2
3
6
7
8
9
14
18
10
15
19
10
15
19
11
16
Vj
S1
Vk
S2
Qj
RS
Qk
Reservation Stations:
Time
Exec Write
Issue CompResult
Name Busy Op
Add1
No
Add2
No
Add3
No
Mult1 Yes Multd
Mult2 No
R(F2) Load3
Busy Addr
Load1
Load2
Load3
Store1
Store2
Store3
No
No
Yes
No
Yes
Yes
Code:
LD
MULTD
SD
SUBI
BNEZ
F0
F4
F4
R1
R1
Fu
64
72
64
[72]*R2
Mult1
0
F0
0
R1
Loop
R1
F2
R1
#8
...
F30
Register result status
Clock
19
F0
R1
56
Fu Load3
Computer Science, Rutgers
F2
F4
F6
F8
F10 F12
Mult1
64
CS 505: Computer Structures
Loop Example Cycle 20
Instruction status:
ITER Instruction
1
1
1
2
2
2
LD
MULTD
SD
LD
MULTD
SD
F0
F4
F4
F0
F4
F4
j
k
0
F0
0
0
F0
0
R1
F2
R1
R1
F2
R1
1
2
3
6
7
8
9
14
18
10
15
19
10
15
19
11
16
20
Vj
S1
Vk
S2
Qj
RS
Qk
Reservation Stations:
Time
Exec Write
Issue CompResult
Name Busy Op
Add1
No
Add2
No
Add3
No
Mult1 Yes Multd
Mult2 No
R(F2) Load3
Busy Addr
Fu
Load1
Load2
Load3
Store1
Store2
Store3
Yes
No
Yes
No
No
Yes
56
64
Mult1
Code:
LD
MULTD
SD
SUBI
BNEZ
F0
F4
F4
R1
R1
0
F0
0
R1
Loop
R1
F2
R1
#8
...
F30
64
Register result status
Clock
R1
F0
F2
F4
F6
F8
F10 F12
Fu Load1
Mult1
• Once again: In-order issue, out-of-order execution and out-of-order
completion.
20
56
Computer Science, Rutgers
65
CS 505: Computer Structures
Why can Tomasulo overlap iterations of
loops?
Register renaming
Multiple iterations use different physical destinations for
registers (dynamic loop unrolling).
Reservation stations
Permit instruction issue to advance past integer control flow
operations
Also buffer old values of registers - totally avoiding the WAR
stall that we saw in the scoreboard.
Other perspective: Tomasulo building data flow
dependency graph on the fly.
Computer Science, Rutgers
66
CS 505: Computer Structures
Tomasulo’s scheme offers 2 major
advantages
(1) the distribution of the hazard detection logic
distributed reservation stations and the CDB
If multiple instructions waiting on single result, & each
instruction has other operand, then instructions can be
released simultaneously by broadcast on CDB
If a centralized register file were used, the units would have
to read their results from the registers when register buses
are available.
(2) the elimination of stalls for WAW and WAR hazards
Computer Science, Rutgers
67
CS 505: Computer Structures
What about Precise Exception?
Coming your way soon …
Computer Science, Rutgers
68
CS 505: Computer Structures
Relationship Between Precise Exceptions
and Specultation:
Speculation: guess and check
Important for branch prediction:
Need to “take our best shot” at predicting branch direction.
If we speculate and are wrong, need to back up and
restart execution to point at which we predicted
incorrectly:
This is exactly same as precise exceptions!
Technique for both precise interrupts/exceptions and
speculation: in-order completion or commit
Computer Science, Rutgers
69
CS 505: Computer Structures
HW support for precise interrupts
Need HW buffer for results of
uncommitted instructions: reorder
buffer
3 fields: instr, destination, value
Use reorder buffer number instead of
reservation station when execution
completes
Supplies operands between execution
complete & commit
Reorder
Buffer
FP
Op
Queue
FP Regs
(Reorder buffer can be operand
source => more registers like RS)
Instructions commit
Res Stations
Once instruction commits,
result is put into register
FP Adder
Res Stations
FP Adder
As a result, easy to undo speculated
instructions
on mispredicted branches
or exceptions
Computer Science, Rutgers
70
CS 505: Computer Structures
Four Steps of Speculative Tomasulo
Algorithm
1. Issue—get instruction from FP Op Queue
If reservation station and reorder buffer slot free, issue instr &
send operands & reorder buffer no. for destination (this stage
sometimes called “dispatch”)
2. Execution—operate on operands (EX)
When both operands ready then execute; if not ready, watch
CDB for result; when both in reservation station, execute;
checks RAW (sometimes called “issue”)
3. Write result—finish execution (WB)
Write on Common Data Bus to all awaiting FUs
& reorder buffer; mark reservation station available.
4. Commit—update register with reorder result
When instr. at head of reorder buffer & result present, update
register with result (or store to memory) and remove instr from
reorder buffer. Mispredicted branch flushes reorder buffer
(sometimes called “graduation”)
Computer Science, Rutgers
71
CS 505: Computer Structures
Tomasulo With Reorder buffer:
Done?
FP Op
Queue
ROB7
ROB6
Newest
ROB5
Reorder Buffer
ROB4
ROB3
ROB2
F0
LD F0,10(R2)
Registers
Dest
Computer Science, Rutgers
ROB1
Oldest
To
Memory
from
Memory
Dest
FP adders
N
Reservation
Stations
Dest
1 10+R2
FP multipliers
72
CS 505: Computer Structures
Tomasulo With Reorder buffer:
Done?
FP Op
Queue
ROB7
ROB6
Newest
ROB5
Reorder Buffer
ROB4
ROB3
F10
F0
ADDD F10,F4,F0
LD F0,10(R2)
Registers
Dest
2 ADDD R(F4),ROB1
FP adders
Computer Science, Rutgers
N
N
ROB2
ROB1
Oldest
To
Memory
from
Memory
Dest
Reservation
Stations
Dest
1 10+R2
FP multipliers
73
CS 505: Computer Structures
Tomasulo With Reorder buffer:
Done?
FP Op
Queue
ROB7
ROB6
Newest
ROB5
Reorder Buffer
ROB4
F2
F10
F0
DIVD F2,F10,F6
ADDD F10,F4,F0
LD F0,10(R2)
Registers
Dest
2 ADDD R(F4),ROB1
FP adders
Computer Science, Rutgers
N
N
N
ROB3
ROB2
ROB1
Oldest
To
Memory
Dest
3 DIVD ROB2,R(F6)
Reservation
Stations
from
Memory
Dest
1 10+R2
FP multipliers
74
CS 505: Computer Structures
Tomasulo With Reorder buffer:
Done?
FP Op
Queue
ROB7
Reorder Buffer
F0
F4
-F2
F10
F0
ADDD F0,F4,F6
LD F4,0(R3)
BNE F2,<…>
DIVD F2,F10,F6
ADDD F10,F4,F0
LD F0,10(R2)
Registers
Dest
2 ADDD R(F4),ROB1
6 ADDD ROB5, R(F6)
FP adders
Computer Science, Rutgers
N
N
N
N
N
N
ROB6
Newest
ROB5
ROB4
ROB3
ROB2
ROB1
Oldest
To
Memory
Dest
3 DIVD ROB2,R(F6)
Reservation
Stations
FP multipliers
75
from
Memory
Dest
1 10+R2
5 0+R3
CS 505: Computer Structures
Tomasulo With Reorder buffer:
FP Op
Queue
Reorder Buffer
-- ROB5
F0
F4
-F2
F10
F0
Done?
ST 0(R3),F4
N ROB7
ADDD F0,F4,F6
N ROB6
LD F4,0(R3)
N ROB5
BNE F2,<…>
N ROB4
DIVD F2,F10,F6 N ROB3
ADDD F10,F4,F0 N ROB2
LD F0,10(R2)
N ROB1
Registers
Dest
2 ADDD R(F4),ROB1
6 ADDD ROB5, R(F6)
FP adders
Computer Science, Rutgers
Newest
Oldest
To
Memory
Dest
3 DIVD ROB2,R(F6)
Reservation
Stations
FP multipliers
76
from
Memory
Dest
1 10+R2
6 0+R3
CS 505: Computer Structures
Tomasulo With Reorder buffer:
FP Op
Queue
Reorder Buffer
-- M[10]
F0
F4 M[10]
-F2
F10
F0
Done?
ST 0(R3),F4
Y ROB7
ADDD F0,F4,F6
N ROB6
LD F4,0(R3)
Y ROB5
BNE F2,<…>
N ROB4
DIVD F2,F10,F6 N ROB3
ADDD F10,F4,F0 N ROB2
LD F0,10(R2)
N ROB1
Registers
Dest
2 ADDD R(F4),ROB1
6 ADDD M[10],R(F6)
FP adders
Computer Science, Rutgers
Newest
Oldest
To
Memory
Dest
3 DIVD ROB2,R(F6)
Reservation
Stations
from
Memory
Dest
1 10+R2
FP multipliers
77
CS 505: Computer Structures
Tomasulo With Reorder buffer:
FP Op
Queue
Reorder Buffer
Done?
-- M[10] ST 0(R3),F4
Y ROB7
F0 <val2> ADDD F0,F4,F6 Ex ROB6
F4 M[10] LD F4,0(R3)
Y ROB5
-BNE F2,<…>
N ROB4
F2
DIVD F2,F10,F6 N ROB3
F10
ADDD F10,F4,F0 N ROB2
F0
LD F0,10(R2)
N ROB1
Registers
Dest
2 ADDD R(F4),ROB1
FP adders
Computer Science, Rutgers
Newest
Oldest
To
Memory
Dest
3 DIVD ROB2,R(F6)
Reservation
Stations
from
Memory
Dest
1 10+R2
FP multipliers
78
CS 505: Computer Structures
Tomasulo With Reorder buffer:
FP Op
Queue
Reorder Buffer
What about memory
hazards???
Done?
-- M[10] ST 0(R3),F4
Y ROB7
F0 <val2> ADDD F0,F4,F6 Ex ROB6
F4 M[10] LD F4,0(R3)
Y ROB5
-BNE F2,<…>
N ROB4
F2
DIVD F2,F10,F6 N ROB3
F10
ADDD F10,F4,F0 N ROB2
F0
LD F0,10(R2)
N ROB1
Registers
Dest
2 ADDD R(F4),ROB1
FP adders
Computer Science, Rutgers
Newest
Oldest
To
Memory
Dest
3 DIVD ROB2,R(F6)
Reservation
Stations
from
Memory
Dest
1 10+R2
FP multipliers
79
CS 505: Computer Structures
Memory Disambiguation:
Sorting out RAW Hazards in memory
Question: Given a load that follows a store in
program order, are the two related?
Eg:
st
ld
0(R2),R5
R6,0(R3)
Can we go ahead and start the load early?
Store address could be delayed for a long time by some
calculation that leads to R2 (divide?).
We might want to issue/begin execution of both
operations in same cycle.
Today: Answer is that we are not allowed to start load
until we know that address 0(R2)  0(R3)
Next Week: We might guess at whether or not they are
dependent (called “dependence speculation”) and use
reorder buffer to fixup if we are wrong.
Computer Science, Rutgers
80
CS 505: Computer Structures
Hardware Support for Memory
Disambiguation
Need buffer to keep track of all outstanding stores to memory, in
program order.
Keep track of address (when becomes available) and value (when
becomes available)
FIFO ordering: will retire stores from this buffer in program order
When issuing a load, record current head of store queue (know
which stores are ahead of you).
When have address for load, check store queue:
If any store prior to load is waiting for its address, stall load.
If load address matches earlier store address (associative lookup),
then we have a memory-induced RAW hazard:
store value available  return value
store value not available  return ROB number of source
Otherwise, send out request to memory
Actual stores commit in order, so no worry about WAR/WAW
hazards through memory.
Computer Science, Rutgers
81
CS 505: Computer Structures
Memory Disambiguation:
FP Op
Queue
Done?
ROB7
ROB6
Newest
ROB5
Reorder Buffer
F4
-F0
-- <val 1>
LD
ST
LD
ST
F4, 10(R3)
10(R3), F5
F0,32(R2)
0(R3), F4
Registers
Dest
Computer Science, Rutgers
ROB4
ROB3
ROB2
ROB1
Oldest
To
Memory
from
Memory
Dest
FP adders
N
N
N
Y
Reservation
Stations
FP multipliers
82
Dest
2 32+R2
4 ROB3
CS 505: Computer Structures
Relationship between precise
interrupts and speculation:
Speculation is a form of guessing
Branch prediction, data prediction
If we speculate and are wrong, need to back up and restart execution to
point at which we predicted incorrectly
This is exactly same as precise exceptions!
Branch prediction is a very important
Need to “take our best shot” at predicting branch direction.
If we issue multiple instructions per cycle, lose lots of potential
instructions otherwise:
Consider 4 instructions per cycle
If take single cycle to decide on branch, waste from 4 - 7 instruction slots!
Technique for both precise interrupts/exceptions and speculation:
in-order completion or commit
This is why reorder buffers in all new processors
Computer Science, Rutgers
83
CS 505: Computer Structures
Branch Prediction
Next time …
Computer Science, Rutgers
84
CS 505: Computer Structures
Multiple Issue Superscalar
Until now, trying to reduce CPI to 1 (or ideal CPI)
Can we make CPI < 1?
Issue multiple instructions every clock cycle
Static scheduling vs. dynamic scheduling
VLIW will be covered at end of term if have time
Dynamic scheduling
Check for dependences at issue time
Issue logic can become bottleneck
Restrict combinations of instruction types that can be issued together
Pipeline issue stage further (2 steps of ½ cycle each)
Widen issue logic
Computer Science, Rutgers
85
CS 505: Computer Structures
Summary
Reservations stations: implicit register renaming to larger set of
registers + buffering source operands
Prevents registers as bottleneck
Avoids WAR, WAW hazards of Scoreboard
Allows loop unrolling in HW
Not limited to basic blocks
(integer units gets ahead, beyond branches)
Today, helps cache misses as well
Don’t stall for L1 Data cache miss (insufficient ILP for L2 miss?)
Lasting Contributions
Dynamic scheduling
Register renaming
Load/store disambiguation
360/91 descendants are Pentium III; PowerPC 604; MIPS R10000;
HP-PA 8000; Alpha 21264
Computer Science, Rutgers
86
CS 505: Computer Structures