Transcript ppt

Systems I
Pipelining II
Topics



Pipelining hardware: registers and
feedback paths
Difficulties with pipelines: hazards
Method of mitigating hazards
Adding Pipeline Registers
valE, valM
W_icode, W_valM
Write back
valM
W_valE, W_valM, W_dstE, W_dstM
valM
W
valM
Data
Data
memory
memory
Memory
Memory
Data
Data
memory
memory
M_icode,
M_Bch,
M_valA
Addr, Data
Addr, Data
M
valE
Bch
Bch
CC
CC
Execute
ALU
ALU
valE
CC
CC
Execute
ALU
ALU
aluA, aluB
aluA, aluB
E
valA, valB
Decode
valA, valB
srcA, srcB
dstA, dstB
icode, valC
valP
A
B
Register M
Register
file
file
d_srcA,
d_srcB
Decode
A
B
Register
RegisterM
file
file E
E
Write back
valP
icode, ifun
rA, rB
valC
Fetch
Instruction
Instruction
memory
memory
D
icode, ifun,
rA, rB, valC
PC
PC
increment
increment
Instruction
Instruction
memory
memory
Fetch
PC
PC
valP
valP
PC
PC
increment
increment
predPC
pState
PC
f_PC
F
2
valM
SEQ+ Hardware
data out
read
Data
Data
memory
memory
Mem.
control
Memory
write
Addr


Still sequential
implementation
Reorder PC stage to put at
beginning
Execute
Bch
valE
CC
CC
ALU
ALU
ALU
A
Data
ALU
fun.
ALU
B
PC Stage

Task is to select PC for
current instruction
valA
valB
dstE dstM srcA srcB
dstE dstM srcA srcB

Based on results computed
by previous instruction
Decode

PC is no longer stored in
register
B
Write back
Processor State

A
Register
Register M
file
file E
icode
Fetch
ifun
rA
rB
valC
Instruction
Instruction
memory
memory
But, can determine PC based
on other stored information
valP
PC
PC
increment
increment
PC
PC
PC
pIcode pBch
pValM
pValC
pValP
3
Write back
PIPE- Hardware
W
icode
valE
Mem.
control
write
Pipeline registers hold
intermediate values
from instruction
execution
Forward (Upward) Paths


Values passed from one
stage to next
Cannot jump past
stages
dstE dstM
data out
read

valM
Data
Data
memory
memory
Memory
data in
Addr
M_valA
M_Bch
M
icode
Bch
valE
valA
dstE dstM
e_Bch
Execute
E
ALU
fun.
ALU
ALU
CC
CC
icode ifun
ALU
A
ALU
B
valC
valA
valB
d_srcA d_srcB
Select
A
Decode
d_rvalA
A
dstE dstM srcA srcB
W_valM
B
Register
Register M
file
file
W_valE
E
 e.g., valC passes
through decode
dstE dstM srcA srcB
D
Fetch
icode ifun
rA
rB
Instruction
Instruction
memory
memory
valC
valP
PC
PC
increment
increment
Predict
PC
f_PC
M_valA
Select
PC
F
W_valM
predPC
4
Write back
Feedback Paths
W
icode
valE
dstE dstM
data out
read
Mem.
control
write
Predicted PC
valM
Data
Data
memory
memory
Memory
data in
Addr
M_valA
M_Bch

Guess value of next PC
M
icode
Bch
valE
valA
dstE dstM
e_Bch
Branch information


Jump taken/not-taken
Fall-through or target
address
Execute
E
ALU
fun.
ALU
ALU
CC
CC
icode ifun
ALU
A
ALU
B
valC
valA
valB
dstE dstM srcA srcB
d_srcA d_srcB
Return point

Decode
To register file write
ports
d_rvalA
A
dstE dstM srcA srcB
W_valM
B
Register
Register M
file
file
Read from memory
Register updates

Select
A
W_valE
E
D
Fetch
icode ifun
rA
rB
Instruction
Instruction
memory
memory
valC
valP
PC
PC
increment
increment
Predict
PC
f_PC
M_valA
Select
PC
F
W_valM
predPC
5
Pipeline Demonstration
irmovl
$1,%eax
#I1
irmovl
$2,%ecx
#I2
irmovl
$3,%edx
#I3
irmovl
$4,%ebx
#I4
halt
#I5
1
2
3
4
5
6
7
8
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
9
W
Cycle 5
File: demo-basic.ys
W
I1
M
I2
E
I3
D
I4
F
I5
6
Data Dependencies: 3 Nop’s
# demo-h3.ys
1
2
3
4
5
0x000: irmovl $10,%edx
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
0x006: irmovl
$3,%eax
0x00c: nop
0x00d: nop
0x00e: nop
0x00f: addl %edx,%eax
0x011: halt
6
7
8
9
10
Cycle 6
W
R[ %eax] f 3
Cycle 7
D
valA f R[ %edx] = 10
valB f R[ %eax] = 3 7
11
W
Data Dependencies: 2 Nop’s
# demo-h2.ys
1
2
3
4
5
0x000: irmovl $10,%edx
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
0x006: irmovl
$3,%eax
0x00c: nop
0x00d: nop
0x00e: addl %edx,%eax
0x010: halt
6
7
8
9
10
W
Cycle 6
W
R[ %eax] f 3
Can’t transport value
produced by first
instruction back in
time
•
•
•
D
valA f R[ %edx] = 10
valB f R[ %eax] = 0
Error
8
Data Dependencies: 1 Nop
# demo-h1.ys
1
2
3
4
5
0x000: irmovl $10,%edx
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
0x006: irmovl
$3,%eax
0x00c: nop
0x00d: addl %edx,%eax
0x00f: halt
6
7
8
9
W
Cycle 5
W
R[ %edx] f 10
M
Now a
problem with
both operands
M_valE = 3
M_dstE = %eax
•
•
•
D
valA f R[ %edx] = 0
valB f R[ %eax] = 0
Error
9
Data Dependencies: No Nop
# demo-h0.ys
1
2
3
4
5
0x000: irmovl $10,%edx
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
0x006: irmovl
$3,%eax
0x00c: addl %edx,%eax
0x00e: halt
6
7
8
W
Cycle 4
M
M_valE = 10
M_dstE = %edx
E
Wow - we really
missed the boat here…
e_valE f 0 + 3 = 3
E_dstE = %eax
D
valA f R[ %edx] = 0
valB f R[ %eax] = 0
Error
10
Predicting the
PC
D
M_icode
M_Bch
M_valA
W_icode
W_valM
icode ifun
rA
rB
valC
valP
Predict
PC
Need
valC
Instr
valid
Need
regids
Split
Split
PC
PC
increment
increment
Align
Align
Byte 0
Bytes 1-5
Instruction
Instruction
memory
memory
Select
PC
F

predPC
Start fetch of new instruction after current one has completed
fetch stage
 Not enough time to reliably determine next instruction

Guess which instruction will follow
 Recover if prediction was incorrect
11
Our Prediction Strategy
Instructions that Don’t Transfer Control


Predict next PC to be valP
Always reliable
Call and Unconditional Jumps


Predict next PC to be valC (destination)
Always reliable
Conditional Jumps


Predict next PC to be valC (destination)
Only correct if branch is taken
 Typically right 60% of time
Return Instruction

Don’t try to predict
12
Recovering
from PC
Misprediction
M_icode
M_Bch
M_valA
W_icode
W_valM
D
icode ifun
rA
rB
valC
valP
Predict
PC
Need
valC
Instr
valid
Need
regids
Split
Split
PC
PC
increment
increment
Align
Align
Byte 0
Bytes 1-5
Instruction
Instruction
memory
memory
Select
PC
F

predPC
Mispredicted Jump
 Will see branch flag once instruction reaches memory stage
 Can get fall-through PC from valA

Return Instruction
 Will get return PC when ret reaches write-back stage

In both cases
 Need to throw away instructions fetched between prediction and resolution
13
Branch Misprediction Example
demo-j.ys
0x000:
xorl %eax,%eax
0x002:
jne t
0x007:
irmovl $1, %eax
0x00d:
nop
0x00e:
nop
0x00f:
nop
0x010:
halt
0x011: t: irmovl $3, %edx
0x017:
irmovl $4, %ecx
0x01d:
irmovl $5, %edx

# Not taken
# Fall through
# Target (Should not execute)
# Should not execute
# Should not execute
Should only execute first 7 instructions
14
Branch Misprediction Trace
# demo-j
0x000:
xorl %eax,%eax
0x002:
jne t # Not taken
1
2
3
4
5
6
F
D
F
E
D
M
E
W
M
W
F
D
F
E
D
F
M
E
D
0x011: t: irmovl $3, %edx # Target
0x017:
irmovl $4, %ecx # Target+1
0x007:
irmovl $1, %eax # Fall Through
7
8
9
W
M
E
W
M
W
Cycle 5
M

Incorrectly execute two
instructions at branch target
M_Bch = 0
M_valA = 0x007
E
valE f 3
dstE = %edx
D
valC = 4
dstE = %ecx
F
valC f 1
rB f %eax
15
Return Example
0x000:
0x006:
0x007:
0x008:
0x009:
0x00e:
0x014:
0x020:
0x020:
0x021:
0x022:
0x023:
0x024:
0x02a:
0x030:
0x036:
0x100:
0x100:
demo-ret.ys
irmovl Stack,%esp # Intialize stack pointer
nop
# Avoid hazard on %esp
nop
nop
call p
# Procedure call
irmovl $5,%esi
# Return point
halt
.pos 0x20
p: <op>
# procedure
<op>
<op>
ret
irmovl $1,%eax
# Should not be executed
irmovl $2,%ecx
# Should not be executed
irmovl $3,%edx
# Should not be executed
irmovl $4,%ebx
# Should not be executed
.pos 0x100
Stack:
# Stack: Stack pointer
16
Incorrect Return Example
# demo-ret
0x023:

ret
0x024:
D
irmovl $1,%eax # Oops! F
0x02a:
irmovl $2,%ecx # Oops!
0x030:
irmovl $3,%edx # Oops!
0x00e:
irmovl $5,%esi # Return
Incorrectly execute 3
instructions following ret
F
E
D
F
M
E
D
F
W
M
E
D
F
W
M
E
D
W
M
E
W
M
W
W
valM = 0x0e
M
valE = 1
dstE = %eax
E
valE f 2
dstE = %ecx
D
valC = 3
dstE = %edx
F
valC f 5
rB f %esi
17
Pipeline Summary
Concept


Break instruction execution into 5 stages
Run instructions through in pipelined mode
Limitations


Can’t handle dependencies between instructions when
instructions follow too closely
Data dependencies
 One instruction writes register, later one reads it

Control dependency
 Instruction sets PC in way that pipeline did not predict correctly
 Mispredicted branch and return
18
The problem is hazards
Make the pipelined processor work!
Data Hazards


Instruction having register R as source follows shortly after
instruction having register R as destination
Common condition, don’t want to slow down pipeline
Control Hazards

Mispredict conditional branch
 Our design predicts all branches as being taken
 Naïve pipeline executes two extra instructions

Getting return address for ret instruction
 Naïve pipeline executes three extra instructions
Making Sure It Really Works

What if multiple special cases happen simultaneously?
19
How do we fix the Pipeline?
Pad the program with NOPs

Yuck!
Stall the pipeline

Data hazards
 Wait for producing instruction to complete
 Then proceed with consuming instruction

Control hazards
 Wait until new PC has been determined
 Then begin fetching
Forward data within the pipeline

Grab the result from somewhere in the pipe
 After it has been computed
 But before it has been written back
20
Stalling for Data Dependencies
# demo-h2.ys
1
2
3
4
5
0x000: irmovl $10,%edx
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
E
M
W
D
D
E
M
W
F
F
D
E
M
0x006: irmovl
$3,%eax
0x00c: nop
0x00d: nop
6
bubble
0x00e: addl %edx,%eax
0x010: halt



F
7
8
9
10
11
W
If instruction follows too closely after one that writes
register, slow it down
Hold instruction in decode
Dynamically inject nop into execute stage
21
Write back
Stall Condition
W
icode
valE
dstE
dstM
data out
read
Mem.
control
Source Registers
valM
Data
Data
memory
memory
write
Memory
data in
Addr
M_valA
M_Bch

srcA and srcB of current
instruction in decode
stage
Destination Registers


dstE and dstM fields
Instructions in execute,
memory, and write-back
stages
Special Case

Don’t stall for register ID
8
M
Bch
valE
valA
dstE
dstM
dstE
dstM srcA
e_Bch
Execute
E
icode
ALU
fun.
ALU
ALU
CC
CC
ifun
ALU
A
ALU
B
valC
valA
valB
srcB
d_srcA d_srcB
Select
A
Decode
d_rvalA
A
dstE
dstM srcA
srcB
W_valM
B
Register
RegisterM
file
file
W_valE
E
D
Fetch
icode
ifun
rA
rB
Instruction
Instruction
memory
memory
valC
valP
PC
PC
increment
increment
Predict
PC
f_PC
M_valA
Select
PC
 Indicates absence of
register operand
icode
F
W_valM
predPC
22
Detecting Stall Condition
# demo-h2.ys
1
2
3
4
5
0x000: irmovl $10,%edx
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
E
M
W
D
D
E
M
W
F
F
D
E
M
0x006: irmovl
$3,%eax
0x00c: nop
0x00d: nop
6
bubble
0x00e: addl %edx,%eax
0x010: halt
F
7
8
9
10
11
W
Cycle 6
W
W_dstE = %eax
W_valE = 3
•
•
•
D
srcA = %edx
srcB = %eax
23
Stalling X3
# demo-h0.ys
1
2
3
4
5
0x000: irmovl $10,%edx
F
D
E
M
W
F
D
E
M
W
E
M
W
E
M
W
E
M
W
0x006: irmovl
$3,%eax
bubble
bubble
6
bubble
0x00c: addl %edx,%eax
F
0x00e: halt
7
8
9
10
D
D
D
D
E
M
W
F
F
F
F
D
E
M
11
W
Cycle 6
W
Cycle 5
W_dstE = %eax
M
Cycle 4
M_dstE = %eax
E
•
•
•
D
E_dstE = %eax
D
srcA = %edx
srcB = %eax
srcA = %edx
srcB = %eax
•
•
•
D
srcA = %edx
srcB = %eax
24
What Happens When Stalling?
# demo-h0.ys
0x000: irmovl $10,%edx
0x006: irmovl
$3,%eax
0x00c: addl %edx,%eax
0x00e: halt



Cycle 8
4
5
6
7
Write Back
Memory
Execute
Decode
Fetch
0x000: bubble
0x006:
irmovl $10,%edx
$3,%eax
0x000: bubble
0x006:
irmovl $10,%edx
$3,%eax
0x006: bubble
0x00c:
irmovl
addl
%edx,%eax
$3,%eax
0x00c: halt
0x00e:
addl %edx,%eax
0x00e: halt
Stalling instruction held back in decode stage
Following instruction stays in fetch stage
Bubbles injected into execute stage
 Like dynamically generated nop’s
 Move through later stages
25
Implementing Stalling
W_dstM
W_dstE
W
icode
valE
valM
dstE dstM
valE
valA
dstE dstM
M_dstM
M_dstE
M
icode
Bch
E_dstM
Pipe
control
logic
E_dstE
E_bubble
E
icode ifun
valC
valA
valB
dstE dstM srcA srcB
d_srcB
d_srcA
srcB
D_icode
D_stall
F_stall
D
srcA
icode ifun
F
rA
rB
valC
valP
predPC
Pipeline Control


Combinational logic detects stall condition
Sets mode signals for how pipeline registers should update
26
Pipeline Register Modes
Input = y
Output = x
x
Normal
stall
=0
Output = x
x
stall
=1
Output = x
x
stall
=0
y
_
Rising
clock
_
Output = x
x
bubble
=0
Input = y
Bubble
_
Output = y
bubble
=0
Input = y
Stall
_
Rising
clock
_
Rising
clock
_
n
o
p
Output = nop
bubble
=1
27
Summary
Today



Data hazards (read after write)
Control hazards (branch, return)
Mitigating hazards through stalling
Next Time



Hazard mitigation through pipeline forwarding
Hardware support for forwarding
Forwarding to mitigate control (branch) hazards
28