Transcript ppt

Systems I
Pipelining II
Topics



Pipelining hardware: registers and
feedback paths
Difficulties with pipelines: hazards
Method of mitigating hazards
valM
SEQ+ Hardware
data out
read
Data
Data
memory
memory
Mem.
control
Memory
write
Addr


Still sequential
implementation
Reorder PC stage to put at
beginning
Execute
Bch
valE
CC
CC
ALU
ALU
ALU
A
Data
ALU
fun.
ALU
B
PC Stage

Task is to select PC for
current instruction
valA
valB
dstE dstM srcA srcB
dstE dstM srcA srcB

Based on results computed
by previous instruction
Decode

PC is no longer stored in
register
B
Write back
Processor State

A
Register
Register M
file
file E
icode
Fetch
ifun
rA
rB
valC
Instruction
Instruction
memory
memory
But, can determine PC based
on other stored information
valP
PC
PC
increment
increment
PC
PC
PC
pIcode pBch
pValM
pValC
pValP
2
Adding Pipeline Registers
valE, valM
W_icode, W_valM
Write back
valM
W_valE, W_valM, W_dstE, W_dstM
valM
W
valM
Data
Data
memory
memory
Memory
Memory
Data
Data
memory
memory
M_icode,
M_Bch,
M_valA
Addr, Data
Addr, Data
M
valE
Bch
Bch
CC
CC
Execute
ALU
ALU
valE
CC
CC
Execute
ALU
ALU
aluA, aluB
aluA, aluB
E
valA, valB
Decode
valA, valB
srcA, srcB
dstA, dstB
icode, valC
valP
A
B
Register M
Register
file
file
d_srcA,
d_srcB
Decode
A
B
Register
RegisterM
file
file E
E
Write back
valP
icode, ifun
rA, rB
valC
Fetch
Instruction
Instruction
memory
memory
D
icode, ifun,
rA, rB, valC
PC
PC
increment
increment
Instruction
Instruction
memory
memory
Fetch
PC
PC
valP
valP
PC
PC
increment
increment
predPC
pState
PC
f_PC
F
3
Pipeline Stages
Fetch



Select current PC
Read instruction
Compute incremented PC
Decode

Read program registers
Execute

Operate ALU
Memory

Read or write data memory
Write Back

Update register file
4
Write back
PIPE- Hardware
W
icode
valE
Mem.
control
write
Pipeline registers hold
intermediate values
from instruction
execution
Forward (Upward) Paths


Values passed from one
stage to next
Cannot jump past
stages
dstE dstM
data out
read

valM
Data
Data
memory
memory
Memory
data in
Addr
M_valA
M_Bch
M
icode
Bch
valE
valA
dstE dstM
e_Bch
Execute
E
ALU
fun.
ALU
ALU
CC
CC
icode ifun
ALU
A
ALU
B
valC
valA
valB
d_srcA d_srcB
Select
A
Decode
d_rvalA
A
dstE dstM srcA srcB
W_valM
B
Register
Register M
file
file
W_valE
E
 e.g., valC passes
through decode
dstE dstM srcA srcB
D
Fetch
icode ifun
rA
rB
Instruction
Instruction
memory
memory
valC
valP
PC
PC
increment
increment
Predict
PC
f_PC
M_valA
Select
PC
F
W_valM
predPC
5
Signal Naming Conventions
S_Field

Value of Field held in stage S pipeline
register
s_Field

Value of Field computed in stage S
6
Write back
Feedback Paths
W
icode
valE
dstE dstM
data out
read
Mem.
control
write
Predicted PC
valM
Data
Data
memory
memory
Memory
data in
Addr
M_valA
M_Bch

Guess value of next PC
M
icode
Bch
valE
valA
dstE dstM
e_Bch
Branch information


Jump taken/not-taken
Fall-through or target
address
Execute
E
ALU
fun.
ALU
ALU
CC
CC
icode ifun
ALU
A
ALU
B
valC
valA
valB
dstE dstM srcA srcB
d_srcA d_srcB
Return point

Decode
To register file write
ports
d_rvalA
A
dstE dstM srcA srcB
W_valM
B
Register
Register M
file
file
Read from memory
Register updates

Select
A
W_valE
E
D
Fetch
icode ifun
rA
rB
Instruction
Instruction
memory
memory
valC
valP
PC
PC
increment
increment
Predict
PC
f_PC
M_valA
Select
PC
F
W_valM
predPC
7
Pipeline Demonstration
irmovl
$1,%eax
#I1
irmovl
$2,%ecx
#I2
irmovl
$3,%edx
#I3
irmovl
$4,%ebx
#I4
halt
#I5
1
2
3
4
5
6
7
8
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
9
W
Cycle 5
File: demo-basic.ys
W
I1
M
I2
E
I3
D
I4
F
I5
8
Data Dependencies: 3 Nop’s
# demo-h3.ys
1
2
3
4
5
0x000: irmovl $10,%edx
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
0x006: irmovl
$3,%eax
0x00c: nop
0x00d: nop
0x00e: nop
0x00f: addl %edx,%eax
0x011: halt
6
7
8
9
10
Cycle 6
W
R[ %eax] f 3
Cycle 7
D
valA f R[ %edx] = 10
valB f R[ %eax] = 3 9
11
W
Data Dependencies: 2 Nop’s
# demo-h2.ys
1
2
3
4
5
0x000: irmovl $10,%edx
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
0x006: irmovl
$3,%eax
0x00c: nop
0x00d: nop
0x00e: addl %edx,%eax
0x010: halt
6
7
8
9
10
W
Cycle 6
W
R[ %eax] f 3
Can’t transport value
produced by first
instruction back in
time
•
•
•
D
valA f R[ %edx] = 10
valB f R[ %eax] = 0
Error
10
Data Dependencies: 1 Nop
# demo-h1.ys
1
2
3
4
5
0x000: irmovl $10,%edx
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
0x006: irmovl
$3,%eax
0x00c: nop
0x00d: addl %edx,%eax
0x00f: halt
6
7
8
9
W
Cycle 5
W
R[ %edx] f 10
M
Now a
problem with
both operands
M_valE = 3
M_dstE = %eax
•
•
•
D
valA f R[ %edx] = 0
valB f R[ %eax] = 0
Error
11
Data Dependencies: No Nop
# demo-h0.ys
1
2
3
4
5
0x000: irmovl $10,%edx
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
0x006: irmovl
$3,%eax
0x00c: addl %edx,%eax
0x00e: halt
6
7
8
W
Cycle 4
M
M_valE = 10
M_dstE = %edx
E
Wow - we really
missed the boat here…
e_valE f 0 + 3 = 3
E_dstE = %eax
D
valA f R[ %edx] = 0
valB f R[ %eax] = 0
Error
12
Predicting the
PC
D
M_icode
M_Bch
M_valA
W_icode
W_valM
icode ifun
rA
rB
valC
valP
Predict
PC
Need
valC
Instr
valid
Need
regids
Split
Split
PC
PC
increment
increment
Align
Align
Byte 0
Bytes 1-5
Instruction
Instruction
memory
memory
Select
PC
F

predPC
Start fetch of new instruction after current one has completed
fetch stage
 Not enough time to reliably determine next instruction

Guess which instruction will follow
 Recover if prediction was incorrect
13
Our Prediction Strategy
Instructions that Don’t Transfer Control


Predict next PC to be valP
Always reliable
Call and Unconditional Jumps


Predict next PC to be valC (destination)
Always reliable
Conditional Jumps


Predict next PC to be valC (destination)
Only correct if branch is taken
 Typically right 60% of time
Return Instruction

Don’t try to predict
14
Recovering
from PC
Misprediction
M_icode
M_Bch
M_valA
W_icode
W_valM
D
icode ifun
rA
rB
valC
valP
Predict
PC
Need
valC
Instr
valid
Need
regids
Split
Split
PC
PC
increment
increment
Align
Align
Byte 0
Bytes 1-5
Instruction
Instruction
memory
memory
Select
PC
F

predPC
Mispredicted Jump
 Will see branch flag once instruction reaches memory stage
 Can get fall-through PC from valA

Return Instruction
 Will get return PC when ret reaches write-back stage

In both cases
 Need to throw away instructions fetched between prediction and resolution
15
Branch Misprediction Example
demo-j.ys
0x000:
xorl %eax,%eax
0x002:
jne t
0x007:
irmovl $1, %eax
0x00d:
nop
0x00e:
nop
0x00f:
nop
0x010:
halt
0x011: t: irmovl $3, %edx
0x017:
irmovl $4, %ecx
0x01d:
irmovl $5, %edx

# Not taken
# Fall through
# Target (Should not execute)
# Should not execute
# Should not execute
Should only execute first 7 instructions
16
Branch Misprediction Trace
# demo-j
0x000:
xorl %eax,%eax
0x002:
jne t # Not taken
1
2
3
4
5
6
F
D
F
E
D
M
E
W
M
W
F
D
F
E
D
F
M
E
D
0x011: t: irmovl $3, %edx # Target
0x017:
irmovl $4, %ecx # Target+1
0x007:
irmovl $1, %eax # Fall Through
7
8
9
W
M
E
W
M
W
Cycle 5
M

Incorrectly execute two
instructions at branch target
M_Bch = 0
M_valA = 0x007
E
valE f 3
dstE = %edx
D
valC = 4
dstE = %ecx
F
valC f 1
rB f %eax
17
Return Example
0x000:
0x006:
0x007:
0x008:
0x009:
0x00e:
0x014:
0x020:
0x020:
0x021:
0x022:
0x023:
0x024:
0x02a:
0x030:
0x036:
0x100:
0x100:
demo-ret.ys
irmovl Stack,%esp # Intialize stack pointer
nop
# Avoid hazard on %esp
nop
nop
call p
# Procedure call
irmovl $5,%esi
# Return point
halt
.pos 0x20
p: <op>
# procedure
<op>
<op>
ret
irmovl $1,%eax
# Should not be executed
irmovl $2,%ecx
# Should not be executed
irmovl $3,%edx
# Should not be executed
irmovl $4,%ebx
# Should not be executed
.pos 0x100
Stack:
# Stack: Stack pointer
18
Incorrect Return Example
# demo-ret
0x023:

ret
0x024:
D
irmovl $1,%eax # Oops! F
0x02a:
irmovl $2,%ecx # Oops!
0x030:
irmovl $3,%edx # Oops!
0x00e:
irmovl $5,%esi # Return
Incorrectly execute 3
instructions following ret
F
E
D
F
M
E
D
F
W
M
E
D
F
W
M
E
D
W
M
E
W
M
W
W
valM = 0x0e
M
valE = 1
dstE = %eax
E
valE f 2
dstE = %ecx
D
valC = 3
dstE = %edx
F
valC f 5
rB f %esi
19
Pipeline Summary
Concept


Break instruction execution into 5 stages
Run instructions through in pipelined mode
Limitations


Can’t handle dependencies between instructions when
instructions follow too closely
Data dependencies
 One instruction writes register, later one reads it

Control dependency
 Instruction sets PC in way that pipeline did not predict correctly
 Mispredicted branch and return
20
The problem is hazards
Make the pipelined processor work!
Data Hazards


Instruction having register R as source follows shortly after
instruction having register R as destination
Common condition, don’t want to slow down pipeline
Control Hazards

Mispredict conditional branch
 Our design predicts all branches as being taken
 Naïve pipeline executes two extra instructions

Getting return address for ret instruction
 Naïve pipeline executes three extra instructions
Making Sure It Really Works

What if multiple special cases happen simultaneously?
21
How do we fix the Pipeline?
Pad the program with NOPs

Yuck!
Stall the pipeline

Data hazards
 Wait for producing instruction to complete
 Then proceed with consuming instruction

Control hazards
 Wait until new PC has been determined
 Then begin fetching
Forward data within the pipeline

Grab the result from somewhere in the pipe
 After it has been computed
 But before it has been written back
22
Stalling for Data Dependencies
# demo-h2.ys
1
2
3
4
5
0x000: irmovl $10,%edx
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
E
M
W
D
D
E
M
W
F
F
D
E
M
0x006: irmovl
$3,%eax
0x00c: nop
0x00d: nop
6
bubble
0x00e: addl %edx,%eax
0x010: halt



F
7
8
9
10
11
W
If instruction follows too closely after one that writes
register, slow it down
Hold instruction in decode
Dynamically inject nop into execute stage
23
Stall Condition
Source Registers

srcA and srcB of current
instruction in decode
stage
Destination Registers


dstE and dstM fields
Instructions in execute,
memory, and write-back
stages
Special Case

Don’t stall for register ID
15 (0xF)
 Indicates absence of
register operand

Don’t stall for failed
conditional move
24
Detecting Stall Condition
# demo-h2.ys
1
2
3
4
5
0x000: irmovl $10,%edx
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
F
D
E
M
W
E
M
W
D
D
E
M
W
F
F
D
E
M
0x006: irmovl
$3,%eax
0x00c: nop
0x00d: nop
6
bubble
0x00e: addl %edx,%eax
0x010: halt
F
7
8
9
10
11
W
Cycle 6
W
W_dstE = %eax
W_valE = 3
•
•
•
D
srcA = %edx
srcB = %eax
25
Stalling X3
# demo-h0.ys
1
2
3
4
5
0x000: irmovl $10,%edx
F
D
E
M
W
F
D
E
M
W
E
M
W
E
M
W
E
M
W
0x006: irmovl
$3,%eax
bubble
bubble
6
bubble
0x00c: addl %edx,%eax
F
0x00e: halt
7
8
9
10
D
D
D
D
E
M
W
F
F
F
F
D
E
M
11
W
Cycle 6
W
Cycle 5
W_dstE = %eax
M
Cycle 4
M_dstE = %eax
E
•
•
•
D
E_dstE = %eax
D
srcA = %edx
srcB = %eax
srcA = %edx
srcB = %eax
•
•
•
D
srcA = %edx
srcB = %eax
26
What Happens When Stalling?
# demo-h0.ys
0x000: irmovl $10,%edx
0x006: irmovl
$3,%eax
0x00c: addl %edx,%eax
0x00e: halt



Cycle 8
4
5
6
7
Write Back
Memory
Execute
Decode
Fetch
0x000: bubble
0x006:
irmovl $10,%edx
$3,%eax
0x000: bubble
0x006:
irmovl $10,%edx
$3,%eax
0x006: bubble
0x00c:
irmovl
addl
%edx,%eax
$3,%eax
0x00c: halt
0x00e:
addl %edx,%eax
0x00e: halt
Stalling instruction held back in decode stage
Following instruction stays in fetch stage
Bubbles injected into execute stage
 Like dynamically generated nop’s
 Move through later stages
27
Implementing Stalling
W_dstM
W_dstE
W
icode
valE
valM
dstE dstM
valE
valA
dstE dstM
M_dstM
M_dstE
M
icode
Bch
E_dstM
Pipe
control
logic
E_dstE
E_bubble
E
icode ifun
valC
valA
valB
dstE dstM srcA srcB
d_srcB
d_srcA
srcB
D_icode
D_stall
F_stall
D
srcA
icode ifun
F
rA
rB
valC
valP
predPC
Pipeline Control


Combinational logic detects stall condition
Sets mode signals for how pipeline registers should update
28
Pipeline Register Modes
Input = y
Output = x
x
Normal
stall
=0
Output = x
x
stall
=1
Output = x
x
stall
=0
y
_
Rising
clock
_
Output = x
x
bubble
=0
Input = y
Bubble
_
Output = y
bubble
=0
Input = y
Stall
_
Rising
clock
_
Rising
clock
_
n
o
p
Output = nop
bubble
=1
29
Summary
Today



Data hazards (read after write)
Control hazards (branch, return)
Mitigating hazards through stalling
Next Time



Hazard mitigation through pipeline forwarding
Hardware support for forwarding
Forwarding to mitigate control (branch) hazards
30