Transcript Lecture 4: Pipeline Complications: Data and Control Hazards Professor Alvin R. Lebeck

Lecture 4: Pipeline Complications:
Data and Control Hazards
Professor Alvin R. Lebeck
Computer Science 220
Fall 2001
Administrative
• Homework #1 Due Tuesday, September 11
• Start Reading Chapter 4
• Projects
© Alvin R. Lebeck 2001
CPS 220
2
Review: A Single Cycle Processor
ALUop
RegDst
op
Main
6
Control
Instr<31:26>
1
32
Clk
Imm16
MemtoReg
MemWr
0
32
Data In 32
ALUSrc
Rd
WrEn Adr
Data
Memory
32
Mux
ALU
Extender
16
Zero
Rs
<0:15>
busA
Rw Ra Rb
32
32 32-bit
busB
Registers
0
32
imm16
Instr<15:0>
Rt
ALUctr
<11:15>
5
Rt
Mux
busW
32
Clk
5
Instruction
Fetch Unit
Jump
Clk
Rs
<21:25>
RegWr 5
Instruction<31:0>
Branch
Rt
1 Mux 0
ALUctr
3
<16:20>
:
ALU
Control
func
Instr<5:0> 6
ALUSrc
Rd
RegDst
3
1
ExtOp
© Alvin R. Lebeck 2001
CPS 220
3
Review: Pipelining Lessons
6 PM
7
30 40
T
a
s
k
A
B
O
r
d
e
r
C
D
© Alvin R. Lebeck 2001
8
40
40
• Pipelining doesn’t help
latency of single task, it
9
helps throughput of
entire workload
Time
• Pipeline rate limited by
slowest pipeline stage
40 20
• Multiple tasks operating
simultaneously
• Potential speedup =
Number pipe stages
• Unbalanced lengths of
pipe stages reduces
speedup
• Time to “fill” pipeline and
time to “drain” it reduces
speedup
CPS 220
4
Review: The Five Stages of a Load
Cycle 1 Cycle 2
Load
Ifetch
Reg/Dec
Cycle 3 Cycle 4 Cycle 5
Exec
Mem
WrB
• Ifetch: Instruction Fetch
– Fetch the instruction from the Instruction Memory
•
•
•
•
Reg/Dec: Registers Fetch and Instruction Decode
Exec: Calculate the memory address
Mem: Read the data from the Data Memory
WrB: Write the data back to the register file
© Alvin R. Lebeck 2001
5
Review: Pipelining the Load Instruction
Cycle 1 Cycle 2
Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7
Clock
1st lw Ifetch
Reg/Dec
2nd lw Ifetch
3rd lw
Exec
Mem
WrB
Reg/Dec
Exec
Mem
WrB
Ifetch
Reg/Dec
Exec
Mem
WrB
• The five independent pipeline stages are:
–
–
–
–
–
Read Next Instruction: The Ifetch stage.
Decode Instruction and fetch register values: The Reg/Dec stage
Execute the operation: The Exec stage.
Access Data-Memory: The Mem stage.
Write Data to Destination Register: The WrB stage
• One instruction enters the pipeline every cycle
– One instruction comes out of the pipeline (completed) every cycle
– The “Effective” Cycles per Instruction (CPI) is 1; ~1/5 cycle time
© Alvin R. Lebeck 2001
6
Review: Delay R-type’s Write by One Cycle
• Delay R-type’s register write by one cycle:
– Now R-type instructions also use Reg File’s write port at Stage 5
– Mem stage is a NO-OP stage: nothing is being done. Effective CPI?
1
R-type
2
Ifetch
Cycle 1 Cycle 2
Reg/Dec
4
3
Exec
5
Mem
Wr
Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9
Clock
R-type Ifetch
R-type
Reg/Dec
Exec
Mem
WrB
Ifetch
Reg/Dec
Exec
Mem
WrB
Ifetch
Reg/Dec
Exec
Mem
WrB
Reg/Dec
Exec
Mem
WrB
Reg/Dec
Exec
Mem
Load
R-type Ifetch
R-type Ifetch
© Alvin R. Lebeck 2001
WrB
7
Review: A Pipelined Datapath
Clk
Ifetch
Reg/Dec
Exec
ExtOp ALUOp
RegWr
1
0
Rb
Rt Rw Di
Rd
0
1
RegDst
© Alvin R. Lebeck 2001
ALUSrc
Zero Data
Mem
RA Do
WA
Di
MemWr
1
Mux
RFile
Branch
Mem/Wr Register
Rt
Ra
Imm16
busA
busB
Exec
EX
Unit
Unit
Ex/Mem Register
Rs
WrB
PC+4
Imm16
ID/Ex Register
IF_Unit
I
PC+4
IF/ID Register
PC+4
PC
A
Mem
0
MemtoReg
8
Its Not That Easy for Computers
• What could go wrong?
• Limits to pipelining: Hazards prevent next instruction
from executing during its designated clock cycle
– Structural hazards: HW cannot support this combination of
instructions
– Data hazards: Instruction depends on result of prior instruction still
in the pipeline
– Control hazards: Pipelining of branches & other instructions
© Alvin R. Lebeck 2001
CPS 220
9
Speed Up Equation for Pipelining
Speedup from pipelining = Ave Instr Time unpipelined
Ave Instr Time pipelined
= CPIunpipelined x Clock Cycleunpipelined
CPIpipelined x Clock Cyclepipelined
= CPIunpipelined
Clock Cycleunpipelined
x
CPIpipelined
Clock Cyclepipelined
Ideal CPI = CPIunpipelined/Pipeline depth
Speedup = Ideal CPI x Pipeline depth
x
CPIpipelined
© Alvin R. Lebeck 2001
CPS 220
Clock Cycleunpipelined
Clock Cyclepipelined
10
Speed Up Equation for Pipelining
CPIpipelined = Ideal CPI + Pipeline stall clock cycles per instr
Speedup = Ideal CPI x Pipeline depth
x
Ideal CPI + Pipeline stall CPI
Speedup =
Pipeline depth
x
1 + Pipeline stall CPI
© Alvin R. Lebeck 2001
CPS 220
Clock Cycleunpipelined
Clock Cyclepipelined
Clock Cycleunpipelined
Clock Cyclepipelined
11
Example: Dual-port vs. Single-port
• Machine A: Dual ported memory
• Machine B: Single ported memory, but its pipelined
implementation has a 1.05 times faster clock rate
• Ideal CPI = 1 for both
• Loads are 40% of instructions executed
SpeedUpA = Pipeline Depth/(1 + 0) x (clockunpipe/clockpipe)
= Pipeline Depth
SpeedUpB = Pipeline Depth/(1 + 0.4 x 1)
x (clockunpipe/(clockunpipe / 1.05)
= (Pipeline Depth/1.4) x 1.05
= 0.75 x Pipeline Depth
SpeedUpA / SpeedUpB = Pipeline Depth/(0.75 x Pipeline Depth) = 1.33
• Machine A is 1.33 times faster
© Alvin R. Lebeck 2001
CPS 220
12
Three Generic Data Hazards
• InstrI followed by InstrJ
• Read After Write (RAW)
InstrJ tries to read operand before InstrI writes it
© Alvin R. Lebeck 2001
CPS 220
13
Three Generic Data Hazards
• InstrI followed by InstrJ
• Write After Read (WAR)
InstrJ tries to write operand before InstrI reads it
• Can’t happen in DLX 5 stage pipeline because:
– All instructions take 5 stages,
– Reads are always in stage 2, and
– Writes are always in stage 5
© Alvin R. Lebeck 2001
CPS 220
14
Three Generic Data Hazards
• InstrI followed by InstrJ
• Write After Write (WAW)
InstrJ tries to write operand before InstrI writes it
– Leaves wrong result ( InstrI not InstrJ)
• Can’t happen in DLX 5 stage pipeline because:
– All instructions take 5 stages, and
– Writes are always in stage 5
• Will see WAR and WAW in later more complicated
pipes
© Alvin R. Lebeck 2001
CPS 220
15
Data Hazards
• We must deal with instruction dependencies.
• Example:
sub $2, $1, $3
and $12, $2, $5
or $13, $6, $2
add $14, $2, $2
sw $15, 100($2)
Cycle 1 Cycle 2
#
#
#
#
$12 depends on the result in $2
but $2 is updated 3 clock
cycles later.
We have a problem!! Data Hazard
Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8
Clock
0: sub
Ifetch
4: and
Reg/Dec
Exec
Mem
WrB
Ifetch
Reg/Dec
Exec
Mem
WrB
Ifetch
Reg/Dec
Exec
Mem
8: or
12: add Ifetch Reg/Dec Exec
16: sw
Ifetch Reg/Dec
© Alvin R. Lebeck 2001
WrB
Mem
WrB
Exec
Mem
WrB
16
RAW Data Hazard Solution: Register Forwarding
ALU
© Alvin R. Lebeck 2001
17
RAW Data Hazard for Load
Cycle 1 Cycle 2
Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8
Clock
I0: Load Ifetch
Plus 1
Reg/Dec
Exec
Mem
Wr
Ifetch
Reg/Dec
Exec
Mem
Wr
Ifetch
Reg/Dec
Exec
Mem
Wr
Ifetch
Reg/Dec
Exec
Mem
Wr
Ifetch
Reg/Dec
Exec
Mem
Plus 2
Plus 3
Plus 4
Wr
• Load is fetched during Cycle 1:
– The data is NOT written into the Reg File until the end of Cycle 5
– We cannot read this value from the Reg File until Cycle 6
– 3-instruction delay before the load takes effect
• This is a Data Hazard:
– Register forwarding reduces the load delay to ONE instruction
– It is not possible to entirely eliminate the load Data Hazard!
© Alvin R. Lebeck 2001
18
Load Data Forwarding
© Alvin R. Lebeck 2001
19
Dealing with the Load Data Hazard
• There are two ways to deal with the load data hazard:
– Insert a NOOP bubble into the data path.
– Use Delayed load semantic (see a next slide)
Insert NOOP Here
Stall0
Stall1
Stall2
ExtOp ALUOp
RegWr
1
0
Rt Rw Di
Rd
0
1
RegDst
© Alvin R. Lebeck 2001
ALUSrc
Dat
a
RA
MeDo
WA
m
Di
MemWr
1
Mux
RFile
Zero
How?
Mem/Wr Register
Rb
PC+4
Imm16
busA
busB
Exec
EX
Unit
Unit
Ex/Mem Register
Rt
ID/Ex Register
IF_Unit
I
IF/ID Register
PC+4
PC
A
PC+4
Imm16
Rs
Ra
Branch
0
MemtoReg
20
Delayed Load
• Load instructions are defined such that immediate
successor instruction will not read result of load.
BAD
ld
r1, 8(r2)
sub r3, r1, r3
add r2, r2, 4
OK
ld
add
sub
r1, 8(r2)
r2, r2, 4
r3, r1, r3
© Alvin R. Lebeck 2001
21
Software Scheduling to Avoid Load Hazards
Try producing fast code for
a = b + c;
d = e – f;
assuming a, b, c, d ,e, and f in memory.
Slow code:
LW
LW
ADD
SW
LW
LW
SUB
SW
© Alvin R. Lebeck 2001
Rb,b
Rc,c
Ra,Rb,Rc
a,Ra
Re,e
Rf,f
Rd,Re,Rf
d,Rd
Fast code:
LW
LW
LW
ADD
LW
SW
SUB
SW
CPS 220
Rb,b
Rc,c
Re,e
Ra,Rb,Rc
Rf,f
a,Ra
Rd,Re,Rf
d,Rd
22
Compiler Avoiding Load Stalls
unscheduled
scheduled
54%
gcc
31%
42%
spice
14%
65%
tex
25%
0%
20%
40%
60%
80%
% loads stalling pipeline
© Alvin R. Lebeck 2001
CPS 220
23
Review: Data Hazards
• RAW
– only one that can occur in DLX pipeline
• WAR
• WAW
• Data Forwarding (Register Bypassing)
– send data from one stage to another bypassing the register file
• Still have load use delay
© Alvin R. Lebeck 2001
CPS 220
24
Pipelining Summary
• Just overlap tasks, and easy if tasks are independent
• Speed Up ~ Pipeline Depth; if ideal CPI is 1, then:
Speedup =
Pipeline Depth
Clock Cycle Unpipelined
X
1 + Pipeline stall CPI
Clock Cycle Pipelined
• Hazards limit performance on computers:
– Structural: need more HW resources
– Data: need forwarding, compiler scheduling
– Control: discuss today
• Branches and Other Difficulties
• What makes branches difficult?
© Alvin R. Lebeck 2001
CPS 220
25
Control Hazard on Branches: Three Stage Stall
time
cc1
beq r1, foo
cc2
cc3
I
add r3, r4, r6
and r3, r2, r4
sub r2, r3, r5
add r3, r2, r5
© Alvin R. Lebeck 2001
cc
4
D
cc
5
I
cc
6
cc
7
cc
8
cc
9
D
I
D
I
D
I
CPS 220
D
26
Control Hazard
Cycle 4 Cycle 5
Cycle 6 Cycle 7 Cycle 8 Cycle 9 Cycle 10 Cycle 11
Clk
12: Beq Ifetch Reg/Dec Exec
(target is 1000)
16: R-type Ifetch Reg/Dec
20: R-type
Ifetch
24: R-type
Mem
Wr
Exec
Mem
Wr
Reg/Dec
Exec
Mem
Wr
Ifetch
Reg/Dec
Exec
Mem
Wr
Ifetch
Reg/Dec
Exec
Mem
1000: Target of Br
Wr
• Although Beq is fetched during Cycle 4:
– Target address is NOT written into the PC until the end of Cycle 7
– Branch’s target is NOT fetched until Cycle 8
– 3-instruction delay before the branch take effect
• This is called a Control Hazard:
Branch Stall Impact
• If CPI = 1, 30% branch, Stall 3 cycles => new CPI = 1.9!
• How can you reduce this delay?
• Two part solution:
– Determine branch taken or not sooner, AND
– Compute taken branch address earlier
• DLX branch tests if register = 0 or != 0
• DLX Solution:
– Move Zero test to ID/RF stage
– Adder to calculate new PC in ID/RF stage
– 1 clock cycle penalty for branch versus 3
© Alvin R. Lebeck 2001
CPS 220
28
Branch Delays
IF/ID
Example:
sub
beq
add
. .
go:
lw
ID/EX
$10, $4, $8
$10, $3, go
$12, $2, $5
.
$4, 16($12)
© Alvin R. Lebeck 2001
29
Branch Hazard
• Can we eliminate the effect of this one cycle branch
delay?
© Alvin R. Lebeck 2001
30
Four Branch Hazard Alternatives
#1: Stall until branch direction is clear
#2: Predict Branch Not Taken
–
–
–
–
–
Execute successor instructions in sequence
“Squash” instructions in pipeline if branch actually taken
Advantage of late pipeline state update
47% DLX branches not taken on average
PC+4 already calculated, so use it to get next instruction
#3: Predict Branch Taken
– 53% DLX branches taken on average
– But haven’t calculated branch target address in DLX
» DLX still incurs 1 cycle branch penalty
» Other machines: branch target known before outcome
© Alvin R. Lebeck 2001
CPS 220
31
Four Branch Hazard Alternatives
#4: Delayed Branch
– Define branch to take place AFTER a following instruction
branch instruction
sequential successor1
sequential successor2
........
sequential successorn
branch target if taken
Branch delay of length n
– 1 slot delay allows proper decision and branch target
address in 5 stage pipeline
– DLX uses this
© Alvin R. Lebeck 2001
CPS 220
32
Delayed Branch
• Where to get instructions to fill branch delay slot?
–
–
–
–
Before branch instruction
From the target address: only valuable when branch taken
From fall through: only valuable when branch not taken
Cancelling branches allows more slots to be filled
• Compiler effectiveness for single branch delay slot:
– Fills about 60% of branch delay slots
– About 80% of instructions executed in branch delay slots useful in
computation
– About 50% (60% x 80%) of slots usefully filled
© Alvin R. Lebeck 2001
CPS 220
33
Evaluating Branch Alternatives
Pipeline speedup =
Scheduling
Branch
scheme
penalty
Stall pipeline
3
Predict taken
1
Predict not taken
1
Delayed branch
0.5
Pipeline depth
1 +Branch frequency Branch penalty
CPI
1.42
1.14
1.09
1.07
speedup v.
unpipelined
3.5
4.4
4.5
4.6
speedup v.
stall
1.0
1.26
1.29
1.31
Branches = 14% of insts, 65% of them change PC
© Alvin R. Lebeck 2001
CPS 220
34
Compiler “Static” Prediction of
Taken/Untaken Branches
• Improves strategy for placing instructions in delay slot
• Two strategies
– Backward branch predict taken, forward branch not taken
– Profile-based prediction: record branch behavior, predict branch
based on prior run
16%
80%
14%
tomcatv
swm256
ora
Taken backwards
Not Taken Forwards
Always taken
© Alvin R. Lebeck 2001
mdljsp2
tomcatv
swm256
ora
mdljsp2
hydro2d
gcc
0%
espresso
0%
doduc
2%
compress
10%
hydro2d
4%
gcc
20%
6%
espresso
30%
8%
doduc
40%
10%
compress
50%
12%
alvinn
Misprediction Rate
60%
alvinn
Misprediction Rate
70%
CPS 220
35
Evaluating Static Branch Prediction
100
Profile-based
© Alvin R. Lebeck 2001
CPS 220
tomcatv
swm256
ora
mdljsp2
hydro2d
gcc
espresso
doduc
1
compress
10
alvinn
Instructions per mispredicted branch
• Misprediction
ignores frequency
of branch
100000
• “Instructions
10000
between
mispredicted
1000
branches” is a
better metric
Direction-based
36
Pipelining Complications
• Interrupts (Exceptions)
–
–
–
–
5 instructions executing in 5 stage pipeline
How to stop the pipeline?
How to restart the pipeline?
Who caused the interrupt?
Stage
IF
ID
EX
MEM
© Alvin R. Lebeck 2001
Problem interrupts occurring
Page fault on instruction fetch; misaligned memory
access; memory-protection violation
Undefined or illegal opcode
Arithmetic interrupt
Page fault on data fetch; misaligned memory
access; memory-protection violation
CPS 220
37
Pipelining Complications
• Simultaneous exceptions in > 1 pipeline stage
– Load with data page fault in MEM stage
– Add with instruction page fault in IF stage
• Solution #1
– Interrupt status vector per instruction
– Defer check til last stage, kill state update if exception
• Solution #2
– Interrupt ASAP
– Restart everything that is incomplete
• Exception in branch delay slot,
– SW needs two PCs
• Another advantage for state update late in
pipeline!
© Alvin R. Lebeck 2001
CPS 220
38
Next Time
• Next time
– More pipeline complications
– Longer pipelines (R4000) => Better branch prediction, more
instruction parallelism?
Todo
• Read Chapter 3 and 4
• Homework #1 due
• Project selection by September 30
© Alvin R. Lebeck 2001
CPS 220
39
Pipeline Complications
• Complex Addressing Modes and Instructions
• Address modes: Autoincrement causes register
change during instruction execution
– Interrupts? Need to restore register state
– Adds WAR and WAW hazards since writes no longer last stage
• Memory-Memory Move Instructions
– Must be able to handle multiple page faults
– Long-lived instructions: partial state save on interrupt
• Condition Codes
© Alvin R. Lebeck 2001
CPS 220
40
Pipeline Complications: Floating Point
EX
M1
IF
M2
MEM
M3
M4
M5
M6
M7
WB
ID/RF
A1
A2
A3
A4
FP/INT Divide Unit
Not Pipelined
25 Clocks
© Alvin R. Lebeck 2001
41
Pipelining Complications
• Floating Point: long execution time
• Also, may pipeline FP execution unit so they can
initiate new instructions without waiting full latency
FP Instruction
Add, Subtract
Multiply
Divide
Square root
Negate
Absolute value
FP compare
Latency
4
8
36
112
2
2
3
Initiation Rate
3
4
35
111
1
1
2
Cycles before
use result
© Alvin R. Lebeck 2001
(MIPS R4000)
(interrupts,
WAW, WAR)
Cycles before issue
instr of same type
CPS 220
42
Summary of Pipelining Basics
• Hazards limit performance
– Structural: need more HW resources
– Data: need forwarding, compiler scheduling
– Control: early evaluation & PC, delayed branch, prediction
• Increasing length of pipe increases impact of
hazards; pipelining helps instruction bandwidth, not
latency
• Compilers reduce cost of data and control hazards
– Load delay slots
– Branch delay slots
– Branch prediction
• Interrupts, Instruction Set, FP makes pipelining
harder
• Handling context switches.
© Alvin R. Lebeck 2001
CPS 220
43
Case Study: MIPS R4000
(100 MHz to 200 MHz)
• 8 Stage Pipeline:
– IF–first half of fetching of instruction; PC selection happens here as
well as initiation of instruction cache access.
– IS–second half of access to instruction cache.
– RF–instruction decode and register fetch, hazard checking and also
instruction cache hit detection.
– EX–execution, which includes effective address calculation, ALU
operation, and branch target computation and condition evaluation.
– DF–data fetch, first half of access to data cache.
– DS–second half of access to data cache.
– TC–tag check, determine whether the data cache access hit.
– WB–write back for loads and register-register operations.
• 8 Stages: What is impact on Load delay? Branch
delay? Why?
© Alvin R. Lebeck 2001
CPS 220
44
Case Study: MIPS R4000
IF
IS
IF
RF
IS
IF
EX
RF
IS
IF
DF
EX
RF
IS
IF
DS
DF
EX
RF
IS
IF
TC
DS
DF
EX
RF
IS
IF
WB
TC
DS
DF
EX
RF
IS
IF
IF
THREE Cycle
Branch Latency
(conditions evaluated
during EX phase)
IS
IF
RF
IS
IF
EX
RF
IS
IF
DF
EX
RF
IS
IF
DS
DF
EX
RF
IS
IF
TC
DS
DF
EX
RF
IS
IF
WB
TC
DS
DF
EX
RF
IS
IF
TWO Cycle
Load Latency
Delay slot plus two stalls
Branch likely cancels delay slot if not taken
© Alvin R. Lebeck 2001
CPS 220
45
MIPS R4000 Floating Point
• FP Adder, FP Multiplier, FP Divider
• Last step of FP Multiplier/Divider uses FP Adder HW
• 8 kinds of stages in FP units:
–
–
–
–
–
–
–
–
–
Stage
A
D
E
M
N
R
S
U
© Alvin R. Lebeck 2001
Functional unit Description
FP adder
Mantissa ADD stage
FP divider
Divide pipeline stage
FP multiplier
Exception test stage
FP multiplier
First stage of multiplier
FP multiplier
Second stage of multiplier
FP adder
Rounding stage
FP adder
Operand shift stage
Unpack FP numbers
CPS 220
46
MIPS FP Pipe Stages
FP Instr
Add, Subtract
Multiply
Divide
Square root
Negate
Absolute value
FP compare
Stages:
M
N
R
S
U
© Alvin R. Lebeck 2001
1
U
U
U
U
U
U
U
2
S+A
E+M
A
E
S
S
A
3
4
A+R R+S
M
M
R
D28
(A+R)108
7
8
…
5
6
M
…
…
N
N+A R
D+A D+R, D+R, D+A, D+R, A, R
A
R
R
First stage of multiplier
Second stage of multiplier
Rounding stage
Operand shift stage
Unpack FP numbers
CPS 220
A
D
E
Mantissa ADD stage
Divide pipeline stage
Exception test stage
47
R4000 Performance
• Not ideal CPI of 1:
– Load stalls (1 or 2 clock cycles)
– Branch stalls (2 cycles + unfilled slots)
– FP result stalls: RAW data hazard (latency)
– FP structural stalls: Not enough FP hardware (parallelism)
Base
Load stalls
Branch stalls
FP result stalls
tomcatv
su2cor
spice2g6
ora
nasa7
doduc
li
gcc
espresso
eqntott
4.5
4
3.5
3
2.5
2
1.5
1
0.5
0
FP structural
stalls
© Alvin R. Lebeck 2001
CPS 220
48
Next Time
• Homework #1 is Due
• Instruction Level Parallelism (ILP)
• Read Chapter 4
© Alvin R. Lebeck 2001
CPS 220
49