Transcript CMPE421 Par. Computer Architecture

CMPE 421 Advanced
Computer Architecture
Control Hazard and Prediction
PART4
Control Hazards
• When the flow of instruction addresses is not sequential
(i.e., PC = PC + 4); incurred by change of flow
instructions
– Conditional branches (beq, bne)
– Unconditional branches (j, jal, jr)
– Exceptions
• Possible approaches
– Stall (impacts CPI)
– Move decision point as early in the pipeline as possible, thereby
reducing the number of stall cycles
– Delay decision (requires compiler support)
– Predict and hope for the best !
• Control hazards occur less frequently than data hazards,
but there is nothing as effective against control hazards
as forwarding is for data hazards
2
Data Hazards for Branches
• If a comparison register is a destination of
2nd or 3rd preceding ALU instruction
add $1, $2, $3
IF
add $4, $5, $6
…
beq $1, $4, target
ID
EX
MEM
WB
IF
ID
EX
MEM
WB
IF
ID
EX
MEM
WB
IF
ID
EX
MEM
WB
• Can resolve using forwarding
3
Data Hazards for Branches
• If a comparison register is a destination of
preceding ALU instruction or 2nd preceding
load instruction
– Need 1 stall cycle
lw
$1, addr
IF
add $4, $5, $6
beq stalled
beq $1, $4, target
ID
EX
MEM
WB
IF
ID
EX
MEM
WB
IF
ID
ID
EX
MEM
WB
4
Data Hazards for Branches
• If a comparison register is a destination of
immediately preceding load instruction
– Need 2 stall cycles
lw
$1, addr
IF
beq stalled
beq stalled
beq $1, $0, target
ID
EX
IF
ID
MEM
WB
ID
ID
EX
MEM
WB
5
Jumps Incur One Stall
• Jumps not decoded until ID, so one flush is needed
flush
j target
IM
Reg
IM
Reg
DM
Fix jump
hazard by
waiting –
stall – but
affects CPI
Reg
DM
ALU
O
r
d
e
r
Reg
ALU
I
n
s
t
r.
IM
ALU
j
Reg
DM
Reg
• Fortunately, jumps are very infrequent – only ~3% of the
instruction mix
6
Supporting ID Stage Jumps
Jump
PCSrc
ID/EX
Shift
left 2
IF/ID
EX/MEM
Control
Add
PC+4[31-28]
4
PC
Instruction
Memory
Read
Address
Shift
left 2
Add
Read Addr 1
Data
Memory
Register Read
0
Read Addr 2Data 1
File
Write Addr
Write Data
16
Sign
Extend
Read
Data 2
32
MEM/WB
Branch
ALU
Address
Read
Data
Write Data
ALU
cntrl
Forward
Unit
7
8
9
Datapath Branch and Jump Hardware
Jump
PCSrc
ID/EX
Shift
left 2
IF/ID
EX/MEM
Control
Add
PC+4[31-28]
4
PC
Instruction
Memory
Read
Address
Shift
left 2
Add
Read Addr 1
Data
Memory
Register Read
Read Addr 2Data 1
File
Write Addr
Write Data
16
Sign
Extend
Read
Data 2
32
MEM/WB
Branch
ALU
Address
Read
Data
Write Data
ALU
cntrl
Forward
Unit
11
Branch Hazard
• The delay is determining the next instruction to
fetch is called “Control” or “Branch” hazard
• Arising from the need to make decision based
on the results of one instruction while other are
executing
• Pipeline can not know what the next instruction
should be (in only receives branch instruction
from memory)
• Whether do branch or not is not know (taken)
until MEM stage
• These type of hazards occur less frequently
– 15-20 % of all instructions are branches
12
Solution 1: Always stall
• Assume extra H/W is added to test
registers calculate the branch target
address and update the PC during the
second stage (see fig 6.7 and page 13)
– The lw instr. is executed if the branch fails (fig
6.8 A) is stalled 200ps clock cycle before
starting.
– Disadvantage: For longer pipelines will
slowdown the total exec. time.
13
Branch Hazard - Stall on Branch
FIGURE 6.7 Pipeline showing stalling on every conditional branch as solution to control hazards. This example
assumes the conditional branch is taken, and the instruction at the destination of the branch is the OR instruction. There
is a one-stage pipeline stall, or bubble, after the branch. In reality, the process of creating a stall is slightly more
complicated, as we will see in Section 6.6. The effect on performance, however, is the same as would occur if a bubble
were inserted. Page 380
14
Solution 2: (Prediction) need to add H/W for
flushing instructions if are wrong
• Assume branch is not taken (Fig 6.8)
– If so, pipeline proceeds at full speed
– Fetch the next instruction in program order
– When the branch is resolved:
• If the branch is not taken, keep going, no problem
• If the branch is taken, we need to flush 3 instructions in the
pipeline
– We use nops to discard instructions in the IF, ID, EX
stage.
– In the case of branches, we need to flush the
instructions from the pipeline, so that they don't have
any effect
15
Branch Hazard - Prediction
FIGURE 6.8 Predicting that branches are not taken as a solution to control hazard. (A)The top drawing shows the
pipeline when the branch is not taken. (B) The bottom drawing shows the pipeline when the branch is taken. As we noted
in Figure 6.7, the insertion of a bubble in this fashion simplifies what actually happens, at least during the first clock cycle
immediately following the branch. Section 6.6 will reveal the details.
16
Case of Dependence
• We do not know which instructions actually
follows the branch, until MEM stage of branch
instruction
• The pipeline, however will have already fetched
3 instructions (and or add) from the not taken
path. (See Figure 6.37)
• Stalls reduce performance
– But are required to get correct results
• Compiler can arrange code to avoid hazards
and stalls
– Requires knowledge of the pipeline structure
17
The trouble with branches
Flush these
instructions
(Set control
values to 0)
PC
FIGURE 6.37 The impact of the pipeline on the branch instruction. The numbers to the left of the instruction (40, 44,
. . . ) are the addresses of the instructions. Since the branch instruction decides whether to branch in the MEM stage—
clock cycle 4 for the beq instruction above—the three sequential instructions that follow the branch will be fetched and
begin execution. Without intervention, those three following instructions will begin execution before beq branches to lw at
location 72. (Figure 6.7 assumed extra hardware to reduce the control hazard to one clock cycle; this figure uses the
nonoptimized datapath.)
18
The trouble with branches
O
r
d
e
r
beq
IM
Reg
ALU
I
n
s
t
r.
DM
Reg
Fix branch
hazard by
waiting –
stall – but
affects CPI
flush
flush
flush
IM
Reg
ALU
beq target
DM
Reg
19
Reducing Branch Delay
• Move hardware to determine outcome to ID
stage
– Target address adder
– Register comparator
• Example: branch taken
36:
40:
44:
48:
52:
56:
72:
sub
beq
and
or
add
slt
...
lw
$10,
$1,
$12,
$13,
$14,
$15,
$4,
$3,
$2,
$2,
$4,
$6,
$8
7
$5
$6
$2
$7
$4, 50($7)
20
Example: Branch Taken
21
Example: Branch Taken
22
Solutions for branch hazards
• Say, we move branch logic earlier in the
pipeline:
– ID stage is the earliest time possible
– Need circuitry to calculate branch target address
• Easy, since we have PC and offset from the IF stage
– Need circuitry to evaluate branch condition:
• Harder!
• Branch condition may be dependent on earlier instructions!
• Moving the condition checking earlier introduces more data
hazards between the branch and earlier instructions on which
the branch depends! (see page 3, 4 and 5)
23
Solutions for branch hazards
• Say, we move branch logic earlier in the pipeline:
– Need to take care of data hazards before the branch
– Forwarding from the EX/MEM and the MEM/WB stage if
the branch depends on prior instruction (page 4)
– Data hazard can still occur, if the immediately preceding
instruction generates a register which is used for the
comparison in the branch.
– At decode stage we need to decide whether we should
bypass the ALU and use the dedicated branch condition
logic (see page 21)
24
Moving Branch Decisions Earlier in Pipe
• Move the branch decision hardware back to the EX stage
– Reduces the number of stall (flush) cycles to two
– Adds an and gate and a 2x1 mux to the EX timing path
• Add hardware to compute the branch target address and
evaluate the branch decision to the ID stage
– Reduces the number of stall (flush) cycles to one
(like with jumps)
• But now need to add forwarding hardware in ID stage
– Computing branch target address can be done in parallel with
RegFile read (done for all instructions – only used when needed)
– Comparing the registers can’t be done until after RegFile read,
so comparing and updating the PC adds a mux, a comparator,
and an and gate to the ID timing path
• For deeper pipelines, branch decision points can be even
later in the pipeline, be needing more stalls
25
ID Branch Forwarding Issues
• MEM/WB “forwarding” is
taken care of by the
normal RegFile write
before read operation
(during same clock cycle)
• Need to forward from the
EX/MEM pipeline stage to
the ID comparison
hardware for cases like
if (IDcontrol.Branch
and (EX/MEM.RegisterRd
and (EX/MEM.RegisterRd
ForwardC = 1
if (IDcontrol.Branch
and (EX/MEM.RegisterRd
and (EX/MEM.RegisterRd
ForwardD = 1
WB
MEM
EX
ID
IF
add3
$1,
add2
$3,
add1
$4,
beq
$1,$2,Loop
next_seq_instr
WB
MEM
EX
ID
IF
add3
$3,
add2
$1,
add1
$4,
beq
$1,$2,Loop
next_seq_instr
Forwards the
result from the
second
previous instr.
!= 0)
to either input
= IF/ID.RegisterRt)) of the compare
!= 0)
= IF/ID.RegisterRs))
26
27
Supporting ID Stage Branches
Branch
PCSrc
IF/ID
Add
PC
Instruction
Memory
Read
Address
IF.Flush
4
ID/EX
0
0
Control
Shift
left 2
EX/MEM
1
Add
Read Addr 1
MEM/WB
Compare
Hazard
Unit
Data
Memory
RegFile
0
Read Addr 2
Read Data 1
Write Addr
ReadData 2
Write Data
16
Sign
Extend 32
ALU
Read Data
Address
Write Data
ALU
cntrl
Forward
Unit
Forward
Unit
28
Summary: Static Branch Prediction
•
Resolve branch hazards by assuming a given outcome
and proceeding without waiting to see the actual branch
outcome
1. Predict not taken – always predict branches will not be
taken, continue to fetch from the sequential instruction
stream, only when branch is taken does the pipeline stall
– If taken, flush instructions after the branch (earlier in the pipeline)
•
•
•
in IF, ID, and EX stages if branch logic in MEM – three stalls
In IF and ID stages if branch logic in EX – two stalls
in IF stage if branch logic in ID – one stall
– ensure that those flushed instructions haven’t changed the
machine state – automatic in the MIPS pipeline since machine
state changing operations are at the tail end of the pipeline
(MemWrite (in MEM) or RegWrite (in WB))
– restart the pipeline at the branch destination
29
Flushing
•
DM
IM
Reg
IM
Reg
DM
Reg
Reg
DM
ALU
20 or r8,$1,$9
Reg
ALU
16 and $6,$1,$7
O
r
d
e
r
IM
ALU
8 flush
sub $4,$1,$5
Reg
ALU
4 beq $1,$2,2
I
n
s
t
r.
IM
Reg
DM
Reg
To flush the IF stage instruction, assert IF.Flush to
zero the instruction field of the IF/ID pipeline register
(transforming it into a noop)
31
32
33
Dynamic Branch Prediction
(Topic for Term Project)
• In deeper and superscalar pipelines, branch
penalty is more significant
• Use dynamic prediction
–
–
–
–
Branch prediction buffer (aka branch history table)
Indexed by recent branch instruction addresses
Stores outcome (taken/not taken)
To execute a branch
• Check table, expect the same outcome
• Start fetching from fall-through or target
• If wrong, flush pipeline and flip prediction
34
Dynamic Branch Prediction
• A branch prediction buffer (aka branch history table
(BHT)) in the IF stage addressed by the lower bits of the
PC, contains a bit passed to the ID stage through the
IF/ID pipeline register that tells whether the branch was
taken the last time it was execute
– Prediction bit may predict incorrectly (may be a wrong prediction
for this branch this iteration or may be from a different branch
with the same low order PC bits) but the doesn’t affect
correctness, just performance
• Branch decision occurs in the ID stage after determining that the
fetched instruction is a branch and checking the prediction bit
– If the prediction is wrong, flush the incorrect instruction(s) in
pipeline, restart the pipeline with the right instruction, and invert
the prediction bit
• A 4096 bit BHT varies from 1% misprediction (nasa7, tomcatv) to
18% (eqntott)
35
Branch Target Buffer
• The BHT predicts when a branch is taken, but does not
tell where its taken to!
– A branch target buffer (BTB) in the IF stage can cache the
branch target address, but we also need to fetch the next
sequential instruction. The prediction bit in IF/ID selects which
“next” instruction will be loaded into IF/ID at the next clock edge
• Would need a two read port
instruction memory
– Or the BTB can cache the
branch taken instruction while the
instruction memory is fetching the
next sequential instruction
PC
BTB
Instruction
Memory
0
Read
Address
• If the prediction is correct, stalls can be avoided no matter
which direction they go
36
•
1-bit Prediction Accuracy
A 1-bit predictor will be incorrect twice when not taken
– Assume predict_bit = 0 to start (indicating
branch not taken) and loop control is at
Loop:
the bottom of the loop code
1. First time through the loop, the predictor
mispredicts the branch since the branch is
taken back to the top of the loop; invert
prediction bit (predict_bit = 1)
2. As long as branch is taken (looping),
prediction is correct
3. Exiting the loop, the predictor again
mispredicts the branch since this time the
branch is not taken falling out of the loop;
invert prediction bit (predict_bit = 0)
•
1st loop instr
2nd loop instr
.
.
.
last loop instr
bne $1,$2,Loop
fall out instr
For 10 times through the loop we have a 80% prediction
accuracy for a branch that is taken 90% of the time
37
1-Bit Predictor: Shortcoming
• Inner loop branches mispredicted twice!
outer: …
…
inner: …
…
beq …, …, inner
…
beq …, …, outer
– Mispredict as taken on last iteration of
inner loop
– Then mispredict as not taken on first
iteration of inner loop next time around
38
2-bit Predictors
• A 2-bit scheme can give 90% accuracy since a prediction must be
wrong twice before the prediction bit is changed
right 9 times
wrong on loop
Taken
fall out
1 Predict 11
Taken
Taken
Taken
Predict 1
10
Taken
right on 1st
iteration
0 Predict 01
Not Taken
Not taken
Not taken
Not taken
00Predict 0
Not Taken
Taken
Loop: 1st loop instr
2nd loop instr
.
.
.
last loop instr
bne $1,$2,Loop
fall out instr
Not taken
•
BHT also
stores the
initial FSM
state
40