Question? Leave a message!

Branch Prediction and Speculative Execution

Branch Prediction and Speculative Execution
1 Branch Prediction and Speculative Execution Arvind Computer Science and Artificial Intelligence Laboratory M.I.T. Based on the material prepared by Krste Asanovic and Arvind 6.823 L132 Arvind Outline • Control transfer penalty • Branch prediction schemes • Branch misprediction recovery schemes October 26, 2005 6.823 L133 Arvind Phases of Instruction Execution PC Fetch: Instruction bits retrieved Icache from cache. Fetch Buffer Decode: Instructions placed in appropriate issue (aka “dispatch”) stage buffer Issue Buffer Execute: Instructions and operands sent to Func. execution units . Units When execution completes, all results and exception flags are available. Result Buffer Commit: Instruction irrevocably updates architectural state (aka “graduation” or Arch. “completion”). State October 26, 2005 6.823 L134 Arvind Fetch Stage PC Instruction Cache Hit Opcode Rd Rsrc1 Rsrc2/Imm Fetch Buffer Instructions To Decode Stage October 26, 2005 6.823 L135 Arvind Decode Rename Stage Opcode Rd Rsrc1 Rsrc2/Imm (Renaming is shown only for Rsrc2, similar for Rsrc1) R31 R31 V Tag SignExt Committed R30 Rename R30 V Tag Architectural Table Regfile R0 R0 V Tag 1 X 0 1 0 1 0 1 1 ImmSel 0 1 0 1 Opcode U E P1 Tag1 Data1 P2 Tag2 Data2 Pd Rd Datad Cause t1 t2 ROB tn October 26, 2005 6.823 L136 Arvind Execute Stage • Arbiter selects one ready instruction (P1=1 AND P2=1) to execute • Instruction reads operands from ROB, executes, and broadcasts tag and result to waiting instructions in ROB. Also saves result and exception flags for commit. Opcode U E P1 Tag1 Data1 P2 Tag2 Data2 Pd Rd Datad Cause t1 t2 ROB Opcode U E P1 Tag1 Data1 P2 Tag2 Data2 Pd Rd Datad Cause tn Func. Unit October 26, 2005 6.823 L137 Arvind Commit Stage • When instruction at ptr2 (commit point) has completed, write back result to architectural state and check for exceptions • Check if rename table entry for architectural register written matches tag, if so, clear valid bit in rename table t1 ptr2 Opcode U E P1 Tag1 Data1 P2 Tag2 Data2 Pd Rd Datad Cause ROB tn Exception R31 Rename V Tag R31 Committed R30 V Tag R30 Table Architectural Regfile R0 R0 V Tag = Clear rename valid October 26, 2005 6.823 L138 Arvind Branch Penalty PC Next fetch started Fetch Icache Fetch Modern processors may Buffer Decode have 10 pipeline stages between next PC calculation Issue and branch resolution Buffer Execute Func. Units Result Branch executed Buffer Commit Arch. State October 26, 2005 6.823 L139 Arvind Average RunLength between Branches Average dynamic instruction mix from SPEC92: SPECint92 SPECfp92 ALU 39 13 FPU Add 20 FPU Mult 13 load 26 23 store 9 9 branch 16 8 other 10 12 SPECint92: compress, eqntott, espresso, gcc, li SPECfp92: doduc, ear, hydro2d, mdijdp2, su2cor What is the average run length between branches October 26, 2005 6.823 L1310 Arvind Reducing Control Transfer Penalties Software solution • loop unrolling Increases the run length • instruction scheduling Compute the branch condition as early as possible (limited) Hardware solution • delay slots replaces pipeline bubbles with useful work (requires software cooperation) • branch prediction speculative execution of instructions beyond the branch October 26, 2005 6.823 L1311 Arvind MIPS Branches and Jumps Need to know (or guess) both target address and whether the branch/jump is taken or not Instruction Taken known Target known After Reg. Fetch After Inst. Fetch BEQZ/BNEZ Always Taken After Inst. Fetch J Always Taken After Reg. Fetch JR October 26, 2005 6.823 L1312 Arvind Branch Penalties in Modern Pipelines UltraSPARCIII instruction fetch pipeline stages (inorder issue, 4way superscalar, 750MHz, 2000) A PC Generation/Mux P Instruction Fetch Stage 1 F Instruction Fetch Stage 2 Branch Target B Branch Address Calc/Begin Decode Address I Complete Decode Known J Steer Instructions to Functional units Branch R Register File Read Direction E Integer Execute Jump Register Remainder of execute pipeline Target (+ another 6 stages) Known October 26, 2005 6.823 L1313 Arvind Outline • Control transfer penalty • Branch prediction schemes • Branch misprediction recovery schemes October 26, 2005 6.823 L1314 Arvind Branch Prediction Motivation: branch penalties limit performance of deeply pipelined processors Modern branch predictors have high accuracy (95) and can reduce branch penalties significantly Required hardware support: Prediction structures: branch history tables, branch target buffers, etc. Mispredict recovery mechanisms: • Inorder machines: kill instructions following branch in pipeline • Outoforder machines: shadow registers and memory buffers for each speculated branch October 26, 2005 6.823 L1315 Arvind Static Branch Prediction Overall probability a branch is taken is 6070 but: JZ backward forward 90 50 JZ ISA can attach additional semantics to branches about preferred direction, e.g., Motorola MC88110 bne0 (preferred taken) beq0 (not taken) ISA can allow arbitrary choice of statically predicted direction (HP PARISC, Intel IA64) October 26, 2005 6.823 L1316 Arvind Dynamic Branch Prediction learning based on past behavior Temporal correlation The way a branch resolves may be a good predictor of the way it will resolve at the next execution Spatial correlation Several branches may resolve in a highly correlated manner (a preferred path of execution) October 26, 2005 6.823 L1317 Arvind Branch Prediction Bits • Assume 2 BP bits per instruction • Change the prediction after two consecutive mistakes ¬take wrong ¬taken taken taken taken ¬take take ¬taken right right taken ¬taken ¬taken take wrong BP state: (predict take/¬take) x (last prediction right/wrong) October 26, 2005 6.823 L1318 Arvind Branch History Table Fetch PC 00 k k 2 entry BHT, ICache BHT Index 2 bits/entry Instruction Opcode offset + Branch Taken/¬Taken Target PC 4Kentry BHT, 2 bits/entry, 8090 correct predictions October 26, 2005 6.823 L1319 Arvind TwoLevel Branch Predictor Pentium Pro uses the result from the last two branches to select one of the four sets of BHT bits (95 correct) 00 Fetch PC k 2bit global branch history shift register Shift in Taken/¬Taken results of each branch Taken/¬Taken October 26, 2005 6.823 L1320 Arvind Exploiting Spatial Correlation Yeh and Patt, 1992 if (xi 7) then y += 1; if (xi 5) then c = 4; If first condition false, second condition also false History bit: H records the direction of the last branch executed by the processor Two sets of BHT bits (BHT0 BHT1) per branch instruction H = 0 (not taken) ⇒ consult BHT0 H = 1 (taken) ⇒ consult BHT1 October 26, 2005 6.823 L1321 Arvind Limitations of BHTs Cannot redirect fetch stream until after branch instruction is fetched and decoded, and target address determined A PC Generation/Mux Correctly P predicted Instruction Fetch Stage 1 taken branch F Instruction Fetch Stage 2 penalty B Branch Address Calc/Begin Decode I Complete Decode Jump Register J Steer Instructions to Functional units penalty R Register File Read E Integer Execute Remainder of execute pipeline (+ another 6 stages) UltraSPARCIII fetch pipeline October 26, 2005 6.823 L1322 Arvind Branch Target Buffer predicted BPb target Branch Target IMEM Buffer k (2 entries) k PC target BP BP bits are stored with the predicted target address. IF stage: If (BP=taken) then nPC=target else nPC=PC+4 later: check prediction, if wrong then kill the instruction and update BTB BPb else update BPb October 26, 2005 6.823 L1323 Arvind Address Collisions 132 Jump 100 Assume a 128entry 1028 Add ..... BTB target BPb 236 take Instruction Memory What will be fetched after the instruction at 1028 BTB prediction = 236 Correct target = 1032 ⇒ kill PC=236 and fetch PC=1032 Is this a common occurrence Can we avoid these bubbles October 26, 2005 6.823 L1324 Arvind BTB should be for Control Transfer instructions only BTB contains useful information for branch and jump instructions only ⇒ it should not be updated for other instructions For all other instructions the next PC is (PC)+4 How to achieve this effect without decoding the instruction October 26, 2005 6.823 L1325 Arvind Branch Target Buffer (BTB) k 2 entry directmapped BTB ICache PC (can also be associative) Entry PC predicted Valid target PC k = match target valid • Keep both the branch PC and target PC in the BTB • PC+4 is fetched if match fails •Only taken branches and jumps held in BTB • Next PC determined before branch fetched and decoded October 26, 2005 6.823 L1326 Arvind Consulting BTB Before Decoding 132 Jump 100 entry PC target BPb 1028 Add ..... 132 236 take • The match for PC=1028 fails and 1028+4 is fetched ⇒ eliminates false predictions after ALU instructions • BTB contains entries only for control transfer instructions ⇒ more room to store branch targets October 26, 2005 6.823 L1327 Arvind Combining BTB and BHT • BTB entries are considerably more expensive than BHT, but can redirect fetches at earlier stage in pipeline and can accelerate indirect branches (JR) • BHT can hold many more entries and is more accurate A PC Generation/Mux P BTB Instruction Fetch Stage 1 F Instruction Fetch Stage 2 BHT in later B BHT Branch Address Calc/Begin Decode pipeline stage I Complete Decode corrects when J Steer Instructions to Functional units BTB misses a predicted R Register File Read taken branch E Integer Execute BTB/BHT only updated after branch resolves in E stage October 26, 2005 6.823 L1328 Arvind Uses of Jump Register (JR) • Switch statements (jump to address of matching case) BTB works well if same case used repeatedly • Dynamic function call (jump to runtime function address) BTB works well if same function usually called, (e.g., in C++ programming, when objects have same type in virtual function call) • Subroutine returns (jump to return address) BTB works well if usually return to the same place ⇒ Often one function called from many different call sites How well does BTB work for each of these cases October 26, 2005 6.823 L1329 Arvind Subroutine Return Stack Small structure to accelerate JR for subroutine returns, typically much more accurate than BTBs. fa() fb(); fb() fc(); fc() fd(); Pop return address Push call address when when subroutine function call executed return decoded fd() k entries (typically k=816) fc() fb() October 26, 2005 6.823 L1330 Arvind Outline • Control transfer penalty • Branch prediction schemes • Branch misprediction recovery schemes Fiveminute break to stretch your legs October 26, 2005 6.823 L1331 Arvind Mispredict Recovery Inorder execution machines: – Assume no instruction issued after branch can writeback before branch resolves – Kill all instructions in pipeline behind mispredicted branch Outoforder execution –Multiple instructions following branch in program order can complete before branch resolves October 26, 2005 6.823 L1332 Arvind InOrder Commit for Precise Exceptions Inorder Outoforder Inorder Commit Reorder Buffer Fetch Decode Kill Kill Kill Execute Exception Inject handler PC • Instructions fetched and decoded into instruction reorder buffer inorder • Execution is outoforder ( ⇒ outoforder completion) • Commit (writeback to architectural state, i.e., regfile memory, is inorder Temporary storage needed in ROB to hold results before commit October 26, 2005 6.823 L1333 Arvind Extensions for Precise Exceptions Inst use exec op p1 src1 p2 src2 pd dest data cause ptr 2 next to commit ptr 1 next available Reorder buffer • add pd, dest, data, cause fields in the instruction template • commit instructions to reg file and memory in program order ⇒ buffers can be maintained circularly • on exception, clear reorder buffer by resetting ptr =ptr 1 2 (stores must wait for commit before updating memory) October 26, 2005 6.823 L1334 Arvind Branch Misprediction Recovery Inst use exec op p1 src1 p2 src2 pd dest data cause ptr 2 next to commit BEQZ rollback next Speculative Instructions available ptr 1 next available Reorder buffer On mispredict • Roll back “next available” pointer to just after branch • Reset use bits • Flush misspeculated instructions from pipelines • Restart fetch on correct branch path October 26, 2005 6.823 L1335 Arvind Branch Misprediction in Pipeline Inject correct PC Branch Branch Kill Resolution Prediction Kill Kill Commit Reorder Buffer Fetch Decode PC Complete Execute • Can have multiple unresolved branches in ROB • Can resolve branches outoforder by killing all the instructions in ROB that follow a mispredicted branch October 26, 2005 6.823 L1336 Arvind Recovering Renaming Table t v t v t v r t v 1 Register Rename Rename r 2 File Snapshots Table Ins use exec op p1 src1 p2 src2 pd dest data t 1 t 2 Reorder . buffer . t n Commit Load Store FU FU FU Unit Unit t, result Take snapshot of register rename table at each predicted branch, recover earlier snapshot if branch mispredicted October 26, 2005 6.823 L1337 Arvind Speculating Both Directions An alternative to branch prediction is to execute both directions of a branch speculatively • resource requirement is proportional to the number of concurrent speculative executions • only half the resources engage in useful work when both directions of a branch are executed speculatively • branch prediction takes less resources than speculative execution of both paths With accurate branch prediction, it is more cost effective to dedicate all resources to the predicted direction October 26, 2005 38 Thank you
Document Information
User Name:
User Type:
United States
Uploaded Date: