Question? Leave a message!




Branch Prediction and Speculative Execution

Branch Prediction and Speculative Execution
1 Branch Prediction and Speculative Execution Arvind Computer Science and Artificial Intelligence Laboratory M.I.T. Based on the material prepared by Krste Asanovic and Arvind 6.823 L13-2 Arvind Outline • Control transfer penalty • Branch prediction schemes • Branch misprediction recovery schemes October 26, 2005 6.823 L13-3 Arvind Phases of Instruction Execution PC Fetch: Instruction bits retrieved I-cache from cache. Fetch Buffer Decode: Instructions placed in appropriate issue (aka “dispatch”) stage buffer Issue Buffer Execute: Instructions and operands sent to Func. execution units . Units When execution completes, all results and exception flags are available. Result Buffer Commit: Instruction irrevocably updates architectural state (aka “graduation” or Arch. “completion”). State October 26, 2005 6.823 L13-4 Arvind Fetch Stage PC Instruction Cache Hit? Opcode Rd Rsrc1 Rsrc2/Imm Fetch Buffer Instructions To Decode Stage October 26, 2005 6.823 L13-5 Arvind Decode & Rename Stage Opcode Rd Rsrc1 Rsrc2/Imm (Renaming is shown only for Rsrc2, similar for Rsrc1) R31 R31 V Tag SignExt Committed R30 Rename R30 V Tag Architectural Table Regfile R0 R0 V Tag 1 X 0 1 0 1 0 1 1 ImmSel 0 1 0 1 Opcode U E P1 Tag1 Data1 P2 Tag2 Data2 Pd Rd Datad Cause t1 t2 ROB tn October 26, 2005 6.823 L13-6 Arvind Execute Stage • Arbiter selects one ready instruction (P1=1 AND P2=1) to execute • Instruction reads operands from ROB, executes, and broadcasts tag and result to waiting instructions in ROB. Also saves result and exception flags for commit. Opcode U E P1 Tag1 Data1 P2 Tag2 Data2 Pd Rd Datad Cause t1 t2 ROB Opcode U E P1 Tag1 Data1 P2 Tag2 Data2 Pd Rd Datad Cause tn Func. Unit October 26, 2005 6.823 L13-7 Arvind Commit Stage • When instruction at ptr2 (commit point) has completed, write back result to architectural state and check for exceptions • Check if rename table entry for architectural register written matches tag, if so, clear valid bit in rename table t1 ptr2 Opcode U E P1 Tag1 Data1 P2 Tag2 Data2 Pd Rd Datad Cause ROB tn Exception? R31 Rename V Tag R31 Committed R30 V Tag R30 Table Architectural Regfile R0 R0 V Tag =? Clear rename valid? October 26, 2005 6.823 L13-8 Arvind Branch Penalty PC Next fetch started Fetch I-cache Fetch Modern processors may Buffer Decode have 10 pipeline stages between next PC calculation Issue and branch resolution Buffer Execute Func. Units Result Branch executed Buffer Commit Arch. State October 26, 2005 6.823 L13-9 Arvind Average Run-Length between Branches Average dynamic instruction mix from SPEC92: SPECint92 SPECfp92 ALU 39 % 13 % FPU Add 20 % FPU Mult 13 % load 26 % 23 % store 9 % 9 % branch 16 % 8 % other 10 % 12 % SPECint92: compress, eqntott, espresso, gcc, li SPECfp92: doduc, ear, hydro2d, mdijdp2, su2cor What is the average run length between branches October 26, 2005 6.823 L13-10 Arvind Reducing Control Transfer Penalties Software solution • loop unrolling Increases the run length • instruction scheduling Compute the branch condition as early as possible (limited) Hardware solution • delay slots replaces pipeline bubbles with useful work (requires software cooperation) • branch prediction & speculative execution of instructions beyond the branch October 26, 2005 6.823 L13-11 Arvind MIPS Branches and Jumps Need to know (or guess) both target address and whether the branch/jump is taken or not Instruction Taken known? Target known? After Reg. Fetch After Inst. Fetch BEQZ/BNEZ Always Taken After Inst. Fetch J Always Taken After Reg. Fetch JR October 26, 2005 6.823 L13-12 Arvind Branch Penalties in Modern Pipelines UltraSPARC-III instruction fetch pipeline stages (in-order issue, 4-way superscalar, 750MHz, 2000) A PC Generation/Mux P Instruction Fetch Stage 1 F Instruction Fetch Stage 2 Branch Target B Branch Address Calc/Begin Decode Address I Complete Decode Known J Steer Instructions to Functional units Branch R Register File Read Direction & E Integer Execute Jump Register Remainder of execute pipeline Target (+ another 6 stages) Known October 26, 2005 6.823 L13-13 Arvind Outline • Control transfer penalty • Branch prediction schemes • Branch misprediction recovery schemes October 26, 2005 6.823 L13-14 Arvind Branch Prediction Motivation: branch penalties limit performance of deeply pipelined processors Modern branch predictors have high accuracy (95%) and can reduce branch penalties significantly Required hardware support: Prediction structures: branch history tables, branch target buffers, etc. Mispredict recovery mechanisms: • In-order machines: kill instructions following branch in pipeline • Out-of-order machines: shadow registers and memory buffers for each speculated branch October 26, 2005 6.823 L13-15 Arvind Static Branch Prediction Overall probability a branch is taken is 60-70% but: JZ backward forward 90% 50% JZ ISA can attach additional semantics to branches about preferred direction, e.g., Motorola MC88110 bne0 (preferred taken) beq0 (not taken) ISA can allow arbitrary choice of statically predicted direction (HP PA-RISC, Intel IA-64) October 26, 2005 6.823 L13-16 Arvind Dynamic Branch Prediction learning based on past behavior Temporal correlation The way a branch resolves may be a good predictor of the way it will resolve at the next execution Spatial correlation Several branches may resolve in a highly correlated manner (a preferred path of execution) October 26, 2005 6.823 L13-17 Arvind Branch Prediction Bits • Assume 2 BP bits per instruction • Change the prediction after two consecutive mistakes ¬take wrong ¬taken taken taken taken ¬take take ¬taken right right taken ¬taken ¬taken take wrong BP state: (predict take/¬take) x (last prediction right/wrong) October 26, 2005 6.823 L13-18 Arvind Branch History Table Fetch PC 00 k k 2 -entry BHT, I-Cache BHT Index 2 bits/entry Instruction Opcode offset + Branch? Taken/¬Taken? Target PC 4K-entry BHT, 2 bits/entry, 80-90% correct predictions October 26, 2005 6.823 L13-19 Arvind Two-Level Branch Predictor Pentium Pro uses the result from the last two branches to select one of the four sets of BHT bits (95% correct) 00 Fetch PC k 2-bit global branch history shift register Shift in Taken/¬Taken results of each branch Taken/¬Taken? October 26, 2005 6.823 L13-20 Arvind Exploiting Spatial Correlation Yeh and Patt, 1992 if (xi 7) then y += 1; if (xi 5) then c -= 4; If first condition false, second condition also false History bit: H records the direction of the last branch executed by the processor Two sets of BHT bits (BHT0 & BHT1) per branch instruction H = 0 (not taken) ⇒ consult BHT0 H = 1 (taken) ⇒ consult BHT1 October 26, 2005 6.823 L13-21 Arvind Limitations of BHTs Cannot redirect fetch stream until after branch instruction is fetched and decoded, and target address determined A PC Generation/Mux Correctly P predicted Instruction Fetch Stage 1 taken branch F Instruction Fetch Stage 2 penalty B Branch Address Calc/Begin Decode I Complete Decode Jump Register J Steer Instructions to Functional units penalty R Register File Read E Integer Execute Remainder of execute pipeline (+ another 6 stages) UltraSPARC-III fetch pipeline October 26, 2005 6.823 L13-22 Arvind Branch Target Buffer predicted BPb target Branch Target IMEM Buffer k (2 entries) k PC target BP BP bits are stored with the predicted target address. IF stage: If (BP=taken) then nPC=target else nPC=PC+4 later: check prediction, if wrong then kill the instruction and update BTB & BPb else update BPb October 26, 2005 6.823 L13-23 Arvind Address Collisions 132 Jump 100 Assume a 128-entry 1028 Add ..... BTB target BPb 236 take Instruction Memory What will be fetched after the instruction at 1028? BTB prediction = 236 Correct target = 1032 ⇒ kill PC=236 and fetch PC=1032 Is this a common occurrence? Can we avoid these bubbles? October 26, 2005 6.823 L13-24 Arvind BTB should be for Control Transfer instructions only BTB contains useful information for branch and jump instructions only ⇒ it should not be updated for other instructions For all other instructions the next PC is (PC)+4 How to achieve this effect without decoding the instruction? October 26, 2005 6.823 L13-25 Arvind Branch Target Buffer (BTB) k 2 -entry direct-mapped BTB I-Cache PC (can also be associative) Entry PC predicted Valid target PC k = match target valid • Keep both the branch PC and target PC in the BTB • PC+4 is fetched if match fails •Only taken branches and jumps held in BTB • Next PC determined before branch fetched and decoded October 26, 2005 6.823 L13-26 Arvind Consulting BTB Before Decoding 132 Jump 100 entry PC target BPb 1028 Add ..... 132 236 take • The match for PC=1028 fails and 1028+4 is fetched ⇒ eliminates false predictions after ALU instructions • BTB contains entries only for control transfer instructions ⇒ more room to store branch targets October 26, 2005 6.823 L13-27 Arvind Combining BTB and BHT • BTB entries are considerably more expensive than BHT, but can redirect fetches at earlier stage in pipeline and can accelerate indirect branches (JR) • BHT can hold many more entries and is more accurate A PC Generation/Mux P BTB Instruction Fetch Stage 1 F Instruction Fetch Stage 2 BHT in later B BHT Branch Address Calc/Begin Decode pipeline stage I Complete Decode corrects when J Steer Instructions to Functional units BTB misses a predicted R Register File Read taken branch E Integer Execute BTB/BHT only updated after branch resolves in E stage October 26, 2005 6.823 L13-28 Arvind Uses of Jump Register (JR) • Switch statements (jump to address of matching case) BTB works well if same case used repeatedly • Dynamic function call (jump to run-time function address) BTB works well if same function usually called, (e.g., in C++ programming, when objects have same type in virtual function call) • Subroutine returns (jump to return address) BTB works well if usually return to the same place ⇒ Often one function called from many different call sites How well does BTB work for each of these cases? October 26, 2005 6.823 L13-29 Arvind Subroutine Return Stack Small structure to accelerate JR for subroutine returns, typically much more accurate than BTBs. fa() fb(); fb() fc(); fc() fd(); Pop return address Push call address when when subroutine function call executed return decoded &fd() k entries (typically k=8-16) &fc() &fb() October 26, 2005 6.823 L13-30 Arvind Outline • Control transfer penalty • Branch prediction schemes • Branch misprediction recovery schemes Five-minute break to stretch your legs October 26, 2005 6.823 L13-31 Arvind Mispredict Recovery In-order execution machines: – Assume no instruction issued after branch can write-back before branch resolves – Kill all instructions in pipeline behind mispredicted branch Out-of-order execution? –Multiple instructions following branch in program order can complete before branch resolves October 26, 2005 6.823 L13-32 Arvind In-Order Commit for Precise Exceptions In-order Out-of-order In-order Commit Reorder Buffer Fetch Decode Kill Kill Kill Execute Exception? Inject handler PC • Instructions fetched and decoded into instruction reorder buffer in-order • Execution is out-of-order ( ⇒ out-of-order completion) • Commit (write-back to architectural state, i.e., regfile & memory, is in-order Temporary storage needed in ROB to hold results before commit October 26, 2005 6.823 L13-33 Arvind Extensions for Precise Exceptions Inst use exec op p1 src1 p2 src2 pd dest data cause ptr 2 next to commit ptr 1 next available Reorder buffer • add pd, dest, data, cause fields in the instruction template • commit instructions to reg file and memory in program order ⇒ buffers can be maintained circularly • on exception, clear reorder buffer by resetting ptr =ptr 1 2 (stores must wait for commit before updating memory) October 26, 2005 6.823 L13-34 Arvind Branch Misprediction Recovery Inst use exec op p1 src1 p2 src2 pd dest data cause ptr 2 next to commit BEQZ rollback next Speculative Instructions available ptr 1 next available Reorder buffer On mispredict • Roll back “next available” pointer to just after branch • Reset use bits • Flush mis-speculated instructions from pipelines • Restart fetch on correct branch path October 26, 2005 6.823 L13-35 Arvind Branch Misprediction in Pipeline Inject correct PC Branch Branch Kill Resolution Prediction Kill Kill Commit Reorder Buffer Fetch Decode PC Complete Execute • Can have multiple unresolved branches in ROB • Can resolve branches out-of-order by killing all the instructions in ROB that follow a mispredicted branch October 26, 2005 6.823 L13-36 Arvind Recovering Renaming Table t v t v t v r t v 1 Register Rename Rename r 2 File Snapshots Table Ins use exec op p1 src1 p2 src2 pd dest data t 1 t 2 Reorder . buffer . t n Commit Load Store FU FU FU Unit Unit t, result Take snapshot of register rename table at each predicted branch, recover earlier snapshot if branch mispredicted October 26, 2005 6.823 L13-37 Arvind Speculating Both Directions An alternative to branch prediction is to execute both directions of a branch speculatively • resource requirement is proportional to the number of concurrent speculative executions • only half the resources engage in useful work when both directions of a branch are executed speculatively • branch prediction takes less resources than speculative execution of both paths With accurate branch prediction, it is more cost effective to dedicate all resources to the predicted direction October 26, 2005 38 Thank you
Website URL
Comment