Question? Leave a message!




Complex Pipelining: Out-of-Order Execution & Register Renaming

Complex Pipelining: Out-of-Order Execution & Register Renaming
1 Complex Pipelining: OutofOrder Execution Register Renaming Arvind Computer Science and Artificial Intelligence Laboratory M.I.T. Based on the material prepared by Krste Asanovic and Arvind 6.823 L122 Arvind InOrder Issue Pipeline ALU Mem IF ID Issue WB Fadd GPR’s FPR’s Fmul Fdiv October 24, 2005 6.823 L123 Arvind Scoreboard for Inorder Issues BusyFU : a bitvector to indicate FU’s availability. (FU = Int, Add, Mult, Div) These bits are hardwired to FU's. WPreg : a bitvector to record the registers for which writes are pending. These bits are set to true by the Issue stage and set to false by the WB stage Issue checks the instruction (opcode dest src1 src2) against the scoreboard (Busy WP) to dispatch FU available BusyFU RAW WPsrc1 or WPsrc2 WAR cannot arise WAW WPdest October 24, 2005 6.823 L124 Arvind InOrder Issue Limitations: an example latency 2 1 1 LD F2, 34(R2) 1 2 LD F4, 45(R3) long 3 4 3 MULTD F6, F4, F2 3 4 SUBD F8, F2, F2 1 5 5 DIVD F4, F2, F8 4 6 ADDD F10, F6, F4 1 6 Inorder: 1 (2,1) . . . . . . 2 3 4 4 3 5 . . . 5 6 6 Inorder restriction prevents instruction 4 from being dispatched October 24, 2005 6.823 L125 Arvind OutofOrder Issue ALU Mem IF ID Issue WB Fadd Fmul • Issue stage buffer holds multiple instructions waiting to issue. • Decode adds next instruction to buffer if there is space and the instruction does not cause a WAR or WAW hazard. • Any instruction in buffer whose RAW hazards are satisfied can be issued (for now at most one dispatch per cycle). On a write back (WB), new instructions may get enabled. October 24, 2005 6.823 L126 Arvind InOrder Issue Limitations: an example latency 2 1 1 LD F2, 34(R2) 1 2 LD F4, 45(R3) long 3 4 3 MULTD F6, F4, F2 3 4 SUBD F8, F2, F2 1 5 5 DIVD F4, F2, F8 4 6 ADDD F10, F6, F4 1 6 Inorder: 1 (2,1) . . . . . . 2 3 4 4 3 5 . . . 5 6 6 Outoforder: 1 (2,1) 4 4 . . . . 2 3 . . 3 5 . . . 5 6 6 Outoforder execution did not allow any significant improvement October 24, 2005 6.823 L127 Arvind How many Instructions can be in the pipeline Which features of an ISA limit the number of instructions in the pipeline Number of Registers Which features of a program limit the number of instructions in the pipeline Control transfers Outoforder dispatch by itself does not provide any significant performance improvement October 24, 2005 6.823 L128 Arvind Overcoming the Lack of Register Names Floating Point pipelines often cannot be kept filled with small number of registers. IBM 360 had only 4 Floating Point Registers Can a microarchitecture use more registers than specified by the ISA without loss of ISA compatibility Robert Tomasulo of IBM suggested an ingenious solution in 1967 based on onthefly register renaming October 24, 2005 6.823 L129 Arvind InstructionLevel Parallelism with Renaming latency 2 1 1 LD F2, 34(R2) 1 2 LD F4, 45(R3) long 3 4 3 MULTD F6, F4, F2 3 X 4 SUBD F8, F2, F2 1 5 5 DIVD F4’, F2, F8 4 6 ADDD F10, F6, F4’ 1 6 Inorder: 1 (2,1) . . . . . . 2 3 4 4 3 5 . . . 5 6 6 Outoforder: 1 (2,1) 4 4 5 . . . 2 (3,5) 3 6 6 Any antidependence can be eliminated by renaming. (renaming ⇒ additional storage) Can it be done in hardware yes October 24, 2005 6.823 L1210 Arvind Register Renaming ALU Mem IF ID Issue WB Fadd Fmul • Decode does register renaming and adds instructions to the issue stage reorder buffer (ROB) ⇒ renaming makes WAR or WAW hazards impossible • Any instruction in ROB whose RAW hazards have been satisfied can be dispatched. ⇒ Outoforder or dataflow execution October 24, 2005 6.823 L1211 Arvind Renaming Outoforder Issue An example Renaming table Reorder buffer p data Ins use exec op p1 src1 p2 src2 t F1 1 t F2 2 . F3 . F4 . F5 F6 F7 F8 data / t i 1 LD F2, 34(R2) • When are names in sources 2 LD F4, 45(R3) replaced by data 3 MULTD F6, F4, F2 Whenever an FU produces data 4 SUBD F8, F2, F2 5 DIVD F4, F2, F8 • When can a name be reused 6 ADDD F10, F6, F4 Whenever an instruction completes October 24, 2005 6.823 L1212 Arvind DataDriven Execution Renaming table reg file Ins use exec op p1 src1 p2 src2 t 1 t Reorder 2 . buffer . t n Replacing the tag by its value Load Store FU FU is an expensive Unit Unit operation t, result • Instruction template (i.e., tag t) is allocated by the Decode stage, which also stores the tag in the reg file • When an instruction completes, its tag is deallocated October 24, 2005 6.823 L1213 Arvind Dataflow execution Ins use exec op p1 src1 p2 src2 t 1 ptr t 2 2 . next to . deallocate . prt 1 next available t n Reorder buffer Instruction slot is candidate for execution when: •It holds a valid instruction (“use” bit is set) •It has not already started execution (“exec” bit is clear) •Both operands are availble (p1 and p2 are set) October 24, 2005 6.823 L1214 Arvind Simplifying Allocation/Deallocation Ins use exec op p1 src1 p2 src2 t 1 ptr t 2 2 . next to . deallocate . prt 1 next available t n Reorder buffer Instruction buffer is managed circularly •“exec” bit is set when instruction begins execution •When an instruction completes its “use” bit is marked free •ptr is incremented only if the “use” bit is marked free 2 October 24, 2005 6.823 L1215 Arvind IBM 360/91 Floating Point Unit R. M. Tomasulo, 1967 data instructions 1 p 1 data load Floating 2 2 buffers Point 3 3 (from Reg 4 4 ... 5 memory) 6 distribute 1 pdata p data instruction 2 pdata p 1 data templates 3 2 by functional Adder Mult units t, result Common bus ensures that data is made p data store buffers available immediately to all the instructions (to memory) waiting for it October 24, 2005 6.823 L1216 Arvind Effectiveness Renaming and Outoforder execution was first implemented in 1969 in IBM 360/91 but did not show up in the subsequent models until mid Nineties. Why Reasons 1. Exceptions not precise 2. Effective on a very small class of programs One more problem needed to be solved Control transfers October 24, 2005 17 Fiveminute break to stretch your legs 6.823 L1218 Arvind Precise Interrupts It must appear as if an interrupt is taken between two instructions (say I and I ) i i+1 • the effect of all instructions up to and including I is i totally complete • no effect of any instruction after I has taken place i The interrupt handler either aborts the program or restarts it at I . i+1 October 24, 2005 6.823 L1219 Arvind Effect on Interrupts Outoforder Completion I DIVD f6, f6, f4 1 I LD f2, 45(r3) 2 I MULTD f0, f2, f4 3 I DIVD f8, f6, f2 4 I SUBD f10, f0, f6 5 I ADDD f6, f8, f2 6 outoforder comp 1 2 2 3 1 4 3 5 5 4 6 6 restore f2 restore f10 Consider interrupts Precise interrupts are difficult to implement at high speed want to start execution of later instructions before exception checks finished on earlier instructions October 24, 2005 6.823 L1220 Arvind Exception Handling Commit (InOrder FiveStage Pipeline) Point Inst. Data Decode PC D E M W + Mem Mem Illegal Data Addr Kill Overflow Select Opcode Except Writeback Handler PC Address PC Exceptions Exc Exc Exc Cause D E M PC PC PC EPC D E M Kill F Kill D Kill E Asynchronous Stage Stage Stage Interrupts • Hold exception flags in pipeline until commit point (M stage) • Exceptions in earlier pipe stages override later exceptions • Inject external interrupts at commit point (override others) • If exception at commit: update Cause and EPC registers, kill all stages, inject handler PC into fetch stage October 24, 2005 6.823 L1221 Arvind Phases of Instruction Execution PC Fetch: Instruction bits retrieved Icache from cache. Fetch Buffer Decode: Instructions placed in appropriate issue (aka “dispatch”) stage buffer Issue Buffer Execute: Instructions and operands sent to Func. execution units . Units When execution completes, all results and exception flags are available. Result Buffer Commit: Instruction irrevocably updates architectural state (aka “graduation” or Arch. “completion”). State October 24, 2005 6.823 L1222 Arvind InOrder Commit for Precise Exceptions Inorder Outoforder Inorder Commit Reorder Buffer Fetch Decode Kill Kill Kill Execute Exception Inject handler PC • Instructions fetched and decoded into instruction reorder buffer inorder • Execution is outoforder ( ⇒ outoforder completion) • Commit (writeback to architectural state, i.e., regfile memory, is inorder Temporary storage needed to hold results before commit (shadow registers and store buffers) October 24, 2005 6.823 L1223 Arvind Extensions for Precise Exceptions Inst use exec op p1 src1 p2 src2 pd dest data cause ptr 2 next to commit ptr 1 next available Reorder buffer • add pd, dest, data, cause fields in the instruction template • commit instructions to reg file and memory in program order ⇒ buffers can be maintained circularly • on exception, clear reorder buffer by resetting ptr =ptr 1 2 (stores must wait for commit before updating memory) October 24, 2005 6.823 L1224 Arvind Rollback and Renaming Register File (now holds only committed state) Ins use exec op p1 src1 p2 src2 pd dest data t 1 t 2 Reorder . buffer . t n Commit Load Store FU FU FU Unit Unit t, result Register file does not contain renaming tags any more. How does the decode stage find the tag of a source register Search the “dest” field in the reorder buffer October 24, 2005 6.823 L1225 Arvind Renaming Table r tag t v 1 Register Rename r valid bit 2 File Table Ins use exec op p1 src1 p2 src2 pd dest data t 1 t 2 Reorder . buffer . t n Commit Load Store FU FU FU Unit Unit t, result Renaming table is a cache to speed up register name look up. It needs to be cleared after each exception taken. When else are valid bits cleared Control transfers October 24, 2005 6.823 L1226 Arvind Branch Penalty PC Next fetch started Fetch Icache How many instructions Fetch need to be killed on a Buffer Decode misprediction Issue Buffer Modern processors may have 10 pipeline stages Execute Func. between next pc calculation Units and branch resolution Result Branch executed Buffer Commit Arch. State October 24, 2005 6.823 L1227 Arvind Average RunLength between Branches Average dynamic instruction mix from SPEC92: SPECint92 SPECfp92 ALU 39 13 FPU Add 20 FPU Mult 13 load 26 23 store 9 9 branch 16 8 other 10 12 SPECint92: compress, eqntott, espresso, gcc , li SPECfp92: doduc, ear, hydro2d, mdijdp2, su2cor What is the average run length between branches next lecture: Branch prediction Speculative excecution October 24, 2005 28 Thank you