In-Order Superscalar Architecture¶

This section is a brief overview of fascinating but complex topics. In my DDCA notes, I have some really good examples regarding the techniques introduced here. If you want to learn more details, I highly recommend to check out those notes.

Pipelining exploits the potential parallelism among instructions. This parallelism is called, naturally enough, instruction-level parallelism (ILP). There are two primary methods for increasing the potential amount of instruction-level parallelism:

1. Deep Pipelining
2. Multiple Issue: A scheme whereby multiple instructions are launched in one clock cycle.

There are also two main ways to implement a multiple-issue processor, with the major difference being the division of work between the compiler and the hardware:

1. Static Multiple Issue: An approach to implementing a multiple-issue processor where many decisions are made by the compiler before execution.
2. Dynamic Multiple Issue: An approach to implementing a multiple-issue processor where many decisions are made during execution by the processor.

Static multiple-issue processors all use the compiler to assist with packaging instructions and handling hazards. In a static issue processor, you can think of the set of instructions issued in a given clock cycle, which is called an issue packet, as one large instruction with multiple operations. This view is more than an analogy. Since a static multiple-issue processor usually restricts what mix of instructions can be initiated in a given clock cycle, it is useful to think of the issue packet as a single instruction allowing several operations in certain predefined fields. This view also led to the original name for this approach: Very Long Instruction Word (VLIW).

In Mach-V, instead of seeking help from the compiler, I am going to implement an in-order superscalar processor. There will be a hardware unit to handle the instruction packaging during the execution and some other units used for different purposes, so technically it is a dynamic multiple-issue processor. As this architecture is rather complex, I will explain it part-by-part.

Instruction Issue Unit¶

The overal architecture of the instruction issue unit (IIU) is shown below.

The IIU is implemented in the Decode stage. Whenever dependency arises between the instructions on the Instruction 1 wire and the Instruction 2 wire, the second instruction will be stored in a hold register and a "rollback" signal will be asserted to adjust next-PC value so that the next-PC coming into the IROM will be PC+4 instead of PC+8 (We will talk more about the Next-PC Logic in detail later). The held instruction will be issued in the next clock cycle.

The three multiplexers in the figure each has their own purpose:

• The Hold Mux (Top Left): Controls what gets saved into the Hold Register. Inputs are Instruction 1 (0), Instruction 2 (1), and NOP (2).
• The Pipe 1 Mux (Top Right): Controls what enters the first execution pipeline. Inputs are Instruction 1 (0) and held instruction (1).
• The Pipe 2 Mux (Bottom Right): Controls what enters the second execution pipeline. Inputs are Instruction 1 (0), Instruction 2 (1), and NOP (2).

And to understand its flow better, let's look at all the 4 possible cases:

The behavior of superscalar instruction fecthin mechanism is illustrated in the following table assuming only the first two instructions are having dependency.

From this table, we can see clearly that the instructions on the Instruction 1 and Instruction 2 wires in the IIU in the Decode Stage are actually the current instructions in the Fetch Stage in the previous cycle. In the Decode Stage, a rollback signal will be generated combinationally and this immediately affects the next instructions in the current clock cycle.

Instruction Dependency Detection¶

In the IIU, the following conventions are used for the instruction dependency detection:

• Two instructions are issued to the pipeline only if the second instruction does not have any dependency on the first instruction. In case of dependencies, only the first instruction is issued and the second instruction is held at the Decode stage.
• Two memory access instructions (load or store) or two branch instructions are never issued together.
• Load, branch, multiply/divide instructions are issued only through the first pipeline. In other words, only one load, branch, or multiply/divide instruction can be issued a time.

With these conventions, besides the change of the IROM we have mentioned at the beginning, the rest of changes for the Mach-V microarchitecture are:

1. Add two read ports and one write port to the register file.
2. Add one more ALUs so now we have two ALUs.

Next-PC Logic¶

As the normal Next-PC Logic, this part will cover several pipeline stages and which stages are covered really depends on the program execution flow. The architecture for the Next-PC logic is shown below.

The Next-PC Logic has two paths:

1. The upper path: This is to deal with the "rollback" signal from the IIU, which is already explained in the previous section.
2. The lower path: This is to deal with the superscalar branch prediction unit.

The structure of the BPU Issuing Unit is shown below.

The BPU Issuing Unit is implemented in the Fetch Stage for feeding the PC value to the BPU.

1. If the held instruction is a branch, the PC value from the hold register in the Decode Stage is fed to the BPU.
2. Otherwise, the PCF will be fed to the BPU as normal.

Coming back to the Next-PC Logic, we have three multiplexers:

1. The Rollback Mux (Top Left): Whenever the rollback signal is asserted in the Decode Stage, the next PC value will be PCF+4; otherwise, it will be PCF+8.
2. The Prediction Mux (Middle): In the Fetch Stage, if the BPU predicts the target PC value of an instruction (PrPCSrcF == 1'b1), the predicted PC from the BPU (PrBTAF) will be loaded. Otherwise, it passes the sequential PC through.
3. The Correction Mux (Right): If the prediction goes wrong (BranchMispredictM == 1'b1), then the correct PC value will be loaded from the Memory Stage (PC_ResolvedM). This overrides everything else fetched in the current cycle.

Forwarding Unit¶

The forwarding unit is implemented in the Decode, Execute and Memory stages to handle the data hazards arising from the dependencies between instructions.