Modern Processor Design Fundamentals of Superscalar Processors 130225191042 Phpapp02

7/18/2019 Modern Processor Design Fundamentals of Superscalar Processors 130225191042 Phpapp02

http://slidepdf.com/reader/full/modern-processor-design-fundamentals-of-superscalar-processors-130225191042 1/331

MODERN

PROCESSORDESIGN

Fundamentalsof SuperscalarProcessors

JOHN PAUL SHEN●

MIKKO H.LIPASTI

John Paul Shen

Jo hn P aul Shen is the Director of Intel's Microarchitecture

Research Lab (MRL), providing leadership to about two-

dozen highly skilled researchers located in Santa Clara, C A;

Hillsboro, OR; and Austin. TX. MRL is responsible for de-

veloping innovative microarchitecture techniques that can

I potentially be used in future microprocessor products from

Intel. MRL researchers collaborate closely with microarchi-

I tects from product teams in joint advanced-development

I efforts. MRL frequently hosts visiting faculty and Ph.D.

I interns and conducts joint research projects with academic

I research groups.

Prior to joining Intel in 2000, John was a professor in the

electrical and computer engineering department of Carnegie

Mellon University, where he headed up the CMU Microarchitecture Research Team

(CMuART). He has supervised a total of 16 Ph.D. students during his years at CMU.

Seven are currently with Intel, and five have faculty positions in academia. He won

multiple teaching awards at CMU. He was an NSF Presidential Young Investigator.

He is an IEEE Fellow and has served on the program committees of ISCA, MICRO,

HPCA, ASPLOS, PACT, ICCD, ITC, and FTCS.He has published over 100 research papers in diverse areas, including fault-

tolerant computing, built-in self-test, process defect and fault analysis, concurrent

error detection, application-specific processors, performance evaluation, compila-

tion for instruction-level parallelism, value locality and prediction, analytical mod-

eling of superscalar processors, systematic microarchitecture test generation, per-

formance simulator validation, precomputation-based prefetching, database workload

analysis, and user-level helper threads.

John received his M.S. and Ph.D. degrees from the University of Southern

California, and his B.S. degree from the University of Michigan, all in electrical

engineering. He attended Kimball High School in Royal Oak, Michigan. He is

happily married and has three daughters. His family enjoys camping, road trips, and

reading The Lord of the Rings.

(continued on back inside cover)

Modern Processor

Fundamentals of Sup

John Paul Shen Intel Corporation

Mikko H. LipastiUniversity of Wisconsin

Tata McGraw-Hill PublishinN E W D E L H I

McGraw-Hill Offices

New Delhi New York St Lou is Sa n Fra

Kua la Lumpur L isbon Lond on Madr id

S an Juan San t iago S in gapor e Sy dn e

Our parents:

Paul and Sue Shen

Tarja and Simo Lipasti

Our spouses:

Amy C. Shen

Erica Ann Lipasti

Our children:Priscilla S. Shen, Rachael S. Shen, and Valentia C. Shen

Emma Kristiina Lipasti and Elias Joel Lipasti

Tata McGraw-Hiil

MODERN PROCESSOR DESIGN: FUNDAMENTALS OF

SUPERSCALAR PROCESSORS

in any form or by any means, or stored in a data base or retrieval system, without theprior written consent of The McGraw-Hill Companies, Inc., including, but notlimited to, in any network or other electronic storage or transmission, or broadcast

for distance learning

Some ancillaries, including electronic and print components, may not be available

to customers outside the United States

Tata McGraw-Hill Edition

RZXQCRBIRQQDD

Reprinted in India by arrangement with The McGraw-Hill Companies, Inc.,

New York

Sales territories: India, Pakistan, Nepal, Bangladesh, Sri Lanka and Bhutan

ISBN 0-07-059033-8

Published by Tata McGraw-Hill Publishing Company Limited,

7 West Patel Nagar, New Delhi 110 008, and printed at

Shivam Printers, Delhi 110 032

The McGraw-Hill Companies

Table of Content

Table of Contents

Additional Resources

Preface

1 Processor Design

1.1 The Evolut ion of Mic

1.2 Instructio n Set Proces

1.2.1 Digital System

1.2.2 Architecture, I

Realization

1.2.3 Instruction Set

1.2.4 Dynamic-Stati

1.3 Principle s of Processo

1.3.1 Processor Perf

1.3.2 Processor Perf

1.3.3 Performance E

1.4 Instructio n-Level Par

1.4.1 From Scalar to

1.4.2 Limits of Instr1.4.3 Machines for I

1.5 Summary

2 Pipelined Processors

2.1 Pipelining Fundamen

2.1.1 Pipelined Desi

2.1.2 Arithmetic Pip

2.1.3 Pipelining Ide

2.1.4 Instruction Pip

2.2 Pipelined Processor D

2.2.1 Balancing Pip

2.2.2 Unifying Instr

2.2.3 Minimizing Pi

2.2.4 Commercial P

2.3 Deeply Pipelined Pro

2.4 Summary

3 Memory and I/O Systems

3.1 Introduction

3.2 Comp uter System Ov

3.3 Key Concept s: Latenc

2 M O D E R N P R O C E S S O R D E S I G N

Instruction

Old Mix,

New Mix,

Latency Old CPI Cycles % fait ructions Cycles NewCPf

_oa d 25.0 2 0.50 50 0

Store I S O 1 " T " ' 0 1 5 150

Ar ithme ti c 30.0 T 0.30 30 0

Logical 10.0 1 0.10 100

Branch-T 8.0 3 0.24 240

Branch-NT 6.0 2 0.12 120

Jump 5.0 2 0.10 100

Jump register 1.0 3 6.03 30

Total 160.6"" 154" " ' 1540

P2.16 In a TYP-based pipeline design with a data cache, load instructions

check the tag array for a cache hit in parallel with accessing the data

array to read the corresponding memory location. Pipelining stores

to such a cache is more difficult, since the processor must check

the tag first, before it overwrites the data array. Otherwi se, in the

case of a cache miss, the wrong memory location may be overwrittenby the store. Design a solution to this problem that does not require

sending the store down the pipe twice, or stalling the pipe for every

store instruction, or dual-porting the data cache. Referring to

Figure 2.15, are there any new RAW, WAR, and/or WAW memory

hazards?

P2.17 The MIPS pipeline shown in Table 2.7 employs a two-phase clocking

scheme that makes efficient use of a shared TLB, since instruction fetch

accesses the TLB in phase one and data fetch accesses in phase two.

However, when resolving a conditional branch, both the branch target

address and the branch fall-through address need to be translated during

phase one—in parallel with the branch condition check in phase one of

the ALU stage—to enable instruction fetch from either the target or the

fall-through during phase two. This seems to imply a dual-ported TLB.

Suggest an architected solution to this problem that avoids dual-portingthe TLB.

Problems 18 through 24: InstructionPipeline DesignThis problem explores pipeline design. As discussed eadier, pipelining involves

balancing the pipe stages. Good pipeline implementations minimize both internal

and external fragmentation to create sim

lined implementation of a simple micro

tions, with no data hazards:

1Instr

18 2 MODERN PROCESSOR DESIGN

(a) No parallelism (b) Temporal parallelism

(c) Spatial parallelism

(d) Parallel pipeline

Figure 4.2Machine Parallelism: (a) No Parallelism (Nonpipelined); (b) Temporal

Parallelism (Pipelined); (c) Spatial Parallelism (Multiple Units);

(d) Combi ne d Tempor al and Spatial Parallelism

Intel [Intel Corp., 1993]. The Pentium microprocessor is a superscalar machine

implementing a parallel pipeline of width s = 2. It essentially implements two i486:

pipelines; see Figure 4.4. Multiple instructions can be fetched and decoded by the

first two stages of the parallel pipeline in every machine cycle. In each cycle,

potentially two instructions can be issued into the t w o execution pipelines, i.e., the

U pipe and the V pipe. The goal is to maximize the number of dual-issue cycles.

The superscalar Pentium micropro cessor can achieve a peak execution rate of two

instructions per machine cycle.

As compared to the scalar pipeline of i486, the Pentium parallel pipeline

requires significant additional hardware resources. First, the five pipeline stages

i — *

TFigure 4.3

A Parallel Pipe line of Wi

Figure 4.4

(a) The Five- Stage i48 6 Scalar P

(b) The Five-Stage Pentium Par

o f W i d t h i = 2 .

21 4 MODERN PROCESSOR DESIGN

decoder simply inserts the add/multiply instruction pair into the execution

window. Identify and discuss at least two reasons why this approach

could reduce performance as measured in instructions per cycle.

P4.1 8 Does the ALU mix for the Motorola 88110 processor shown in Figure 4.7

agree with the IBM instruction mix provided in Section 2.2.4.3? If not,

how would you change the ALU mix?

Terms and BuzzwordsThese problems are similar to the "Jeopardy Game" on TV. The "answers" are

given and you are to provide the best correct "questions." For each "answer" there

may be more than one appropriate "question"; you need to provide the best one.

P4.19 A: A m echanism that tracks out-of-order execution and maintains specu-

lative machine state.

Q: What is.

P4.20 A: It will significantly reduce the machine cycle time, but can increase

the branch penalty.

Q: What is.

P4.21 A: Additional I-cache bits generated at cache refill time to ease the

decoding/dispatching task.

Q: What are ?

P4.22 A: A program attribute that causes inefficiencies in a superscalar fetch

Q: What is ?

P4.23 A: The internal RISC-like instruction executed by the Pentium Pro (P6)

microarchitecture.

Q: What is ?

P4.24 A: The logical pipeline stage that assigns an instruction to the appropri-

ate execution unit.

Q: What is ?

P4.25 A: An early processor design that incorporated 10 diverse functional

units.Q: Wha t is ?

P4.26 A: A new instruction that allows a scalar pipeline to achieve more than

one floating-point operation per cycle.

Q: What is ?

P4.27 A: An effective technique for all

to be performed per cycle.

Q: Wha t is

P4.28 A: A useful architectura l prope

low-level operating system code

Q: What is

P4.29 A: The first research paper to

of one instruction set to another

Q: What was

P4.30 A: The first real processor to im

of one instruction set to another

Q: What was

P4.31 A: This attribute of most RISC

the task of decoding multiple in

Q: What was

Superscalar Techniq

CHAPTER OUTLINE

Instruction Flow Techniques

Register Data Flow Techniques

Memory Data Flow Techniques

Summary

References

Hornework Problems*

In Chapter 4 we focused on the structural, or

scalar pipeline and dealt with issues that were so

types of instructions being processed. In thi

dynamic behavior of a superscalar processor

with specific types of instructions. The ultimat

pipeline is to achieve max imum throughput of in

to view instruction processing as involving thr

and/or data, namely, instruction flow, regist

This partitioning into three flow paths is simi

1991 textbook entitled Superscalar Microproce

overall performance objective is to maximize th

paths. Of course, what makes this task interest

not independent and their interactions are quiand presents superscalar microarchitecture tec

with the three flow paths.

The three flow paths correspond roughly t

types of instructions, namely, branch, ALU,

quently, maximizing the throughput of the th

minimizing of the branch, ALU, and load pena

220 M O D E R N P R O C E S S O R D E S I G N

-- Branch

Execute

Finish

Decode

IDispatch

iComplete

Decode buffer

Dispatch buffer

i L X D L X a L X D C x a| Reservationstations

J Completion buffer

Store buffer

Retire

Figure 5.2

Disruption of Sequential Control Flow by Branch Instructions.

slots as in the scalar pipeline, but the number of empty instruction slots must be

multiplied by the width of the machine. For example, for a four-wide machine thetotal penalty is 12 instruction "bubbles" in the superscalar pipeline. Also recall

from Chapter 1, tha t such pipeline stall cycles effectively corr espond to the

sequential bottleneck of Amdahl's law and rapidly and significantly reduce the

actual performance from the potential peak performance.

For conditional branches, the actual number of stalled or penalty cycles can be

dictated by either target address generation or condition resolution. Figure 5.3 illus-

trates the potential cycles that can be incurred by target address generation. The

Register

indirectwith

offset

Register

Indirect

relative

Branch

Execute

Finish

Figure 5.3

Branch Target Address Generation Penalties.

actual number of penalty cycles is deter

branch instructions. For the PC-relative ad

can be generated during the fetch stage, r

register indirect addressing mode is used,

decode stage to access the register. In this

register indirect with an offset addressing m

access and a total three-cycle penalty can

the penalty due to target address generatio

branch condition resolution latency must a

286 MODERN PROCESSOR DESIGN

P5.14 The figure shows the control flow graph of a simple program. The

CFG is annotated with three different execution trace paths. For each

execution trace, circle which branch predictor (bimodal, local, or gse-

lect) will best predict the branching behavior of the given trace. More

C o ? ) Circle one:

p ^ Bimodal

Gselect

than one predictor may perform

However, you are to use each

choosing the best predictors fo

for each of the three traces and

many times and every node in

branch history register for the l

limited to 4 bits.)

Problems 5.15 and 5.16: CombinGiven a combining branch predictor with

branch direction predictor, a gshare predic

entries, and a two-entry selector table, simu

branches as shown in the rows.of the table

made by the predictor before the branch is

predictor entries after the branch resolves.

Branch address

Bimodal

Selector

Use the following assumptions:

• Instructions are a fixed 4 bytes long

branch address should be shifted out

the next lowest-order bit to index int

• Each predictor and selector entry

counter with the initial states shown.

• A taken branch (T) incremen ts the p

decrements the predictor entry.

• A predictor entry less than 2 (0 or 1)

• A predictor entry greater than or equ

prediction.

S92 MODERN PROCESSOR DESIGN

Load/store unit

Example:

Dispatch cycle Instr uction

1 Load A

1 LoadB

1 Store C

Retire cycle

LoadRS

Store RS

t bAddress 1

unit I

Store buffer

Cache address Cache write data

Load Store

Reservation Reservation

Cycle Station Station

1 Ld A l_d B S t C

2 Ld B

Cache WriteStore Buffer Address Data

St C Ld A

S t C Ld B

St C data

Load/store unit

Dispatch cycle

Instruction

Store A

Load A

Store D

Load E

Load A

Retire cycle

LoadRS

Addressunit

Store RS

Addressunit

TStore buffer

3 — R: FCache address Cache write data

Load Store

Reservation Reser

Cycle Station Statio

P5.24 In one or two sentences comp

load bypassing.

P5.2S Woul d load forwarding improve

from Problem 5.23? Why or wh

Problems 5.26 through 5.28The goal of lockup-free cache designs is to

parallelism in the processor by overlapping

cessing or other cache misses. For this p

workload running on a processor with a p

16-byte memory bus. Assume that f miss

block is 1 cycle. Hence, for a simple blockin

'transfer = 5 + 4 x 1 = 9 cycles. Here is the wo

f o r ( i = l ; i < 1 0 0 0 0 ; + + i )

a += A [ i ] + B [ i ] ;

In RISC assembly language, assuming r3 po

l i r 2 , 9 9 9 9 # l o a d

l o o p : l f d u r 5 , 8 ( r 3 ) # l o a d

l f d u r 6 , 8 ( r 4 ) # l o a d

294 MODERN PROCESSOR DESIGN

a d d r 7 , r 7 , r 5 # a d d A [ i ] t o a

a d d r 7 , r 7 , r 6 # a d d B [ i ] t o a

b d n z r 2 , l o o p # d e c r e m e n t r 2 , b r a n c h i f n o t z e r o

Here is a timing diagram for the loop body assuming neither array hits in the cache

and the cache is a simple blocking design (m = miss latency, t = transfer latency,

A = load from A, B = load from B, a = add, and b = branch):

Cycl«

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 8 7 8 9 0 1 2 3 4 S 6 7 8 9

Array A m m m m m t t t t

ArrayB m m m m m t t t t

A B a a b A B a a b A ~ T ~

For Problems 5.26 through 5.28 assume that each instruction takes a single

cycle to execute and that all subsequent instructions are stalled on a cache miss

until the requested data are returned by the cache (as shown in the timing dia-

gram). Furthermore, assume that no portion of either array (A or B) is in the cache

initially, but must be fetched on demand misses. Also, assume there are enough

loop iterations to fill all the table entries provided.

P5.26 Assume a blocking cache design with critical word forwarding (i.e., the

requested word is forwarded as soon as it has been transferred), but

support for only a single outstanding miss. Fill in the timing diagram

and explain your work.

0 1 2 3 4 5 6 7 8 9 0 1 2

Array A m m m m m

ArrayB

Execution A

3 4 5 6 7 6 9 0 1 2 3 4 5 6 7 « #

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 S 9 0 1 2 3 4 5 6 7 8 9

Array A

Array B

Execution

P5.27 Now fill in the timing diagram

multiple outstanding misses, an

0 1 2 3 4 5 0 7 8 9

Array A m m m m m

ArrayB

Execution A

- ! " S ~ ^ * 0 1 2 3 4 5 6 > 8

Array A

Array B

Execution

P5.28 Instead of a 64-byte cache line

and fill in the timing diagram a

0 1 2 3 4 5 6 7 8 9

Array A m m m m m

Array B

Execution A

0 1 2 3 4 5 6 7 S 9

Array A

ArrayB

Execution

Problems 5.29 through 5.30The goal of prefetching and nonblocking

of concurrency or parallelism in the proc

with other processing or other cache mi

simple workload from Problem 5.26 run

with 16-byte lines and an 4-byte memor

Trung A. Diep

The PowerPC 620

CHAPTER OUTLINE <p

6.1 INTRCKJUCTIORI

6.2 Experimental Framework

63 Instruction Fetching

6.4 Instruction Dispatching

65 Instruction Execution

6.6 Instruction Completion

6.7 Conclusions and Observations

6. 8 Bricigingtothel^

6.9 Summary •

References

Homework Problems

The Po werPC family of microprocessors inc

processor. The 620 was the first 64-bit super

order execution, aggressive branch predicti

stations, dynamic renaming for all register fi

completion buffer to ensure precise exceptio

previously implemented in a single-chip mic

is of great interest to both academic researchchapter presents an instruction-level, or mac

tion of the 620 microarchitecture using a V

of the 6 20 (VMW is the Visualization-base

Carnegie Mellon University) [Levitan et al., 1

We also describe the IBM POWER3 an

how they differ from the predecessor Power

Robert P.

Dave B. Papworth

Glenn J. Hinton

Mike A. Fetterman

An dy F. Glew

Intel's P6 Microarc

CHAPTER OUTLINE

7.1 Introduction

7.2 Pipelining

7.3 The In-Order Front End

7.4 The Out-of-Order Core

7.5 Retirement

7.6 Memory Subsystem

7.7 Summary

7.8 Acknowledgments

References

Homework Problems

In 1990, Intel began development of a n

microarchitecture core known as the P6. Intr

and Steck, 1995], it was named the Pentium

lar in workstation and server systems. A des

Pentium II processor, was launched in May

tions to the basic P6 engine. Th e P6-based Pe

which included MMX and SSE instructions.

P6 and to the products by their respective pr

The P6 microarchitecture is a 32-bit

performance, superpipelined dynamic executi

uses out-of-order and speculative execution

execution core. P6 includes nonblocking cac

bus. This chapter describes the various com

combine to deliver exbaordinary performance

Mark Smotherman

Survey of Supersca

CHAPTER OUTLINE

ftl * Development of Superscalar Proc

82 A Classification of Recent Design

8.3 PrccessctfCJescrjptiortt

8.4 Verification of Superscalar Proces

8.5 Ad(novv4edgrnents

References

Homework Problems

The 1990s was the decade in which sup

However, the idea of decoding and issuing

single instruction stream dates back 25 years

the history of superscalar design and examin

8.1 Development of SuperscalarThis section reviews the history of supersc

Stretch and its direct superscalar descend

(ACS), and follows developments up throug

8.1.1 Early Advances in Uniprocessor P

The first efforts at what we now call super

IBM machine directly descended from the I

gressive implementation techniques (such as

speculative execution, branch misprediction

because it was a precursor to the IBM ACS in

Gabriel H. Loh

Advanced InstructFlow Techniques

CHAPTER OUTLINE

9.1 Introduction

9.2 Static Branch Prediction Techn

9.3 Dynamic Branch Prediction Te

9.4 Hybrid Branch Predictors

9.5 Other Instruction Flow Issues a

9.6 Summary

References

Homework Problems

9.1 IntroductionIn Chapter 5, it was stated that the instruc

provides an upper bound on the throughpu

conditional branches in programs are a se

instruction flow and, hence, the perform

tional branch is resolved in a pipelined pro

should follow the branch. To increase theparallel, modern processors make a bran

the instructions in the predicted path of pr

covered later on to have been mispredicte

of the processor to the point before the

resumed along the correct path.

The penalty associated with mispredi

cessors has a great impact on performanc

Modern Processor Design Fundamentals of Superscalar Processors 130225191042 Phpapp02

Documents

The Microarchitecture of Superscalar Processors ...ece447/s15/lib/exe/fetch.php?media=00476078.pdfThe Microarchitecture of Superscalar Processors JAMES E. SMITH, MEMBER, IEEE, AND

CAPS team Compiler and Architecture for superscalar and embedded processors

ECE/CS 552: Introduction to Superscalar Processors

Superscalar Processors by Sherri Sparks. Overview 1.What are superscalar processors? 2.Program Representation, Dependencies, & Parallel Execution 3.Micro

Multiple Issue Processors: Superscalar and VLIW. Multiple-Issue Processor Styles Dynamic multiple-issue processors (superscalar) l Decisions on which

An Approach for Implementing Efficient Superscalar CISC Processors

Chapter 13 Instruction-Level Parallelism and Superscalar Processors

Complexity-Effective Superscalar Processorsftp.cs.wisc.edu/pub/sohi/theses/subbarao.pdf · Complexity-Effective Superscalar Processors by ... answering my questions during ... 3.2.5

A Study of Control Independence in Superscalar Processors · A Study of Control Independence in Superscalar Processors December 18, 1998 4 1.1 Prior work Lam and Wilson’s limit

COMP381 by M. Hamdi 1 Commercial Superscalar and VLIW Processors

Fundamentals of Superscalar Processors

The Microarchitecture of Superscalar Processors · 2013-05-17 · The Microarchitecture of Superscalar Processors James E. Smith Department of Electrical and Computer Engineering

Computer Organization and Architecture Instruction-Level Parallelism and Superscalar Processors

The Microarchitecture of Superscalar Processors - Computing

CSL718 : Superscalar Processors

Task Superscalar: Using Processors as Functional … · Task Superscalar: Using Processors as Functional Units ... (Latency related) ... • Building a large pipeline • Multiplex

Pipeline Behavior Prediction for Superscalar Processors by

Chapter 14 Superscalar Processors. What is Superscalar? “Common” instructions (arithmetic, load/store, conditional branch) can be executed independently

Modern Processor Design: Fundamentals of Superscalar Processors

Superscalar Processors Superscalar Processors: Branch ...bhagiweb/cs211/lectures/superscalar.pdfSuperscalar Processors: Branch Prediction Dynamic Scheduling Superscalar Processors