Basic Instruction Timings - Virginia Techcourses.cs.vt.edu › ~cs2504 › fall2008 › Notes › PDF › L23... · Computer Science Dept Va Tech January 2006 ©2006 McQuain & Ribbens

Pipelining

1Basic Instruction Timings

Mak

ing som

e as

sum

ptions re

gar

din

g the

oper

atio

n tim

es for so

me

of th

e bas

ic h

ardw

are

units in

our dat

apat

h, w

e hav

e th

e fo

llow

ing tim

ings:

Instruction

class

Instruction

fetch

Register

read

ALU

operation

Data

access

Register

write

Total time

lw200 ps

100 ps

200 ps

200 ps

100 ps

800 ps

sw

200 ps

100 ps

200 ps

200 ps

700 ps

R-form

at

200 ps

100 ps

200 ps

100 ps

600 ps

beq

200 ps

100 ps

200 ps

500 ps

Intro Computer Organization

Computer Science Dept Va Tech January 2006

©2006 McQuain & Ribbens

How

long w

ould

it ta

ke

to e

xec

ute

the

follow

ing seq

uen

ce o

f in

stru

ctio

ns?

lw

$1, 100($0)

lw

$2, 200($0)

lw

$3, 300($0)

But, m

aybe

ther

e’s a

way

we

can c

hea

t an

d c

om

ple

te the

sequen

ce fas

ter.

Pipelining

2Basic Idea

What

if w

e th

ink o

f th

e dat

apat

h a

s a

linea

r se

quen

ce o

f st

ages

?

Note: single-cycle

datapath

We have 5 stages,

which will m

ean

that on any given

cycle up to 5

different




Can

we

oper

ate

the

stag

es indep

enden

tly,

usi

ng a

n e

arlier

one

to b

egin

the

nex

t in

stru

ctio

n

bef

ore

the

pre

vio

us one

has

com

ple

ted?

instructions will be

in various points of

execution.

Pipelining

3Pipelining

We’

ve

only

consi

der

ed u

nim

agin

ativ

e ex

ecution; co

nsi

der

our lo

nges

t in

stru

ctio

n:

Impro

ve

per

form

ance

by incr

easi

ng inst

ruct

ion throughput:




Idea

l speedup

is n

um

ber

of st

ages

in the

pip

elin

e. D

o w

e ac

hie

ve

this

?

Impro

ve

per

form

ance

by incr

easi

ng inst

ruct

ion throughput:

Pipelining

4Pipelining

Det

ails

:




The average time

between initiating

instructions has

dropped to 200 ps.

Why do we have

idle “gaps”?

Assume:

-register file write occurs in

first half of a cycle

-register file read occurs in

second half of a cycle

Total time here is 1400 ps versus 2400 ps for the original version…

…but consider how this would look if we were had 1,000,000 more lw

instructions in our sequence…

Pipelining

5MIPS addPipeline

Her

e's how

the

pip

elin

e st

ages

would

look fro

m the

per

spec

tive

of an

add

inst

ruct

ion:

Shading indicates when

the instruction is using a

Note that the computed

result isn't written into

the register file until the

5thstage.




the instruction is using a

particular hardware

resource.

What

if th

e nex

t in

stru

ctio

n n

eeds th

e re

sult fro

m the add

inst

ruct

ion?

Dep

endin

g o

n w

hen

the

resu

lt is nee

ded

, w

e m

ay h

ave

to stall

the

pip

elin

e until th

e re

sult

bec

om

es a

vai

lable

.

Pipelining

6Pipelining

What

mak

es it ea

sy

-al

l in

stru

ctio

ns ar

e th

e sa

me

length

-ju

st a

few

inst

ruct

ion form

ats

-m

emory

oper

ands ap

pea

r only

in load

s an

d sto

res

What

mak

es it har

d?

-st

ruct

ura

l haz

ards:

suppose

we

had

only

one

mem

ory

-co

ntrol haz

ards:

nee

d to w

orry a

bout bra

nch

inst

ruct

ions

-dat

a haz

ards:

an

inst

ruct

ion d

epen

ds on a

pre

vio

us in

stru

ctio

n




We’

ll b

uild a

sim

ple

pip

elin

e an

d look a

t th

ese

issu

es

We’

ll tal

k a

bout m

oder

n p

roce

ssors

and w

hat

rea

lly m

akes

it har

d:

-ex

ception h

andling

-tryin

g to im

pro

ve

per

form

ance

with o

ut-of-

ord

er e

xec

ution, et

c.

Pipelining

7Pipeline Hazards

In som

e ca

ses, the

nex

t in

stru

ctio

n c

annot ex

ecute

in the

follow

ing c

lock

cycl

e. W

e w

ill

introduce

som

e of th

e pote

ntial

iss

ues

in the

nex

t fe

w slides

.

structural hazard

-har

dw

are

cannot su

pport the

nec

essa

ry c

om

bin

atio

n o

f oper

atio

ns at

once

-re

consi

der

the

earlie

r ex

ample

with a

sin

gle

mem

ory

unit a

nd a

fourth lw

inst

ruct

ion




data hazard

-dat

a th

at is nec

essa

ry to e

xec

ute

the

inst

ruct

ion is not yet

avai

lable

-co

nsi

der

:

add

$s0, $t0, $t1

sub

$t2, $s0, $s3

-load-use

haz

ard o

ccurs

when

dat

a im

ported

by a

load

inst

ruct

ion is not av

aila

ble

w

hen

it is

req

ues

ted

Pipelining

8

Her

e, the

seco

nd inst

ruct

ion n

eeds th

e final

res

ult fro

m the

firs

t in

stru

ctio

n d

uring the

regis

ter fe

tch p

ortio

n o

f th

e in

stru

ctio

n d

ecode

phas

e:

Data Hazard Example: Forwarding

Obvio

usl

y the

val

ue

will not be

avai

lable

in reg

iste

r $s0

until th

e firs

t in

stru

ctio

n h

as




Obvio

usl

y the

val

ue

will not be

avai

lable

in reg

iste

r $s0

until th

e firs

t in

stru

ctio

n h

as

com

ple

ted.

How

ever

, th

e co

mpute

d v

alue

IS a

ctual

ly a

vai

lable

after

the

firs

t in

stru

ctio

n fin

ishes

its

third sta

ge,

just

in tim

e to

sat

isfy

the

nee

d o

f th

e A

LU

when

the

seco

nd inst

ruct

ion rea

ches

its th

ird sta

ge.

This

is in

dic

ated

above

by a

forwarding

link.

In p

rinci

ple

, th

e haz

ard h

ere

could

be

det

ecte

d a

nd h

andle

d.

But, w

hat

if th

e "f

orw

ardin

g" link a

ctual

ly w

ent bac

kw

ards?

Pipelining

9Data Hazard Example: Stalling

Her

e th

e firs

t in

stru

ctio

n is a

load

, an

d its

res

ult sim

ply

won't b

e av

aila

ble

in tim

e:




As in

dic

ated

, th

is c

an b

e re

solv

ed b

y stalling

the

pip

elin

e, d

elay

ing the

initia

tion o

f th

e

seco

nd inst

ruct

ion for 1 c

ycl

e.

Agai

n, if w

e ca

n d

etec

t th

is situat

ion, w

e ca

n in p

rinci

ple

im

pose

the

solu

tion show

n

above.

A p

ipel

ine

stal

l is

often

ref

erre

d to a

s a bubble

.

Pipelining

10

Control Hazards: Stall on Branch

One

appro

ach w

ould

be

to sta

ll w

hen

a b

ranch

inst

ruct

ion is dis

cover

ed, until th

e

nec

essa

ry c

om

puta

tions ar

e co

mple

ted a

nd then

fet

ch the

corr

ect in

stru

ctio

n n

ext.

A control hazard

occ

urs

when

the

inst

ruct

ion that

was

fet

ched

is not th

e one

that

is

nee

ded

.

Note

that

our pip

elin

e dis

cuss

ion so far

ass

um

es seq

uen

tial

exec

ution.

When

the

curr

ent in

stru

ctio

n is a

conditio

nal

bra

nch

, th

is m

ay b

e in

corr

ect.




nec

essa

ry c

om

puta

tions ar

e co

mple

ted a

nd then

fet

ch the

corr

ect in

stru

ctio

n n

ext.

Pipelining

11

Control Hazards: Branch Prediction

A sec

ond a

ppro

ach is to

predictw

het

her

the

bra

nch

will be

taken

and load

the

corr

espondin

g inst

ruct

ion into

the

pip

elin

e.

If w

e gues

s th

at the

bra

nch

will N

OT b

e ta

ken

, w

e ju

st incr

emen

t th

e PC

, fe

tch a

nd

pro

ceed

:




This

work

ed o

ut per

fect

ly. T

her

e w

as n

o d

elay

… h

ow

ever

, w

hat

if th

e bra

nch

HA

D b

een

taken

?

More

sophis

tica

ted v

aria

nts

act

ual

ly ret

ain info

rmat

ion (his

tory

) ab

out in

div

idual

bra

nch

inst

ruct

ions an

d u

se that

his

tory

to p

redic

t fu

ture

beh

avio

r.

Pipelining

12

Control Hazards: Delayed Branch

A third a

ppro

ach is to

delay

the

tim

e at

whic

h the

bra

nch

tak

es e

ffec

t by a

lway

s ex

ecuting

the

nex

t se

quen

tial

inst

ruct

ion follow

ing a

bra

nch

inst

ruct

ion, an

d then

mak

ing the

bra

nch

(if nec

essa

ry) im

med

iate

ly a

fter

than

one-

inst

ruct

ion d

elay

.

To d

o this

, th

e as

sem

ble

r w

ill au

tom

atic

ally

acc

om

plish

this

by p

laci

ng a

n inst

ruct

ion

imm

edia

tely

after

the

bra

nch

inst

ruct

ion that

is not af

fect

ed b

y the

bra

nch

.

This

appro

ach is use

d in the

MIP

S a

rchitec

ture

.




;; programmer writes:

add

$4, $5, $6

beq

$1, $2, 40

or

$7, $8, $9

;; assembler writes:

beq

$1, $2, 40

add

$4, $5, $6

or

$7, $8, $9

Of co

urs

e, it’s not al

way

s th

at sim

ple

. W

hat

would

we

do if th

e add

inst

ruct

ion h

ad

store

d its

res

ult in o

ne

of th

e re

gis

ters

use

d b

y the beq

inst

ruct

ion?

Pipelining

13

Hav

e as

sem

ble

r guar

ante

e no h

azar

ds. O

ne

appro

ach w

ould

be

to rea

rran

ge

stat

emen

ts;

anoth

er w

ould

be

to inse

rt n

o-o

p (no o

per

atio

n) st

atem

ents

, to

induce

the

nec

essa

ry sta

lls.

Wher

e do w

e in

sert the

“no-o

ps”

?

sub

$2, $1, $3

and

$12, $2, $5

or

$13, $6, $2

add

$14, $2, $2

sw

$15, 100($2)

Software Solution




sw

$15, 100($2)

Pro

ble

m: this

rea

lly slo

ws us dow

n!

Pipelining

14

Check Yourself

What

difficu

ltie

s ca

n y

ou iden

tify

in the

follow

ing c

ode

sequen

ces?

;; 1

lw

$t0, 0($t0)

add

$t1, $t0, $t0

;; 2

add

$t1, $t0, $t0

addi

$t2, $t0, 5

The result of the lwis needed by the add

during its second cycle; a stall is needed.

The result of the addis needed by second

addiduring its second cycle, but isn’t written

to the register file until the next cycle;




addi

$t2, $t0, 5

addi

$t4, $t1, 5

;; 3

addi

$t1, $t0, 1

addi

$t2, $t0, 2

addi

$t3, $t0, 3

addi

$t4, $t0, 4

addi

$t5, $t0, 5

to the register file until the next cycle;

however, we can forward the value since it’s

been computed two cycles before it’s written.

No problems here.

Pipelining

15

Basic Idea Redux

What

do w

e nee

d to a

dd/m

odify to a

ctual

ly split th

e dat

apat

h into

sta

ges

?

Inst

ruct

ions an

d d

ata

gen

eral

ly

move

from

lef

t to

rig

ht.

Tw

o e

xce

ptions:

-w

rite

-bac

k o

f dat

a to

regis

ter file




regis

ter file

-se

lect

ing the

nex

t val

ue

for

the

PC

(in

crem

ente

d P

C

ver

sus bra

nch

addre

ss)

The

case

s w

her

e dat

a flow

s right to

lef

t do n

ot af

fect

the

curr

ent in

stru

ctio

n, but ra

ther

they

affec

t la

terin

stru

ctio

ns.

The

firs

t ca

se c

an lea

d to a

dat

a haz

ard; th

e se

cond c

an lea

d to a

control haz

ard.

Pipelining

16

Analysis

Consi

der

a tim

e line

show

ing

over

lappin

g p

ipel

ine

logic

for a

set of in

stru

ctio

ns:

Pro

ble

m: the

origin

al c

onte

nts

of

the

IR w

ill be

lost

when

the

nex

t

inst

ruct

ion is fe

tched

, but th

ose

origin

al c

onte

nts

are

nee

ded

at a

late

r cy

cle

as w

ell. (W

hy?)




So, how

do w

e fix this

?

Bas

ical

ly, w

e nee

d the

ability to p

rese

rve

resu

lts gen

erat

ed in e

ach sta

ge

until th

ey a

re

actu

ally

nee

ded

.

So, w

e ca

n a

dd a

ban

k o

f st

ora

ge

loca

tions bet

wee

n e

ach p

air of stag

es.

Pipelining

17

Datapath with Pipeline Registers

Her

e's a

firs

t at

tem

pt; w

e ju

st a

dd a

n u

nsp

ecifie

d a

mount of stora

ge

bet

wee

n d

atap

ath

stag

es:

Original IR contents are

passed forward… for later use

Incremented PC value

is passed forward




IR is embedded here

How large must the IF/ID register storage be?

The

nex

t ord

er o

f busi

nes

s is

to e

xam

ine

the

oth

er inte

r-st

age

registe

rs.

Pipelining

18

Boundary Analysis

Let

's c

onsi

der

the

"boundar

ies"

:




No pipeline register is

needed after the W

B stage.

Why?

The

nex

t ord

er o

f busi

nes

s is

to e

xam

ine

the

oth

er inte

r-st

age

registe

rs.

What about the PC?

In effect it IS a pipeline register,

feeding the IF stage.

Pipelining

19

Load Instruction Analysis: IF

PC is incremented by 4.

Result is written back into

PC, but also into IF/ID

pipeline register in case it is

needed later…

… don't know what the

instruction actually is yet.

Instruction is fetched into the




Reg

iste

r sh

adin

g indic

ates

whet

her

a w

rite

(le

ft h

alf)

or a

read

(right hal

f) is occ

urr

ing.

Instruction is fetched into the

pipeline register.

Pipelining

20

Load Instruction Analysis: ID

Values read from register

file, and extended

immediate field are stored in

the next pipeline register.

Incremented PC value is

also passed forward to next-

state pipeline register.




Register read numbers are

supplied to register file.

16-bit immediate field is

supplied to the sign-extender.

Pipelining

21

Load Instruction Analysis: EX

Incremented PC value is NOT

carried forward… why?




Contents of first read register and

sign-extended im

mediate are sent to

the ALU from the pipeline register.

Resulting sum is then

placed into next-stage

pipeline register.

Pipelining

22

Load Instruction Analysis: MEM




Computed address is passed from

pipeline register to memory unit.

Retrieved data is written into

next-stage pipeline register.

Pipelining

23

Load Instruction Analysis: WB

Data is retrieved from the pipeline

register and written into the register

file.




Pipelining

24

Load Instruction Analysis

So, w

hat

hav

e w

e le

arned

?

One

key

poin

t: a

logic

al c

om

ponen

t of th

e dat

apat

h, like

the A

LU

, m

ust

be

use

d o

nly

in a

single

pip

elin

e st

age…

oth

erw

ise,

we

hav

e a

stru

ctura

l haz

ard.

If y

ou w

ere

pay

ing v

ery c

lose

atten

tion, w

e've

unco

ver

ed a

bug in the

pro

pose

d h

andling

of a

load

inst

ruct

ion.

Tak

e an

oth

er look a

t w

hat

hap

pen

s in

the

final

sta

ge…

wher

e does

the

num

ber

of th

e




Tak

e an

oth

er look a

t w

hat

hap

pen

s in

the

final

sta

ge…

wher

e does

the

num

ber

of th

e

write

reg

iste

r co

me

from

?

Ala

s, w

e w

ill no longer

hav

e th

e origin

al inst

ruct

ion in the

IF/ID

pip

elin

e re

gis

ter, a

nd so

we

won't h

ave

the

info

rmat

ion w

e nee

d.

Solu

tion: p

ass th

e w

rite

reg

iste

r num

ber

forw

ard to the

MEM

/WB p

ipel

ine

regis

ter, so it

is still a

vai

lable

during the

final

sta

ge.

Pipelining

25

Summary

Further

, si

milar

anal

ysi

s of oth

er inst

ruct

ions le

ads to

a c

orr

ecte

d, but in

com

ple

te, pip

elin

e

des

ign:




An im

portan

t ques

tion is ju

st h

ow

much

sto

rage

must

eac

h p

ipel

ine

regis

ter pro

vid

e?

That

is le

ft to the

read

er.

Pipelining

26

We

hav

e 5 sta

ges

. W

hat

nee

ds to

be

controlled

in e

ach sta

ge?

-In

stru

ctio

n F

etch

and P

C Incr

emen

t

-In

stru

ctio

n D

ecode

/ R

egis

ter Fet

ch

-Exec

ution

-M

emory

Sta

ge

-W

rite

Bac

k

How

would

control be

han

dle

d in a

n a

uto

mobile

pla

nt?

-a

fancy

control ce

nte

r te

llin

g e

ver

yone

what

to d

o?

-sh

ould

we

use

a fin

ite

stat

e m

achin

e?

Pipeline Control




-sh

ould

we

use

a fin

ite

stat

e m

achin

e?

Pipelining

27

Pipeline Control

Iden

tify

the

nec

essa

ry c

ontrol si

gnal

s:




Pipelining

28

Pas

s co

ntrol si

gnal

s al

ong just

lik

e th

e dat

a:

Pipeline Control




Execution/Address Calculation

stage control lines

Memory access stage

control lines

Write-back

stage control

lines

Instruction

Reg

Dst

ALU

Op1

ALU

Op0

ALU

Src

Branch

Mem

Read

Mem

Write

Reg

write

Mem to

Reg

R-form

at

11

00

00

01

0lw

00

01

01

01

1sw

X0

01

00

10

Xbeq

X0

10

10

00

X

Pipelining

29

Datapath with Control




Documents

Basic Instruction Timings - Virginia Techcourses.cs.vt.edu › ~cs2504 › fall2008 › Notes › PDF › L23... · Computer Science Dept Va Tech January 2006 ©2006 McQuain & Ribbens