Sustaining the Exponential Growth of Embedded DSP Capability † High Performance Embedded Computing Workshop 28 September 2004 Dr. Gary Shaw Dr. Mark Richards

Sustaining the Exponential Growth of Embedded DSP Capability†

High Performance Embedded Computing Workshop

28 September 2004

Dr. Gary Shaw Dr. Mark Richards MIT Lincoln Laboratory Georgia Institute of Technology

MIT Lincoln Laboratory

† This work was sponsored by the Department of the Air Force under Contract F19628-00-C-0002. Opinions, interpretations, conclusions, and recommendations are those of the author and are not necessarily endorsed by the United States Government.

MIT Lincoln Laboratory040928-2

HPEC GAS

Elements Contributing to Embedded Processor Performance

Signal Processor

Hardware Software

Computer Architecture

IC Devices Algorithms Functionality

Signal Processor

Hardware SoftwareHardware Software


IC Devices Algorithms FunctionalityComputer Architecture



HPEC GAS

Outline

• Historical perspective – fulfillment of Moore’s Law

• Impediments to continued IC density growth

• Algorithms – the softer side of exponential growth

• Implications regarding sustaining exponential growth

• Summary and Conclusions


HPEC GAS

Original Prediction(Source: Electronics 1965)

Moore’s Law: Prediction and Realization

Source: IntelSource: IntelSource: IntelSource: Intel

2x every year 2x every year

2x every ~1.9 years 2x every ~1.9 years

Gordon Moore: “If Al Gore invented the Internet, I invented the exponential”Gordon Moore: “If Al Gore invented the Internet, I invented the exponential”

John von Neumann: “Truth is much too complicated to allow anything but approximations.”


HPEC GAS

Top 500 Computer Growth

0.10

1.00

10.00

100.00

1000.00

10000.00

100000.00

Ju

n-9

3

Ju

n-9

4

Ju

n-9

5

Ju

n-9

6

Ju

n-9

7

Ju

n-9

8

Ju

n-9

9

Ju

n-0

0

Ju

n-0

1

Ju

n-0

2

Ju

n-0

3

Ju

n-0

4

Pe

ak

Pe

rfo

rma

nc

e (

GF

LO

PS

)

N=1

N=100

N=5002x every 1.1 year2x every 1.1 year


HPEC GAS

Outline

• Historical perspective - fulfillment of Moore’s Law

• Impediments to continued IC density growth– Heat dissipation– Quantum effects– Production technology





HPEC GAS

Performance Implications of Shrinking Feature Size

Performance Metric

Geometrical Dependency

Clock Frequency

Transistor Power

Transistor Density 2

Total Device Power

Power Density

Energy/Instruction 2

feature size

FrantzSmailagicIntel trend

-2x every 1.8 years

Year

nJ/

Inst

ruct

ion

0.001

0.01

0.1

1

10

100

1000

10000

100000

1965 1970 1975 1980 1985 1990 1995 2000 2005 2010 2015 2020

Energy per Instruction


HPEC GAS

Moore’s Law Growth in Power Density

Po

wer

Den

sity

(W

/cm

2 )

Year

1985 1990 1995 2000 2005 2010

1

10

100

i386

i486Pentium

Pentium MMX

Pentium 4

Pentium II

hot plate

space shuttle tile

nuclear reactor fuel cell

1000

2x in 3.3 years


HPEC GAS

Moore’s Law is Dead, Long Live Moore’s Law!Theory & Practice: Feature Size for MOSFET Devices

Sources:Combined graph and original concept: Lance Glasser, former Director, DARPA/ETOTheory: Provided by Prof. David Ferry, Arizona State UniversityPractice: The National Technology Roadmap for Semiconductors (SIA Publication, 1994)

Gat

e L

eng

th (m

)

Year

YearElect.Per Bit

Now ~100

+10 ~10

+20 ~1

It's tough to make predictions, especially about the future. - Yogi Berra

MIT Lincoln Laboratory040928-10HPEC GAS

Capitalization Cost Impediments

Extrapolation

ROI Risk Limited?

1970 1980 1990 2000 2010 2020

0.01

0.1

1

10

100W

afe

r-F

ab

Ca

pit

al C

os

t ($

B)

Year

2x every 3 years

12” fab $3-4B

Adapted from:

Intel withdraws from DRAM market due to estimated ~ $400M capitalization cost


• Silicon CMOS IC fabrication technology

• Examples of far-reaching impact

Fulfillment and Impact of Moore’s Prediction

Funda-mentalLimits

ParadigmShift?

QuantumComputing?

BiologicalComputing?

Exponential ImprovementsIn Computing at a Fixed

Price Point

Embedded Processors For Real-time Digital

Signal Processing

Low-power Wireless Applications

Loosely-CoupledHardware & Software

Design Methodologies

1965 2005 2015?Year

Transistors 102 109

Naïve, low-order implementations

ScalingRules

ProcessInnovations

Design ToolInnovations

Altair 8800, 1975

Itanium, 2005


Outline







Elements Contributing to Embedded Processor Performance

Signal Processor

Hardware Software



Signal Processor

Hardware Software


IC Devices Algorithms FunctionalityComputer Architecture

Algorithms

Hardware Software

IC Devices Functionality


Different Character of Hardware (IC) Vs. Algorithm Improvements

Improvement Metrics Hardware Algorithms

Regularity Predictable Unpredictable

Dependent variable Time Order complexity

Impact on applications Incremental Leap-ahead

Useful lifetime 3 years or less 10 years or more

R&D Cost growth 2x in 3 years 1.11x in 3 years


Computational Complexity Reduction Afforded by the FFT Over a Sum-of-Products DFT

1

10

100

1000

10000

4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384

FFT Length

Co

mp

lexi

ty R

edu

ctio

n F

acto

r


Moore’s-Law Equivalent Years Required to Match FFT Computational Speedup

Equivalent Years of Hardware Improvement

0

5

10

15

20

25

4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384

FFT Length

Eq

uiv

alen

t M

oo

re's

Law

Im

pro

vem

ents

Y

ears

Radix-2 FFT Length

Yea

rs o

f H

ard

war

e Im

pro

vem

ent

Req

uir

ed

fo

r E

qu

al C

om

pu

tati

on

al s

pee

du

p


100

1000

10000

100000

1984 1986 1988 1990 1992 1994 1996 1998 2000

Year

Do

wn

stre

am D

ata

Rat

e (b

ps)

Exponential Improvement in Modem Rates

2x every 1.8 years2x every 1.8 years


Application Maturation Cycle


Pulse-Doppler Radar Example

• Algorithmically naïve implementation

• Reduced-order implementation with digital I/Q

BPF

LPF

A/D

A/D

I

Q

Time-DomainMatched Filter

MatrixTranspose

DopplerDFTcos(ot)

sin(ot)

LPF

LO

RF

MatrixTranspose

DopplerFFT

RangeFFT

I + jQ

Chirp

LPF A/D

j n

LPFRF


0.01

0.1

1

10

100

Pulse CompressionDoppler FilteringDigital I/Q

Pulse-Doppler Radar Algorithm Improvements

Year

GO

PS

Goe

rtze

l

Rad

ix-2

FFT

Rad

ix-4

FFT

Fast

Con

volu

tion

Str

etch

Pro

cess

ing

Dig

ital I

/Q

Moore’s LawReference slopeMoore’s LawReference slope


Outline







IC Vs. Algorithm Development (A Contrived but Useful Analogy)

• ICs

Silicon lithographic fabrication technology

Funda-mentalLimits

ParadigmShift?

QuantumComputing?

BiologicalComputing?

Naïve, low-order implementations

ScalingRules

ProcessInnovations

Design ToolInnovations

• Algorithms

Funda-mentalLimits

ParadigmShift?

Quantum SignalProcessing?

NonlinearAlgorithms?

Linear algebraic representations and processing

Naïve, high-order implementations

OrderReductions

ArchitectureInnovations

H/W & S/WCodesign Tools


Increased Emphasis on Codesign Methodologies

Signal Processor

Hardware Software

IC Devices FunctionalityComputer AlgorithmsArchitecture

Signal Processor

Hardware SoftwareHardware Software

IC Devices FunctionalityComputer AlgorithmsArchitecture


1970 1980 1990 2000 2010 2020

Wafer-Fab Capitalization Cost Compared to Annual DSP Algorithm R&D Costs

0.01

0.1

1

10

100

$B

Year

2x every 3 years

Capital cost for state-of-the-art wafer fab facility

1.11x every 3 years†

† Salary inflation rate based on US Bureau of Labor and Statistics Median Engineering Salaries 1983-2003

Annual R&D support for entire IEEE SP Society membership (18,500 x $150K in 2001)


Summary and Conclusions

• Fulfilling Moore’s Law– Enabled by diverse, innovative R&D aimed at realizing a common

vision (ITRS semiconductor roadmap)– Continued improvements may be impeded by a combination of

thermal, quantum, and capital cost limits

• Taking up the slack– Over same 40-year time frame as Moore’s Law, algorithm innovation

has yielded exponentially improving performance as well– Algorithm innovation also enabled by diverse R&D, but without as clear

of an industry-wide common vision– Algorithm R&D cost growth significantly lower than fab capital cost

growth (1.1x vs. 2x every 3 years)

• Increasing the effectiveness of algorithm R&D– Develop better methods for quantifying the return on investment for

algorithm R&D– Consider mechanisms for developing a broader industry vision and

commitment to a long-term R&D roadmap

• Hardware/software codesign methods increasingly important

Documents

Sustaining the Exponential Growth of Embedded DSP Capability † High Performance Embedded Computing Workshop 28 September 2004 Dr. Gary Shaw Dr. Mark Richards