Multi/Many Core SystemsMulti/Many Core Systems

By Ahmed Shah Mashiyat

Processor Trends So Far..Processor Trends So Far..• Smarter Brain Smarter Brain

– –(e.g. x386 (e.g. x386 x486 x486 Pentium Pentium P2 P2 P3 P3 P4)P4)

• Larger MemoryLarger Memory

– –Larger caches, DRAM, DiskLarger caches, DRAM, Disk• Smaller HeadSmaller Head

– –Fewer chips (integrate more things onto a Fewer chips (integrate more things onto a chip)chip)

• More Power Consumption More Power Consumption

– –few Watts few Watts 120+ Watts!120+ Watts!• More ComplexMore Complex

– –1Billion Transistors; design + verification 1Billion Transistors; design + verification complexitycomplexity

Processor-Memory Processor-Memory Performance GapPerformance Gap

µProc60%/yr.

DRAM7%/yr.

1

10

100

1000

1980

1981

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

DRAM

CPU

1982

Per

form

ance

“ Moore’s Law”

From D. Patterson, CS252, Spring 1998 ©UCB

Major Problem Today: End Major Problem Today: End of the Road!of the Road!

• Can’t increase Power Can’t increase Power Consumption (~100W!)Consumption (~100W!)

• Can’t increase Design ComplexityCan’t increase Design Complexity• Can’t increase Verification Can’t increase Verification

RequirementsRequirements

No further improvements to a No further improvements to a Processor?!Processor?!

Obvious Answer: Use Obvious Answer: Use Multiple BrainsMultiple Brains

• If single brain can’t be If single brain can’t be improved, use multiple brains!improved, use multiple brains!

• Put multiple simple CPUs on a Put multiple simple CPUs on a single chipsingle chip

Multi-Core!Multi-Core!

The First Multi-Core: IBM The First Multi-Core: IBM Power 4 Processor Power 4 Processor

DefinitionsDefinitions

• A A Multi-CoreMulti-Core processor combines two or processor combines two or more independent more independent corescores into a single package into a single package composed of a single integrated circuit (IC), composed of a single integrated circuit (IC), called a die, or more dies packaged together. called a die, or more dies packaged together.

A A dual-coredual-core processor contains two cores, processor contains two cores, and a and a quad-corequad-core processor contains four processor contains four cores. cores.

• Typically the term Typically the term Many-CoreMany-Core is sometimes is sometimes used to describe multi-core architectures with used to describe multi-core architectures with an especially high number of cores (tens or an especially high number of cores (tens or hundreds). hundreds).

Multi coreMulti core• Single ChipSingle Chip

• Multiple distinct processing EngineMultiple distinct processing Engine

• E.g.) Shared-cache Dual Core ArchitectureE.g.) Shared-cache Dual Core Architecture

Core 0

CPU

L1 Cache

Core 1

CPU

L1 Cache

L2 Cache

Multi coreMulti core

• Cores in a multi-core device may share a Cores in a multi-core device may share a single coherent cache at the highest on- single coherent cache at the highest on- device cache level (e.g. L2 for the Intel Core device cache level (e.g. L2 for the Intel Core 2) or may have separate caches (e.g. current 2) or may have separate caches (e.g. current AMD dual-core processors).AMD dual-core processors).

• The processors also share the same The processors also share the same interconnect to the rest of the system.interconnect to the rest of the system.

• Each "core" independently implements Each "core" independently implements optimizations such as superscalar execution, optimizations such as superscalar execution, pipelining, and multithreading.pipelining, and multithreading.

• A system with n cores is effective when it is A system with n cores is effective when it is presented with n or more threads presented with n or more threads concurrently.concurrently.

Systems using Systems using Multi coreMulti core

• The most commercially significant (or at The most commercially significant (or at least the most 'obvious') multi-core least the most 'obvious') multi-core processors are those used in personal processors are those used in personal computers (primarily from Intel and AMD) computers (primarily from Intel and AMD) and game consoles (e.g., the eight-core Cell and game consoles (e.g., the eight-core Cell processor in the PS3 and the three-core processor in the PS3 and the three-core Xenon processor in the Xbox 360). Xenon processor in the Xbox 360).

• "multi" typically means a relatively small "multi" typically means a relatively small number of cores. However, the technology number of cores. However, the technology is widely used in other technology areas, is widely used in other technology areas, especially those of embedded processors, especially those of embedded processors, such as network processors and digital such as network processors and digital signal processors, and in GPUs.signal processors, and in GPUs.

Advantages of Multi-coreAdvantages of Multi-core• The proximity of multiple CPU cores on the The proximity of multiple CPU cores on the

same die allows the cache coherency circuitry same die allows the cache coherency circuitry to operate at a much higher clock rate than is to operate at a much higher clock rate than is possible if the signals have to travel off-chip. possible if the signals have to travel off-chip. Combining equivalent CPUs on a single die Combining equivalent CPUs on a single die significantly improves the performance of significantly improves the performance of cache snoop (alternative: Bus snooping) cache snoop (alternative: Bus snooping) operations. Put simply, this means that operations. Put simply, this means that signals between different CPUs travel shorter signals between different CPUs travel shorter distances, and therefore those signals distances, and therefore those signals degrade less. These higher quality signals degrade less. These higher quality signals allow more data to be sent in a given time allow more data to be sent in a given time period since individual signals can be shorter period since individual signals can be shorter and do not need to be repeated as often.and do not need to be repeated as often.

Advantages of Multi-Advantages of Multi-core(cont..)core(cont..)

• The largest boost in performance will likely The largest boost in performance will likely be noticed in improved response time while be noticed in improved response time while running CPU-intensive processes, like running CPU-intensive processes, like antivirus scans, ripping/ burning media antivirus scans, ripping/ burning media (requiring file conversion), or searching for (requiring file conversion), or searching for folders. For example, if the automatic virus folders. For example, if the automatic virus scan initiates while a movie is being scan initiates while a movie is being watched, the application running the movie watched, the application running the movie is far less likely to be starved of processor is far less likely to be starved of processor power, as the antivirus program will be power, as the antivirus program will be assigned to a different processor core than assigned to a different processor core than the one running the movie playback.the one running the movie playback.

Advantages of Multi-Advantages of Multi-core(cont..)core(cont..)

• Assuming that the die can fit into the Assuming that the die can fit into the package, physically, the multi-core CPU package, physically, the multi-core CPU designs require much less Printed Circuit designs require much less Printed Circuit Board (PCB) space than multi-chip SMP Board (PCB) space than multi-chip SMP designs. Also, a dual-core processor uses designs. Also, a dual-core processor uses slightly less power than two coupled single-slightly less power than two coupled single-core processors, principally because of the core processors, principally because of the decreased power required to drive signals decreased power required to drive signals external to the chip. Furthermore, the cores external to the chip. Furthermore, the cores share some circuitry, like the L2 cache and share some circuitry, like the L2 cache and the interface to the front side bus (FSB).the interface to the front side bus (FSB).

The Result…The Result…

• Today’s CPU manufacturers have turned Today’s CPU manufacturers have turned to to multi-coremulti-core designs e.g. putting more  designs e.g. putting more than one “brain” on a single chip. Intel than one “brain” on a single chip. Intel offers two, four or eight core chips with offers two, four or eight core chips with more to come, Sun Microsystems has more to come, Sun Microsystems has chips with up to 64 cores, AMD has its chips with up to 64 cores, AMD has its quad-core chip (and others), etc. So it quad-core chip (and others), etc. So it seems the future belongs to multi-core seems the future belongs to multi-core systems, especially in the server systems, especially in the server marketplace. marketplace.

A Crisis in the Industry!A Crisis in the Industry!

• Parallelism is Difficult to Find!Parallelism is Difficult to Find!

––Writing Parallel Programs is much Writing Parallel Programs is much harder!harder!

––Parallel Architectures, Compilers, and Parallel Architectures, Compilers, and Programmers have been researched for Programmers have been researched for many decadesmany decades

• But, now everything will have multi-core:But, now everything will have multi-core:

––Desktops, Servers, Supercomputers, Desktops, Servers, Supercomputers, Palmtops, Embedded Computers Palmtops, Embedded Computers

Reason for the crisis…Reason for the crisis…

• Memory bandwidthMemory bandwidth

-- Memory bandwidth-- Memory bandwidth is a major limitation  is a major limitation with multi-core systems. All cores must share with multi-core systems. All cores must share access to the same memory and access to the same memory and therefore therefore have to take turns to read from or write ithave to take turns to read from or write it..

• Disk speedDisk speed

-- -- Disk speed can throttle performance. If a Disk speed can throttle performance. If a disk can read only 40MB/sec (a common disk can read only 40MB/sec (a common figure for even a fast laptop hard drive), then figure for even a fast laptop hard drive), then a program that needs to read a 400MB file a program that needs to read a 400MB file will take a minimum of 10 seconds to open it, will take a minimum of 10 seconds to open it, even if it has no computation to do. even if it has no computation to do.

Reason for the crisis…Reason for the crisis…

• Software and scalabilitySoftware and scalability

----softwaresoftware today is frequently not written  today is frequently not written to take advantage of multiple cores to take advantage of multiple cores efficiently; it is technically challenging to efficiently; it is technically challenging to write write efficientefficient threaded code that is free  threaded code that is free of threading bugs (threading bugs are of threading bugs (threading bugs are those that result from incorrectly-written those that result from incorrectly-written threaded code). threaded code).

-- A highly -- A highly scalablescalable program is difficult to  program is difficult to build. It requires a lot of expertise.build. It requires a lot of expertise.

Crisis Crisis Opportunity!Opportunity!

• Industry is willing/forced to consider:Industry is willing/forced to consider:

––Code-rewrite to better expose parallelismCode-rewrite to better expose parallelism

––Newer ways of expressing parallelismNewer ways of expressing parallelism

––Newer programming languagesNewer programming languages

––Newer architecturesNewer architectures

––Overall: Newer lines of thinking and Overall: Newer lines of thinking and research!research!

• Marvelous Opportunity to do Research Marvelous Opportunity to do Research and Development in this area!and Development in this area!

ConclusionsConclusions

• That fancy new computer won’t deliver That fancy new computer won’t deliver on its speed promise unless it offers on its speed promise unless it offers adequate memory bandwidth for the adequate memory bandwidth for the program you actually run. Those program you actually run. Those programs need to be especially well programs need to be especially well designed to exploit more than one or designed to exploit more than one or two CPU cores.two CPU cores.

• So for now we are not ready to So for now we are not ready to capitalize the power of multi/many core capitalize the power of multi/many core system to full extent.system to full extent.

ReferencesReferences• Multi-core Computers @ Multi-core Computers @

http://diglloyd.com/diglloyd/free/MultiCore/index.html• Multi-core @ http://en.wikipedia.org/wiki/Multi-coreMulti-core @ http://en.wikipedia.org/wiki/Multi-core• Multi-core state of play @ Multi-core state of play @

http://www.builderau.com.au/strategy/architecture/soa/Multi-http://www.builderau.com.au/strategy/architecture/soa/Multi-core-state-of-play/0,339028264,339291257,00.htm?core-state-of-play/0,339028264,339291257,00.htm?feed=pt_programmingfeed=pt_programming

• Central processing unit @ http://en.wikipedia.org/wiki/CPUCentral processing unit @ http://en.wikipedia.org/wiki/CPU• Multi Core Processors: The Possibilities for a New Era in

Computer Architecture… IBM Research @ http://203.197.173.12/faer/treach/pdf/Multicore-Possibilities.pdf

• HETEROGENEOUS MULTICORE PROCESSORS @ http://132.236.67.210/EngrWords/issues/ew04/Vitkalov_talk.pdf

• OVERVIEW OF MULTICORE, PARALLEL COMPUTING,OVERVIEW OF MULTICORE, PARALLEL COMPUTING,AND DATA MINING @ AND DATA MINING @ http://grids.ucs.indiana.edu/ptliupages/presentations/http://grids.ucs.indiana.edu/ptliupages/presentations/

• Introduction to High Performance Computing @ Introduction to High Performance Computing @ http://cse.stfx.ca/~ltyang/csci-455/ http://cse.stfx.ca/~ltyang/csci-455/

THANK YOU!THANK YOU!

QUESTIONS?QUESTIONS?