45
EECC756 - Shaaban EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2 Cache Coherence in Bus-Based Shared Memory Multiprocessors Multiprocessors Shared Memory Multiprocessors Variations Cache Coherence in Shared Memory Multiprocessors A Coherent Memory System: Intuition Formal Definition of Coherence Shared Memory Access Consistency Models Sequential Consistency (SC) Model Sequential Consistency (SC) Model Cache Coherence Approaches Bus-Snooping Cache Coherence Protocols Write-invalidate Bus-Snooping Protocol Write-invalidate Bus-Snooping Protocol For Write-Through For Write-Through Caches Caches Write-invalidate Bus-Snooping Protocol For Write-Back Write-invalidate Bus-Snooping Protocol For Write-Back Caches Caches MSI Write-Back Invalidate Protocol MESI Write-Back Invalidate Protocol Write-update Bus-Snooping Protocol For Write-Back Caches Write-update Bus-Snooping Protocol For Write-Back Caches Dragon Write-back Update Protocol Dragon Write-back Update Protocol PCA Chapter 5

EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

  • View
    233

  • Download
    2

Embed Size (px)

Citation preview

Page 1: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#1 lec # 10 Spring2006 5-4-2006

Cache Coherence in Bus-Based Shared Memory MultiprocessorsMultiprocessors

• Shared Memory Multiprocessors Variations • Cache Coherence in Shared Memory Multiprocessors• A Coherent Memory System: Intuition• Formal Definition of Coherence• Shared Memory Access Consistency Models

– Sequential Consistency (SC) ModelSequential Consistency (SC) Model

• Cache Coherence Approaches• Bus-Snooping Cache Coherence Protocols

– Write-invalidate Bus-Snooping ProtocolWrite-invalidate Bus-Snooping Protocol For Write-Through CachesFor Write-Through Caches– Write-invalidate Bus-Snooping Protocol For Write-Back CachesWrite-invalidate Bus-Snooping Protocol For Write-Back Caches

• MSI Write-Back Invalidate Protocol• MESI Write-Back Invalidate Protocol

– Write-update Bus-Snooping Protocol For Write-Back CachesWrite-update Bus-Snooping Protocol For Write-Back Caches• Dragon Write-back Update ProtocolDragon Write-back Update Protocol

PCA Chapter 5

Page 2: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#2 lec # 10 Spring2006 5-4-2006

• Direct support in hardware of shared address space (SAS) parallel programming model: address translation and protection in hardware (hardware SAS).

• Any processor can directly reference any memory location

– Communication occurs implicitly as result of loads and stores

• Normal uniprocessor mechanisms used to access data (loads and stores) + synchronization

– Key is extension of memory hierarchy to support multiple processors.

• Memory may be physically distributed among processors

• Caches in the extended memory hierarchy may have multiple inconsistent copies of the same data leading to data consistency or cache coherence problem that have to addressed by hardware architecture.

Shared Memory MultiprocessorsShared Memory Multiprocessors

Extended memory hierarchy

Page 3: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#3 lec # 10 Spring2006 5-4-2006

Shared Address Space (SAS) Parallel Programming Model• Process: virtual address space plus one or more threads of control

• Portions of address spaces of processes are shared

• Writes to shared address visible to other threads (in other processes too)• Natural extension of the uniprocessor model:

• Conventional memory operations used for communication• Special atomic operations needed for synchronization• OS uses shared memory to coordinate processes

St or e

P1

P2

Pn

P0

Load

P0 pr i vat e

P1 pr i vat e

P2 pr i vat e

Pn pr i vat e

Virtual address spaces for acollection of processes communicatingvia shared addresses

Machine physical address space

Shared portionof address space

Private portionof address space

Common physicaladdresses

From lecture 1

Page 4: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#4 lec # 10 Spring2006 5-4-2006

Shared Memory Multiprocessors:Shared Memory Multiprocessors: Support of Programming ModelsSupport of Programming Models

• Address translation and protection in hardware (hardware SAS).

• Message passing using shared memory buffers:• Can offer very high performance since no OS involvement necessary.

• The focus here is on supporting a consistent or coherent shared address space.

Multiprogramming

Shared address space

Message passing Programming models

Communication abstractionUser/system boundary

Compilationor library

Operating systems support

Communication hardware

Physical communication medium

Hardware/software boundary

Page 5: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#5 lec # 10 Spring2006 5-4-2006

• Uniform Memory Access (UMA) MultiprocessorsMultiprocessors :

– All processors have equal access to all memory addresses.

– Can be further divided into three types:• Bus-based shared memory multiprocessors

– Symmetric Memory Multiprocessors (SMPs).

• Shared cache multiprocessorsShared cache multiprocessors

• Dancehall multiprocessorsDancehall multiprocessors

• Non-uniform Memory Access (NUMA) or distributed memory MultiprocessorsMultiprocessors :– Shared memory is physically distributed locally among

processors (nodes). Access to remote memory is higher.– Most popular design to build scalable systems (MPPs).

– Cache coherence achieved by directory-based methods.

Shared Memory Multiprocessors VariationsShared Memory Multiprocessors Variations

Page 6: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#6 lec # 10 Spring2006 5-4-2006

Shared Memory Multiprocessors VariationsShared Memory Multiprocessors Variations

I/O devicesMem

P1

$ $

Pn

P1

Switch

Main memory

Pn

(Interleaved)

(Interleaved)

P1

$

Interconnection network

$

Pn

Mem Mem

(b) Bus-based shared memory

(c) Dancehall

(a) Shared cache

First-level $

Bus

P1

$

Interconnection network

$

Pn

Mem Mem

(d) Distributed-memory

Symmetric Memory Multiprocessors (SMPs)

Bus or point-to-point interconnects

NUMA

UMAUMA

Or SMP nodes

Scalable networkp-to-p or MIN

UMA

Scalable DistributedShared Memory

Page 7: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#7 lec # 10 Spring2006 5-4-2006

• Bus-based Bus-based Multiprocessors: (SMPs)– A number of processors (commonly 2-4) in a single node share physical memory via A number of processors (commonly 2-4) in a single node share physical memory via system bussystem bus or or

point-to-point interconnectspoint-to-point interconnects (e.g. AMD64 via. HyperTransport) (e.g. AMD64 via. HyperTransport)

– Symmetric access to all of main memory from any processor.• Commonly called: Symmetric Memory Multiprocessors (SMPs).

– Building blocks for larger parallel systems (MPPs, clusters)

– Also attractive for high throughput servers

– Bus-snooping mechanisms used to address the cache coherency problem.

• Shared cache Multiprocessor Systems:Shared cache Multiprocessor Systems:– Low-latency sharing and prefetching across processors.– Sharing of working sets.– No cache coherence problem (and hence no false sharing either).– But high bandwidth needs and negative interference (e.g. conflicts).– Hit and miss latency increased due to intervening switch and cache size.– Used in mid 80s to connect a few of processors on a board (Encore, Sequent).– Used currently in chip multiprocessors (CMPs): 2-4 processors on a single chip. e.g IBM Power

4, 5: two processor cores on a chip (shared L2).

• Dancehall: – No local memory associated with a node.

– Not a popular design: All memory is uniformly costly to access over the network for all processors.

Uniform Memory Access (UMA) Multiprocessors

Page 8: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#8 lec # 10 Spring2006 5-4-2006

Uniform Memory Access Example: Uniform Memory Access Example: Intel Pentium Pro QuadIntel Pentium Pro Quad

P-Pro bus (64-bit data, 36-bit addr ess, 66 MHz)

CPU

Bus interface

MIU

P-Promodule

P-Promodule

P-Promodule256-KB

L2 $Interruptcontroller

PCIbridge

PCIbridge

Memorycontroller

1-, 2-, or 4-wayinterleaved

DRAM

PC

I bus

PC

I busPCI

I/Ocards

• All coherence and multiprocessing glue in processor module

• Highly integrated, targeted at high volume

Bus-Based Symmetric Memory Processors (SMPs).

A single Front Side Bus (FSB) is shared among processorsThis severely limits scalability to only 2-4 processors

Circa 1997

Shared FSB

Repeated here from lecture 1

Page 9: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#9 lec # 10 Spring2006 5-4-2006

Non-Uniform Memory Access (NUMA) Non-Uniform Memory Access (NUMA) Example: AMD 8-way Opteron Server NodeExample: AMD 8-way Opteron Server Node

Dedicated point-to-point interconnects (Coherent HyperTransport links) used to connect processors alleviating the traditional limitations of FSB-based SMP systems (yet still providing the cache coherency support needed)Each processor has two integrated DDR memory channel controllers:memory bandwidth scales up with number of processors.NUMA architecture since a processor can access its own memory at a lower latency than access to remote memory directly connected to other processors in the system.

Total 16 processor cores when dual core Opteron processors used

Circa 2003

Repeated here from lecture 1

Page 10: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#10 lec # 10 Spring2006 5-4-2006

Chip Multiprocessor (Shared-Cache) Example:Chip Multiprocessor (Shared-Cache) Example: IBM Power 4 IBM Power 4

Two Processor cores on a chipshare level 2 cache

CMP

Page 11: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#11 lec # 10 Spring2006 5-4-2006

Generic Distributed Memory Generic Distributed Memory OrganizationOrganization

• Network bandwidth?• Bandwidth demand?

– Independent processes?– Communicating processes?

• Latency? O(log2P) increase?

• Cost scalability of system?

Scalable network

CA

P

$

Switch

M

Switch Switch

Multi-stageinterconnection network (MIN)?Point-to-point?Custom-designed?SAN?

Node:O(10) Bus-based SMP

OS Supported?Network protocols?

Communication AssistExtent of functionality? DMA, user-level access user-level handlers

MessagetransactionDMA?

Global physical or virtual Shared address space support?

Page 12: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#12 lec # 10 Spring2006 5-4-2006

Complexities of MIMD Shared Memory Access• Relative order (interleaving) of instructions in different

streams in not fixed.– With no synchronization among instructions streams, a

largelarge number of instruction interleavings is possible.

• If instructions are reordered in a stream then an even largerlarger of number of instruction interleavings is possible.

• If memory accesses are not atomic with multiple copies of the same data coexisting (cache-based systems) then different processors observe different interleavings during the same execution. The total number of possible observed execution

orders becomes even largerlarger.

i.e Effect of access not visible to memory, all processors in the same order

i.e orders

Page 13: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#13 lec # 10 Spring2006 5-4-2006

Cache Coherence in Shared Memory Cache Coherence in Shared Memory MultiprocessorsMultiprocessors

• Caches play a key role in all shared memory multiprocessor system variations:– Reduce average data access time (AMAT).– Reduce bandwidth demands placed on shared interconnect.

• Replication in cache reduces artifactual communication.

• Cache coherence or inconsistency problem.– Private processor caches create a problem:

• Copies of a variable can be present in multiple caches. • A write by one processor may not become visible to others:

– Processors accessing stale (old) value in their private caches.

• Also caused by:– Process migration.– I/O activity.

– Software and/or hardware actions needed to ensure write visibility to all processors thus maintaining cache coherence.

• i.e. Processors must see the most updated value

Page 14: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#14 lec # 10 Spring2006 5-4-2006

Cache Coherence Problem Example Cache Coherence Problem Example

– Processors see different values for u after event 3.– With write back caches, a value updated in cache may not have been written

back to memory: • Processes even accessing main memory may see very stale value.

– Unacceptable: leads to incorrect program execution.

I/O devices

Memory

P1

$ $ $

P2 P3

12

34 5

u = ?u = ?

u:5

u:5

u:5

u = 7

1 P1 reads u=5 from memory2 P3 reads u=5 from memory3 P3 writes u=7 to local P3 cache4 P1 reads old u=5 from local P1 cache5 P2 reads old u=5 from memory

Write Back Caches Assumed

Frommain memory

Write by P3not visible to P1, P2

PCA page 274

Page 15: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#15 lec # 10 Spring2006 5-4-2006

A Coherent Memory System: IntuitionA Coherent Memory System: Intuition

Reading a memory location should return the latest value written (by any process).

• Easy to achieve in uniprocessors:– Except for DMA-based I/O: Coherence between DMA I/O devices and

processors.– Infrequent so software solutions work:

• Uncacheable memory regions, uncacheable operations, flush pages, pass I/O data through caches.

• The same should hold when processes run on different processors: – e.g. Results should be the same as if the processes were interleaved (or

running) on a uniprocessor.• Coherence problem much more critical in multiprocessors:

– Pervasive.

– Performance-critical.

– Must be treated as a basic hardware design issue.

PCA page 275

Page 16: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#16 lec # 10 Spring2006 5-4-2006

Basic DefinitionsBasic DefinitionsExtend definitions in uniprocessors to multiprocessors:

• Memory operation: a single read (load), write (store) or read-modify-write access to a memory location.– Assumed to execute atomically (visible) with respect to (w.r.t) each other

and in the same order.

• Issue: A memory operation issues when it leaves processor’s internal environment and is presented to memory system (cache, buffer …).

• Perform: operation appears to have taken place, as far as processor can tell from other memory operations it issues.– A write performs w.r.t. the processor when a subsequent read by the

processor returns the value of that write or a later write (no RAW).– A read perform w.r.t the processor when subsequent writes issued by

the processor cannot affect the value returned by the read (no WAR).

• In multiprocessors, stay same but replace “the” by “a” processor– Also, complete: perform with respect to all processors.

– Still need to make sense of order in operations from different processes.

PCA page 276

or WAW

Page 17: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#17 lec # 10 Spring2006 5-4-2006

Shared Memory Access ConsistencyShared Memory Access Consistency

• A load by processor Pi is performed with respect to processor Pk at a point in time when the issuing of a subsequent store to the same location by Pk cannot affect the value returned by the load (no WAW, WAR).

• A store by Pi is considered performed with respect to Pk at one time when a subsequent load from the same address by Pk returns the value by this store (no RAW).

• A load is globally performed if it is performed with respect to all processors and if the store that is the source of the returned value has been performed with respect to all processors.

Page 18: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#18 lec # 10 Spring2006 5-4-2006

Formal Definition of CoherenceFormal Definition of Coherence• Results of a program: values returned by its read (load) operations• A memory system is coherent if the results of any execution of a

program are such that for each location, it is possible to construct a hypothetical serial order of all operations to the location that is consistent with the results of the execution and in which:

1. operations issued by any particular process occur in the order issued by that process, and

2. the value returned by a read is the value written by the latest write to that location in the serial order

• Two necessary conditions:– Write propagation: value written must become visible to others – Write serialization: writes to location seen in same order by all

• if one processor sees w1 after w2, another processor should not see w2 before w1

• No need for analogous read serialization since reads not visible to others.

PCA page 276-277

i.e processors

Page 19: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#19 lec # 10 Spring2006 5-4-2006

Shared Memory Access Consistency ModelsShared Memory Access Consistency Models• Memory Access Consistency Models: Specify the order by

which shared memory access events of one process should be observed by other processes in the system.– Strict Consistency: A read always returns with the most recent write to

the same location

– Sequential Consistency (SC) Model: The result of any execution appears the same as the result of the interleaving of individual programs strictly in sequential program order.

– Weak or Relaxed Consistency Models.• Improve shared memory access performance while ensuring correctness

• Program Order: The order in which memory accesses appear in the execution of a single process without program reordering.

• Event Ordering: Used to declare whether a memory event is legal (for a given memory access consistency model) when several processes access a common set of memory locations.

Page 20: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#20 lec # 10 Spring2006 5-4-2006

Sequential Consistency (SC) ModelSequential Consistency (SC) Model• Lamport’s Definition of SC:

[Hardware is sequentially consistent if] the result of any execution is the same as if the operations of all the processors were executed in some sequential order, and the operations of each individual processor appear in this sequence in the order.

• Sufficient conditions to achieve SC in shared-memory access:– Every process issues memory operations in program order– After a write operation is issued, the issuing process waits for the write to

complete before issuing its next operation.– After a read operation is issued, the issuing process waits for the read to

complete, and for the write whose value is being returned by the read to complete, before issuing its next operation (provides write atomicity).

• According to these Sufficient, but not necessary, conditions:– Clearly, compilers should not reorder for SC, but they do!

• Loop transformations, register allocation.– Even if issued in order, hardware may violate for better performance

• Write buffers, out of order execution.– Reason: uniprocessors care only about dependences to same location

• Makes the sufficient conditions very restrictive for performance.

i.e multiprocessor

for multiprocessing

Page 21: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#21 lec # 10 Spring2006 5-4-2006

Sequential Consistency (SC) ModelSequential Consistency (SC) ModelProcessors issuing memory references as per program order

P1 P2 Pn

Memory

The “switch” is randomly set after each memoryreference

• As if there were no caches, and a only single memory exists.

• Total order achieved by interleaving accesses from different processes.

• Maintains program order, and memory operations, from all processes,

appear to [issue, execute, complete] atomically w.r.t. others

• Programmer’s intuition is maintained.PCA page 287

Page 22: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#22 lec # 10 Spring2006 5-4-2006

SC ExampleSC Example

• Possible outcomes for (A,B): (0,0), (1,0), (1,2); impossible under SC: (0,2)

• We know 1a->1b and 2a->2b by program order

• A = 0 implies 2b->1a, which implies 2a->1b

• B = 2 implies 1b->2a, which leads to a contradiction

• BUT, actual execution 1b->1a->2b->2a is SC (A,B)=(1,2) despite not program order

• Appears just like 1a->1b->2a->2b as visible from results.

• Actual execution 1b->2a->2b-> is not SC (A,B)= (0,2)

P1 P2

/*Assume initial values of A and B are 0*/

(1a) A = 1; (2a) print B;

(1b) B = 2; (2b) print A;

PCA page 286

i.e results

i.e not SC

Page 23: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#23 lec # 10 Spring2006 5-4-2006

Implementing SCImplementing SC• Two types of requirements:

– Program order:• Memory operations issued by a process must appear to become

visible (to others and itself) in program order.

– Atomicity:• In the overall total order, one memory operation should appear to

complete with respect to all processes before the next one is issued.• Needed to guarantee that total order is consistent across processes. • Making writes atomic is involved

– due to the presence of multiple caches in the system requiring a cache coherence mechanism .

PCA page 287-288

To meet the “write serialization” necessary condition for coherence

Write-back caches change this order

i.e to the same location

Page 24: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#24 lec # 10 Spring2006 5-4-2006

Write AtomicityWrite Atomicity• Write Atomicity: Position in total order at which a write appears to

perform should be the same for all processes:– Nothing a process does after it has seen the new value produced by a

write W should be visible to other processes until they too have seen W.– In effect, extends write serialization to writes from multiple processes.

• Transitivity implies A should print as 1 under SC• Problem if P2 leaves loop, writes B, and P3 sees new B but old A (from its cache, i.e cache coherence problem violating SC).

P1 P2 P3

A=1; while (A==0);

B=1; while (B==0);

print A;

PCA page 289

Page 25: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#25 lec # 10 Spring2006 5-4-2006

Further Interpretation of SCFurther Interpretation of SC• Each process’s program order imposes partial order on set of

all operations.

• Interleaving of these partial orders defines a total order on all operations.

• Many total orders may be SC (SC does not define particular interleaving). Synchronization does.

• SC Execution: An execution of a program is SC if the results it produces are the same as those produced by some possible total order (interleaving)

• SC System: A system is SC if any possible execution on that system is an SC execution.

PCA page 288

Page 26: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#26 lec # 10 Spring2006 5-4-2006

Cache Coherence ApproachesCache Coherence Approaches• Bus-Snooping Protocols: Used in bus-based systems where all

processors observe memory transactions and take proper action to invalidate or update local cache content if needed.

• Directory Schemes: Used in scalable cache-coherent distributed-memory multiprocessor systems where cache directories are used to keep a record on where copies of cache blocks reside.

• Shared Caches: – No private caches.– This limits system scalability (limited to chip multiprocessors, CMPs).

• Non-cacheable Data: – Not to cache shared writable data:

• Locks, process queues.• Data structures protected by critical sections.

– Only instructions or private data is cacheable.– Data is tagged by the compiler.

• Cache Flushing: – Flush cache whenever a synchronization primitive is executed. – Slow unless special hardware is used.

Not Scalable

Scalable

Page 27: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#27 lec # 10 Spring2006 5-4-2006

Cache Coherence Using A BusCache Coherence Using A Bus• Built on top of two fundamentals of uniprocessor systems:

1 Bus transactions.2 State transition diagram of cache blocks.

• Uniprocessor bus transaction:– Three phases: arbitration, command/address, data transfer.

– All devices observe addresses, one is responsible for transaction

• Uniprocessor cache block states:– Effectively, every block is a finite state machine.– Write-through, write no-allocate has two states:

valid, invalid.– Write-back caches have one more state: Modified (“dirty”).

• Multiprocessors extend both these two fundamentals somewhat to implement cache coherence using a bus.

PCA page 274

i.e .SMPs

Page 28: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#28 lec # 10 Spring2006 5-4-2006

Bus-Snooping Cache Coherence Bus-Snooping Cache Coherence ProtocolsProtocols

Basic Idea:• Transactions on bus are visible to all processors.

• Processors or bus-watching (bus snoop) mechanisms can snoop (monitor) the bus and take action on relevant events (e.g. change state) to ensure data consistency among private caches and shared memory.

Basic Protocol Types:• Write-invalidate: Invalidate all remote copies of when a local cache block is

updated. • Write-update:

When a local cache block is updated, the new data block is broadcast to all caches containing a copy of the block updating them.

Page 29: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#29 lec # 10 Spring2006 5-4-2006

Write-invalidate & Write-update Coherence Write-invalidate & Write-update Coherence Protocols for Write-through CachesProtocols for Write-through Caches

• See handout• Figure 7.14 in Advanced Computer

Architecture: Parallelism, Scalability, Programmability, Kai Hwang.

Page 30: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#30 lec # 10 Spring2006 5-4-2006

Implementing Bus-Snooping ProtocolsImplementing Bus-Snooping Protocols• Cache controller now receives inputs from both sides:

1 Requests from local processor

2 Bus requests/responses from bus snooping mechanism .

• In either case, takes zero or more actions:– Possibly: Updates state, responds with data, generates new bus

transactions.

• Protocol is a distributed algorithm: Cooperating state machines.– Set of states, state transition diagram, actions.

• Granularity of coherence is typically a cache block– Like that of allocation in cache and transfer to/from cache.

– False sharing of a cache block may generate unnecessary coherence protocol actions over the bus.

PCA page 280

Change Block StateBus action

Page 31: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#31 lec # 10 Spring2006 5-4-2006

Coherence with Write-through CachesCoherence with Write-through Caches

• Key extensions to uniprocessor: snooping, invalidating/updating caches:• Invalidation- versus update-based protocols.

• Write propagation: even in invalidation case, later reads will see new value:• Invalidation causes miss on later access, and memory update via write-through.

I/O devicesMem

P1

$

Bus snoop

$

Pn

Cache-memorytransaction

Possible Action: Invalidate or update cache block in P1 if shared

(Write)

Snoop Snoop

Write throughto memory

Invalidate or update shared copies

Page 32: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#32 lec # 10 Spring2006 5-4-2006

Write-invalidate Bus-Snooping Protocol:Write-invalidate Bus-Snooping Protocol: For Write-Through CachesFor Write-Through Caches

The state of a cache block copy of local processor i can take one of two states (j represents a remote processor):Valid State:

• All processors can read (R(i), R(j)) safely.

• Local processor i can also write W(i)

• In this state after a successful read R(i) or write W(i)

Invalid State: not in cache or, • Block being invalidated.

• Block being replaced Z(i) or Z(j)

• When a remote processor writes W(j) to its cache copy, all other cache copies become invalidated.

– Bus write cycles are higher than bus read cycles due to request invalidations to remote caches.

TwoStates:

i = Local Processorj = Other (remote) processor

Page 33: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#33 lec # 10 Spring2006 5-4-2006

Write-invalidate Bus-Snooping ProtocolWrite-invalidate Bus-Snooping Protocol

For Write-Through CachesFor Write-Through CachesState Transition DiagramState Transition Diagram

W(i) = Write to block by processor i

W(j) = Write to block copy in cache j by processor j iR(i) = Read block by processor i.

R(j) = Read block copy in cache j by processor j iZ(i) = Replace block in cache .

Z(j) = Replace block copy in cache j i

InvalidInvalid ValidValid

R(i), W(i)

W(j), Z(i)

R(i)W(i)R(j)Z(j)

R(j)Z(j)Z(i)W(j)

i local processor j other processor

For a cache block in local processor iLocal write

Write by other processor j, invalidate

Page 34: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#34 lec # 10 Spring2006 5-4-2006

Write-invalidate Bus-Snooping ProtocolWrite-invalidate Bus-Snooping Protocol For Write-Through CachesFor Write-Through Caches

– Two states per block in each cache, as in uniprocessor.

• state of a block can be seen as p-vector (for all p processors).

– Hardware state bits associated with only blocks that are in the cache.

• other blocks can be seen as being in invalid (not-present) state in that cache

– Write will invalidate all other caches (no local change of state).

• can have multiple simultaneous readers of block,but write invalidates them.

I

V

PrRd/BusRd

PrRd/—

PrWr/BusWr

BusWr/—

Processor-initiated transactions

Bus-snooper-initiated transactions

PrWr/BusWr

Alternate State Transition DiagramAlternate State Transition DiagramV = Valid I = InvalidA/B means if A is observed B is generated.Processor Side Requests: read (PrRd) write (PrWr)Bus Side or snooper/cache controller Actions: bus read (BusRd) bus write (BusWr)

Snooper sensesa write by other processor to same block -> invalidate

A write by other processor to block

A read by this processor

A write by this processor

A read by this processor

i.e W(j)

PCA page 281

i.e R(i) i.e W(i)

i.e W(j)

p bits

Page 35: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#35 lec # 10 Spring2006 5-4-2006

Problems With Write-ThroughProblems With Write-Through• High bandwidth requirements:

– Every write from every processor goes to shared bus and memory.

– Consider 200MHz, 1 CPI processor, and 15% of the instructions are 8-byte stores.

– Each processor generates 30M stores or 240MB data per second.– 1GB/s bus can support only about 4 processors without

saturating.– Write-through especially is unpopular for SMPs.

• Write-back caches absorb most writes as cache hits:– Write hits don’t go on bus.– But now how do we ensure write propagation and serialization?

• Requires more sophisticated coherence protocols.

PCA page 283

Visible to all In correct order

Page 36: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#36 lec # 10 Spring2006 5-4-2006

Write-invalidate Bus-Snooping Protocol:Write-invalidate Bus-Snooping Protocol: For Write-Back CachesFor Write-Back Caches

• Corresponds to ownership protocol.• Valid state in write-through protocol is divided into two states (3 states total):

RW (read-write): (this processor i owns block) or Modified M• The only cache copy existing in the system; owned by the local processor.• Read (R(i)) and (W(i)) can be safely performed in this state.

RO (read-only): or Shared S• Multiple cache block copies exist in the system; owned by memory.• Reads ((R(i)), ((R(j)) can safely be performed in this state.

INV (invalid): I

• Entered when : Not in cache or, – A remote processor writes (W(j) to its cache copy. – A local processor replaces (Z(i) its own copy.

• A cache block is uniquely owned after a local write W(i)

• Before a block is modified, ownership for exclusive access is obtained by a read-only bus transaction broadcast to all caches and memory.

• If a modified remote block copy exists, memory is updated (forced write back), local copy is invalidated and ownership transferred to requesting cache.

ThreeStates:

i = Local Processorj = Other (remote) processor

i.e which processor owns the block

For a cache block in local processor i

Page 37: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#37 lec # 10 Spring2006 5-4-2006

Write-invalidate Bus-Snooping Protocol For Write-Back CachesFor Write-Back CachesState Transition Diagram

RW RO

INV

RW: Read-WriteRO: Read OnlyINV: Invalidated or not in cache

W(i) = Write to block by processor iW(j) = Write to block copy in cache j by processor j iR(i) = Read block by processor i.R(j) = Read block copy in cache j by processor j iZ(i) = Replace block in cache .Z(j) = Replace block copy in cache j i

W(j), Z(i)

R(i)R(j)Z(j)

R(i)W(i) Z(j)

R(j), Z(j), W(j), Z(i)

R(j)

W(i)

W(i)

R(i)W(j) Z(i)

i local processor j other processor

(Processor iowns block)

Owned bymemory

Force write back

Other processor writesinvalidate

For a cache block in local processor i

Page 38: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#38 lec # 10 Spring2006 5-4-2006

Basic MSI Write-Back Invalidate ProtocolBasic MSI Write-Back Invalidate Protocol• States:

– Invalid (I). – Shared (S): Shared unmodified copies exist.– Dirty or Modified (M): One only valid, other copies must be invalidated.

• Processor Events: – PrRd (read).– PrWr (write).

• Bus Transactions:– BusRd: Asks for copy with no intent to modify.– BusRdX: Asks for copy with intent to modify.– BusWB: Updates memory.

• Actions:– Update state, perform bus transaction, flush value onto

bus (forced write back).

MSI is similar to previous protocol just different representation

PCA page 293

ThreeStates:

MSIModified Shared Invalid

e.g write back to memory

Page 39: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#39 lec # 10 Spring2006 5-4-2006

Basic MSI Write-Back Invalidate ProtocolBasic MSI Write-Back Invalidate ProtocolState Transition DiagramState Transition Diagram

– Replacement changes state of two blocks: Outgoing and incoming.

PrRd/—

PrRd/—

PrWr/BusRdXBusRd/—

PrWr/—

S

M

I

BusRdX/Flush

BusRdX/—

BusRd/Flush

PrWr/BusRdX

PrRd/BusRd

M = Dirty or Modified, main memory is not up-to-date, owned by local processorS = Shared, main memory is up-to-date owned by main memoryI = Invalid

Processor Side Requests: read (PrRd) write (PrWr)Bus Side or snooper/cache controller Actions: Bus Read (BusRd) Bus Read Exclusive (BusRdX) bus write back (BusWB) Flush

ForcedWrite Back

Invalidate

Processor InitiatedBus-Snooper Initiated

Block ownedby local processor

Block ownedby main memory

PCA page 295

ThreeStates:

BusWB

Page 40: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#40 lec # 10 Spring2006 5-4-2006

MESI (4-state) Invalidation ProtocolMESI (4-state) Invalidation Protocol• Problem with MSI protocol:

– Reading and modifying data is 2 bus transactions, even if not sharing:

• e.g. even in sequential program.

• BusRd ( I-> S ) followed by BusRdX ( S -> M ).• Add exclusive state (E): Write locally without a bus transaction, but not

modified:– Main memory is up to date, so cache is not necessarily the owner.– Four States:

• Invalid (I).• Exclusive or exclusive-clean (E): Only this cache has a copy, but not

modified; main memory has same copy.• Shared (S): Two or more caches may have copies.• Modified (M): Dirty.

– I -> E on PrRd if no one else has copy.• Needs “shared” signal S on bus: wired-or line asserted in response to

BusRd.

i.e no other cache has a copy

Modified Exclusive Shared Invalid

S = 0 Not SharedS = 1 Shared

Page 41: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#41 lec # 10 Spring2006 5-4-2006

– BusRd(S) Means shared line asserted on BusRd transaction.

– Flush: If cache-to-cache sharing, only one cache flushes data.

PrWr/—

BusRd/Flush

PrRd/

BusRdX/Flush

PrWr/BusRdX

PrWr/—

PrRd/—

PrRd/—BusRd/Flush

E

M

I

S

PrRd

BusRd(S)

BusRdX/Flush

BusRdX/Flush

BusRd/Flush

PrWr/BusRdX

PrRd/BusRd (S)

MESI State Transition DiagramMESI State Transition Diagram

S = shared signal = 0 not shared = 1 shared

Four States:M = Modified or DirtyE = ExclusiveS = SharedI = Invalid Invalidate / Forced

Write Back

Not Shared

Shared

Page 42: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#42 lec # 10 Spring2006 5-4-2006

Invalidate Versus UpdateInvalidate Versus Update• Basic question of program behavior:

– Is a block written by one processor read by others before it is rewritten (i.e. written-back)?

• Invalidation:

– Yes => Readers will take a miss.

– No => Multiple writes without additional traffic.• Clears out copies that won’t be used again.

• Update:

– Yes => Readers will not miss if they had a copy previously.• Single bus transaction to update all copies.

– No => Multiple useless updates, even to dead copies.

• Need to look at program behavior and hardware complexity.

• In general, invalidation protocols are much more popular.

– Some systems provide both, or even hybrid protocols.

Page 43: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#43 lec # 10 Spring2006 5-4-2006

Update-Based Bus-Snooping ProtocolsUpdate-Based Bus-Snooping Protocols• A write operation updates values in other caches.

– New, update bus transaction.

• Advantages:– Other processors don’t miss on next access: reduced latency

• In invalidation protocols, they would miss and cause more transactions.

– Single bus transaction to update several caches can save bandwidth.

• Also, only the word written is transferred, not whole block

• Disadvantages:– Multiple writes by same processor cause multiple update

transactions.• In invalidation, first write gets exclusive ownership, others local

• Detailed tradeoffs more complex.

Page 44: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#44 lec # 10 Spring2006 5-4-2006

Dragon Write-back Update ProtocolDragon Write-back Update Protocol• 4 states:

– Exclusive-clean or exclusive (E): I and memory have this block.

– Shared clean (Sc): I, others, and maybe memory, but I’m not owner.

– Shared modified (Sm): I and others but not memory, and I’m the owner.• Sm and Sc can coexist in different caches, with only one Sm.

– Modified or dirty (D): I have this block and no one else, stale memory.

• No explicit invalid state (implied).– If in cache, cannot be invalid.– If not present in cache, can view as being in not-present or invalid state.

• New processor events: PrRdMiss, PrWrMiss.– Introduced to specify actions when block not present in cache.

• New bus transaction: BusUpd.– Broadcasts single word written on bus; updates other relevant caches.

PCA page 301

Fifth (Invalid) State Implied

Page 45: EECC756 - Shaaban #1 lec # 10 Spring2006 5-4-2006 Multiprocessors Cache Coherence in Bus-Based Shared Memory Multiprocessors Shared Memory Multiprocessors

EECC756 - ShaabanEECC756 - Shaaban#45 lec # 10 Spring2006 5-4-2006

Dragon State Transition DiagramDragon State Transition Diagram

E Sc

Sm M

PrW r/—PrRd/—

PrRd/—

PrRd/—

PrRdMiss/BusRd(S)PrRdMiss/BusRd(S)

PrW r/—

PrWrMiss/(BusRd(S); BusUpd) PrWrMiss/BusRd(S)

PrWr/BusUpd(S)

PrWr/BusUpd(S)

BusRd/—

BusRd/Flush

PrRd/— BusUpd/Update

BusUpd/Update

BusRd/Flush

PrWr/BusUpd(S)

PrWr/BusUpd(S)

S = shared signal = 0 not shared = 1 shared

Not shared

BusUpd = Broadcast word written on bus to update other caches

Four States:E = ExclusiveSc = Shared cleanSm = Shared modifiedM = Modified

S = Shared

S = Not Shared

Shared

Update others

This Processoris owner

I I

SharedNot shared

Shared

Shared

Not shared