Power Efficient Cache Coherence

Craig SaldanhaMikko Lipasti

Motivation Power consumption becoming a

serious design constraint. Market demand for faster and

more complex servers. Complex coherence protocol and

interconnect. fclock + Complexity = Pinterconnect

Motivation Traditional power saving

methodologies ineffective. Minimize number of transaction

packets. At the end-points: Jetty. At the source: Serial Snoop.

OUTLINE

Overview of Snoop based protocols and opportunity for power savings

Latency and Power consumption of parallel snooping techniques

Serial Snooping Results Conclusions Future Work

Snoop Based Coherence

Memory

Local Node Tag Lookup Bus Arbitration Snoop Transmission

Remote Node Tag Lookup Snoop Response Combination of Responses Data Fetch Data Transmit

Tag Array

Data Array

P1 P2 P3 P4

Degrees of Speculation 3 Degrees of Freedom to SpeculateSnooping Data Fetch Data Transmit

DFetch

D-Xmit

Parallel Snoop,Spec Dfetch Spec DXmit

Parallel Snoop,Spec Dfetch Non-Spec DXmit

Parallel Snoop,Non-Spec Dfetch Non-Spec DXmit

Serial Snoop,Spec Dfetch Spec DXmit

Serial Snoop,Spec Dfetch Non-Spec DXmit

Serial Snoop,Non-Spec Dfetch Non-Spec DXmitX

Latency and Power assumptions

MEMORY

Consider only load misses

Tree of point-point connections.

Latency to traverse a link: 1Bus cycle (7ns)

Tag Look up :1 bus cycle (7ns)

D-Fetch: 2 bus cycles (14ns)

DRAM access: 10 bus cycles (70ns)

P1 P2 P3 P4

Root Node

Switch1 Switch2

Board1

Backplane

Parallel Snoop Speculative Fetch Speculative Transmit (PS/SF/ST)

MEMORY

Snoop Broadcast

Latency

MEMORY

Snoop Broadcast

Latency

Plink+Pswitch

MEMORY

Snoop BroadcastLatency

2Plink+Pswitch

0 7 14 21

MEMORY

2Plink+2Pswitch

0 7 14 21 28

MEMORYSnoop BroadcastLatency

5Plink+2Pswitch

0 7 14 21 28 35

MEMORY

5Plink+4Pswitch

0 7 14 21 28 35 42

MEMORY

8Plink+4Pswitch

0 7 14 21 28 35 42 49

Memory Access :Data FetchLatency

35 91 105

MEMORY

Memory Access :Data TransmitLatency

Pmem+3Plink+2Pswitch

35 91 105 140

MEMORY

Remote Node: Tag LookupLatency

MEMORY

MEMORYRemote Node :Snoop Response Latency

3Ptag+3*(Pswitch+2Plink)+Plink+2Pswitch+2Plink

49 56 105

Remote Node: Data FetchLatency

3Pcache

MEMORY

MEMORYRemote Node :Data Transmit Latency

3Ptag+3*(4Plink+3Pswitch)

49 63 112

Local Node

0TL Snoop BRDCSTTL49

105 14091Data XmitMemory Access

35Memory

105 14091Data XmitMemory Access

49Remote Node11263

DF Data Xmit49

10556 RSP + CMB

49Remote Node

10556 RSP + CMB

Latency

Power29Plink+18Pswitch+3Ptag+3Pcache+PmemRemote Node supplies the data

Memory supplies the data20Plink+11Pswitch+3Ptag+3Pcache+Pmem

Parallel Snoop Speculative Fetch Non-Speculative Transmit (PS/SF/NT)

105 14091

Snoop BRDCST

Data XmitMemory Access35

Local Node

Remote Node

Memory

Remote Node

105 154

Data Xmit

Power21Plink+12Pswitch+3Ptag+Pcache+PmemRemote Node supplies the data

Memory supplies the data20Plink+11Pswitch+3Ptag+3Pcache+Pmem

Latency

Parallel Snoop Non-Speculative Fetch Non-Speculative Transmit (PS/NF/NT)

196161

Snoop BRDCST

Data XmitMemory Access91

Local Node

Remote Node

Memory

119 168

Remote Node

DF Data Xmit

Latency

Power21Plink+12Pswitch+3Ptag+PcacheRemote Node supplies the data

Memory supplies the data 20Plink+11Pswitch+3Ptag+Pmem

MEMORY

Serial Snooping

Avoids Speculative transmission of Snoop packets.

MEMORY

Serial Snooping

Avoids Speculative transmission of Snoop packets.

Check the nearest neighbor

Data supplied with minimum latency and power

MEMORY

Serial Snooping

Forward snoop to next level

MEMORY

Serial Snooping

Forward snoop to next level

MEMORY

Serial Snooping

Search other half of tree

MEMORY

Serial Snooping

Search leaf nodes serially

MEMORY

Serial Snooping

Search leaf nodes serially

Serial Snooping : Features Latency to satisfy a request dependent on

distance from requestor. Data resident at the nearest neighbor

supplied with the lowest latency and power. Requests visible to memory controller only

at root node. Latency is adversely affected when

requested data present at the farthest node Worst case power consumption is still less

than the parallel snooping.

MEMORY

Serial Snooping :

Request satisfied by Nearest NodeLatency

0 21 SNPTL

35 56DF Data Xmit

28 49 RSPTL

Xmit Snoop: 2Plink + Pswitch

P2 Tag access and snoop response: Ptag + 2Plink + Pswitch

P2 Data Fetch and Xmit: Pcache +2Plink + Pswitch

Ptotal= 6Plink+3Pswitch+Ptag+Pcache

MEMORY

Serial Snooping :

Request satisfied by Next-Nearest NeighborLatency

0 21 SNPTL

RSPTL21 63 77

77 84 133

77 91 140DF

TL Xmit

XmitMemory Access63 133 168

Memory

16Plink+10Pswitch+2Ptag+Pcache

MEMORY

Serial Snooping :

Request satisfied by farthest nodeLatency

XmitTL

28 49 RSPTL

21 63 77

77 84 10TL Xm

Spec-Memory

Non-Spec-Memory

XmitMemory Access 133 168

105 161

DF Xmit

105 168119

XmitMemory Access217 252

XmitMemory Access133 182147

18Plink+11Pswitch+3Ptag+PcacheRemote Node supplies the data

If Memory supplies the data 17Plink+10Pswitch+3Ptag+Pmem

RESULTS : Load Miss Distributions

Load Miss Distribution

NearestNeighbourNext NearestNodeFarthestNodeMemory

raytrace specwebspecjbb tpc-w

RESULTS: Average Latencies to satisfy load misses

Average Latencies to satisfy load misses

0 20 40 60 80 100 120 140 160 180 200

Avg. Latencies (ns)

SSNFNT

SSSFNT

SSSFST

PSNFNT

PSSFNT

PSSFST

raytrace

specjbb

btpc-w

RESULTS: Relative Power Savings

Factors contributing to power consumption

PS/SF/ST

SS/SF/ST

PS/SF/NT

SS/SF/NT

PS/NF/NT

SS/NF/NT

Plink Psw Ptag Pcache Pmem

CONCLUSIONS Reducing degree of speculation has

potential for significant power savings

Performance degradation is minimal for the set of benchmarks studied.

Serial Snooping with speculative memory fetch provides optimal latency and power consumption.

Future Work Develop detailed execution-driven

Power Model Explore different interconnect

topologies. Examine the viability of adaptive

mechanisms for protocol policy.

MEMORY

Serial Snooping

Nearest NeighborLatency

2Plink+Pswitch

0 7 14 21

Questions

Local Node

0TL Snoop BRDCSTTL49

MEMORY

Memory Access35

Memory Access35 91

Power Efficient Cache Coherence

Documents

CACHE COHERENCE DIRECTORIES FOR SCALABLE … · Directory-based protocols have been proposed as an efficient means of implementing cache coherence in large-scale shared-memory multiprocessors

Cache Coherence in Scalable Machines

Cache Coherence for GPU Architectures - Iowa State Universityclass.ece.iastate.edu/.../HPCA13GPUCachecoherence.pdf · Cache Coherence for GPU Architectures Inderpreet Singh1 Arrvindh

Database backed coherence cache

Cache Coherence CS433 Spring 2001

Lecture 3. Directory-based Cache Coherence

Efficient and Scalable Cache Coherence for Many-Core Chip

CoNDA: Efficient Cache Coherence Support for Near-Data ...€¦ · Application Analysis Analysis of Existing Coherence Mechanisms Architecture Support Evaluation CoNDA consistently

A Tagless Coherence Directorymoshovos/research/tagless...taining cache coherence in an efficient manner. Directory-based protocols avoid the bandwidth overhead of snoop-based protocols,

Dynamic Verification of Cache Coherence Protocols

05 Coherence de Cache

Source Snooping Cache Coherence Protocols - Par Labparlab.eecs.berkeley.edu/.../parlab/files/20091029-goodman-ssccp.pdfSource Snooping Cache Coherence Protocols ... – Supports all

Cache Coherence

Cache Coherence(Part 1)

Cache Coherence “Can we do a better job of supporting cache coherence ?”

Cache coherence problem and its solutions

Cache Coherence Techniques - Dipartimento di Informaticavannesch/SPA 2011-12/Silvia-cache-coherence-.pdf · Figure 1: The cache coherence problem 1 Cache Coherence In multiprocessor

Cache coherence - University of Pittsburghmelhem/courses/xx45p/coherence.pdf · 2 3 Approaches to cache coherence • Do not cache shared data • Do not cache writeable shared data

Power Efficient Cache Coherence Craig Saldanha Mikko Lipasti

Cache coherence, etc… - MIMD –