Amoeba-Cache Adaptive Blocks for Eliminating Waste in the Memory Hierarchy

Snehasish KumarArrvindh ShriramanEric MatthewsLesley Shannon

Hongzhou ZhaoSandhya Dwarkadas

Amoeba Cache : Adaptive blocks for Eliminating Waste in the Memory Hierarchy

Fixed granularity cache organisation

Tag Array Data Array

Cache data utilization

Tags Data UntouchedData

Utilization = Fraction of words touched in cache block at the time of eviction

4Ser0%

100%64K L1 – 4 ways – 64B/block

h2 jbb

Cache utilization

18%5%4%

Block Distribution

# Words Touched

64K – 64B/block

58%20%

Block Distribution

# Words Touched

64K – 64B/block 1M – 64B/block

Application specific behaviour ― Inefficient data structure access

patterns

Interaction with cache geometry— Way conflicts reduce block lifetime

and cause poor utilization

Factors affecting cache utilization

Application Specific Behaviour

struct TIE {long long X, Y, Z;long long V, H;long long data[3];

} Imperial[1024];

Data[3]X Y HZ V

Access in a loop

Data Arrayfor (int i=0; i<1024; i++){

Imperial[i].X = …;Imperial[i].Y = …;Imperial[i].Z = …;Imperial[i].V = …;

Cache Geometry

Data Array – 4 ways

Problem : Lots of data map to same set

1. Shrinks effective cache space

2. Increases miss rate

3. Wastes on-chip bandwidth

4. Increases on-chip cache energy consumption

Implications

Miss Rate

Space Utilisation

Bandwidth

AmoebaCache

Target Metrics

Variable Granularity Blocks

How to support variable # of blocks / set ?

How to support variable granularity for each block?

Our Approach : Amoeba Cache

Unified SRAM Array

Amoeba Cache

• Insert• Lookup• Partial Miss• Overheads

SRAM Array

Region Tag Start End

1 word 1+ words

SRAM Array

Tag Data Block

Bitmaps

0000Valid? Tag?

0000 0000

Tag - Regions

Memory

Region

RMAXbytes

Region Tag ByteStart / EndSet Index

64 bit address

Example

} Imperial;

Imperial.X = … ;

Invoke Spatial Granularity Predictor(PC/Region based)

Tag X Y Z V

00000000

Valid? Tag?

Amoeba Cache – Insert (8words/set)

00000000SRAM Array / Set

Insert 4+1 words

00000 substring()

1Pos: 0

Tag X Y Z V

00000000Valid? Tag?

Amoeba Cache – Insert (8words/set)

00000000

SRAM Array / Set

11111000

Tag X Y Z V

Refill

210000000

Tag X Y Z V

Example

} Imperial;

Imperial.Y = … ;Lookup Data from the cacheData[3]X Y HZ VX Y Z V

Tag X Y Z V

Amoeba Cache – Lookup (8words/set)

RegionTag

Set Index

Word (W)

Tag X Y Z V

SRAM Array / Set

10000000

2x1 2x12x1 2x1

2 𝐴𝑑𝑑𝑟 ∈𝑇𝑎𝑔Region

==Start ≤ W

End > W Word SelectorHit?

Tag X Y Z VOutput Buffer

Partial MissIdentify Sub-Blocks Step 1 of 2

New ∩ Tags

MSHR 2 Evict Overlap

Fetch NewTag X Y Z V

Tag X Y Tag V H

Partial MissInsert New Block Step 2 of 2

Allocate 6 words

Miss 4

5Patch Missing ?’s

Occurs ≈ 5 in 1000 accesses

Tag X Y Z V H

X Y ? V HZ

Hardware Overheads

SRAM Array

Metadata

0000Valid? Tag?

0000 0000

0000 0000 Criti

ritica

Latency +4%

Evaluation

• Parameters for latency and energy• Workloads

Latency Parameters (cycles)

64K L1

1M LLC

1.04 Latency +4%

On-Chip Energy Parameters (pJ)

64K L1

1M LLC

≈ 7 / word

• 22 diverse workloads from• PARSEC• SPEC-CPU 2000 & 2006• DaCapo ( Java Benchmarks )• Apache, Firefox and PostgreSQL

Workloads

Results

% Improvement in L1 Miss-Rate

canneal

lbm h2 jbb

apache

firefoxtpcc

eclipse

Reduces L1 and L2 miss rate by 18%

% Improvement in L1 Miss-Bandwidth

Reduces on-chip bandwidth by 46% Reduces off-chip bandwidth by 38%

% Improvement in memory energy

canneal

lbm h2 jbb

apache

firefoxtpcc

eclipse

Reduces energy by 11%

% Improvement in execution time

canneal

lbm h2 jbb

apache

firefoxtpcc

eclipse

Improves performance by 10%

Results SummaryAmoeba-Cache

• Reduce cache pollution for applications with low cache utilization

• Improve performance for moderate cache utilization

• Maintain performance for high cache utilization workloads

• Save energy for streaming applications by keeping out unused words

Additional Results

Lookup as an extra cache pipeline stage vs. throttling the CPU

Spatial Granularity Predictor— Indexing— Training — Table Size

For extra pipeline stage, 8 of 22 applications show improvement

18 of 22 – Address region betterEvictions and First Touch

256 – PC and 1024 – Region

Additional Results

Multicore Shared Cache

Comparison against other designs— Fixed Granularity 2X— Sector Cache variants— Multi-$

Reduces miss rate (avg 18%) and LLC miss bandwidth (16%-39%)

Amoeba Cache

What? —Enable variable granularity data caching

Why?—Eliminate waste

How?—Unify tag and data into a single SRAM array

—Afforded by recent technology trendsWhere?

—Definitely at the L2, possibly at the L1

Frequently Asked Questions

1. Multiple threads?

2. Compare against other designs

3. Spatial Pattern Predictor

4. Replacement Policy

Multicore Shared Cache

Miss Miss Miss Miss BW

Mix T1 T2 T3 T4 (All)

jbb x2, tpc-c x2 12.38% 12.38% 22.29% 22.37% 39.07%

Firefox x2, x264 x2 3.82% 3.61% –2.44% 0.43% 15.71%

cactus, fluid., omnet., sopl. 1.01% 1.86% 22.38% 0.59% 18.62%

canneal, astar, ferret, milc 4.85% 2.75% 19.39% –4.07% 17.77%

Comparison

Impact on Miss-RateImpact on BandwidthLow tag overheadTradeoff data and tag spaceDynamically resize blocks

Amoeba Cache

Multi -$Sector Variants

YesYes~

~NoYesNoNo

Comparison – Moderate Group – 64K

1.0 1.1 1.2 1.3 1.4 1.5 1.60.4

Miss Rate Ratio

atio Sector

(x:2.9)

Sector-Pre

Fixed-2X

AmoebaMulti$-25

Multi$-50

Spatial Pattern Predictor

Index Pattern

PC / Region 01011111

Predictor History Table

PC : Read Addr 0 0 0 1 1 1 0 1

Critical Word

Policy Miss vs Policy-Bandwidth

What to do when there is no entry?

Predictor Training

Data Array

Index Pattern

Add / update entry on evict

Predictor – L1 Miss Rate (1 of 2)

10Aligned Finite Infinite Finite+FT History

Predictor – L1 Miss Rate (2 of 2)

mcf jbb020406080

100120140

Aligned Finite Infinite Finite+FT History

Predictor – L1 Miss Bandwidth (1 of 2)

Predictor – L1 Miss Bandwidth (2 of 2)

mcf jbb0

Predictor – Summary

For majority applications Region Predictor with

— 1024 entry table— Table with 8 ways x 128 sets

PC Predictor is good for 5 applications— apache, art, mcf, lbm and omnetpp

Pseudo LRU Replacement

• Logically partition the set into a Nways

• Pick a block at random from way• Unset the T? (Tag) and V? (Valid) bits

Way 0 Way 1

50apac

he art

ctus can

se fac

q. h2 jbb

1-2 Words 3-4 Words 5-6 Words 7-8 WordsW

45 20 39 79 30 80 77 82 49 62 55 38 40 32 29 81 33 21 53 73 29 46 50

Access Distribution for L1W

ributi

Amoeba block size distribution for L1Bl

ributi

he art

ctus can

se fac

q. h2 jbb

1-2 Words 3-4 Words 5-6 Words 7-8 Words%

92 80 98 100

67 98 88 99 78 100

94 82 89 89 93 100

83 91 91 97 70 91 90

L1 FSM

Miss-Rate ( 64K L1 )

canneal

lbm h2 jbb

apache

firefoxtpcc

eclipse

Amoeba

Miss Bandwidth Rate ( 64K L1 )

canneal

lbm h2 jbb

apache

firefoxtpcc

eclipse

10000Fixed

Amoeba

Energy Rate ( L1 + LLC ) – (nJ/KI)

canneal

lbm h2 jbb

apache

firefoxtpcc

eclipse

100Fixed

Amoeba

Reduction in execution time

Amoeba

Amoeba-Cache Adaptive Blocks for Eliminating Waste in the Memory Hierarchy

Documents

The Amoeba First-Kit / クラウドサービス

AMOEBA PROTEUS - RU Press

Life Processes - Amoeba

Roxie Amoeba

Obat Parasitic Anti Amoeba

Protozoans Lecture Amoeba

Amoeba-Cache: Adaptive Blocks for Eliminating Waste in the

Vtc Team Amoeba 6 10

Pathgenic free living amoeba

Free Living Amoeba 5

Amoeba & Water Children Book

AMOEBA: HIERARCHICAL CLUSTERING BASED ON · amoeba reproduces itself by dividing into two equal parts, a process known as binary fission. AMOEBA also reproduces itself by dividing

Mulay Saie - Amoeba

Amoeba Defense

Amoeba Server

Amoeba: dynamic management system - IMT...Amoeba and its introduction, Kamoshiga University •Naoki Hori (2018), The Amoeba management system, KCCS •Junichi Kawasaki (2017), My

Amoeba pure

AMOEBA – A DISTRIBUTED OPERATING SYSTEM

Case Study 1: AMOEBA. History of Amoeba Amoeba originated at the Vrije Universiteit, Amsterdam, The Netherlands in 1981 as a research project in distributed

Amoeba Sisters Answer Key