1

Decoupled Compressed Cache: Exploiting Spatial Locality for Energy-Optimized Compressed Caching

Somayeh Sardashti and David A. WoodUniversity of Wisconsin-Madison

2

3

Where does energy go?

Communication vs. Computation

Keckler Micro 2011

Improving cache utilization is critical for energy-efficiency!

~200X

Compressed Cache: Compress and Compact Blocks+ Higher effective cache size+ Small area overhead+ Higher system performance+ Lower system energy

Previous work limit compression effectiveness:- Limited number of tags- High internal fragmentation- Energy expensive re-compaction

6

Decoupled Compressed Cache (DCC)

Saving system energy by improving LLC utilization through cache compression.

Non-Contiguous Sub-Blocks

Previous work limit compression effectiveness:- Limited number of tags- High Internal Fragmentation- Energy expensive re-compaction

Decoupled Super-Blocks

7

Decoupled Compressed Cache (DCC)

Saving system energy by improving LLC utilization through cache compression.

Outperform 2X LLC1.08X LLC area14% higher performance12% lower energy

8

Outline Motivation Compressed caching Our Proposals: Decoupled compressed cache Experimental Results Conclusions

9

Uncompressed Caching

A fixed one-to-one tag/data mapping

Tags DataBAB

C

C A

10

Compressed Caching

Compress cache blocks.

Tags DataA A BB A B CC

Compact compressed blocks, to make room.Add more tags to increase effective capacity.

11

Compression

(1) Compression: how to compress blocks? • There are different compression algorithms.• Not the focus of this work. • But, which algorithm matters!

C64 bytes

C20 bytes

Compressor

12

Compression Potentials

High compression ratio potentially large normalized effective cache capacity.

1.5

2.8

3.9

Compression Ratio = Original Size / Compressed Size

Cycles to Decompress

Compression Algorithm

We use C-PACK+Z for the rest of the talk!

13

Compaction

(2) Compaction: how to store and find blocks?• Critical to achieve the compression potentials.• This work focuses on compaction.

Tags DataAB

Fixed Sized Compressed Cache (FixedC) [Kim’02, WMPI, Yang Micro 02]

Internal Fragmentation!

CBAC

14

Compaction

(2) Compaction: how to store and find blocks?

Tags DataAB

Variable Sized Compressed Cache (VSC) [Alameldeen, ISCA 2002]

C

A B CSub-block

D D

15

Previous Compressed Caches

(Limit 1) Limited Tag/Metadata– High Area Overhead by Adding 4X/more Tags

(Limit 2) Internal Fragmentation– Low Cache Capacity Utilization

A10B

A16B 2.6

2.3 2.01.7

Potential: 3.9

3.1

Normalized Effective Capacity = LLC Number of Valid Blocks / MAX Number of (Uncompressed) Blocks

16

(Limit 3) Energy-Expensive Re-Compaction

3X higher LLC dynamic energy!

Tags DataABC AD B DC

VSC requires energy-expensive re-compaction.

B

Update BB needs 2 sub-blocks

17

Outline Motivation Compressed caching Our Proposals: Decoupled compressed cache Experimental Results Conclusions

18

Decoupled Compressed Cache(1) Exploiting Spatial Locality

Low Area Overhead(2) Decoupling tag/data mapping

Eliminate energy expensive re-compactionReduce internal fragmentation

(3) Co-DCC: Dynamically co-compacting super-blocksFurther reduce internal fragmentation

19

(1) Exploiting Spatial Locality

Neighboring blocks co-reside in LLC.

89%

20

(1) Exploiting Spatial Locality

DCC tracks LLC blocks at Super-Block granularity.

E4X Tags

Tags DataABCD AB C D2X Tags

EQuad (Q): A, B, C, DSingleton (S): E

Super-Block Tag Q stat

e A

stat

e B

stat

e C

stat

e D

Super TagsQ S

Up to 4X blocks with low area overheads!

21

(2) Decoupling tag/data mappingDCC decouples mapping to eliminate re-compaction.

Quad (Q): A, B, C, DSingleton (S): E

Super TagsQ S AB D E

Quad (Q): A, B, C, DSingleton (S): E

Flexible Allocation

CB1 B2

Update B

22

(2) Decoupling tag/data mappingBack pointers identify the owner block of each sub-block.

Quad (Q): A, B, C, DSingleton (S): E

Super TagsQ S A C D E

Quad (Q): A, B, C, DSingleton (S): E

DataBack

Pointers

Tag ID Blk ID

B2B1

23

(3) Co-compacting super-blocks

Co-DCC dynamically co compacts super-blocks.‑ Reducing internal fragmentation

A B DCA sub-blockABCD

B

Quad (Q): A, B, C, D

24

Outline Motivation Compressed caching Our Proposals: Decoupled compressed cache Experimental Results Conclusions

25

Experimental Methodology Integrated DCC with AMD Bulldozer Cache.

– We model the timing and allocation constraints of sequential regions at LLC in detail.

– No need for an alignment network. Verilog implementation and synthesis of the tag

match and sub-block selection logic.– One additional cycle of latency due to sub-block selection.

26

Experimental Methodology Full-system simulation with a simulator based on GEMS. Wide range of applications with different level of cache

sensitivities:– Commercial workloads: apache, jbb, oltp, zeus– Spec-OMP: ammp, applu, equake, mgrid, wupwise– Parsec: blackscholes, canneal, freqmine– Spec 2006 mixes (m1-m8): bzip2, libquantum-bzip2, libquantum, gcc, astar-

bwaves, cactus-mcf-milc-bwaves, gcc-omnetpp-mcf-bwaves-lbm-milc-cactus-bzip, omnetpp-lbm

Cores Eight OOO cores, 3.2 GHzL1I$/L1D$ Private, 32-KB, 8-wayL2$ Private, 256-KB, 8-wayL3$ Shared, 8-MB, 16-way, 8 banksMain Memory 4GB, 16 Banks, 800 MHz bus frequency DDR3

Effective LLC Capacity

27

Components FixedC/VSC-2X DCC Co-DCC

Tag Array 6.3% 2.1% 11.3%

Back Pointer Array 0 4.4% 5.4%

(De-)Compressors 1.8% 1.8% 1.8%

Total Area Overhead 8.1% 8.3% 18.5%

1

2

1 2 3

Nor

mal

ized

LLC

Area

Baseline

2X Baseline

VSC DCCCo-DCC

FixedC

Normalized Effective LLC Capacity

Components FixedC/VSC-2X

Tag Array 6.3%

Back Pointer Array 0

(De-)Compressors 1.8%

Total Area Overhead 8.1%

Components FixedC/VSC-2X DCC

Tag Array 6.3% 2.1%

Back Pointer Array 0 4.4%

(De-)Compressors 1.8% 1.8%

Total Area Overhead 8.1% 8.3%

28

(Co-)DCC Performance

0.930.96 0.95

0.900.86

(Co-)DCC boost system performance significantly.

29

(Co-)DCC Energy Consumption

0.930.96 0.97

0.91 0.88

(Co-)DCC reduce system energy by reducing number of accesses to the main memory.

30

SummaryAnalyze the limits of compressed caching

• Limited number of tags• Internal fragmentation• Energy-expensive re-compaction

Decoupled Compressed Cache• Improving performance and energy of compressed caching• Decoupled super-blocks• Non-contiguous sub-blocks

Co-DCC further reduces internal fragmentation

Practical designs [details in the paper]

31

(De-)Compression overhead DCC data array organization with AMD Bulldozer DCC Timing DCC Lookup Applications Co-DCC design LLC effective capacity LLC miss rate Memory dynamic energy LLC dynamic energy

Backup

32

(De-)Compression Overhead

Parameters Compressor DecompressorPipeline Depth 6 2

Latency (cycles) 16 9

Area () 0.016 0.016

Power Consumption (mW) 25.84 19.01

33

DCC Data Array OrganizationAMD Bulldozer

B Ph

ase

Flop

A Ph

ase

Flop

A Ph

ase

Flop

B Ph

ase

Flop

A0: uncompressed; B1 and C2 are compressed to 2 sub-blocks

SR0SR 1SR 2SR3

A0.3C3.0

A0.2

B1.1

A0.1B1.0

A0.0C3.1

NSet Addr

4SR0 Addr

SR3 Addr

4

44

SR1 AddrSR2 Addr

Read Data

34

DCC Timing

35

DCC Lookup

1. Access Super Tags and Back Pointers in parallel2. Find the matched Back Pointers3. Read corresponding sub-blocks and decompress

Quad (Q): A, B, C, DSingleton (S): E

Super Tags

Data

AB EDBack Pointers

C1 C0

Read C

Q1

0 S

11

11

11

36

Applications

Spec2006 (m1-m8)

bzip2, libquantum-bzip2, libquantum, gcc, astar-bwaves, cactus-mcf-milc-bwaves, gcc-omnetpp-mcf-bwaves-lbm-milc-cactus-bzip, omnetpp-lbm

Sensitive to Cache Capacity and Latency

Sensitive to Cache Capacity

Cache Insensitive

Sensitive to Cache

Latency

37

Co-DCC Design

A2.1

A: <A2,A1,A0>A-ENDA1-Begin

Sub-block 7

…Sub-block 6 Sub-block 5

…Sub-blocks 4-2

A0.0

Sub-block 1 Sub-block 0

A0-Begin

A2.2A0.1A1A2.0

Tag ID Sh

arer

s

4b

Super-Block Tag Cs

tate

3Co

mp3

Csta

te2

Com

p2

Begi

n3

7b

Begi

n2Cs

tate

1Co

mp1

Csta

te0

Com

p0

Begi

n1

Begi

n0

END

7b3b 1b 1b7b3b1b 7b3b 1b 7b3b1b

38

LLC Effective Cache Capacity

1.0

1.5

2.0

2.5

3.0

3.5

4.0N

orm

LLC

Effec

tive

Capa

city

39

LLC Miss Rate

0.20

0.40

0.60

0.80

1.00N

orm

LLC

Miss

Rat

e

40

Memory Dynamic Energy

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Nor

m M

emor

y Dy

nam

ic E

nerg

y

41

LLC Dynamic Energy

0

1

2

3

4

5

6N

orm

LLC

Dyna

mic

Ene

rgy