KineticModelingofDataEvictioninCache

XiamengHu,XiaolinWang,LanZhou,Yingwei LuoPekingUniversity

ChenDingUniversityofRochester

ZhenlinWangMichiganTechnologicalUniversity

6/29/16 Usenix ATC'16 1

Background�MissRatioCurve(MRC)isapowerfulmetricforcacheoptimization:� Allocation,Partition,Scheduling,QoS managing…

� OnlineMRCprofilingtechniqueshavebeendevelopedfordecades.

� Ultimategoals:� Lessspaceconsumption.� Lowertimecomplexity.

6/29/16 Usenix ATC'16 2

0%

50%

100%MRC

Background

6/29/16 Usenix ATC'16 3

•AbriefhistoryofMRCtechniques.

Our Model: Average Eviction Time

6/29/16 Usenix ATC'16 4

•Linear time•Constant space•Composability

EvictionTime

6/29/16 Usenix ATC'16 5

d a b c

a d b c

c a d b

b c a d

e b c a

a d b c

2nd access

Eviction

d a b c

a b c e

1st access

a

d

a

d

a

c

b

e

data

Residence time

Eviction time

1

2

3

4

5

6

7

8

time

d e b cd93nd access

MRU LRU

EvictionTime

�Theevictiontimeisthetimebetweenthelastaccessandtheeviction.�Property of eviction time:

� Ifthereusetimeofanaccessislargerthanit’sevictiontime,it’samiss.

� Reusetime:thetimebetweenanaccessanditsnextreuse.Thereusetimeofcoldmissisdefinedasinfinite.

6/29/16 Usenix ATC'16 6

Backtotheexample

6/29/16 Usenix ATC'16 7

d a b c

a d b c

c a d b

b c a d

e b c a

a d b c

d a b c

a b c ea

d

a

d

a

c

b

e

data

Eviction time = 4

d e b cd

Hit!

Cold Miss!

Miss!

Reuse time = ∞

Reuse time = 2

Reuse time = 5

1

2

3

4

5

6

7

8

time

9

MRU LRU

AverageEvictionTime

�AverageEvictionTime(AET)isthemeanevictiontimeofalldataevictionsinafullyassociativeLRUcache.�WecanassumealldatareferenceswithareusetimelargerthanAETaremisses.

6/29/16 Usenix ATC'16 8

HowtomodelAET?� Movecondition#1:

� Cachehitinsertsthelower priority position datatotheLRUstack top.

� Movecondition#2:� Cachemissinsertsamissed datatotheLRUstack top.

6/29/16 Usenix ATC'16 9

d a b c

a d b c

d a b c

e d a b

access: a

access: e

MRU LRU

MRU LRU

HowtomodelAET?� Staycondition:

� Cachehitinsertsthehigher priority position datatotheLRUstack top.

6/29/16 Usenix ATC'16 10

a b d c

b a d caccess: b

MRU LRU

HowtomodelAET?�Wedefinethearrivaltime𝑇% astheexpected timeittakesforan evicting data toreachthem-th position(fromitslastaccess).

� Adatablockatposition𝑚moveonestepdownwheneverthereusetimeofcurrentaccessisgreaterthanthe𝑇%.

� 𝑃(𝑡) istheprobabilityforanaccesswithareusetimegreaterthan𝑡.

� Themovementconditionisnowaprobability.Everyaccess,adatablockatstackposition𝑚movesby𝑃(𝑇%) .

6/29/16 Usenix ATC'16 11

KineticModel� Datatravelsinonedirectionwithchangingspeed:

� Ingeneral,ifthetimethatevictingdataalreadytraveledis𝑡, its’currentevictingspeedis𝑃(𝑡).

6/29/16 Usenix ATC'16 12

top d bottom… …

𝑉(𝑡) = 𝑃(𝑡)

AverageEvictionTime� Physics:theintegrationofspeedovertimeistraveldistance.� ThelengthofLRU lististhetraveldistanceofeveryeviction.Whichisthecachesize𝑐.

1 𝑃 𝑡 𝑑𝑡 = 𝑐345(6)

7

� With𝑃,wecalculateAETsofdifferentcachesizesinlineartime.� 𝑃 canbeacquiredonlinebymonitoringthereusetimehistogram.

6/29/16 Usenix ATC'16 13

FromAETtoMRC

6/29/16 Usenix ATC'16 14

•Themissratio𝑚𝑟(𝑐)atcachesizec istheprobabilitythatareusetimeisgreaterthantheaverageevictiontime𝐴𝐸𝑇 𝑐 :

𝑚𝑟(𝑐) = 𝑃(𝐴𝐸𝑇(𝑐))

AET Design Overview

6/29/16 UsenixATC'15 15

ProgramMonitoring

AccessTrace

Reuse TimeHistogram AET Miss Ratio

Curve

RandomSampling

� Randomlypickcurrentaccesseddatatomonitoritsreusetime.

� Thedistancebetweentwosampledisarandomvalue.� Constraintherandomvaluerangetocontrolsamplingrate.� Ahashtableisrequired.Itmaintainscurrentmonitoreddata.� Thespaceconsumptionislinearbutlimited.

6/29/16 Usenix ATC'16 16

ReservoirSampling

� Toboundthespacecosttoconstant.𝑂(1)�Whenthe𝑖-th sampleddataarrives,reservoirsamplingkeepsthenewdatainmonitoringsetwithprobabilitymin(1, 𝑘/𝑖)andrandomlydiscardsanolddatawhenthesetisfull.

� Itensurestheequalprobabilityforeverysampledreusetoupdatereusetimehistogram.

�Whilethenumberofsamplesberecordedisbounded.

6/29/16 Usenix ATC'16 17

AETinSharedCache� Composability：co-runbehaviorcanbecomputedfromthemetricofsolo-runs.

�When𝑛 programssharethecacheofsize𝑐,all𝑛 + 1co-run𝐴𝐸𝑇𝑠,𝐴𝐸𝑇I(𝑐) foreachprogrami and𝐴𝐸𝑇(𝑐)forthegroup,arethesame:

� Detailedmodelingis describedinpaper.

6/29/16 Usenix ATC'16 18

Evaluation

�AET vs Counter Stacks (OSDI’14)�AET vs SHARDS (FAST’15)� Shared Cache AET

6/29/16 Usenix ATC'16 19

AETvsCounterStacks

� CounterStacks:� Onlyrequiresextremelysmallspacewhilemaintaininganacceptableaccuracy.

� HyperLogLog countertotrackreusedistance.� Balanceaccuracyandspacebylimitingthenumberofcounters.

� Benchmarks:� MicrosoftResearchCambridge(MSR)storagetraces.� Configuredwithonlyreadrequestsof4KBcacheblocks.

6/29/16 Usenix ATC'16 20

AETvsCounterStacks

6/29/16 Usenix ATC'16 21

AETvsCounterStacks

6/29/16 Usenix ATC'16 22

AETRandomSampling(𝟏 ∗ 𝟏𝟎M𝟓)

AETReservoirSampling8kentries

CounterStacksHighfidelity(d=1M,s=60,δ=0.02)

CounterStacksLowfidelity(d=1M,s=3600,δ=0.1)

MeanAbsoluteError

0.96% 1.12% 0.77% 1.26%

AverageSpace Cost

452KB 384KB 7363KB 1292KB

AverageThroughput

63.99Mreqs/sec

61.99Mreqs/sec

1.73Mreqs/sec

5.86Mreqs/sec

AETvsSHARDS� SHARDS:

� hash-basedspatialsampling� asplaytreetotrackthereusedistancesofthesampleddata.� Limitsthespaceoverheadtoaconstantbyadaptivelyloweringthesamplingrate.

� Benchmarks:� “master”MSR,whichisa2.4billion-accesstracecombiningall13MSRtracesbyrankingthetimestampsofallaccesses.

6/29/16 Usenix ATC'16 23

AETvsSHARDS

6/29/16 Usenix ATC'16 24

AETvsSHARDS

6/29/16 Usenix ATC'16 25

AETRandomSampling(𝟏 ∗ 𝟏𝟎M𝟓)

AETReservoirSampling8k samples

SHARDS8k samples

CounterStacks

MeanAbsoluteError

1% 1% 0.6% 0.3%

AverageSpace Cost

1.7MB 1.4MB 2.3MB 80MB

AverageThroughput

79Mreqs/sec

66.6Mreqs/sec

81.4Mreqs/sec

3.2Mreqs/sec

SharedCacheAET

�We choose Four MSRstoragetraces{prn,src2,web,stg}asaco-rungroup.

� Generateacombinedtracefromthefourtracesunderequalspeedassumption.

�WecompareMRCcomposedbyindividualAETmodelingofeachtrace,aswellastherealMRCofthecombinedtrace.

6/29/16 Usenix ATC'16 26

SharedCacheAET

6/29/16 Usenix ATC'16 27

Summary

6/29/16 Usenix ATC'16 28

•Anewmodeltocharacterizecachebehavior.• EnablefastMRCprofilingwithO(1)spaceandO(n)time.•PredictsharedcacheMRCwithoutco-runtesting.•Perfectforonlinedeploymentwithlimitedoverhead.

Thankyouforyourattention!

Q&AEmail:hxm@pku.edu.cn

6/29/16 Usenix ATC'16 29

AETvsStatStack� StatStack:

� DesignedforCPUworkloads.� Itsamplescacheblocksandmeasurestheirreusetimeusingperformancecountersandwatchpoints.

� Reusetimehistogram->Reusedistancehistogram.� Benchmarks:

� SPECCPU2006,30benchmarks.� Foreachbenchmark,weintercept1billionreferencesfromtheirexecutionusingtheinstrumentationtoolPin.

� Wemeasurethecumulativedistributionfunction(CDF)ofabsoluteerroroffull-traceStatStack,full-traceAET,samplingAET.

6/29/16 Usenix ATC'16 30

AETvsStatStack

6/29/16 Usenix ATC'16 31