The Computational Sprinting Game

Benjamin C. Lee

benjamin.c.lee@duke.edu

Songchun Fan, Andrew Hilton, Ziqiang Huang,Jose Joao, Alejandro Rico, Seyed Majid Zahedi

Acknowledgements: Semiconductor Research Corporation; National Science Foundation.

Computational Sprints

Enhance performance briefly with extra power

3 cores @1.2GHz → 12 cores @ 2.7GHz

Nor

mal

ized

Spe

edup

0

1

2

3

4

5

6

naive

decis

ion

grad

ientsv

mlin

ear

kmea

ns als

corre

lation

page

rank cc

trian

gle

Nor

mal

ized

Pow

er

0.0

0.5

1.0

1.5

naive

decis

ion

grad

ientsv

mlin

ear

kmea

ns als

corre

lation

page

rank cc

trian

gle

Ave

rage

Tem

pera

ture

(°C

)

0

10

20

30

40

50

Non−sprinting Sprinting

naive

decis

ion

grad

ientsv

mlin

ear

kmea

ns als

corre

lation

page

rank cc

trian

gle

Fan et al. “The computational sprinting game” [ASPLOS’16]

2 / 16

Sprint Management

When should processors sprint?

Which processors should sprint?

Phases and processors that benefit most

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●

●

● ●

●

0 5 10 15 20 25 30

56

78

Epoch

Util

ity fr

om S

prin

t

●

?

3 / 16

Sprint Management

Sprint consumes thermal headroom

Sprint constrains future action

Sprint requires recovery time

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●

●

● ●

●

0 5 10 15 20 25 30

56

78

Epoch

Util

ity fr

om S

prin

t

●

××

××

××

×××

4 / 16

DynaSprint

Software Run-Time

Predict workload’s sprint utility, cost

Track system’s thermal headroom

Decide whether to sprint

Hardware Support

Alloate resources (e.g., cache capacity)

Monitor performance, energy

Implement safety fallback

Huang et al. “DynaSprint: µarchitectural sprints with dynamic utility, thermal management” [MICRO’19]

5 / 16

DynaSprint

(1-2) Classify eventsi → phasei

(3-5) Predict phasei → phasei+1 → utilityi+1

(6-8) Control sprint if utilityi+1 > utilitythresh

Huang et al. “DynaSprint: µarchitectural sprints with dynamic utility, thermal management” [MICRO’19]

6 / 16

Phase Classifier

eventsi → phasei

Define signaturewith events

Cluster signatures todefine phase

Huang et al. “DynaSprint: µarchitectural sprints with dynamic utility, thermal management” [MICRO’19]

7 / 16

Phase Predictor

phasei → phasei+1

Track phase history

Use history topredict phase

Huang et al. “DynaSprint: µarchitectural sprints with dynamic utility, thermal management” [MICRO’19]

8 / 16

Utility Predictor

phasei+1 → utilityi+1

Use phase average

to predict utility

Huang et al. “DynaSprint: µarchitectural sprints with dynamic utility, thermal management” [MICRO’19]

9 / 16

Sprint Decision

Ui+1 > Umax ×Hconsumed/Htotal

Ui+1 Predicted utilityUmax Max utility in recent epochs

Hconsumed/Htotal Fraction of thermal headroom consumed

Huang et al. “DynaSprint: µarchitectural sprints with dynamic utility, thermal management” [MICRO’19]

10 / 16

Performance

Improve performance by 17% (average), 40% (max)

Outperform heuristics

Compete with oracles

Huang et al. “DynaSprint: µarchitectural sprints with dynamic utility, thermal management” [MICRO’19]

11 / 16

Dynamics

Sprint during high-utility epochs

Tune management policy dynamically

Huang et al. “DynaSprint: µarchitectural sprints with dynamic utility, thermal management” [MICRO’19]

12 / 16

Utility Distributions

Perform well for dynamic workloads

Sprint judiciously given bimodal speedups

Huang et al. “DynaSprint: µarchitectural sprints with dynamic utility, thermal management” [MICRO’19]

13 / 16

Prediction Accuracy

Predict utility from phase with low error

Track modest number of unique phases

Huang et al. “DynaSprint: µarchitectural sprints with dynamic utility, thermal management” [MICRO’19]

14 / 16

Conclusion

Dynamic Phase Analysis

Manage sprints online

Sprint with cache capacity

Algorithmic Game Theory

Arbitrate multi-programmed sprints

Decentralize power management

Future Directions

Coordinate multi-resource sprints

Define sprints dynamically

Huang, Joao, Rico, Hilton, Lee. “DynaSprint: Microarchitectural sprints with dynamic utility and thermalmanagement.” MICRO 2019.

Fan, Zahedi, Lee. “Distributed strategies for computational sprints.” CACM 2019.

Fan, Zahedi, Lee. “The computational sprinting game.” ASPLOS 2016.

15 / 16

The Computational Sprinting Game

Benjamin C. Lee

benjamin.c.lee@duke.edu

Songchun Fan, Andrew Hilton, Ziqiang Huang,Jose Joao, Alejandro Rico, Seyed Majid Zahedi

Acknowledgements: Semiconductor Research Corporation; National Science Foundation.