Real-Time Performance Analysis of Adaptive Link Rate

Baoke Zhang, Karthikeyan Sabhanatarajan, Ann Gordon-Ross*, Alan D. George*

This work was supported by the U.S. National Science Foundation

The Energy Efficient Internet ProjectHigh-performance Computing & Simulation Research Lab

ECE Department, University of Florida, Gainesville

* Also affiliated with NSF Center for High Performance Reconfigurable Computing

Joseph Antoon - Speaker

1

2

Introduction

INTERNET

Introduction

• Connected edge devices account for 2% of the total power consumed in the US [EPA-06]– 130 TWh/Year– 1 single-unit nuclear power

plant outputs 8 TWh/Year– Translates to 16 single-unit

nuclear power plants!

3

Introduction

• Ethernet links – As they grow faster…– They eat more power!

• But, this corresponds to data capacity, right?

1Gbps

+4W

100Mbps

10Gbps

+16W

4

Introduction

• Wrong!• Average traffic has

5% utilization

10Gbps

+16W

(Actually needed)

Wasted!1Gbps

+4W

100Mbps

5

13

57

911

1315

1719

2123

2527

2931

3335

3739

4143

4547

4951

5355

5759

6163

6567

6971

7375

7779

8183

8587

8991

9395

9799

101103

105107

109111

113115

117119

121123

125127

129131

133135

137139

141143

145147

149151

153155

157159

161163

165167

169171

173175

177179

181183

185187

189191

193195

197199

0

20

40

60

80

100

120 13

57

911

1315

1719

2123

2527

2931

3335

3739

4143

4547

4951

5355

5759

6163

6567

6971

7375

7779

8183

8587

8991

9395

9799

101103

105107

109111

113115

117119

121123

125127

129131

133135

137139

141143

145147

149151

153155

157159

161163

165167

169171

173175

177179

181183

185187

189191

193195

197199

0

20

40

60

80

100

120

Adaptive Link Rate

• Ethernet links rarely utilized

• Only need high capacity for short bursts

• Adaptive Link Rate– Use low link rate– During idle periods

Time

link utilization

link waste

6

Adaptive Link Rate

PHY

Tx Buffer

Rx Buffer

Netw

ork

MAC

Network Interface Card

100MUh oh, need

more throughput!

1G

7

• Can’t send packets while switching• Long switching time interrupts connections

• Result: Performance hit!

To MACFrom PC

Drawbacks

Tx Buffer

8

• Switching costs energy• Goal: Positive net energy consumption

• Result: Minimum time to stay in low rate!

Drawbacks

SwitchEnergy

SavedEnergy

Link Rate

9

Goals

• How long to switch?– Previous work assumes 1-10ms

• How much energy to switch• How long to stay in low rate?

• Solution: Build a prototype– Virtex II Pro Development Kit– RiceNIC-based architecture

Goals

10

Modified RiceNIC Architecture

CRC Addr Add

CRC

Flow Control

MIIM

PHYPCI Interface

MAC Core

FIFO

FIFO Tx Control

FIFO Rx Control

ControlPolicy

MAC Control

Unit

11

Determines whenTo change link rate

Coordinates link rate changeGenerates synthetic traffic

Experiment• MAC Handshake time

– How long to coordinate switch

• PHY Register Time– How long to trigger rate change

• PHY Resynchronization time– How long to physically reset link

PHYMAC

100M1G

12

Results – Average Times

1

10

100

1000

10000

100000

1000000

Handshake PHY Config PHY Resync

Tim

e (u

s) >700,000%

Longer

13

Results – PHY Resync Time

0

100

200

300

400

500

600

700

To 1Gbps To 100Mbps To 10Mbps

Tim

e (m

s)

14

Results – Switching Power

0

0.5

1

1.5

2

2.5

3

3.5

4

10M ↔ 100M 100M ↔ 1G 10M ↔ 1G 1Gpbs Link

Pow

er (W

)

Not too bad!

But what about energy?15

Results – Energy

• Why not just switch to 100M, then 10M?

0

0.5

1

1.5

2

2.5

10M ↔ 100M 100M ↔ 1G 10M ↔ 1G Step-down

Ener

gy (J

)

+ <

10M ↔ 1G costs so much when…

16

Minimum Time in Low Link Rate

0

1

2

3

4

5

6

10M ↔100M

100M ↔ 1G 10M ↔ 1G Step-down

Tim

e (s

)

Observed Previously Simulated

20.8

Assuming a switch time makes a big difference!

17

Conclusions

• First hardware prototype of an Adaptive Link Rate (ALR) enabled NIC

• Observed switching times much longer than previously assumed

• “Step down” method uses less energy than switching from 1Gbps than 10Mbps

18

Thank you for attending!

Any Questions?

19