THE COST OF CONDOR: MEASURING POWER USAGE OF SCIENTIFIC COMPUTATION USING THE DESKTOP FLEET

Supervisors:•Brian Davis•Sam Moskwa

Summer Scholar:•Monish Parajuli

What is Condor?

High Throughput Computing framework

Used to create a cluster of computers

Powerful and simple way running tasks

Commonly used to perform CPU cycle scavenging

Image form http://research.cs.wisc.edu/htcondor/index.htmlhttp://research.cs.wisc.edu/htcondor/description.html

Condor at CSIRO

Installed on 4000 desktops

Peak capacity 13 TFlop/sImagine doing

3.14159265359 * 2.7182818284 trillion times in 1 second

Current usage ~30,000 CPU hours

Purpose

http://www.livinggreener.gov.au/energy/energy-efficiency-home/use-appliances-efficiently

Switching off power can reduce up to 10% electricity use

Computers need to be left on to run condor

Measure, measure, measure Hardware

DellCPU (Windows based High Performance Computing cluster)

Optiplex 755 (Old computers with Allendale processors)

Optiplex 7170 (New computers with Ivy bridge processors)

Operations APSIM (Agricultural simulation program)

CPU Bound (Tasks limited by processor speed)

Memory Bound (Tasks limited by speed of memory)

Disk Intensive (Tasks reading/writing large amounts of data)

Optilex 755 Optiplex 7170 DellCPU0

50

100

150

200

250

300

350

400

Energy needed to complete one task (lower is better)

Power(mWh)

Optilex 755 Optiplex 7170 DellCPU0.62

0.64

0.66

0.68

0.7

0.72

0.74

0.76

0.78

0.8

Task completed by a core in one minute (higher is better)

Tasks/core/min

APSIM

CPU B

ound

Mem

ory

Bound

Disk

Inte

nsiv

eId

le Off05

1015202530354045

Average Power Usage Per Core

Optilex 755Optiplex 7170DellCPU

Tasks

Power(Watts)

What do those figures mean? It costs about $2500/year to leave

computers turned off!!

It costs an extra $4500/year to run condor during work hours

It costs $100,000/year to run condor all the time

But that is not much!!

CSIRO's Elec-tricity Cost for

2012

ASC Budget 2012

Cost of running Condor

24/7/365

0

5000000

10000000

15000000

20000000

25000000Cost comparison

Cost($)

Another comparison

Burnet Peak capacity: 13.64 TFlop/s Cost : $840,000

Condor Peak capacity: ~13 TFlop/s Cost: $0 well not exactly...

So throw away the supercomputers?

No! Condor is not suitable for all types of tasks Limited by the specification of the desktop

computers

Supercomputers are really efficient Perform more calculation per minute Consume less energy per task

Then why use condor?

Utilize compute capacity

Compliment the HPC

Future

Look at other HPCs (Burnet, Bragg, Cherax)

Look at tasks other than APSIM

Use the energy usage profile to predict energy cost

Conclusion

Conclusion

Cost of running condor is negligible compared to the cost of not being able to do science

Thank you!

Questions/comments?

Cost while leaving computers turned off =

Non-work hours * power while idle * cost of electricity * number of computers * workdays/year

=16 hours/day * 0.00133 kW * $0.0817/kWh * 4000 * 365 days/year=$2538.32/year

Extra cost to run condor during work hours =

Idle hours * cost of electricity * number of computers * power draw while working * workdays/year * utilization %

=2 hours/day * 0.0817/kWh * 4000 * 0.05765 kW * 365 days/year * 32.46%

=$4464.29/year

Extra cost to run condor all the time =Non work hours * cost of electricity * number of computers * power draw while working * workdays/year

+ Extra cost to run condor during work hours

=16 hours/day * $0.0817/kWh * 4000 * (0.05765 kW * 32.46% + 0.04632 * 67.54% )* 365 days/year

+ Extra cost to run condor during work hours=$95421.24/year + $4464.29/year=$99885.53/year

Calculations