View
0
Download
0
Category
Preview:
Citation preview
ShiningaLightonSmallDataCentersintheU.S.
Authors:
MohanGaneshalingam,ArmanShehabi,Louis-BenoitDesroches
EnergyAnalysisandEnvironmentalImpactsDivisionLawrenceBerkeleyNationalLaboratory
June2017
ThisworkwassupportedbytheFederalEnergyManagementProgramoftheU.S.DepartmentofEnergyunderLawrenceBerkeleyNationalLaboratoryContractNo.DE-AC02-05CH1131.
LBNL-2001025
1
AbstractLargedatacentersarewellknownfortheirhigh-energyintensityandhavemadedramaticefficiencyimprovementsoverthepastdecade.Smallclosetandroomdatacentershavereceivedmuchlessattention,yetconstituteasignificantfractionofthetotalnumberofserversintheUnitedStates.Theoftenmakeshift,adhocnatureofsmalldatacentersoftenresultsinlittleattentionpaidtoenergyefficiencyandinadequatecoolingequipment.Thesmallphysicalfootprintofthesedatacenters,typicallyembeddedwithinalargerbuilding,makesitdifficulttoidentifyandtargetforefficiencymeasures.Theseconditionsmakesmalldatacentersnotoriouslyinefficientrelativetotheirlargercounterparts.Inthisreport,wepresentananalysisofsmallandmidsizedatacentersintheUS,drawingfromsurveysofcommercialbuildingstock.Wefindthatserversinsmalldatacentersmakeupapproximately40%ofinstalledserverstock,withthevastmajorityofsitesutilizingonly1-2servers.Weidentifyindustrieswheresmalldatacentersaremostprevalent,findingthatthehighestsaturationsareinmedical,retail,office,andeducationsectors.Smalldatacenterstypicallylackdedicatedcoolingequipment,oftenrelyingonbuildingairconditioningandventilationequipmentforcooling.Wefurtherfindthatthetypeofcoolingequipmentusedishighlycorrelatedwiththenumberofoperationalserverracks,withlessefficientcoolingoptionsusedwithfewerracks.WedevelopgeospatialmapsofsmallandmidsizedatacenterstovisuallyidentifyregionsofhighserverconcentrationandcalculateassociatedCO2emissions.Smalldatacentersconsume13billionkWhofenergyannually,emitting7millionmetrictons(MMT)ofcarbondioxide–theequivalentemissionsofapproximately2.3coal-firedplants.WediscussefficiencymeasuresthatcouldbeimplementedandestimatepotentialenergyandCO2savings.
2
Acknowledgements
TheauthorswouldliketothankHeidiFuchsandBlakeRingeisenforusefuldiscussionsregardingtheservermarket.RichardBrownprovidedhelpfulsuggestionsandfeedbackthatimprovedtheclarityofthispaper.
3
1. IntroductionThedigitizationofthemodernU.S.economyhasmadedatacentersessentialinfrastructureforcommercialbusinessesacrossallindustries.Moderndatacentersarepopulatedbycomputingequipment,whichoperatecontinuouslytosupporton-demandnetworkrequests.Theyrangeinsizefromindividualserversfoundinliteralclosetstoexpansivewarehousesfilledwiththousandsofservers.Datacentersofallsizesareusedtomanagecommunications,supportbusinessoperations,andcontrolaccesstodata.Attheheartofdatacenteroperationsareservers,networking,anddatastorageequipment.Serverstypicallyrundedicatedapplicationsandprocessrequestsreceivedviaadistributednetwork.Evenduringperiodsoflowutilization,thepowerdrawofaconventionalservermaybeasmuchas150-350W(Brownetal.,2008;Koomey,2011;Masanetetal.,2011;Shehabietal.,2016).Factoringinspaceconditioningandotherfacilityoverheadequipmentcandoubledatacenterelectricityconsumption(CenterofExpertiseforEnergyEfficiencyinDataCenters,n.d.).Consequently,buildingswithdatacentershaveahighenergyintensityrelativetoothertypesofcommercialbuildings(EIA,n.d.).Inaggregate,datacentersareestimatedtocurrentlyconsume70billionkWhintheU.S.alone,representingapproximately1.8%ofU.S.electricityconsumption(Shehabietal.,2016).Severalstudieshaveanalyzedtheevolvinglandscapeofdatacentersmostlyfocusingonlargestdatacentersthathousethousandsofservers(Brownetal.,2008;Koomey,2011;Masanetetal.,2011;Shehabietal.,2016).Inarecentstudy,Shehabietal.(2016)foundthatalthoughthereisoverallgrowthinthesizeandnumberofdatacentersovertime,thetotalenergyconsumptionattributedtodatacentershasremainedrelativelyconstantforthepastfewyears.Thisresultisaproductofconcertedeffortstooptimizeoperationsforthelargestdatacenters,whichareabletoreaplargeoperationalcostsavingsfromefficiencyimprovements.Technologycompaniesoperatinglargedatacentersbenefitfromeconomiesofscaleandhavetheresourcestoinvestinefficiencymeasures.Givenhighoperationalcostsoflargedatacenters,upfrontcapitalexpendituresonefficiencymeasuresarequicklyrecoupedbyoperatingcostsavings.Googlepresentsacasestudyinwhicha$25,000expenditureonefficiencymeasuresledtoannualsavingsof$67,000-apaybackof5months(Google,2011).Forthisparticulardatacenter,Googleestimatedayearlyenergysavingsof670MWh.Whenconsideringthelargenumberofdatacentersoperatedbysimilartechnologycompanies,undertakingefficiencymeasuresleadstomillionsofdollarsinsavings.Significantlylessattentionhasbeenpaidtosmall-scaledatacenters,typicallyreferredtoasdataclosetsanddatarooms(or“embedded”datacenters).Thesmallphysicalfootprintofthesedatacenters,typicallyembeddedwithinalargerbuilding,makesitdifficulttoidentifyandtargetforefficiencymeasures.Thesespacesaregenerallyrunbyinstitutionslessfamiliarwithinformationtechnologysystemsandbestpracticesfordatacentermanagement.Theoftenmakeshift,adhocnatureofsmalldatacentersresultsinlittleattentionpaidtoenergyefficiencyandinadequatecoolingequipment.Theseconditionsmakesmalldatacentersconsiderablylessefficientrelativetotheirlargecounterparts.
4
AstudyperformedbyCheungetal.(2014)surveyed30smallserverclosetsandroomswithfoursitesselectedfordetailedassessments.Theauthorsfoundthatmostsiteswerenotdesignedforefficientserveroperationandnotedmanyexamplesofpoorserverroommanagement.Forexample,inmanycasesconditionedcoolairwasinefficientlydirectedwithintheserverroomleadingtomixingwithexhaustedwarmair.ManyofthepotentialefficiencymeasuresoutlinedinCheungetal.rangefromno-tolow-cost.However,theauthorsnotebarriersimpedingmoreefficientoperationareorganizationalratherthantechnological.Feworganizationshadpoliciestopromoteefficiencyandmostlackedproperlytrainedstafftoresearchandimplementefficiencymeasures.BennettandDelforge(2012)performedasurveyof30businessesoperatingsmalldatacenterslookingspecificallyatthepenetrationofefficientoperatingpractices.Thesurveycovereddatacentersoperatingbetween1-30servers.Ingeneral,theyfoundalackofawarenessandorganizationalprioritizationofefficiencymeasures.Datacenteroperatorsweretypicallynotresponsibleforpayingenergybills,limitingmotivationforpursuingefficiency.SimilartothefindingsofCheungetal.,barrierstoefficiencywereorganizationalasopposedtotechnological.BennettandDelforgehighlighttheneedforlocalandstatepolicymeasurestocreateincentivestopromoteefficiencymeasuresspecificallyaimedtowardssmalldatacenters.Todate,nostudyhasanalyzedarepresentativesampleofcommercialbusinessesoperatingsmalldatacentersduetoadearthofdata.Thelackofdatahaslimitedtheabilityofpolicymakerstoeffectivelycraftpolicytoaddressenergywasteinsmalldatacenters.Inthisreport,weuserecentlyreleasedsurveydataofcommercialbuildingstocktoanalyzecharacteristicsofsmallandmidsizedatacenters.Weusetherecentlyreleased2012CommercialBuildingEnergyConsumptionSurvey(CBECS)administeredbythefederalEnergyInformationAdministration(EIA)toanalyzehowserversaredistributedgeographicallyandinwhattypesofbuildingstheyarelocated(EIA,2015).ThesedatarepresentanationallyrepresentativesurveydetailingenergyconsumptionpracticesofU.S.commercialbuildings.CombiningthesedatawithoccupationdatafromtheBureauofLaborStatistics(BLS)weconstructmapsofgeographiclocationofsmalldatacentersincommercialenterprises,andofserverlocationforsmallanddatacentersintheU.S.Additionally,weperformananalysisofthe2014CommercialBuildingStockAssessment(CBSA)administeredbytheNorthwestEnergyEfficiencyAlliance(NEEA).1AlthoughlimitedtodatacentersinthePacificNorthwestregion,thesurveyprovidesadetailedglimpseintothespacecoolingandservervirtualizationpracticeswithinsmalldatacenters.Thereportisorganizedasfollows.InSection2,wediscussthedatasourcesandmethodologyusedinouranalysisofdatacenters.Althoughthefocusofthisreportissmalldatacenters,wealsoincludeanalysisofmidsizedatacenters,whichfallinthegapbetweensmalldataclosetsandroomsandlargehyper-scaledatacenters.InSection3,wediscusstheresultsofouranalysisoftheCBECSandCBSAsurveys.InSection4,wepresentgeospatialmapsofserverintensityofsmallandmidsizedatacenters.WethendiscussaggregateenergyandCO2emissionsfromsmalldatacenterconsumptionandpotential1http://neea.org/resource-center/regional-data-resources
5
energysavingsfromefficiencymeasures.Section5providesasummaryofresultspresentedinthisreport.
2. DataandMethodsInthissectionwedescribethedatausedinouranalysisofsmallandmidsizedatacenters,aswellasthemethodologytoestimatenationalenergyconsumptionandCO2emissions.WewillusetheCBECSandCBSAstudiesofcommercialbuildingstocktocharacterizevariousaspectsofthesmallandmidsizedatacentermarket.WewillalsoutilizethenationallyrepresentativeCBECSdatatoestimatenationalenergyuseandCO2emissionsfromsmallandmidsizedatacenters.WestartwithU.S.commercialbuildingstocksurvey,estimatethenumberofservers,anddisaggregategeographicallybasedonregionalemploymentstatistics.
CommercialBuildingEnergyConsumptionSurvey2012CBECSisanationallyrepresentativesurveyofcommercialbuildingsintheUnitedStatesconductedbytheEIA.Thesurveyprovidesasnapshotofenergy-relatedbuildingcharacteristicsofU.S.commercialbuildingstock.CBECSdefinescommercialbuildingsasthosethatarenotprimarily(i.e.,>50%offloorspace)usedforresidential,industrial,oragriculturalpurposes.ForCBECS2012,EIAsurveyed6,720buildingsandweightedtheirsampletobenationallyrepresentativeofcommercialbuildingstock(EIA,2015).Thepubliclyreleasedmicrodatausedforthisanalysisisanonymizedtoremoveanycharacteristicsthatcouldpossiblybeusedtoidentifyindividualbuildings.Aspartofthisprocess,thelocationofbuildingsisonlymadeavailableattheCensusdivisionlevel(groupsof4-9states).
In2012,EIAincludedsurveyquestionstospecificallytargetserverusageincommercialbuildings.Whenquestioningrespondents,CBECSdefinedserversas“usuallyjusttheCPU,or‘case,’portionofacomputerthatmanagesnetworkresourcessuchascomputerfiles,printers,databases,ornetworktraffic;serversdonotrequiremuchhumanoperation,somostdonothavekeyboardsormonitors.”
Buildingswith500ormoreserversarecodedinCBECSwith‘9,995’inplaceoftheactualnumberofservers.Thesebuildingscorrespondto26recordsrepresentingapproximately2,000buildings.Thesesitesmayrepresentlargeenterpriseand‘hyper-scale’datacentersoutsidethescopeofthisreportandomittedfromouranalysis.However,theyarenotclearlyidentifiedasdatacentersbyactivityinthedataset.ItisalsopossiblethatthesedatacentersareclassifiedasindustrialfacilitiesoutsidethescopeofCBECS.
CBECScharacterizestheprincipalbuildingactivity(PBA)ofeachsampledbuildingintocategoriestogroupbuildingswithsimilarenergyconsumptionpatterns.Activitiescoverabroadrangeofcategoriesfrom‘Education’to‘WarehouseandStorage’.2,3AlthoughCBECS
2Fordefinitionsseehttp://www.eia.gov/consumption/commercial/building-type-definitions.cfm3CBECSalsoprovidesanadditionalmoredetailedcharacterizationofthebuildingactivitycodedasPBAPLUS.Forexample,the‘Education’PBAisfurthersubdividedinto‘elementary/middleschool’,‘highschool’,
6
includesaPBAfor‘DataCenters’(separatefromaquestionofwhetherthereisadatacenterorserverfarminthebuilding),therearenobuildingscategorizedassuchinthedataset.
Forthiswork,weadoptaclassificationsystemofeitherasmallormidsizedatacenterforeachbuildinginCBECSwithatleastoneserverbasedonthenumberofreportedservers.OurclassificationisrootedinthedatacentertaxonomydefinedbymarketreportsreleasedbytheInternationalDataCorporation(IDC).IDCcategorizesdatacentersinto5types,rangingfromsmallclosets(1-4servers)toenterprise-level,high-enddatacenters(>500servers).Indescriptionsofeachspace,IDCreportsthatbothclosetandroomdatacentersarehousedinrelativelysmallspaces(<500squarefeet),oftenlackdedicatedspacecooling(Baileyetal.,2007),andareusedprimarilybysmallbusinesses.Forourpaper,wedefineasmalldatacenterashavingbetween1-25servers,capturingbothclosetandroomservers.Allotherdatacentershavingmorethan25servers,butlessthan500areclassifiedasamidsizedatacenter.Table1providestypicalpropertiesforIDC’sdatacentertaxonomydatacentersandtherelationshiptoourclassification.Table1:Datacentercharacteristics
ClassificationThisWork
TaxonomyIDC
SpaceDimensions(squarefeet)
TypicalNumberofServers
SmallDataCenter
Closet <=100 1-4Room 101-1000 5-25
MidsizeDatacenter
LocalizedDatacenter 1,001-2000 26-100
Mid-tierDatacenter 2,001-20,000 101-499
NotAnalyzed High-endDatacenter >20,000 >500
CBECSreportsdatacentersquarefootageforthosebuildingswhererespondentsidentifiedadedicatedspaceforservers.Ideally,thesedatacanbeusedtoinformourserverspacecategorization.However,wefindthesedatanotusefulforcategorizingserverspaceforallbuildingsthatreporthavingservers.Forexample,98%ofrespondentswith5orlessserversdidnotidentifyhavingadatacenterandthusdidnotreportadatacentersquarefootage.Respondentswhoidentifiedadatacenterintheirbuildingappearbiasedtolargerspacetypes.Basedonthelackofreportingforallbuildingswithservers,wedonotusethedatacentersquarefootageestimatesfromCBECSinourcategorizationofserverspacetype.
‘university/college’,and‘othereducation’.Ingeneral,ouranalysisdoesnotbenefitgreatlyfromtheextradetailprovidedbythesemorespecificPBAs.Ingeneral,wewillpresentresultsintermsofPBA.However,whennoteworthy,wehighlightinterestingtrendsbythemorespecificbuildingactivitywithindiscussionofresults.
7
ItisunclearhowCBECSreportsbuildingswithmultipleseparatedatacenters.WeassumethatCBECSreportsthetotalnumberofserversinthebuildingcorrectly,butdoesnotdifferentiatebetweenmultipledatacenters.Althoughthisaddsalevelofuncertaintytoourestimatesofthenumberofdatacenters,wedonotexpectthistogreatlyimpactourdatacenterclassificationofbuildingsgiventhebroadcategorizationusedinourclassification.ItispossiblethatCBECSresultsmaysufferfromvariousformsofreportingbiasdependingonhowrespondentschosetoestimatethenumberofserversintheirbuilding.RespondentsansweringquestionsinCBECSarenotnecessarilytheindividualsmaintainingITequipment.CBECSreliesonestimatesfromrespondentsandresponsesarenotverifiedindependently.Inparticular,ourdatacentercategorizationsassumethattherespondentcan(1)positivelyidentifyserversand(2)provideareasonablyaccurateestimateofthenumberofserverson-premise.Thereislittlewecandotowardscharacterizinguncertaintiesduetothefirstassumption.Thesurveycouldpotentiallyunder-orover-reportthenumberofserversdependingoneachrespondent’sabilitytorecognizeandquantifyservers.Forthesecondassumption,ourclassificationofdatacentersaseithersmall,midsize,orhigh-endde-emphasizestheneedforexactestimatesforclassification.Forexample,arespondentmaypotentiallyconfuse5serversfor7or8servers,butisunlikelytoconfuse5serversformorethan25.
CommercialBuildingStockAssessment2014The2014CBSAsurveywasconductedbyNEEAinordertoprovideasnapshotofenergy-consumingdevicesfoundinregionalcommercialsites(NavigantConsulting,2014;NorthwestEnergyEfficiencyAlliance,2014).ThesurveyisspecifictocommercialbuildingsinthePacificNorthwestcoveringOregon,Washington,Idaho,andMontana.Detailedauditsofcommercialstockweregatheredfor859randomlyselectedcommercialsitesacrosstwelvebuildingtypes.Selectedsitescoveredbothurbanandruralcommerciallocations.DatawereanonymizedtoavoididentificationofindividualsitesandweightedtoberepresentativeofthePacificNorthwest.AlthoughlimitedtodatacentersinthePacificNorthwestregion,thesurveyprovidesadetailedglimpseintothespacecoolingandservervirtualizationpracticeswithinsmalldatacenters.CBSA2014includedmanyquestionsspecifictodatacentercharacteristicsandoperation.UnlikeCBECS,whichprovidesserverdataforeachbuilding,CBSAdataareprovidedatthedatacenterlevel,allowingforasinglecommercialsitetoreportmultipledatacenters.Foreachdatacenter,thesurveyrecordedthetotalnumberofracksintheserverroom,datacenterfloorspace,andtypeofspacecooling.Althoughtherewereadditionalquestionsregardingthepresenceofuninterruptiblepowersupplies(UPS),mostresponseswereleftblank.Thesurveydidnotincluderackdimensionsorthenumberofserversineachdatacenter,makingdirectcomparisonstoCBECSdifficult.NEEAspecificallysurveyedheating,ventilation,andairconditioning(HVAC)characteristicsforeachsite.Onsitesurveyorsrecordedthepresenceofdedicateddatacentercoolingandthetypeofconditioningprovided(e.g.,computerroomAC,buildingtransfer,waterchillers,etc.),evenifconditionedairwassharedwithspaceoutsideofthedatacenter.
8
Datacollectionforhospitalsanduniversitiesfollowedadifferentsurveyprotocolthanotherbuildingtypesduetothecomplexitiesinrecordingresultsforalargecampus.Forhospitalsanduniversities,CBSArecordsthedatacentersquarefootage,butdoesnotrecordthenumberofracks.Additionaldatafor521buildingswascollectedforthe‘CBSAOversampleStudy’(NorthwestEnergyEfficiencyAlliance,2014).Thesedatawerecollectedasautility-orstakeholder-fundedextensionoftheoriginalsurvey.Althoughthesedatarepresentasignificantadditionofthedataset,NEEArecommendsuseoftheCoredataset,presumablytoavoidintroducingbiasfromtheadditionoftargetedcommercialsites.OuranalysismakesuseoftheCoredataset.
EnergyCalculationsThemostcommonindustrymetricforquantifyingoveralldatacenterenergyefficiencyismeasurementofthePowerUsageEffectiveness(PUE)coefficient(TheGreenGrid,2012).PUEisdefinedastheratioofthetotalenergyusedfordatacenteroperationstotheelectricitydrawnbyITequipment:
𝑃𝑈𝐸 = 𝐸!"#"$%&#%'
𝐸!"
PUEcapturestheoverheadenergyrequiredtomaintaindatacenteroperationsincludingspacecooling,lighting,electricitydistribution,etc.Forexample,aPUEof2indicatesthatforeverykilowatt-hourdrawnbyITequipment,anotherkilowatt-hourisrequiredtomaintainserveroperations.TheenergydrawofITequipmentcanbeexpressedasthesumoftheenergyrequiredtopowerservers,network,anddatastorageequipment
𝐸!" = 𝐸!"#$"#! + 𝐸!"#$%&' + 𝐸!"#$%&' Thetotaldatacenterenergyconsumptioncanbeexpressedas
𝐸!"#"$%&#%' = 𝐸!" × 𝑃𝑈𝐸 𝐸!"#"$%&#%' = 𝐸!"#$"#! + 𝐸!"#$%&' + 𝐸!"#$%&' ×𝑃𝑈𝐸
Bytheverynatureofdatacenteroperations,serversoperatecontinuouslyinanactivemode.Serverpowerdrawdependsonutilization.Studieshavefoundthatserversrarelyoperateathighlevelsofutilization(Brownetal.,2008;WSPEnvironment&EnergyandNaturalResourcesDefenseCouncil,2012).Shehabietal.(2016)foundthatactiveserversinsmalldatacenterstypicallyoperateataround10-15%utilizationassuming10%ofserversare“zombie”serverswhichareconnectedtopowerbutneverutilized(KoomeyandTaylor,2015).Weassumethatatypicalserverfoundinasmallormidsizedatacenterconsumes,onaverage,approximately180W(Shehabietal.,2016),basedonthepower-
9
scalingabilityandcomputationalprocessorcountgenerallyfoundinthesedatacentertypes.FollowingMasanetetal.(2011),weuseatop-downapproachtoestimateenergyconsumptionfromnetworkandstoragedevices.Weassumenetworkandexternalstorageenergyconsumptionisproportionaltotheenergyconsumptionofserverhardware.Weassumenoexternalstorageinsmalldatacentersand20%formidsizedatacentersbasedontheircalculationsforthenumberofexternalharddrivesderivedfromshipmentsinBaileyetal.(2007).Ontopoftheserverenergyconsumption,weattributeanadditional5%ofserverenergyconsumptiontonetworkequipmentinsmalldatacentersand10%ofserverenergyconsumptiontonetworkequipmentinmidsizedatacenters.PreviousworkhasfoundthatsmallerdatacentersaremorelikelytoexhibithigherPUEvaluesduetopoorspacecooling(Cheungetal.,2014;Shehabietal.,2016).DetailedonsitecasestudiesofsmalldatacentersperformedbyCheungetal.(2014)foundthatdataroomsandclosetswereoftenfashionedoutofrepurposedspacenotintendedforservers.Manyclosetsandroomsstartedastemporaryhousingforsmall-scaleserverneedswhichincrementallyexpandedovertimetoadedicatedserverspace.ThelackofattentionpaidtoproperspacecoolingledtowarmairexhaustfromITequipmentmixingwithcooledairfromHVACequipment.Insomecases,theauthorsnotedthatservercoolingwasprovidedbybuildingcoolingequipment,theimpactofwhichisreducedcoolingefficiency.DetailedassessmentsoffoursmalldataspacesfoundPUEsrangingfrom1.5-2.1,withtwodatacentershavingPUEsof2.1.Shehabietal.(2016)reportvaluesof2.5forclosetdatacentersand2.0forroomandlocalizeddatacenters.Forcomparison,verylargedatacenteroperatorsreportedPUEsaslowas1.1throughconcertedefficiencyefforts(Google,2016).Forthiswork,weadoptatypicalPUEvalueof2.1forsmallandmidsizedatacenters.
OccupationalEmploymentStatisticsCBECSprovidesgeographicdataattheresolutionofCensusdivision(i.e.,groupsof4-9states)allowingustoquantifyserverownershipasafunctionofcensusdivisionandPBA.Toproducehigherspatialresolutions,wemaketheassumptionthatserversarespatiallydistributedproportionallytoworkerswhowouldworkinabuildingwithaserver.Forthispurpose,wemakeuseofoccupationalemploymentdata,availableatahighergeographicresolution,toapproximatethenumberofserversbyU.S.zipcode.
WeutilizeOccupationEmploymentStatistics(OES)releasedinMay2014fromtheBureauofLaborStatistics(BLS,2015).Thesurveyisconductedsemi-annuallyviamailanddesignedtocollectemploymentandwageestimatesfor821occupations.Thedataareprovidedforover650geographicalareasthatareacombinationofmetropolitanstatisticalarea(MSA)andnon-metropolitanareas.TheU.S.OfficeofManagementandBudgetdefinesMSAsashavingarelativelyhighpopulationdensitywithcloseeconomictiesthroughthearea.MSAsgenerallyincludealargecityandassociatedsurroundingareas,oftencoveringmultiplecounties.RegionswithineachstatethatdonotfallintoanMSAaredefinedas“nonmetropolitanareas”.Althoughastatemayhavemultiplenon-metropolitanareas,wecombineemploymentstatisticsforallnon-metropolitanareaswithinastate.
10
TheBLScollectsdataonthenumberofemployedpersonsin821detailedoccupationsthataregroupedinto22majoroccupationcategoriesforeachregionalunit.WeusethemajoroccupationcategoriesprovidedbyBLSandidentifyCBECSPBAswherethoseoccupationsaremostlikelytobefound.ForsomeoccupationsthereisadirectcorrespondencebetweenoccupationandPBA.Forexample,“LegalOccupations”canbemappeddirectlyto“Office”buildings.However,amajorityofoccupationsinOEShaveaone-to-manymappingofoccupationtoPBAs.Insuchcases,workersforthatoccupation,inagivenMSA/non-metropolitanarea,aresplitproportionallyaccordingthenumberofworkersinthoseCBECSPBAsforthecensusdivisioncorrespondingtothearea.Table2showsourmappingofoccupationscodestoPBA.Table2:MappingofOESjobcodestoCBECSPBA
OESJobCode CBECSPBAManagementOccupations OfficeBusinessandFinancialOperationsOccupations OfficeComputerandMathematicalOccupations Office,LaboratoryArchitectureandEngineeringOccupations Office,LaboratoryLife,Physical,andSocialScienceOccupations Office,LaboratoryCommunityandSocialServiceOccupations Office,ReligiousworshipLegalOccupations OfficeEducation,Training,andLibraryOccupations Education,PublicassemblyArts,Design,Entertainment,Sports,andMediaOccupations Publicassembly
HealthcarePractitionersandTechnicalOccupations
Outpatienthealthcare,Inpatienthealthcare,Nursing
HealthcareSupportOccupations Outpatienthealthcare,Inpatienthealthcare,Nursing
ProtectiveServiceOccupations PublicorderandsafetyFoodPreparationandServingRelatedOccupations Foodservice
BuildingandGroundsCleaningandMaintenanceOccupations Office,Laboratory,Lodging
PersonalCareandServiceOccupations Service
SalesandRelatedOccupationsFoodsales,Retailotherthanmall,Enclosedmall,Stripshoppingmall,Foodservice
OfficeandAdministrativeSupportOccupations Office,LodgingFarming,Fishing,andForestryOccupations AgriculturalConstructionandExtractionOccupations IndustrialInstallation,Maintenance,andRepairOccupations Other
ProductionOccupations Non-refrigeratedwarehouse,Refrigeratedwarehouse,Other
11
TransportationandMaterialMovingOccupations Non-refrigeratedwarehouse,Other
Wefirstdeterminethenumberofworkersinagivenmetropolitan/non-metropolitanarea,𝑀𝑆𝐴! ,andinagivenPBA,𝑃𝐵𝐴! ,whichwedenoteas𝑁! 𝑀𝑆𝐴! ,𝑃𝐵𝐴! :
𝑁! 𝑀𝑆𝐴! ,𝑃𝐵𝐴! = 𝑇(𝑂𝐶𝐶! → 𝑃𝐵𝐴!) ×𝑁!(𝑀𝑆𝐴! ,𝑂𝐶𝐶!)Inthisequation,weapplyourmappingofoccupationstoPBA,𝑇(𝑂𝐶𝐶! → 𝑃𝐵𝐴!),to𝑁!(𝑀𝑆𝐴! ,𝑂𝐶𝐶!)whichisthenumberofworkersin𝑀𝑆𝐴! ineachoccupationcode,𝑂𝐶𝐶! .𝑇(𝑂𝐶𝐶! → 𝑃𝐵𝐴!)isamatrixof0sand1swhichdescribeshoweachoccupationcodeisrelatedtoeachPBA.Nextwecalculatetheparameter𝜂,whichisthefractionofworkersinagiven𝑀𝑆𝐴! and𝑃𝐵𝐴! relativetothetotalnumberofworkersassociatedwith𝑃𝐵𝐴! inthecensusdivisionthatcontains𝑀𝑆𝐴! ,𝐶𝐷𝐼𝑉!"!!:
𝜂 𝑀𝑆𝐴! ,𝑃𝐵𝐴! = 𝑁!(𝑀𝑆𝐴! ,𝑃𝐵𝐴!)
𝑁!(𝑀𝑆𝐴! ,𝑃𝐵𝐴!)!"!!∈!"#!!"!!
FromCBECS,wehavethenumberofserversisagivencensusdivisionandPBA.Togetthenumberofserversinagiven𝑀𝑆𝐴! and𝑃𝐵𝐴! ,wemultiplythenumberofserversfoundinthe𝐶𝐷𝐼𝑉!"!! and𝑃𝐵𝐴! by𝜂:
𝑁! 𝑀𝑆𝐴! ,𝑃𝐵𝐴! = 𝑁! 𝐶𝐷𝐼𝑉!"!! ,𝑃𝐵𝐴! ×𝜂(𝑀𝑆𝐴! ,𝑃𝐵𝐴!)Andfinallytodeterminethenumberofserversin𝑀𝑆𝐴! ,wesumoverallPBAs:
𝑁! 𝑀𝑆𝐴! = 𝑁!(𝑀𝑆𝐴! ,𝑃𝐵𝐴!)!!!"" !"#$
Asimplifiedexamplefollowstoillustratetheprocess,beginningbymappingoccupationalcodestoagivenPBA.Considerauniversewheretherearetwooccupations(𝑂𝐶𝐶!and𝑂𝐶𝐶!)withtwopossiblePBAs(𝑃𝐵𝐴!and𝑃𝐵𝐴!).Allworkersin𝑂𝐶𝐶!areassumedtomaponto𝑃𝐵𝐴!.Workersin𝑂𝐶𝐶!canbeineither𝑃𝐵𝐴!or𝑃𝐵𝐴!.Weassumethat50%ofworkersin𝑂𝐶𝐶!willbein𝑃𝐵𝐴!and50%willbein𝑃𝐵𝐴!.Soifweassumethatthereare100workersin𝑂𝐶𝐶!and100in𝑂𝐶𝐶!,150workerswillbein𝑃𝐵𝐴!and50workerswillbein𝑃𝐵𝐴!.Next,consideracensusdivisionthatcontains3MSAs(𝑀𝑆𝐴!,𝑀𝑆𝐴!,and𝑀𝑆𝐴!).Inthiscensusdivision,wefind100serversassociatedwiththeretailPBA.Fromoccupationdata,wefind100workersin𝑀𝑆𝐴!,150in𝑀𝑆𝐴!,and250in𝑀𝑆𝐴!resultingin500totalworkersinourcensusdivisionassociatedwithretail.Wewouldthenassumethat20%(100/500)of
12
the100serversassociatedwithretailarein𝑀𝑆𝐴!.Similarly,30serversarefoundin𝑀𝑆𝐴!and50serversarein𝑀𝑆𝐴!.Lastly,forthepurposesofcreatinggeospatialmaps,serverswithinanMSAaredistributedevenlyacrossthezipcodesinthatMSA.Ourmethodtodisaggregateserverlocationbyzipcodeexplicitlyassumesthatthespatialdistributionofemployedworkersinbuildingswithserversisareasonableproxyforservers.
eGRID2012TheEmissionsandGenerationIntegratedDatabase(eGRID)administeredbytheU.S.EnvironmentalProtectionAgency(EPA)collectselectricitygenerationandemissionsdataforthemajorityofpowerplantsthatsupplyelectricityontheU.S.electricgrid(EPA,n.d.).ThesedataprovidedetailedprofilesofelectricitygenerationandassociatedgreenhousegasemissionratesacrosstheU.S.Dataarecollectedatthepowerplantlevel,thenaggregatedinto26eGRIDsubregionsdefinedbytheEPA.Variationsinemissionfactorsamongsubregionsareduetothemixofresourcesusedtopowertheregionalgrid.Subregionswithanelectricityportfoliowithsignificantcontributionsfromrenewableresources(i.e.,hydro,wind,solar,nuclear)haveloweremissionfactorsincomparisontoregionsthatrelypredominantlyoncoalandnaturalgas.
eGRIDprovidesemissionfactorsforbothbaseloadandnon-baseloadelectricitydemand.Forourestimates,weusebaseloademissionfactorswhencalculatingcurrentCO2emissionsfromservers.Reductionsinelectricityconsumptionimpactpowersuppliedbyancillarypowerplantsthatoperatecoincidentlywithpeakdemand.Weusenon-baseloadfactorswhencalculatingpotentialreductionsinCO2fromreducedelectricitydemand.eGRIDemissionfactorsarebasedonelectricitygenerationatthepowerplantandnotatthepointofconsumption.Tocapturelossesfromdistributionandtransmissionfromthepowerplanttotheconsumer,eGRIDderivesgrossgridlossfactorsfor5sectionsoftheU.S.electricgrid:Alaska,Hawaii,EasternU.S.,WesternU.S.,andERCOTwhichcoversmostofTexas.Thegrossgridlossfactorsarethenmappedontothe26eGRIDsubregionsandappliedtoourestimatesforgreenhousegasemissionstoaccuratelyaccountforalllosses.TotalCO2emissionsaretherefore:
𝐶𝑂! 𝑒𝑚𝑖𝑠𝑠𝑖𝑜𝑛𝑠 =𝐸!"#$
1− 𝐺𝐺𝐿!"#$%&× 𝐹 !"#$%&
!!!
where𝐸!"#$ representsenergyatthepointofconsumption,𝐺𝐺𝐿!"#$%&isthegridlossfactor,and𝐹 !"#$%&
!!! istheemissionsfactorrelatingtheamountofCO2emittedperunitofelectricitygenerated.Notethat𝐺𝐺𝐿!"#$%&and𝐹 !"#$%&
!!! arefunctionsofeGridsubregionasdenotedbytheirsubscript.
13
WeuseEPA’sPowerProfilerspreadsheettooltomapU.S.zipcodetoeGRIDsubregion(EPA,2015).Inaminorityofcases(2801outof41335),azipcodeiscoveredbymultipleutilitiesbelongingtodifferenteGRIDsubregions.Inthesecases,weusethelisted“PrimaryeGRIDSubregion”assumingthisrepresentsamajorityoftheelectricitysuppliedtothezipcode.
3. ResultsServersbydatacenterspacetypeInTable3,wereportgeneralstatisticsforsmallandmidsizedatacentersfoundinCBECS.Wefindapproximately3.7millionserversresideinsmalldatacenterscorrespondingto72%ofinstalledstockinCBECSbuildings.Smalldatacentersarereportedin31%ofallcommercialbuildingsinCBECSandin99%ofbuildingswithservers.Table3:SummaryoftheNumberofDatacentersandServersinCBECS*
DataCenterType
NumberofBuildings(millions)
NumberofServers(millions)
AverageNumber
ofServers
%ofServersinCBECS
NumberofCBECSRecords
Small(1-25servers)
1.725 3.70 2.1 72% 3163
Midsize(26-500servers)
0.021 1.47 69.4 28% 360
*Excludesbuildingsthatcontainmorethan500servers.IDCestimatesthetotalnumberofvolumeserversinstalledin2012acrossdatacentersofalltypestobe13.2million(IDC,2014a).OurestimateofthetotalnumberofserversinsmallandmidsizedatacentersinCBECSis5.2millionservers,implying40%ofserversarefoundinsmallandmidsizedatacenters.Figure1providesadetailedlookatthedistributioninthenumberofserverswithinourdatacenterclassifications.Thetoptwopanelsdisplaythedistributioninthenumberofserversperbuildingrelativetothetotalnumberofserversforsmallandmidsizedatacenters,respectively.Eachdistributionisnormalizedbythetotalnumberofserversinthatdatacenterspacetype.Thebottompanelsdisplaythedistributioninthenumberofserversperbuildingrelativetothenumberofsmallandmidsizedatacenters.Inthesepanels,eachdistributionisnormalizedbythetotalnumberofdatacentersineachdatacenterspacetype.
14
Figure1:Distributioninthenumberofserversbydatacentertype
Thetop-leftpanelshowsthatapproximately30%ofthetotalnumberofserversinsmalldatacentersarefoundinlocationswithonlyoneserver.Datacenterswithasingleservermayrepresentausecasewhereoperatorsneedjustoneserverforfilesharingacrossanofficeorfore-mailorwebhosting.AswithotherresultsfromCBECS,thisresultisdependentonthereportingaccuracyfromthesurveyrespondent.Aninterestingfeatureinthetop-leftpanelofFigure1istheslight,yetsignificant,increaseinfrequencyat10and20servers.Thismaybeareportingbiastowardsavalueroundedtothenearest10orpossiblyanindicationthatserveroperatorsarebiasedtowardsowningserversinincrementsof10servers.Asseeninthebottom-leftpanel,anoverwhelmingmajorityofcommercialbuildingswithsmalldatacentersoperateoneserver(~70%).Thefrequencyofsmalldatacenterswithmultipleserversdropsoffdramatically,withlessthan6%havingmorethan5servers.Thisresultindicatesthatalthoughsingleserverdatacentersonlymakeup30%ofthefractionofserversinsmalldatacenters,suchdatacentersarefarmoreprevalentandrepresentapopularusecaseforbusinesses.
15
Whenlookingatmidsizedatacenters,mostlocationshavelessthan100servers.Thereisanotablerelativedearthofdatacenterswithintherangeof200-500serversindicatingthismaybeanintermediatescaleofinfrastructurethatdoesnotserveapracticalpurposeinbusinessorindustry.CBSAreports36,500datacentersinitssampleofPacificNorthwestcommercialsites.TheCBSAsurveydidnotrecordthenumberofserversineachdatacenter,insteadcountingthenumberofserverracksinoperation(seesection2fordetails).ThisdifferencefromCBECS,whichrecordsthenumberofservers,likelyindicatesthatCBSAmaybeexcludinglocationsthatoperatewithonlyafewservers.Asdiscussedpreviously,ourCBECSanalysisshowsthatsingle-serveroperationisamajorusecase.Figure2showsthedistributionofdatacentersasafunctionofthereportednumberofoperationalserverracks.Themajorityofdatacentersreporthavingbetween1-5racks(84%),withoverhalfhavingasinglerack(55%).Detailsregardingthedimensionsoftherackwerenotrecorded.Dependingonthesizeandtypeofrack,anindividualrackcouldholdanywherefrom1serverupto40serversmakingitdifficulttotranslaterackcounttoservercountforacomparisontoresultsfromCBECS.However,theseresultsqualitativelysupporttheassertionthatthemajorityofdatacentersarerelativelysmallinsize.
Figure2:DistributioninthenumberofracksinCBSAdatacenters
Notably,1%ofidentifieddatacenters(correspondingto11records)donotreporthavinganyoperationalracks.Thisleadstoquestionsregardinghowhardwareisoperatedandspaceconditioned.Thesesitesmaybeoperatinghardwareinadhocspacesnotintendedforserveroperationandmaybenefitfromsimpleefficiencymeasures.
16
CBSAclassifieseachsiteaseitherurbanorruralbasedontheRural-UrbanContinuumCodeofthesitecounty.4Approximately33%ofsurveyeddatacentersarefoundinrurallocations.Figure3showstheurbanandruralbreakdownforeachrackcategory.Ofdatacentersthatreported0racks,59%arefoundinrurallocations,significantlymorethantheoverallbreakdown.Forotherdatacentersthatreportedhavingarack,thefractionofresponsesinrurallocationsisroughly33%,asexpectedfromtheoverallbreakdownbetweenurbanandrurallocations.Ruralareashavealargerfractionofdatacenterswith0rackscomparedtourbanareas.Theskewtowardssmaller,lessefficient(intermsofPUE)datacentersinruralareaspotentiallyindicatesanopportunityforenergysavings.
Figure3:UrbanandruralbreakdownbyracksizeinCBSAdata
ServersbyCensusDivisionInthissectionwereportgeographicresultsfromdatacentersinCBECSaggregatedbycensusdivision.Table4showsthepercentageofserversineachcensusdivisionfortheconsidereddatacentertypes.Wealsoshowthepercentageofcommercialbuildingsineachcensusdivision(CBECSWeight)forcomparison.Table4:GeographicdistributionofdatacentersbyCensusDivision.
CensusDivision SmallDatacenters
MidsizeDatacenters
CBECSWeight
NewEngland 5.8% 2.7% 5.4% MiddleAtlantic 9.2% 19% 9.1% EastNorthCentral 13% 12% 13% WestNorthCentral 6.7% 4.8% 9.0% 4Formoredetails,seehttp://www.ers.usda.gov/data-products/rural-urban-continuum-codes.aspx
17
SouthAtlantic 21% 8.2% 20% EastNorthCentral 5.6% 7.3% 6.7% WestSouthCentral 13% 16% 14% Mountain 6.1% 10% 6.1% Pacific 20% 20% 17% ThespatialdistributionofsmalldatacentersgenerallytracksthedistributionofcommercialbuildingsinCBECSwithafewnotableexceptions.WefindthattherearemoresmalldatacentersintheSouthAtlanticandPacificrelativetothenumberofcommercialbuildings.TherearefewersmalldatacenterserversinWestSouthCentralandEastNorthCentralrelativetonumberofcommercialbuildings.Overall,thesimilarityinspatialdistributionsunderscorestheprevalenceofsmalldatacentershousedwithincommercialspacesacrosstheU.S.Contrarytoourfindingsforsmalldatacenters,thespatialdistributionofmidsizedatacentersdifferssignificantlyfromthatofcommercialbuildings.MidsizedatacentersaresignificantlyoverrepresentedintheMid-AtlanticandMountainrelativetothenumberofcommercialbuildingsintheseregions.Giventhatmidsizedatacentersarefoundlessfrequentlythansmalldatacenters(asshowninTable1),midsizedatacentersmayonlybeutilizedbycertainindustriesthatrequiresignificantcomputingpoweranddonotnecessarilytrackthegeographicdistributionofallCBECsbuildings.
ServersbyPrincipleBuildingActivityInthissectionweinvestigatehowserversaredistributedbyCBECSbuildingactivity.CBECSprovidesaprincipalbuildingactivitycodeandamorespecificbuildingactivitycode.Inouranalysis,weanalyzedresultsbybothPBAandspecificPBA.Fortheresultspresentedinthissection,wefocusourattentiononresultsbyPBAaslittlemeaningfulinformationisgainedbythefinergradationprovidedbythespecificbuildingactivitycode.However,whennoteworthy,wediscussresultsbyspecificbuildingactivity.Table5showsthepercentageofserversbyPBAforeachdatacentertype.Althoughsmalldatacentersarefoundinadiversevarietyofsettings,thepluralityofserversassociatedwithsmalldatacentersarefoundinofficebuildings.Thisisunsurprisinggiventheutilitythatserversprovidewithinofficesettingssuchasfilestorageacrossanetwork,accesstodatabases,andrunningsharedenterpriseapplications.LookingatspecificPBAswithinofficebuildings,wefindthat26%ofserversinsmalldatacentersarefoundinadministrativeoffices.Similartosmalldatacenters,serversinmidsizedatacentersareprimarilyfoundinofficebuildings,specificallywithinadministrativeoffices(38%).Medicalbuildingsalsohouseasignificantnumberofserverswithinlargedatacenters,withmostassociatedwithInpatientCare(12%).
18
Table5:Percentageofserversbybuildingactivityforeachconsidereddatacentertype
BuildingActivity
SmallDataCenters
MidsizeDataCenters
Education 9.8% 7.4%FoodSales 1.8% 0.0%FoodService 3.0% 0.0%Lodging 1.2% 0.9%Medical 6.9% 22%Office 43% 57%PublicAssembly 3.3% 5.4%PublicOrder 2.1% 1.7%ReligiousWorship 2.0% 0.0%Retail 13% 0.1%Service 4.4% 0.2%Warehouse 8.2% 5.4%Vacant 0.71% 0.0%Other 1.4% 0.3%Total 100% 100%AnotherwayofinvestigatingtheprevalenceofdatacentersisbylookingatthemarketsaturationofdatacentersineachPBA.Foragivenbuildingactivity,thedatacentersaturationisdefinedastheratioofthenumberofbuildingsofthatactivitywithadatacentertothetotalnumberofbuildingsofthatactivity. Table6showssaturationratesofsmallandmidsizedatacentersacrossPBA.Forsmalldatacenters,saturationratesrangefrom14%forreligiousworshipupto59%formedicalcarefacilities(notincludingadministrativemedicaloffices)withtypicalvaluesbetween20-30%.Theseresultsshowthatsmalldatacentersarerelativelycommoninawiderangeofindustries.Again,wefindthatsmalldatacentersareespeciallyprevalentinofficesettings.Notethatwefindhighsaturationratesformedical(56%),courthouses(94%),andgovernmentbuildings(55%),whichhouseactivitieswherefederalprivacylawsmayimpedeadoptionofoff-sitecolocation,highlightingtheneedtocustomizeefficiencystrategiesforthesesmalldatacenters.Forexample,privacylawmaypreventmedicalcompaniesfrommovingtheirin-housedatacentertoacloud-basedsystemoperatingfromalarger,moreefficienthyperscaledatacenter.Insuchcases,therearepotentiallymethodsofleveragingthecomputationalpowerofhyperscaledatacentersthatinteractwithon-siteprotecteddata,whichcouldyieldsignificantenergysavings.Bycontrast,highsaturationratesforprivatebusinessesinofficeandretailspacesmayprovideanopportunityforsubstantialenergysavingsviacloud-basedsolutionswithouthavingtoaddressasmanyobstaclesregardingprivacy.
19
Table6:Serversaturationbybuildingactivityforeachconsidereddatacentertype
BuildingActivity SmallDataCenters
MidsizeDataCenters
Education 38% 0.54%FoodSales 31% 0.0%FoodService 22% 0.0%Lodging 22% 0.21%Medical 59% 2.2%Office 56% 1.1%PublicAssembly 20% 0.21%PublicOrder 28% 0.55%ReligiousWorship 14% 0.0%Retail 40% 0.0%Service 19% 0.0%Warehouse 21% 0.24%Vacant 3.0% 0.0%Other 32% 0.0%
SpaceCoolinginDatacentersTheCBSAsurveyallowsustoanalyzeHVACequipmentasafunctionofspacetypeandnumberofserverracks.AlthoughthesedataareweightedtoberepresentativeofthePacificNorthwest,wenotethatthedatamaynotreflectspaceconditioningacrossothergeographicregions.DifferentgeographicregionswillhavedifferentspacecoolingrequirementswhendeterminingHVACneedsforthebuilding.ThePacificNorthwestcovers‘Marine’and‘Cold’climatezonesasdefinedbytheBuildingAmericaprogram(PacificNorthwestNationalLaboratory,2015),sponsoredbytheU.S.DepartmentofEnergy’sOfficeofEnergyEfficiencyandRenewableEnergy(EERE).Buildingsinthesezonesgenerallyhavelowercoolingloadscomparedtohighertemperatureclimatezones,whichwillbefactoredintoHVACsystemdesigns.Thiscouldpotentiallyimpactspaceconditioningfordatacentersites,particularlythosethatutilizeeconomizersforcooling.
20
Figure4:SpacecoolingcharacteristicsofdatacentersintheCBSA
ThetoppanelsofFigure4displayresultsfordatacenterswithdedicatedairconditioning.Whencategorizingdatacentersbysquarefootage(leftpanel),wefindthatamajorityofdatacenterslessthan101squarefeetdonothavededicatedspaceconditioning,relyingoncoolingsystemsthatservicealargerpartofthebuildingornocoolingsystem.Surprisingly,datacentersinthe101-1000squarefootrangehavealargerproportionofdedicatedcooling(79%)comparedtodatacenterslargerthan1000squarefeet(62%),althoughwe
21
notethatthereareonly13recordsofdatacenterslargerthan1000squarefeetwithnodedicatedcooling.Furtherinvestigationfindsthatonlyonerecordofthe13reportshavingnotypeofHVACsystem.Inthetop,rightpanel,wefindthattheproportionofdatacenterswithoutdedicatedcoolingdecreasesasthenumberofoperationalserverracksincreases.ThebottompanelsshowtheproportionofdatacentersthatusedifferenttypesofHVACsystems.Notethatthesesystemsdonotnecessarilycorrespondtodedicatedconditioningforonlytheserverspaceandmayalsobeusedforconditioningotherpartsofthebuilding.Theleftpanelshowsthefractionofresponsesbysquarefootageandtherightpanelbynumberofoperationalserverracks.Amajorityofdatacenterslessthan1000squarefeetuseair-cooledcomputerroomairconditioners(CRACs),whicharelabeled‘AirDX’inthefigure.Asizablefractionofdatacenterslessthan1000squarefeetrelyonbuildingair-handlingunits(labeled‘BuildingTransfer’).Datacenterslessthan101squarefeethavethelargestfractionofrespondentsthatusenoconditioning(27%).Datacenterslargerthan1000squarefeetdatacentersrelymostlyonwater-cooleddirectexpansionsystems(42%).Thebottom,rightpanelshowssimilarresultsbynumberofoperationalserverracks.Over80%ofdatacenterswith0racksrepliedtheyhavenoHVACsystem.Curiously,wefind67recordsfordatacentersthatreporthavingeitheranair-cooledorwater-cooledCRAC,butalsoreporteitherhaving‘unknown’ornodedicatedairconditioning.ThisissurprisinggiventhatCRACsaredesignedspecificallyforuseindatacentersandwouldbeillsuitedforgeneralbuildingcooling.ItisunclearwhetherrespondentsansweredthequestionsincorrectlyincludingothertypesofcoolingsystemswithCRACsorifthereareofficeswhereCRACsarealsousedmoregenerallyinabuilding.
ServerVirtualizationServervirtualizationallowsasinglephysicalservertoactasmultiplevirtualmachinesrunningindependenttasksandapplications,ratherthandedicateindividualserversforeachapplication.Bycreatingvirtualmachines,datacenteroperatorscanconsolidatehardwareandmorefullyutilizeseverprocessingpower.However,previousresearchonsmallbusinessdatacenterpracticesindicatesthatservervirtualizationisnotheavilypracticed.BennettandDelforge(2012)foundonly37%ofsmallbusinessespracticeservervirtualizationinasurveyperformedin2011.Theauthorsnotethatmanysmallbusinesseshadneverheardofservervirtualizationordidnotthinktheinvestmentinvirtualizationwouldbecost-effective.Asurveyoflarge-scaleenterprisesin2011foundthatapproximately92%ofbusinessesutilizedvirtualization(VansonBourne,2011)andamorerecentsurveyin2016foundthat75%ofbusinessesofallsizesutilizedvirtualization(Spiceworks,n.d.).TheCBSAstudyrecordedthedegreeofvirtualizationundertakenbyITstaffforeachdatacenter.Unfortunately,itisunclearwhetherthedegreeofvirtualizationreferstothenumberofserversthatarevirtualserversorthenumberofphysicalserversthatcontainvirtualizationsoftware.Notably,mostsitesdidnotprovidearesponse.Giventhelackofvirtualizationinsmallbusinesseswithvirtualizationknowledge(BennettandDelforge,2012),itislikelythatmostofthesesitesdonotmakeuseofdatavirtualization.However,theremaybesomesiteswheretherespondentmaynothavebeenanITprofessional
22
capableofproperlyansweringthequestion.Giventheuncertainty,unknownresponsesareseparatelyidentifiedinourreporting.InFigure5,weshowresultsforthereporteddegreeofvirtualizationbysquarefootage.Datacenterslessthan1000squarefeetshowahighproportionofunknownandnovirtualizationresponses.Only8%ofdatacenterslessthan101squarefeethaveanydegreeofvirtualization,with7%reportingmorethan50%virtualization.Approximately15%ofdatacentersoperatewithservervirtualization.Datacenterslargerthan1000squarefeetoperatewiththehighestproportionofservervirtualizationwith31%.Wewouldexpectthatlargercentershavemoreopportunitytoconsolidate,sincetheywouldberunningalargernumberofapplications,andwouldbenefitthemostfromvirtualization.
Figure5:Degreeofservervirtualizationbydatacenterspacetype
4. DiscussionSpatialDistributionofServersInthissectionwecombinedatafromCBECSandOEStoconstructspatialmapsofserverintensity(i.e.,serversperunitarea).AsdiscussedinSection2,occupationstatisticsatthezipcodelevelareusedasaproxytoestimatethespatialdistributionofservers.Figure6showsthecombinedgeographicserverintensityforbothsmallandmidsizedatacenters.Serverintensitycloselytrackspopulationcenterswithhighintensitiesinurbanareas.Thisisunsurprisinggiventhatmostjobsthatwouldutilizeaserverarefoundincities.
23
MapsofCO2emissionintensitiesdisplayedaverysimilardistributiontowhatwefindforserverintensity.TheyarenotincludedherebecausedifferencesbetweentheserverintensityandCO2emissionmapswerehardtodiscern.However,itisworthmentioningthattherearesubtledifferencesbetweenthegeographicdistributionforserverandcarbondioxideemissionintensity.Forexample,thePacificNorthwesthasloweremissionsperserverduetotheregion’srelianceonhydropower.TheDenvermetropolitanregioninColoradohashigheremissionsperserverduetotheregion’srelianceoncoal.
Figure6:Geographicdistributionofservers
Inthenextsection,wewillcombineourderivedspatialdistributionofserverswithtypicalpowerconsumptionvaluesandgeographic-specificwithcarbonemissionfactorsfromeGridtoestimateregionalcarbondioxideemissions.
NationalEnergyUseandCarbonDioxideEmissionsInthissection,weaggregateourresultstoestimatetheannualenergyusedueonlytoservers,totalsiteenergyuseincludingenergyusedtomaintainandoperatethedatacenter,andtotalCO2emissionsincludinggrossgridlossesfortheconsidereddatacenters.ResultsforsmallandmidsizedatacenterscanbefoundinTable7.
Table7:AggregateannualenergyconsumptionandCO2emissionsbydatacentertype
DataCenterType
NumberofServers(millions)
ServerEnergyUse(billionkWh)
TotalEnergyUse(billionkWh)
TotalCO2Emissions(MMT)
Small 3.70 5.83 12.9 6.71Midsize 1.47 2.32 6.34 3.29Total 5.17 8.15 19.2 10.0
24
Inaggregate,smallandmidsizedatacentersuseapproximately20billionkWhandemit10millionmetrictonsofCO2peryear,equivalenttotheannualelectricityuseof1.5millionhomes.5Theaverageserverinasmalldatacenterisresponsiblefor1.81metrictonsofCO2peryearandinamidsizedatacentertheaverageserverisresponsiblefor2.23metrictonsofCO2peryear.
ComparisontoIDCMarketReportMarketresearchfirmIDCauthoreda2014studyanalyzingthestateofthedatacenterindustry(IDC,2014b).ThestudyprovidesacensusofUSdatacentersbasedonin-depthinterviewswithITbuyersandsuppliersincludingvendorsthatmanufacturepower,cooling,server,storage,andnetworkingequipment.Thestudypresentshistoricaldataandprojectionsofthedatacentermarketbasedonunderlyingindustrytrends.WeusedatapresentedintheIDCstudytocomparetotheestimatesderivedfromCBECS.IDCreportsthenumberofdatacentersbyspacetypeusingataxonomythatcapturestherelativelybroaddefinitionofdatacenter.Fromsmallesttolargest,theirtaxonomyincludescloset,room,localizedinternal,mid-tierinternal,andhigh-endinternaldatacenters.AsdescribedinSection2,thetaxonomyusedinthisworkcombinesclosetandroomdatacentersinto“small”andlocalizedinternalandmid-tierinternalinto“midsize”.InTable8,wecomparethenumberofdatacentersreportedbyIDCandourestimatesfromCBECS.Therearelargediscrepanciesbetweenthetwodatasets.IDCreportsnearlyonemillionmoresmalldatacenterscomparedtothenumberofbuildingsclassifiedasasmalldatacenterinCBECS.Formidsizedatacenters,CBECSreportsroughlyaquarterofdatacentersfoundbyIDC.Table4:SummaryofcomparisonindatacentercountsbetweenIDCandthiswork
Withinsmalldatacenters,IDCreportsaroughlyevensplitbetweenclosetandroomdatacenters.However,assumingbuildingsinCBECSwithfewerthan5serverscorrespondtoaclosetdatacenter,wefindthat91%ofsmalldatacentersareclosets.Theskewtowards
5UsingEPA’sgreenhousegasequivalencycalculator:https://www.epa.gov/energy/greenhouse-gas-equivalencies-calculator
IDCTaxonomy IDCDataCenterCount
Taxonomy(thiswork)
IDCDataCenterCount
CBECS(thiswork)
Closet 1,512,829 Small 2,846,125 1,724,783Room 1,333,296Localized 70,419 Midsize 80,383 21,203
Mid-tier 9,964
High-end 8,283 NotAnalyzed
8,283 NotAnalyzed
25
closetdatacenterscanbeseeninthebottomleftpanelofFigure1whichshowsthevastmajorityofsmalldatacentershaveonly1server.IfweapplytheIDCdefinitionofclosetdatacentertoCBECSbuildings(i.e.,adatacenterwithlessthan5servers),weestimate1.56millionclosetdatacenters.ThisvalueisveryclosetoIDC’sreportedestimateof1.51milliondatacenters.However,asinthecaseofmidsizeandhigh-enddatacenters,CBECSreportssignificantlyfewerroomdatacentersbyapproximatelyafactorof8.IDC’srangeinthenumberofserversisintendedasaroughguidetocategorizedatacentersandnotasastrictdefinition.However,itisdifficulttoseehowmisalignedclassificationsofdatacentersinCBECScouldpotentiallyexplainthedifferenceforlargeandhigh-enddatacentersbetweenthetwodatasets.Forthesedatacenters,CBECSreportsroughlyaquarterfewerdatacenters.OnepossibleexplanationforthedifferenceisCBECSgivesacountbybuilding,whereasIDCisestimatingthenumberofdatacenters.Thatis,abuildinginCBECSwithmultipledatacenterswouldbeaggregatedandrecordedasasinglerecord.AsCBECSgivesestimatesofthetotalnumberofserversineachbuilding,wehavenowayofinferringthetruenumberofdatacentersinbuildingswithmultipleseparatedatacenters.ThesystematicdifferencescouldpotentiallyberelatedtodifferencesinsurveymethodologybetweenIDCandCBECS.IDCreliesoninformationfromtheITindustrytodevelopestimatesofdatacenters,whereasCBECSrespondentsarecommercialbuildingownersandoperators.However,notethatsinceourenergyuseandCO2calculationsrelyonthenumberofservers,ourestimatesarenotsensitivetothisdiscrepancyinthenumberofdatacenters.InadditiontotheIDCdatacenterreport,wealsoobtaineddataoninstalledserverstock(IDC,2014a).IDCestimatesapproximately13.4millionserversin2012.FromCBECSdata,weestimate5.17millionserversinstalledinsmallandmidsizedatacentersimplyingtheseembeddeddatacenterspacetypesaccountforapproximately40%ofserverstock.Thelargefractionofserversinsmallandmidsizedatacentersagainunderscorestheneedtobetterunderstandthesetypesofdatacenterstoachieveenergysavingspotential.
EnergySavingOpportunities
ServerVirtualizationResultsfromtheCBSA2014showthatservervirtualizationremainsmostlyunderutilizedinsmallandmidsizedatacenters.Similarly,BennettandDelforge(2012)report26%servervirtualizationsaturationinsmallbusinessessurveyedin2011.OuranalysisofCBSAdatainSection3foundapproximately10%ofdatacentersutilizedsomelevelofservervirtualization.Theselowestimatesofvirtualizationindicatethereissignificantavailableenergysavingspotentialfromincreasingsaturation.BennettandDelforge(2012)estimatethatdatacenterswith10ormoreserverswouldbenefitfromvirtualization.Asascenario,weassumethat50%ofdatacenterswith10ormoreserversinCBECSareabletoreachvirtualizationratioof5to1,reportedasan
26
“average”usecasebyWSPandtheNationalResourceDefenseCouncil(2012).Inthisscenario,weestimateannualsiteenergysavingsof3.8billionkWh,correspondingto2millionmetrictonsofCO2emissions.Therearemanybarrierstotheadoptionofvirtualizedserversinsmallbusinesses.Inmanycases,businessespayaflatfeeforenergyserviceseliminatingenergycostsasanincentivetoreduceenergydemand.OrganizationbarriersmayincludelackofcapitaltoupgradeandlackofITknowledge.Additionally,virtualizationwillnotbenefitsmalldatacentersthatoperatefewserverswithlittleroomtoeliminateexcesshardware.
CloudComputingTheriseofcloudcomputingoverthepastdecadehasprovidedanewmeansforbusinessestomeetITneeds.TheavailabilityofSoftwareasaService(SaaS),InfrastructureasaService(IaaS),andPlatformasaService(PaaS)cloudsolutionshasmovedcomputingdemandawayfromsmall,on-premisedatacentersintohighlyoptimizedcloud-computingdatacenters.Inasmalldatacenter,individualserversareoftenusedforrunningindividualapplicationsortasks.Thisleadstoserversthat,onaverage,operateatverylowutilizationlevels.Inlarge,hyper-scaledatacenters,applicationsandtasksaredistributedacrossahugenumberofserversandoperationalizedtorunatutilizationlevelscloserto50%(Shehabietal.,2016).Cloud-computingdatacentersbenefitfromeconomiesofscaleandareabletomakeupfrontinfrastructureinvestmentstosupportefficientdatacenteroperations(WSPEnvironment&EnergyandNaturalResourcesDefenseCouncil,2012).Giventhelargecostofoperatingthousandsofservers,evensmallimprovementsinefficiencycanleadtosignificantoperatingcostsavings.TheUptimeInstitute(2014)estimatedanaveragePUEof1.7fordatacentersthatself-reportedthePUEfortheirlargestdatacenter.SomecloudcomputingdatacentershaveachievedevenlowerPUEs.Forexample,Googlehasreachedafleet-wideaveragePUEof1.12(Google,2016).Pushingthelimitsevenfurther,Gao(2014)presentsaframeworkforapplyingmachine-learningartificialintelligencetooperationaldatatopotentiallydecreaseGoogledatacenterPUEsbelow1.10.Forcomparison,smalldatacentershavetypicalPUEscloserto2.0(Cheungetal.,2014;Shehabietal.,2016).Hereweperformasimpleanalysistoapproximatethepotentialsavingsfrommovingcomputingneedsawayfromasmalldatacentertoacloud-computingdatacenter.Weassumethatserverloadsfromprivate-runofficeandretailstorescanbeeasilyshiftedfromin-housedatacenterstocloudcomputingdatacenters.Shehabietal.(2016)estimateacloud-computingserverrunningundertypicalconditionsconsumesapproximately360W.However,movinganapplicationortaskfromasmalldatacentertoacloud-computingserverwillonlyconsumeamarginalfractionoftheserver’scomputingpower.Previousstudiesestimatethattheloadofasmalldatacenterserverisapproximatelyequivalentto1/5theloadofhigh-endcloudserver(Masanet,2014;Shehabietal.,2016).Ifweassumeconservativelya10%reductioninserversfromindustriesotherthanfinancial,health,andgovernmentrelatedfields,weestimateapotential1.2billionkWhreductioninenergyconsumptionand0.85MMTofCO2emissionsperyear.
27
Asurveyofbusinessownerswithsmalldatacentersfoundthatformanyowners,amainbarriertocloudadoptionstemsfromprivacyandsecurityconcerns(BennettandDelforge,2012).However,theseareconcernsthatarenotnecessarilyrootedinreality.Cloudcomputingdatacentersoftentakemoremeasurestoensurecustomerdataismaintainedsecurelyanddedicatestaffresourcesforthispurpose.Additionally,manycloudcomputingoperatorsofferprivatecloudsolutionsforcompaniesthatwishtosilotheirdatafrompubliccloudresources.IncreasingpenetrationofclouddeploymentwithinsmallerdatacenterswilltakeaconcertedefforttoeducateITprofessionals.Althoughcloudcomputingoffersenergysavingspotentialbyshiftingloadsawayfromsmalldatacenterstomoreefficienthyperscaledatacenters,easyaccesstocloudservicescouldleadtoincreaseddemandforcomputingneedsthatwerenotneededpreviously.Forexample,businessesthatpreviouslyreliedonpaperrecordsthatmovetocloudserviceswilladdtooverallenergydemandorstartprovidingservicesthatrequirecloud-computingresources.Thesebusinesseswilllikelybenefitfromtheefficiencyofdigitizingrecordsandonlinemanagement.Forsomenewusers,thecloudoffersaneaseofsetupandmaintenanceinthatitdoesnotrequireadedicatedITprofessionaltoinstallanewsystem.
5. ConclusionTargetingsmallerdatacentersremainschallengingduetotheadhocnatureofsmalldatacenters,whichareoftensituatedinconfinedspaceswithinlargerbuildings.WemakeuseofrecentlyreleaseddataonserversanddatacentersincommercialbuildingstocharacterizethesmallandmidsizedatacentermarketandconstructspatialmapsoftheirlocationsacrosstheUnitedStates.Wecategorizebuildingswithlessthan25serversassmalldatacentersandbuildingswithmorethan25butfewerthan500datacentersasmidsizedatacenters.Wefindapproximately5millionservers(approximately40%ofinstalledstock)insmallandmidsizedatacenters,consistentwithpreviousestimates(Baileyetal.,2007;DelforgeandWhitney,2014).Thevastmajorityofserversinsmall&mediumdatacentersarefoundinsmalldatacenters.Amajorityofsmalldatacentersarefoundinofficesandretailbuildings.Thehighestsaturationsareinmedical,retail,office,andeducationsectors.Wefindthatapproximately26%ofserversinsmalldatacentersarefoundinadministrativeoffices.BasedondatafromtheCBSAconductedinthePacificNorthwest,amajorityofsmalldatacenterswithmorethanoneoperationalrackmakeuseofair-cooleddirectexpansionCRACsforspaceconditioning.ThemajorityofdatacenterswithoneorlessoperationalserverrackdidnotreportusinganyspecializedcoolingsystemandarelikelyrelyingonthebuildingHVACsystem.Siteswithtwoormoreoperationrackstendtorelyondedicatedspacecoolingwithair-cooledCRACsbeingthemostcommon.Water-cooledCRACsaremainlyusedindatacentersthatarelargerthan1000squarefeet,makingupapproximately40%ofcoolingunitsinthesesites.
28
Inaggregate,wefindthatsmallandmidsizedatacentersconsumeapproximately20billionkWhofsiteenergy.UsingdatafromtheBureauofLaborofStatistics,wecorrelatethenumberofserversfoundintheCBECSdatasetstoemploymentdatawithinoccupationalfieldsidentifiedasusingdatacenters.ThisallowsustodisaggregateresultsbyzipcodetocreatedetailedgeospatialmapsofserverintensityacrosstheUS.Unsurprisingly,highestserverintensitiesarelocatedindensemetropolitanareas,withhighestconcentrationsontheEastandWestCoasts.Intotal,smallandmidsizedatacentersemitover10MMTonCO2emissionsannually.Servervirtualizationprovidesameansforanindividualservertoactasmultiplemachinesallowingforserverconsolidation.Althoughapracticalmeanstodecreasedatacenterenergyconsumption,wefindthatitisnotacommonpracticeinsmalldatacenters.Thevastmajorityofdatacentersrespondedthattheydidnotemployvirtualizationordidnotknowthedegreeofvirtualizationwithintheirdatacenter.Only15%ofsmalldatacentersreportedhavinganylevelofvirtualizationwithintheirfleet.Modestconsolidationofhardwarethroughvirtualizationindatacenterswithmorethan10serverscouldsave3.8billionkWhannually,correspondingto2millionmetrictonsofCO2.Themostchallengingbarriertoadoptionofvirtualizationisnottechnical.Mostsmallbusinesseshaveeitherneverheardofservervirtualizationordidnotthinktheinvestmentinvirtualizationwouldbecost-effective(BennettandDelforge,2012).Theeraofcloudcomputingrepresentsanopportunitytoshiftresourcesfrominefficientsmalldatacenterstohighlyoptimizedhyper-scaledatacenters.Although,thereareindustrieswhereprivacyregulationsmaypreventtheuseofthirdpartyclouddatacenters,suchasinhealthcare,manyindustriesthatcurrentlyrelyonsmalldatacentersshouldbeabletomovecomputingtocloudcenters.Assumingofficesotherthanfinancial,healthcare,andgovernmentofficescanshift10%oftheirloadtocloud-computingresources,wecalculateapproximateenergysavingsof1.2billionkWhperyear.Despiteincreasingconsolidationofsmalldatacentersintolargerones,thereremainasizablenumberofsmallandmidsizedatacenters.Thisreporttakesafirststeptowardsidentifyingandcharacterizinghowthesecentersoperate.However,asdiscussedthroughoutthisreport,amainbarrierimpedingefficientoperationislackofeducationaboutbestpracticesandopportunities.Futureprogresswillbemadebymeasuresandpoliciesthatareabletodirectlytargetsmalldatacenters.
29
ReferencesBailey,M.,Eastwood,M.,Grieser,L.,Turner,V.,Gray,R.C.,2007.IDCSpecialStudy:Data
CenteroftheFuture(No.IDC#06C4799).InternationalDataCorporation(IDC).Bennett,D.,Delforge,P.,2012.SmallServerRooms,BigEnergySavings:Opportunitiesand
BarrierstoEnergyEfficiencyontheSmallServerRoomMarket.NaturalResourcesDefenseCouncil.
BLS,2015.OccupationalEmploymentStatisticsHomePage[WWWDocument].URLhttps://www.bls.gov/oes/(accessed6.27.17).
Brown,R.,Energy,A.toS.,Incorporated,I.C.F.,Incorporated,E.R.G.,Agency,U.S.E.P.,2008.ReporttoCongressonServerandDataCenterEnergyEfficiency:PublicLaw109-431.LawrenceBerkeleyNatl.Lab.
CenterofExpertiseforEnergyEfficiencyinDataCenters,n.d.CenterofExpertiseforEnergyEfficiencyinDataCenters[WWWDocument].URLhttps://datacenters.lbl.gov/(accessed3.27.17).
Cheung,I.,Greenberg,S.,Mahdavi,R.,Brown,R.,Tschudi,W.,2014.EnergyEfficiencyinSmallServerRooms:FieldSurveysandFindings.Presentedatthe2014ACEEESummerStudyonEnergyEfficiencyinBuildings.
Delforge,P.,Whitney,J.,2014.DataCenterEfficiencyAssessment-ScalingUpEnergyEfficiencyAcrosstheDataCenterIndustry-EvaluatingKeyDriversandBarriers.NaturalResourcesDefenseCouncil.
EIA,2015.EnergyInformationAdministration(EIA)-CommercialBuildingsEnergyConsumptionSurvey(CBECS)Data[WWWDocument].URLhttps://www.eia.gov/consumption/commercial/data/2012/index.php?view=microdata(accessed6.27.17).
EIA,n.d.Officebuildingswithdatacentersusesignificantlymoreelectricitythanotheroffices-TodayinEnergy-U.S.EnergyInformationAdministration(EIA)[WWWDocument].URLhttps://www.eia.gov/todayinenergy/detail.php?id=28232(accessed5.8.17).
EPA,2015.PowerProfilerZipcodeTool[WWWDocument].URLfromhttp://www2.epa.gov/sites/production/files/2015-10/power_profiler_zipcode_tool_2012_v6-0.xlsx(accessed10.21.16).
EPA,n.d.Emissions&GenerationResourceIntegratedDatabase(eGRID)[WWWDocument].URLhttps://www.epa.gov/energy/emissions-generation-resource-integrated-database-egrid(accessed5.8.17).
Gao,J.,2014.MachineLearningApplicationsforDataCenterOptimization.Google,2016.Efficiency:Howwedoit–DataCenters–Google[WWWDocument].URL
https://www.google.com/about/datacenters/efficiency/internal/(accessed9.1.16).
Google,2011.Google’sGreenDataCenters:NetworkPOPCaseStudy.IDC,2014a.IDC’sWorldwideQuaterlyServerShipmentTracker,2010-2018.International
DataCorporation(IDC),Framingham,MA.IDC,2014b.U.S.DatacenterCensusandConstruction2014–2018Forecast:Realigning
Workloads,ManagingObsolescence,andLeveragingHyperscale(No.252712).Koomey,J.,2011.Growthindatacenterelectricityuse2005to2010.Rep.Anal.Press
Complet.Req.N.Y.Times9.
30
Koomey,J.,Taylor,J.,2015.Newdatasupportsfindingthat30percentofserversare“Comatose”,indicatingthatnearlyathirdofcapitalinenterprisedatacentersiswasted.
Masanet,E.,2014.TheEnergyEfficiencyPotentialofCloud-BasedSoftware:AU.S.CaseStudy.eScholarship.
Masanet,E.R.,Brown,R.E.,Shehabi,A.,Koomey,J.G.,Nordman,B.,2011.EstimatingtheenergyuseandefficiencypotentialofUSdatacenters.Proc.IEEE99,1440–1453.
NavigantConsulting,2014.2014CommercialBuildingStockAssessment:FinalReport.NorthwestEnergyEfficiencyAlliance.
NorthwestEnergyEfficiencyAlliance,2014.NorthwestEnergyEfficiencyAlliance:2014CBSADatabase[Dataset].Portland,OR.
PacificNorthwestNationalLaboratory,2015.GuidetoDeterminingClimateRegionsbyCounty,BuildingAmericaBestPracties.
Shehabi,A.,Smith,S.J.,Horner,N.,Azevedo,I.,Brown,R.,Koomey,J.,Masanet,E.,Sartor,D.,Herrlin,M.,Lintner,W.,2016.UnitedStatesDataCenterEnergyUsageReport(No.LBNL-1005775).
Spiceworks,n.d.2016LatestTrendsinIT&TechnologyAdoption[WWWDocument].URLhttps://www.spiceworks.com/marketing/resources/reports/2016-state-of-it/(accessed5.8.17).
TheGreenGrid,2012.PUE:AComprehensiveExaminationoftheMetric.UptimeInstitute,2014.2014DataCenterIndustrySurvey[WWWDocument].UptimeInst.
EJournal.URLhttps://journal.uptimeinstitute.com/2014-data-center-industry-survey/(accessed9.9.16).
VansonBourne,2011.VeeamlaunchesV-indextomeasurevirtualizationpenetrationrate[WWWDocument].VeeamSoftw.URLhttps://www.veeam.com/news/veeam-launches-v-index-to-measure-virtualization-penetration-rate.html(accessed9.8.16).
WSPEnvironment&Energy,NaturalResourcesDefenseCouncil,2012.TheCarbonEmissionsofServerComputingforSmall-toMedium-SizedOrganizations:APerformanceStudyofOn-Premisevs.TheCloud.WSPEnvironment&EnergyandNaturalResourcesDefenseCouncil.
Recommended