Edge Computing - OpenStack

EdgeComputingNextStepsinArchitecture,DesignandTestingEdgeComputing:NextStepsinArchitecture,DesignandTesting

Introduction

Whileedgecomputinghasrapidlygainedpopularityoverthepastfewyears,therearestillcountlessdebatesaboutthedefinitionofrelatedtermsandtherightbusinessmodels,architecturesandtechnologiesrequiredtosatisfytheseeminglyendlessnumberofemergingusecasesofthisnovelwayofdeployingapplicationsoverdistributednetworks.

InourpreviouswhitepapertheOSFEdgeComputingGroupdefinedcloudedgecomputingasresourcesandfunctionalitydeliveredtotheendusersbyextendingthecapabilitiesoftraditionaldatacentersouttotheedge,eitherbyconnectingeachindividualedgenodedirectlybacktoacentralcloudorseveralregionaldatacenters,orinsomecasesconnectedtoeachotherinamesh.Fromabird’seyeview,mostofthoseedgesolutionslooklooselylikeinterconnectedspiderwebsofvaryingsizesandcomplexity.

Inthesetypesofinfrastructures,thereisnoonewelldefinededge;mostoftheseenvironmentsgroworganically,withthepossibilityofdifferentorganizationsowningthevariouscomponents.Forexample,apubliccloudprovidermightsupplysomeofthecoreinfrastructure,whileothervendorsaresupplyingthehardware,andyetathirdsetofintegratorsarebuildingthesoftwarecomponents.Tryingtocreateaonesizefitsallsolutionisimpossibleforedgeusecasesduetotheverydifferentapplicationneedsinvariousindustrysegments.Interestingly,whilecloudtransformationstartedlaterinthetelecomindustry,operatorshavebeenpioneersintheevolutionofcloudcomputingouttotheedge.Asownersofthenetwork,telecominfrastructureisakeyunderlyingelementinedgearchitectures.

Afterfouryears,whilethereisnoquestionthatthereiscontinuinginterestinedgecomputing,thereislittleconsensusonastandardedgedefinition,solutionorarchitecture.Thatdoesn’tmeanthatedgeisdead.Edgemustbebyitsverynaturehighlyadaptable.Adaptabilityiscrucialtoevolveexistingsoftwarecomponentstofitintonewenvironmentsorgivethemelevatedfunctionality.Edgecomputingisatechnologyevolutionthatisnotrestrictedtoanyparticularindustry.Asedgeevolves,moreindustriesfinditrelevant,whichonlybringsfreshrequirementsorgivesexistingonesdifferentcontexts,attractingnewpartiestosolvethesechallenges.Nowmorethanever,edgecomputinghasthepromiseforaverybrightfutureindeed!

ThisdocumenthighlightstheOSFEdgeComputingGroup’sworktomorepreciselydefineandtestthevalidityofvariousedgereferencearchitectures.Tohelpwithunderstandingthechallenges,thereareusecasesfromavarietyofindustrysegments,demonstratinghowthenewparadigmsfordeployinganddistributingcloudresourcescanusereferencearchitecturemodelsthatsatisfytheserequirements.

Challengesindifferentindustries

Inanutshell,edgecomputingmovesmorecomputationalpowerandresourcesclosertoendusersbyincreasingthenumberofendpointsandlocatingthemnearertotheconsumers--betheyusersordevices.Fundamentally,edgecomputingarchitecturesarebuiltonexistingtechnologiesandestablishedparadigmsfordistributedsystems,whichmeansthattherearemanywellunderstoodcomponentsavailabletocreatethemosteffectivearchitecturestobuildanddeliveredgeusecases.

Thissectionwillguideyouthroughsomeusecasestodemonstratehowedgecomputingappliestodifferentindustriesandhighlightthebenefitsitdelivers.Wewillalsoexploresomeofthedifferentiatingrequirementsandwaystoarchitectthesystemssotheydonotrequirearadicallynewinfrastructurejusttocomplywiththerequirements.

5GBringsYoutheEdgeorViceVersa?

5Gtelecomnetworkspromiseextrememobilebandwidth,buttodeliver,theyrequiremassivenewandimprovedcapabilitiesfromthebackboneinfrastructurestomanagethecomplexities,includingcriticaltrafficprioritization.Thenetworkneedstoprovidebothhighthroughputandlowlatencycombinedwithefficientuseoftheavailablecapacityinordertosupporttheperformancedemandsoftheemerging5Gofferings.

SignalingfunctionsliketheIMScontrolplaneorPacketCorenowrelyoncloudarchitecturesinlargecentralizeddatacenterstoincreaseflexibilityandusehardwareresourcesmoreefficiently.However,togetthesamebenefitsforuserplaneandradioapplicationswithoutbumpingintothephysicallimitationsofthespeedoflight,computepowerneedstomovefurtherouttotheedgesofthenetwork.Thisenablesittoprovidetheextremehighbandwidthrequiredbetweentheradioequipmentandtheapplicationsortofulfilldemandsforlowlatency.

Themostcommonapproachistochoosealayeredarchitecturewithdifferentlevelsfromcentraltoregionaltoaggregatededge,orfurtherouttoaccessedgelayers.Theexactnumberoflevelswilldependonthesizeoftheoperatornetwork.Thecentrallocationsaretypicallywellequippedtohandlehighvolumesofcentralizedsignalingandareoptimizedforworkloadswhichcontrolthenetworkitself.Formoreinformationaboutsignalingworkloads,referenceChapter2.1oftheCNTTReferenceModelunderControlPlaneforalistofexamples.Toincreaseend-to-endefficiency,itisimportanttopayattentiontotheseparationofthesignalprocessingandtheusercontenttransfer.Theclosertheendusersaretothedataandsignalprocessingsystems,themoreoptimizedtheworkflowwillbeforhandlinglowlatencyandhighbandwidthtraffic.

Todescribewhatitallmeansinpractice,takeaRadioAccessNetwork(RAN)asanexample.Edgearchitecturesrequireare-thinkofthedesignoftheBaseBandUnit(BBU)component.Thiselementisusuallylocatedneararadiotowersitewithcomputationalandstoragecapabilities.Ina5Garchitecturetargetingtheedgecloud,aCloudRAN(C-RAN)approach,theBBUcanbedisaggregatedintoaCentralUnit(CU),aDistributedUnit(DU)andaRemoteRadioUnit(RRU)wheretheDUfunctionalityisoftenvirtualized(vDU)withcloseproximitytotheusers,combinedwithhardwareoffloadingsolutionstobeabletohandletrafficmoreeffectively.TheillustrationoftheaboveedgearchitectureshowshowtheCUcomponentcanbelocatedinanaggregatedorregionaledgesitewhilethevDUwouldbelocatedintheedgedatacenters.ThissetupallowsmoreflexibilityinmanagingtheCUandDUwhilekeepingthebandwidthutilizationoptimal,fulfillingtheincreasinguserdemands.

Thesearchitecturalchangesintroducenewchallengesforthelifecycleofthebuildingblocks:

Automation:tomanagetens,hundredsorthousandsofedgenodesRemoteprovisioning:allowstheoptiontoprovisioncloudinfrastructurethroughWANconnectionforsitesatremotelocations‘Singlepaneofglass’:acentraldashboardtomonitortheedgesites’statusincludingalarmsandmetricsRemoteupgrade:edgesitesareupgradedremotelywithcompatibilitybetweenthedifferentversionsofthesoftwarethroughoutthewholeinfrastructureResiliency:theabilitytorunworkloadswithoutinterruptionincaseofeventslikenetworkconnectiondisruptionbetweendatacenters

ContentCachingattheEdge

Reducingbackhaulandlatencymetricsandimprovingqualityofservice(QoS)aregoodreasonsforpushingcontentcachingandmanagementouttothenetworkedge.Acachingsystemcanbeassimpleasabasicreverse-proxyorascomplexasawholesoftwarestackthatnotonlycachescontentbutprovidesadditionalfunctionality,suchasvideotranscodingbasedontheuserequipment(UE)deviceprofile,locationandavailablebandwidth.

Contentdeliverynetworks(CDN)arenotanewconcept.However,thecreationofmoreCDNnodeswithregionalpointsofpresence(PoP)areoneofthefirstexamplesofwhatcannowbeconsiderednear-edge-computing.Withtheexplosionofvideostreaming,onlinegamingandsocialmedia,combinedwiththeroll-outof5Gmobilenetworks,theneedtopushcachingouttothefar-edgehasincreaseddramatically.The"last-mile"mustbecomeincreasinglyshortertomeetcustomerdemandforbetterperformanceanduserexperiencewiththeseapplicationsthatarehighlysensitivetonetworklatency.Thisisencouragingcontentproviderstomigratefromatraditional,regionalPoPCDNmodeltoedge-basedintelligentandtransparentcachingarchitectures.

TheParetoPrinciple,or80-20rule,appliestovideostreaming;thatis,80%ofcustomerswillonlyconsume20%oftheavailablecontent.Therefore,byonlycaching20%oftheircontent,serviceproviderswillhave80%oftrafficbeingpulledfromedgedatacenters.Thisgreatlyreducesloadonbackbonenetworkswhileimprovinguserexperience.

Cachingsystemsinedgeenvironmentsneedtotakeenduserdevice(EUD)proximity,systemloadandadditionalmetricsasfactorsindeterminingwhichedgedatacenterwilldeliverthepayloadstowhichendpoints.Inrecentprototypes,smartcachingframeworksuseanagentinthecentralcloudthatredirectscontentrequeststotheoptimumedgedatacenterusingalgorithmsbasedonmetricssuchasUElocationandloadonthegivenedgesite.

ManufacturingintheDigitalEra

Industry4.0isoftenidentifiedwiththefourthindustrialrevolution.Theconceptisthatfactoriesareusingcomputersandautomationinnewwaysbyincorporatingautonomoussystemsandmachinelearningtomakesmarterfactories.Thisparadigmshiftincludestheuseofopenhardwareandsoftwarecomponentsinthesolutions.

Factoriesareusingmoreautomationandleveragingcloudtechnologiesforflexibility,reliabilityandrobustness,whichalsoallowsforthepossibilityofintroducingnewmethodssuchasmachinevisionandlearningtoincreaseproductionefficiency.Theamountofdataprocessingandcomputationalpowerneededtosupportthesetechnologiesisincreasingbyordersofmagnitude.Manyapplicationsmovethedatafromthefactoryfloortoapublicorprivatecloud,butinmanycasesthelatencyimpactsandtransmissioncostscanleadtodisruptionsontheassemblyline.Tofulfillthehighperformanceandlowlatencycommunicationneeds,atleastsomeofthedataprocessingandfilteringneedstostaywithinthefactorynetwork,whilestillbeingabletousethecloudresourcesmoreeffectively.Furtherprocessingofthedatacollectedbyvarioussensorsisdoneinthecentralizedclouddatacenter.Reusableportablemicroserviceslocatedattheedgenodesfulfilltasksthatarepartofnewvisionapplicationsordeeplearningmechanisms.

Similarlytothetelecommunicationindustry,manufacturingalsohasverystrictrequirements.Tofulfillthecontrolsystems’real-timeandfunctionalsafetyneeds,theycanusetechnologiessuchasTimeSensitiveNetworking(TSN)onthelowerlayersofthearchitecture.

EdgeComputingforIntelligentAquaculture

Aquacultureissimilartoagriculture,exceptthatinsteadofdomesticanimals,itbreedsandharvestsfish,shellfish,algaeandotherorganismsthatliveinavarietyofsaltorfreshwaterenvironments.Theseenvironmentscanbeveryfragile;therefore,itrequireshighprecisiontocreateandsustainhealthyandbalancedecosystems.Toincreaseproductionwhileprovidingasafeandhealthyenvironmentfortheanimals,automationishighlydesirable.Thisusecaseisalsoagreatexampleofwhereequipmentisdeployedandrunninginpoorenvironmentalconditions.

Thissectiondescribesshrimpfarms,whicharecontrolledecosystemswherehumansandautomatedtoolsoverseetheentirelifecycleoftheanimalsfromthelarvaphasetothefullygrownharvestablestage.Thesystemsevenfollowthetransportationoftheshrimpaftertheyareharvested.Likeagriculture,theenvironmentalconditionshighlyaffecttheanimals’conditions,andthereforethepondsneedtobecloselymonitoredforanychangesthatmightaffectthewell-beingoftheshrimp,sothatpromptactionscanbetakentoavoidloss.

Thearchitecturediagrambelowshowsadetailedviewoftheedgedatacenterwithanautomatedsystemusedtooperateashrimpfarm.

Someofthesystemfunctionsandelementsthatneedtobetakenintoconsiderationinclude:

Environmentalmonitoring,includingdatacollectionandreportingofmetricssuchas:outdoortemperatureandhumidity,waterquality,PHvalueandtemperature,dissolvedoxygen,ammonia,nitrogen,andnitriteVideosurveillanceforillegalintrusiondetectionandcomplianceidentificationofstaffusingclothingandfacerecognitionSmartbreedingthatincludesautomatedfeedingandinventorytracking(food,medicine,auxiliarymaterials,disinfectantandsoforth)Platformtraceabilitytoqueryanddisplaytheentiresupplychaintoensurehighqualityofaquaticproducts

Byautomatingandconnectingthesefarms,thesolutionminimizestheisolationthatexistsinthisindustry.Theplatformprovidesdatatobecollectedandanalyzedbothlocallyonthefarmsandcentrallytoimprovetheenvironmentalconditionsandpreventmistakeswhileusingchemicalslikeauxiliarymaterialsanddisinfectants.

Withmorecomputationalpowerattheedgedatacenters,itispossibletostoreandanalyzelocalmonitoringdataforfasterreactiontimetomanagechangesinenvironmentalconditionsormodifyfeedingstrategy.Thesystemcanalsopre-filterdatabeforesendingittothecentralcloudforfurtherprocessing.Forinstance,thesystemcanpre-processwaterqualitydatafromthemonitoringsensorsandsendstructuredinformationbacktothecentralcloud.Thelocalnodecanprovidemuchfasterfeedbackcomparedtoperformingalloperationsinthecentralcloudandsendinginstructionsbacktotheedgedatacenters.

Digitalizationhasalreadyprovidedmuchinnovation,butthereisstillroomforimprovement,suchasreducingthelaborcostsrelatedtocollectingdataandimprovingdataanalysistobefasterandmorereliable.Withedgecomputingtechniques,itispossibletobuildintelligentaquacultureinfrastructureinordertointroduceartificialintelligenceandmachinelearningtechniquesthatwilloptimizefeedingstrategyorreducecostbyminimizinghumanerrorandreactingfastertomachinefailures.

TechnologyConsiderations

Ascanbeseenfromthesefewusecases,therearebothcommonchallengesandfunctionalitythatbecomeevenmorecrucialinedgeandhybridenvironments.Asusecasesevolveintomoreproductiondeployments,thecommoncharacteristicsandchallengesoriginallydocumentedinthe“CloudEdgeComputing:BeyondtheDataCenter”whitepaperremainrelevant.

Thehighestfocusisstillonreducinglatencyandmitigatingbandwidthlimitations.Furthersimilaritybetweenthedifferentusecases,regardlessoftheindustrytheyarein,istheincreaseddemandforfunctionslikemachinelearningandvideotranscodingontheedge.Duetothethroughputdemandsofapplicationsliketheseandworkloadssuchasvirtualnetworkfunctions(VNF)for5G,variousoffloadingandaccelerationtechnologiesarebeingleveragedtoboostperformancethroughsoftwareandhardware,suchas:

Single-rootinput/outputvirtualization(SR-IOV):ThistechnologyallowsVMsandcontainerstosharedirectaccesstoadistinctpartofadevice,suchasnetworkadapters,usingthePCIExpressinterface.DataPlaneDevelopmentKit(DPDK):DPDKallowshighernetworkpacketthroughputbyusingoffloadingand

schedulingtechniques.Whilethisisnotatechnologyspecificforedgeitiscrucialtoenabletheoptiontouseittofulfillstrictrequirementsinhighlyresourceconstrainedenvironments.Non-uniformmemoryaccess(NUMA):Thisisanothermethodtoincreasethroughputbyallocatingdedicatedmemoryblockstoaninstance.ForfurtheroptimizationthememoryblockislocaltotheCPUcoreonwhichtheinstanceisworking.Forverysmalledgesitesthiscouldbecomeabottleneck,dependingontheedgesiteworkloadmix.SmartNics/Field-programmablegatearray(FPGA):ItisahardwareaccelerationoptionthatisalreadyusedforvRANdeploymentstoincreasetheperformanceofsiteswithcompute-intensiveworkloads.TheFPGAunitsareprogrammedwithworkload-specificsoftwaretooffloadtheexecutionofsomeapplication-specificalgorithms.GraphicsProcessingUnit(GPU):GPUshaveahigh-numberofcoreswhichmaybeutilizedbyawidevarietyofparallel-processingintensiveworkloadssuchasMapReduce,machinelearning(ML),IoTandgaming.WhileusingGPUsisagoodwaytoincreasethesystemperformancewheretheworkloaddemandsit,itintroducesthequestionofcostconstraints,especiallyifthenumberofedgesitesstartstogrow.

ReferenceArchitectures

Architecturedesignisalwaysspecifictotheusecase,takingintoaccountalltheneedsofthegivenplannedworkloadandfinetuningtheinfrastructureondemand.Asdiscussedearlier,thereisnosinglesolutionthatwouldfulfilleveryneed.However,therearecommonmodelsthatdescribehigh-levellayoutswhichbecomeimportantforday-2operationsandtheoverallbehaviorofthesystems.

Beforegoingintodetailabouttheindividualsitetypeconfigurations,thereisadecisionthatneedstobemadeonwheretolocatethedifferentinfrastructureservices’controlfunctionsandhowtheyneedtobehave.Thesemodelsanddecisionsarenotspecifictothetechnologiesnordotheydependontheparticularsoftwaresolutionchosen.

Theusecasesinthisdocumentaremostlyenvisionedasaspiderwebtypeofarchitecturewithhierarchyautomaticallyabletoscalethenumberofendpoints.Dependingonneeds,therearechoicesonthelevelofautonomyateachlayerofthearchitecturetosupport,manageandscalethemassivelydistributedsystems.Thenetworkconnectivitybetweentheedgenodesrequiresafocusonavailabilityandreliability,asopposedtobandwidthandlatency.

Thissectioncoverstwocommonhigh-levelarchitecturemodelsthatshowthetwodifferentapproaches.TheyaretheCentralizedControlPlaneandtheDistributedControlPlanemodels.Sincethisisahigh-leveldiscussion,theassumptionisthattherewillbeenoughcompute,storageandnetworkingfunctionalitytotheedgetocoverthebasicneeds;anyspecializedconfigurationsorfeaturesareoutofscope.ThearchitecturemodelsalsoshowrequiredfunctionalityforeachsitebutdonotdiscusshowtorealizeitwithanyspecificsolutionsuchasKubernetes,OpenStack,andsoforth.However,aspectsandtoolsthatwereconsideredduringthedevelopmentofthemodelsinclude:

Challengesofmanagingalargenumberofedgedatacenters:Availablefunctionalityattheedgedatacentervs.orchestrationoverheadPreparingthearchitecturetohandleonefailureatatime:e.g.:NetworkconnectionlossordegradationtothecentralorregionaldatacenterProvidingminimalviablefunctionalityonsmallfootprints

Thereareotherstudiesthatcoversimilararchitecturalconsiderationsandholdsimilarcharacteristicswithoutbeingfullyalignedwithonemodelortheother.Forinstance,arecentstudypresentsadisruptiveapproachconsistingofrunningstandaloneOpenStackinstallationsindifferentgeographicallocationswithcollaborationbetweenthemondemand.Theapproachdeliverstheillusionofasingleconnectedsystemwithoutrequiringintrusivechanges.

DiscussinganddevelopingadditionaldetailsaroundtherequirementsandsolutionsinintegratingstoragesolutionsandfurthernewcomponentsintoedgearchitecturesispartofthefutureworkoftheOSFEdgeComputingGroup.

CentralizedControlPlane

FortheCentralizedControlPlanemodel,theedgeinfrastructureisbuiltasatraditionalsingledatacenterenvironmentwhichisgeographicallydistributedwithWANconnectionsbetweenthecontrollerandcomputenodes.Ifadistributednodebecomesdisconnectedfromtheothernodes,thereisariskthattheseparatednodemightbecomenon-functional.

Duetotheconstraintsofthismodel,thenodesrelyheavilyonthecentralizeddatacentertocarrytheburdenofmanagementandorchestrationoftheedgecompute,storageandnetworkingservicesbecausetheyrunallthecontrollerfunctions.Computeservicesincorporaterunningbaremetal,containerizedandvirtualizedworkloadsalike.Relatedfunctionswhichareneededtoexecutetheworkloadoftheinfrastructurearedistributedbetweenthecentralandtheedgedatacenters.

Thediagramaboveshowsthatallofthekeycontrolfunctionalityislocatedinthecentralsite,includingallidentitymanagementandorchestrationfunctions.Ifyousetasidethegeographicallydistributednature,thisapproachfacesverysimilarchallengesasoperatinglarge-scaledatacenters.Ontheplusside,itprovidesacentralizedviewoftheinfrastructureasawhole,whichhasitsadvantagesfromanoperationalperspective.

Whilethemanagementandorchestrationservicesarecentralized,thisarchitectureislessresilienttofailuresfromnetworkconnectionloss.Theedgedatacenterdoesn'thavefullautonomy,thereforedistributingconfigurationchangesmightfailifthereislostaccesstotheimagelibraryortheidentitymanagementservice.Theconfigurationneedstoallowapplicationstocontinuerunningevenincaseofnetworkoutagesiftheusecaserequirestheworkloadtobehighlyavailable,i.e.aPointofSalessysteminaretaildeploymentortheindustrialrobotsoperatinginanIoTscenario.Thiscanbechallengingbecausemostdatacentercentricdeploymentstreatcomputenodesasfailedresourceswhentheybecomeunreachable.InadditiontheIdentityProvider(IdP)servicecaneitherbeplacedinthecentraldatacenterorremotelywithconnectiontotheidentitymanagementservicewhichlimitsusermanagementandauthentication.Dependingonthesituation,thismightbeconsideredmoresecureduetothecentralizedcontrollers,orlessflexiblebecauseitmightmeanlostaccessbyusersatacriticaljuncture.

Typically,buildingsucharchitecturesusesexistingsoftwarecomponentsasbuildingblocksfromwell-knownprojectssuchasOpenStackandKubernetes.SomeedgesitesmightonlyhavecontainerizedworkloadswhileothersitesmightberunningVMs.ItisrecommendedtoreviewtheDistributedComputeNode(DCN)deploymentconfigurationofTripleOwhichisalignedwiththismodel.

Insummary,thisarchitecturemodeldoesnotfulfilleveryusecase,butitprovidesanevolutionpathtoalreadyexistingarchitectures.Plus,italsosuitstheneedsofscenarioswhereautonomousbehaviorisnotarequirement.

DistributedControlPlane

Alargersetofusecasesdemandsedgesitestobemorefullyfunctionalontheirown.Thismeanstheyaremoreresilienttonetworkconnectivityissuesaswellasbeingabletominimizedisruptioncausedbylatencybetweenedgesites.

TheDistributedControlPlanemodeldefinesanarchitecturewherethemajorityofthecontrolservicesresideonthelarge/mediumedgedatacenters.Thisprovidesanorchestrationaloverheadtosynchronizebetweenthesedatacentersandmanagethemindividuallyandaspartofalarger,connectedenvironmentatthesametime.

Therearedifferentoptionsthatcanbeusedtoovercometheoperationalchallengesofthismodel.Onemethodistousefederationtechniquestoconnectthedatabasestooperatetheinfrastructureasawhole;anotheroptionistosynchronizethedatabasesacrosssitestomakesuretheyhavethesameworkingsetofconfigurationsacrossthedeployment.Thismodelstillallowsfortheexistenceofsmalledgedatacenterswithsmallfootprintswheretherewouldbealimitedamountofcomputeservices,andthepreferencewouldbetodevotethemajorityoftheavailableresourcestotheworkloads.

Themostcommonexampleiswhenthelocationofthecomponentsoftheidentitymanagementservicearechosenbasedonthescenarioalongwithoneoftheaforementionedmethodstoconnectthem.Thechoicedependsonthecharacteristicsoftheindividualusecaseandthecapabilitiesofthesoftwarecomponentsused,becausetheoverallbehaviorandmanagementofeachconfigurationisdifferent.Forinstance,usingtheOpenStackIdentityManagementservice(Keystone)tolocateitintoanedgedeploymentwithoutthelimitationoftechnologiesasitsAPIsupportsbothOpenStackandKubernetesorthecombinationofboth.

Thisarchitecturemodelismuchmoreflexibleincaseofanetworkconnectionlossbecausealltherequiredservicestomodifytheworkloadsorperformusermanagementoperationsareavailablelocally.Therearestillpotentialobstacles,suchasnothavingalltheimagesavailablelocallyduetolimitationsofstorageandcachesizes.Therearealsonewchallengesduetotheadditionalburdenofrunningalargenumberofcontrolfunctionsacrossageographicallydistributedenvironmentthatmakesmanagingtheorchestrationtypeservicesmorecomplex.

Asinthepreviouscase,thisarchitecturesupportsacombinationofOpenStackandKubernetesservicesthatcanbedistributedintheenvironmenttofulfillalltherequiredfunctionalityforeachsite.AnexampleofthisisStarlingX,asitsarchitecturecloselyresemblesthedistributedmodel.

Therearehybridsolutionsonthemarketthattrytoleveragethebestofbothworldsbydeployingfullinstallationsinthecentralnodesaswellaslarge/mediumedgedatacentersandhaveanorchestrationtypeserviceontop,suchasONAP,anorchestrationtoolusedinthetelecomindustry.

Futurearchitecturalconsiderations

Theabovedescribedmodelsarestillunderdevelopmentasmoreneedsandrequirementsaregatheredinspecificareas,suchas:

Storage:Considerationsincludelocalstoragetoenablehighperformanceandlowlatencyprocessingofdataaswellasprovidingoptionstoconnecttoremotestoragesolutions.Severalalternativesareavailable,rangingfromsmallandsimplesystemslikesoftwareRAIDorLVMtolargeandhighlyreliabledistributedstoragemanagerslike

Ceph.Baremetalmanagement:Itcanbeintroducedonmultiplelayers,onebeingtheinfrastructureoperatorinneedofmanagingandscalingtheirinfrastructuretoincludezerotouchprovisioningmethods,andtheotherbeingtheuseroftheinfrastructurewhomaygetthepermissionandoptiontocreatenewedgesitesondemand.

Thereareotherstudiesthatcoversimilararchitecturalconsiderationsandholdsimilarcharacteristicswithoutbeingfullyalignedwithonemodelortheother.Forinstance,arecentstudypresentsadisruptiveapproachconsistingofrunningstandaloneOpenStackinstallationsindifferentgeographicallocationswithcollaborationbetweenthemondemand.Theapproachdeliverstheillusionofasingleconnectedsystemwithoutrequiringintrusivechanges.

DiscussinganddevelopingadditionaldetailsaroundtherequirementsandsolutionsinintegratingstoragesolutionsandfurthernewcomponentsintoedgearchitecturesispartofthefutureworkoftheOSFEdgeComputingGroup.

Testingconsiderations

Definingcommonarchitecturesforedgesolutionsisacomplicatedchallengeinitself,butitisonlythebeginningofthejourney.Thenextstepistobeabletodeployandtestthesolutiontoverifyandvalidateitsfunctionalityandensureitperformsasexpected.Astheedgearchitecturesarestillintheearlyphase,itisimportanttobeabletoidentifyadvantagesanddisadvantagesofthecharacteristicsforeachmodeltodeterminethebestfitforagivenusecase.

ThebuildingblocksarealreadyavailabletocreateedgedeploymentsforOpenStackandKubernetes.Thesearebothopensourceprojectswithextensivetestingeffortsthatareavailableinanopenenvironment.Whileitiscommontoperformfunctionalandintegrationtestingaswellasscalabilityandrobustnesschecksonthecodebase,thesedeploymentsrarelygetextendedbeyondoneormaybeafewphysicalservers.Inthecaseofedgearchitecturesitiscrucialtocheckfunctionalitythatisdesignedtoovercomethegeographicaldistributionoftheinfrastructure,especiallyinthecircumstancewheretheconfigurationsofthearchitecturalmodelsarefundamentallydifferent.Inordertoensurestableandtrustableoutcomesitisrecommendedtolookintothebestpracticesofthescientificcommunitytofindthemostrobustsolution.Onecommonstandardpracticeistheartifactreviewandbadgingapproach.

Testingisasmuchanartformasitisapreciseengineeringprocess.Testingcodeonlowerlevels,suchasunittestsorcheckingresponsesofcomponentsthroughAPItests,isstraightforward.Thisallowsframeworkstobecreatedthatsupportrunninganautomatedunittestsuitethataddressesrequirementssuchasrepeatability,replicabilityandreproducibility.Testingtheintegratedsystemstoemulatetheconfigurationandcircumstancesofproductionenvironmentscanbequitechallenging.Thediagrambelowdescribesthegeneralprocessthatisexecutedwhenperformingexperimentalcampaigns.Thisprocess,thatisappliedinthefieldofresearch,canalsobeutilizedtohelpbuildnewcomponentsandsolutionsthatfittherequirementsofedgecomputingusecaseseventhoughsomeofthestepsstillneedmoretoolstoperformallchecksasiftheyweresimpleunittests.

Thefirstseeminglytrivialstepdescribingtheacquisitionofresourcesfromatestbedisnotspecifictoedgecomputingscenarios.Theassignedresources(e.g.,compute,storage,network)representthephysicalinfrastructurethatwillbeusedtoconducttheevaluation.

Thesecondphaseismoredifficult.Itincorporatesmultiplesubstepstopreparethephysicalinfrastructureaswellasthedeploymentofthesystemundertest(SUT).Asedgeenvironmentscanbeverycomplex,theyalsoneedtobetestedfortheirabilitytobepreparedforcircumstancessuchasanunreliablenetworkconnection.Therefore,havingadeploymenttoolthatsupportsadeclarativeapproachispreferredtospecifythecharacteristicsoftheinfrastructuresuchaslatency,throughputandnetworkpacketlossratiotoemulatethetargetedreallifescenarioandcircumstances.

Oncethedeploymentplanhasbeencreatedandtheresourceshavebeenselected,itneedstobeconfirmedthattheinfrastructureisconfiguredcorrectlyduringthepre-deploymentphasebeforeinstallingtheapplicationsandservicesontop.Thisisespeciallytrueinedgearchitectureswhereresourcesmustbeavailableovercomplexnetworkingtopologies.Forinstance,profileattributesmayhaveallbeensetcorrectly,butarealltheresourcesreachable,ingoodhealth,andcancommunicatetoeachotherasexpected?Thecheckscanbeassimpleasusingthepingcommandbi-directionally,verifyingspecificnetworkportstobeopenandsoforth.Thepurposeofthisprocedureistoensurethatthedeploymentstepwillbecompletedsuccessfullyandresultinatestenvironmentthatisalignedwiththerequirementsandplans.Thecomplexityofedgearchitecturesoftendemandsagranularandrobustpre-deploymentvalidationframework.

Nowthatthetestbedispreparedandtested,thenextstepistodeploythesoftwareapplicationsontheinfrastructure.ForsystemsbuiltonenvironmentssuchasOpenStackandKubernetesservices,frameworkslikeKolla,TripleO,KubesprayorAirshipareavailableasstartingpoints.Beawarethatthemajorityofthesetoolsaredesignedwiththelimitationsofonedatacenterastheirscope,whichmeansthatthereisanassumptionthattheenvironmentcanscalefurtherduringoperation,whileedgeinfrastructuresaregeographicallydistributedandoftenhavelimitedresourcesintheremotenodes.Inaddition,theconfigurationoptionsaresignificantlydifferentamongthedifferentmodels.Aspartoftestingedgearchitectures,thedeploymenttoolsneedtobevalidatedtoidentifytheonesthatcanbeadaptedandreusedforthesescenarios.Toensurethesuccessoftesting,theinstallationitselfneedstobeverified,forinstance,checkingtheservicestoensuretheywereinstalledandconfiguredcorrectly.Thisoperationshouldpreferablybeafunctionalityofthedeploymenttool.

Whenallthepreparationsaredone,thenextstepisbenchmarkingtheentireintegratedframework.Benchmarkingisoftendefinedasperformancetesting,buthereitappliestoabroaderscopethatincludesintegrationandfunctionaltestingaswell.Itisalsoimportanttonotethatthetestsuitescanbeheavilydependentontheusecase,sotheyneedtobefinetunedforthearchitecturemodelbeingused.Whileafewtoolsexisttoperformnetworktrafficshapingandfaultinjections,thechallengeliesmoreintheidentificationofvaluesthatarerepresentativetotheaforementionededgeusecases.

Buildinganedgeinfrastructureconsistsofvariouswellknowncomponentsthatwerenotimplementedspecificallyforedgeusecasesoriginally.Becauseofthat,therearesituationswheretherewillbeaneedtotestbasicfunctionalityintheseenvironmentsaswelltomakesuretheyworkasexpectedinotherscenarios.Examplefunctionsinclude:

create/deletearesource(user,flavor,image,etc);scope:oneormoreedgesiteslistinstances(VM,container);scope:anedgesiteor‘singlepaneofglass’dashboardcreateresourcesforcross-data-centernetworks

Furthertestingoftheedgeinfrastructureneedstotakethechoiceofarchitecturalmodelintoconsideration:

UsingOpenStackinthecentralizedcontrolplanemodeldependsonthedistributedvirtualrouter(DVR)featureoftheOpenStackNetworkConnectivityasaService(Neutron)component.Thebehavioroftheedgedatacentersincaseofanetworkconnectionlossmightbedifferentbasedonthearchitecturalmodels.Insomecases,thedecisionmightbetochoosetoconfigurethesystemtokeeptheinstancesrunningwhileinothercases,therightapproachwouldbetodestroytheworkloadsincasethesitebecomesisolated.Inadditiontotheseconsiderations,theexpectationsonfunctionssuchasauto-scalingwillalsobedifferentduetopossibleresourceconstraints,whichneedtobereflectedinthetestsuitesaswell.

Thefinaltwostepsaretrivial.Thetestresultsneedtobecollectedandevaluated,beforereturningtheSUTinfrastructuretoitsoriginalstate.

ToolssuchasEnos,Enos-Kubernetesandenoslibareavailableintheexperiment-drivenresearchcommunitytoevaluateOpenStackandKubernetesinadistributedenvironmentoverWideAreaNetwork(WAN)connection.Theycanbeextendedorleveragedasexamplesofsolutionsthatcanbeusedtoperformtheabovedescribedprocesstoevaluatesomeofthearchitectureoptionsforedge.Furthercomponentsareneededtoensuretheabilitytotestmorecomplexenvironmentswheregrowingnumbersofbuildingblocksareintegratedwitheachother.

Conclusion

Edgecomputingishighlydependentonlessonslearnedandsolutionsimplementedinthecloud.Evenifthemajorityofbuildingblocksareavailabletocreateanenvironmentthatfulfillsmostrequirements,manyofthesecomponentsneedfinetuningorAPIextensionstoprovideamoreoptimizedandfitforpurposesolution.Deploymentandtestingrequirementsarefurtherhighlightedforthesenewarchitecturalconsiderations,andthereforeexistingsolutionsneedtobeenhanced,customizedandinsomecasesdesignedandimplementedfromscratch.

Therealchallengeliesinefficientandthoroughtestingofthenewconceptsandevolvingarchitecturemodels.Newtestcasesneedtobeidentifiedalongwithvaluesthatarerepresentativetotypicalcircumstancesandsystemfailures.Testingcanhelpwithbothenhancingarchitecturalconsiderationsaswellasidentifyingshortcomingsofdifferentsolutions.

Ascanbeseenfromthesediscussions,edgecomputingrelatedinnovationandsoftwareevolutionisstillverymuchinitsearlystages.Yes,therearesystemsrunninginproductionthatresembleatleastsomeoftheconsiderations—uCPEorvRANdeployments,forexample.Thearchitecturemodelsdiscussedherecoverthemajorityoftheusecases,however,theystillneedadditionaleffortstodetailtherequiredfunctionalitytogobeyondthebasics,outlinefurtherpreferablesolutionsanddocumentbestpractices.

ThisistheperfecttimeforgroupsintheITindustry,bothopengroupsandsemi-openorclosedconsortiums,aswellasstandardizationbodies,tocollaborateontakingthenextstepsforarchitecturedesignandtestinginordertobeabletoaddresstheneedsofthevariousedgecomputingusecases.Consideringthehighlevelofintegrationneeded,itiscrucialthatthesubjectmatterexpertsofthevariouscomponentsstarttocontributetoacommoneffort.

Authors

BethCohen,DistinguishedMemberofTechnicalStaff,VerizonGergelyCsatári,SeniorOpenSourceSpecialist,NokiaShuquanHuang,TechnicalDirector,99CloudBruceJones,StarlingXArchitect&ProgramManager,IntelCorp.AdrienLebre,ProfessorinComputerScience,IMTAtlantique/Inria/LS2NDavidPaterson,Sr.PrincipalSoftwareEngineer,DellTechnologiesIldikóVáncsa,EcosystemTechnicalLead,OpenStackFoundation