Embed Size (px)
Citation preview
EdgeComputingNextStepsinArchitecture,DesignandTestingEdgeComputing:NextStepsinArchitecture,DesignandTesting
Introduction
Whileedgecomputinghasrapidlygainedpopularityoverthepastfewyears,therearestillcountlessdebatesaboutthedefinitionofrelatedtermsandtherightbusinessmodels,architecturesandtechnologiesrequiredtosatisfytheseeminglyendlessnumberofemergingusecasesofthisnovelwayofdeployingapplicationsoverdistributednetworks.
InourpreviouswhitepapertheOSFEdgeComputingGroupdefinedcloudedgecomputingasresourcesandfunctionalitydeliveredtotheendusersbyextendingthecapabilitiesoftraditionaldatacentersouttotheedge,eitherbyconnectingeachindividualedgenodedirectlybacktoacentralcloudorseveralregionaldatacenters,orinsomecasesconnectedtoeachotherinamesh.Fromabird’seyeview,mostofthoseedgesolutionslooklooselylikeinterconnectedspiderwebsofvaryingsizesandcomplexity.
Inthesetypesofinfrastructures,thereisnoonewelldefinededge;mostoftheseenvironmentsgroworganically,withthepossibilityofdifferentorganizationsowningthevariouscomponents.Forexample,apubliccloudprovidermightsupplysomeofthecoreinfrastructure,whileothervendorsaresupplyingthehardware,andyetathirdsetofintegratorsarebuildingthesoftwarecomponents.Tryingtocreateaonesizefitsallsolutionisimpossibleforedgeusecasesduetotheverydifferentapplicationneedsinvariousindustrysegments.Interestingly,whilecloudtransformationstartedlaterinthetelecomindustry,operatorshavebeenpioneersintheevolutionofcloudcomputingouttotheedge.Asownersofthenetwork,telecominfrastructureisakeyunderlyingelementinedgearchitectures.
Afterfouryears,whilethereisnoquestionthatthereiscontinuinginterestinedgecomputing,thereislittleconsensusonastandardedgedefinition,solutionorarchitecture.Thatdoesn’tmeanthatedgeisdead.Edgemustbebyitsverynaturehighlyadaptable.Adaptabilityiscrucialtoevolveexistingsoftwarecomponentstofitintonewenvironmentsorgivethemelevatedfunctionality.Edgecomputingisatechnologyevolutionthatisnotrestrictedtoanyparticularindustry.Asedgeevolves,moreindustriesfinditrelevant,whichonlybringsfreshrequirementsorgivesexistingonesdifferentcontexts,attractingnewpartiestosolvethesechallenges.Nowmorethanever,edgecomputinghasthepromiseforaverybrightfutureindeed!
ThisdocumenthighlightstheOSFEdgeComputingGroup’sworktomorepreciselydefineandtestthevalidityofvariousedgereferencearchitectures.Tohelpwithunderstandingthechallenges,thereareusecasesfromavarietyofindustrysegments,demonstratinghowthenewparadigmsfordeployinganddistributingcloudresourcescanusereferencearchitecturemodelsthatsatisfytheserequirements.
Challengesindifferentindustries
Inanutshell,edgecomputingmovesmorecomputationalpowerandresourcesclosertoendusersbyincreasingthenumberofendpointsandlocatingthemnearertotheconsumers--betheyusersordevices.Fundamentally,edgecomputingarchitecturesarebuiltonexistingtechnologiesandestablishedparadigmsfordistributedsystems,whichmeansthattherearemanywellunderstoodcomponentsavailabletocreatethemosteffectivearchitecturestobuildanddeliveredgeusecases.
Thissectionwillguideyouthroughsomeusecasestodemonstratehowedgecomputingappliestodifferentindustriesandhighlightthebenefitsitdelivers.Wewillalsoexploresomeofthedifferentiatingrequirementsandwaystoarchitectthesystemssotheydonotrequirearadicallynewinfrastructurejusttocomplywiththerequirements.
5GBringsYoutheEdgeorViceVersa?
5Gtelecomnetworkspromiseextrememobilebandwidth,buttodeliver,theyrequiremassivenewandimprovedcapabilitiesfromthebackboneinfrastructurestomanagethecomplexities,includingcriticaltrafficprioritization.Thenetworkneedstoprovidebothhighthroughputandlowlatencycombinedwithefficientuseoftheavailablecapacityinordertosupporttheperformancedemandsoftheemerging5Gofferings.
SignalingfunctionsliketheIMScontrolplaneorPacketCorenowrelyoncloudarchitecturesinlargecentralizeddatacenterstoincreaseflexibilityandusehardwareresourcesmoreefficiently.However,togetthesamebenefitsforuserplaneandradioapplicationswithoutbumpingintothephysicallimitationsofthespeedoflight,computepowerneedstomovefurtherouttotheedgesofthenetwork.Thisenablesittoprovidetheextremehighbandwidthrequiredbetweentheradioequipmentandtheapplicationsortofulfilldemandsforlowlatency.
Themostcommonapproachistochoosealayeredarchitecturewithdifferentlevelsfromcentraltoregionaltoaggregatededge,orfurtherouttoaccessedgelayers.Theexactnumberoflevelswilldependonthesizeoftheoperatornetwork.Thecentrallocationsaretypicallywellequippedtohandlehighvolumesofcentralizedsignalingandareoptimizedforworkloadswhichcontrolthenetworkitself.Formoreinformationaboutsignalingworkloads,referenceChapter2.1oftheCNTTReferenceModelunderControlPlaneforalistofexamples.Toincreaseend-to-endefficiency,itisimportanttopayattentiontotheseparationofthesignalprocessingandtheusercontenttransfer.Theclosertheendusersaretothedataandsignalprocessingsystems,themoreoptimizedtheworkflowwillbeforhandlinglowlatencyandhighbandwidthtraffic.
Todescribewhatitallmeansinpractice,takeaRadioAccessNetwork(RAN)asanexample.Edgearchitecturesrequireare-thinkofthedesignoftheBaseBandUnit(BBU)component.Thiselementisusuallylocatedneararadiotowersitewithcomputationalandstoragecapabilities.Ina5Garchitecturetargetingtheedgecloud,aCloudRAN(C-RAN)approach,theBBUcanbedisaggregatedintoaCentralUnit(CU),aDistributedUnit(DU)andaRemoteRadioUnit(RRU)wheretheDUfunctionalityisoftenvirtualized(vDU)withcloseproximitytotheusers,combinedwithhardwareoffloadingsolutionstobeabletohandletrafficmoreeffectively.TheillustrationoftheaboveedgearchitectureshowshowtheCUcomponentcanbelocatedinanaggregatedorregionaledgesitewhilethevDUwouldbelocatedintheedgedatacenters.ThissetupallowsmoreflexibilityinmanagingtheCUandDUwhilekeepingthebandwidthutilizationoptimal,fulfillingtheincreasinguserdemands.
Thesearchitecturalchangesintroducenewchallengesforthelifecycleofthebuildingblocks:
Automation:tomanagetens,hundredsorthousandsofedgenodesRemoteprovisioning:allowstheoptiontoprovisioncloudinfrastructurethroughWANconnectionforsitesatremotelocations‘Singlepaneofglass’:acentraldashboardtomonitortheedgesites’statusincludingalarmsandmetricsRemoteupgrade:edgesitesareupgradedremotelywithcompatibilitybetweenthedifferentversionsofthesoftwarethroughoutthewholeinfrastructureResiliency:theabilitytorunworkloadswithoutinterruptionincaseofeventslikenetworkconnectiondisruptionbetweendatacenters
ContentCachingattheEdge
Reducingbackhaulandlatencymetricsandimprovingqualityofservice(QoS)aregoodreasonsforpushingcontentcachingandmanagementouttothenetworkedge.Acachingsystemcanbeassimpleasabasicreverse-proxyorascomplexasawholesoftwarestackthatnotonlycachescontentbutprovidesadditionalfunctionality,suchasvideotranscodingbasedontheuserequipment(UE)deviceprofile,locationandavailablebandwidth.
Contentdeliverynetworks(CDN)arenotanewconcept.However,thecreationofmoreCDNnodeswithregionalpointsofpresence(PoP)areoneofthefirstexamplesofwhatcannowbeconsiderednear-edge-computing.Withtheexplosionofvideostreaming,onlinegamingandsocialmedia,combinedwiththeroll-outof5Gmobilenetworks,theneedtopushcachingouttothefar-edgehasincreaseddramatically.The"last-mile"mustbecomeincreasinglyshortertomeetcustomerdemandforbetterperformanceanduserexperiencewiththeseapplicationsthatarehighlysensitivetonetworklatency.Thisisencouragingcontentproviderstomigratefromatraditional,regionalPoPCDNmodeltoedge-basedintelligentandtransparentcachingarchitectures.
TheParetoPrinciple,or80-20rule,appliestovideostreaming;thatis,80%ofcustomerswillonlyconsume20%oftheavailablecontent.Therefore,byonlycaching20%oftheircontent,serviceproviderswillhave80%oftrafficbeingpulledfromedgedatacenters.Thisgreatlyreducesloadonbackbonenetworkswhileimprovinguserexperience.
Cachingsystemsinedgeenvironmentsneedtotakeenduserdevice(EUD)proximity,systemloadandadditionalmetricsasfactorsindeterminingwhichedgedatacenterwilldeliverthepayloadstowhichendpoints.Inrecentprototypes,smartcachingframeworksuseanagentinthecentralcloudthatredirectscontentrequeststotheoptimumedgedatacenterusingalgorithmsbasedonmetricssuchasUElocationandloadonthegivenedgesite.
ManufacturingintheDigitalEra
Industry4.0isoftenidentifiedwiththefourthindustrialrevolution.Theconceptisthatfactoriesareusingcomputersandautomationinnewwaysbyincorporatingautonomoussystemsandmachinelearningtomakesmarterfactories.Thisparadigmshiftincludestheuseofopenhardwareandsoftwarecomponentsinthesolutions.
Factoriesareusingmoreautomationandleveragingcloudtechnologiesforflexibility,reliabilityandrobustness,whichalsoallowsforthepossibilityofintroducingnewmethodssuchasmachinevisionandlearningtoincreaseproductionefficiency.Theamountofdataprocessingandcomputationalpowerneededtosupportthesetechnologiesisincreasingbyordersofmagnitude.Manyapplicationsmovethedatafromthefactoryfloortoapublicorprivatecloud,butinmanycasesthelatencyimpactsandtransmissioncostscanleadtodisruptionsontheassemblyline.Tofulfillthehighperformanceandlowlatencycommunicationneeds,atleastsomeofthedataprocessingandfilteringneedstostaywithinthefactorynetwork,whilestillbeingabletousethecloudresourcesmoreeffectively.Furtherprocessingofthedatacollectedbyvarioussensorsisdoneinthecentralizedclouddatacenter.Reusableportablemicroserviceslocatedattheedgenodesfulfilltasksthatarepartofnewvisionapplicationsordeeplearningmechanisms.
Similarlytothetelecommunicationindustry,manufacturingalsohasverystrictrequirements.Tofulfillthecontrolsystems’real-timeandfunctionalsafetyneeds,theycanusetechnologiessuchasTimeSensitiveNetworking(TSN)onthelowerlayersofthearchitecture.
EdgeComputingforIntelligentAquaculture
Aquacultureissimilartoagriculture,exceptthatinsteadofdomesticanimals,itbreedsandharvestsfish,shellfish,algaeandotherorganismsthatliveinavarietyofsaltorfreshwaterenvironments.Theseenvironmentscanbeveryfragile;therefore,itrequireshighprecisiontocreateandsustainhealthyandbalancedecosystems.Toincreaseproductionwhileprovidingasafeandhealthyenvironmentfortheanimals,automationishighlydesirable.Thisusecaseisalsoagreatexampleofwhereequipmentisdeployedandrunninginpoorenvironmentalconditions.
Thissectiondescribesshrimpfarms,whicharecontrolledecosystemswherehumansandautomatedtoolsoverseetheentirelifecycleoftheanimalsfromthelarvaphasetothefullygrownharvestablestage.Thesystemsevenfollowthetransportationoftheshrimpaftertheyareharvested.Likeagriculture,theenvironmentalconditionshighlyaffecttheanimals’conditions,andthereforethepondsneedtobecloselymonitoredforanychangesthatmightaffectthewell-beingoftheshrimp,sothatpromptactionscanbetakentoavoidloss.
Thearchitecturediagrambelowshowsadetailedviewoftheedgedatacenterwithanautomatedsystemusedtooperateashrimpfarm.
Someofthesystemfunctionsandelementsthatneedtobetakenintoconsiderationinclude:
Environmentalmonitoring,includingdatacollectionandreportingofmetricssuchas:outdoortemperatureandhumidity,waterquality,PHvalueandtemperature,dissolvedoxygen,ammonia,nitrogen,andnitriteVideosurveillanceforillegalintrusiondetectionandcomplianceidentificationofstaffusingclothingandfacerecognitionSmartbreedingthatincludesautomatedfeedingandinventorytracking(food,medicine,auxiliarymaterials,disinfectantandsoforth)Platformtraceabilitytoqueryanddisplaytheentiresupplychaintoensurehighqualityofaquaticproducts
Byautomatingandconnectingthesefarms,thesolutionminimizestheisolationthatexistsinthisindustry.Theplatformprovidesdatatobecollectedandanalyzedbothlocallyonthefarmsandcentrallytoimprovetheenvironmentalconditionsandpreventmistakeswhileusingchemicalslikeauxiliarymaterialsanddisinfectants.
Withmorecomputationalpowerattheedgedatacenters,itispossibletostoreandanalyzelocalmonitoringdataforfasterreactiontimetomanagechangesinenvironmentalconditionsormodifyfeedingstrategy.Thesystemcanalsopre-filterdatabeforesendingittothecentralcloudforfurtherprocessing.Forinstance,thesystemcanpre-processwaterqualitydatafromthemonitoringsensorsandsendstructuredinformationbacktothecentralcloud.Thelocalnodecanprovidemuchfasterfeedbackcomparedtoperformingalloperationsinthecentralcloudandsendinginstructionsbacktotheedgedatacenters.
Digitalizationhasalreadyprovidedmuchinnovation,butthereisstillroomforimprovement,suchasreducingthelaborcostsrelatedtocollectingdataandimprovingdataanalysistobefasterandmorereliable.Withedgecomputingtechniques,itispossibletobuildintelligentaquacultureinfrastructureinordertointroduceartificialintelligenceandmachinelearningtechniquesthatwilloptimizefeedingstrategyorreducecostbyminimizinghumanerrorandreactingfastertomachinefailures.
TechnologyConsiderations
Ascanbeseenfromthesefewusecases,therearebothcommonchallengesandfunctionalitythatbecomeevenmorecrucialinedgeandhybridenvironments.Asusecasesevolveintomoreproductiondeployments,thecommoncharacteristicsandchallengesoriginallydocumentedinthe“CloudEdgeComputing:BeyondtheDataCenter”whitepaperremainrelevant.
Thehighestfocusisstillonreducinglatencyandmitigatingbandwidthlimitations.Furthersimilaritybetweenthedifferentusecases,regardlessoftheindustrytheyarein,istheincreaseddemandforfunctionslikemachinelearningandvideotranscodingontheedge.Duetothethroughputdemandsofapplicationsliketheseandworkloadssuchasvirtualnetworkfunctions(VNF)for5G,variousoffloadingandaccelerationtechnologiesarebeingleveragedtoboostperformancethroughsoftwareandhardware,suchas:
Single-rootinput/outputvirtualization(SR-IOV):ThistechnologyallowsVMsandcontainerstosharedirectaccesstoadistinctpartofadevice,suchasnetworkadapters,usingthePCIExpressinterface.DataPlaneDevelopmentKit(DPDK):DPDKallowshighernetworkpacketthroughputbyusingoffloadingand
schedulingtechniques.Whilethisisnotatechnologyspecificforedgeitiscrucialtoenabletheoptiontouseittofulfillstrictrequirementsinhighlyresourceconstrainedenvironments.Non-uniformmemoryaccess(NUMA):Thisisanothermethodtoincreasethroughputbyallocatingdedicatedmemoryblockstoaninstance.ForfurtheroptimizationthememoryblockislocaltotheCPUcoreonwhichtheinstanceisworking.Forverysmalledgesitesthiscouldbecomeabottleneck,dependingontheedgesiteworkloadmix.SmartNics/Field-programmablegatearray(FPGA):ItisahardwareaccelerationoptionthatisalreadyusedforvRANdeploymentstoincreasetheperformanceofsiteswithcompute-intensiveworkloads.TheFPGAunitsareprogrammedwithworkload-specificsoftwaretooffloadtheexecutionofsomeapplication-specificalgorithms.GraphicsProcessingUnit(GPU):GPUshaveahigh-numberofcoreswhichmaybeutilizedbyawidevarietyofparallel-processingintensiveworkloadssuchasMapReduce,machinelearning(ML),IoTandgaming.WhileusingGPUsisagoodwaytoincreasethesystemperformancewheretheworkloaddemandsit,itintroducesthequestionofcostconstraints,especiallyifthenumberofedgesitesstartstogrow.
ReferenceArchitectures
Architecturedesignisalwaysspecifictotheusecase,takingintoaccountalltheneedsofthegivenplannedworkloadandfinetuningtheinfrastructureondemand.Asdiscussedearlier,thereisnosinglesolutionthatwouldfulfilleveryneed.However,therearecommonmodelsthatdescribehigh-levellayoutswhichbecomeimportantforday-2operationsandtheoverallbehaviorofthesystems.
Beforegoingintodetailabouttheindividualsitetypeconfigurations,thereisadecisionthatneedstobemadeonwheretolocatethedifferentinfrastructureservices’controlfunctionsandhowtheyneedtobehave.Thesemodelsanddecisionsarenotspecifictothetechnologiesnordotheydependontheparticularsoftwaresolutionchosen.
Theusecasesinthisdocumentaremostlyenvisionedasaspiderwebtypeofarchitecturewithhierarchyautomaticallyabletoscalethenumberofendpoints.Dependingonneeds,therearechoicesonthelevelofautonomyateachlayerofthearchitecturetosupport,manageandscalethemassivelydistributedsystems.Thenetworkconnectivitybetweentheedgenodesrequiresafocusonavailabilityandreliability,asopposedtobandwidthandlatency.
Thissectioncoverstwocommonhigh-levelarchitecturemodelsthatshowthetwodifferentapproaches.TheyaretheCentralizedControlPlaneandtheDistributedControlPlanemodels.Sincethisisahigh-leveldiscussion,theassumptionisthattherewillbeenoughcompute,storageandnetworkingfunctionalitytotheedgetocoverthebasicneeds;anyspecializedconfigurationsorfeaturesareoutofscope.ThearchitecturemodelsalsoshowrequiredfunctionalityforeachsitebutdonotdiscusshowtorealizeitwithanyspecificsolutionsuchasKubernetes,OpenStack,andsoforth.However,aspectsandtoolsthatwereconsideredduringthedevelopmentofthemodelsinclude:
Challengesofmanagingalargenumberofedgedatacenters:Availablefunctionalityattheedgedatacentervs.orchestrationoverheadPreparingthearchitecturetohandleonefailureatatime:e.g.:NetworkconnectionlossordegradationtothecentralorregionaldatacenterProvidingminimalviablefunctionalityonsmallfootprints
Thereareotherstudiesthatcoversimilararchitecturalconsiderationsandholdsimilarcharacteristicswithoutbeingfullyalignedwithonemodelortheother.Forinstance,arecentstudypresentsadisruptiveapproachconsistingofrunningstandaloneOpenStackinstallationsindifferentgeographicallocationswithcollaborationbetweenthemondemand.Theapproachdeliverstheillusionofasingleconnectedsystemwithoutrequiringintrusivechanges.
DiscussinganddevelopingadditionaldetailsaroundtherequirementsandsolutionsinintegratingstoragesolutionsandfurthernewcomponentsintoedgearchitecturesispartofthefutureworkoftheOSFEdgeComputingGroup.
CentralizedControlPlane
FortheCentralizedControlPlanemodel,theedgeinfrastructureisbuiltasatraditionalsingledatacenterenvironmentwhichisgeographicallydistributedwithWANconnectionsbetweenthecontrollerandcomputenodes.Ifadistributednodebecomesdisconnectedfromtheothernodes,thereisariskthattheseparatednodemightbecomenon-functional.
Duetotheconstraintsofthismodel,thenodesrelyheavilyonthecentralizeddatacentertocarrytheburdenofmanagementandorchestrationoftheedgecompute,storageandnetworkingservicesbecausetheyrunallthecontrollerfunctions.Computeservicesincorporaterunningbaremetal,containerizedandvirtualizedworkloadsalike.Relatedfunctionswhichareneededtoexecutetheworkloadoftheinfrastructurearedistributedbetweenthecentralandtheedgedatacenters.
Thediagramaboveshowsthatallofthekeycontrolfunctionalityislocatedinthecentralsite,includingallidentitymanagementandorchestrationfunctions.Ifyousetasidethegeographicallydistributednature,thisapproachfacesverysimilarchallengesasoperatinglarge-scaledatacenters.Ontheplusside,itprovidesacentralizedviewoftheinfrastructureasawhole,whichhasitsadvantagesfromanoperationalperspective.
Whilethemanagementandorchestrationservicesarecentralized,thisarchitectureislessresilienttofailuresfromnetworkconnectionloss.Theedgedatacenterdoesn'thavefullautonomy,thereforedistributingconfigurationchangesmightfailifthereislostaccesstotheimagelibraryortheidentitymanagementservice.Theconfigurationneedstoallowapplicationstocontinuerunningevenincaseofnetworkoutagesiftheusecaserequirestheworkloadtobehighlyavailable,i.e.aPointofSalessysteminaretaildeploymentortheindustrialrobotsoperatinginanIoTscenario.Thiscanbechallengingbecausemostdatacentercentricdeploymentstreatcomputenodesasfailedresourceswhentheybecomeunreachable.InadditiontheIdentityProvider(IdP)servicecaneitherbeplacedinthecentraldatacenterorremotelywithconnectiontotheidentitymanagementservicewhichlimitsusermanagementandauthentication.Dependingonthesituation,thismightbeconsideredmoresecureduetothecentralizedcontrollers,orlessflexiblebecauseitmightmeanlostaccessbyusersatacriticaljuncture.
Typically,buildingsucharchitecturesusesexistingsoftwarecomponentsasbuildingblocksfromwell-knownprojectssuchasOpenStackandKubernetes.SomeedgesitesmightonlyhavecontainerizedworkloadswhileothersitesmightberunningVMs.ItisrecommendedtoreviewtheDistributedComputeNode(DCN)deploymentconfigurationofTripleOwhichisalignedwiththismodel.
Insummary,thisarchitecturemodeldoesnotfulfilleveryusecase,butitprovidesanevolutionpathtoalreadyexistingarchitectures.Plus,italsosuitstheneedsofscenarioswhereautonomousbehaviorisnotarequirement.
DistributedControlPlane
Alargersetofusecasesdemandsedgesitestobemorefullyfunctionalontheirown.Thismeanstheyaremoreresilienttonetworkconnectivityissuesaswellasbeingabletominimizedisruptioncausedbylatencybetweenedgesites.
TheDistributedControlPlanemodeldefinesanarchitecturewherethemajorityofthecontrolservicesresideonthelarge/mediumedgedatacenters.Thisprovidesanorchestrationaloverheadtosynchronizebetweenthesedatacentersandmanagethemindividuallyandaspartofalarger,connectedenvironmentatthesametime.
Therearedifferentoptionsthatcanbeusedtoovercometheoperationalchallengesofthismodel.Onemethodistousefederationtechniquestoconnectthedatabasestooperatetheinfrastructureasawhole;anotheroptionistosynchronizethedatabasesacrosssitestomakesuretheyhavethesameworkingsetofconfigurationsacrossthedeployment.Thismodelstillallowsfortheexistenceofsmalledgedatacenterswithsmallfootprintswheretherewouldbealimitedamountofcomputeservices,andthepreferencewouldbetodevotethemajorityoftheavailableresourcestotheworkloads.
Themostcommonexampleiswhenthelocationofthecomponentsoftheidentitymanagementservicearechosenbasedonthescenarioalongwithoneoftheaforementionedmethodstoconnectthem.Thechoicedependsonthecharacteristicsoftheindividualusecaseandthecapabilitiesofthesoftwarecomponentsused,becausetheoverallbehaviorandmanagementofeachconfigurationisdifferent.Forinstance,usingtheOpenStackIdentityManagementservice(Keystone)tolocateitintoanedgedeploymentwithoutthelimitationoftechnologiesasitsAPIsupportsbothOpenStackandKubernetesorthecombinationofboth.
Thisarchitecturemodelismuchmoreflexibleincaseofanetworkconnectionlossbecausealltherequiredservicestomodifytheworkloadsorperformusermanagementoperationsareavailablelocally.Therearestillpotentialobstacles,suchasnothavingalltheimagesavailablelocallyduetolimitationsofstorageandcachesizes.Therearealsonewchallengesduetotheadditionalburdenofrunningalargenumberofcontrolfunctionsacrossageographicallydistributedenvironmentthatmakesmanagingtheorchestrationtypeservicesmorecomplex.
Asinthepreviouscase,thisarchitecturesupportsacombinationofOpenStackandKubernetesservicesthatcanbedistributedintheenvironmenttofulfillalltherequiredfunctionalityforeachsite.AnexampleofthisisStarlingX,asitsarchitecturecloselyresemblesthedistributedmodel.
Therearehybridsolutionsonthemarketthattrytoleveragethebestofbothworldsbydeployingfullinstallationsinthecentralnodesaswellaslarge/mediumedgedatacentersandhaveanorchestrationtypeserviceontop,suchasONAP,anorchestrationtoolusedinthetelecomindustry.
Futurearchitecturalconsiderations
Theabovedescribedmodelsarestillunderdevelopmentasmoreneedsandrequirementsaregatheredinspecificareas,suchas:
Storage:Considerationsincludelocalstoragetoenablehighperformanceandlowlatencyprocessingofdataaswellasprovidingoptionstoconnecttoremotestoragesolutions.Severalalternativesareavailable,rangingfromsmallandsimplesystemslikesoftwareRAIDorLVMtolargeandhighlyreliabledistributedstoragemanagerslike
Ceph.Baremetalmanagement:Itcanbeintroducedonmultiplelayers,onebeingtheinfrastructureoperatorinneedofmanagingandscalingtheirinfrastructuretoincludezerotouchprovisioningmethods,andtheotherbeingtheuseroftheinfrastructurewhomaygetthepermissionandoptiontocreatenewedgesitesondemand.
Thereareotherstudiesthatcoversimilararchitecturalconsiderationsandholdsimilarcharacteristicswithoutbeingfullyalignedwithonemodelortheother.Forinstance,arecentstudypresentsadisruptiveapproachconsistingofrunningstandaloneOpenStackinstallationsindifferentgeographicallocationswithcollaborationbetweenthemondemand.Theapproachdeliverstheillusionofasingleconnectedsystemwithoutrequiringintrusivechanges.
DiscussinganddevelopingadditionaldetailsaroundtherequirementsandsolutionsinintegratingstoragesolutionsandfurthernewcomponentsintoedgearchitecturesispartofthefutureworkoftheOSFEdgeComputingGroup.
Testingconsiderations
Definingcommonarchitecturesforedgesolutionsisacomplicatedchallengeinitself,butitisonlythebeginningofthejourney.Thenextstepistobeabletodeployandtestthesolutiontoverifyandvalidateitsfunctionalityandensureitperformsasexpected.Astheedgearchitecturesarestillintheearlyphase,itisimportanttobeabletoidentifyadvantagesanddisadvantagesofthecharacteristicsforeachmodeltodeterminethebestfitforagivenusecase.
ThebuildingblocksarealreadyavailabletocreateedgedeploymentsforOpenStackandKubernetes.Thesearebothopensourceprojectswithextensivetestingeffortsthatareavailableinanopenenvironment.Whileitiscommontoperformfunctionalandintegrationtestingaswellasscalabilityandrobustnesschecksonthecodebase,thesedeploymentsrarelygetextendedbeyondoneormaybeafewphysicalservers.Inthecaseofedgearchitecturesitiscrucialtocheckfunctionalitythatisdesignedtoovercomethegeographicaldistributionoftheinfrastructure,especiallyinthecircumstancewheretheconfigurationsofthearchitecturalmodelsarefundamentallydifferent.Inordertoensurestableandtrustableoutcomesitisrecommendedtolookintothebestpracticesofthescientificcommunitytofindthemostrobustsolution.Onecommonstandardpracticeistheartifactreviewandbadgingapproach.
Testingisasmuchanartformasitisapreciseengineeringprocess.Testingcodeonlowerlevels,suchasunittestsorcheckingresponsesofcomponentsthroughAPItests,isstraightforward.Thisallowsframeworkstobecreatedthatsupportrunninganautomatedunittestsuitethataddressesrequirementssuchasrepeatability,replicabilityandreproducibility.Testingtheintegratedsystemstoemulatetheconfigurationandcircumstancesofproductionenvironmentscanbequitechallenging.Thediagrambelowdescribesthegeneralprocessthatisexecutedwhenperformingexperimentalcampaigns.Thisprocess,thatisappliedinthefieldofresearch,canalsobeutilizedtohelpbuildnewcomponentsandsolutionsthatfittherequirementsofedgecomputingusecaseseventhoughsomeofthestepsstillneedmoretoolstoperformallchecksasiftheyweresimpleunittests.
Thefirstseeminglytrivialstepdescribingtheacquisitionofresourcesfromatestbedisnotspecifictoedgecomputingscenarios.Theassignedresources(e.g.,compute,storage,network)representthephysicalinfrastructurethatwillbeusedtoconducttheevaluation.
Thesecondphaseismoredifficult.Itincorporatesmultiplesubstepstopreparethephysicalinfrastructureaswellasthedeploymentofthesystemundertest(SUT).Asedgeenvironmentscanbeverycomplex,theyalsoneedtobetestedfortheirabilitytobepreparedforcircumstancessuchasanunreliablenetworkconnection.Therefore,havingadeploymenttoolthatsupportsadeclarativeapproachispreferredtospecifythecharacteristicsoftheinfrastructuresuchaslatency,throughputandnetworkpacketlossratiotoemulatethetargetedreallifescenarioandcircumstances.
Oncethedeploymentplanhasbeencreatedandtheresourceshavebeenselected,itneedstobeconfirmedthattheinfrastructureisconfiguredcorrectlyduringthepre-deploymentphasebeforeinstallingtheapplicationsandservicesontop.Thisisespeciallytrueinedgearchitectureswhereresourcesmustbeavailableovercomplexnetworkingtopologies.Forinstance,profileattributesmayhaveallbeensetcorrectly,butarealltheresourcesreachable,ingoodhealth,andcancommunicatetoeachotherasexpected?Thecheckscanbeassimpleasusingthepingcommandbi-directionally,verifyingspecificnetworkportstobeopenandsoforth.Thepurposeofthisprocedureistoensurethatthedeploymentstepwillbecompletedsuccessfullyandresultinatestenvironmentthatisalignedwiththerequirementsandplans.Thecomplexityofedgearchitecturesoftendemandsagranularandrobustpre-deploymentvalidationframework.
Nowthatthetestbedispreparedandtested,thenextstepistodeploythesoftwareapplicationsontheinfrastructure.ForsystemsbuiltonenvironmentssuchasOpenStackandKubernetesservices,frameworkslikeKolla,TripleO,KubesprayorAirshipareavailableasstartingpoints.Beawarethatthemajorityofthesetoolsaredesignedwiththelimitationsofonedatacenterastheirscope,whichmeansthatthereisanassumptionthattheenvironmentcanscalefurtherduringoperation,whileedgeinfrastructuresaregeographicallydistributedandoftenhavelimitedresourcesintheremotenodes.Inaddition,theconfigurationoptionsaresignificantlydifferentamongthedifferentmodels.Aspartoftestingedgearchitectures,thedeploymenttoolsneedtobevalidatedtoidentifytheonesthatcanbeadaptedandreusedforthesescenarios.Toensurethesuccessoftesting,theinstallationitselfneedstobeverified,forinstance,checkingtheservicestoensuretheywereinstalledandconfiguredcorrectly.Thisoperationshouldpreferablybeafunctionalityofthedeploymenttool.
Whenallthepreparationsaredone,thenextstepisbenchmarkingtheentireintegratedframework.Benchmarkingisoftendefinedasperformancetesting,buthereitappliestoabroaderscopethatincludesintegrationandfunctionaltestingaswell.Itisalsoimportanttonotethatthetestsuitescanbeheavilydependentontheusecase,sotheyneedtobefinetunedforthearchitecturemodelbeingused.Whileafewtoolsexisttoperformnetworktrafficshapingandfaultinjections,thechallengeliesmoreintheidentificationofvaluesthatarerepresentativetotheaforementionededgeusecases.
Buildinganedgeinfrastructureconsistsofvariouswellknowncomponentsthatwerenotimplementedspecificallyforedgeusecasesoriginally.Becauseofthat,therearesituationswheretherewillbeaneedtotestbasicfunctionalityintheseenvironmentsaswelltomakesuretheyworkasexpectedinotherscenarios.Examplefunctionsinclude:
create/deletearesource(user,flavor,image,etc);scope:oneormoreedgesiteslistinstances(VM,container);scope:anedgesiteor‘singlepaneofglass’dashboardcreateresourcesforcross-data-centernetworks
Furthertestingoftheedgeinfrastructureneedstotakethechoiceofarchitecturalmodelintoconsideration:
UsingOpenStackinthecentralizedcontrolplanemodeldependsonthedistributedvirtualrouter(DVR)featureoftheOpenStackNetworkConnectivityasaService(Neutron)component.Thebehavioroftheedgedatacentersincaseofanetworkconnectionlossmightbedifferentbasedonthearchitecturalmodels.Insomecases,thedecisionmightbetochoosetoconfigurethesystemtokeeptheinstancesrunningwhileinothercases,therightapproachwouldbetodestroytheworkloadsincasethesitebecomesisolated.Inadditiontotheseconsiderations,theexpectationsonfunctionssuchasauto-scalingwillalsobedifferentduetopossibleresourceconstraints,whichneedtobereflectedinthetestsuitesaswell.
Thefinaltwostepsaretrivial.Thetestresultsneedtobecollectedandevaluated,beforereturningtheSUTinfrastructuretoitsoriginalstate.
ToolssuchasEnos,Enos-Kubernetesandenoslibareavailableintheexperiment-drivenresearchcommunitytoevaluateOpenStackandKubernetesinadistributedenvironmentoverWideAreaNetwork(WAN)connection.Theycanbeextendedorleveragedasexamplesofsolutionsthatcanbeusedtoperformtheabovedescribedprocesstoevaluatesomeofthearchitectureoptionsforedge.Furthercomponentsareneededtoensuretheabilitytotestmorecomplexenvironmentswheregrowingnumbersofbuildingblocksareintegratedwitheachother.
Conclusion
Edgecomputingishighlydependentonlessonslearnedandsolutionsimplementedinthecloud.Evenifthemajorityofbuildingblocksareavailabletocreateanenvironmentthatfulfillsmostrequirements,manyofthesecomponentsneedfinetuningorAPIextensionstoprovideamoreoptimizedandfitforpurposesolution.Deploymentandtestingrequirementsarefurtherhighlightedforthesenewarchitecturalconsiderations,andthereforeexistingsolutionsneedtobeenhanced,customizedandinsomecasesdesignedandimplementedfromscratch.
Therealchallengeliesinefficientandthoroughtestingofthenewconceptsandevolvingarchitecturemodels.Newtestcasesneedtobeidentifiedalongwithvaluesthatarerepresentativetotypicalcircumstancesandsystemfailures.Testingcanhelpwithbothenhancingarchitecturalconsiderationsaswellasidentifyingshortcomingsofdifferentsolutions.
Ascanbeseenfromthesediscussions,edgecomputingrelatedinnovationandsoftwareevolutionisstillverymuchinitsearlystages.Yes,therearesystemsrunninginproductionthatresembleatleastsomeoftheconsiderations—uCPEorvRANdeployments,forexample.Thearchitecturemodelsdiscussedherecoverthemajorityoftheusecases,however,theystillneedadditionaleffortstodetailtherequiredfunctionalitytogobeyondthebasics,outlinefurtherpreferablesolutionsanddocumentbestpractices.
ThisistheperfecttimeforgroupsintheITindustry,bothopengroupsandsemi-openorclosedconsortiums,aswellasstandardizationbodies,tocollaborateontakingthenextstepsforarchitecturedesignandtestinginordertobeabletoaddresstheneedsofthevariousedgecomputingusecases.Consideringthehighlevelofintegrationneeded,itiscrucialthatthesubjectmatterexpertsofthevariouscomponentsstarttocontributetoacommoneffort.
Authors
BethCohen,DistinguishedMemberofTechnicalStaff,VerizonGergelyCsatári,SeniorOpenSourceSpecialist,NokiaShuquanHuang,TechnicalDirector,99CloudBruceJones,StarlingXArchitect&ProgramManager,IntelCorp.AdrienLebre,ProfessorinComputerScience,IMTAtlantique/Inria/LS2NDavidPaterson,Sr.PrincipalSoftwareEngineer,DellTechnologiesIldikóVáncsa,EcosystemTechnicalLead,OpenStackFoundation
