New 20160926 - Introducing pScheduler - Feit · 2016. 10. 24. · Introducing pScheduler perfSONAR’sNew Scheduler Presented by the perfSONAR Project ... • Round-Trip Ping

IntroducingpSchedulerperfSONAR’s NewScheduler

PresentedbytheperfSONAR Projecthttp://www.perfsonar.net

Internet2TechnologyExchange2016September,2016ThisdocumentisaresultofworkbytheperfSONAR Project(http://www.perfsonar.net)andislicensedunderCCBY-SA4.0

(https://creativecommons.org/licenses/by-sa/4.0/).

pScheduler• pScheduler isnewsoftwareforschedulingandsupervisingmeasurements.

• SuccessortotheBandwidthTestController(BWCTL)

October24,2016©2016,http://www.perfsonar.net 2

WhyreplaceBWCTL?• BWCTLhasservedperfSONARwellbutisshowingitsage.– Maintenanceandexpansionhavebecomedifficult.

• Demandfornewfeaturesandadditionalsophisticationcalledforaclean-slateimplementationbuiltwithaneyetowardthefuture.

October24,2016©2016http://www.perfsonar.net 3

WeGetLetters• TestsandTools.CanperfSONARmeasure ____?• Schedule.Whatran?What’srunning?What’splannedtorun?

• StatusandDiagnostics.Whathappened?WhydoIhavetologinandpickthroughthelogs?


WeGetLetters• RegularTesting. Canitworkwith____?Canitbebetter?Lesskludgy?

• Archiving. Canresultsbesentto____?Reliably?• Persistence.Whydoestheschedulenotsurviveareboot?

• OpenIssuesinGitHub.Lotsofthem.


OurGoals1. Lookateveryaspectofmeasurement

throughasmanylensesaspossible.

2. Designsomethingadaptabletonewneeds.

3. Buildittolastalongtime.


NOMENCLATUREANDCONCEPTS


pScheduler Terminology• Test Definitionofameasurement• Participant Pointinvolvedinthetest• Tool Programthatdoesthemeasuring• Run Themakingofameasurement• Result Theendproductofarun• Archiver Awaytosendresulttostorage• Task AjobforpScheduler todo


TestsintheAbstract• Measurementhasbeenabstracted.– Testparameters Standardformat– Result foreachtest.

• Testsdescribetheparameters,toolsdothemeasuring.

• Tool-specificfeaturesareaccommodated.October24,2016©2016,http://www.perfsonar.net 9

WHAT’SNEW


Extensibility• ManyofthethingsBWCTLandotherpartsofperfSONARusedtodointernallyhavebeendelegatedtoplugins:– Tests Measurenewthings.– Tools Usenewthingstodomeasurements.– Archivers Sendresultswhereyouwant.– Otherthingstocomeinthefuture


Extensibility• DocumentedAPI– Therewillbetraining.• Minimalinvolvementfromcoredevelopmentteamrequired– Weaskearlyadopterstoworkwithus

• Supportfornicheapplications– DomorewithyourperfSONARdeployments

• Shorterdevelopmentcycle


TestsandTools• Throughput Iperf2,Iperf3,Nuttcp• Latency OWAMP,Powstream• Round-Trip Ping• Trace Traceroute,tracepath,ParisTraceroute• DevelopmentandDemo:– Simplestream Simplestreamer– Idle Sleep,Snooze


ToolSelection• JustOne owamp• PreferredOrder iperf3,iperf2• Automatic– Toolavailabilityandhard-wiredpreferences(e.g.,traceroute overtracepath).

– Allowssmoothdeprecationofoldertoolsandeasyintroductionofnewones.


Archivers• Shipping Esmond,Syslog• DevandDemo BitBucket,Failer• NearFuture HTTP,RabbitMQ,Others

• Archiverscanbeselectedpertaskorappliedgloballytoeveryresult.


RepetitiveTesting• Bakedintothecoreofthesystem.• Nowpartofschedulingrunsforevery task.• Alltasksworkthesameway:– Somerepeat,somedon’t.– Alltestfeaturesareavailable,repeatingornot.– Nospecialdevelopmentrequiredfornewtestsortools.


LimitSystem• Allnew– Therewillbetraining• FourPhases– Identification Who’sasking?– Classification Whatgroup(s)?– Limits Whataretherestrictions?– Application Forcetesttoconformto…


LimitSystem• Identification Who’sasking?– CIDRs(explicitlist,download/update),ReverseDNS,Bogons/Martians,Hints(IP,protocol),Forced

• Classification Whatgroups?– E.g.,bogons,selectedIPsandalistdownloadedfromIDSfallintoahostile classification.

– IPsinadownloadedlistofR&ECIDRsfallintoaresearch classification.


LimitSystem• Limits Whataretherestrictions?– Testtypeandparameters,scheduleattributes

• Applications Howarethelimitsapplied?– Denyrequestsinthehostile classification– Allowrequestsintheresearch classificationthatpasslimitsresearch-throughput orinert-tests


ComputerandHumanInterfaces• REST– Allotherinterfacesoperatethroughthisone.– 100%JavaScriptObjectNotation(JSON)

• Command-Line– Notidenticaltobw* familyofcommandsbutnotadifficultadjustment

• GUI(Future)October24,2016©2016,http://www.perfsonar.net 20

CLIFeatures• Exportandimporttasksto/fromJSON– Editparametersonthefly

• Watchrepeatingtasksastheyrun• Retrievepreviously-storedresults• Querytheschedule


OtherFeatures• Per-taskreferencedata– AddarbitraryJSONtoanytask.– QueryagainstthisandothertaskparametersviatheRESTAPI

• Extensivediagnosticsanddebug– Reducedemailvolleysrequiredtosolveproblems


BackwardCompatibility• pScheduler isincompatiblewithBWCTL• BWCTLwillbekeptatleastthroughfall,2017• Duringthetransition:– ToolpluginsthatcantalktoBWCTL-onlysystemsforthroughput,latency,round-triptimeandtraceroute

– perfSONAR3.xRegularTestingandMeshConfig havebeenadaptedtoworkwithpScheduler.


TECHNICALFEATURES


RelationalDatabaseBackend• PostgreSQL9.5– Selectedforscaleup/down,maturityandfullfeatureset

• Heavyliftingthroughqueriesinsteadofhand-rolledcode

• Integrityfeaturescutdownonbugsandconsistencyproblemsinducedbysoftwarebugsandsystemcrashes.

• Upgradeprocessis100%transactional.


Plug-inArchitecture• APIforextendingpartsofthesystem• NomodificationstopScheduler needed• Processboundary– Developinanylanguage– AllcommunicationisJSONviastandardI/O– pScheduler isprotectedfromplug-in-relatedcrashes


SupportingCode• WritteninPython2.x– Well-usedandwell-understoodbythecommunity

• Recommendedlanguageforplug-ins– Notrequired– pscheduler Pythonmodulewillmakecommonutilitiesavailabletospeeddevelopment(e.g.,inhalingJSON,standardexitmethods,saferunningofexternalprocesses).


AQUICKLOOKATTHECLI


% pscheduler task throughput \--dest ps.bar.net --duration PT15S \--interval PT3S

Submitting task...Task URL:

https://ps.foo.net/pscheduler/tasks/9d30dbf7-746f-4fef-9640-814896b2b8a5Fetching first run...Next run:https://ps.foo.net/pscheduler/tasks/9d30dbf7-746f-4fef-9640-814896b2b8a5/runs/a785d67e-08af-4614-ab91-593425ee04d5


Starts 2016-09-12T01:29:06Z (~3 seconds)

Ends 2016-09-12T01:29:26Z (~19 seconds)Waiting for result...

* Stream ID 4Interval Throughput Retransmits Current Window 0.0 - 3.0 854.60 Mbps 630 1.40 MBytes 3.0 - 6.0 599.98 Mbps 12 849.98 KBytes

6.0 - 9.0 551.65 Mbps 0 923.82 KBytes9.0 - 12.0 575.41 Mbps 0 931.06 KBytes12.0 - 15.0 464.74 Mbps 3 750.06 KBytes

SummaryInterval Throughput Retransmits

0.0 - 15.0 609.29 Mbps 645


FINALLY…


BearWithUs• pScheduler representsaverysignificantchange.

• Weexpecttherewillbesometeething.


Training,SeminarsandShortSubjects• pScheduler Overview(LongVersion)• Command-LineInterface• PluginDevelopment• Archiving Moretocome.

• Announcementsonperfsonar-users


Feedback• Yourcomments,questionsandideasarethesourceofmanypScheduler features.– Pleasekeepthemcoming.

• perfSONARDeveloperMailingList:https://lists.internet2.edu/sympa/info/perfsonar-developer

• Emailmedirectly:[email protected],2016©2016,http://www.perfsonar.net 34

TheEnd

Questions?Comments?


Documents

New 20160926 - Introducing pScheduler - Feit · 2016. 10. 24. · Introducing pScheduler perfSONAR’sNew Scheduler Presented by the perfSONAR Project ... • Round-Trip Ping