View
0
Download
0
Category
Preview:
Citation preview
Copyright©2016SplunkInc.
TeddHellmann/DavidPoncelowProductManager/SeniorSoDwareEngineer,Splunk
STEPUpYourAppDevelopmentGame
Disclaimer
2
DuringthecourseofthispresentaMon,wemaymakeforwardlookingstatementsregardingfutureeventsortheexpectedperformanceofthecompany.WecauMonyouthatsuchstatementsreflectourcurrentexpectaMonsandesMmatesbasedonfactorscurrentlyknowntousandthatactualeventsorresultscoulddiffermaterially.Forimportantfactorsthatmaycauseactualresultstodifferfromthosecontainedinourforward-lookingstatements,pleasereviewourfilingswiththeSEC.Theforward-lookingstatementsmadeinthethispresentaMonarebeingmadeasoftheMmeanddateofitslivepresentaMon.IfreviewedaDeritslivepresentaMon,thispresentaMonmaynotcontaincurrentor
accurateinformaMon.WedonotassumeanyobligaMontoupdateanyforwardlookingstatementswemaymake.InaddiMon,anyinformaMonaboutourroadmapoutlinesourgeneralproductdirecMonandis
subjecttochangeatanyMmewithoutnoMce.ItisforinformaMonalpurposesonlyandshallnot,beincorporatedintoanycontractorothercommitment.SplunkundertakesnoobligaMoneithertodevelopthefeaturesorfuncMonalitydescribedortoincludeanysuchfeatureorfuncMonalityinafuturerelease.
HowshouldIbuildmyapp?
EVERYTHINGYOUNEEDTOBUILD
SplunkDeveloperGuidance
STEPupyourgame STEP-interacMvelearningenvironment ExploretopicsthroughTechniquesandRecipes Technique:explorethedetailsoffeaturesyoucanuseinapps(modularinputs,customvisualizaMons,customalertacMons,…)
Recipe:diveintothedetailsofbringingseveraltechniquestogethertoaddressabusinessgoal
STEPupyourgame
6
DataIngest
HEC
BasicDataInput
IndexerAck
…
ModularInput
CheckpointResults
InputValidaMon
…
……
VisualizaMons
…
Search
…
SimpleXML
CustomViz
...
CustomCommands
Workflows
...
STEPupyourgame
7
DataIngest
HEC
BasicDataInput
IndexerAck
…
ModularInput
CheckpointResults
InputValidaMon
…
……
VisualizaMons
…
Search
…
SimpleXML
CustomViz
...
CustomCommands
Workflows
...
STEPupyourgame
8
STEPPreview2Techniques1Recipe
Nextsteps
9
Telluswhattechniquesandrecipesyouneed!
STEPintoreal-worldexamples
Planningajourney
Pla'ormandtools:akitbagforourjourney
UIandvisualiza2ons:whattheappslooklike
Workingwithdata:whereitcomesfrom&howwemanageit
Addingcode:usingJavaScriptandSearchProcessingLanguage
Packaginganddeployment:reachingourfirstdesMnaMon
DealingwithOAuth
Aler2ng
Buildingintelemetrywithhigh-performancedatacollec2on
splk.it/devguide
1.StartwithaQuesMonsBacklog Architecture– WhatdoesatypicalSplunkapplicaMonreferencearchitecturelooklike?– WhatcommonparadigmsareapplicabletoSplunkappdevelopment?– Whatarethetypicaldeploymenttopologies?WhyshouldIchooseaspecificone?Whataretheconfoundingfactors
onthechoiceofmytopology?– HowdoIparMMonmySplunksoluMons?– Whatarethetradeoffsofvarioustypesofinputs?– HowdoIarchitectmySplunksoluMonanddeploymentforaverylargescale?– HowdoIarchitectmySplunksoluMonforthecloud?WhatarespecificconsideraMonsfordeployingtoAWSorAzure?– What’sthelandscapeofSplunkextensionpoints?– HowdoIintegratedatafromSplunkintoexisMngapplicaMonsandsystems?– HowdoIplananddesignarobustalerMngandmonitoringsubsystemontopofSplunk?– WhatshouldIconsiderformysizingrequirements?– WhatarerecommendedconfiguraMonsofSplunkdeploymenttomeetmysizingrequirements?– ShouldIarchitectmysoluMontoindexmydatainlocaldatacenter(zone)orcentrally?– WhatarethingswecanautomaMcallydegradesowecanmakesureourcoreexperienceisworking?– Whensomethinghappens,howdoIeffecMvelypropagatetheinfoandreacttoit?– HowareothersoluMonsonSplunkbuilt?Whatwerethechallenges?Howhavetheybeenaddressed?
PackagingandDeployment– HowdoIpiecetogethervariouspartsofaSplunkapp(customsearchcommands,modinputsetc.)?– HowdoIpackageaSplunksoluMonwithasingleinstallthatautomaMcallyrollsoutallthenecessarydependencies?– HowdoImanagemySplunksoluMonversioning,backwardandfuturecompat?– What'sthebestwaytosplitupcustomappsfordeployment?
Development– HowshouldIsetupmydevelopmentenvironmenttobeproducMvewithSplunk?– WhataredifferentwaysofhowIdevelopmySplunkapp?ProsandconsofusingspecificSDKvsRESTAPIs?
ProsandconsofusingSimpleXMLvsAdvancedXMLvsWebFramework…– HowdoIanalyzeadatasourceforaTA?– WhatarethedifferentwaysofenrichingthedatainSplunk?Whataretheirtradeoffs?– WhenshouldIuseeventtypesandtransacMonsfordataclassificaMon?– HowdoIextendSplunktodefineacustominputcapability?– WhenshouldIusemodularinputsvsscriptedinputsvs..?– Whatarestreamingvsnon-streamingoutputsconsideraMons?– HowdoIdealwithlong-runningscripts?Handlingshutdown/restartofSplunk?Concurrency?Statepersistenceetc.
– WhyshouldInotusetransacMons?– WhenshouldIusepivotvststats?– WhyshouldIusedatamodels?– Whenmydatasourcetouchesonmanydatamodels,shouldIassumecompleteseparaMonorheavyinheritance?– HowdoIextendanexisMngdatamodel?– WhatdoesCIMofferandwhyshouldIbuildCIM-compliantapps?– InthecontextofCIM,whatarethetradeoffsofusingmyprops.confandtransforms.confandrewriMngthemon
indexing,completelydiscardingthevendorsuppliedfieldnames?HowdoIreconciletheadvantagesofacleaninterface&normalisaMon,butatthecostoflosingalignmentwithpublishedvendordocumentaMon,andalearningcurveforexisMngusers?
– HowdoImanagemysoluMondeclaraMveconfiguraMon?HowdoIdetect/troubleshootbadconfig?– HowdoIlogandanalyzedatathatisnoteventdriven(certainwebfeeds,htmlparsing,imagemetadata)?– Compareandcontrastad-hocsearchingvsbackgroundsearching– HowdoIhandletransientfaults?– HowdoIeffecMvelymanagecredenMals?– What’stheeffectofsearchheadlocaMononmyappandtheoveralluserexperience?– HowdoIdevelopanintegratedmechanismtoletmeconnectSplunktomyMOM(messagingmiddleware)andindex
mymessages?– HowdoIhandletherequirementthatappconfigsmustbedifferentacrossdifferentservertypesinadistributed
environment(e.g.appsonsearchheadsshouldn'thaveinputsenabled)? Quality/Compliance
– WhatqualitygatesshouldIconsider?Whatkindofpara-funcMonalcharacterisMcsareimportanttoconsider?– WhatheurisMcsdoIusetobless/blockarelease?– HowdoItestadatamodel?– HowdoIprepareeventgeneraMonwhenbuilding/tesMnganapp?– WhatkindofperftesMngshouldIdoandhow?– HowdoItestUI?– HowdoIsecuritycerMfymysoluMon?– HowdoIdesigntosaMsfymyretenMonandcompliancepolicies?– HowdoIarchitecttodesignmyavailabilityrequirements?– HowdoIhandlegeographicdisasterrecovery/faulttolerance?– HowdoIproperlyinstrumentmysoluMonsothatIknowwhat’shappening?
SustainedEngineering– HowdoImaintain/service/supportSplunkapps?– HowdomycustomershandleupdaMngtheircustomizedconfigsoncenewversionsofmyappcomeout?
Business– WhyshouldIbuildonSplunk?– WhatkindofskilldoIneedmydevstohavetobuildaSplunksoluMon?– Whatisthecommunitybuilding?HowarecurrentdevscreaMnguniqueexperiencesusingSplunk–Itypicallywantto
seesomemarketplacesuccess – Costandpricingareveryimportanttomeasaentrepreneurdeveloper.IfIamcomingintobuildatoolthatwillbe
commercializedIneedtoknowthatthecoststructureofSplunkwon’tcausemyservicetobeeconomicallyunprofitable.
WhatdoesatypicalSplunkapplica2onarchitecturelooklike?
HowshouldIsetupmydevenvironmenttobeproduc2vewith
Splunk?
HowdoIintegrateSplunkintoexis2ngsystems?
HowdoIpreparemyeventgenera2onwhendeveloping&
tes2nganapp?
HowdoIpackageanapp?dealwithappversioningandupdates?
12
2.IdenMfyExtensibilitySurfaceArea§ Datainges2on&indexing
– Inputê Modularinputsê Custom(trained)sourcetypesê Customsourcesê HTTPEventCollecMon
– Datainges2onpipelineê FieldextracMonsê FieldtransformaMons
– Indexingê Customindexes
§ Searching– Searchauthoring
ê Customsearchcommandsê Macros(basic,parametrized)ê Savedsearches
– Dataclassifica2onê Eventtypesê TransacMons
– Dataenrichmentê Lookupsê KVstorecollecMonsê WorkflowacMons
– Datanormaliza2onê Tagsê Aliases
– Dataminingê cluster&dedupê anomalousvalueê kmeansê predictcommands…
§ Processing&repor2ng– Search-2memapping
ê Datamodels
– CIMextensions
– CustomVisualiza2ons
– CustomUI– Pages,views&dashboards
ê JS,CSSExtensionsê Customsetupscreens
– Scheduledprocessingê Scheduledreports
– Aler2ngê Scriptedalertsê CustomalertacMons
– Branding&naviga2onê CustomappnavigaMonê Appbranding
– Manageabilityê Customsplunkwebcontrollersê Customsplunkdendpoints
13
3.Minebusinessrequirements4.FormulatelearningobjecMves5.Designaround3and4
…
14
§ Data§ Searchlanguage§ AggregaMngsiloedmetricsinto
meaningfulKPIs§ DatamanipulaMon§ DatanormalizaMon§ Sub-searches§ Config-driven§ PersistencewithKVstore§ Macros
§ Viz:§ Dynamicscaling§ Customizingin-theboxviz
controls
§ Generalsearchpaserns§ SearchopMmizaMons§ UxPrototyping§ AdapMng3rdpartyvizlibrary§ CompositechartswithinteracMons§ Dealingwithhigh-volumedatasets§ TroubleshooMngperfissues§ Post-processornot-post-process–
deploymentimplicaMons§ AutomatedUItesMng(w.Selenium)
§ Setngthestage§ OverallSplunkappstructure§ UItechnologyselecMon:
SimpleXMLvsSplunkJS§ Modularity§ Dev&testenv§ Devworkflow§ Modularity§ Dataonboarding§ CIMcompliance§ Tools
§ Post-processing§ IntegraMngwith3rdparty
component§ UnittesMng(w.Mocha)§ PersisMngstate(peruser)
§ Datamodeling§ Usinglookups§ Buildingabaselinelookuptable§ WindowsofMme/CustomMmeranges§ OverlayingMmedata
§ Usingsub-searchestocorrelatedata§ TroubleshooMngsearches
§ Customnav§ UxacMviMespermeaMngalldev
§ Datamining:§ ExploraMon§ PreparaMon:filtering/deduping/
buckeMng§ UsingadvancedstaMsMcsfuncMons§ Threshold-basedanomalydetecMon§ EvaluaMnggoodness/accuracy
Plusnon-funcMonaltopics: § Appversioning§ PackagingInstallaMon§ Securityreview
§ Deployment§ Publishingtosplunkbase§ AppcerMficaMon
BuildingSoluMonsontheSplunkPlauorm1. SplunkReferenceApps
Complete,workingreal-worldSplunksoluMonsbuilttogetherwithpartners(Conducive,Auth0)
2. SplunkDeveloperGuide
Thisisunbelievable,itcoversmosteverythingIlearnedthehardway…–BernieMacias,TechnicalArchitect,Zillow
dev.splunk.com/goto/devguide
SplunkReferenceAppDemo
SplunkReferenceApp–PluggableAuditSystemsplunkbase.splunk.com/app/1934/ORsearchnamefromSplunkWebUI
Takeaways Appdevelopment!=rocketscience
STEPupyourgamewithtechniquesandrecipes
Getintouchwithus
– STEP@.splunk.com
– LeavefeedbackforSTEP
– Comebyourbooth,getsomeswag
Resources
19
Relatedbreakoutsessions&acMviMes
20
FasterSplunkAppCer2fica2onwithSplunkAppInspect(GrigoriMelnik/AndyNortrup)BestPrac2cesforWorkingwithSplunkCloud(DennisBourg/EricSix)HTTPEventCollectorinSplunk6.4-MoreSuperPowers!(GlennBlock/ItayNeeman)
BuildingSplunkVisualiza2onswiththeNewCustomVisualiza2onAPI(MarshallAgnew)DashboardWizardry(NicholasFilippi/SiegfriedPuchbauer)
BestPrac2cesforDevelopingSplunkAppsandAdd-ons
(JasonConger)
HowtoBuildaSolu2onfromScratch:ACaseStudyofPartnerEngagementandCo-Development
(VladimirMelnik/IgalVanier)
OnboardYourDataFasterUsingtheSplunkAdd-onBuilder
(EliasHaddad/GuodongWang)
THANKYOU
Recommended