Embed Size (px)
Citation preview
November15,2016
Monitoring,HoldtheInfrastructureGettingthemostfromAWSLambda
MattWilliams– Evangelist@Datadog@technovangelist - [email protected]
November15,2016
ThePlanBriefDatadog OverviewThewhatandthewhyofLambdaAddingLambdatoyourinfrastructureMonitoring101(tl;dr version)AddingLambdatoyourmonitoring
November15,2016
Datadog Overview
SaaS-basedInfrastructureMonitoring
FocusonModernInfrastructure
(cloud,containers,micro-services)
Processingaboutatrillionmetricsperday
Intelligentalerting
November15,2016
MonitorEverything
November15,2016
ThePlanBriefDatadog OverviewThewhatandthewhyofLambdaAddingLambdatoyourinfrastructureMonitoring101(tl;dr version)AddingLambdatoyourmonitoring
November15,2016
WhatisLambda
Thenextstageinalongprogression...
physicalboxes
virtualmachines
onphysicalboxes
containers lambdaAWSinstances
November15,2016
ImportantLambdaConcept
November15,2016
ImportantLambdaConcepts
November15,2016
ImportantLambdaConcepts
SERVERLESS
AS FAR AS YOU ARE CONCERNED
November15,2016
WhatisaAWSLambdaFunctionEachAWSLambdaFunctionis:• CodeinacontaineronLXConAMZNLinux
– Containerimplementationhidden• Norestrictionsonwhatitcando
– Runtimes forJava,Node,Python– Otherlanguagesviaexec/spawn
• Pricedbasedonexectime,quantity,andmemory
November15,2016
WhatcantriggerAWSLambda?VariousAWSServicesincluding:Files(S3)DatabaseUpdates(DynamoDB)Streams(Kinesis)Messages(SNS,SES)IoT (Echo,APIGateway)
November15,2016
SoWhatAreSomeGoodUseCasesBootstrappingastartup(first1Mrequests/monthfree,upto3.2Mfree
seconds/month)CronFilemanipulation(transcode,compress,etc)APItranslationStreamprocessing(socialmediaanalysis)ScalinginfrastructureMobile/IoT BackendsAlexaSkills
November15,2016https://goad.io/
November15,2016
OtherExamplesBittorrent Tracker• Hugelyscalable• UsesS3forassets• Capableofhandling9,000,000peershttps://blog.zappa.io/posts/zappa-bittorrent-tracker
ChatBots• Easytobuildhttps://claudiajs.com/claudia-bot-builder.html
S3DataLoading• SimplerthanthetypicalKinesisprocess• (butslower)https://www.trek10.com/blog/serverless-architectures-s3-data-loading/
November15,2016
ThePlanDatadog overviewThewhatandthewhyofAWSLambdaAddingLambdatoyourinfrastructureMonitoring101(thetl;dr version)AddingAWSLambdatoyourmonitoring
November15,2016
HowtoCreateaLambdaFunction
1. Writethecode2. SetuponAWS(mem,timeout,role,handler)
– Webinterface– node-lambda,apex,serverless,etc
3. Debug– logs
November15,2016
AWSLambdaDemo
November15,2016
Usingnode-lambda• npm install node-lambda• npm run setup• npm run dry-run• npm run deploy
November15,2016
Usingapex• apex init• apex deploy• apex deploy --dry-run• apex invoke
November15,2016
Usingserverless• npm install serverless –g• serverless create --template aws-nodejs
• serverless deploy• serverless invoke
November15,2016
NativeGotchas• RememberthatLambdarunsonanAmazonLinuxcontainer
• …andyourdevboxprobablyisn’tAmazonLinux…
• …sonativeNPMmodulesareproblematic
November15,2016
Nativenpm1. LaunchanAmazonLinuxec22. Installnvm (https://github.com/creationix/nvm)3.nvm install 4.34.npm install <chosen modules>5. Zipupnode_modules6. scp todevbox,unzip7. Deploy
November15,2016
OtherAWSLambdaTools• Apex(http://apex.run)• Serverless framework(http://serverless.com)• GoSparta (http://gosparta.io)• go-lambda(https://github.com/xlab/go-lambda)
• AWSChalice(https://github.com/awslabs/chalice)
November15,2016
ThePlanDatadog overviewThewhatandthewhyofAWSLambdaAddingAWSLambdatoyourinfrastructureMonitoring101(thetl;dr version)AddingAWSLambdatoyourmonitoring
November15,2016
November15,2016
November15,2016
Collectingdataischeap
November15,2016
Collectingdataischeap;Nothavingitwhenyouneeditcanbeexpensive
November15,2016
November15,2016
November15,2016
November15,2016
November15,2016
November15,2016
November15,2016
November15,2016
ThePlanDatadog overviewThewhatandthewhyofAWSLambdaAddingAWSLambdatoyourinfrastructureMonitoring101(thetl;dr version)AddingLambdatoyourmonitoring
November15,2016
WhataretheAWSLambdaMetricsaws.lambda.duration,min,max,sumaws.lambda.errorsaws.lambda.invocationsaws.lambda.throttles
November15,2016
3OptionsforMonitoringLambda
1. AddacustommetrictoCloudWatch2. Addmetricdirectlytomonitoringapplication3. AddalinetotheCloudWatch logs
November15,2016
TheDownsidesoftheFirstTwo• CloudWatch Metrics
• CloudWatch metricsareupdatedeveryminute• Sendmorethanoneperminuteandyougeta
histogram• DirectlytoMonitoringApp
• Addscomplexityandsizetoyourlambda(shouldbekeptsmall)
• Butnolagtime
November15,2016
HowDatadogDoesItAddlinetoCloudWatch logs:MONITORING|unix_epoch_timestamp|value|type|my.metric.name|#tag1:value,tag2
metrictypecanbecountorgauge
InPython: print(monitoringstring)InNodeJS:console.log(monitoringstring)
November15,2016
Monitoringexample
November15,2016
MonitorAlltheLambdaThingsMonitorthetriggersMonitorthefunctionMonitortheoutput
November15,2016
November15,2016
SummaryAWSLambdacanbeaninterestingpartofyourinfrastructureVerylowcosttostartServerless doesn’tmeanmonitorless
MattWilliams- [email protected]@technovangelist
November15,2016
November15,2016
![[Document title] Water Monitoring Report... · Web viewIn the absence of competition, infrastructure operators hold market power, which can result in prices, quality, service levels](https://img.pdfslide.us/doc/110x75/5ae85e977f8b9a6d4f8f66e6/document-title-water-monitoring-reportweb-viewin-the-absence-of-competition.jpg)
![[Brochure] SevOne Infrastructure Performance Monitoring](https://img.pdfslide.us/doc/110x75/58ef2d931a28abeb2f8b4643/brochure-sevone-infrastructure-performance-monitoring.jpg)
