Embed Size (px)
Citation preview
DIYIoTBackendVoonSiongWONG
greenfields
greenfields4yearsago,wantedIoTplatform
noclearrequirementsgreworganicallyflawsinhindsightneverinproduction,neverfixedhobbyproject
revisitedtheprojectmadelistofgood,bad,andhowitcanbebetter
development~2weeks,~800LOCRubyrunson1xt2.smallinstanceresponsetimeaverage50ms,95%<80ms
learntfromalotofbadexperiences
disclaimernotnecessarilyadvocatingDIY
outsourcingto3rdpartyvendorscanbecheaper
dependsonin-housecapabilities
Let'sGetStarted
item1:timeseriesdon'tDIYtimeseries
v1hadtodoit,notmanyopensource
v2usedopensourcedatabase,threwoutatonofcode
read/writeoptimisationishard
use1-Dsortforindexoptimisationusuallytimeasprimaryindexoverwriteorappendforsametimestamp?aggregatingacrossmultipletables?
geo-spatialcapabilities
eg.measurementsinthisareaatthistime4-D"sort"
wetriedbuildingonebasedon
carefulplanningofRow-key&Column-key
onlysupportedoverwrite,noappends
fixedtimeintervals
aggregationwerescheduledtasks,ratherthanqueries
poorperformance
HBase
unlessyourjobistobuilddatabases,don't
outsideofyourcontrolifyou'reusingvendor
item2:authenticationtreatbrowsersandiotdevicesequallyasfirstclasscitizens
accessiblewithfixedorvariableAPIkeys
optionallywithusername/password
avoidclientSSL
cumbersometogenerate/manageimpracticaltoinstallonbrowsersthusrequiresproxyserver
useAPIKeys
fixedAPIkeysvariableAPIkey,HMACrequestandtimedeploywithenvironmentvariables
interactivelogins
username/passwordoauth
item3:expectchangeplanforit
veryrelevantwhenDIY
butuseascriteriaforjudgingvendors
manyvendorsarestillyoung
ensureAPIsareversioned
usingpath,headers,requestparams,etc.
deprecations
devicesdon't/can'tunderstandwarningmessageswarndevelopersdirectly
returnsuccessondeadAPIs
don'tcareabouttheirbehaviouranymoreavoidolddevices'sretrylogicfromhammeringit
APIownershipAPIscanmakeorbreakyourapplication
oneofstrongestreasonstoDIY
don'tforgetthatitchanges!
Item4:modelassociationsdon'tenforceaschemaformodelrelationships
embraceschema-less,becausesensorsare
schema/syntactic
fixedassociations,inflexibleneedtoknow"position"inhierarchy/groupingconflictswhenreusingnames
schema-less/semantic
uniquenessbycredentialsandnamehierarchiesorgroupingstoredintext,eg."state/suburb/street",arbitrarilydelimited
item5:dataflowstricklesfromdevicetoserver
streamsfromservertoadvancedclients
useregularsession-lessHTTPforsmallvolumes
forgreatervolume,useapub-submechanism
,subscribetodesiredchannels
modernclientsuse(uni-directional)websockets,resumeonconnectionfailuresolderclientsfallbacktolong-polling
firehoseserver
item*:nuts&boltsmeasurementunits
overwritevsappend
don'trequireSNI
prefertimesinceepoch
textoverbinary
IoTBackend:SolvedProblem?
IoTforhireauthentication,dataaggregation,remotecontrol,etc.forhire
stillmaturingsector...
focusonvaluethislistapplywhetheryou'reDIYbackend,orevaluatingvendors
notsuggestingoneortheother,that'suptoyou
suggestyouthinkaboutvalue,ratherthantechnicalcapability
Ican,shouldI?
worthit?inourcase,DIYwasworthit
(re)developmenteffortwassmall
lessonsfrompreviousgeneration
in-houseknowledgeonhowtofixthings
Real-time Energy Monitor
Your Energy Use
CONSUMPTION
6.06kW
COST
$1.51per hour
HISTORY
kW
6
5.8
6.2
6.4
10:40 10:45
Last 10 mins
2015 DiUS Computing Inc.
22/09/2015@10:00
22-September-2015
+-
SmokeAlert Enter a location Legend About Privacy Terms
disclaimer(repeat)notnecessarilyadvocatingDIY
outsourcingto3rdpartyvendorscanbecheaper
dependsonin-housecapabilities
Thanks!
