Good eveningandthanksforhavingmehere.TodayIwanttolookathowourrelationshiptotheworldchangeswhenwe’resurroundedbydevicesthatanticipateourneedsandactonthem.Thatmeansitsitsattheintersectionoftheinternetofthings,userexperiencedesignandmachinelearning,andalthoughpeoplehavedealtwithoneofthosedisciplinesbefore,Idon’tthinkthey’veeverbeencombinedinquitethewaystheyarenow,orwiththecurrententhusiasm.And,tobeclear:Iamneitherafanof,noracriticof,thesetechnologies.Ithinkthey’retoocomplextobereducedthatwayandtomaximizetheirpositiveimpactwehavetoactivelyengagewiththem,andthat’swhatthistalkistryingtodo.

Thetalkisdividedintoseveralparts:itstartswithanoverviewofhowIthinkInternetofThingsdevicesareprimarilycomponentsofservices,ratherthanbeingself-containedexperiences,howpredictivebehaviorenableskeycomponentsofthoseservices,andthenIfinishbyexploringsomespeculativeideasofwhatkindofimpactthey’regoingtohaveonus,asindividualsandasasociety.AtitscoreisanargumentthateverythingisgoingtobeconnectedtotheInternet,thatthosethingswilleachtrytopredictourimmediatefuture,andthatthisisgoingtofundamentallychangeourrelationshiptotheworld.

Acoupleofcaveats:- Mycurrentworkinthisfieldfocusesalmostexclusivelyontheconsumerinternetofthings,soIseemostthingsthroughthatlens.- IwanttopointoutthatfewifanyoftheissuesIraisearenew.Thoughtheterms“internetofthings”and“machinelearning”arehotrightnow,theideashavebeendiscussedinresearchcirclesfordecades.Searchfor“ubiquitouscomputing,”“ambientintelligence,”and“pervasivecomputing”andyou’llseealotofgreatthoughtinthespace.Ifyou’rereallyambitious,youcanreadtheArtificialIntelligenceandCyberneticsworksofthe50sand60sandyou’llbesurprisedbytheprescienceofthepeopleworkinginthisspacewhentheentireworld’scomputepowerwasaboutasmuchasmykeyfob.- Therearealotofideashere,andIwillalmostcertainlyunder-explainsomething.ForthatIapologizeinadvance.Mygoalhereistogiveyouageneralsenseofhowthesethepiecesconnect,ratherthananin-depthexplanationofanyoneofthepieces.- Finally,mostofmyslidesdon’thavewordsonthem,soI’llmakethecompletedeckwithatranscriptavailableassoonI’mdone.

0

Let me begin by telling you a bit about my background. I�m a user experience designer. I was one of the first professional Web designers. This is the navigation for a hot sauce shopping site I designed in the spring of 1994.

1

I’vealsoworkedontheuserexperiencedesignofalotofconsumerelectronicsproductsfromcompaniesyou’veprobablyheardof.

2

Now, aquickaside.WhatisUserExperienceDesign?UXdesignisnotgraphicdesign,interfacedesign,ergonomics,industrialdesign,orproductdesign,butitincludesaspectsofallofthosethings.

UXdesignisahumanisticproblemsolvingapproachthatbringstogethertheneedsofpeopleandbusinessestocreatetechnologicalproductsthatarevaluableforbothgroups.It’smuchmoreaboutprocessthanmakingthingslookgood.

Thefieldisabout20yearsold.Thisishowitlookedabout15yearsago.

DiagrambyJessMcMullin.

3

It’salittlemorecomplextoday,but’sroughlythesamething.

DiagrambyCoreyStern.

4

Iwroteacoupleofbooksbasedonmyexperienceasadesigner.Oneisacookbookofuserresearchmethods,andtheseconddescribeswhatIthinkaresomeofthecoreconcernswhendesigningnetworkedcomputationaldevices.I’malsomarriedtooneoftheauthorsofthisbook,sothinkingabouttheimpactofthedesignofconnecteddevicesonpeopleiskindofafamilybusiness.

5

Ialsostartedacoupleofcompanies.Thefirst,AdaptivePath,wasprimarilyfocusedontheweb, andwiththesecondone,ThingM,Igotdeepintodevelopinghardware.

6

TodayIworkforPARC,thefamousresearchlabthatinventedthepersonalcomputer,objectorientedsoftware,thetabletcomputer,andlaserprinter,asaprincipalinitsInnovationServicesgroup.Wehelpcompaniesreducetheriskofadoptingnoveltechnologiesusingamixofsocialresearch,designandbusinessstrategy.

7

PARCalsostartedthinkingaboutwhatwecalltheIoT longbeforemost othercompanies.

ItwasatPARCin1971thatDickShoup,andearlyPARCresearcher,wrotethateventuallyprocessorswouldbeascommon,andasinvisibleaselectricmotors.Thisclearlyoutlinesthedestinyofconnectedcomputer:thateventuallyitwillbecomeasboringandascommonaselectricmotorsaretoday.

8

Inthelate80s,alsoatPARC,MarkWeisercoinedthetermubiquitouscomputingtodescribeafuturewhenthenumberofcomputerssurpassedthenumberofpeopleusingthem.Inthischartfrom20yearsago,hepredictedthatwouldhappenaround2005.Hedidn’tlivetoseethatcrossover,buthewasbasicallyright—theiPhonelaunchedin2007—andwenowliveintheworldheenvisioned.

Essentially,whatwenowseeasanovelphenomenonhasbeenforseen bypeopleintheindustryformanydecades.Thequestionshavealwaysbeennotaboutwherewe’regoing,butwhenwe’llgetthere,andhow.

9

Buttheendvisiondoesn’tappearallatonce.We’veonlystartedthetransitiontotheubiquitouscomputingworld,andassuch,we’reseeingalotofbadideasaboutwhattheInternetofThingsisanditisn’t.Essentially,everythingthatcanbeconnectedtotheInternetwillbe,whichincludesalotofthingsthatshouldn’tbe.TherearesomanybadideasnowthatthereareentireTumblrs dedicatedtomockingstupidIoTideas.Oneisaboutdumbsmartthingsandtheotherisjustaboutdumbsmartrefrigerators.

10

Mostofthesethingsarebadideasbecausesimplyconnectingexistingstufftotheinternetdoesnotproducecustomervalue…

11

Simpleconnectivityhelpswhenyou’retryingtomaximizetheefficiencyofafixedprocess,butthat’snotaproblemthatmostpeoplehave.We’vebeenabletosimplyconnectvariousdevicestoacomputersinceaTandyColorComputerscouldlightsoffandonoverX10in1983.Theproblemisthatthatwasn’tveryusefulthen,andit’snotveryusefulnow.IfyoureplacetheTandywithaniPhoneandthelampwithawashingmachine…

12

…oraneggcarton,youstillhavethesameproblem,andit’sauserexperienceproblem.

TheUXproblemisthatendusershavetoconnectallthedotstocoordinatebetweenawidevarietyofdevices,andtointerpretthemeaningofallofthesesensorstocreatepersonalvalue.Formanysimplyconnectedproductsthereissolittleefficiencytobehadrelativetothecognitiveloadthatit’s justnotworthit.What’sworse,theextracognitiveloadisexactlyoppositetowhattheproductpromises,andcustomersfeelintenselydisappointed,perhapsevenbetrayed,whentheyrealizehowlittletheygetoutofsuchaproductThatmakesmostsuchproductseffectivelyWORSEthanuseless.

Thatpromisegapiswhatdistinguishesagadgetfromatool,whythiseggcartonisfunny,andwhyQuirkywhomadeit,filedforbankruptcyafterburningthroughhundredsofmillionsofdollars.

13

How doyoucreateatoolthatreducescognitiveloadinsteadofcreatingit,thatexchangespeople’sprecioustimeforsignificantvalue?Oneapproachistocouplecloud-basedserviceswithpredictivemachinelearningmodelstoanticipatewhatbehaviorswillmaximizethechancesofadesirableoutcomeinagivensituation.

14

WhenI talkaboutservices,I’mtalkingaboutthinkingofhardwaredevicesasphysicalrepresentativesofcloudservices,whichmakesthemverydifferentthantraditionalconsumerelectronics.Historically,acompanymadeanelectronicproduct,sayaturntable,theyfoundpeopletosell itforthem,theyadvertiseditandpeopleboughtit.Thatwastraditionallytheendofthecompany’srelationshipwiththecustomeruntilthatpersonboughtanotherthing,andallofthevalueoftherelationshipwasinthedevice.WiththeIoT,thesaleofthedeviceisjustthebeginningoftherelationshipandphysicalthingholdsalmostnovalueforeitherthecustomerorthemanufacturer.

15

Value now shifts to services and the devices, software applications and websites used to access it—its avatars—become secondary. A camera becomes a really good appliance for taking photos for Instagram, while a TV becomes a nice Instagram display that you don’t have to log into every time, and a phone becomes a convenient way to check your friends’ pictures on the road.

Hardware, physical things, become simultaneously more specialized and devalued as users see “through” each device to the service it represents. The avatars exist to get better value out of the service.

16

Amazon reallygetsthis.Here�satellingolderadfromAmazonfortheKindle. It’ssaying�Look,usewhateverdevice youwant.Wedon�tcare,aslongyoustayloyaltoourservice.Youcanbuyourspecializeddevices,butyoudon�thaveto.�

17

WhenFirewasreleased5yearsago,JeffBezosevencalled itaservice.

18

AmazonDashisaservicethat’senabledbydedicateddevices.ADashbuttonisanetworkedcomputerwhoseonlypurposeistobeanavatarforamacaroniandcheeseservice.

19

Mostlarge-scaleIoT productsareserviceavatars.Theyusespecialized sensorsandactuatorstosupportaservice,buthavelittlevalue—ordon’tworkatall—withoutthesupportingservice.SmartThings,whichwasacquiredbySamsung, clearlystatesitsserviceofferingrightupfrontontheirsite.Thefirstthingtheysayabouttheirproductlineisnotwhatthefunctionalityis,butwhateffecttheirservicewillachievefortheircustomers.Theirhardwareproducts’functionality,howtheywilltechnicallysatisfytheservicepromise,isalmostanafterthought.

20

ComparethattoX10,theirspiritualpredecessorthat’sbeeninthebusinessfor30years.AllthatX10tellsisyouiswhatthedevicesare,notwhattheservicewillaccomplishforyou.Idon’tevenknowifthereISaservice.WhyshouldIcarethattheyhave“modules”?Ishouldn’t,andIdon’t.

21

Ithink therealvalueconnectedservicesofferistheirabilitytomakesenseoftheworldonourbehalf,toreducecognitiveloadbyenablingpeopletointeractwithdevicesatahigherlevelthansimpletelemetry,atthelevelofintentionsandgoals,ratherthandataandcontrol.Humansarenotbuilttocollectandmakesenseofhugeamountsofdataacrossmanydevices,ortoarticulateourneedsassystemsofmutuallyinterdependentcomponents.Computersaregreatatit.

22

Theydothisthroughprocessesthathavemanynames,butI’lllumpthemallunderMachineLearning,whichisabigpartofwhatusedtobecalledArtificialIntelligence.Manyofthecoreideasheregobacktothe1950sandit’sthebasisofeveryemailspamfilter,soifyou’vehadyourspamautomaticallyfiltered,you’veexperiencedthevalueofmachinelearning.

AbigpartofMachineLearningispatternrecognition.Wehumansevolvedverysophisticatedfacultiestorapidlyidentifyvisualimagesinallkindsofdifficultconditions.Youlookatapictureofanorangeonaredplateandyoucantellinstantlythatit’snotasunset,butuntilrecentlythatwasreally,reallyhardforacomputer.BecauseofacombinationofMoore’sLawandsomebreakthroughs,computershavegottenmuchbetteratpatternrecognitioninthelastcoupleofyears.

Foracomputer,recognizingsomethingstartswithaprocesswheresomebasicattributesofanimageareextracted,suchastheshapeofboundariesbetweenclustersofpixels,orthedominantcolorofapatchofanimage.Thesearecalledfeaturesinmachinelearning.Byexamininglotsandlotsofexamplesoffeaturesinanimage,amachinelearningsystembuildsastatisticalmodelofwhatthatclusterrepresents.

Basicformsofthiskindofimagerecognitionhasbeenusedindustriallyfordecades.Mostoftheorangesthatcomefromthecentralvalleyarescanned360timestoseparateoneswithblemishesfromoneswithout.LegohasacompletelyautomatedfactorythatinjectionmoldsamillionLegobricksanhour,examineseverysinglepiece,automaticallysorts,bagsandboxesthem,allusingcomputervision.That’srelativelyold.

Imagesfrom:Region-basedConvolutionalNetworksforAccurateObjectDetectionandSemanticSegmentation,R.Girshick,J.Donahue,T.Darrell,J.Malik,IEEETransactionsonPatternAnalysisandMachineIntelligence

Real-TimeImageandVideoProcessing:FromResearchtoRealitybyKehtarnavaz andGemadia

23

What’snewisaclassofsystemsthatunderstandthecontentofimages.Theydon’tjustlookatfeatures,butclustersoffeatures,andclustersofclustersoffeatures,andtheycannowidentifyanorangefromthesettingsun,orapersonfromanairplane,orapolarbearfromadalmatian.

ThisiswhyFacebookasksyoutosaywhoisinanimage.It’snotjustforyou,it’sfortheirfacerecognizer.

Nowhere’stheinterestingpart:we’rebuilttoidentifypatternsinvisualphenomena,butwe’reprettybadatidentifyingtheminotherkindsofsituations.Forexample,ifyou’veevertriedtounderstandsomeone’sfoodsensitivities,it’sreallyhardtoextractwhatthatpersonisreactingto,evenifyoukeepverycarefultrackofwhatthey’veeaten.We’rejustnotbuiltforit.Itwasneverevolutionarilysufficientlyimportant,sowedidn’tevolveanorganforit.

Computers,ontheotherhand,don’tcare,andnowthatwe’vefoundreallygoodwaystofindpatternsinvisualimages,thesesametechniquescanfindpatternsinanything.

Insteadofamatrixofpixels,whatifyouhadamatrixofmedicalprescriptions,witheachrowasthehistoryofoneperson’sprescriptionsfromthefirsttimethatpersonwenttothedoctorforaproblem,throughwhentheywereprescribedcertainthings,towhentheygotbetter,ortheydidn’t.Thesamekindofsystemcouldlearnthetypicalpatternforprescribing,say,awheelchair.Itwouldessentiallyseethegeneralshapeofthesequencefortheprescriptionofachairovertimeandacrossmanypeople.

Thenifyousawawheelchairbeingprescribedthatwasoutsideofthetypicalpattern,youcouldidentifyit.That’scalledanomalydetection.That’sinfactexactlyhowwebuiltasystemtoidentifyMedicarefraudforthestateofCalifornia.Peopleareterribleatthatstuff,butcomputersaregreat.

24

Whenoneofthedimensionsistimeandanotheristheoutcomeofaseriesofactionsyoucanmakeapatternrecognizerthatassociatesasequenceofactionswithasetofstatisticalprobabilitiesforpossibleoutcomesbasedondatacollectedacrossawidevarietyofsimilarsituations.Inotherwords,becausepeopleandmachinesbehaveinfairlyconsistentways,thesemachinelearningsystemscanincreasinglypredictthefutureandattempttoadaptthecurrentsituationtocreateamoredesirableoutcome.

25

Theinterestingthingisthatthisnotjusttheory.

PredictionandresponseisattheheartofthevaluepropositionmanyofthemostcompellingIoT services,startingwiththeNest.TheNestsaysthatitknowsyou.Howdoesitknowyou?Itpredictswhatyou’regoingtowantbasedonyourpastbehavior.

26

Amazon’sEchospeaker saysit’scontinuallylearning.Howisthat?Predictivemachinelearningbasedonyouractionsandyourwords.

27

The Birdi smartsmokealarmsaysitwilllearnovertime,whichisagainthesamething.

28

Jaguar, learning…ANDintelligent.

29

TheEdyn plantwateringsystemadapts toeverychange.Whatisthatadaptation?Predictivemachinelearning.

30

Canary,ahomesecurity service.

31

Cocoon,anotherhomesecuritysystem knows.Howdoesitknow?Machinelearning.

32

Here’sfoobot,anairqualityservice.

[Ialsolikehowoneof itsimplicitservicepromisesistoidentifywhenyourkidsaresmokingpot.]

33

Silk’sSenseadapts

34

Mistbox sprayswaterintoyourairconditionertoreduceyourenergybill.You’dthinkthat’saprettysimpleprocess,butno,it’salwayslearning.

35

Anumberofcompaniesaremakingchipsthatmakemachinelearningmuchcheaperandmorepower-efficient,whichmeansthatit’sgoingtobeveryeasytoinstallitineverydevice,fromstreetlightstomedicalequipmenttotoys.It’snotjustlikely,it’sinevitable.Here’sonethatwasannouncedacoupleofweeksago.

36

Here’saKickstarter for an“AIButler”thatpostedearlierthismonth.Whatdoesitdo?Idon’tknow,butitlearns.

37

Theidealscenariothesethingspaintisprettyseductive.Imagineaworldofespressomachinesthatstartbrewingasyou’rethinkingit’sagoodtimeforcoffee;officelightsthatdimwhenit’ssunnytosaveenergy,andmacandcheesethatneverrunsout.Theproblemisthatalthoughthevaluepropositionisofabetteruserexperience,it’sunspecificinthedetails.Previousmachinelearningsystemswereusedinareassuchaspredictivemaintenance andfinance.Theyweremadebyandforspecialists.Nowthatthesesystemsareforgeneralconsumers,wehavesomesignificantquestions.Howexactlyhowwillourexperienceoftheworld,ourabilitytouseallthecollecteddata,becomemoreefficientandmorepleasurable?

We’restillearlyinourunderstandingofpredictivedevices,sorightnowtheproblemsareworsethansolutions.IwanttostartbyarticulatingtheissuesI’veobservedinourwork.

38

We’veneverhadmechanicalthingsthatmakesignificantdecisionsontheirown.Asdevicesadapttheirbehavior,howwilltheycommunicatethatthey’redoingso?Dowestickasignonthemthatsays“adapting”,likethelightonavideocamerasays“recording”?Shouldmychairvibratewhenadjustingtomyposture?Howwillusers,orjustpassers-by,knowwhichthingsadapt?Icouldendupsittinguncomfortableforalongtime,waitingformychairtochange,beforerealizingitdoesn’tadaptonitsown.Howshouldsmartdevicessettheexpectationthattheymaybehavedifferentlyinwhatappearstobeidenticalcircumstances?

How doweknowHOWintelligentthesedevicesare?Peoplealreadyoftenprojectmoresmartsondevicesthanthosedevicesactuallyhave,soacoupleofaccuratepredictionsmayimplyamuchbettermodelthanactuallyexists.Howdoweknowwe’renotjust homesteadingtheuncannyvalleyhere?

ChairbyRaffaello D'Andrea,MattDonovanandMaxDean.

39

Theironyinpredictivesystemsisthat they’reprettyunpredictable,atleastatfirst.Whenmachinelearningsystemsarenew,they’reofteninaccurate,whichisnotwhatweexpectfromourdigitaldevices.60%-70%accuracyistypicalforafirstpass,buteven90%accuracyisn’tenoughforapredictivesystemtofeelright,sinceifit’smakingdecisionsallthetime,it’sgoingtobemakingmistakesallthetime,too.It’sfineifyourhouseisacoupleofdegreescoolerthanyou’dlike,butwhatifyourwheelchairrefusestogotoadrinkingfountainnexttoadoorbecauseit’sbeentrainedondoorsanditcan’ttellthat’snotwhatyoumeaninthisoneinstance?Forallthetimesasystemgetsitright,it’sonthemistakesthatwejudgeitandacouplesuchinstancescanshatterpeople’sconfidence.Anxietyisakindofcognitiveload,andalittledoubtaboutwhetherasupposedlysmartsystemisgoingtodotherightthingisenoughtoturnaUXthat’srightmostofthetimeintoonethat’smoretroublethanit’sworth.Whenthathappens,it’s lostyou.

Photo CCBY2.0photo2011PopCultureGeektakenbyDougKline:https://www.flickr.com/photos/popculturegeek/6300931073/

40

Thelastissuecomesasaresultoftheprevioustwo:control.Howcanwemaintainsomelevelofcontroloverthese devices,whentheirbehaviorisbydefinitionstatisticalandunpredictable?

Ontheonehandyoucanmangleyourdevice’spredictivebehaviorbygivingittoomuchdata.WhenIvisitedNestoncetheytoldmethatnoneoftheNestsintheirofficeworkedwellbecausethey’reconstantlyfiddlingwiththem.Inmachinelearningthisiscalledovertraining.Theotherhand,ifIhavenodirectwaytocontrolitotherthanthroughmyownbehavior,howdoIadjustit?AmazonandNetflix’srecommendationsystems,whicharemachinelearningsystemsforpredictingwhatyoumaylike,giveyousomecontextaboutwhytheyrecommendedsomething,butwhatdoIdowhenmyonlyinterfaceisagardenhose?

41

Asinterestingastheseissuesare,Ithinkthat, moreimportantly,whattheyrepresentisthatwe’reentering intoanewrelationshipwithourdeviceecosystem,aseachangeinourrelationshiptothebuiltworld.

42

Thinkofasewingmachine.It’sverycomplex,butitstillonlyactsinresponsetous.

43

Computersacting autonomously erodethissimpletool/userrelationship.

Atthedawnofcomputinginthelate1940scyberneticistslikeNorbertWienerphilosophizedabouttheincreasinglycomplexrelationshipbetweenpeopleandcomputers,andhowitwasfundamentallydifferentthanthewayweinteractwithotherkindsofmachines.Developersworkinginsupervisorycontrolofmanufacturingmachinesandroboticshavehadtodealwiththesequestionspragmaticallyforabout30years,butthankstotheInternetofThings,thisisnowaproblemthateveryonewillhavetograpplewithgoingforward.

Here’sadiagrambythegreatsTomSheridanandBillVerplank from1978,inwhichtheyillustratefourwaysthatsemi-autonomouscomputersandhumanscanworktogethertosolveaproblem.

44

By2000Sheridanexpandedtheseideastocreatethisframework,todefineaspectrumofresponsibilitybetweenpeopleandcomputers.Itrangesfromhumansdoingallthework(thisisyouwritinganessay)tocomputersdoingalltheworkcompletelyautonomously(thisisyourcar ’sfuelinjectioncontroller).Ofcoursethegoalistogetasystemtolevel9or10.That’sthemaximumreductionincognitiveload.However,forasystemtoqualifyforthat,ithastobeverystable,itseffectsneedtobehighlypredictableand,equallyimportantly,it’sroleneedstobeadequatelyembeddedinsociety.ItneedstobeOKforacomputertotakeonthatlevelofresponsibility.Attheairportwetrustthemonorailcomputerstoworkwithouthumanintervention,butwedon’ttrusttheplaneautopilottodothat,eventhough-–asIunderstandit—planescanbasicallyflythemselvesthesedays.

PredictiveIoT devicesgenerallyfallbetween5and7onthisscalerightnow.Theproblemisthatthisistheexactrangewhereyou’remaximizingsomeone’scognitiveload,butnotnecessarilydoingalltheworkforthem,sotheresultoftheautomationhadbetterbeworthit.Thisfundamentallyundermineswhatweexpectfromourtools,andwhenthattoolistryingtoanticipatewhatwe’retryingtodo,itfundamentallychangesourworkingrelationshipwithit.

45

DannyHillis oftheLongNowtalksabouthowwehavegonepasttheEnlightenmentideawherewethoughtthatwecouldunderstandandcontroleverything,andbuilttoolsthatreflectedthatview.Inhisperspective,wearenolongerincontrolasmuchasweareentangledwiththem.

AnneGalloway,aNewZealandresearcherwholooksattheintersectionofanimalsanddigitaltechnology,callsittheendofhumanexceptionalism.Otherswouldsayit’sjustthePostmoderncondition,therecognitionthatthecomplexityoftheworldisbeyondourabilitytocontrol,andwehavetolearntocoaxandcoexist,ratherthancommandandcontrol.

46

Becausesoonerthanweexpect, we’llbelivingwithhundredsofdevicesandservicestryingtomodelusandpredictwhatwillbegoodforus,andmostofthemwillrequireourattention.Theywillwantustoverifythings,touploadthings,toconfirmthings.Theywillwantustovalidatetheirexistence.Andtheywillbewrongalot.Ifyouhave100devicesandeachdevice is99%accurate—andmostpredictivealgorithmsrarelyachievethatlevelofaccuracy,atleastnotatfirst—thenoneisalwayswrong.

So howdoweengagewiththisworld?Howdoweapproachwranglingallthesethinkingtools?

47

Youcanthinkaboutworkingsurroundedbyabunchofapprenticeassistants,asinamiddleageguild.

48

…oryoucantakeananimistviewofassumingeverythingintheworldhasaconsciousness.PhilVanAllenofArtCenterhasrecentlystartedadvocating anapproachlikethis.Well,maybenotlikeTHIS.

ImagefromMiyazaki’sPrincessMononoke.

PhilVanAllen:https://medium.com/@philvanallen/rethink-ixd-e489b843bfb6#.6jszlfw9p

49

I’dliketoexplorefarmingasametaphor,andnotbecauseofthesuperficialironyofusingpre-Enlightenmenttechnologytotalkaboutapost-Enlightenmentproblem.

Ireallywanttocreateausefulwayofthinkingaboutthechallengeofsmarttoolssowecandesignabetterrelationshipwiththemfromthebeginning.

50

Farmingis oneofouroldesttechnologies,oneofthemostadvanced,andoneofthemostbrutalontheland,peopleandanimalsinvolved.Butitgotushere.

Also,anadmission:I’macitykid,myfamilyhasbeenlivingin citiesgoingbackmanygenerations.Ihavenotraisedsomuchasasingleedibleplantorownedapet,thoughIdohavechildren,butIdon’tthinkit’sthesame.Butthelongnowaskedmetodosomethingbrandnewandforageneralaudience,andthisiswhereIendedup,soifthistalkhasn’tgoneofftherailsforyouyet,it’llprobablygoofftherailsnow.

51

For mefarmingisausefulmetaphorabouthowtosimultaneouslymanipulate thestateofmanyautonomous,independent,similarthings,foryourgain.Afarmerdoesn’traiseanearofcorn,sheraisesafieldofcorn,andsheisnotincontroloftheircropsasmuchassheisinsymbiosiswiththem.

Shereducesthecomplexityoffarmingbyplantingmanycopiesofthesameplant,anddividingherlandintoregionsforeachkindofplant.Rightnowslikeeachplantistotallydifferentandrequiresatotallydifferenttechniquetoworkwithit.

Sheselectscropsthatthriveinaspecificsetofconditionsandwhichcansynergisticallyusethesamerawmaterialtomaximizethevalueofthatmaterial.Whatifhadmultiplealgorithmsusingtheinformationfromthesamesensors—sayallthecamerasandtemperaturesensorsinyourenvironment—thenfusingtheirresults?

52

Afarmerusesspecializedtoolstoworkonmanyplantsatthesametime,whetherit’saplow,aharvesterorascarecrow.That’swhyshechoosesmanyofthesamething.Inthealgorithmanalogy,howcanwegrouplargenumbersofalgorithmsandworkonthemallatonce?

Sheexpectspests.Rightnoweveryoneisshockedwhentheirsmartfridgestartspostingspambecauseit’sbeenhacked.That’skindoflikeafungusinfection,andfarmershavetoolsforthatandtrytomaintaingoodpracticestominimizeit,butwhenithappens,nooneissurprised.

Shedoesn’texpecttoextractthevaluefromitimmediately—thatmaytakemonthsoryears—yetsheknowsshewillhavetomaintainitthatwholetimeregardless.Rightnowweexpectourdigitalproductstoworkimmediatelyorwethinkthey’renotworthwhileordefectiveiftheydon’t.Whatifwedesignedthingssothattheywouldonlybeusefulafterwehadlivedwiththemforalongtime,butthenthey’dbeREALLYuseful?

53

Anotheraside:machinelearningalgorithmsarepatternrecognizers,sotheyneedtoknowwhichpatternsareimportant.Wheneveryoumarkemailasspamusingyouremailprogram,youaredoingwhat’scalledtrainingthealgorithmtounderstandwhatyouconsiderspam.

54

Similarlywhenyoumakeachoiceusingvirtuallyanyadigitaldeviceorservice,you’retraininganalgorithm.Facebookasksyoutolabelpeopleinyourpicturestotrainitsalgorithmstoassociateasetoffacialfeatureswiththepersonyoulabeled.

Whathappenswhenyoutrainasingleanimal?Whatareyourmechanismsofcontrol?Whatareyourexpectations?

Well,youexpectthatitwillrequiretimeanditwillrequireacombinationofbothpositiveandnegativereinforcement.Then,youexpectthatitwillregularlymisbehaveandyouhavetoreinforcewhatyouteachit.Conversely,youcanexpectthatitwillprobablylearnabitfromotheranimalswithoutyouhavingtotelliteverythinganditsbehaviorwillsurpriseyouingoodwaysinadditiontobadways.

Imagesource:http://countingsheep.info/permalamb.html (AnneGalloway’sCountingSheepproject)

55

Butwhathappenswhenafarmerhasalotofanimalstocontrol?Shecan’ttrainallofthemindividually,sooverthelast10000yearsshe’sdevelopedsometoolsformanagingthem.

First,sheselects animalsthatworkwellingroups.Ouralgorithmsarecurrentlybuiltoneatatimeandtheexpectationisthatourinteractionwiththemwillbeindividual.Thatdoesn’tscale.Weneedalgorithmsthatareexperiencedwelltogether,orelsewe’renotherdingsheep,we’reherdingcats.Next,shehasacrook.Whenyouneedtoassertcontrol,youneedaclearwaytodothatwhichworksonawidevarietyofanimalsandweneedconsistentwaystoassetimmediatecontroloverawidevarietyofsmartdevices.Shehas adog, whichisasmarterentitythatalso needstobetrained,butoncetrainedcanbeusedtoautonomouslycontrolmultipleotherindependententitiesitself.Shecanhandofftheworktoanassistant.Infarmingawholeclassofpeoplewhocantakeresponsibilityforallofthethingsandwhocanworktogether.Responsibilitycanbedelegated.AsTomCoatesofThington pointsout,mostIoT systemsarenotbuiltformanypeopletocontrolthemsimultaneously,eventhoughtheireffectsareoftenexperiencedinsharedenvironments.

56

Todaywedon’thaveanInternetofThings,wehavemanyAOLsofthings.They’vebeenintentionallymademutuallyincompatibleandalthoughsomemaybecuteontheirown,whenyouhavealotofthem,andtheyhavetobedealtwithindividually,it’sabigproblem.

57

Ithinkin 1000years,maybe100years fromnow,thisentirediscussionwillseemabsurd,likearguingaboutwhetherironisagoodthingorabadthing.We’llseeitasjustthewaytheworldis.Ourbodiesaregoingtobesemi-autonomouscomponentsthatwehavesomecontrolover,inanecosystemthatcombinesotherbiologicalanddigitalsemi-autonomouscomponents.Everythingisgoingtohavesomecontroloverandbecontrolledbyotherthings.

Someofthemaresmarterthanothers,somearemoreautonomousthanothers,someareevensmarterthanweareincertainways,somehavepositivesymbioticrelationships,someareparasites.Theboundariesbetweenmindsandbodies,betweennaturalandartificial,andbetweenhumanandnon-humanwillhavebeeneroded.Ourworldwillhavereconfigureditselfaroundassumptionsthateverythingismuchpermeableandmuchlessclearlydelineatedthanwehadfooledourselvesintobelieving.Wearenotasgods.Weare,andalwayshavebeen,animalsinanecosystem.

Anditwon’tallbegood.Therewillprobablybeterriblethingsthathappentopeople’sbodies,mindsandsocieties.Theremayalso,hopefully,begoodthings.

Image:CamillePissarro,“Shepherdesses,”1887

58

Thisisthe lookingglassthatwe’vemade,andit’stimeforustostepthrough,andexplorethefield beyond,becausewehavenochoicebuttoengagewithit,tomakeitbewhatwewantittobe,whatweneedittobe,becauseitisnotandroidswhowilldreamingofelectricsheep,itwillbeus.

59

Thankyou.

60