Augmen'ngIntellectthroughWearablesandAr'ficialIntelligence

ProfessorThadStarnerContextualCompu'ngGroupGeorgiaIns'tuteofTechnologyTechLead/Manager,GoogleGlass

PathstoAugmen'ngIntellect

•  Networkedhumanminds

•  Ar2ficialIntelligence•  AugmentedIntelligence

LarryPage&SergeyBrin

•  Reducethe2mebetweeninten2onandac2on.

ProvidingBeGerCare

ProvidingBeGerCare

ImprovingJobTraining

ImprovingJobTraining

ImprovingAccesstoInforma'on:Extrac'onDiagram

ImprovingAccesstoInforma'on:Naviga'on

ImprovingCommunica'on:SpeechTransla'on

ImprovingCommunica'on:SpeechTransla'on

ImprovingCommunica'on:TextTransla'on

ImprovingCommunica'on:RemoteExpert

ImprovingCommunica'on:Transcrip'onforHardofHearing

ReducingDisability

WhyNow?

•  Bluetooth4.0LowEnergybodynetworkstandardonmobilephonesstar'ng2013-now

•  Available,highqualitymobilespeechrecogni'on~2010•  Lowlatencycellularnetwork~2008:LTE/HSPDA•  Windowsinterfacemonopolybrokenbysmartphones~2008•  Highsensi'vityGPS~2007•  Qualityplas2cop2cs~2003•  GPSselec2veavailabilityeliminated2000•  Highresolu'onsmalldisplays~2000•  DC-DCconverters70->95%efficiency~2000•  SmallIMUs~1998•  ConsumerlithiumbaXeries~1997•  Powerefficientprocessors(StrongARM)1996•  CMOScamera1995->2007

70YearsofInvestment

VannevarBush1945•  Na2onalScienceAdvisor•  OSRD(scienceforwarfare)•  Raytheon•  NASAprecursor•  NSFprecursor

IvanSutherland1966

Microdisplays:Upton@Bell(1967-1980s)

•  Cuedspeech•  HelicopterHUDsfordefense•  Scanningfiberbundles

Microdisplays:Upton@Bell(1967-1980s)

•  PrivateEye•  LEDsandscanningmirrors•  Military&beginningsof

industrialinterest•  ConsumerVR1995

Microdisplays(1990-now):Panels(Kopin,Displaytech,etc.)

Networking:802.15

•  StartedasAdHocCommiXeeMee2ngforWearablesStandards1997

•  LedbyDickBraleyatFedex•  IncorporatedBluetoothandZigbee

Sensing:CMOSCamera(early1990s)

•  EricFossum•  NASAJPL(1993-1995)

→PhotobitCorp.→webcam→cameraphones

Sensing:Iner'alheadtrackers(late1990s)

•  EricFoxlin,MIT(NASA,AFOSR,NRL)•  Intersense(SBIRs)•  3x3mmhighprecisionIMUchips

today

Sensing:GPS

Sensing:SpeechRecogni'on

•  BellLabAudrey1952;IBMShoebox•  HMMs:Ins2tuteforDefenseAnalyses(60s)•  DARPASpeechUnderstandingResearch1971-1976:CMU

Hearsay,BBNHWIM,…•  DARPA/NISTTasks80s-90s(ATIS,Switchboard,WSJ,Resource

Management):CMUSphinx,BBNBYBLOS,SRIDECIPHER

Sensing:SpeechRecogni'on

Suddenly!WearableCompu'ng!

Symbio'cAI

Interweaveacomputerwithsomethingthatisalreadyintelligent(i.e.,theuser)• Putcameraswheremyeyesare• Putmicrophoneswheremyearsare• Putmo2onsensorsatmyhands• Ifpossible,lookintobrainforfocusofaXen2onand“intent”

Learnhowtointeractwiththehumanworldbyobservingahumaninterac2ngwiththeworld

Bonusslides

ImprovingLearning(op'onal)

PHL&PHR(MorseCodeonGlass,audioforpianoonGlass,Braille,etc.

DouglasEngelbart1962

By'augmen2nghumanintellect'wemeanincreasingthecapabilityofamantoapproachacomplexproblemsitua2on,togaincomprehensiontosuithispar2cularneeds,andtoderivesolu2onstoproblems…weincludetheprofessionalproblemsofdiplomats,execu2ves,socialscien2sts,lifescien2sts,physicalscien2sts,aXorneys,designers—whethertheproblemsitua2onexistsfortwentyminutesortwentyyears…wayoflifeinanintegrateddomainwherehunches,cut-and-try,intangibles,andthehuman"feelforasitua2on"usefullyco-existwithpowerfulconcepts,streamlinedterminologyandnota2on,sophis2catedmethods,andhigh-poweredelectronicaids.

DouglasEngelbartNLS/Augment

•  themouse•  2-dimensionaldisplayedi2ng•  in-fileobjectaddressing,linking•  hypermedia•  outlineprocessing•  flexibleviewcontrol•  mul2plewindows•  cross-fileedi2ng

•  integratedhypermediaemail•  remoteprocedurecallprotocols•  hypermediapublishing•  documentversioncontrol•  shared-screenteleconferencing•  computer-aidedmee2ngs•  formasngdirec2ves•  context-sensi2vehelp•  distributedclient-server

architecture•  uniformcommandsyntax•  universal"userinterface"front-

endmodule•  protocolsforvirtualterminals

JCRLicklider(DARPAIPTO)1960

“Man-computersymbiosis”isasubclassofman-machinesystems.Therearemanyman-machinesystems.Atpresenthowever,therearenoman-computersymbioses....Thehopeisthat,innottoomanyyears,humanbrainsandcompu2ngmachineswillbecoupledtogethervery2ghtlyandthattheresul2ngpartnershipwillthinkasnohumanbrainhaseverthoughtandprocessdatainawaynotapproachedbytheinforma2on-handlingmachinesweknowtoday.

RodBrooks:EmbodiedCogni'on

PursuingConsumerProduct:Possible→Prac'calin25years

Interface:Consumer(GeorgiaTech&MIT)

(USAirForceLab.Grad.Fellow,IndustrialConsor2a,NSF)

Interface:TaskGuidance(CMU)

(NSFERC,NSF,DARPA)

Interface:AugmentedReality(Columbia)

(ONR, NSF)

Microdisplays:VRHMDs(early1990s)

(NASA,DARPA)

Microdisplays:Military,Industrial,Hobbyist(90s)

Microdisplays:VideoViewers(early2000s)

Microdisplays:On-the-goDisplays

(MicroOptical 1997) (Google Glass 2013)

BonusSlides

OtherTopics

•  Websearch(GoogleNSF)•  Ques2onanswering(IBMWatson,govtacademia,industry)•  Boneconduc2onaudio•  Auto-awesome•  BaXeries•  DC-DC