November 13, 2017 Sam Siewert

CS415Human Computer Interaction

Lecture 12 – Ubiquitous ComputingVR, AR, IoT, Pervasive (Ch. 20)

Remaining AssignmentsAssignment #5 – Explore HCI’s, Propose Group Project (Groups of 2, 3 or 4)

Assignment #6 – Proof-of-Concept or Evaluation with Final Report

Final Oral Exam – Presentation of Project

STEM Room 107 (Inside Robotics Lab), [FINAL: 10:15-12:15 Thursday, 12/14]– Oral Presentation of Assignment #6 Report– With Updates for any Progress since

Sam Siewert 2

Advanced HCI … Future

Microsoft Research HCI Vision

Interesting HCI Innovations and Experiments (Examples)– MS Hololens, – Magic Leap, – jibo, – sixthsense, – Oculus VR, – NVIDIA AR/VR, PX, – Intel Real Sense, – Google Project Tango, – Interactive Film, – Gear VR

Sam Siewert 3

Ubiquitous ComputingTed Talk on Minority Report Style UIPranav Mistry Ted Talk on InteractionDavid Merrill Ted Talk on SiftablesMIT Media Lab – “reinvent—how humans experience, and can be aided by, technology.” - HistoryMIT Media Lab Founder Ted TalkJohn Barrett, IoT Ted Talk (Cork Inst. Of Tech – Embedded Systems Research)Marco Annunziata, GE, IIoT Ted TalkJPL – AR for Mars Exploration

Week 12 … Augmented Reality, VR, IoTWeek 13 … Advanced Avionics and UAV/UASWeek 14 – Quiz, Review for Exam #2, A#6 HelpWeek 15 – Exam #2Final Oral Exam - 12/14

Sam Siewert 4

Emergent Markets for Ubiquitous ComputingVirtuous link between Games & Animation (Digital Cinema) and HPC (High Performance Computing)

– Clusters (ILM, Pixar, Dreamworks, …)– GPU and GP-GPU (NVIDIA, ATI, Intel)– Serious Gaming (Education, Therapy, Social Science)

Transportation (Industrials)– Movement of people, animals, goods and commodities– Current modes – air, rail, road, water, cable, pipeline, space– UAS – new aerial transportation for goods at lower cost?– Self-Driving Cars – new road transportation for people, animals,

goods at lower cost?

Energy & Utilities (IIoT)– Smart GRID – efficient generation and distribution– Alternative Energy Sources and storage– Clean Nuclear and Coal (e.g. TeraPower)– Carbon sequestration

Services (IT and Telecomm)– Mobile and Wireless for IoT, IIoT (5th Generation)

Health Care (IoT)– Elder care (at home)– Therapeutics at home– Patient monitoring, Health and Fitness monitoring– Big data analytics – pathology, disease cures, control and

prevention

Sam Siewert 5

Nov, 2017

Games

Cable, Internet, Mobile

HPC, PC, BI, Enterprise systems

Enterprise systems

Systems

Cyberphysical

Cyberphysical

Cyberphysical

DefinitionsUbiquitous Computing– Merge computational artifacts smoothly with world of physical artifacts

(applications integrated with things)– Superimpose relevant information over top of the object– Objects in the physical world to manipulate virtual (graspable)– Context aware applications (Who, What, Where, When, Why) –

Recommendation Engines, Adware, Personal Assistants– Vannevar Bush (Memex) – Often compared to Hypertext Today

VR (Virtual Reality)– Immersive– Desktop– Simulated World

AR (Augmented Reality)– Project Onto World– See World Through Mobile Screen / Glasses With Annotation– Requires Computer Vision (Object Recognition) for True AR

Sam Siewert 6

Internet of ThingsA new form of Ubiquitous Computing?

Build processing into industry and everyday life– IIoT – Industrial Internet of Things

E.g. Honeywell – IIoT, Intel IIoT, GE IoT– IoT – Everyday and Workday Internet of Things

E.g. Intel IoT, IBM IoT– Network infrastructure for IoT (Wireless)

E.g. ATT IoT, Verizon IoT, Cisco IoT

Security Challenges (Cyberphysical)– Classic Risk, Benefit, Impact, Cost to Mitigate Design Trade-offs– Remote Monitoring and Proactive Maintenance

Sam Siewert 7

IoT ProcessorsVery Small, Wireless (BLE), Wired Ethernet, Built-in Sensors, Very Low-Power

E.g. Intel’s Quark, NXP, Texas Instruments TM4C, CC32XX

Arduino – Education, Makers and Quick Prototypes

Zephyr (Linux Foundation) –Micro / Nano kernel

Simple Executives – E.g. Tiva with PDL

Sam Siewert 8

NSF View of IoTBenefits and Risks of the “Internet of Things”

Embedded Computing –Cyberphysical Systems, Extension of Embedded Systems (Networked)

Sam Siewert 9

Ubiquitous ComputingRelation to Cyberphysical SystemsRelation to IoTRelation to Embedded Systems

Sam Siewert 10

Cyberphysical Systems (M2M,HCI,Secure)

Embedded Systems

IoT

Ubiquitous Computing(context aware, merged physical with virtual)

Ubiquitous ComputingTerm Credited to Mark Weiser– “The Computer of the 21st Century”, Scientific American,

265(3):66-75, September 1991– Vision Described Previously at Many HCI Research

Organizations [e.g. Xerox PARC, MIT Media Lab, GIT, …]

Some Enabling Technologies– High speed wireless networks– Computer Vision – Generalization of Machine Vision (for

Automation) to Use Visual Perception in the Field and to Emulate Human Visual Perception

– Projection, 3D, High Resolution Large Displays and Mobile Computing (Fields of View: Inches, Feet, Yards)

– Head tracking, Independent Display Eyeware, 3D Glasses (Polarized, Shutters)

– Large Scale Interactive Surfaces (Large Touch Screens)– Data Gloves and Body Suits

Sam Siewert 11

Playing with ConceptsCost Reduction is Key to Become Ubiquitous (Killer App)MERL DiamondTouch (2003)CU Computer Vision Chess, Automatic Duck Hunter

Sam Siewert 12

VR, AR Killer ApplicationsRemains to Be Seen

Today– Scientific Visualization (VR)– Digital Cinema Animation (VR, AR)– Training Systems (Surgery, AR Overlay to Identify Objects)– Games and Entertainment (VR)– First Person Viewing and Sensing (Remote Presence), E.g.

UAVs (AR)

Ubiquitous Use – Not Yet?– Cost and Convenience (Pranav Mistry Pendant – Good Start -

$350, Easy to Wear)– Compelling Value (Wear Computer or Build into Things – IoT)

Sam Siewert 13

New HCI is RiskyDoes the HCI Define the Product?Is this the Right Place to Innovate the HCI?Catalogue of Catastrophe – Why Products Have FailedClassic Mistakes [According to One PM Consulting Firm]1. The underestimation of complexity, cost and/or schedule – NRE

Cost, Capex, Opex2. Failure to establish appropriate control over requirements and/or

scope – Creeping Featurism3. Lack of communications – Expectations and Goals4. Failure to engage stakeholders – Users and Evaluation5. Failure to address culture change issues – Social Acceptance6. Lack of oversight / poor project management – Micro-management

vs. No management7. Poor quality workmanship - SQA8. Lack of risk management – Initial assessment and adjustment9. Failure to understand or address system performance

requirements – Interactive as well as batch systems10. Poorly planned / managed transitions - Deployment

Sam Siewert 14

Minute PaperBeyond Today’s Vertical Applications Previously Enumerated…

VR, What is it Good for?

AR, What is it Good for?

IoT, What is Good for?

Will Ubiquitous Computing Envisioned by Mark Weiser be Realized?

Sam Siewert 15