Challenges in Ubiquitous Computing

Challenges in Ubiquitous Computing

Patrick MalatackPatrick Malatack

Today’s ReadingsToday’s Readings

Charting Past, Present, and Future Research in Ubiquitous Computing by Gregory Abowd and Elizabeth Mynatt

The Origins of Ubiquitous Computing Research at PARC in the Late 1980sby Mark Weiser, Rich Gold, John Seely Brown

Some Computer Science Issues in Ubiquitous Computing by Mark Weiser

Charting Past, Present, and Future Research in Ubiquitous Computing by Gregory Abowd and Elizabeth Mynatt

The Origins of Ubiquitous Computing Research at PARC in the Late 1980sby Mark Weiser, Rich Gold, John Seely Brown

Some Computer Science Issues in Ubiquitous Computing by Mark Weiser

Charting Past, Present, and Future Research in Ubiquitous ComputingCharting Past, Present, and Future Research in Ubiquitous Computing

Natural interfacesContext-aware applicationsAutomated capture and accessContinuous interactionPrivacySecurityVisibilityControl

Natural interfacesContext-aware applicationsAutomated capture and accessContinuous interactionPrivacySecurityVisibilityControl

Computing with Natural InterfacesComputing with Natural Interfaces

Ubicomp inspires off the desktop applicationsNeeds off the desktop means of interactionSpeech, gestures, writing

More accessibleEasier to use???

Ubicomp inspires off the desktop applicationsNeeds off the desktop means of interactionSpeech, gestures, writing

More accessibleEasier to use???

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Problems with Natural InterfacesProblems with Natural Interfaces

First Class Data TypesKeyboard and mouse

Characters and x,y coordsMost natural interfaces try to convert to first class type

Pen to text, speech to textCurrent Research is working with freeform input as a

primitive.Need to find other ways of structuring inputWhat other problems does creating new primitive types

imposeDevelopment issues

First Class Data TypesKeyboard and mouse

Characters and x,y coordsMost natural interfaces try to convert to first class type

Pen to text, speech to textCurrent Research is working with freeform input as a

primitive.Need to find other ways of structuring inputWhat other problems does creating new primitive types

imposeDevelopment issues

Problems with Natural InterfacesProblems with Natural Interfaces

Error Prone Interaction Permit new and numerous

mistakes People do not have perfect

recognition People don’t recognize

increases in accuracy Needs to be 5-10% to be

recognized Three areas

Error Reduction Error Discovery Reusable toolkit for error

handling

Error Prone Interaction Permit new and numerous

mistakes People do not have perfect

recognition People don’t recognize

increases in accuracy Needs to be 5-10% to be

recognized Three areas

Error Reduction Error Discovery Reusable toolkit for error

handling

Questions about Natural InterfacesQuestions about Natural Interfaces

Can we ever capture subtleties of natural human communication??

What different natural interfaces might exist?? Nintendo Wii

Trauma Center, Wii Sports iPhone

How is accidental communication handled? StarTrek

“Computer:play music” Why do developers want to build apps for these

interactions? More errors, more edge cases

Can we ever capture subtleties of natural human communication??

What different natural interfaces might exist?? Nintendo Wii

Trauma Center, Wii Sports iPhone

How is accidental communication handled? StarTrek

“Computer:play music” Why do developers want to build apps for these

interactions? More errors, more edge cases

Context Aware ComputingContext Aware Computing

Current Systems Generally using position and

identification of objects Still do not provide a complete

context Definition of context is limited

Research Areas Context Toolkits

Toolkit for sensing environmentExplicit use of sensed

information is up to program What is Context? How is Context represented?

Current Systems Generally using position and

identification of objects Still do not provide a complete

context Definition of context is limited

Research Areas Context Toolkits

Toolkit for sensing environmentExplicit use of sensed

information is up to program What is Context? How is Context represented?

What is Context?What is Context? Who

Currently generally tailored to one user How important are others in determining our behavior

How could this be captured? What

Attempt to figure out what is currently happening Sense environment, use calendar software etc. What is the cost of the wrong guess?

TiVo Problem Where

Location based information GPS Most explored of context information

When Easily obtained information - Computer good a remembering time

Although determining when one event stops and another begins is not easy Why

Even harder than the “what” question, biometric sensors might help

Who Currently generally tailored to one user How important are others in determining our behavior

How could this be captured? What

Attempt to figure out what is currently happening Sense environment, use calendar software etc. What is the cost of the wrong guess?

TiVo Problem Where

Location based information GPS Most explored of context information

When Easily obtained information - Computer good a remembering time

Although determining when one event stops and another begins is not easy Why

Even harder than the “what” question, biometric sensors might help

Other Problems with ContextOther Problems with Context How do computers represent context

No universal context schemes, toolkits must be developed with standard context representations

Context may contain a lot of information with only a few pieces relevant for determining context.

Context Sensing and Fusion Sensors are not as flexible as human sensors

Right sensor for right jobUse multiple sensors in parallel to offset noise

Bit of a chicken-and-egg problemHow do you know when to switch sensor? Use context

How do computers represent context No universal context schemes, toolkits must be developed

with standard context representations Context may contain a lot of information with only a few

pieces relevant for determining context. Context Sensing and Fusion

Sensors are not as flexible as human sensorsRight sensor for right jobUse multiple sensors in parallel to offset noise

Bit of a chicken-and-egg problemHow do you know when to switch sensor? Use context

Automated Capture and AccessAutomated Capture and Access Recording information and data as it occurs

Computers are inherently good at recording, people are not People freed up to summarize and understand Most work in academic/ classroom settings

Time stamping lectures, digital whiteboards Challenges in capture and access

Sometime we don’t know we want to capture something until after its already happened

How could the computer know that? If it captures everything then we need a system of sorting and

filtering (access) Gmail Problem

Access is a problem because capturing of raw data can be burdensome for sifting through…systems need to recognize important events facilitate access

Recording information and data as it occurs Computers are inherently good at recording, people are not People freed up to summarize and understand Most work in academic/ classroom settings

Time stamping lectures, digital whiteboards Challenges in capture and access

Sometime we don’t know we want to capture something until after its already happened

How could the computer know that? If it captures everything then we need a system of sorting and

filtering (access) Gmail Problem

Access is a problem because capturing of raw data can be burdensome for sifting through…systems need to recognize important events facilitate access

Challenges Everyday ComputingChallenges Everyday Computing

No clear beginning and end Both fundamental activities like communication and Long-term endeavors do

not have predefined starts and ends Traditional HCI deals with transition users from current state to a goal state

Not necessarily a goal state

Interruptions People are constantly interrupted Constantly on computer systems must recognize interruption and change

state Also computers must appropriately interrupt users

Concurrency People multitask and rapidly switch task based on external unpredictable

environment Systems need to adapt to this opportunistic behavior and change accordingly

No clear beginning and end Both fundamental activities like communication and Long-term endeavors do

not have predefined starts and ends Traditional HCI deals with transition users from current state to a goal state

Not necessarily a goal state

Interruptions People are constantly interrupted Constantly on computer systems must recognize interruption and change

state Also computers must appropriately interrupt users

Concurrency People multitask and rapidly switch task based on external unpredictable

environment Systems need to adapt to this opportunistic behavior and change accordingly

Research in Everyday computingResearch in Everyday computing

Continuously present interfaceNo current model of continuously present interfaces,

even people are not continuously presentCreate an interface that doesn’t get annoying

Determine what information should require my attention and what should be display peripherally

Comfort and style If constantly worn or carried must be stylish and

comfortable

Continuously present interfaceNo current model of continuously present interfaces,

even people are not continuously presentCreate an interface that doesn’t get annoying

Determine what information should require my attention and what should be display peripherally

Comfort and style If constantly worn or carried must be stylish and

comfortable

Other Challenges for UbicompOther Challenges for Ubicomp

Hard to evaluate Ubicomp SystemsLittle publish on ubicomp evaluationSystems often required to be fully connected leading

to systems that are hard to buildLack of development toolkits make system creation

difficultSystems often need to be integrated into peoples

lives which using big clunky prototypes does not lead itself well too

Task/Goal centric approaches don’t work in ubicomp

Hard to evaluate Ubicomp SystemsLittle publish on ubicomp evaluationSystems often required to be fully connected leading

to systems that are hard to buildLack of development toolkits make system creation

difficultSystems often need to be integrated into peoples

lives which using big clunky prototypes does not lead itself well too

Task/Goal centric approaches don’t work in ubicomp

Social issues in UbicompSocial issues in Ubicomp

PrivacyPotential for misuseHow should users be informed of computer

monitoring (Don’t be invisible??) Is it ok to give up this privacy in some settings,

healthcareDo we give the hackers too much power

SecurityConstant monitoring take away personal

responsibilityWhat happens if I lose my data

PrivacyPotential for misuseHow should users be informed of computer

monitoring (Don’t be invisible??) Is it ok to give up this privacy in some settings,

healthcareDo we give the hackers too much power

SecurityConstant monitoring take away personal

responsibilityWhat happens if I lose my data

Killer App in UbicompKiller App in Ubicomp

Article talked about Ubicomp as killer existence rather than just killer appThat said killer apps like spreadsheets and word

processing drove desktop proliferationDoes Ubicomp need a killer app to spur is adoption?Where would such a killer app come from?

Business setting or home users

Article talked about Ubicomp as killer existence rather than just killer appThat said killer apps like spreadsheets and word

processing drove desktop proliferationDoes Ubicomp need a killer app to spur is adoption?Where would such a killer app come from?

Business setting or home users