Ubiquitous Ubiquitous CommunicationsCommunications

Henning Schulzrinne(with Knarig Arabshian, Stefan Berger, Stelios Sidiroglou, Kundan Singh, Xiaotao Wu, Weibin

Zhao and the RPIDS authors)Columbia University IRT Lab

Univ. of Pennsylvania – February 2004

OverviewOverview

What is ubiquitous computing? Core ubiquitous communications

functionality Brief introduction to SIP Ubiquitous computing in SIP and

SLP On-going work at Columbia

What is ubiquitous What is ubiquitous computing?computing? “Ubiquitous computing has as its goal the

enhancing computer use by making many computers available throughout the physical environment, but making them effectively invisible to the user.” (Weiser, 1993)

“Ubiquitous computing is not virtual reality, it is not a Personal Digital Assistant (PDA) such as Apple's Newton, it is not a personal or intimate computer with agents doing your bidding. Unlike virtual reality, ubiquitous computing endeavers to integrate information displays into the everyday physical world. It considers the nuances of the real world to be wonderful, and aims only to augment them.” (Weiser, 1993)

Ubiquitous computing Ubiquitous computing aspectsaspects Also related to pervasive computing Mobility, but not just cell phones Computation and communications Integration of devices

“borrow” capabilities found in the environment composition into logical devices

seamless mobility session mobility adaptation to local capabilities environment senses instead of explicit user

interaction from small dumb devices to PCs

light switches and smart wallpaper

Components of ubiquitous Components of ubiquitous communicationscommunications

Service discovery discover devices

Service mobility configuration information moves to new devices

Event notification for context awareness

Context-awareness location, user actions, location properties, …

Example “ubicomp” Example “ubicomp” projectsprojects Ambient Devices EU IST Disappearing Computer Project Aura, CMU user

attention UNC “office of real soon now” augmented surfaces [Reki99] Microsoft Easy Living Oxygen, MIT Portolano, Univ. of

Washington Endeavour, Berkeley CoolTown, HP Labs

Ubiquitous computing using Ubiquitous computing using SIP – what’s different?SIP – what’s different? Traditionally, focus on closed environments

(lab, single company, home, …) Often, proprietary protocols self-

contained environment Here,

operate across Internet ( no Corba…) trusted, semi-trusted and untrusted participants use standard protocol mechanisms where

possible make minimal assumptions on homogeneity respect user privacy

Context-aware Context-aware communicationscommunications Traditional emphasis: communicate anywhere, anytime, any

media largely possible today New challenge: tailor reachability Context-aware communications

modify when, how, where to be reached machine: context-dependent call routing human: convey as part of call for human usage

context-aware services leveraging local resources awareness of other users

sources of location information voluntary and automatic

location-based services privacy concerns applies to other personal information activity, reachability, capabilities, bio sensor data, …

emergency services as a location-based service

ContextContext context = “the interrelated conditions

in which something exists or occurs” anything known about the participants

in the (potential) communication relationship

both at caller and callee

time CPL

capabilities caller preferences

location location-based call routinglocation events

activity/availability presence

sensor data (mood, bio)

not yet, but similar in many aspects to location data

What is SIP?What is SIP? Session Initiation Protocol protocol

that establishes, manages (multimedia) sessions also used for IM, presence & event

notification uses SDP to describe multimedia sessions

Developed at Columbia U. (with others) Standardized by IETF, 3GPP (for 3G

wireless), PacketCable About 60 companies produce SIP

products Microsoft’s Windows Messenger (4.7)

includes SIP

Basic SIP message flowBasic SIP message flow

SIP trapezoidSIP trapezoid

SIP trapezoid

outbound proxy

a@foo.com: 128.59.16.1

registrar

SIP event notificationSIP event notification Named events Typically, used for events within conferences (“Alice

joined”) and call legs (e.g., call transfer) Supports arbitrary notification bodies, typically XML

SUBSCRIBE sip:alice@vmail.example.com SIP/2.0To: <sip:alice@example.com>From: <sip:alice@example.com>;tag=78923Call-Id: 1349882@alice-phone.example.comContact: <sip:alice@alice-phone.example.com>

NOTIFY sip:alice@alice-phone.example.com SIP/2.0…Event: message-summarySubscription-State: active

Messages-Waiting: yesMessage-Account: sip:alice@vmail.example.comVoice-Message: 2/8 (0/2)

SIP event architectureSIP event architecture Does not try to route notifications (“application

layer multicast”) as in SIENA Filtering at PA under discussion (for low-bandwidth

devices) rate content

But most ubicomp notification groups are probably small

and message volume not likely to provide much bandwidth saving via network-based filtering

Greatly simplifies trust model: no intermediaries that need to inspect content

can encrypt via S/MIME However, can build redistribution “exploders” and

list subscriptions (“subscribe to engineering@hp.com”)

SIP presence architectureSIP presence architecture

PA

a@foo.com: 128.59.16.1

watcher

PUAs

Alice Bob

PUBLISH

REGISTERSUBSCRIBE

NOTIFY

<?xml version="1.0" encoding="UTF-8"?><p:presence xmlns:p="urn:…" entity="pres:alice@example.com"><p:tuple id="sg89ae"> <p:status> <p:basic>open</p:basic> </p:status> <p:contact>tel:09012345678</p:contact></p:tuple></p:presence>

Session mobilitySession mobility Walk into office,

switch from cell phone to desk phone

call transfer problem SIP REFER

related problem: split session across end devices

e.g., wall display + desk phone + PC for collaborative application

assume devices (or stand-ins) are SIP-enabled

third-party call control

Session mobility via 3PCCSession mobility via 3PCC

INVITE speakerphonem=audioc=pc42

INVITE displaym=videoc=pc42

192.0.2.1

192.0.2.7

INVITE pc42m=videoc=192.0.2.7m=audioc=192.0.2.1

pc42

How to find services?How to find services? Two complementary developments:

smaller devices carried on user instead of stationary devices devices that can be time-shared

large plasma displays projector hi-res cameras echo-canceling speaker systems wide-area network access

Need to discover services in local environment SLP (Service Location Protocol) allows querying for services

“find all color displays with at least XGA resolution” slp://example.com/SrvRqst?public?type=printer

SLP in multicast mode SLP in DA mode

Need to discover services before getting to environment “is there a camera in the meeting room?” SLP extension: find remote DA via DNS SRV

Service Location Protocol Service Location Protocol (SLP)(SLP)

Version 2 standardized June 1999

UA

DA

SA

SA

SrvReg

SrvRply

SrvRqst

SrvRqst SrvReg

DAAdvert

SLP attribute exampleSLP attribute exampleURL service:printer:lpr://igore.wco.ftp.com/draft

scope-list Development

Language tag

en

Attributes (Name=Igore),(Description=For developers only), (Protocol=LPR),(location-description=12th floor), (Operator=James Dornan \3cdornan@monster\3e), (media-size=na-letter),(resolution=res-600),x-OK

Other service location Other service location mechanismmechanism DNS SRV/NAPTR DNS TXT records (Apple Rendezvous) DNS-SD UPnP uses SSDP:

multicast HTTP over UDP

M-SEARCH * HTTP/1.1S: uuid:ijklmnop-7dec-11d0-a765-00a0c91e6bf6Host: 239.255.255.250:reservedSSDPportMan: "ssdp:discover“ST: ge:fridgeMX: 3

HTTP/1.1 200 OKS: uuid:ijklmnop-7dec-11d0-a765-00a0c91e6bf6Ext: Cache-Control: no-cache="Ext", max-age = 5000ST: ge:fridgeUSN: uuid:abcdefgh-7dec-11d0-a765-00a0c91e6bf6AL: <blender:ixl><http://foo/bar>

Service mobilityService mobility Allow access to service parameters anywhere –

“payphone problem” address book incoming call rules source name (SIP From)

Existing solutions: SIM card cumbersome to change synchronization (e.g., Palm) not suitable for borrowed

devices Server-based services easier with SIP (service-routing),

if carrier allows Emerging solutions for SIP systems:

Small user token (smart card, RFID, i-button) identifying user

Temporarily download configuration from home server

Context-based Context-based communication servicescommunication services Observable state and actions State:

location of users user activities

Derive state from sensors (time, location, environment, user

interaction) data (calendars, address books) network inputs (messages)

Actions incoming and outgoing calls incoming and outgoing IMs, SMS, email, …

Initially, focusing on location at key context

Location-based servicesLocation-based services Finding services based on location

physical services (stores, restaurants, ATMs, …) electronic services (media I/O, printer, display, …) not covered here

Using location to improve (network) services communication

incoming communications changes based on where I am configuration

devices in room adapt to their current users awareness

others are (selectively) made aware of my location security

proximity grants temporary access Privacy rules for access to context data

Location-based servicesLocation-based services Presence-based approach:

UA publishes location to presence agent (PA) becomes part of general user context other users (human and machines) subscribe

to context call handling and direction location-based anycast (“anybody in the room”) location-based service directory

Languages for location-based services building on experience with our XML-based

service creation languages CPL for user-location services LESS for end system services

Location-based SIP Location-based SIP servicesservices Services:

Location-aware call routing “do not forward call if time at callee location is [11 pm, 8

am]” “only forward time-for-lunch if destination is on campus” “contact nearest emergency call center” “do not ring phone if I’m in a theater” “send delivery@pizza.com to nearest branch”

Location-based events subscribe to locations, not people “Alice has entered the meeting room” subscriber may be device in room our lab stereo

changes CDs for each person that enters the room Person + location events

We’re implementing SIP, caller-preferences and CPL extensions for these services

LocationsLocations Geographic location

latitude, longitude, altitude, velocity, heading Civil location (≠ postal location!)

time zone, street address, city some countries are a bit difficult…

Categorical office, library, theater, hospital, …

Behavioral “public location, don't expect privacy” “silence is encouraged, don't ring the phone”

Determining locationsDetermining locations SIP entities are often far away from physical user or his

current network (intentionally) For many devices, can’t afford hardware to determine location

different precision requirements: “in Fayette County” (within driving distance of service or person) “on campus” “in room 815” “in corner, talking to Bob”

GPS doesn’t work indoors, but Assisted GPS (A-GPS) may Use location beacons: BlueTooth, 802.11

may not offer network connectivity see our 7DS project: offer local content + location

Physically close by network entities: DHCP (same broadcast domain) PPP (tail circuit)

Not always true with VPNs, but end system knows that it’s using a VPN

Determining locationDetermining location Two types of sensors:

end system determines location “handset-based” GPS, 802.11 triangulation

network conveys location to end system or other component

MAC backtracking AP-based 802.11 triangulation swipe cards, iButtons, active badges

Two modes: explicit user action: swipe card, touch iButton involuntary: network-based tracking

GPS may not be practical (cost, power, topology) Add location beacons

extrapolate based on distance moved odometer, pedometer, time-since-sighting

idea: meet other mobile location beacons estimate location based on third-party information

Determining locationsDetermining locations For many devices,

can’t afford hardware to determine location Implementing

BlueTooth-based location sensor networks

CU 7DS project: offer local content + location

Developing programmable active badges with IR and RF capabilities

DHCP for locationsDHCP for locations modified dhcpd (ISC) to generate location information use MAC address backtracing to get location information

DHCPserver

458/17 Rm. 815458/18 Rm. 816

DHCP answer:sta=DC loc=Rm815lat=38.89868 long=77.03723

8:0:20:ab:d5:d

CDP + SNMP8:0:20:ab:d5:d 458/17

Location-based services & Location-based services & SIPSIP We’re using SIP (and SIMPLE) as generic

protocols for effecting change (“actuators”)

send MESSAGE to devices distributing event information (“sensors”)

Advantages: people and rooms identified by URIs

sip:hgs@cs.columbia.edu sip:cepsr815@cs.columbia.edu

cross-domain, with extensive security mechanisms domains don’t need to trust each other

scalable to global system many other systems are mostly local

Architectures for (geo) Architectures for (geo) information accessinformation access Claim: all using protocols fall into one of these categories Presence or event notification

“circuit-switched” model subscription: binary decision

Messaging email, SMS basically, event notification without (explicit)

subscription but often out-of-band subscription (mailing list)

Request-response RPC, HTTP; also DNS, LDAP typically, already has session-level access control (if

any at all) Presence is superset of other two

SIP URIs for locationsSIP URIs for locations Identify confined

locations by a SIP URI, e.g., sip:rm815@cs.columbia.edu

Register all users or devices in room

Allows geographic anycast: reach any party in the room

a@foo.com: 128.59.16.1

Room 815

sip:rm815

location beacon

Contact: alice

Contact: bob

RPID: rich presence dataRPID: rich presence data Basic IETF presence (CPIM) only gives you

contact information (SIP, tel URI) priority “open” or “closed”

Want to use presence to guide communications

PA

watcher

PUA watcher

watcher

PUBLISH

NOTIFY

everything

"vague"

CPL

INVITE

Aside: SIP caller Aside: SIP caller preferencespreferences

SIP core philosophy: many devices, one identifier Address people, not plastic

Aside: SIP caller Aside: SIP caller preferencespreferences But caller sometimes has preferences among devices SIP caller guides call routing:

“I hate voicemail!” “I hate people!” “I prefer voicemail” Multilingual lines

“Caller proposes, callee disposes”

a@foo.com: 128.59.16.1

sip:isabel@a.com;languages="es"sip:isabel@a.com;languages="en";q=0.2

sip:bob@a.com;languages="en"

INVITE sip:sales@a.comAccept-Contact: *;languages="en"

INVITE

REGISTER

RPID: Rich presence data RPID: Rich presence data Integrates caller preferences information into

presence announcements <activity>: on-the-phone, away,

appointment, holiday, meal, meeting, steering, in-transit, travel, vacation, busy, permanent-absence

<placetype>: home, office, public <privacy>: public, private, quiet <sphere>: facet of life (home, work, …) <idle>: activity for device <relationship>: family, associate, assistant,

supervisor <class>: grouping

Event filteringEvent filtering

Events are core attribute of ubiquitous computing systems tell devices about people actions tell people about device presence e.g., “Alice has entered Room 815”

devices that know Alice’s preferences subscribe to Alice

locations may also have presence e.g., for occupancy sensors, switches

Location filtering languageLocation filtering language SIP presence information will be updated using REGISTER and

UPDATE Need to constrain

who is allowed to see what detail presentity privacy who wants to see what detail

how often what granularity of change

Proposal to allow SUBSCRIBE to include frequency limitation Working on CPL-like language invoked (logically) at publication

time classes of users, e.g., based on entry in my address book classes get mapped to restriction

“12 bits of long/lat resolution, 6 bits of altitude resolution, 0 bits of velocity”

“time zone only”, “category only” watchers can then add filters that restrict the delivery:

location difference > threshold entering or leaving certain area entering or leaving category or behavioral type

Presence modelPresence model

subscriptionpolicy

event generatorpolicy

subscriberfilter

rate limiter

change to previousnotification?

for eachwatcher

subscriber (watcher)

SUBSCRIBE

NOTIFY

Policy rulesPolicy rules There is no sharp geospatial boundary Presence contains other sensitive data

(activity, icons, …) and others may be added

Example: future extensions to personal medical data “only my cardiologist may see heart rate,

but notify everybody in building if heart rate = 0”

Thus, generic policies are necessary

GEOPRIV and SIMPLE GEOPRIV and SIMPLE architecturesarchitectures

targetlocationserver

locationrecipient

rulemaker

presentity

caller

presenceagent

watcher

callee

GEOPRIV

SIPpresence

SIPcall

PUBLISHNOTIFY

SUBSCRIBE

INVITE

publicationinterface

notificationinterface

ruleinterface

INVITE

Columbia SIP servers Columbia SIP servers (CINEMA)(CINEMA)

InternalTelephoneExtn: 7040

SIP/PSTN Gateway

Department PBX

Web based configuration

Web server

Telephoneswitch

SQLdatabase

sipd:Proxy, redirect, registrar server

Extn: 7134

xiaotaow@cs NetMeeting

H.323

rtspd: media server

sipum: Unified messaging

Quicktime

RTSP clients

RTSP

Extn: 7136

713x

Single machine

SNMP(Network Management)

sipconf: Conference server

siph323: SIP-H.323 translator

Local/long distance1-212-5551212

Location-based services in Location-based services in CINEMACINEMA Initial proof-of-concept implementation Integrate devices:

lava lamp via X10 controller set personalized light mood setting

Pingtel phone add outgoing line to phone and register user

painful: needs to be done via HTTP POST request stereo change to audio CD track based on

user Sense user presence and identity:

passive infrared (PIR) occupancy sensor magnetic swipe card ibutton BlueTooth equipped PDA IR+RF badge (in progress) RFID (future) biometrics (future)

PA

devicecontroller

SUBSCRIBEto each room

SUBSCRIBE to configurationfor users currently in rooms

1. discover room URI2. REGISTER as contact for room URI

tftp

HTTP

Example: user-adaptive Example: user-adaptive device configurationdevice configuration

SLP

“all devices that are in the building”RFC 3082?

802.11 signal strength

location

REGISTERTo: 815cepsrContact: alice@cs

SIP

room 815

Emergency calling as an Emergency calling as an LBSLBS Existing emergency call systems (“911”) will no longer

work in IP-based networks current 911 system uses outmoded operator trunk

technology Emergency calling =

call identification sip:sos@domain or tel:112 destination identification

is this really an emergency call center? call routing to nearest emergency call center (ECC)

Call routing is hardest must work internationally end system + network-based location determination

Once solved: roadside emergency (AAA, ADAC, …) pizza emergency (nearest PizzaHut) but different privacy trade-offs voluntary disclosure

Location-based call routing – Location-based call routing – UA knows its locationUA knows its location

GPS

48° 49' N 2° 29' E

INVITE sips:sos@

DHCP

outboundproxy server

48° 49' N 2° 29' E Paris fire department

Location-based call routing – Location-based call routing – network knows locationnetwork knows location

IP

48° 49' N 2° 29' E

TOA

include locationinfo in 302

INVITE sips:sos@ INVITE sips:sos@paris.gendarme.fr

map location to (SIP) domain

outbound proxy

Service creationService creation

programmer, carrier

end user

network servers

SIP servlets, sip-cgi

CPL

end system VoiceXML VoiceXML (voice),LESS

Tailor a shared infrastructure to individual users traditionally, only vendors (and sometimes carriers) learn from web models

Call Processing Language Call Processing Language (CPL)(CPL) XML-based “language” for processing requests intentionally restricted to branching and

subroutines no variables (may change), no loops thus, easily represented graphically

and most bugs can be detected statically termination assured

mostly used for SIP, but protocol-independent integrates notion of calendaring (time ranges) structured tree describing actions performed on

call setup event top-level events: incoming and outgoing

CPLCPL Location set stored as implicit global variable

operations can add, filter and delete entries Switches:

address language time, using CALSCH notation (e.g., exported from Outlook) priority

Proxy node proxies request and then branches on response (busy, redirection, noanswer, ...)

Reject and redirect perform corresponding protocol actions

Supports abstract logging and email operation

CPL exampleCPL example

String-switchfield: from

match:*@example.com

otherwise

proxytimeout: 10s

locationurl: sip:jones@

example.comvoicemail.

merge: clear

locationurl: sip:jones@

example.com

redirect

Call

busy

timeout

failure

CPL exampleCPL example<?xml version="1.0" ?><!DOCTYPE call SYSTEM "cpl.dtd">

<cpl> <incoming> <lookup source="http://www.example.com/cgi-bin/locate.cgi?

user=jones" timeout="8"> <success> <proxy /> </success> <failure> <mail url="mailto:jones@example.com&Subject=lookup%20failed" /> </failure> </lookup> </incoming></cpl>

CPL example: anonymous CPL example: anonymous call screeningcall screening<cpl>

<incoming><address-switch field="origin"

subfield="user"><address is="anonymous">

<reject status="reject"reason="I don't accept anonymous

calls" /></address>

</address-switch></incoming>

</cpl>

Service creation for presence Service creation for presence services (work-in-progress)services (work-in-progress) Accept or deny subscriptions Shape presence notifications

different level of detail for family, friends and colleagues

particularly important for geo data Subscriber can filter detail

primarily, wireless bandwidth constraints rate limit notifications XPath?

Mostly, condition/reaction CPL can be extended to most of these functions

Pushing context-sensitive Pushing context-sensitive data to usersdata to users User with mobile device should get location

information when entering city, campus or building

flight and gate information maps and directions local weather forecast special advisories (“choose security checkpoint 2”)

Often does not require knowing user but interface with (e.g.) calendar

Example Columbia implementation: OBEX data exchange over BlueTooth PDA pushes current appointment or event name base station delivers directions and map

ConclusionConclusion SIP + auxiliary protocols supports many of the core

requirements for ubiquitous computing and communications:

mobility modalities: terminal, user, session, service service negotiation for devices with different capabilities automatic configuration and discovery

with SLP or similar event notification and triggered actions automatic actions: event filtering, CPL, LESS (for end

system services) SIP offers a loosely-coupled approach (cf. Jini or object

models) Also need data push functionality Avoid tendency to assume SIP users are human – want

to interconnect different components and devices SIP device configuration needs automation, rather than

screen-scraping