© 2010 Sense 127/04/10

Sensors are everywhere

Steven Mulder steven@sense-os.nl

© 2010 Sense 227/04/10

Mission statement

Modern smartphonesMany sensorsMany networking optionsEasily available

Other sensors/actuatorsEven more sensing optionsCustomized for specific purposes

Networked intelligenceSmart combining of sensor dataSmart sharing of useful informationDon't depend on having 1 perfect sensor

Sense aims to improve human communication and coordination by achieving situational awareness through:

© 2010 Sense 327/04/10

Smartphone and sensor node

Microphone

Camera Accelerometer

Light sensor

Touch sensor

Battery level

Bluetooth

Wi-Fi

GPRS

UMTS

GSM

GPS

Speaker

Vibration

Display

USB Calendar data

Communication records

Human sources

Orientation

Proximity

Sensors

Actuators

Connectivity

Virtual Sensors

Internet sources

Accelerometer

Temperature

LightInfrared (movement)

Bluetooth*

MyriaNed

Serial port

Orientation

CO2

Indoorlocation

© 2010 Sense 427/04/10

Three Steps in Situational Awareness

Status observation and predictionReachability of the user

Location of the user

Activities (in general) of the user

Properties of the environment

Networked intelligenceShare sensor data with other devices

Combine multi-modal sensor signals for richer data

Smart interaction Don't just observe what is happening!

Share information with other people when this is useful

Support and coordinate human

© 2010 Sense 527/04/10

Sense software suite

Local networkMesh network with gossip protocol

Linked to external world via smartphones

Does sensing and actuation

Preprocesses sensed data

Interacts with other platforms (e.g. ASK or Regas)

Remote host, “CommonSense”Cloud intelligence

Remembers, combines, analyzes sensor data

Also interacts with other platforms (e.g. ASK)

Sense delivers a development platform consisting of two parts:

Local applications

CommonSense

Otherplatforms

© 2010 Sense 627/04/10

Advantages

Do more things with more intelligenceno more manual logging: automatically register what is happening

immediately notice abnormal behavior

minimize coordination problems between links

real-time information means better planning possibilities

Ease of use of wireless sensor networksvirtually no infrastructure costs

up and running within hours

reliable against breakdowns of individual devices

Open architectureno proprietary black boxes

works with pre-existing infrastructure

flexibility for scaling/updating

© 2010 Sense 727/04/10

Current situation

Working on pilot projects:

Building management system

Location-based time registration (Regas)

Sharing reachability status via webservice (ASK)

Indoor localization

Looking for more!

© 2010 Sense 827/04/10

Example 1: ReachabilityKey issue in emergency response

Observe and predict if / when a user can be reached.

Automatically pick the right communication channels.

Already in a call, arrange callback?Too noisy to talk, send SMS?

Not near phone,call a friend?

© 2010 Sense 927/04/10

Example 2: LocationObserve and predict where the user is.

Multiple sensors:GPS

Calendar

MyriaNed sensor nodes

GSM cell towers

Known Wi-Fi access points

Other users

Bringing the kids to school, back in 20 minutes.

In traffic, will be 30 minutes late for the meeting.Having a walk in the park, everything is fine.

© 2010 Sense 1027/04/10

Example 3: Activities

Recognize activities of the user and learn which ones are relevant.

Check with user or other devices for feedback.

Attending G20 summit with Barack and AngelaMeasuring a patient's blood pressureHaving a 10 minute rest stop

© 2010 Sense 1127/04/10

Example 4: Networked intelligence

Use the network of sensors that exists around us

Share sensor data with neighboring devices

Combine multi-modal sensor signals

Does anyone know where we are?

No problem! I have GPS.

I think I heard something fall.

I just hit some-thing hard!

I could call the neighbours...

Somethingjust moved.

I heardbreaking glass!

The light was switched on.

Somethingis moving.

© 2010 Sense 1227/04/10

Example 5: Smart interaction

Act upon the knowledge about the user and his environment

Help the user in smart ways

Hello, I am here for ameeting with Mr. Lee That's my guy! I'll

send you his business card.

You just bought toma-toes, pasta, chips...

OK, I will scratch those off the grocery list and notify Mrs. Carrot.

Careful! You are moving away fromthe nearest CPR-person.

© 2010 Sense 1327/04/10