Mitja LuštrekJožef Stefan Institute

Department of Intelligent Systems

Environment should be◦ Intelligent◦ Require no special skills of the user◦ Require minimal interaction from the user

The technology should disappear Its advantages should remain

Defined by objectives, not methods Interdisciplinary

On the go:◦ Wearable sensors◦ Smart phone applications

At home:◦ Sensors◦ Computer controlled appliances◦ Home automation

Living labs (Philips...)

Pupulation is aging – over 65 in Europe:◦ 17.9 % in 2007◦ 53.5 % in 2060

Not enough young people to care for the old Technology must step in

◦ Assistance with activities of daily living (ADL)◦ Detection of health problems

Equip elderly with radio tags

Sensors determine tag coordinates:◦ Installed in the

appartments◦ Included in tags and

portable device outdoors Detect falls and other

health problems

Portable

device

Body tags

Sensors in the

appartment

Equip elderly with radio tags

Sensors determine tag coordinates:◦ Installed in the

appartments◦ Included in tags and

portable device outdoors Detect falls and other

health problems

Intelligence

Radio tags and sensors to be developed in the project◦ Distance to tag – time needed for signal to travel

from tag to sensor◦ Direction of tag – angle of arrival of the signal

Expected standard deviation of noise:◦ ~5 cm when stationary (Ubisense × 1)◦ ~10 cm when moving (Ubisense × 2)

6 infrared cameras 12 reflective markers on the

body Multiple cameras see a marker

⇒ location can be computed

Standard deviation of noise:◦ ~1 mm

Add more noise to simulate radio hardware

815 recordings:◦ Walking◦ Sitting◦ Lying◦ Falling – 11 types◦ Lying down◦ Sitting down◦ Health problems:

Limping Hemiplegia (stroke) Parkinson’s disease Dizziness Epilepsy

Six

basic

activ

ities

Input: sequence of snapshots of tags (each consisting of coordinates of all tags)

Attributes

Output: posture/activity (walking, lying...)

Class

Manually segment and label recordings Compute attributes for each snapshot Concatenate to create attribute vectors

Z coordinates of tags Absolute, z velocities of tags Absolute, z distances between tags

Attributes – Attributes – anglesangles

All coordinates of tags Velocities of tags

(absolute, direction)

One coordinate systemper snapshot

One coordinate systemper 1-second interval

Two options

Two more options: each coordinate system can use reference z axis

Attributes: reference coordinate system Machine learning algorithms:

◦ SVM◦ Random forest◦ Bagging◦ Adaboost M1 boosting◦ 3-nearest neighbor

Winner:◦ Reference coordinate system + angles◦ SVM

Sitting down, no noise

Falling, Ubisense × 1 noise

Tag placement◦ More tags ⇒ better performance◦ More tags ⇒ worse user acceptance

Noise level◦ We are only estimating noise of the radio

hardware

12 11 10 9 8 7

6 5 4 3 2 1

LR

We can recognize walking Can we recognize abnormal walking?

Gait (way of walking) important to physicians

Used to recognize health problems in clinical setting

Support (foot on the ground), swing (foot off the ground) and step (support + swing) times

Double support time (both feet on the ground) Step length and width Maximal distance of the foot from the ground Ankle, knee and hip angles upon touching the ground Knee angle when the ankle of the leg on the ground is

directly below the hip and knee angle of the opposite leg at that time

Minimal and maximal knee and hip angles, the angle of the torso with respect to the ground, and the range for each

Hip and shoulder sway (the difference between the extreme left and right deviation from the line of walking)

X, y coordinates of ankles L: lowest distance travelled (standing still) H: highest distance travelled (moving)

Normal:◦ Completely normal◦ With a burden

Abnormal:◦ Limping◦ Hemiplegia (stroke)◦ Parkinson’s disease◦ Dizziness

In-depth analysis of activities other than walking

Attributes other than walking signature

Macroscopic movement (about the appartment)