Ubiquitous Health: Wearable Computing Systems that Promote Healthy Living and Transform Health Care

Prof. Bjoern Eskofier, PhD Endowed Professorship of the adidas AG Digital Sports & Health Lab December 1, 2016

Introduction

The Medical Valley of Germany

3

Digital Sports in Erlangen

4

Digital Sports Group

Digital Sports Group

Data Mining Biomechanics Physiology

Wearable Systems Sensors Algorithms

5

Sports Applications

Biomedical Applications

“… to increase human health …”

Research Environment

6

Sports Science

Medical Experts

Industry

Dr. B. Krabbe

Prof. M. Lochmann Prof. J. Klucken

Digital Sports Group

Prof. B. Eskofier Team: 14 PhDs / 1 PDoc

Hi!$

Wearable Computing Systems

Origins – adidas_1 (2008)

8

Eskofier et al.: Embedded Surface Classification in Digital Sports. Pat Rec Let 30(16), 2009

Internet

Human-Machine-Interface (Speech, Display, Vibration,…)

M.D. Athlete Coach

Apps for Live-Feedback, Updates miFitness

miTeam

Web 2.0

miHealth

miCoach Bluetooth

ZigBee

ANT

ANDROID Mobile Sensor Framework ASTRUM miLife

WebService

Feedback, Monitoring and Social Networking Feedback Training

Sensor Integration

Synchronization Communication

Volume'2'280'000'€' ' ''

European'Fund'for'Reg.'Devt.'

Follow;up'project'(2015;2018):'

“Urban'Sports”,'1'558'000'€''

miLife Research Project: 2011

9

Wearable Computing Results

10

Running$Analysis$Schuldh.$et$al.,$2012$

Synchroniza<on$Kugler$et$al.,$2012$

Research$Sensor$Blank$et$al.,$2014$

Golf$PuDng$Jensen$et$al.,$2015$

Swimming$Classifi.$Jensen$et$al.,$2016$

ECG$Classifica<on$Gradl$et$al.,$2012$

Sleep$Monitoring$Gradl$et$al.,$2013$

eGaIT$System$Rampp$et$al.,$2015$

Nykturia$Monit.$Huppert$et$al.,$2015$

Wearable$ECG$Richer$et$al.,$2016$

Cycling$System$Richer$et$al.,$2015$

Skateboard$Classif.$Groh$et$al.,$2016$

Beach$Volleyball$Kautz$et$al.,$2016$

Ski$Jumping$Groh$et$al.,$2016$

Soccer$System$Zhou$et$al.,$2016$

Gradl, S.; Kugler, P.; Lohmüller, C.; Eskofier, B.: Real-time ECG monitoring and arrhythmia detection using Android-based mobile devices. In: Proc. of the Int. Conf. of the IEEE EMBS (EMBC2012). Elgendi, M.; Eskofier, B.; Dokos, S. Abbott, D.: Revisiting QRS Detection Methodologies for Portable, Wearable, Battery-Operated, and Wireless ECG Systems. PLoS ONE 9(1), e84018, 2014.

HRV

QRS detection

ECG signal classifcation

HR features

Hearty – realtime ECG analysis & arrythmia detection

Biosignal Analysis

11

The FitnessSHIRT

12 H Leutheuser, [...], BM Eskofier. Textile Integrated Wearable Technologies for Sports and Medical Applications. Springer, Berlin, Germany, 2016

Smart shoes reach the clinic: Wearable sensor-based instrumented gait analysis in Parkinson’s disease

Movement Disorders

0

10.000

20.000

30.000

40.000

50.000

60.000

2002 2004 2006 2008

21.559 22.293 23.609 24.780

18.677 19.411 20.481

22.546

6.508 6.612 6.541

6.577

Cost of Movement Disorders (Mio Euro/ Year)

weitere ambulant stationär

Federal statistical office, Germany 14

The patient view

‘Just imagine what we could achieve if we start working together – as equals with different but complementary areas of expertise!’

15

Care scenario Sy

mpt

om S

ever

ity

Disease Progression

DiagnosKcs'

Therapy'D TD TD TD TD

Chronic Disease

DiagnosKcs'

Therapy'

Acute Illness

Incomplete Remission

Complete Remission

16

PDXNurse$Pa<ent$

Physician'Expert'

Physician'MDU*'

*Movement Disorder Unit

Sectors'of'Care'

Care scenario

Chronic Disease: Parkinson Syndrome

Telemedicine'

Medical'

InformaKon'Medical''

Technology'

IT'PlaUorm''

CommunicaKon'

Individualised'PaKent'History'

GxP,$Data$safety,$Privacy,$Security,$etc.$

17

$$

„Pa#ent'needs“$Technically$&$Medically$Validated$Technology$

Nurse Patient

Care Scenario – Clin. Application

IT'PlaUorm''

CommunicaKon'

Individualised'PaKent'History'

GxP,$Data$safety,$Privacy,$Security,$etc.$

Technology'

EMG ECG, Respiration, Temperature

Instrumented Gait Analysis

Video based

Diagnostics

Activity

18

Embedded Gait Analysis using Information Technology Specific focus on Parkinson‘s Disease

Funding source

Bavarian Research Foundation

Volume 878 000 €

New funding source

FAU Emerging Fields Project Volume 860 000 €

eGaIT Research Project: 2011

19

1000 PD-Specific Datasets eGaIT shoes IMMU sensors

Movement exercises

Clinical routine assessment

Barth, [...], Eskofier; EMBC 2011 / Klucken, Barth, [...], Eskofier, Winkler; PLoS ONE 8(2), 2013

25.03.2015: Bayerischer Innovationspreis Gesundheitstelematik 2015 für eGAIT

22.10.2014: Erlanger Medizintechnikpreis 2014 (Kategorie Versorgung) für eGAIT

20

Analysis Paradigm Movement recording

Robust Stride Segmentation Barth, [...], Klucken*, Eskofier*; EMBC 2013 & Sensors 2015

Stride Signatures Machine Learning: Waveshape

Klucken, [...], Eskofier, Winkler; PLoS ONE 8(2), 2013

Stride Parameters Signal Analysis: Biomechanics

Rampp, Barth [...], Klucken, Eskofier; TBME 2015

× TO + HS ! MS

21

Signal-processing-driven Stride Parameter Calculation

Rampp, Barth [...], Klucken, Eskofier; TBME 62(4), 2015

IMU DataAccelerometer

Gyroscope

NormalizationCalibration Invert Axes

Stride Segmentation

msDTW

Gait Event DetectionMid Stance (MS)

Heel Strike (HS)

Toe Off (TO)

Spatial Gait ParametersOrientation Estimation (MS to MS)

Gravity Cancellation

Zero Velocity Update

Angle Course

De-Drifted Integration

DistanceEstimation

Sensor Clearance Estimation

(SC)

Sensor-Toe-Distance

Estimation

Stride Length

Angle Heel Strike

Angle Toe Off

Temporal Gait ParametersStride Time

Stance Time

Swing Time

Time HS to HS

Time HS to TO

Time TO to HS Max Toe Clearance

Toe Clearance Estimation

Angle Dependent Correction of SC

Stride Parameters

22

Timed-Up & Go Instrumentation

Angular velocity [°/s]

First Turn Walking

Second Turn Turn-to-Sit Walking

Sit-to-Walk

Time [s]

TUG-Phases in PD patients

n = 265 PD patients, * ANOVA (0.05), post-hoc Bonferroni. Mean time (+/- SEM)

* *

* *

Results of the analysis

Reinfelder, S.; […]; Klucken, J.; Eskofier, B.: Timed Up-and-Go Phase Segmentation in Parkinson's Disease Patients using Unobtrusive Inertial Sensors. EMBC 2015. 24

This$is$great,$but…$

C I II III C 1 2 3 C L M H

Minimum foot clearance

Monocenter IIT 193 PD patients 145 controls

C I II III C 1 2 3 C L M H

Stride length

Gait parameter changes in PD

Schlachetzki, J.; […]; Eskofier, B.; Klucken, J.: Smart shoes reach the clinic: Wearable Sensor-based Instrumented gait analysis in Parkinson’s disease. Lancet Neurol, under review, 2015.

H&Y UPDRS-GAIT UPDRS-III

25

Gait parameter changes in PD

Longitudinal measurement – intra-individual Long term monitoring

Stride length Stance phase Swing phase

UPDRS-III Change at follow-up visit

Schlachetzki, J.; […]; Eskofier, B.; Klucken, J.: Smart shoes reach the clinic: Wearable Sensor-based Instrumented gait analysis in Parkinson’s disease. Lancet Neurol, under review, 2015. 26

This$is$fantas<c!$

Need To Go Ambulatory

Stationary lab systems Mobile sensor systems

Non-natural scenario Limited subject numbers

Home and everyday life Big Data!

27

Espay, A.; [...]; Klucken, J.; Eskofier, B.; [...]; Papapetropoulos, S.: Technology in Parkinson disease: Challenges and Opportunities. Submitted to Movement Disorders 12/2015. On behalf of the MDS Taskforce on Technology. Pasluosta, C.; Gassner, H.; Winkler, J.; Klucken, J.; Eskofier, B.: An Emerging Era in the Management of Parkinson’s disease: Wearable Technologies and the Internet of Things. IEEE J Biomed Health Inform 19(6), 1873-1881, 2015.

8 hours of unsupervised gait of PD patients

Unsupervised Gait Analysis

Single'strides'&'

Individual'raKngs'

Gait'signatures'&'

Gait'parameters'

Daytime [hour]

Reinfelder, Marxreiter, Klucken*, Eskofier*; Unpublished, in preparation for TBME 28

Time Sync

Sensor Data

Patient Rating

Unsupervised Gait Analysis

ON OFF INTERMED. Motor Fluctuations

Gait parameters Stride length (cm)

Freezing Gait Pattern

Daytime [hour]

29

Transforming Healthcare

New reimbursement paradigm: •  At present: reimbursement per prescription & treatment •  In future: reimbursement per objectively measured

treatment success? New chronic disease management concepts: •  Present concept:

•  Future concept:

6 months 6 months

variable, dep. on needs variable

30

Digital Biobank

Biobank of individual signatures from a diversity of movement disorders: • Neurologic: Parkinson, … • Musculoskeletal: OA, ... Signatures consist of: •  Inertial sensor data • Biomechanical data •  Imaging data • Clinical scales

31

EU Data Platform?

Comprehensive Center for Movement Medicine

Physician / Patient Pharma / Industry

Database

Provide Data

Controls Access

Engage Organize

32

EIT Health

33

Our Vision:

EIT Health is a catalyst for change. Our community creates novel

solutions that make healthy lives a reality for all.

Funding by EU:

2 billion / 10 years

EIT Health – Partners

Menno$Kok$Interim$CLC$Director$Belgium/Netherlands$

CLC'UK/Ireland'

CLC'France'CLC'Spain'

CLC'Belgium/Netherlands'

InnoStars'

CLC'Germany'

CLC'Scandinavia'

34

Future Synergies

Fitness'and'sport'Disease'and'early'

detecKon'Chronic'disease'

Morbidity

Mortality'

35

36

Fall 2015

Summer 2014

Digital Sports Group

See$you!$

Thank You!