kHealth: Proactive Personalized Actionable Information forBetter Healthcare

Put Knoesis Banner

PDA@IoT, in conjunction with VLDB, September, 2014

Amit Sheth, Pramod Ananthram, T.K. PrasadThe Ohio Center of Excellence in Knowledge-enabled Computing (Kno.e.sis)

Wright State, USA

2

A Historical Perspective on Collecting Health Observations

Diseases treated onlyby external observations

First peek beyond justexternal observations

Information overload!

Doctors relied only on external observations

Stethoscope was the first instrument to go beyond just external

observations

Though the stethoscope has survived, it is only one among many observations

in modern medicine

http://en.wikipedia.org/wiki/Timeline_of_medicine_and_medical_technology

2600 BC ~1815 Today

Imhotep

Laennec’s stethoscope

Image Credit: British Museum

3

“The next wave of dramatic Internet growth will come through the confluence of people, process, data, and things — the Internet of Everything (IoE).”

- CISCO IBSG, 2013

http://www.cisco.com/web/about/ac79/docs/innov/IoE_Economy.pdf

Beyond the IoE based infrastructure, it is the possibility of developing applications that spansPhysical, Cyber and the Social Worlds that is very exciting.

What has changed now?

4Petabytes of Physical(sensory)-Cyber-Social Data everyday! More on PCS Computing: http://wiki.knoesis.org/index.php/PCS

‘OF human’ : Relevant Real-time Data Streams for Human Experience

6http://www.technologyreview.com/featuredstory/426968/the-patient-of-the-future/

MIT Technology Review, 2012

The Patient of the Future

7

Detection of events, such as wheezing sound, indoor temperature, humidity,

dust, and CO level

Weather Application

Asthma Healthcare Application

Close the window at home during day to avoid CO in

gush, to avoid asthma attacks at night

‘FOR human’ : Improving Human Experience (Smart Health)

Population Level

Personal

Public Health

Action in the Physical World

Luminosity

CO levelCO in gush during day time

8

Electricity usage over a day, device at work, power consumption, cost/kWh,

heat index, relative humidity, and public events from social stream

Weather Application

Power Monitoring Application

‘FOR human’ : Improving Human Experience (Smart Energy)

Population Level Observations

Personal Level Observations

Action in the Physical World

Washing and drying has resulted in significant cost

since it was done during peak load period. Consider

changing this time to night.

kHealthKnowledge-enabled Healthcare

Four current applications: To reduce preventable readmissions of patients with

ADHF and GI; Asthma in children; patients with Dementia

9

Brief Introduction Video

11

Through physical monitoring and analysis, our cellphones could act as an early warning system to detect serious health conditions, and provide actionable information

canary in a coal mine

Empowering Individuals (who are not Larry Smarr!) for their own health

kHealth: knowledge-enabled healthcare

What?

• kHealth is a knowledge-based approach/application for patient-centric health-care that exploits:(a) Web based tools and social media, (b) Mobile phone technology and wireless sensors, (c) For synthesizing personalized actions from heterogeneous health data

(i) For disease prevention and treatment(ii) For health, fitness and well-being

12

Weight Scale

Heart Rate Monitor

Blood PressureMonitor

13

Sensors

Android Device (w/ kHealth App)

Readmissions cost $17B/year: $50K/readmission; Total kHealth kit cost: <

$500

kHealth Kit for the application for reducing ADHF readmission

ADHF – Acute Decompensated Heart Failure

Sensordrone (Carbon monoxide,

temperature, humidity) Node Sensor

(exhaled Nitric Oxide)

15

Sensors

Android Device (w/ kHealth App)

Total cost: ~ $500

kHealth Kit for the application for Asthma management

*Along with two sensors in the kit, the application uses a variety of population level signals from the web:

Pollen level Air Quality Temperature & Humidity

Why?

• “Unintelligible” health data deluge due to – Continuous monitoring of patients using passive and

active sensors– Continuous monitoring of environment using

sensors– Public health reports– Population level information– Social media conversations– Personal Electronic Medical Records (EMRs)– Wide use of affordable mobile/wireless technologies

19

Why?

• Empowering patients to improve health by– Abstracting and integrating low-level sensor data

to more meaningful health signals – Recommending personalized actions

• Ubiquitous, timely and effective health management and telemedicine– Involve patient and health-care team without

causing “interaction fatigue”

20

kHealth: Health Signal Processing Architecture

Personal level Signals

Public level Signals

Population level Signals

Domain Knowledge

Risk Model

Events from Social Streams

Take Medication before going to work

Avoid going out in the evening due to high pollen levels

Contact doctor

AnalysisPersonalized Actionable

Information

Data Acquisition & aggregation

21

How?

• Data collection from various sources– Active and passive sensing devices– Social media crawling– EMR

• Syntactic and semantic integration – Qualitative/imprecise citizen observations– Quantitative/precise sensor observations

• Provide complementary and collaborative information• Using Semantic Web technologies, e.g., SemSOS

22

How?

• Semantic Perception: Reasoning for decision making and action generation– Perception cycle– Personalized action recommendation using

• Patient health score (linear scale, RYG-abstraction) • Patient vulnerability score (personalization)

– Qualify vs quantify

• Domain (e.g. disease) specific knowledge

23

24

Asthma Domain Knowledge

Domain Knowledge

ICS= inhaled corticosteroid, LABA = inhaled long-acting beta2-agonist, SABA= inhaled short-acting beta2-agonist ; *consider referral to specialist

Asthma Control and Actionable Information

25

Patient Health Score (diagnostic)

Risk assessment model

Semantic Perception

Personal level Signals

Public level Signals

Domain Knowledge

Population level Signals

GREEN -- Well Controlled YELLOW – Not well controlledRed -- poor controlled

How controlled is my asthma?

26

Patient Vulnerability Score (prognostic)

Risk assessment model

Semantic Perception

Personal level Signals

Public level Signals

Domain Knowledge

Population level Signals

Patient health Score

How vulnerable* is my control level today?

*considering changing environmental conditions and current control level

31

Population Level

Personal

Wheeze – YesDo you have tightness of chest? –Yes

Observations Physical-Cyber-Social System Health Signal Extraction Health Signal Understanding

<Wheezing=Yes, time, location>

<ChectTightness=Yes, time, location>

<PollenLevel=Medium, time, location>

<Pollution=Yes, time, location>

<Activity=High, time, location>

Wheezing

ChectTightness

PollenLevel

Pollution

Activity

Wheezing

ChectTightness

PollenLevel

Pollution

Activity

RiskCategory

<PollenLevel, ChectTightness, Pollution,Activity, Wheezing, RiskCategory><2, 1, 1,3, 1, RiskCategory><2, 1, 1,3, 1, RiskCategory><2, 1, 1,3, 1, RiskCategory><2, 1, 1,3, 1, RiskCategory>

.

.

.

Expert Knowledge

Background Knowledge

tweet reporting pollution level and asthma attacks

Acceleration readings fromon-phone sensors

Sensor and personal observations

Signals from personal, personal spaces, and community spaces

Risk Category assigned by doctors

Qualify

Quantify

Enrich

Outdoor pollen and pollution

Public Health

Health Signal Extraction to Understanding

Well Controlled - continueNot Well Controlled – contact nursePoor Controlled – contact doctor

36

RDF OWL

How are machines supposed to integrate and interpret sensor data?

Semantic Sensor Networks (SSN)

39

W3C Semantic Sensor Network Ontology

Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-Castro, R., Graybeal, J., Herzog, A., Janowicz, K., Neuhaus, H., Nikolov, A., and Page, K.: Semantic Sensor Network XG Final Report, W3C Incubator Group Report (2011).

41

What if we could automate this sense making ability?

… and do it efficiently and at scale

SSNOntology

2 Interpreted data(deductive)[in OWL] e.g., threshold

1 Annotated Data[in RDF]e.g., label

0 Raw Data[in TEXT]e.g., number

Levels of Abstraction

3 Interpreted data (abductive)[in OWL]e.g., diagnosis

Intellego

“150”

Systolic blood pressure of 150 mmHg

ElevatedBlood

Pressure

Hyperthyroidism

less

use

ful …

…

mor

e us

eful

……

42

43

Making sense of sensor data with

44

People are good at making sense of sensory input

What can we learn from cognitive models of perception?• The key ingredient is prior knowledge

Semantic Perception : Perception Cycle

Semantic perception in kHealth involves:• Abductive reasoning to derive candidate

explanations for sensor data, and• Deductive reasoning to disambiguate among

multiple explanations with patient inputs and additional targeted sensor observations.

Intellego

45

46* based on Neisser’s cognitive model of perception

ObserveProperty

PerceiveFeature

Explanation

Discrimination

1

2

Perception Cycle*

Translating low-level signals into high-level knowledge

Focusing attention on those aspects of the environment that provide useful information

Prior Knowledge

47

To enable machine perception,

Semantic Web technology is used to integrate sensor data with prior knowledge on the Web

48

Prior knowledge on the Web

W3C Semantic Sensor Network (SSN) Ontology Bi-partite Graph

49

Prior knowledge on the Web

W3C Semantic Sensor Network (SSN) Ontology Bi-partite Graph

50

ObserveProperty

PerceiveFeature

Explanation1

Translating low-level signals into high-level knowledge

Explanation

Explanation is the act of choosing the objects or events that best account for a set of observations; often referred to as hypothesis building

51

Discrimination is the act of finding those properties that, if observed, would help distinguish between multiple explanatory features

ObserveProperty

PerceiveFeature

Explanation

Discrimination2

Focusing attention on those aspects of the environment that provide useful information

Discrimination

52

Discrimination

Discriminating Property: is neither expected nor not-applicable

DiscriminatingProperty ≡ ¬ExpectedProperty ¬NotApplicableProperty⊓

elevated blood pressure

clammy skin

palpitations

Hypertension

Hyperthyroidism

Pulmonary Edema

Discriminating Property Explanatory Feature

Semantic Perception : Abstraction

• Mapping low-level sensor values to coarse-grain abstract values– E.g., Blood pressure: 150/100 => High bp

• Extracting signatures for high-level human comprehensible features from low-level sensor data stream.– E.g., Parkinson disease : unsteady walk, fall,

slurred speech, etc.

53

57

How do we implement machine perception efficiently on aresource-constrained device?

Use of OWL reasoner is resource intensive (especially on resource-constrained devices), in terms of both memory and time

• Runs out of resources with prior knowledge >> 15 nodes• Asymptotic complexity: O(n3)

58

intelligence at the edge

Approach 1: Send all sensor observations to the cloud for processing

Approach 2: downscale semantic processing so that each device is capable of machine perception

Henson et al. 'An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices, ISWC 2012.

59

Efficient execution of machine perception

Use bit vector encodings and their operations to encode prior knowledge and execute semantic reasoning

0101100011010011110010101100011011011010110001101001111001010110001101011000110100111

60

O(n3) < x < O(n4) O(n)

Efficiency Improvement

• Problem size increased from 10’s to 1000’s of nodes• Time reduced from minutes to milliseconds• Complexity growth reduced from polynomial to

linear

Evaluation on a mobile device

61

2 Prior knowledge is the key to perceptionUsing SW technologies, machine perception can be formalized and integrated with prior knowledge on the Web

3 Intelligence at the edgeBy downscaling semantic inference, machine perception can

execute efficiently on resource-constrained devices

Semantic Perception for smarter analytics: 3 ideas to takeaway

1 Translate low-level data to high-level knowledgeMachine perception can be used to convert low-level sensory signals into high-level knowledge useful for decision making

62

thank you, and please visit us at

http://knoesis.org

Smart Data to Big Data; Physical-Cyber-Social Computing