Event processing within the human body - Tutorial

Speaker: Opher Etzion

Event Processing within the Human Body – Tutorial

2

Event processing within the human body may, in time, be the most important use of this technology and the Internet of Things in general

3

In the human body, we sense, make sense of what we sense, make constant decisions and carry them out. Autonomous systems do the same. When the domain is the human body itself it’s getting interesting….

Sensing

Making sense from the sensing

Real-time decision making

Acting

In this tutorial we’ll talk about current and future applications of IoT and autonomous systems within the human body

OUTLINE

Background: events, autonomous systems, Body

Area Networks

Medical oriented applications of event

processing within the human body

Other applications of event processing within the

human body

The future perspective

4

Topic I

TOPIC II

Topic III

Topic IV

OUTLINE


Area Networks




human body


5

Topic I

TOPIC II

Topic III

Topic IV

None of the

authorized drivers

location is near the

car’s location

theft is concluded

Use a built-in car

stopper to slow the

intruder and dispatch

the security company

A person enters a

car and the car

starts moving;

the person does not

look like one of the

authorized drivers

Such applications

become possible

since everything is

connected

6

7

The term “Internet of Things” was coined by Kevin Ashton in 1999.

His observation was that all the data on the Internet has been created by a human.

His vision was: “we need to empower computers with their own means of gathering information, so they can see, hear, and smell the world by themselves”.

8

The world of sensors

1 Acoustic, sound, vibration2 Automotive, transportation3 Chemical4 Electric current, electric potential, magnetic, radio5 Environment, weather, moisture, humidity6 Flow, fluid velocity7 Ionizing radiation, subatomic particles8 Navigation instruments9 Position, angle, displacement, distance, speed, acceleration10 Optical, light, imaging, photon11 Pressure12 Force, density, level13 Thermal, heat, temperature14 Proximity, presence

9

The value of sensors

Kevin Ashton: “track and count everything, and greatly reduce waste, loss, and cost. We could know when things needs replacing, repairing or recalling, and whether they were fresh or past their best”

The value is in the ability to know and react in a timely manner to situations that are detected by sensors

10

Differences between the traditional Internet to the Internet of Things

Topic Traditional Internet Internet of Things

Who creates content? Human Machine

How is the content

consumed?

By request By pushing information

and triggering actions

How content is

combined?

Using explicitly defined

links

Through explicitly

defined operators

What is the value? Answer questions Action and timely

knowledge

What was done so far? Both content creation

(HTML…) and content

consumption (search

engines)

Mainly content creation

Two separate but connected goals: Awareness and Reaction

Awareness Reaction

Event

Detect Derive Decide Do

Detect

Something

that may be

important

The act of bringing into a system’s sphere of understanding knowledge about an event.

The detection is done by sensors, instrumentation and human reports.

Sw

im L

an

e

Trigger Event

Activity

StateChange

13

Intelligent Detection

Determine what is actually been sensed: vision understanding, voice understanding, text understanding.

Derive

The act of becoming aware of events that are not directly detectable by bringing together events with other events, data, patterns and publishing the observation as a derived event.

Raw

eventsRaw

eventsRaw

events

A Person or a computer recognizes the pattern and enters the derived event or just reacts to it directly.

15

Event processing: making sense from what we sense…

Combining data from multi-sensors to get observations, alerts, and actions in real-time gets us to the issue of detecting patterns in event streams

16

Intelligent derivation

Find the causality between events and situations.

Decide

Automated decision by

decision management

system

The act of determining the course of action to do in response to the situation. This includes the background information needed to be collected to make the decision.

Pass through: Sometimes there is no decision. There is only one course of action.

management system that seeks a optimizing quantitative goals.

18

Intelligent Decision

Finding the best decision some times under real-time constraints may require an intelligent process.

DoThe act of performing the course of action that was decided upon.

Notification: Sending a signal of sort to either a person or system. This would include calling a web-service or subscription to alerts.

action.

20

Intelligent Actions

Intelligent actuators

21

Knowledge acquisition for IoT based systems

How do we know how to make sense of all these data?

22

Contemporary

healthcare trends:

Telehealth

23

Contemporary healthcare trends: Personalized Medicine

24

Contemporary healthcare trends: Sensors and actuators

25

Body Area Networks

26

Body Area Networks

Source: Min Chen et al – Body Area Networks: A Survey. Mobile Networks Applications, 16: 171-193, 2011.

27

Body Area Networks

Source: Min Chen et al – Body Area Networks: A Survey. Mobile Networks Applications, 16: 171-193, 2011.

OUTLINE


Area Networks




human body


28

Topic I

TOPIC II

Topic III

Topic IV

29

Soldier in Battle (including: policeman, fire fighter…)

Wearables:

Camera

GPS

Biometric Sensors

Goal:

Warning

Orientation

Assessment of readiness by commander.

30

Wearables:

Camera

Motion sensors

Biometric SensorsGoal:

Self feedback

Trainer feedback

Real-time game management for the coach.

Training and Sport

31

Fitbit Surge:

Wristwatch that contains GPS, tracks steps, monitors heart-rate, connects to community

Training monitoring

32

NICU: Pre-mature babies monitoring

Personalized alerts based on collection of monitors: when nurse should be alerted, when physician should be alerted.

There are many false alerts that are ignored, Missing or ignored alert is sometimes fatal

33

iFeel Labs:

Remove stress by bio-feedback. Using games.

Stress control

34

Goal:

Monitor and diagnose sleep disorders .

Sleep disorders monitoring

Source: N. Oliver & F. Flores-Mangas: Health Gear – Automatic Sleep Apnea Detection and Monitoring with Mobile Phone. Journal of Communication, 2(2), 2007

35

Goal:

Monitor and diagnose sleep disorders .

Sleep disorders monitoring

Source: N. Oliver & F. Flores-Mangas: Health Gear – Automatic Sleep Apnea Detection and Monitoring with Mobile Phone. Journal of Communication, 2(2), 2007

36

Pebble time:

Wrist watch that measures sleep, sleep patterns, finds the best time to wake up.

Sleep control

37

Wearables:

Wristband, chest patch and spirometer –

Monitoring both the person and the environment for allergy factors

Goal:

Prevent, predict, monitor asthma attacks .

Monitoring Asthma Patients

Source: https://news.ncsu.edu/2016/06/wearable-tech-asthma-2016/

38

Sensors and actuators –technology still under development

Goal:

to improve insulin replacement therapy until glycemic control is practically normal, and to ease the burden of therapy for the insulin-dependent.

Implants for diabetes patients

Source: https://www.slideshare.net/energexsystems/pancreas-presentation

39

Implantable cardioverter defibrillators (ICDs) are used to detect dangerously fast heartbeats and give a lifesaving shock to correct the heart’s rhythm. A person with an ICD has the equivalent of a paramedic sitting on his shoulder, always watching and ready to give the heart “the paddles,”

Implants for cardiovascular diseases

Source: http://www.webmd.com/heart-disease/implantable-cardioverter-defibrillator

40

Smart pacemaker

A pacemaker is a small device that's placed in the chest or abdomen to help control abnormal heart rhythms. This device uses electrical pulses to prompt the heart to beat at a normal rate.

Implants for cardiovascular diseases

source: https://www.cambridgeconsultants.com/media/press-releases/setting-pace

41

Monitoring for cancer cells, replacing biopsy tests. Monitoring for impact of treatments and medications for personalized treatment.

In the future: also actuators that serve to treat and eliminate

Implants for cancer treatment

42

Closed loop to monitor and regulate DBS Helping Essential Tremor and Parkinson Patients.

Controlling deep brain stimulation

43

Connecting audio signals to hearing nerves.

Implants that bypass damaged senses

https://www.foundationforsightandsound.org/wordpress/?page_id=636

44

Aiding blind people


45

OrCam MyEye is a tiny camera that uses recognition technology to help the blind and visually impaired identify people, text and things. The portable, intuitive device is comprised of a small camera that can be discreetly affixed to the wearer’s own glasses, plus a lightweight computer processor that’s small enough to be carried in a pocket. OrCam MyEye works by capturing an image of an object and translating it into audio for the wearer. The device is able to read text, identify products, read street signs and even recognize faces.


46

Almost the entire electromagnetic spectrum is invisible to the human eye. Hyper imaging technology and AI will provide super human vision that will help to spot dangers that are hidden from the view

Source: IBM Predictions for 2022.

Super human vision

47

AI based diagnostic of mental disorders based on behavior, speech, writing etc…

Source: IBM Predictions for 2022.

Detection of mental disorders

48

Personalized aides for elderly to maintain independent life

Motion sensor

Door sensor

ChairSensor

Voice Sensor

Alert family member

Alerts example:Door was not locked within 2 minutes after entranceFalling event detectedVocal distress detectedNo motion for certain time period detected

While much technology exists, it is not widely used. It needs to be more personalized, more affordable, and much simpler…

The research required is multi-disciplinary:

Technology oriented, human oriented, economic oriented and particular domain oriented

49

Family planning andcontraceptives based onwristband that measures andsystems that accumulatespersonal data

OUTLINE


Area Networks




human body


50

Topic I

TOPIC II

Topic III

Topic IV

51

Implant in the brain retrieves electric signals.

Brain computer interface

Source: Ed Grabaianowski: How Brain-Computer Interfaces Works: http://computer.howstuffworks.com/brain-computer-interface.htm

http://computer.howstuffworks.com/brain-computer-interface.htm

52

Brain computer interface

Source: Ed Grabaianowski: How Brain-Computer Interfaces Works: http://computer.howstuffworks.com/brain-computer-interface.htm

http://computer.howstuffworks.com/brain-computer-interface.htm

53

Microchipimplanted in the body to identify persons.

Identification implants

Applications for security, open doors, payment (activating credit cards) presence and more.

54

Next generation of glasses that connect to the Internet, identify persons, interface for autonomic assitant

Seeing Virtual

55

Automated personal assistant

Sensors that determine the context are linked to active advisors. They understand your context and even listen to your conversations and give you suggestions of what to say (e.g. through google glass).

2018

56

Chemical plants safety

Monitoring people movement and compliance with safety regulations, leak of chemical materials…

57

Summary: Detect-Derive-Decide-Do in the human body level

Detect –

Wearables + implants

Derive -knowledge + learning + personalization

Decide – AI + decision science

Do –Actuators: autonomic and manual

OUTLINE


Area Networks




human body


58

Topic I

TOPIC II

Topic III

Topic IV

59

The vision: Human Sapiens 2.0 –next phase of the evolution

60

Reprogramming the body to eliminate aging.

Eternal Life?

61

Kurzweil’s time

Stamp for singularity.

Eternal Life?

62

63

Source: S.A. Dash – Mind Uploading

https://www.slideshare.net/prabin210/mind-uploading

https://www.slideshare.net/prabin210/mind-uploading

64

Lung on a chip

65

Arthery on a chip

66

Human bodyWith multiple organson a chip

67

Microsoft Biological Computation: Goal to program the body in order to eliminate cancer

By treating cancer like an information processing system, Microsoft researchers are able to adapt tools typically used to model computational processes to model biological ones.

Ultimately, the company hopes to create molecular computers to program the body to fight cancer cells immediately after detection

Andrew PhillipsSource: Christina Farr, Mind and Machine, September 2016, https://www.fastcompany.com/3063835/microsofts-biological-computing-lab-wants-to-fight-diseases-by-reprogrammin

https://www.fastcompany.com/3063835/microsofts-biological-computing-lab-wants-to-fight-diseases-by-reprogrammin

68

Biological computers destroy cancer cells

69

Hacking the DNA – programming the cells

CELLS ARE BASICALLY tiny computers: They send and receive inputs and output accordingly. If you chug a Frappuccino, your blood sugar spikes, and your pancreatic cells get the message. Output: more insulin.

A cell could be programmed, for example, with a so-called NOT logic gate. This is one of the simplest logic instructions: Do NOT do something whenever you receive the trigger…. Biologist Wilson Wong of Boston University, who led the research, refers to these engineered cells as “genetic circuits.”

Applications: diagnostics, disease fighting, in the future: hacking the aging mechanism.

Source: Sophia Chen, Wired, March 2017

https://www.wired.com/2017/03/biologists-made-logic-gates-dna/

70

Biological molecules are used for storing information and transmitting it to other cells

“For example, suppose we could partner with microbes and plants to record events, natural or otherwise, and convert this information into easily observed signals. That would greatly expand our ability to monitor the environment,” Drew Andy, Stanford University

71

Privacy considerationsThe traditional Internet and social networks are already compromising privacy in the virtual world

Wearable devices and implants expose a lot of information about humans. There are already debates about biometric stores.

72

Security considerations

Murder by the Internet

“With so many devices being Internet connected, it makes murdering people remotely relatively simple, at least from a technical perspective. That’s horrifying,” said IID president and CTO Rod Rasmussen. “Killings can be carried out with a significantly lower chance of getting caught, much less convicted, and if human history shows us anything, if you can find a new way to kill, it will be eventually be used.”

EXAMPLES: Turn off pacemakers, Shutdown car systems while driving, stop IV drip from functioning

73

Dangers and challenges

Confusing a sensor

Changing the rules of the game

Abusing an actuator

74

Confusing a sensor

The same as confusing the human eyes. See things that don’t exist, don’t see things that exist, distort picture…

75

Confusing a sensor

Can be used to sabotage, to commit crimes, frauds and more.

76

Confusing a sensor

Example from another domain: the Twitter hoax

77


The logic is available. The ease of modification can be abused to add/delete/modify rules, change thresholds…

78


Changing data relevant for the system: maps, pictures, person’s data…

79

Abusing actuators

Deviating from course, shutting down, activating in wrong mode…

80

The Solaria effect

||machines|| >> ||men||

81

The Driverless Car Dilemma as an example

to theautonomous computing dilemmas.

82

The Deep Learning Dilemma

Autonomous systems employ deep learning techniques for self-improvement. The designer loses control over the behavior.

83

Who will move to the next phase of evolution?

84

Social Implications

Spreading the progress or keeping it at the hands of small fraction of the

population

85

Social Implications

Question: What are the implications of long life?

86

Social Implications

Question: What are the implications of Homo Sapiens 2.0 with embedded

computers in the brain and body?

87

Processing human related events is one of the enabling and essential

technologies to the future, and mightbe the area where event processing eventually will have the most impact.

Data & Analytics

Event processing within the human body - Tutorial