Towards networks and services supporting the human cognition

Anastasius Gavras, Eurescom GmbH

Definition Augmented cognition refers to all technological means

based on ICT technology and neurotechnology, which are used to alter the power and scope of the cognitive processes and sensory perceptions of human beings

Context is the reality-virtuality continuum

Augmented human cognition - Supporting networks and servicesMilgram & Kishino, 1994

Timelines Short-term: almost mature, conceivable in

the near future 5-10 years Mid-term: trends in R&D, predictions,

extrapolations 10-20 years Long Term: technologies in their infancies

and some reasonable chance of commercial availability beyond the next 20 years

Augmented human cognition - Supporting networks and services

Augmented human cognition - Supporting networks and services

Motivation Future innovations are more likely to happen at the

borderline of ICT and other science disciplines Neuroscience, medicine Restorative

Advances in real 3D, smell, taste, haptics and similar technologies improve the reality feeling Might not be distinguishable from the real

environment anymore Perceived time travel or teleportation Simple augmented reality up to augmented cognition

in a broader sense

Only restoration? 400 m in 45”07 Qualified for 2011

world championship part of South Africa's

silver medal winning relay team 4x400m

first amputee to win an able-bodied world track medal, although he was not selected for the final

Qualified for the 2012 Olympics

Augmented human cognition - Supporting networks and services

Fast on artificial legs: South African Paralympic runner Oscar Pistorius

CyborgsTerm Cyborg coined in 1960 by Manfred Clynes and Nathan Kline in an article about the advantages of self-regulating human-machine systems in outer space

Is everyone with a pacemaker a cyborg?

1958 first pacemaker implant1970 first pacemaker with TTM (transtelephopnic monitoring)

Augmented human cognition - Supporting networks and services

Arne H. W. Larsson (1915 – 2001), the first recipient of an implantable pacemaker

Vision

“Solar” powered 50µm thick Stimulation electrodes

connect to nerve-cells Creates visual perceptions

Augmented human cognition - Supporting networks and services

Diagramme on the functioning of a retinal chip implant by German medical technology company

Retina Implant AG – http://retina-implant.de

Augmented reality

Augmented human cognition - Supporting networks and services

Microvision (www.microvision.com)

Google

Augmented reality

Augmented human cognition - Supporting networks and services

University of Washington

Brain

Augmented human cognition - Supporting networks and services

Intel: Brain implants could control computers by 2020

Brain computer interface to control the Honda Asimo robot

US patent 7,486,986, Feb. 3, 2009

A cerebral interface system has a housing mechanism configured to be at least partially spaced in a cavity formed in the subject’s skull…

A bit uncomfortable

Challenges, risks, socio-economics Privacy, IPR regulations need significant update

Private bystanders and IPR owners in public places who come into the view of AR applications

IPR issues with annotating protected information or superimposing physical billboards

“Very serious copyright issues around places databases” Attributed to Eric Schmidt, Google, Inc.

Tampering can lead to malicious injection by a third party, of tampered data, resulting into false cognitive information

Unauthorized and unsolicited cognitive content Ethical and privacy implications of affect sensors

Augmented human cognition - Supporting networks and services

Challenges, risks, socio-economics Mental and physical health and safety

AR visors – psychological effects observed on fighter pilots “Change blindness” occurs on rapid eye movement or focus

change from the display to the external environment Failure of attentional capture, when the human is totally

fixated on the symbology Invasive BCI bears all risks associated with surgery and

unlikely to receive healthcare regulatory approval except for extreme medical indications

Applications for emergency scenarios subject to regulatory approval, which leads to a significant cost to address compliance issues

Acceptability

Augmented human cognition - Supporting networks and services

Question leading to objectives The network and service infrastructure must support

these visions Short term scenarios related to mass market introduction

Augmented reality Mid term scenarios for future orientation

“Real virtuality” – an undistinguishable virtual reality Long term scenarios only science fiction?

Augmented cognition Networking requirements? Network and service infrastructure? Timeline of potential deployment?

Augmented human cognition - Supporting networks and services

Technology enablers

Mobile computing Computation at the edge Vision based technologies

Recognition, processing, 3D Mobile sensors

Gyro, Accelerometer, Inertial navigation

Brain computer interface (non-invasive)

Machine learning Multilanguage speech

technologies

Displays Stereoscopic experience, retinal implants

Auditory sensors and displays

Olfactory sensors and displays

Biosensors, Lab-on-a-chip, DNA chip

Brain computer interfaces (partially invasive and cerebral)

Nano-communications, molecular electronics

Augmented human cognition - Supporting networks and services

Short term – mid to long term

Some number estimates Today:

Average mobile data volume originating from standard mobile devices: 3.3 MB per month

Average mobile data volume originating from smart phones: 79 MB per month

Tomorrow (2015): Global subscriber penetration expected to grow to 100% (7.2 billion) –

factor 2 compared to today. Penetration of smart phones expected to grow to 35 % - factor 10

compared to today By 2020:

Up to thousand-fold increase in total mobile broadband trafficSources: (i) Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2010–2015, (ii) John Spindler

(ADC), “Augmented reality needs a better network”, (iii) Perey Research & Consulting, Switzerland, (iv) Measurements by Univ. of Kaiserslautern, (v) Joe Ludwigs blog, “7 Requirements for an Augmented Reality Positioning System”, and discussion, (vi) Nokia Mixed Reality - Nokia World, (vii) NSN – 2020: Beyond 4G

Augmented human cognition - Supporting networks and services

Some number estimates for AR only With Augmented Reality:

Data rate per AR user: 50 kb/s – 1 Mb/s depending on who you ask and applications to be supported

1 – 4 hours use per day (for leisure and professional users) 5 – 25 % of smart phones will be AR enabled Resulting increase of traffic compared to no-AR: Mobile data volume per AR-enabled smart phone (25 days /

month): 562 - 45000 MB/month (factor 7 – 570 compared to today)

Average increase of overall mobile data volume (compared to today): 20 x more smart phones up to 2.500 more traffic in the worst case (and this is real-time traffic)

Latency < 50ms

Augmented human cognition - Supporting networks and services

Requirements Computation and content caching close to the edge

Minimise latency Optimised access to air interface

Intelligent choice of transmission technology Optimised mobility management Context awareness

Access through any available infrastructure Different operator Service handling in the proximity of the users

Local break-out and interconnection of mobile devices directly Through point-to-point wireless technologies But still maintain service session control

High-bandwidth upload from multiple mobile users Lower OPEX through local communication management

Augmented human cognition - Supporting networks and services

Work by Eurescom study P2155 Deutsche Telekom AG Laboratories Portugal Telecom Inovação Orange Labs – France Télécom Eurescom GmbH

http://www.eurescom.eu/services/eurescom-study-programme.html

Contact: gavras@eurescom.eu

Augmented human cognition - Supporting networks and services