Includes comments from (see http://forum.complexevents.com/viewtopic.php?f=13&t=316&p=1325#p1321 ff)- Dimitris Iakovidis 27 Dec 2011- Andrew Hunter 3 Jan 2012- Dimitris 4. Jan- Bernard de Bono 5. Jan- Dimitris 6. Jan- Bernard de Bono 6 Jan, 9 Jan- Wolfgang Maass 13 Jan- RvA 18 Jan--RvA 5 Feb SotA and What is Beyond materials

BioMedCEP shaping for NBIS Call V0.91

Objective ICT-2011.9.11 NBIS

• learn more about the relationship between structure, dynamics and function in neuronal circuits and assemblies, and how information is represented or “coded” in a brain.• develop deeper and more comprehensive theories of neural processing, possibly building on results obtained in the domains of dynamic and complex systems.• close the gap between neuroscience and engineering by motivating interdisciplinary work that ties data with theories, novel computing paradigms, models and implementations.Target outcome

• Developing and applying radically new neural recording, imaging or interfacing concepts and designs for a deeper understanding of neural information processing.• New multi-scale dynamical theories of neural representation for the development of neuro-bio-ICT systems that can perform high-level tasks (e.g. robust object recognition, or classification), going beyond purely sensory-driven information processing. Exocortex systems• Development and prototyping of modular brain-like computing architectures that combine neural processing primitives to give a better understanding of brain function and facilitate the design of more complex processing systems for real-time and optimized performance.• World-class global research cooperation and alliances in this area, and links with similar actions outside Europe, in particular with participants from USA and Japan.

Expected impact

3 Target outcome in the case of IP:• New computing paradigms leading to advanced bio-inspired sensing and processing systems, which are naturally able to learn and adapt• New concepts leading to new brain-computer interface technologies• Target outcome in the case of CSA: New EU and global collaborations between researchers in multiple disciplines spanning engineering, physical and life science domains

The objectives of NBIS are more IT- than Health-related. If the IT objectives are met it would be a “plus” to have an impact on health, although it is not a requirement of the call.

Main idea of NBIS is: “Study, analyze, model brain function(s), not necessarily of a human brain, and use this/these models to build artificial processing/sensing system(s) more efficient/effective than the state of the art ones, exploitable in the framework of novel brain-computer interface technologies.”

Essence of the NBIS Call:

Middleware

Monitor / Analyze / Act

ProcessModeler

EventModeler

EventStore

analysehistory…

Low Level Event Streams

Event Type Adapters

Normalized events,build higher level events

Process Enginetbd: Process ExecutionLanguage

CEP Engine tbd: Event ProcessingLanguage for U-CEP

IF …AND …FOLLOWED BY…WITHIN…ACTION

Event ProcessingModel

e.g. GPS-signal

• Which events are important?

• How are events correlated?

• When and how should we react?

Process Models

„unus mundus“

- Internetservices and their events

e.g. Traffic Message Controlse.g. Weather Forecast

e.g. JMS pub/sub

e.g. RFID …

Integration with other proposals: e.g. - Ray Kurzweil: Singularity is Near / Henry Markram Blue Brain / - Bruce H. Lipton: Epigenetics – Intelligent cells / - Karlheinz Meier: Design, construction and Operation of a Neuromorphic Computation facility - Plamen Simeonov Integral Biomathics, - Francois Képès, Marc Schoenauer : Using Evolution to compute - Kevin Warwick: Brain Computer Interface - Cyborg

Enhancing human intelligence and cognitive or physical abilitiesconnect humans to more events of the universe (e.g. also Internet of Things and Services, “smart dust”)

Extracellular receptors = event adapters

Intracellular effectors = Event Processing Agents

Protein machinery

Protein machinery

…

Intracellular effectors = Event Processing AgentsIntracellular effectors = Event Processing Agents

NeuroColumn

FET-F U-CEP presentation Jan, June 2010

BioMedCEP and clinical applications

Carel Meskers/LUMC

BioMedCEP and clinical applications – U-CEP based automatic treatment - Biomarkers as complex event patterns, and therapeutics as (pre-) modelled processes- „Understanding“ and „rebooting“ the brain- Mind reading for clinical applications and for HETs

VPH-FET 2011, Marco Viceconti et al.Paper 2012 Filippo Castiglione, Andrea Gaggioli et al „Physio-environmental sensing and live modeling based on U-CEP“

Biosensors and biomarkers, e.g.

BrainPort and CN-NINMBCI as interface to the brain(U Wisconsin / TCNL)

Mind reading – based on (complex) event patterns BCI as interface from the brainU Würzburg, U Tübingen, MPI Leipzig, UCLA, MIT, IBM… http://www.silicon.de/technologie/mobile/0,39044013,41556997,00/gedanken_mal

en_am_computer_bilder.htm,

BioMedCEP abstract-tbd –

- http://forum.complexevents.com/viewtopic.php?f=13&t=316&p=1332#p1330- http://www.citt-online.com/downloads/BioMedCEP-preprop_en.doc

BioMedCEP EPSS para. 1.1ff “State of the Art / What is Beyond”1 Scientific and Technical Quality, Relevant to the Topics addressed by the Call 1.1 Concept and Objectives 1.2 Progress beyond the State-of-the-art

-Tbd, this extended abstract/paper has to adapted or enhanced according to the Call/Objective –

http://www.citt-online.de/downloads/3-Danilov-Tyler-Ammon-Etzion.pdf+ e.g. enhancement of Erwin Vlugt about clinical applications and Ageing/Mobility of elder people, etc.

-Tbd, overview comparison Brain projects and BioMedCEP positioning-http://forum.complexevents.com/viewtopic.php?f=13&t=257&p=1364#p1364

The basic workpackage structure of BioMedCEP

WP1Management

WP2Reference model + reference architecture

(for integrating U-CEP, Brain Computer Brain Interfaces, BAN, Quantum Computing or cogntive comp. chips, etc.)

WP3(Complex Event) Processing, Analysis and Modeling

WP 4Model validation

WP6.2De-pression

WP6.3Diabetes

WP6.1Ageing

WP6.4Obesity

WP5.1Sensitivity Range

WP5.2Add senses

WP 5Beyond senses BCI

WP2 (Reference Model and Reference Architecture for the BioMedCEP approach):RM and RA integrate U-CEP, Event adapters, Brain Computer Brain Interfaces, BAN, Quantum Computing or Cognitive Comp. Chips, etc as a combined technology for WP5.1 „Enhancing sensitivity range“ and WP5.2 „Add new senses“ via an Exocortex.The term “reference model” could also mean the selection of the biological model which we would like to get inspired from. If this is what it means then we should provide 2-3 justified alternatives. So far, BioMedCEP uses the 6-layered neuro-columns and the model that “old” or originally born event patterns are stored on the lower levels, the acquired or “historical” event patterns are stored in the middle layers and current, “inflying” events are processed on the top-level layer. Although the human brain can only process 120.000 events per second unconsciously, and around 7 event patterns logically or consciously, but the performance comes from the combination of the event patterns of several layers and the fact that event patterns are not processed every time again. (more in our papers and references)However, in the case of a “fault” in event processing (or trauma situations, etc.), this is the reason for diseases which could be treated or healed with the proposed U-CEP based approach, what should be covered by WP6.

Comments wrt the Workpackages:

Andrew Hunter (Ulincoln): …the focus that now really makes sense for us is on sensor/signal interpretation, real-time, using specialized parallel hardware (FPGA) and neurally-inspired architectures. In the BRAINS project, funded by the UK's Trade and Strategy Board, we have built a neurally-inspired architecture that has a number of modular computational capabilities, including collision detection, pattern recognition, anomaly detection and operates at high-speed, low current-consumption and therefore suitable for portable and/or wearable devices, and because FPGA-based has low reconfiguration/redesign costs. On the back of this we have built the world's first fully functional video-based security surveillance system programmed solely on FPGA, incorporation the full pipeline of video decoding, filtering, feature extraction, tracking, pattern recognition, anomaly detection of signal output. We would therefore like to contribute to WP2 in the development of FPGA-based cognitive computing chips, the key concepts being: Design a neurally-inspired architecture with an architecture sufficiently well-aligned with natural neuronal structure to allow a "sympathetic" mapping, while respecting the architectural differences of the hardware platform (a key realisation for us is that you need to "respect the architecture" to make a really functional FPGA-based neural analogue);Ensure the resulting specialized hardware-based architecture can be integrated directly with sensors on a wearable system, is low-cost, low energy consumption and capable of massive computational demands on real-time scale, exploiting the FPGA inherently parallel architecture;Build neurally-inspired algorithms for signal translation/interpretation for this architecture.

WP3(Complex Event) Processing, Analysis and Modeling):Signal and complex event processing/analysis from multisource data in order to get a deeper understand neural information processing and build a novel dynamic model that would serve the simulation of complex dynamic systems. This model should be able to solve more efficiently real-world problems.Modelling approaches of (class) diagrams for thoughts or bio-markers will be some of the most challenging tasks in the BioMedCEP project."Mind reading„ andIf we model a class diagram of "Thought" as software engineers, we would probably model it similarly to a "Notification Event Architecture of <domain>" like NEAR for the Retail domain - as a reference model of "Thought", not of "Thinking", because a class diagram does not model dynamics.The class diagram might become very complex, starting from a superclass of "Thought" with a lot of subclasses of special thoughts or components (Gen/Spec) and their attributes and operations or methods and a lot of associations, compositions, aggregations and so on... When we have this, we can define patterns and look for them in the brain and map them to machines or robots and manage or control them via "just thinking". Or vice versa and enhance the human by HET or by an exocortex as we have sketched it in our papers and references.

Mind reading and what a human is thinking - based on a catalogue of thoughts or thinking patterns correlated with realtime fMRI brain activity patterns. The most challenging part is probably the modelling of the class diagram

We shall cooperate with other accordant projects from John Dylan-Haynes, Max-Planck-Institute für Kognitions- und Neurowissenschaften in Leipzig , UCLA's Laboratory of Integrative Neuroimaging Technology, etc.See also the Comparison of neuro-science or brain projects http://forum.complexevents.com/viewtopic.php?f=13&t=257&p=1312#p1301

The problem of modeling the context: What "context" or “world knowledge" means and why it is important as a precondition to model and trigger the accordant (re-) actions based on complex event patterns…When do we have to start on which level with modeling the context? Is there a hierarchy of contexts or a relationship between contexts? How do we model context? By UML and its class-, state- or activity diagrams? Or directly as SQL-like or another EPL code without any abstraction level? We have to check what the "studios" of all the CEP- or ed(B)PM platforms can offer or must be enhanced. There was also already an initiative of the OMG (Jim Odell) to standardize a notation, but is not started yet. We should such standardizing make a task in BioMedCEP. http://forum.complexevents.com/viewtopic.php?f=13&t=252&start=20#p1317This WP has to work not only on modelling notations but also on executable models http://www.citt-online.com/downloads/EDBPM-workshop09.ppt, slide 20, fig. 2 in http://www.citt-online.com/downloads/Integrating_Complex_Events_for_Collaborating_and_Dynamically_Changing_Business_Processes_MONA_Final.pdf

Uncertainty modeling (as also mentioned in Ch. 5 of the VPH-FET roadmap). We experiment with modeling approaches from UAI (Uncertainty in Artificial Intelligence, http://www.auai.org/, http://citeseerx.ist.psu.edu/viewdoc/do ... 1&type=pdf, http://citeseerx.ist.psu.edu/viewdoc/do ... 1&type=pdf) as well as the suggestion of a reference model for non-deterministic U-CEP applications (see workshop paper Danilov/Tyler/Ammon/Etzion) and special aspects of uncertainty in the disciplin of CEP (Etzion/Niblett 2010).

Bernard de Bono:knowledge representation (and associated visualization) of biological structure, and related disease mechanisms, across multiple scales, as well as the use of ontology-based reasoning in support of discovering novel relations between resources annotated by such ontologies. If this IP plans to manage clinically-related resources (i.e. models and data), and to ensure that the modeling frameworks emerging from its effort is semantically interoperable with standards being established by the VPH and Pharma community, then the application of the RICORDO metadata infrastructure may have key contributions to offer to BioMedCEP. The newly-emerging ApiNATOMY toolkit is being developed for a GUI to sit on top of the RICORDO infrastructure and provide a visual schematic of anatomy ontologies (including, therefore, neuroanatomy) as a way to overview brain nuclei, pathways and associated metadata. We are also in the early stage of applying RICORDO and ApiNATOMY in the knowledge-based resource management of the Virtual Physiological Rat co-ordinated by Wisconsin

Wolfgang Maass:It would be fun to see, whether some of our theories for spike based processing and learning could also be fruitful in this expanded context. http://scholar.google.com/scholar_url?hl=de&q=http://193.54.228.31/fr_vers/documents/thorpe_s_01_715.pdf&sa=X&scisig=AAGBfm0NI--D7r024kKVfT9InkYrGNIGDA&oi=scholarrhttp://scholar.google.com/scholar_url?hl=de&q=http://www.t35.ph.tum.de/addons/publications/Kempter-2001b.pdf&sa=X&scisig=AAGBfm2CyQgvivUYtfTUrw4DMZt7A0Vjuw&oi=scholarrhttp://scholar.google.com/scholar_url?hl=de&q=http://neurophysics.huji.ac.il/~guetig/papers/guetig06.pdf&sa=X&scisig=AAGBfm2vzJudatx95o_e6ew1Eq4PFSuKPQ&oi=scholarr http://www.em.mpg.de/index.php?id=281

Rainer von Ammon: Perhaps this idea of spike based processing could be brought together/connected with indexing idea/problem (of Jeff Hawkins On Intelligence 2005) of how to find the “stored” and already processed “historical” or acquired or a”already born with” event patterns with the current inflying events depending on the context or the situational conditions. Question is: how to retrieve and correlate event patterns from the different 6 layers of the neocortex. What would spikes do?

Rainer von Ammon:Another question in this connection might be where the memory or the memories (= stored event patterns?) is/are? Neuroscientists identify a master controller of memory: When you experience a new event, your brain encodes a memory of it by altering the connections between neurons. This requires turning on many genes in those neurons. Now, neuroscientists at MIT’s McGovern Institute for Brain Research have identified what may be a master gene that controls this complex process.How would that be related to the theory of the six layers of the neurocolumns and where the "old" or "historically" acquired event patterns are stored (on the lower levels) or the current events are processed (on the top levels)? (e.g. Kevin Hawkins, On Intelligence, 2005). What would we find respectively what would be marked with the help of Npas4?http://forum.complexevents.com/viewtopic.php?f=13&t=257&p=1316#p1316

Comments wrt the Workpackages:

WP4 (Model Validation):This WP validates the theoretically based results of WP 3 and brings together the international key players of relevant technologies as named in WP3. …

Comments wrt the Workpackages:

WP5 (Beyond Senses BCI)We focus on enhancing the sensitivity range of a sense and adding new senses – through brain-computer interfaces (BCI) exploiting the model developed in WP 3. This WP will develop and exploit an U-CEP based Exocortex and will experiment with an approach to map such enhanced sensitivity ranges and additional senses to the limited event processing capabilities of a human brain.Examples might be to smell pollution or radioactivity via new body area networks, bio-sensors, event adapters, etc., as sketched http://forum.complexevents.com/viewtopic.php?f=13&t=299 plus to (re-) act accordingly via (pre-) modeled processes.Another example might be to add a magnetic sense and to experiment with human behaviour and adapting brain functioning (according to first hints from belt technology, see U Osnabrück).More applications and examples for enhanced sensivity ranges and additional senses will be elaborated.

Comments wrt the Workpackages:

WP6 (Real-World Clinical Applications):All clinical applications are organized in WP6 where we work on understanding the human brain and healing diseases by changing the event processing of a human brain or to “reboot” a brain (see BrainPort, …) or to delete “false event patterns” in the memory (see http://forum.complexevents.com/viewtopic.php?f=13&t=257&start=10#p1316). This is seen asa “fault” in event processing (as a consequence of trauma situations, accidents, etc.), what is the reason for diseases (balance, depression, …) which could be treated or healed with the proposed U-CEP based approach. To find accordant “treatments” will also help to learn more about the brain and to find more functions and applications of neuro-biologically inspired systems. Typical, main health problems and diseases of the today’s society (like ageing, depression, diabetes, obesity) are addressed. Each problem or disease represents a discriminant challenge of the BioMedCEP approach. …

Comments wrt the Workpackages:

WP7 (Impact Assessment):…. Insert your suggestions

Comments wrt the Workpackages:

WP8 (Training):…. Insert your suggestions

Comments wrt the Workpackages:

WP9 (Dissemination and Exploitation)…. Insert your suggestions… isn’t exploitation already covered by WP5 and WP6??? So we should only focus on Dissemination with WP9???

Dimitris:“Exploitation” usually refers to a formal viability study/exploitation plan that refers to the way the project’s results will and can be exploited after the end of project e.g. commercial products, possible chain reaction of future research that could be triggered etc.

Comments wrt the Workpackages:

UC1: Enhancing sensitivity range of a sense: Increasing / decreasing sensitivity range, How to map to the brain, Superimposition of „normal“ events by new events,

Will the brain change / adapt, Ego States / Consciousness / Reality… Applications of an Exocortex

UC2: Adding new senses: Applications of an Exocortex, How to map new senses to the brain, Superimposition of „normal“ events by new events,

Will the brain change / adapt, Ego States / Consciousness / Reality…

BioMedCEP - Use Cases related to the Objectives, Outcomes and Impacts- Will be changed or deleted

UC3: Learn more about Ageing and the brainSubstitute typical problems or diseases wrt Ageing, Provide mobility of elder people,

<insert bullets, …>

UC4: Depression as a fault of event processingTreatments on organ level and cell- or protein level, change metabolism by U-CEP or BioMedCEP approach,

<insert bullets, …>

UC5: Diabetes and new treatments based on U-CEP Biosensing and BiomarkersChange or influence thinking and behaviour, addiction, Treatments on organ level and cell- or protein level, change

metabolism by U-CEP or BioMedCEP approach, <insert bullets, …>

UC6: Obesity and new treatments based on U-CEP Biosensing and Biomarkers Change or influence thinking and behaviour, addiction, Treatments on organ level and cell- or protein level, change

metabolism by U-CEP or BioMedCEP approach , <insert bullets, …>

Exploitation and dissemination: Use cases and the workpackage structure of BioMedCEP

Will be changed or deletedW

P1M

anag

emen

t

WP2

Refe

renc

e m

odel

+ re

fere

nce

arch

itect

ure

(for i

nteg

ratin

g U-

CEP,

Bra

in C

ompu

ter B

rain

Inte

rfac

es, B

AN, Q

uant

um C

ompu

ting

or c

ognti

ve c

omp.

ch

ips,

etc

.)

WP3

(Com

plex

Eve

nt) P

roce

ssin

g, A

naly

sis a

nd M

odel

ing

WP

4M

odel

val

idati

on

WP6

.2De

-pre

ssio

nW

P6.3

Diab

etes

WP6

.1Ag

eing

WP6

.4O

besit

y

WP5

.1Se

nsiti

vity

Ran

geW

P5.2

Add

sens

es

WP

5Be

yond

sens

es B

CI

UC1: Enhancing sensitivity range of a sense:

UC2: Adding new senses:

UC3: Learn more about Ageing and the brain

UC4: Depression as a fault of event processing

UC5: Diabetes and new treatments based on U-CEP Biosensing and Biomarkers

UC6: Obesity and new treatments based on U-CEP Biosensing and Biomarkers

WP1

Man

agem

ent

WP2

Refe

renc

e m

odel

+ re

fere

nce

arch

itect

ure

(for i

nteg

ratin

g U-

CEP,

Bra

in C

ompu

ter B

rain

Inte

rfac

es, B

AN, Q

uant

um C

ompu

ting

or c

ognti

ve c

omp.

ch

ips,

etc

.)

WP3

(Com

plex

Eve

nt) P

roce

ssin

g, A

naly

sis a

nd M

odel

ing

WP

4M

odel

val

idati

on

WP6

.2De

-pre

ssio

nW

P6.3

Diab

etes

WP6

.1Ag

eing

WP6

.4O

besit

y

WP5

.1Se

nsiti

vity

Ran

geW

P5.2

Add

sens

es

WP

5Be

yond

sens

es B

CI

UC1: Enhancing sensitivity range of a sense:

UC2: Adding new senses:

UC3: Learn more about Ageing and the brain

UC4: Depression as a fault of event processing

UC5: Diabetes and new treatments based on U-CEP Biosensing and Biomarkers

UC6: Obesity and new treatments based on U-CEP Biosensing and Biomarkers

Tasks related to the use cases and the workpackage structure of BioMedCEP

Will be changed or deleted

T2.1

T2.2

T2.3

T2.4

T2.5

T2.6

T3.1

T3.2

T3.3

T3.4

T2.5

T3.6

T4.1

T4.2

T4.3

T4.4

T4.5

T4.6

T5.1

T5.2

T5.3

T5.4

T5.5

T5.6

T5.1.1

T5.2.1

T7.1

T7.2

T7.3

T7.4

T7.5

T7.6

T6.2

T6.1

T6.4

T6.3

T8.1

T8.2

T8.3

T8.4

T8.5

T8.6

T9.1

T9.2

T9.3

T9.4

T9.5

T9.6

Part. No. Participant Organisation Name Part. Short Name Country

1 (Coord.) ??? OFFIS or FZI or MPI Leipzig or U Tübingen, Würzburg

2 Centrum für Informations-Technologie Transfer GmbH CITT Germany

3 TNCL / University of Wisconsin TCNL USA

4 TU Delft NL

5 University Medical Center Leiden (clinical appl., Ageing) NL

6 Center for Technological Research of Central Greece, Technological Educational Institute of Lamia, IVIBIS group CTRCG/TEILAM GR

7 U Lincoln UK

8 Cambridge EMBL-EBI BioInformatics Inst. / Auckland NZ UK

9?U Heidelberg? Other BrainScaleS member? EPFL/ Haynes? Germany

10 ?http://www.igi.tugraz.at/English.html or http://www.csc.kth.se/forskning/cb/cbn/ AT/SE

11 ?MPI Psychiatrie Holsboer Munich (disease Depression)? Germany

12?IBM Almaden? Haifa? Zurich (Cognitive Chip, U-CEP platform)SAG? SAP? Streambase? Starview? FZI ETALIS?

13?MPI Biophysics Frankfurt? U Berkeley? Princeton? event adapter provider Germany

... … Japan

…

ConsortiumConsortium should at least have experts on a) brains (mainly neuroscientists), b) intelligent systems – data/event processing and analysis, c) sensors, d) brain-computer interfaces, and if we would like to address specific health issues such as dementia, obesity etc then e) experts on these specific health issues.

Work Package

No.

Work Package Title Type of Activity

Lead Partic.

No.

Lead Partic. short Name

Person-months

StartMonth

EndMonth

WP 1 Project Management MGT 1 ??? M1 M??

WP 2 RTD 2, 6, 7CITT?

CTRCG?ULinclon?

M1 M??

WP 3 RTD 2,6 CITT?CTRCG? M1 M??

WP 4 RTD M?? M??

WP 5 RTD/DEM 6 CTRCG? M?? M??

WP6 6

WP7

WP8 2,6

WP 9 Dissemination and Exploitation RTD/DEM 2,6 CITT? M?? M??

TOTAL

Workpackages list

Del. No. Deliverable Name WP No. N

Dissemi-nation Level

Delivery Date

WP 1D1.1 1 M PU M?D1.2 1 M PU M?D1.3 1 M CO M?…

WP 2D2.1 2 R PU M?D2.2 2 R, P PU M?D2.3 2 R PU M?D2.4 2 R PU M?D2.5 2 R PU M?D2.6 2 R, P PU M?WP 3D3.1 3 R PU M?D3.2 3 R PU M?D3.3 3 P PU M?...WP 4

D4.1 4 R PU M?

... 4 R,P PU M?

...

Deliverables list

Milestone No. Milestone Name Work Package(s) involved Expected Date

1 … M?

2 WP2, WP3 M?

3 WP3, WP4, WP5 M?

4 WP3, WP4, WP5, WP6 M?

5 WP3, WP4, WP5, WP6 M?

… ... M?

List of milestones

Work package number c 2 Start date or starting event: Month 1

Work package title

Activity type RTD

Participant number 1 2 3 4 5 6 7

Participant short name CITT CTRCG ULINC

Person-months per participant

18 18 18

Objectives ………

Description of Work

WP-Leader: CITT?T2.1: (CITT, ..., ...)

The main topic of this first task is to define... blabla

Deliverables

D2.1 ... (M?)

D2.2 ... (M7)

D...

Work Package Description