Will we ever have Conscious Machines?

Patrick Krauss1,2 and Andreas Maier3

1Neuroscience Lab, Experimental Otolaryngology, University Hospital Erlangen,Germany

2Cognitive Computational Neuroscience Group at the Chair of English Philologyand Linguistics, Department of English and American Studies,

Friedrich-Alexander University Erlangen-Nurnberg (FAU), Germany3Chair of Pattern Recognition, Friedrich-Alexander University

Erlangen-Nurnberg (FAU), Germany

April 1, 2020

Corresponding author:Prof. Andreas MaierChair of Pattern RecognitionFriedrich-Alexander University of Erlangen-Nurnberg (FAU), GermanyE-Mail: andreas.maier@fau.de

Keywords:machine consciousness, artificial intelligence, theories of consciousness

arX

iv:2

003.

1413

2v1

[cs

.AI]

31

Mar

202

0

Abstract

The question of whether artificial beings or machines could becomeself-aware or consciousness has been a philosophical question for cen-turies. The main problem is that self-awareness cannot be observedfrom an outside perspective and the distinction of whether somethingis really self-aware or merely a clever program that pretends to doso cannot be answered without access to accurate knowledge aboutthe mechanism’s inner workings. We review the current state-of-the-art regarding these developments and investigate common machinelearning approaches with respect to their potential ability to becomeself-aware. We realise that many important algorithmic steps towardsmachines with a core consciousness have already been devised. Forhuman-level intelligence, however, many additional techniques haveto be discovered.

Introduction

The question of understanding consciousness is in the focus of philosophersand researchers for more than two millennia. Insights range broadly from “Ig-norabimus” – “We will never know.”1 to mechanistic ideas with the aim toconstruct artificial consciousness following Richard Feynman’s famous words“What I cannot create, I do not understand.”2.

The major issue that precludes the analysis of consciousness is its sub-jectivity. Our mind is able to feel and process our own conscious states. Byinduction, we are also able to ascribe conscious processing to other humanbeings. However, once we try to imagine to be another species, as Nagel de-scribes in his seminal work “What is it like to be a Bat?” [1], we immediatelyfail to follow such experience consciously.

Another significant issue is that we are not able to determine conscious-ness by means of behavioural observations as Searle demonstrates in histhought experiment [2]. Searle describes a room that we cannot enter. Onecan pass messages written in Chinese to the room and the room returns mes-sages to the outside world. All the messages and questions passed to the

1With this simple statement, Emil du Bois-Reymond concluded his talk on the limitsof scientific knowledge about the relation of brain processes and subjective experience atthe 45th annual meeting of German naturalists and physicians in 1872.

2Richard Feynman left these words on his blackboard in 1988 at the time of his deathas a final message to the world.

room are answered correctly as a Chinese person would. A first conclusionwould be that there is somebody in the “Chinese Room” who speaks Chineseand answers the questions. However, the person in the room could also havesimply access to a large dictionary that contains all possible questions andthe respective answers. When we are not able to understand how the infor-mation is actually processed, we will never be able to determine whether asystem is conscious or not.

In this article, we want to explore these and different thoughts in litera-ture to address the problem of consciousness. We therefore revisit works inphilosophy, neuroscience, artificial intelligence, and machine learning. Fol-lowing the paradigm of cognitive computational neuroscience [3], we presenthow the convergence of these fields could potentially also lead to new insightsregarding consciousness.

The Philosophical Perspective

More than two thousand years ago, Aristotle was convinced that only humansare endowed with a rational soul. All animals, however, live only with theinstincts necessary for survival, like biological automata. Along the sameline, in the statement “Cogito ergo sum” also Descartes realised being self-aware is reserved for human beings. In his view, this insight is fundamentalfor any philosophical approach [4].

Modern philosophy went on to differentiate the problem into an easy and ahard problem. While the “easy problem” is to explain its function, dynamics,and structure, the “hard problem of consciousness” [5] is summarised in theInternet Encyclopedia of Philosophy [6] as:

“The hard problem of consciousness is the problem of explaining why anyphysical state is conscious rather than nonconscious. It is the problem ofexplaining why there is “something it is like” for a subject in conscious ex-perience, why conscious mental states “light up” and directly appear to thesubject.”

In order to avoid confusion some scientists prefer to speak of “consciousexperience” or only “experience” instead of consciousness [5]. As alreadynoted, the key problem of deriving models of conscious events is that theycan only be perceived subjectively. As such it is difficult to encode such anexperience in a way that it can be recreated by others. This gives rise to theso-called “qualia problem” [7] as we can never be sure, e.g. that the color

red consciously looks the same to another person. Extension of this line ofthought leads again to Nagel’s thought experiment [1].

According to [6], approaches to tackle the problem from a philosophicalpoint of view are very numerous, but none of them can be considered to beexhaustive:

• Eliminativism [8] demonstrates that the mind is fully functional with-out the experience of consciousness. Being nonfunctional, consciousnesscan be neglected.

• The view of strong reductionism proposes that consciousness canbe deconstructed into simpler parts and be explained by functionalprocesses. Such considerations gave rise to the global work space the-ory [9–11] or integrated information theory [12,13] in neuroscience. Themain critique of this view, is that any mechanistic solution to conscious-ness that is not fully understood will only mimic true consciousness,i.e., one could construct something that appears conscious that simplyisn’t as the Chinese Room argument demonstrates [2].

• Mysterianism proposes that the question of consciousness cannot betackled with scientific methods. Therefore any investigation is in vainand the explanatory gap cannot be closed [14].

• In Dualism the problem is tackled as consciousness being metaphys-ical that is independent of physical substance [4]. Modern versions ofDualism exist, but virtually all of them require to reject that our worldcan be fully described by physical principles. Recently, Penrose andHammeroff tried to close this gap using quantum theory [15, 16]. Wededicate a closer description of this view in a later section of this article.

• Assuming that metaphysical world and physical world simply do notinteract does not require to reject physics and gives rise to Epiphe-nomenalism [17].

There are further theories and approaches to address the hard problem ofconsciousness that we do not want to detail here. To the interested reader, werecommend to study the Internet Encyclopedia of Philosophy [6] as furtherreading into the topic.

In conclusion, we observe that a major disadvantage of exploring thesubject of consciousness by philosophical means is that we are never able

to explore the inside of the Chinese Room. Thought alone will not be ableto open the black box. Neuroscience, however, offers means to explore theinside by various means of measurement and might offer suitable means toaddress the problem.

Consciousness in Neuroscience

Historical Overview

In 1924, Hans Berger recorded, for the first time, electrical brain activityusing electroencephalography (EEG) [18]. This breakthrough enabled theinvestigation of different mental states by means of electrophysiology, e.g.during perception [19] or during sleep [20]. The theory of cell assemblies,proposed by Donald Hebb in 1949 [21], marked the starting point for thescientific investigation of neural networks as the biological basis for percep-tion, cognition, memory, and action. In 1965, Gazzaniga demonstrated thatdissecting the corpus callosum which connects the two brain hemisphereswith each other results in a split of consciousness [22, 23]. Almost ten yearslater, Weiskrantz et al. discovered a phenomenon for which the term “blind-sight” has been coined: following lesions in the occipital cortex, humans loosethe ability to consciously perceive, but are still able to react to visual stim-uli [24,25]. In 1983, Libet demonstrated that voluntary acts are preceded byelectrophysiological readiness potentials that have their maximum at about550ms before the voluntary behavior [26]. He concluded that the role of con-scious processing might not be to initiate a specific voluntary act but ratherto select and control volitional outcome [27]. In contrast to the above men-tioned philosophical tradition from Aristotle to Descartes that consciousnessis a phenomenon that is exclusively reserved for humans, in contemporaryneuroscience most researchers tend to regard consciousness as a gradual phe-nomenon, which in principle also occurs in animals [28], and several maintheories of how consciousness emerges have been proposed so far.

Neural Correlates of Consciousness

Based on Singer’s observation that high-frequency oscillatory responses in thefeline visual cortex exhibit inter-columnar and inter-hemispheric synchroniza-tion which reflects global stimulus properties [29–31] and might therefore be

the solution for the so called “binding problem” [32], Crick and Koch sug-gested Gamma frequency oscillations to play a key role in the emergence ofconsciousness [33]. Koch further developed this idea and investigated neu-ral correlates of consciousness in humans [34, 35]. He argued that activityin the primary visual cortex, for instance, is necessary but not sufficient forconscious perception, since activity in areas of extrastriate visual cortex cor-relates more closely with visual perception, and damage to these areas canselectively impair the ability to perceive particular features of stimuli [36].Furthermore, he discussed the possibility that the timing or synchronizationof neural activity might correlate with awareness, rather than simply theoverall level of spiking [36]. A finding which is supported by recent neu-roimaging studies of visual evoked activity in parietal and prefrontal cortexareas [37]. Based on these findings, Koch and Crick provided a frameworkfor consciousness, where they proposed a coherent scheme to explain theneural activation of visual consciousness as competing cellular clusters [38].Finally, the concept of neural correlates of consciousness has been furtherextended to an index of consciousness based on brain complexity [39], whichis independent of sensory processing and behavior [40], and might be used toquantify consciousness in comatose patients [41].

Consciousness as a Computational Phenomenon

Motivated by the aforementioned findings concerning the neural correlatesof consciousness, Tononi introduced the concept of integrated information,which according to his “Integrated Information Theory of Consciousness”plays a key role in the emergence of consciousness [12, 13]. This theory rep-resents one of two major theories of contemporary research in consciousness.According to this theory, the quality or content of consciousness is identicalto the form of the conceptual structure specified by the physical substratesof consciousness, and the quantity or level of consciousness corresponds toits irreducibility, which is defined as integrated information [42].

Tegmark generalized Tononi’s framework even further from neural-net-work-based consciousness to arbitrary quantum systems. He proposed thatconsciousness can be understood as a state of matter with distinctive in-formation processing abilities, which he calls “perceptronium”, and investi-gates interesting links to error-correcting codes and condensed matter criti-cality [43,44].

Even though, there is large consensus that consciousness can be under-

stood as a computational phenomenon [45–48], there is dissent about whichis the appropriate level of granularity of description and modeling [3]. Pen-rose and Hameroff even proposed that certain features of quantum coher-ence could explain enigmatic aspects of consciousness, and that conscious-ness emerges from brain activities linked to fundamental ripples in spacetimegeometry. In particular, according to their model of orchestrated objectivereduction (Orch OR), they hypothesize that the brain is a kind of quantumcomputer, performing quantum computations in the microtubeles, which arecylindrical protein lattices of the neurons’ cytoskeleton [15,49,50].

However, Tegmark and Koch argue, that the brain can be understoodwithin a purely neurobiological framework, without invoking any quantum-mechanical properties: quantum computations which seek to exploit the par-allelism inherent in entanglement, require that the qubits are well isolatedfrom the rest of the system, whereas on the other hand, coupling the systemto the external world is necessary for the input, the control, and the outputof the computations. Due to the wet and warm nature of the brain, all theseoperations introduce noise into the computation, which causes decoherenceof the quantum states, and thus makes quantum computations impossible.Furthermore, they argue that the molecular machines of the nervous system,such as the pre- and post-synaptic receptors, are so large that they can betreated as classical rather than quantum systems, i.e. that there is nothingfundamentally wrong with the current classical approach to neural networksimulations [51–53].

The Global Workspace Theory of Consciousness

In the 1990s, Baars introduced the concept of a virtual “Global Workspace”that emerges by connecting different brain areas (Figure 1) to describe con-sciousness [9–11, 54]. This idea was taken up and further developed by De-haene [55–60]. Today, besides the Integrated Information Theory, the GlobalWorkspace Theory represents the second major theory of consciousness, be-ing intensively discussed in the field of cognitive neuroscience. Based on theimplications of this theory, i.e., that consciousness arises from specific typesof information-processing computations, which are physically realized by thehardware of the brain, Dehaene argues that a machine endowed with theseprocessing abilities “would behave as though it were conscious; for instance, itwould know that it is seeing something, would express confidence in it,wouldreport it to others, could suffer hallucinations when its monitoring mecha-

Long-term Memory

Evaluative Systems

Attentional Systems

Perceptual Systems

MotorSystems

GlobalWorkspace

Figure 1: The Global Workspace emerges by connecting different brain areasaccording to Dehaene.

nisms break down, and may even experience the same perceptual illusions ashumans” [61].

Damasio’s Model of Consciousness

Damasio’s model of consciousness was initially published in his popular sci-ence book “The feeling of what happens” [62]. Later Damasio also publishedthe central ideas in peer-reviewed scientific literature [63]. With the ideasbeing published first in a popular science book, most publications on con-sciousness neglect his contributions. However, we believe that his thoughtsdeserve more attention. Therefore, we want to introduce his ideas quickly inthis section.

The main idea in Damasio’s model is to relate consciousness to the abilityto identify one’s self in the world and to be able to put the self in relation

Protoself

Core Consciousness

Extended Consciousness

Sensory Input

Emotion

Feelings

Memory,Language

Actions

Figure 2: Simplified view of Damasio’s model of consciousness: The proto-self processes emotions and sensory input unconsciously. Core consciousnessarises from the protoself which allows to put the itself into relation. Projec-tions of emotions give rise to higher-order feelings. With access to memoryand extended functions such as language processing the extended conscious-ness emerges.

with the world. However, a formal definition is more complex and requiresthe introduction of several concepts first.

He introduces three levels of conscious processing:

• The fundamental protoself does not possess the ability to recognizeitself. It is a mere processing chain that reacts to inputs and stimulilike an automaton, completely non-conscious. As such any animal hasa protoself according to this definition. However, also more advancedlifeforms including humans exhibit this kind of self.

• A second stage of consciousness is the core consciousness. It is ableto anticipate reactions in its environment and adapts to them. Fur-thermore, it is able to recognise itself and its parts in its own imageof the world. This enables it to anticipate and to react to the world.However, core consciousness is also volatile and not able to persist forhours to form complex plans.

In contrast to many philosophical approaches, core consciousness doesnot require to represent representations of the world in words or lan-guage. In fact, Damasio believes that progress in understanding con-scious processing has been impeded by dependence on words and lan-guage.

• The extended consciousness enables human-like interaction with theworld. It builds on top of core consciousness and enables further func-tions such as access to memory in order to create a autobiographic self.Also being able to process words and language falls into the categoryextended consciousness and can be interpreted as a form of serialisationof conscious images and states.

In Damasio’s theory emotion and feelings are fundamental concepts [64].In particular Damasio differentiates emotions from feelings. Emotions aredirect signals that indicate a positive or negative state of the (proto-)self.Feelings emerge only in conjunction with images of the world and can beinterpreted as a second-order emotion that is derived from the world repre-sentation and future possible events in the world. Both are crucial for theemergence of consciousness. Fig. 2 schematically puts the described terms inrelation.

After having defined the above concepts, Damasio now goes on to attemptand describe a model of (core) consciousness. In his theory, consciousnessdoes not merely emerge from the ability to identify oneself in the world oran image of the world. For conscious processing, additionally feeling oneselfin the sense of desiring to exist is required. Hence, he postulates a feeling,i.e., a derived second-order emotion, between the protoself and its internalrepresentation of the world. Conscious beings as such want to identify oneselfin the world and want to exist. From an evolutionary perspective as heargues, this is possibly a mechanism to enforce self-preservation.

In the context of this article, Damasio’s theory is interesting for two majorreasons. On the one hand, it describes a biologically plausible model of con-sciousness, as he locates all stages of consciousness to structures in the brainand associates them to the respective function. On the other hand, Dama-sio describes a mechanistic model that can at least in theory be completelyimplemented as a computer program.

Hence, we can conclude that neuroscience is able to describe fundamentalprocesses in the brain that give rise to complex phenomena such as conscious-ness. However, the means of observation in neuroscience are insufficient. Nei-ther EEG nor fMRI have a temporal and spatial resolution that is even closeenough to observe what is happening in the brain in-vivo. At this point, therecent massive progress in artificial intelligence and machine learning comesto our attention and will be the focus of our next section.

Consciousness in Machine Learning and AI

In artificial intelligence (AI) numerous theories of consciousness exist [65,66].Implementations often focus on the global work space theory with only lim-ited learning capabilities [67], i.e. most of the consciousness is hard-coded andnot trainable [68]. An exception is the theory by van Hateren which relatesthe consciousness to close to simultaneous forward and backward processingin the brain [69]. Yet, algorithms that were investigated so far made useof a global work space and mechanistic hard-coded models of consciousness.Following this line, research on minds and consciousness rather focuses onrepresentation than on actual self-awareness [70]. Although representationwill be important to create human-like minds and general intelligence [71–73],a key factor to become conscious is the ability to identify a self in one’s en-vironment [61]. A major drawback of pure mechanistic methods, however,is that the complete knowledge on the model of consciousness is requiredin order to realise and implement them. As such, in order to develop thesemodels to higher forms such as Damasio’s extended consciousness, a completemechanistic model of the entire brain including all connections is required.

Consciousness in Machine Learning

A possible solution to this problem is machine learning, as it allows to formand train complex models. The topic of consciousness, however, is neglectedin the field to a large extent. On the one hand, this is because of the concernsthat the brain and consciousness will never be successfully simulated in acomputer system [16,74]. On the other hand, consciousness is considered tobe an extremely hard problem and current results in AI are still meager [75].

The state-of-the-art in machine learning instead focuses on supervised andunsupervised learning techniques [76]. Another important research directionis reinforcement learning [77] that aims at learning of suitable actions for anagent in a given environment. As consciousness is often associated with anembodiment, reinforcement learning is likely to be important for modellingof consciousness.

The earliest work that the authors are aware of attempting to model andcreate agents that learn their own representation of the world entirely usingmachine learning date back to the early 1990’s. Already in 1990, Schmidhu-ber proposed a model for dynamic reinforcement learning in reactive envi-ronments [78] and found evidence for self-awareness in 1991 [79]. The model

follows the idea of a global work space. In particular, future rewards andinputs are predicted using a world model. Yet, Schmidhuber was missinga theory on how to analyse intelligence and consciousness in this approach.Similar to Tononi [13], Schmidhuber followed the idea of compressed neuralrepresentation. Interestingly, compression is also key to inductive reason-ing, i.e., learning from few examples which we typically deem as intelligentbehaviour.

Solomonoff’s Universal Theory of Inductive Inference [80] gives a theoreticframework to inductive reasoning. It combines information and compressiontheory and results in a formalisation of Occam’s razor preferring simple mod-els over complex ones, as simple models are more likely from an informationtheoretic point of view [81].

Under Schmidhuber’s supervision, Hutter applied Solomonoff’s theory tomachine learning to form a theory of Universal Artificial Intelligence [82]. Inhis theory, intelligent behaviour stems from efficient compression of inputs,e.g. from an environment, such that predictions and actions are performedoptimally. Again, models capable of describing a global work space play animportant role.

Maguire et al. further expand on this concept to extend Solomonoff’sand Hutter’s theories to also describe consciousness. Following the ideas ofTononi and Koch [36] consciousness is understood as data compression, i.e.the optimal integration of information [81]. The actual consciousness emergesfrom binding of information and is inherently complex. As such, conscious-ness can also not be deconstructed into mechanical sub-components, as thedecomposition would destroy the sophisticated data compression. Maguire etal. even provide a mathematical proof to demonstrate that consciousness iseither integrated and therefore cannot be decomposed or there is an explicitmechanistic way of modelling and describing consciousness [81].

Based on the extreme success of deep learning [83], also several scientistsobserved similarities in neuroscience and machine learning. In particular,deep learning allows to build complex models that are hard to analyse andinterpret at the benefit of making complex predictions. As such both fieldsare likely to benefit each other in the ability to understand and interpretcomplex dynamic systems [3,84–89]. In particular, hard-wiring following bi-ological ideas might help to reduce the search space dramatically [90]. Thisis in line with recent theoretical considerations in machine learning as priorknowledge allows to reduce maximal error bounds [91]. Both fields can ben-efit from these ideas as recent discoveries of e.g. successor representation

show [92–94]. Several scientists believe that extension of this approach to so-cial, cultural, economic, and political sciences will create even more synergyresulting in the field of machine behaviour [95].

Can Consciousness emerge in Machine Learn-

ing Systems?

After having reviewed philosophy, neuroscience, and the state-of-the-art inAI and machine learning, we can now analyse the most important conceptsin the field of machine and deep learning to assess whether they have thepotential to create consciousness following one of the previous theories. Inparticular, we focus on the ability of the system to represent a symbol of selfand how this self-awareness is constructed, as all theories of consciousnessrequire at least experiencing the self.

We show a brief overview on important architectures in machine learningin Fig. 3. We denote hard-wired connections as solid arrows to indicaterecurrent modes of specific network parts. Dashed lines indicate trainableconnections. They could be implemented as a single feed-forward layer, i.e.,a universal function approximator. Without loss of generality, they couldalso be implemented by other deep feed-forward architectures [96] and couldthus be inherently complex. Red double arrows indicate training losses toadjust the trainable weights of the dashed arrows.

Fig. 3 A shows a simple feed-forward architecture that requires externallabelled training data y∗ to adjust its trainable weights given input x toproduce output y. Fig. 3 B shows a similar setup, for the recurrent case.Note that we only indicate a simple recurrent cell here with time-dependentstate ht. Without loss of generality, this could also be realised using gatedrecurrent units [97] or long short term memory cells [98]. Both models fallinto the category of supervised learning. Consciousness is not supported byany of the theories presented so far.

In Fig. 3 C, we introduce the concept of “Emotion” following Damasio’swording. In machine learning terms, this reflects an additional loss. Now, thesystem receives an additional input e that is associated to a valence or value.Without loss of generality, we can assume positive entries in e to be associatedto desirable states for the system and negative values to undesirable states.As such, training using e falls into the category of reinforcement learning that

Input x

Output y Reference y*

Self

Training

A) Feed Forward Model

Input x

State ht State ht+1

Reference y*

Self

Training

B) Recurrent Model

Input x

Output y

State ht State ht+1

Input e

Output e‘ Reference y*Training

Reference e* Training

Self

C) Recurrent Model with “Emotion”

Input x

Output y

State ht State ht+1

Expected x‘

Training

State wt State wt+1

Self World Model

D) Auto-Encoder Model

Input x

Output y

State ht State ht+1

Input e

Output e‘

Expected x‘

Training

Reference e* Training

State wt State wt+1

Self World Model

E) Auto-Encoder Model with “Emotion”

Input x

Output y

State ht State ht+1

Input e

Output e‘

Expected x‘

Training

Reference e* Training

State wt State wt+1

Expected e‘‘

Self World Model

F) Auto-Encoder with “Emotion” and “Feelings”

Output y

State ht State ht+1

Training

Output e‘Reference e* Training

State wt State wt+1

Input x Input e

Expected x‘ Expected e‘‘

Self World Model

State h‘t State h‘t+1Training

Output e‘Reference e* Training

G) Core Consciousness Model after Damasio

Model SelfModel Self

Input x

Output y

State ht State ht+1

Input e

Output e‘

Expected x‘

Training

Reference e* Training

Expected e‘‘

Self World Model

State h‘t State h‘t+1Training

Memory / Extended Functions

State wt State wt+1

Ausgabe e‘Referenz e* Training

H) Extended Consciousness Model after Damasio

Model SelfModel Self

Output y

Figure 3: Overview on typical architectures in machine learning. Hard-codedpaths, i.e. recurrent connections, are indicated by solid arrows, trainableconnections by dashed arrows, and training losses are indicated by red doublearrows. While archtiectures A-E do not match theories of consciousness,architectures F-H implement theories by Schmidhuber and Damasio.

aims at maximizing future values of e. In order to model competing interestsand saturation effects, e.g. a full battery does not need to be charged further,we introduce a reference e∗ that is able to model such effects. Note that wedeem the system to be able to predict the expected future reward e′ from itscurrent state ht following a deep Q learning paradigm [99]. Here we use e′

and e∗ to construct a trainable reinforcement loss, to be able to learn fromlow-level rewards/emotions e. Although being able to learn, the system stillneeds supervision to train the weights producing output y using a reference.This setup also does not match any theory of consciousness so far.

As self-awareness is a requirement for base consciousness, we deem a worldmodel to be necessary. Such an approach his shown in Fig. 3 D. Given fromthe produced output y, the world model is used to create an expected futureinput x′. In the figure, we chose a recurrent model capturing the state ofthe world in wt that is independent of the internal state of the actual agentht. To gain consciousness, this model misses at least a link from internalto external state and “emotions” that would guide future decisions. Addinglow-level rewards/emotions results in Fig. 3 E. Again world and self aredisconnected inhibiting self-representation and self-discovery. Approacheslike this are already being explored for video game control [100].

With a world-model being present, we are now able to predict futurerewards e′′ that also take into account the state of the world and the chosenaction. As such Fig. 3 F is the first one that would implement a trainableversion of deep Q learning. Development of consciousness is debatable, asthe model does not feature a link between the state of the world and thestate of the agent. If we would add a trainable connection from ht to wt andvice versa, we would end up with Schmidhuber’s Model from 1990 [78] forwhich Schmidhuber found evidence to develop self representation [79].

Interestingly, Damasio’s descriptions follow a similar line in [62]. Wedepict a model implementing Damasio’s core consciousness in Fig. 3 G. AsSchmidhuber, Damasio requires a connection from the world model wt tothe body control system ht. However, in his view, consciousness does notemerge by itself. It is enforced by a “feeling” that is expressed as a loss in theworld of machine learning. As such, the Damasio model of core consciousnessrequires a loss that aims at the recovery of the image of the self in the worldmodel. If this is implemented as a loss, we are able to express the desire toexist in the world. If implemented merely as trainable weights, we arrive atthe theory of integrated information that creates consciousness as maximizedcompression of the world, the self, and their interaction. Interestingly, these

considerations also allow integration of attention [101] and other concepts ofresolving context information in machine learning. Realised in a biologicallearning framework, e.g. using neuromodulators like Dopamin [102], thenotion of loss and connection will disappear and the models of Damasio,Schmidhuber, Tononi, Koch, and Dehane turn out to be different descriptionsof the same principles.

Note that the models of consciousness that we have discussed so far arevery basic. They do not concern language, memory, or other complex multi-modal forms of processing, planning, induction, or representation. Again, wefollow Damasio at this point in Fig. 3 H in which all of these sophisticatedprocesses are mapped into a block “Memory / Extended Functions”. Note al-though we omit these extended functions, we are able to integrate them usingtrainable paths. As such, the model of core consciousness acts as a “neuraloperating system” that is able to update also higher order functions accord-ing to the needs of the environment. With increasing “extended functions”,the degree of complexity and “integrated information” rises measurably asalso observed by Casarotto [39].

This brings us back to the original heading of our section: There areclearly theories that enable modelling and implementation of consciousness inthe machine. On the one hand, they are mechanistic to the extend that theycan be implemented in programming languages and require similar inputsas humans would do. On the other hand, even the simple models in Fig.3are already arbitrarily complex, as every dashed path in the models could berealised by a deep network. As such also training will be hard. Interestingly,the models follow a bottom-up strategy such that training and developmentcan be performed in analogy to biological development and evolution. Themodels can be trained and grown to more complex tasks gradually.

Discussion

Existence of consciousness in the machine is a hot topic of debate. Evenwith respect to the simple core consciousness, we observe opinions rangingfrom “generally impossible” [103] through “plausible” [61] to “has alreadybeen done” [79]. Obviously, all of the suggested models cannot solve thequalia problem or the general problem on how to demonstrate whether asystem is truly conscious. All of the emerging systems could merely be mim-icking conscious behaviour without being conscious at all (even Fig. 3 A).

Yet as already discussed by Schmidhuber [79], we would be able to measurecorrelates of self recognition similar to neural correlates of consciousness inhumans [35] which could help to understand consciousness in human beings.However, as long as we have not solved how to provide proof of consciousnessin human beings, we will also fail to do so in machines as the experience ofconsciousness is merely subjective.

Koch and Dehaene discussed the theories of global work space and in-tegrated information as being opposed to each other [103]. In the modelsfound in Fig. 3, we see that both concepts require a strong degree of inter-connection. As such, we do not see why both concepts are fundamentallyopposing. A global work space does not necessarily have to be encoded indecompressed state. Also, Maguire’s view of integrated information [81] isnot necessarily impossible to implement mechanistically, as we are able touse concepts of deep learning to train highly integrated processing networks.In fact, as observed by neuroscience [3], both approaches might support eachother yielding methods to construct and reproduce biological processes in amodular way. This allows the integration of representation [71] and process-ing theories [65] as long as they can be represented in terms of deep learningcompatible operations [91].

In all theories that we touched in this article, the notion of self is funda-mental. Hence, all presented theories of consciousness require embodiment asbasis for consciousness. As such, a body is required for conscious processing.Also the role of emotion and feelings is vital. Without emotion or feelings,the system cannot be trained and thus it can not adopt to new environmentsand changes of circumstances. In the machine learning inspired models, weassume that a disconnection between environment and self would cause adegradation of the system similar to the one that is observed in human be-ings in locked-in state [104]. This homeostatsis was also deemed importantby Man et al. [105].

Similar to the problems identified by Nagel, also the proposed mechanisticmachine learning models will not be able to understand “what it is like” tobe a bat. However, the notion of train-/learnable programs and connectionsor adapters might offer a solution to explore this in the future. Analogously,one cannot describe to somebody “what it is like” to play the piano or tosnowboard on expert level unless one really acquires the ability. As such alsothe qualia problem persists in machine consciousness. However, we are ableto investigate the actual configuration of the representation in the artificialneural net offering entirely new mechanisms of insight.

In Damasio’s theory, consciousness is effectively created by a trainingloss that causes the system to “want” to be conscious, i.e., ”Cogito ergosum” becomes ”Sentio ergo sum”. Comparison between trainable connec-tions after [78], attention mechanisms [101], and this approach are withinthe reach of future machine learning models which will create new evidencefor the discussion of integrated information and global work spaces. In fact,Schmidhuber has already taken up the work on combination of his early ideaswith modern approaches from deep learning [106,107].

With models for extended consciousness, even the notion of the Homuncu-lus [108] can be represented by extension of the self with another self pointer.In contrast to common rejection of the Homunculus thought experiment, thisrecurrent approach can be trained using end-to-end systems comparable toAlphaGo [109].

Damasio also presents more interesting and important work that is mostlyomitted in this article for brevity. In [62] he also relates structural braindamage to functional loss of cognitive and conscious processing. Also thenotion of emotion is crucial in a biological sense and is the driving effect ofhomeostasis. In [105], Damasio already pointed out that this concept will befundamental for self-regulating robotic approaches.

With the ideas of cognitive computational neuroscience [3] and the ap-proaches detailed above, we will design artificial systems that approach themechanisms of biological systems in an iterative manner. With the itera-tions, the artificial systems will increase in complexity and similarity to thebiological systems. However, even if we arrive at an artificial system thatperforms identical computations and reveals identical behaviour as the bio-logical system, we will not be able to deem this system as conscious beyondany doubts. The true challenge in being perceived as conscious will be the ac-ceptance by human beings and society. As Alan Turing already proposed inhis imitation game in 1950 to decide whether a machine is intelligent or evenconscious [110], the ascription of such by other humans is a critical factor.For this purpose, Turing’s Test has already been extended to also account forembodiment [111]. However, such tests are only necessary, but not sufficientas Gary Marcus pointed out: Rather simple chat bot models are alreadyable to beat Turing’s Test in some occasions [112]. Hence, requirements forconscious machines will comprise, the similarity to biological conscious pro-cesses, the ability to convince human beings, and even the machine itself. Assuch, we deem it necessary to look as some ethical implications at this point.

Ethical implications

Being able to create systems that are indistinguishable from conscious beingsthat are potentially conscious also raises ethical concerns. First and foremost,in the transformation from core consciousness to extended consciousness, thesystems gain the ability to link new program routines. As such systemsfollowing such a line of implementation need to be handled with care andshould be experimented on in a contained environment. With the right choiceof embodiment in a virtual machine or in a robotic body, one should be ableto solve such problems.

Of course there are also other ethical concerns, the more we approachhuman-like behaviour. A first set of robotic laws has been introduced inAsimov’s novels [113]. Even Asimov considered the rules problematic as canbe seen from the plot twists in his novels. Aside this, being able to followthe robotic laws requires the robot to understand the concepts of “humans”,“harm”, and “self”. Hence, such beings must be conscious. Therefore, tam-pering with their memories, emotions, and feelings is also problematic byitself. Being able to copy and reproduce the same body and mind does notlead to further simplification of the issue and implies the problem that wehave to agree on ethics and standards of AI soon [114].

Conclusion

In this article, we reviewed the state-of-the-art theories on consciousness inphilosophy, neuroscience, AI, and machine learning. We find that all threedisciplines need to interact to push research in this direction further. In-terestingly, basic theories of consciousness can be implemented in computerprograms. In particular, deep learning approaches are interesting as they of-fer the ability to train deep approximators that are not yet well understood toconstruct mechanistic systems of complex neural and cognitive processes. Wereviewed several machine learning architectures and related them to theoriesof strong reductionism and found that there are neural network architecturesfrom which base consciousness could emerge. Yet, there is still a long way toform human-like extended consciousness.

Data availability

All data in this publication are publicly available.

Code availability

All software used in this article is publicly available.

Acknowledgments

This work was funded by the Deutsche Forschungsgemeinschaft (DFG, Ger-man Research Foundation): grant KR5148/2-1 to PK – project number436456810, the Emergent Talents Initiative (ETI) of the University Erlangen-Nuremberg (grant 2019/2-Phil-01 to PK). Furthermore, the research leadingto these results has received funding from the European Research Council(ERC) under the European Union’s Horizon 2020 research and innovationprogramme (ERC grant no. 810316).

Author contributions

PK and AM contributed equally to this work.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

References

[1] Thomas Nagel. What is it like to be a bat? The philosophical review,83(4):435–450, 1974.

[2] John R Searle. Minds, brains, and programs. Behavioral and brainsciences, 3(3):417–424, 1980.

[3] Nikolaus Kriegeskorte and Pamela K Douglas. Cognitive computationalneuroscience. Nature neuroscience, 21(9):1148–1160, 2018.

[4] Rene Descartes. Meditations on First Philosophy/Meditationes dePrima Philosophia: A Bilingual Edition. University of Notre DamePess, 1990.

[5] David J Chalmers. Facing up to the problem of consciousness. Journalof consciousness studies, 2(3):200–219, 1995.

[6] Josh Weisberg. The Hard Problem of Consciousness. https://www.

iep.utm.edu/hard-con/. Accessed: 2020-02-28.

[7] Tim Crane. The origins of qualia. In History of the mind-body problem,pages 177–202. Routledge, 2012.

[8] Georges Rey. A question about consciousness. In Perspectives on mind,pages 5–24. Springer, 1988.

[9] James Newman and Bernard J Baars. A neural attentional model foraccess to consciousness: A global workspace perspective. Concepts inNeuroscience, 4(2):255–290, 1993.

[10] Bernard J Baars. A global workspace theory of conscious experience.Consciousness in philosophy and cognitive neuroscience, pages 149–171, 1994.

[11] Bernard J Baars and James Newman. A neurobiological interpretationof global workspace theory. Consciousness in philosophy and cognitiveneuroscience, pages 211–226, 1994.

[12] Giulio Tononi. An information integration theory of consciousness.BMC neuroscience, 5(1):42, 2004.

[13] Giulio Tononi. Consciousness as integrated information: a provisionalmanifesto. The Biological Bulletin, 215(3):216–242, 2008.

[14] Joseph Levine. Purple haze: The puzzle of consciousness. OxfordUniversity Press, 2001.

[15] Roger Penrose. Mechanisms, microtubules and the mind. Journal ofConsciousness Studies, 1(2):241–249, 1994.

[16] Stuart Hameroff and Roger Penrose. Consciousness in the universe:A review of the ’orch or’ theory. Physics of life reviews, 11(1):39–78,2014.

[17] Keith Campbell. Body and mind. University of Notre Dame Pess, 1992.

[18] Hans Berger. Uber das elektrenkephalogramm des menschen. DMW-Deutsche Medizinische Wochenschrift, 60(51):1947–1949, 1934.

[19] Patrick Krauss, Claus Metzner, Achim Schilling, Konstantin Tziridis,Maximilian Traxdorf, Andreas Wollbrink, Stefan Rampp, Christo Pan-tev, and Holger Schulze. A statistical method for analyzing and com-paring spatiotemporal cortical activation patterns. Scientific reports,8(1):1–9, 2018.

[20] Patrick Krauss, Achim Schilling, Judith Bauer, Konstantin Tziridis,Claus Metzner, Holger Schulze, and Maximilian Traxdorf. Analysisof multichannel eeg patterns during human sleep: a novel approach.Frontiers in human neuroscience, 12:121, 2018.

[21] Donald O Hebb. The organization of behavior. na, 1949.

[22] Michael S Gazzaniga, Joseph E Bogen, and Roger W Sperry. Obser-vations on visual perception after disconnexion of the cerebral hemi-spheres in man. Brain, 88(2):221–236, 1965.

[23] Michael S Gazzaniga. Forty-five years of split-brain research and stillgoing strong. Nature Reviews Neuroscience, 6(8):653–659, 2005.

[24] Lawrence Weiskrantz, Elizabeth K Warrington, MD Sanders, andJ Marshall. Visual capacity in the hemianopic field following a re-stricted occipital ablation. Brain, 97(4):709–728, 1974.

[25] L Weiskrantz and E WARRINGTON. Blindsight-residual vision fol-lowing occipital lesions in man and monkey. Brain Research, 85(1),1975.

[26] Benjamin Libet, Elwood W Wright Jr, and Curtis A Gleason.Preparation-or intention-to-act, in relation to pre-event potentialsrecorded at the vertex. Electroencephalography and clinical Neurophys-iology, 56(4):367–372, 1983.

[27] Benjamin Libet. Unconscious cerebral initiative and the role of con-scious will in voluntary action. Behavioral and brain sciences, 8(4):529–539, 1985.

[28] Melanie Boly, Anil K Seth, Melanie Wilke, Paul Ingmundson, BernardBaars, Steven Laureys, David Edelman, and Naotsugu Tsuchiya. Con-sciousness in humans and non-human animals: recent advances andfuture directions. Frontiers in psychology, 4:625, 2013.

[29] Charles M Gray, Peter Konig, Andreas K Engel, and Wolf Singer.Oscillatory responses in cat visual cortex exhibit inter-columnarsynchronization which reflects global stimulus properties. Nature,338(6213):334–337, 1989.

[30] Andreas K Engel, Peter Konig, Andreas K Kreiter, and Wolf Singer.Interhemispheric synchronization of oscillatory neuronal responses incat visual cortex. Science, pages 1177–1179, 1991.

[31] Wolf Singer. Synchronization of cortical activity and its putative rolein information processing and learning. Annual review of physiology,55(1):349–374, 1993.

[32] Wolf Singer and Charles M Gray. Visual feature integration andthe temporal correlation hypothesis. Annual review of neuroscience,18(1):555–586, 1995.

[33] Francis Crick and Christof Koch. Towards a neurobiological theoryof consciousness. In Seminars in the Neurosciences, volume 2, pages263–275. Saunders Scientific Publications, 1990.

[34] Giulio Tononi and Christof Koch. The neural correlates ofconsciousness-an update. Annals of the New York Academy of Sci-ences, 2008.

[35] Christof Koch, Marcello Massimini, Melanie Boly, and Giulio Tononi.Neural correlates of consciousness: progress and problems. Nature Re-views Neuroscience, 17(5):307, 2016.

[36] Geraint Rees, Gabriel Kreiman, and Christof Koch. Neural correlatesof consciousness in humans. Nature Reviews Neuroscience, 3(4):261–270, 2002.

[37] Melanie Boly, Marcello Massimini, Naotsugu Tsuchiya, Bradley R Pos-tle, Christof Koch, and Giulio Tononi. Are the neural correlates ofconsciousness in the front or in the back of the cerebral cortex? clini-cal and neuroimaging evidence. Journal of Neuroscience, 37(40):9603–9613, 2017.

[38] Francis Crick and Christof Koch. A framework for consciousness. Na-ture neuroscience, 6(2):119–126, 2003.

[39] Silvia Casarotto, Angela Comanducci, Mario Rosanova, SimoneSarasso, Matteo Fecchio, Martino Napolitani, Andrea Pigorini, Ade-nauer G. Casali, Pietro D Trimarchi, Melanie Boly, et al. Stratificationof unresponsive patients by an independently validated index of braincomplexity. Annals of neurology, 80(5):718–729, 2016.

[40] Adenauer G Casali, Olivia Gosseries, Mario Rosanova, Melanie Boly,Simone Sarasso, Karina R Casali, Silvia Casarotto, Marie-AurelieBruno, Steven Laureys, Giulio Tononi, et al. A theoretically basedindex of consciousness independent of sensory processing and behav-ior. Science translational medicine, 5(198):198ra105–198ra105, 2013.

[41] Anil K Seth, Zoltan Dienes, Axel Cleeremans, Morten Overgaard, andLuiz Pessoa. Measuring consciousness: relating behavioural and neuro-physiological approaches. Trends in cognitive sciences, 12(8):314–321,2008.

[42] Giulio Tononi, Melanie Boly, Marcello Massimini, and Christof Koch.Integrated information theory: from consciousness to its physical sub-strate. Nature Reviews Neuroscience, 17(7):450–461, 2016.

[43] Max Tegmark. Consciousness is a state of matter, like a solid or gas.New Scientist, 222(2964):28–31, 2014.

[44] Max Tegmark. Consciousness as a state of matter. Chaos, Solitons &Fractals, 76:238–270, 2015.

[45] Axel Cleeremans. Computational correlates of consciousness. Progressin brain research, 150:81–98, 2005.

[46] Anil Seth. Explanatory correlates of consciousness: theoretical andcomputational challenges. Cognitive Computation, 1(1):50–63, 2009.

[47] James A Reggia, Garrett Katz, and Di-Wei Huang. What are the com-putational correlates of consciousness? Biologically Inspired CognitiveArchitectures, 17:101–113, 2016.

[48] Stephen Grossberg. Towards solving the hard problem of consciousness:The varieties of brain resonances and the conscious experiences thatthey support. Neural Networks, 87:38–95, 2017.

[49] Stuart Hameroff and Roger Penrose. Orchestrated reduction of quan-tum coherence in brain microtubules: A model for consciousness. Math-ematics and computers in simulation, 40(3-4):453–480, 1996.

[50] Stuart Hameroff. Biological feasibility of quantum approaches to con-sciousness. The Physical Nature of Consciousness, Amsterdam: JohnBenjamins, pages 1–61, 2001.

[51] Max Tegmark. Importance of quantum decoherence in brain processes.Physical review E, 61(4):4194, 2000.

[52] Christof Koch and Klaus Hepp. Quantum mechanics in the brain.Nature, 440(7084):611–611, 2006.

[53] Christof Koch and Klaus Hepp. The relation between quantum mechan-ics and higher brain functions: Lessons from quantum computation andneurobiology. Citeseer, 2007.

[54] Bernard J Baars. The global workspace theory of consciousness. TheBlackwell companion to consciousness, pages 236–246, 2007.

[55] Stanislas Dehaene, Michel Kerszberg, and Jean-Pierre Changeux. Aneuronal model of a global workspace in effortful cognitive tasks. Pro-ceedings of the national Academy of Sciences, 95(24):14529–14534,1998.

[56] Stanislas Dehaene and Lionel Naccache. Towards a cognitive neuro-science of consciousness: basic evidence and a workspace framework.Cognition, 79(1-2):1–37, 2001.

[57] Stanislas Dehaene and Jean-Pierre Changeux. Neural mechanisms foraccess to consciousness. The cognitive neurosciences, 3:1145–58, 2004.

[58] Claire Sergent and Stanislas Dehaene. Neural processes underlyingconscious perception: experimental findings and a global neuronalworkspace framework. Journal of Physiology-Paris, 98(4-6):374–384,2004.

[59] Stanislas Dehaene, Jean-Pierre Changeux, and Lionel Naccache. Theglobal neuronal workspace model of conscious access: from neuronalarchitectures to clinical applications. In Characterizing consciousness:From cognition to the clinic?, pages 55–84. Springer, 2011.

[60] Stanislas Dehaene, Lucie Charles, Jean-Remi King, and SebastienMarti. Toward a computational theory of conscious processing. Currentopinion in neurobiology, 25:76–84, 2014.

[61] Stanislas Dehaene, Hakwan Lau, and Sid Kouider. What is conscious-ness, and could machines have it? Science, 358(6362):486–492, 2017.

[62] Antonio R Damasio. The feeling of what happens: Body and emotionin the making of consciousness. Houghton Mifflin Harcourt, 1999.

[63] Antonio Damasio and Kaspar Meyer. Consciousness: An overviewof the phenomenon and of its possible neural basis. The neurology ofconsciousness: Cognitive neuroscience and neuropathology, pages 3–14,2009.

[64] Antonio Damasio. Fundamental feelings. Nature, 413(6858):781–781,2001.

[65] Ron Sun and Stan Franklin. Computational models of consciousness:A taxonomy and some examples., 2007.

[66] Janusz A Starzyk and Dilip K Prasad. Machine consciousness: Acomputational model. Brain-inspired Cognitive Systems (BICS 2010),2010.

[67] Stan Franklin and Art Graesser. A software agent model of conscious-ness. Consciousness and cognition, 8(3):285–301, 1999.

[68] Artemy A Kotov. A computational model of consciousness for artificialemotional agents. Psychology in Russia: state of the aRt, 10(3):57–73,2017.

[69] JH van Hateren. A theory of consciousness: computation, algorithm,and neurobiological realization. Biological cybernetics, 113(4):357–372,2019.

[70] Joshua B Tenenbaum, Charles Kemp, Thomas L Griffiths, and Noah DGoodman. How to grow a mind: Statistics, structure, and abstraction.science, 331(6022):1279–1285, 2011.

[71] Samuel J Gershman, Eric J Horvitz, and Joshua B Tenenbaum. Com-putational rationality: A converging paradigm for intelligence in brains,minds, and machines. Science, 349(6245):273–278, 2015.

[72] Brenden M Lake, Tomer D Ullman, Joshua B Tenenbaum, andSamuel J Gershman. Building machines that learn and think like peo-ple. Behavioral and brain sciences, 40, 2017.

[73] Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B. Tenenbaum,and Jiajun Wu. The neuro-symbolic concept learner: Interpretingscenes, words, and sentences from natural supervision, 2019.

[74] Roger Penrose. Consciousness, the brain, and spacetime geometry: anaddendum: Some new developments on the orch or model for conscious-ness. Annals of the New York Academy of Sciences, 929(1):105–110,2001.

[75] Emma S Brunette, Rory C Flemmer, and Claire L Flemmer. A re-view of artificial intelligence. In 2009 4th International Conference onAutonomous Robots and Agents, pages 385–392. IEEE, 2009.

[76] Christopher M Bishop. Pattern recognition and machine learning.springer, 2006.

[77] Richard S Sutton, Andrew G Barto, et al. Introduction to reinforcementlearning, volume 135. MIT press Cambridge, 1998.

[78] Jurgen Schmidhuber. An on-line algorithm for dynamic reinforcementlearning and planning in reactive environments. In 1990 IJCNN in-ternational joint conference on neural networks, pages 253–258. IEEE,1990.

[79] Jurgen Schmidhuber. A possibility for implementing curiosity and bore-dom in model-building neural controllers. In Proc. of the internationalconference on simulation of adaptive behavior: From animals to ani-mats, pages 222–227, 1991.

[80] Ray J Solomonoff. A formal theory of inductive inference. part i. In-formation and control, 7(1):1–22, 1964.

[81] Phil Maguire, Philippe Moser, and Rebecca Maguire. Understand-ing consciousness as data compression. Journal of Cognitive Science,17(1):63–94, 2016.

[82] Marcus Hutter. Universal artificial intelligence: Sequential decisionsbased on algorithmic probability. Springer Science & Business Media,2004.

[83] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning.nature, 521(7553):436–444, 2015.

[84] Adam H Marblestone, Greg Wayne, and Konrad P Kording. Towardan integration of deep learning and neuroscience. Frontiers in compu-tational neuroscience, 10:94, 2016.

[85] Marcel Van Gerven. Computational foundations of natural intelligence.Frontiers in computational neuroscience, 11:112, 2017.

[86] Demis Hassabis, Dharshan Kumaran, Christopher Summerfield, andMatthew Botvinick. Neuroscience-inspired artificial intelligence. Neu-ron, 95(2):245–258, 2017.

[87] David GT Barrett, Ari S Morcos, and Jakob H Macke. Analyzingbiological and artificial neural networks: challenges with opportunitiesfor synergy? Current opinion in neurobiology, 55:55–64, 2019.

[88] Blake A Richards, Timothy P Lillicrap, Philippe Beaudoin, YoshuaBengio, Rafal Bogacz, Amelia Christensen, Claudia Clopath, Rui PonteCosta, Archy de Berker, Surya Ganguli, et al. A deep learning frame-work for neuroscience. Nature neuroscience, 22(11):1761–1770, 2019.

[89] Neil Savage. Marriage of mind and machine. Nature, pages 15–27,2019.

[90] Anthony M Zador. A critique of pure learning and what artificialneural networks can learn from animal brains. Nature communications,10(1):1–7, 2019.

[91] Andreas K Maier, Christopher Syben, Bernhard Stimpel, Tobias Wurfl,Mathis Hoffmann, Frank Schebesch, Weilin Fu, Leonid Mill, LasseKling, and Silke Christiansen. Learning with known operators reducesmaximum error bounds. Nature machine intelligence, 1(8):373–380,2019.

[92] Kimberly L Stachenfeld, Matthew M Botvinick, and Samuel J Ger-shman. The hippocampus as a predictive map. Nature neuroscience,20(11):1643, 2017.

[93] Samuel J Gershman. The successor representation: its computationallogic and neural substrates. Journal of Neuroscience, 38(33):7193–7200,2018.

[94] Jesse P Geerts, Kimberly L Stachenfeld, and Neil Burgess. Probabilis-tic successor representations with kalman temporal differences. arXivpreprint arXiv:1910.02532, 2019.

[95] Iyad Rahwan, Manuel Cebrian, Nick Obradovich, Josh Bongard, Jean-Francois Bonnefon, Cynthia Breazeal, Jacob W Crandall, Nicholas AChristakis, Iain D Couzin, Matthew O Jackson, et al. Machine be-haviour. Nature, 568(7753):477–486, 2019.

[96] Andreas Maier, Christopher Syben, Tobias Lasser, and Christian Riess.A gentle introduction to deep learning in medical image processing.Zeitschrift fur Medizinische Physik, 29(2):86–101, 2019.

[97] Kyunghyun Cho, Bart Van Merrienboer, Dzmitry Bahdanau, andYoshua Bengio. On the properties of neural machine translation:Encoder-decoder approaches. arXiv preprint arXiv:1409.1259, 2014.

[98] Sepp Hochreiter and Jurgen Schmidhuber. Long short-term memory.Neural computation, 9(8):1735–1780, 1997.

[99] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu,Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, An-dreas K Fidjeland, Georg Ostrovski, et al. Human-level control throughdeep reinforcement learning. Nature, 518(7540):529–533, 2015.

[100] Lukasz Kaiser, Mohammad Babaeizadeh, Piotr Milos, Blazej Osinski,Roy H Campbell, Konrad Czechowski, Dumitru Erhan, Chelsea Finn,Piotr Kozakowski, Sergey Levine, et al. Model-based reinforcementlearning for atari. arXiv preprint arXiv:1903.00374, 2019.

[101] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, LlionJones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attentionis all you need. In Advances in neural information processing systems,pages 5998–6008, 2017.

[102] Evan M Russek, Ida Momennejad, Matthew M Botvinick, Samuel JGershman, and Nathaniel D Daw. Predictive representations can linkmodel-based reinforcement learning to model-free mechanisms. PLoScomputational biology, 13(9):e1005768, 2017.

[103] Olivia Carter, Jakob Hohwy, Jeroen Van Boxtel, Victor Lamme, NedBlock, Christof Koch, Naotsugu Tsuchiya, et al. Conscious machines:Defining questions. Science, 359(6374):400–400, 2018.

[104] Andrea Kubler and Niels Birbaumer. Brain–computer interfacesand communication in paralysis: Extinction of goal directed think-ing in completely paralysed patients? Clinical neurophysiology,119(11):2658–2666, 2008.

[105] Kingson Man and Antonio Damasio. Homeostasis and soft robotics inthe design of feeling machines. Nature Machine Intelligence, 1(10):446–452, 2019.

[106] Jurgen Schmidhuber. On learning to think: Algorithmic informationtheory for novel combinations of reinforcement learning controllers andrecurrent neural world models. arXiv preprint arXiv:1511.09249, 2015.

[107] Juergen Schmidhuber. One big net for everything. arXiv preprintarXiv:1802.08864, 2018.

[108] Anthony Kenny. The homunculus fallacy. In Investigating psychology,pages 169–179. Routledge, 2016.

[109] David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Lau-rent Sifre, George Van Den Driessche, Julian Schrittwieser, IoannisAntonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mastering

the game of go with deep neural networks and tree search. nature,529(7587):484, 2016.

[110] INTELLIGENCE BY AM TURING. Computing machinery andintelligence-am turing. Mind, 59(236):433, 1950.

[111] Robert M French. Moving beyond the turing test. Communications ofthe ACM, 55(12):74–77, 2012.

[112] Moshe Y Vardi. Would turing have passed the turing test? Commu-nications of the ACM, 57(9):5–5, 2014.

[113] Roger Clarke. Asimov’s laws of robotics: implications for informationtechnology-part i. Computer, 26(12):53–61, 1993.

[114] Anna Jobin, Marcello Ienca, and Effy Vayena. The global landscape ofai ethics guidelines. Nature Machine Intelligence, 1(9):389–399, 2019.