Co-evolutionary scenarios for creative prototyping of future robot systems for civil protection

Technological Forecasting & Social Change xxx (2013) xxx–xxx

TFS-17799; No of Pages 8

Contents lists available at ScienceDirect

Technological Forecasting & Social Change

Co-evolutionary scenarios for creative prototyping of futurerobot systems for civil protection

Henrik Carlsen a, Linda Johansson b, Per Wikman-Svahn a,⁎, Karl Henrik Dreborg a

a Swedish Defence Research Agency (FOI), SE-164 90 Stockholm, Swedenb Department of Philosophy and the History of Technology, Royal Institute of Technology (KTH), Teknikringen 78, SE-100 44 Stockholm, Sweden

a r t i c l e i n f o

⁎ Corresponding author. Tel.: +46 8 5550 3882; faxE-mail address: [email protected] (P. Wikm

0040-1625/$ – see front matter © 2013 Elsevier Inc. Ahttp://dx.doi.org/10.1016/j.techfore.2013.07.016

Please cite this article as: H. Carlsen, et al.protection, Technol. Forecast. Soc. Change (

a b s t r a c t

Article history:Received 2 November 2012Received in revised form 25 July 2013Accepted 27 July 2013Available online xxxx

Co-evolutionary scenarios are used for creative prototyping with the purpose of assessingpotential implications of future autonomous robot systems on civil protection. The methodologyis based on a co-evolutionary scenario approach and the development of different evolutionarypaths. Opportunities, threats and ethical aspects in connection with the introduction of roboticsin the domestic security and safety sector are identified using an iterative participatoryworkshopmethodology. Three creative prototypes of robotic systems are described: “RoboMall”,“RoboButler” and “SnakeSquad”. The debate in society thatmight follow the introduction of thesethree robot systems and society's response to the experienced ethical problems and opportunitiesare discussed in the context of two scenarios of different future societies.

© 2013 Elsevier Inc. All rights reserved.

Keywords:Co-evolutionaryScenariosAutonomous systemsRobotsSecuritySafety

1. Introduction

Assessing the future impact of emerging and disruptivetechnologies [1,2] present particular challenges. The history oftechnology shows a number of examples of disruptivetechnologies that have developed into critical infrastructures.The most prominent example in recent decades is the Internet.Private aswell as public communications are highly dependenton a well-functioning Internet. For instance, a major part offinancial services are critically dependent on the Internet as area growing number of public services that municipalities andgovernmental agencies are providing to citizens. The EuropeanCommission defines a critical infrastructure as an asset, systemor part thereof located in member states which is essential forthe maintenance of vital societal functions, health, safety,security, economic or social well-being of people” ([3, p. 77],emphasis added).

The concept of “vital societal functions”used by the EuropeanCommission highlights two points, which are important for this

: +46 8 5550 3866.an-Svahn).

ll rights reserved.

, Co-evolutionary scenar2013), http://dx.doi.org

paper. First,what comprises a vital societal function is dependenton technology aswell as the societal context. And second, the setof vital societal functions is constantly developing according totechnological change and societal development. Hence, in orderto support policymakers with identification and characterizationof the risks associated with emerging and potentially disruptivetechnologies, it is of critical importance to deepen the under-standing of the dynamics between technological change, societaldevelopment and vital societal functions.

Technologies and technical artifacts are shaped in a socialcontext and contribute to shaping society. The evolution oftechnology does not take place in a vacuum and can thereforenot be analyzed in isolation. The same goes for society —

technology shapes our society, it influences our norms andethical standards, which in turn shape regulations and legisla-tion. This calls for more sophisticated methods in order toincorporate the co-evolutionary process that always takes placebetween technologies and society.

This paper explores an emerging technology that has thepotential of having major effects on vital societal functions —

autonomous robotic systems. It is difficult to provide a precisedefinition of autonomous systems and the term is often used

ios for creative prototyping of future robot systems for civil/10.1016/j.techfore.2013.07.016

http://dx.doi.org/10.1016/j.techfore.2013.07.016

mailto:[email protected]


http://www.sciencedirect.com/science/journal/00401625


1 An anonymous reviewer pointed out that a very advanced artificialintelligence (AI) might change on its own, and much faster than humansmight change. This is a very intriguing idea (e.g., explored in the“singularity” literature), but it has not been the focus of the present study.

2 H. Carlsen et al. / Technological Forecasting & Social Change xxx (2013) xxx–xxx

rather vaguely, both in philosophy and in the engineeringcommunity [4]. The word “autonomy” originates from theGreek words “autos” (self) and “nomos” (law) and in generalterms autonomous systems are systems that in some sense areindependent of (human) control. A system can be autonomousto a certain degree, a degree determined by the level of humanoversight. The word “control” implies that autonomy is aproperty of the relation between a system and an agent [5, p.136]. The agent can be the designer, the current user or evenprevious users of the system. The relationship between thesedifferent groups of agents highlights one of the key aspectsconcerning the implementation of autonomous systems insociety, namely the issue of human responsibility in designingand using these technologies [6,7].

Autonomous systems can be implemented as either soft-ware systems or as physical artifacts. The issue of responsibilityis important in both cases, but we will here only focus onphysical artifacts with a degree of autonomy — “roboticsystems” or simply “robots”. In contrast to industrial robots –

which has been around for decades and operate in highlycontrolled environments – a large-scale utilization of robots insociety at large is yet to come. It is difficult to construct robotsthat are able to operate in open environments characterized byrapid changes and high uncertainty. However, there are recentadvances, e.g. from biologically inspired robotics [8] that sparksoptimism among both researchers and potential users of thesesystems. We are indeed beginning to see autonomous robotsystems that are intended to be used in society at large. In Japanrobots that can care for the elderly at homeor provide assistanceto the personnel in hospitals are under development. In Finlandrobot pets have recently been introduced for the care of elderly.Rescue services and police forces already use robots whendealing with explosives. Security firms are looking into thepotential of using robots for surveillance. The combination of anincreasing degree of autonomy of the robots and their growingability to intervene in human activities will most likely lead toethical debate.

These systems are bound to develop in co-evolution withsociety not only in the way other emerging technologies might,but some technologies for building autonomous systemsinclude the possibility of learning from humans and itsenvironment (‘learning machines’), and evolve in a way that isin principle unforeseeable. This idea goes back to one of AlanTuring's seminal papers where he poses the question: “CanMachines Think?” [9]. Here Turing proposes to build computerswhich simulate a child's brain rather than an adult's mind andthat this computer, when subjected to appropriate training,would develop into an adult brain. This represents another levelof co-evolution between technical artifacts and society. For a‘non-learning’ artifact the co-evolution takes place when socialhabits change due to the introduction of a new artifact and thatthe next generation of the artifact is further developed inaccordance with the changed habits and perceived futuredemands. For a learning machine, there is also an inner loop ofco-evolution when the artifact of a given generation is evolvingas a result of its training. As a consequence, the utilizationof these learning robots introduces a “man in the middle”regarding responsibilities. In addition to the designer, theproducer and the user of such a robot, it is possible to arguethat previous owners could also been held responsible if therobot causes any problems [10]. This makes the discussions on

Please cite this article as: H. Carlsen, et al., Co-evolutionary scenarprotection, Technol. Forecast. Soc. Change (2013), http://dx.doi.org

agency and responsibility more interesting when it comes toautonomous systems, compared to most other technologies.1

This paper describes a combined co-evolutionary scenariomethodology for creative prototyping and technology as-sessment applied to a case study of autonomous roboticsystems. The methodology used here is built on a frameworkfor assessing disruptive technologies developed by Carlsen etal. [11] and involves a co-evolutionary scenario approach andthe development of different evolutionary paths. The presentstudy included participatory workshops with stakeholdersrepresenting potential end-users of future robot technologyas well as experts on autonomous systems.

The paper is organized as follows. In Section 2, the rationalefor using co-evolutionary scenarios is described. Section 3presents the operationalization of a co-evolutionary methodol-ogy for this case study. In Section 4, the actual prototypes andscenarios we have developed are presented. A concludingdiscussion is provided in Section 5.

2. The Rationale for using co-evolutionary scenarios increative prototyping

In traditional sequential models of innovation, invention –

ideas – precedes innovation, where innovation is when anartifact is introduced into social practice [12]. In suchmodels it isreasonable to draw a sharp line between intentional and actualuse. However, it has been obviousmany times that such amodelis too simple; that innovations actually evolve in a complexinterplay between innovators and users. This has led to a moveaway from sequential models of innovation to so-called“user-centered” models [13]. These models put a specialemphasis on the role played by users in the improvement andmodification of products. There are many historical examples ofwhat can be seen as user-centered innovation. A classical case isthe marketing of early telephone services in the late 1800s,which focused on broadcast news, church services etc. Anotherexample is that themobile phone textmessaging service (SMS),which was initially intended for notifying users of new voicemessages [14]. Perhaps the most prominent example of userstaking part in the development of new technology is theinvention of the World Wide Web. When Tim Berners-Leewrote the first proposal on a system for information sharing atthe European Organization for Nuclear Research (CERN) heemphasized that it is impossible to predict how users will usethe system [15]. A more recent example is the shift from“ordinary” cell phones to “smart” phones, which have resultedin an explosion of use-cases for mobile phones, much enabledby the development of markets for easy installation ofthird-party software [16]. It has been argued that buildingmore open architectures, and hence increasing the possibilitiesfor users to interact and develop new functionality, hugelyincreases the flexibility of technology based systems [17].

These user-centered models give a strong reason forincluding co-evolutionary elements into technology assessmentand to treat the development of technology and social change asa co-evolutionary process.



3H. Carlsen et al. / Technological Forecasting & Social Change xxx (2013) xxx–xxx

A traditional way of exploring emerging technologies andrelated policy issues is technology assessment (TA) [18,19].The aim of a TA is to assess social consequences as well as thebusiness potential of plausible technological developments.The original ambitions of predicting future impacts using TAhas been downplayed and today several scenarios of possibleor plausible developments are often used in TA studies. Theexpected benefit of a TA study has, as a consequence, shiftedfrom sharp policy recommendations towards a broaderunderstanding of the inherent uncertainties and the multi-tude of possible development paths.

However, few TA-studies explore the possible interplaybetween future technological applications and society overtime, i.e. the sometimes lengthy co-evolutionary process that istaking place between societies and technologies or technologicalartifact during a decade ormore. This is, as alreadymentioned, insharp contrast to studies of the historical development of specifictechnical artifacts or technologies, where the co-evolutionbetween technology and society market is often emphasized[13,14,20].

The issue of co-evolution is also a key to understanding therole played by technology (and technical artifacts) in differentschools of economics. Orthodox economic theory (neoclassicaltheory) confines the analysis to allocations of resources andtechnology is usually taken as “given and asocial” [21, p. 14].Technology is considered as being an exogenous variable andbeyond economic analysis. In contrast, institutional or evolu-tionary economics regard technology as part of the economicsystem, and thus something that should be explained byeconomists [22]. In these approaches the structure of theinterplay between technology and social practices is part ofthe economic analysis, and the inseparability of technology andsociety at large is emphasized.

In another tradition, philosophy of technology [23], casescenarios are used to underpin an ethical argument or toillustrate an ethical dilemma that may arise in connection withfuture technologies. These scenarios take neither a broadersocietal context into account, nor the development over time.Case scenarios are often used as a counterexample against aclaim or a theory. When exploring the potential ethical issuessurrounding a future technology – e.g. autonomous systems –

the method seems to be to imagine a future scenario, and thenuse this case scenario and apply ethical theories to it, or point toshortcomings in those theories. Another approach is to simplypinpoint an ethical or philosophical problem that may arise.In the philosophical literature on autonomous systems mostefforts have been made in areas of agency, responsibility, andhow ethical artificial agents should be programmed orrestrained [24,25] Sometimes it is suggested that the ethics oftoday – the view on responsibility for instance – is unable tocope with situations stemming from the advancement of acertain technology. The philosophical approach is effective inillustrating and illuminating an argument, and we think it isuseful for our purpose.

There are a few TA studies that apply an approach wherethe interaction between technological development andsociety is a key issue. Davison et al. [26] investigate the impactof norms and the laws of society on how the technologicalpotential in a technological area is realized. Society is seen asconservative, to which technology has to adapt, so theinfluence goes mainly in one direction. Stemerding et al. [27]


outline a co-evolutionary development between society – or tobe more precise: ethical norms of society – and the technologyof genetic susceptibility testing regarding common diseases.Stemerding et al. argue that new technologies will often createtensions by opening up for new practices that sometimeschallenge prevailing norms and ethical values. This may lead torestrictions being imposed on the technology, or to the changeof norms and values that change as well as adapt under theinfluence of the other (a truly co-evolutionary process). Acurrent example of this kind of tension is the downloading,sharing and streaming of music and videos over the Internetthat challenges the immaterial rights of the composers andmusicians. In another context – nanotechnology – Robinson[28] developed three co-evolutionary scenarios for the devel-opment of nanotechnology and governance arrangements. Thescenarios described various positions and expectation of actorsin the development of nanotechnology and were used as aninput to a multi-stakeholder workshop.

We believe that the lack of a co-evolutionary scenarioapproach is a major reason why predictions on future technol-ogies have failed. But another important feature, not mentionedin the literature, is that when such approaches have been used,there has been a lack of transparency in the set up of theco-evolutionary scenarios. We share the view of Stemerding etal. [27] and Robinson [28] regarding technological developmentas a co-evolutionary process where society (its values, norms,laws and institutions) and the technology influence each otherand change over time. However, their scenario methodology, bywhich the co-evolutionary paths are developed, is not described.We believe that what is lacking is a more systematic scenariogeneration approach which is transparent as to how thescenarios are developed or chosen and what range of variationit aims to cover. It is also important to underline the necessity of adevelopment process of co-evolutionary scenarios in severalsteps, so that iterative development – and assessment – ofprototypes are made explicit. Our participative approachfacilitates a dynamic view on technological development andsocial change. This is of particular interest to e.g. regulatoryauthorities that might be interested in intervening in thetechnological development process and assess the potentialpros and cons of certain regulations.

3. Operationalizing a co-evolutionary approach

The methodology we used is based on a co-evolutionaryscenario approach and the development of different evolution-ary paths. The scenario approachwe use follows in the traditiondeveloped out of the early works of Kahn and Wiener [29] andthe "scenario planning" approach originating from strategicbusiness applications within the Royal Dutch Shell Group[30–33]. In this tradition, scenarios describe the externalenvironment, essentially focusing on factors beyond the controlof the planning body but with a potentially large impact on goalfulfillment. The scenarios can therefore be called contextscenarios in contrast to themore limited and static case scenariosin the philosophy of technology tradition, c.f. discussion inSection 2. The context scenarios illustrate different “develop-ment logics” and describe the dynamics and not just the endstate of a development. The purpose is to widen the perceptionof what can happen and to support robust or adaptive planningstrategies in the face of uncertainty [34]. Another purpose canbe




to increase the perceptiveness of weak signals thatmay indicatemajor changes in the environment.

The approach of this paper utilizes the scenario planningtradition for two reasons: First, scenario planning offers asystematic way of capturing and structuring societal drivers ofimportance to the uptake and ethical aspects of new technicalartifacts. Second, since we are interested in assessing newtechnologies given different societal contexts, the aimof scenarioplanning to span a broad space of scenarios is appropriate.

The approach we follow in this paper is based on anassessment methodology for disruptive technologies describedin Carlsen et al. [11]. Thismethodology is basedon four premises:

1. An assessment methodology for disruptive technologiesshould take several scenarios of society's developmentinto account.

2. An assessment methodology for disruptive technologiesshould explore co-evolutionary paths for society andartifacts based on the technology.

3. Co-evolutionary scenarios to be used in the assessmentmethodology should be relevant from a policy perspective.They should therefore highlight ethically and politicallycontroversial issues and put focus on situations in which apolicy response is required.

4. A process should be designed for the involvement ofrelevant stakeholder groups and experts on the technol-ogy of interest as well as scenario methodology expertise.

In order to effectively assess challenges associated withfuture technologies, we believe it is necessary to be concreteregarding both the actual implementation of the technologyin terms of actual prototypes of artifacts as well as thedomain where these artifacts are put into use. It does notsuffice to frame the problem like: “What are the futurechallenges of autonomous systems?” Concrete – but of coursehypothetical – prototypes of artifacts therefore have to bedefined and the domain has to be determined.

In this case study we focused on the domain of vitalsocietal function within the sector ‘domestic security andsafety’ as defined by the Swedish Civil Contingencies Agency.Examples of services (cf. vital societal functions) in thissector include the rescue services, the police, supply ofenergy and supply of food.

The co-evolutionary methodology we used consisted ofthree main steps:

Step 1 Developing prototype artifactsThe aim of this step was to develop prototypes ofartifacts, based on the existing or expected user needsand applications that are deemed to be challenging interms of opportunities, risks and potential ethicaldilemmas. The goal was to create prototypes consistingof descriptions of robots that could be on the marketwithin a time frame of 10 years. The appropriate timeframe is dependent on the application, includingcharacteristics of the actual branch of technology (e.g.,information and communication technologies vs. heavyinfrastructures such as the rail sector).We arranged a workshop with a mix of participants:robotic experts (robotics researchers and engineersfrom universities and a robot producing firm), potential


end-users from the sector of domestic security andsafety (from private companies and governmentalagencies) and members of the project group. The aimof the workshop was to identify possibilities, vulnera-bilities and ethical aspects related to the developmentand use of autonomous robots within the domesticsecurity and safety sector. The instruction was to focuson potential applications that in the long runmight havean impact on vital societal functions (cf. Section 1 above)within the domestic security and safety sector. In thisfirst step, the defined time frame was 10 years into thefuture, and the workshop participants were asked toconceive potential robots, the market demand, ethicalproblems associated with robot applications and theneed for society to react in one way or another.The ideas for potential robots were grouped into 8clusters. In order to prioritize among these clusters, theparticipants assigned votes to the clusters, based on theparticipants' judgments on the expected impact for thesector of domestic security and safety. The ordered listconsisted of 8 ideas for artifacts of autonomous robots:1) Fire-fighter robots to support human fire-fighters, 2)Safety controller robots, 3) Service and protection robotsin shopping malls, 4) Portable robots that enhancehuman capabilities, 5) Unmanned aerial vehicle (UAV)robotic systems for detection andpursuit of criminals, 6)Fire detection and fire-fighting robots for use at home,7) Life rescuing robots for accidents at sea, and 8) Firedetection and fire-fighting robots for schools.From this list of 8 proposed prototypes, we selectedthree for further development. The prototypes weresubsequently developed more in detail regarding ad-vantages and future potential. The development of theprototypes was informed by a literature review, espe-cially technology roadmapping studies. The result of thecreative prototyping was a description of the function-ality of the prototype robotic systems (see Section 3below).

Step 2 Constructing a hypothetical debate in societyThe next step was to explore a hypothetical debate insociety concerning security issues and ethical problems –already manifested or expected in the near future –

associated with one or the other technological applica-tion of the robotic systems. After constructing the initialprototypes we evaluated advantages and potentials(expected developments) identified in the artifactprototypes. The prototypes formed the basis for outliningan ethical debate about the use of the robots. First,different attitudes and views on robotswere identified aswell as some plausible events triggering the debate. Theevaluation was structured along the themes integrity,rights and conflicting interests, economy and security.

Step 3 Societal reactionIn this step, society's response to the experiencedethical problems and opportunities was discussed inthe context of different future societies. In order tospan a broader range of possible societal develop-ments, we constructed simple scenarios of futuresocieties. The idea was to explore how the differentartifacts could develop in different societal contexts,depending for instance on how producers, the market




and the public respond to the identified ethical andpractical problems.Because we were interested in the acceptance ornon-acceptance of technologies in society, we wantedto see how two opposite societal stances towards noveltechnologies would respond to the prototypes. Therefore,when examining the reaction from society, we looked attwo different types of societies: 1) the technology-skepticsociety, and 2) the technology-positive society. First, wemade a description of these societies. Using these as astarting point, we then imagined different societalreactions, including governmental strategic principles,and the regulations and incentives used to influence theuse of the technical artifacts for each scenario.The next step would be to explore how future artifactdesigners and producers may respond to the regulationsand incentives imposed by the different societies in theprevious step. For example, new or updated artifacts mayappear on the market, while others may disappear orbecome restricted to a certain application. This canbe seenas an iteration of step 1 again, but starting from differentprototype artifacts and a different society. The process canthen continue to another loop, but now with at least twodifferent development paths where new prototypeartifacts are developed in different future societies.The scenario development process therefore leads to atree-like scenario structure, see Fig. 1. The number ofiterations is partly amatter of resources but the innovativespeed of the actual branch is also important. Ideally, theprocess should at least be carried on for two rounds ormore, but if the time frame for each evolutionary step islong, one may have to stop earlier. In this case study, weonly performed one round, and thus did not constructnew, updated prototype artifacts. In the following sectionthe result of this process is described.

4. Vignette: Future robots in the security and safety sectorand societal response to these robots

4.1. Prototypes of robots

4.1.1. The RoboMallRoboMall is a 1.5 meter high robot with a large screen,

wheels and arms. It is designed for surveillance and assistance inshopping malls and parking garages. In some malls each storehas a RoboMall that is provided by the owner of the mall. Inthese cases there is a supervising RoboMall that can control the

Artefacts

Debate in Society 0

Reaction inSociety 2

Reaction inSociety 1

New/updated artefacts New/updated artefacts

Fig. 1. Schematic figure illustrating the branching tree-like structure in theprocess steps.


other RoboMalls. The RoboMall has some ability to follow athief, but since it is not very fast, it cannot catch up with arunning human being. Instead it can contact the supervisingRoboMall and send photographs and motion patterns to otherrobots that can then continue to follow the suspect. TheRoboMall can tell the thief to surrender, and also inform thethief that the police have been notified. This has actually beensuccessful in many cases.

There are also RoboMalls in different places in the mall,and these robots can show you to a certain store, or helpindividuals to find their car in the garage. The individualsimply tells the robot the numbers on the number plate, andthe robot instantly knows where the car is. It can also contactauthorities in case of an emergency. For instance, if there is afire, or if someone is ill.

There are several advantages with the RoboMall. It canassist in things such as finding one's car, finding a certainstore or merchandise. It can also be an important factor indecreasing shoplifting. Future potential developments ofRoboMall-type of robots include personal shopping, arrestingor holding criminals, early detection of fire or other types ofhazards, and helping to evacuate a mall.

4.1.2. The RoboButler: Always at your service in your homeThe RoboButler is a robot for the private mass market; a

butler or nurse in private homes. This is a robot that can assistelderly people at home by carrying out tasks such as turningon the TV or the radio, making coffee and helping a person tothe bathroom. It can understand oral commands. The robotmoves on wheels, has arms and hands and a monitor throughwhich medical care personnel or relatives can watch the flatand communicate with the person(s) living there. The robotalso has the ability to detect smoke and handle small firesand water leakage as well as contacting emergency servicesin situations requiring expertise or larger resources. It canhelp the elderly people get out in cases of emergency.Furthermore, the robot can take photos of intruders and alsospry ink on them for later identification. Another applicationarea is to assist children after school and to let their parents,still at work, get into contact with them.

The RoboButler may give caregivers an extra sense ofsecurity, especially if the robot is equipped with a monitor, sothat one may be able to phone the robot, and see what isgoing on with a child or an elderly person. The robot mayhelp a disabled person to pick up things, water the flowers,give reminders regarding taking one's medicine and wash thedishes or do other household chores. Other potential areasfor this type of robot are e.g. learning and for use inclassrooms. One interesting question regarding the use ofsuch a robot is what happens when the robot is used in otherareas than the home, for which it is designed. We will getback to this in the ethical debate.

4.1.3. The SnakeSquad: robot system for fire-fighter assistanceThe SnakeSquad consists of a swarm of smaller snake-like

robots acting in concert as a network together with one largerrobot with the capability of dragging people out of a dangerousarea. The system is primarily used in the search and rescue ofpeople in large and complex indoor-environments. The reasonfor this is that it is the size of the area that is to be searched thatis the most sensitive parameter for human fire fighters. There



2 An anonymous reviewer pointed out that in a care situation the peoplewho interact with robots should always be given the choice of whether theywant a robot in their lives. The same reviewer also argued that a referendumwould plausibly be needed before law enforcement robots could beintroduced.


are many examples where the rescue service has to deal withlarge areas, e.g. tube stations and underground train stations,warehouses, shopping centers, factories, sports arenas etc. Thesystem is not optimized for use in private homes.

The SnakeSquad has advantages such as saving morepeople, being able to work in more hazardous areas and forlonger periods of time, and minimizing risks for human firefighters. Important features with the system are the ability toclosely monitor the situation and subsequently use thisinformation in the assessment of operations and henceincrease the ability to improve procedures for complicatedrescue operations.

The major advantage of the system is to support a humanbeing fire fighter in searching large areas by enhancing thehuman's navigation. This is the main task of the swarm ofsmaller snake robots. The snake-like design enhances the robotsability to fast and efficient locomotion and avoidance ofobstacles. The network of robots brings with it its owncommunication network. All robots are linked to each otherand they constantly communicate information in order tomaximize the system search capability. The “collective status”of the swarm is communicated to both the fire fighter at the spotbut also to a central command station outside the fire. Theswarm could provide the fire fighter and the command centralwith information of the location of human being in the fire. Thefirefighter does not directly give the swarm explicit instructions,and he or she instead indirectly influences the behavior of theswarm by his/her movements.

The system also includes one rescue robot that is capableof carrying people. This robot acts only after the explicit orderof the fire fighter. One typical scenario is that the swarm firstidentifies a human being and then the fire fighter – afterreceiving information from the swarm – moves to thelocation of the human being. The fire fighter then decides ifthis human should be prioritized and, if so, he or she calls inthe rescue robot that quickly enters and transfers the humanto a safe place outside the fire. At this time the swarm hasalready begun the search for other victims and the fire fighterfollows the swarm further into the area.

4.2. Debate in society

A general remark concerning security was that the robotswould open up for a “senseless” surveillance. This wasespecially discussed for the RoboMall and one worry wasthat companies would have too much knowledge about theconsumers. It has been argued that this is not something new(c.f. e.g. the debate over CCTVs), but an objection to that isthat surveillance from a robot, which actually can followindividuals, is much more uncomfortable. Individuals arelikely to feel restricted – perhaps intimidated – and be selfconscious about their actions if a robot would follow themaround when shopping; also, there is a risk that this mightlead to a decrease in shopping. In the case of cameras, there isa possibility of avoiding them in areas of blind spots.

Regarding integrity in private homes, which comes intoplay with the RoboButler, an important issue is the possibilityfor an outsider to spy on people in their homes. For instance,if a couple is using the RoboButler in order to have contactwith children living one week at the other partner, it mightbe possible to program or hack the robot in order to film even


though it looks like it has been switched off. The argumentfor the RoboMall, regarding “there are already CCTVs allover”, does not apply in the home environment. Regardingrobots in hazardous environments – the SnakeSquad – thefire fighters are concerned about other people being able towatch their every move. Before the robots, their environmentwas not transparent at all and the possibility to evaluaterescue operations was very limited.

Concerning rights and conflicting interests one issuewas thepossibility of losing jobs. Also, the status of the fire fightersmight decrease, since robots will perform the most dangeroustasks. It seems clear that it is the men, rather than women, whowill be replaced by robots, since it is men rather than womenwho have beenworkingwith dangerous and heavy tasks. In thecase of the RoboButler, women more than men are applaudingthe emergence of the assistance. This is due to the existingpatterns of social life where women in general do more work athome and also take more responsibility for elderly parents. TheRoboButler might lead to a loss of jobs in the service sector,however.

One of the issues discussed under the security theme wasthe possibility of monitoring elderly in their homes. TheRoboButler could monitor key medical parameters (e.g. bloodpressure, insulin) and it could also alarm in case of a collapse.There is also, however, a possibility of numerous unnecessaryalarms, if a robot misinterprets the situation. The RoboButlermight also be a way to increase social control. This raisesinteresting questions concerning the balance between secu-rity and integrity.

A major issue for debate under the security theme had to dowith viruses and “hi-jacking” of robots, especially in caseswheremany robots are controlled by one robot, or a central system, inamall for instance. Some advocate the need for “closed systems”while others suggest that we simply need to work to make thesystems safer. Objections to that kind of reasoning has to dowith the fact that robots actually move about, and that it wouldbe extremely dangerous if criminals would be able to controlthem. This is one example of the difference between physicalartifacts and software based artifacts.2

Table 1 summarizes the ethical debate that followed fromthe introduction of three artifacts based on autonomoussystems technologies.

4.3. Reaction in two different societies

4.3.1. The skeptical societyThis society is skeptical of technology and wishes to control

it. It is only the government that should be allowedmanufactur-ing robots or companies that are certified by the government.The public is not allowed to own robots. Robots in the homemight be permitted, but they will be owned – and controlled –

by hospitals. You may rent such a robot, but you cannot controlit or add any new programs. The idea is for robots to be usedmainly in hazardous environments, but no SnakeSquads, onlyrobots that carry people out of burning buildings. This strict



Table 1Summary of ethical issues debated after the introduction of the different robots.

Topics for debate~2020+

RoboMall RoboButler SnakeSquad

Integrity Ubiquitous surveillance“Moving CCTV”Follow a suspect

Ubiquitous surveillanceThe home as a safe haven is threatenedActions against housebreakersSocial control

Surveillance of fire fightersHazardous environments; responsibility

Economy Loss of jobsCommercialization

Loss of jobsOnly for the wealthy?

Possibility for women

Rights The right to have a robot vs. the right not to have one Status of fire-fightersSecurity Possibility of hi-jacking Possibility of hi-jacking

For owner and for e.g. a housebreaker


anti-robot atmosphere leads, however, to a blackmarket. Peoplebuild and keep their old robots, and repairs and programs aredeveloped for this black market. The government tries toconvince people to “hand in” their robots without criminalcharges, but this rarely happens. People have already gotten ataste for robots. (This is a signal for conservative government:with a black market, there is less control of what happens).People buy robots abroad and smuggles them over the borders.Research and development takes place almost exclusively forrobots in hazardous environments.

4.3.2. Technology positive societyThis society can be described as a liberal, technology-friendly

societywith very few regulations and laws regarding robots. Theonly way this society exercises influence is with support toresearch in robotics, and in terms of certification, indicating somesafety regulations, which are minor, and has to do withfunctionality. There are almost no laws concerning robots, butinstead a free right to go into almost any kind of contract. Thereare also very few restrictions regarding the robots' behavior.They are, for instance, allowed to hold people (capture a criminalin amall). This society can be considered utilitarian-liberal; sincethe pros outnumber the cons, everything is fine, regardingrobots. Politically, the liberal parties get wind in their sails. Butconservative voices worry about the increased number ofdepressed men, who have been replaced with robots in the oldso called “macho-professions” in hazardous environments.However, this is not something that society feels the need toregulate via legislation. It is a free market, is a common slogan.The snakes are also used by the police— and, by the public. Thishas lead to many accidents and situations that raise ethicalissues.

New robots could then be expected developed in thesetwo different societies, as a response to the societal debate. Ifwe look at the original robots at the beginning of the process— the RoboMall, the RoboButler and SnakeSquad, and putthese through the different societies, new artifacts wouldevolve. For instance, in the “Skeptical society”, the extensiveregulation of the robot market may lead to a black market,and a higher demand for the robots on the black marketrather than the robots provided by the government, leadingto ethical problems of its own.

5. Concluding discussion

One visionary dream of future robotics is that thesemachines will do our work for us, leaving human beings free


to choose what to do in life. This dream is built on twopremises: First, it reflects the idea that humans and machineswill continue being separate entities. Second, the dream alsopresupposes that the human beings will continue to besuperior to the machines and hence governing the machinesrather than the other way around. For the dream to cometrue, both premises need to be fulfilled. However, both ofthese premises can be questioned. In the book Robot: MereMachine to Transcendent Mind, the leading robot scientistHans Moravec argues that machines will surpass theintelligence of humans in a couple of decades [35]. Somescholars in the so-called transhumanism movement arguethat the only way we can survive in this game of life andevolution is to become machines, i.e. rejecting the first of thepremise above. The convergence between humans andmachines will evolve into a new being, the posthuman.

Although this may seem too speculative, some considerscenarios like this as serious threats to our future. In theinfluential essay “Why the future doesn't need us” Bill Joy [36]writes about his transformation frombeing a techno-optimist tobecoming more and more worried by the development andprospects in certain technology areas, especiallywithin genetics,nanotechnology and robotics. Joy argues that we need to limitdevelopment of the technologies that are too dangerous. One ofthe key arguments that lead him to this conclusion is that thesenew technologies are not developed within state controllednational laboratories, which was the case for e.g. nuclearweapons during the cold war. This underscores the importanceof studying the dynamical interplay between the developmentof new technical artifacts and society.

Finally, we note that gradual technological change andhuman adaptation are not always taken for granted. A recentexample is the science fiction television series Real Humans –a Swedish production now sold to more than a dozencountries – where extremely realistic and human-like robots(‘humanoids’) exist in a society very similar to today (almosteverything in the fictional world is like today, except therobots). Although this might be an effective way of illustrat-ing possibilities with regard to robot technology itself, we donot think this is a realistic way of portraying how a futurerobotic society will look like. Instead, technological artifactswill interact and co-develop with future society and nottoday's society. It is difficult to operationalize this into anaccessible methodology for technology assessment. Howev-er, we believe that the approach we have used in this paperrepresents a step towards a methodology for the study of theco-evolution of technological and societal change.




Acknowledgments

We thank theworkshop participants thatwere indispensablefor the case study and two anonymous reviewers for valuablesuggestions to improve the manuscript. We acknowledge theSwedish Civil Contingencies Agency (MSB) for research fundsthat made this project possible.

References

[1] C. Christensen, The Innovator's Dilemma: When New TechnologiesCause Great Firms to Fail, Harvard Business School Press, Boston, MA,1997.

[2] C. Christensen, M. Raynor, The Innovator's Solution: Creating andSustaining Successful Growth, Harvard Business School Press, Boston,MA, 2003.

[3] European Commission, Council Directive 2008/114/EC, On the identi-fication and designation of European critical infrastructures and theassessment of the need to improve their protection, Off. J. Eur. UnionL345 (December 8 2008) 75–82.

[4] C.T.A. Schmidt, F. Kraemer, Robots, Dennett and the autonomous: aterminological investigation, Mind and Mach 16 (2006) 73–80.

[5] R. Pfeifer, C. Scheier, Understanding Intelligence, MIT Press, Cambridge,MA, 2001.

[6] A. Matthias, The responsibility gap: ascribing responsibility for theactions of learning automata, Ethics Inf. Technol. 6 (2004) 175–183.

[7] P. Asaro, Robots and responsibility from a legal perspective, Proceed-ings of the IEEE Conference on Robotics and Automation, Workshop ofRoboethics, Rome, April 14, 2007.

[8] R. Pfeifer, M. Lungarella, F. Iida, Self-organization, embodiment, andbiologically inspired robotics, Science 318 (2007) 1088–1093.

[9] A. Turing, Computing machinery and intelligence, Mind 59 (1950)433–460.

[10] L. Johansson, The functional morality of robots, Int. J. Technoetic 1(2010) 65–73.

[11] H. Carlsen, K.H. Dreborg, M. Godman, S.O. Hansson, L. Johansson, P.Wikman-Svahn, Assessing socially disruptive technological change,Technol. Soc. 32 (2010) 209–218.

[12] J. Schumpeter, Capitalism, Socialism and Democracy, Third ed. Harperand Row, New York, 1975.

[13] E. von Hippel, The dominant role of the users in the scientificinstruments innovation process, Res. Policy 5 (1976) 212–239.

[14] I. Toumi, Networks of Innovation. Change and Meaning in the Age ofthe Internet, Oxford University Press, New York, 2002.

[15] T. Berners-Lee, M. Fischetti, Weaving the Web: The Original Design andthe Ultimate Destiny of the World Wide Web by Its Inventors, HarperCollins, San Francisco, 1999.

[16] A. Tiwana, B. Konsynski, A. Bush, Platform evolution: coevolution ofplatform architecture, governance, and environmental dynamics, Inf.Syst. Res. 21 (2010) 675–687.

[17] M. Schilling, Toward a general modular systems theory and itsapplication to interfirm product modularity, Acad. Manag. Rev. 25(2000) 312–334.

[18] G. Ropohl, Ethik und Technikbewertung, Suhrkamp Verlag, Frankfurtam Main, 1996.

[19] T.A. Tran, T. Daim, A taxonomic review of methods and tools applied intechnology assessment, Technol. Forecast. Soc. Change 75 (2008)1396–1405.

[20] F. Geels, Co-evolution of technology and society: the transition in watersupply and personal hygiene in the Netherlands (1850–1930) — a casestudy in multi-level perspective, Technol. Soc. 27 (2005) 363–397.


[21] G. Hodgson, Economics and Institutions: A Manifesto for a ModernInstitutional Economics, Polity Press, 1988.

[22] D. North, Structure and Change in Economic History, Norton, New York,1981.

[23] In: A. Meijers (Ed.), Philosophy of Technology and EngineeringSciences, North Holland, Amsterdam, 2009.

[24] R. Tonkens, A challenge for machine ethics, Minds and Mach 19 (2009)421–438.

[25] S.L. Anderson, Asimov's three laws of robotics and machine metaethics,AI & Soc. 22 (2008) 477–493.

[26] C. Davison, S. Macintyre, G.D. Smith, The potential social impact ofpredictive genetic testing for susceptibility to common chronicdiseases: a review and proposed research agenda, Sociol. Health Ill.16 (1994) 340–371.

[27] D. Stemerding, T. Swierstra, M. Boenink, Exploring the interactionbetween technology and morality in the field of genetic susceptibilitytesting: a scenario study, Futures 42 (2010) 1133–1145.

[28] D.K.R. Robinson, Co-evolutionary scenarios: an application to prospec-tive futures of the responsible development of nanotechnology,Technol. Forecast. Soc. Change 76 (2009) 1222–1239.

[29] H. Kahn, A.J. Wiener, The Year 2000, MacMillan, New York, 1967.[30] P. Wack, Scenarios: shooting the rapids. Part 2 of the gentle art of

reperceiving, Harv. Bus. Rev. 63 (1985) 139–150.[31] K. van der Heiden, The Art of Strategic Conversation, Wiley, Chichester,

1996.[32] G. Ringland, Scenario Planning: Managing for the Future, 2nd ed. Wiley,

Chichester, 2006.[33] R. Bradfield, G. Wright, G. Burt, G. Cairns, K. van der Heijden, The

origins and evolution of scenario techniques in long range businessplanning, Futures 37 (2005) 795–812.

[34] E.A. Eriksson, K.M. Weber Adaptive, Foresight. Navigating the complexlandscape of policy strategies, Technol. Forecast. Soc. Change 75 (2008)462–482.

[35] H. Moravec, Robot: Mere Machine to Transcendent Mind, OxfordUniversity Press, New York, 2000.

[36] B. Joy, Why the Future Doesn't Need Us, Wired 8.04, 2000.

Dr. Henrik Carlsen is a deputy research director at the Division of DefenceAnalysis at the Swedish Defence Research Agency (FOI). His researchinterests include forecasting methodologies, decision analysis, and emergenttechnologies, especially in relation to critical functions of society and climatechange. He has extensive experience in research and consultancy projects intechnology policy and climate change adaptation.

Linda Johansson is finishing her Ph.D. in Philosophy at the Department ofPhilosophy and the History of Technology, the Royal Institute of Technology(KTH). Her doctoral project has included work in research project CopeTech,funded by the Swedish Civil Contingencies Agency. Her research interestsare applied ethics, philosophy of mind, epistemology, and philosophy ofscience.

Dr. Per Wikman-Svahn is a philosopher employed at the Division ofDefence Analysis at the Swedish Defence Research Agency (FOI). He haspublished papers on ethical aspects of risk management, disruptivetechnological change, and strategies for managing uncertainty in futureclimate change.

Dr. Karl Henrik Dreborg is a former deputy research director at the SwedishDefence Research Agency (FOI). He has extensive experience in consulting andresearch in the field of strategic planning, especially scenario methodologies asa tool for coping with structural uncertainty. He has also contributed tothe development and use of participative processes and methodologies inplanning.


http://refhub.elsevier.com/S0040-1625(13)00168-6/rf0005
























































































Documents

Co-evolutionary scenarios for creative prototyping of future robot systems for civil protection