The Future Needs Us

Ian MILESPREST, Institute of Innovation Research

University of Manchester;Manchester M13 9PL; United Kingdom

Abstract. This paper is stimulated by the debate around Bill Joy’s "Why the Future Doesn't Need Us". It argues that the future needs us precisely because it is we who create the future. But the issues that Joy raises, concerning the challenges posed by accumulating knowledge and access to information, and the new forms that some of this knowledge and our associated tools are taking, are very serious ones. The question thus had to be one of how we can shape the future, given that control of it is not possible. The paper proposes that Foresight activities should be oriented so as to get a better grasp on these matters.

Introduction

Bill Joy, co-founder of and then chief scientist at Sun Microsystems, provoked a lively and ongoing debate when he published an article in the April 2000 issue of Wired.1 The title, "Why the Future Doesn't Need Us", came from the magazine editors, but succinctly points to the main arguments. Joy is concerned that current developments in science and technology (S&T) are opening the way to a threatening future.

There is concern among the scientific community about some implications of new S&T. But the running seems largely to be made around the topic of reproductive cloning, very much a side issue in terms of Joy’s analysis. In 2003 67 science academies from the InterAcademy Panel established a united front to call for a worldwide ban on human reproductive cloning at a UN committee on cloning. (Therapeutic cloning to obtain embryonic stem cells would be left to individual countries to determine according to their own circumstances, according to this standpoint.)2 While there are ethical issues about human cloning (and about human reproduction more generally, not to mention augmentation) it is striking how this topic has been able to grab worldwide attention and provide a focal point for action. Meanwhile, Joy has pointed to issues which, if his analysis is even partly correct, are of far more significance.

1. Contradictions of the Knowledge-Based Society

1 Issue 8.04, April 2000, available at http://www.wired.com/wired/archive/8.04/joy_pr.html For some of the subsequent reactions, see http://www.wired.com/wired/archive/8.07/rants_pr.html 2 Pressure is continuing in 2004, in advance of the UN Sixth Committee’s vote on the issue scheduled for October this year. The statement can be downloaded from:http://www4.nationalacademies.org/iap/iaphome.nsf/weblinks/WWWW-5RHFLT?OpenDocument.CF. also the press release, courtesy of The Royal Society (UK): http://www.royalsoc.ac.uk/templates/press/showpresspage.cfm?file=550.txt

1

In his original article, Bill Joy wrote:“…the most compelling 21st-century technologies - robotics, genetic engineering, and nanotechnology - pose a different threat than the technologies that have come before. Specifically, robots, engineered organisms, and nanobots share a dangerous amplifying factor: They can self-replicate. A bomb is blown up only once - but one bot can become many, and quickly get out of control.

Much of my work over the past 25 years has been on computer networking, where the sending and receiving of messages creates the opportunity for out-of-control replication. But while replication in a computer or a computer network can be a nuisance, at worst it disables a machine or takes down a network or network service. Uncontrolled self replication in these newer technologies runs a much greater risk: a risk of substantial damage in the physical world.

Each of these technologies also offers untold promise: …. Yet, with each of these technologies, a sequence of small, individually sensible advances leads to an accumulation of great power and, concomitantly, great danger.

What was different in the 20th century? Certainly, the technologies underlying the weapons of mass destruction (WMD) - nuclear, biological, and chemical (NBC) - were powerful, and the weapons an enormous threat. But building nuclear weapons required, at least for a time, access to both rare - indeed, effectively unavailable - raw materials and highly protected information; biological and chemical weapons programs also tended to require large-scale activities.

The 21st-century technologies - genetics, nanotechnology, and robotics (GNR) - are so powerful that they can spawn whole new classes of accidents and abuses. Most dangerously, for the first time, these accidents and abuses are widelywithin the reach of individuals or small groups. They will not require large facilities or rare raw materials. Knowledge alone will enable the use of them. Thus we have the possibility not just of weapons of mass destruction but of knowledge-enabled mass destruction (KMD), this destructiveness hugely amplified by the power of self-replication….”

1.1 The Three Technologies

Joy points to three broad categories of technology, linking them by the feature of self-replication. However, there are other reasons for paying particular attention to these categories. In particular, each has characteristics that fit definition of a revolutionary technology.3 Technology is about knowing how to transform things, whether these things are symbols, organisms and their products, or materials and artefacts constructed from them. Technological revolutions are associated with major steps forward in our ability to transform fundamental features of our world. New understanding permits all sorts of innovative application of transformative potential – there are a host of new products and processes generated around the new understanding. Our methods of transformation are themselves transformed. Looking back, of course, humanity has had technologies that deal with information, biological processes, and materials into our prehistory. Information technologies have a long history (smoke signals, rock carvings, early symbols communicating the quality and quantity of the contents of containers). Biological processes have been submitted to technological intervention at least since the

3 For a discussion of technological revolutions see C Freeman & F Louçã, 2001, As Time Goes By - From the Industrial Revolutions to the Information Revolution, Oxford: Oxford University Press

2

agricultural revolution in prehistory, and the domestication of the first animals. Materials have been shaped since the Stone Age, and new materials created in the form of pottery and worked metals in subsequent “ages”.

Figure 1 sets out a very broad-brush picture of developments in the three areas of technology associated with earlier industrial revolutions of the nineteenth and twentieth centuries, and the developments characteristic of the current period. This schematic picture is incomplete (notably, we are not including energy technologies, for instance, though past technological revolutions have very much featured these). But it should draw attention to the key features distinguishing the three different epochs. In particular, we see that the important development in modern technologies is the ability to control phenomena in great detail – at microscopic levels. New methods of instrumentation and manipulation have allowed for increasingly fine control of data, organisms and materials. These micro-level achievements have huge consequences at the macro-level of the products that result, as the microscopic properties mean qualitatively different features of the systems to which they contribute.

We seem to be well advanced in the IT revolution, though the pace of change of underlying technologies continues at a rapid rate, while industrial restructuring so as to take advantage of the new possibilities is often slow. Genomics has made great strides and is beginning to make its presence felt in agriculture, health and other application areas. In each of these cases there have been many false starts and much premature optimism that difficult problems would be easily overcome. The early 1980s saw the announcement of the “fourth generation” programme in IT, for instance, many of whose ambitious objectives have yet to be realised. The early promise of gene therapy has run up against unforeseen obstacles, and a far more complex set of interventions is now seen as required if this is to be realised. Whereas new IT has so far run into little social resistance – though this may change with new personal identification and location systems4 – genetic modification of crops has been politically controversial (and we have already noted the alarm about reproductive cloning). Nanotechnology is still emerging as a field, with no obvious core technology to underpin developments, and much debate about the feasibility of some of the more ambitious claims made about atomic-level manipulation and nanorobotics. Nano-products are available, however, and there has been some discussion about political resistance and the need for regulations engendered by, for example, the use of nano-particles whose biological effects are largely unknown.5

There are many areas where advancing technological capacities have security and other risks associated with them. Old-fashioned chemicals and contemporary, though decaying, nuclear waste sites are examples. 6 But revolutionary technologies have several features that make them important areas to examine. They offer a wide range of applications, often with poorly-understood features and long learning processes concerning methods of implementation, complementary technologies and practices, and the like. They display rapid progress and a “swarming” of scientific and entrepreneurial attention. The hyperbole around them attracts the interest of all sorts of agent. Joy’s focus on these three technologies is another reflection of this, but is consonant with the importance of these areas.

4 See our forthcoming report Locating the Personal soon to be available from the PREST website.5 Cf. the Royal Society report, Nanoscience and nanotechnologies: opportunities and uncertainties (2004) at http://www.nanotec.org.uk/finalReport.htm 6 Just what one lad could accomplish with nuclear physics is described in Ken Silverstein, 2004. The Radioactive Boy Scout London, Fourth Estate

3

Figure 1 Three Transformative Technologies Compared 7

Era Information Technology

Biological Technology

Materials Technology

Traditional technology

Knowledge base: empirical experience,

crafts.

Data stored and transmitted in form of signs embodied in materials, and crafted there by human skills.

Harnessing of natural processes of cultivation and fermentation (agriculture, brewing, baking, etc.)

Shaping and casting of part-processed raw materials: early ceramics, metals, alloys.

Industrial era

Knowledge base: applied science; and empirical experience

from industrial processes, production

engineering, etc.

Data stored and transmitted in analogue form – especially using electricity and electronics.(Key science: electrophysics)

“Industrial” fermentation using enzymes and microorganisms.(Key science: biology)

Industrial transformation of materials (esp. steel); later, new materials (e.g. plastics).(Key science: chemistry)

Emerging technologies – Knowledge-based era

Knowledge base: strategic science;

empirical experience from new industrial

processes.

New Information Technology (IT, ICT) – microelectronics etc. Data manipulated in digital form using microelectronics, optronics, and associated software.(Key sciences: physics, computer science)

New biotechnology – genomics and post-genomics.Genetic and microbiological techniques applied to microscopic engineering of living material.(Key science: molecular biology)

NanotechnologyMicroscopic control of structure of materials.(Key sciences: physics/ chemistry)

7 This approach was initially inspired by one developed in P Cohendet, M J Ledoux and E Zuskovitch, 1991 “The Evolution of New Materials: a new dynamic for growth” in Technology and Productivity: the challenge for economic policy Paris, OECD.

4

1.2 The Causes for Concern

There are several distinct categories of concern here, some more immediate than others, some more related to specific technologies than others. We can frame them as follows:

Increasing power to shape the world means that accidents and unexpected consequences can have increasingly wide-ranging and profound impacts. (A corollary argument could be that increased complexity of the systems we are seeking to construct or manage means that accidents and unexpected consequences are more likely. Our increased knowledge of how to transform the world exposes lacunae in our knowledge about the interactions between various transformations, for example.) Note that such circumstances do not necessarily involve the newer technologies – nuclear power plants and proposed macroengineering projects intended to avert climate change8 are other examples that have attracted some discussion.

Many complex or large technologies require considerable effort and investment to develop. But they may be subject to sabotage or “dual use” by malicious individuals or groups with much lower levels of organization and knowledge. 9/11 was a graphic illustration, but various attacks on physical and information infrastructures make the same point. To date the tendency has been more to disrupt systems (computer viruses, sabotage of GM crops) that to apply them as destructive tools (though computer fraud is about controlling assets in the real world). Controlling systems for destructive purposes typically requires more knowledge than simple disruption.

The tools and knowledge required to construct or capture increasingly powerful products are becoming more widely available. (In part this is related to advances in Information technology (IT) that permit wider diffusion of information and that make instrumentation much more effective for a wide range of physical and chemical activities.) We are already in a period where more than basic knowledge of programming, gene-splicing, etc. is in the hands of some teenagers.

Joy also argues that there is a specific issue raised by the emergence of self-replicating products. Computer viruses have been our main experience of maliciously created products to date. But we have also a long history of breeding of plants and animals for specific characteristics that may not be universally welcome, and biosecurity concerns have long focused on the threat of non-native species to indigenous crops and wildlife. The new bioscience enables direct modification of genetic structures, allowing for changes in the functioning of organisms (a worrying case was the creation of a far more virulent form of mousepox than would exist in nature) and the construction of organisms from the bottom up – in which context fears of “biohacking” have been raised.9 There is debate about the imminence and even feasibility of nanotechnological assemblers. The operation of evolutionary principles may mean than self-replicating organisms can evolve in unanticipated ways, of course; genetic algorithms suggest how this can happen in software too. Artificial intelligence is a subject of considerable debate in the IT community, with many commentators arguing that serious AI capabilities will be with us in coming decades. The spectre is raised of human-AI (or robot) competition.

8 See http://www.tyndall.ac.uk/events/past_events/cmi.shtml for useful discussions.9 See The First International Meeting on Synthetic Biology (at http://web.mit.edu/synbio/release/conference/sbconf_overview.pdf ) and in particular the presentation by George Poste entitled “Synthetic Biology: Charting Rational Public Policies For the Oversight and Regulation of Vanguard Technologies” at http://web.mit.edu/synbio/release/conference/talks/George.Poste.SB1.0.pdf For press coverage and the notion of biohacking, see “Experts worry that synthetic biology may spawn biohackers” by Chappell Brown in EE Times June 29, 2004 (1:00 PM EDT) at http://www.eet.com/at/news/showArticle.jhtml?articleID=22102744

5

These are a compelling range of concerns. Even if some are substantially longer-term than others, the implications for security are so profound that Joy’s intervention should stimulate wide-ranging discussion about methods of responding. At the very least, we can see that one feature of the knowledge-based society is – or is potentially –much greater access to, and use of, knowledge with serious security dimensions.

2. Shaping the Future

2.1 Beyond Optimism and Pessimism

Joy, in his original piece, leaned toward the suggestion that research on various topics should be stopped, or at least slowed. Others have proposed that strong restrictions be placed on the diffusion of information about potentially dangerous areas of development.

The UN seems to be slowly being mobilized around human reproductive cloning – though it is an open question whether restrictions promulgated by the UN will carry influence over all member states, and how far abuse might be monitored. It appears to be in principle possible to halt – or more likely, slow – work in specific areas of application of the new bioscience. But where more fundamental research is involved, without conjuring up the knee-jerk responses associated with human cloning, matters are even more complicated. The social and economic benefits associated with applications of the three technology areas are immense, and mustering resolve to reduce advance on these frontiers would require a revolution in social and political attitudes. It is worth exploring what such a revolution might be, but such a future does not seem very likely. We might hope for restrictions on certain types of practice in research and applications – for instance, introduction of a category of nanohazards and rules about disposal of nanoparticulates (which seem to fall well outside of radiological and biohazard rules).

Could these restrictions apply to the dissemination of information? And if so, what sorts of information – intelligence about possible hazards, or more practical information about how to create your own hazards? Despite the tales of conspiracy theorists, it seems to be rather hard to keep S&T information secret for long, and in fields where scientists are competing to make advances, and commercial and other bodies are looking to exploit them, it does not seem likely that information could be restricted for long – though it will be informative to debate this matter further. There might be restrictions on, or surveillance of, the use of particular types of equipment and raw materials. Again, this is liable to be a short-term stopgap, as ongoing advance allows for new approaches to the same problems.

Since the challenges are ones that affect our future, perhaps part of the solution – or of the path toward solutions – is a matter of employing techniques for thinking and acting about the future. Here we can point to the scope for using Foresight methodologies – not the approaches of the nanotechnology-oriented Foresight Institute, but tools and approaches that have been developed in the context of other S&T policy issues.

Across Europe, and more widely, Technology Foresight programmes have multiplied. These (typically national) programmes differ from much other technology-oriented futures work in several ways. Typically they involve a distinctive configuration of elements, though the precise emphases and methods vary considerably.

6

First, they put considerable effort into applying systematic methods to developing longer-term prospectives. Forecasting tools, including elicitation of expert opinion through methods like Delphi and scenario workshops, are employed – but the aim was not specifically to predict the future. The archetypal Technology Foresight programmes examined a wide range of trends and possibilities, with the aim of helping to establish priorities in funding research (and related technology policies, such as training and regulatory development).

Thus, and second, these programmes were policy-oriented – indeed, they were often explicitly intended to inform major policy pronouncements, and timed so as to fit into the rhythms of policymaking. They were sponsored by influential policy actors, rather than being ivory tower or outsider analyses. The Foresight activities brought together key stakeholders, with the futures methodologies providing frameworks that allowed them to communicate and work together with les s tendency to fall into standard positions and speeches.

But, third, they did usually set out to involve outsider views and enlist participation from a much wider pool of knowledgeable stakeholders than “the usual suspects”. This reflects several goals. (1) Enlarging the knowledge base that is drawn upon, in recognition of the point that no single body encompasses all of the knowledge required to understand future opportunities and how to seize them – especially as the world grows more complex (through advances in science and technology, through greater social differentiation, etc.). As well as this technocratic rationale for participation, there is also a democratic rationale, (2) engagement, aimed at enhancing the democratic basis of future visions. This can give Foresight processes and recommendations more legitimacy. A third rationale can also be identified - (3) enlistment, the mobilisation of those involved in the process as actors that can embed the messages of the programme into their own organisations and practices.

The orientation of Technology Foresight programmes has typically been to informing prioritisation processes – where research funds (and related initiatives such as training) should be allocated, for instance. The criteria used for providing such information have typically been those of matching, on the one hand, economic and social benefits (wealth creation and quality of life improvement), and on the other hand the scope for making real progress in (usually national) R&D programmes. Issues of risk and security are relatively less prominent, and while often there is scope for identifying negative economic and quality of life issues, this typically leads to a reduction in the priority associated with the S&T topics, rather than a focus on risks.

However there are exceptions, which indicate that Foresight methods can be brought to bear in analysis of risks and dangers. Consider three examples in the UK.

The Strategic Trends study, undertaken for the UK Ministry of Defence by The Joint Doctrine and Concepts Centre (JDCC)10 is a 'horizon scanning' effort to examine future threats, risks, challenges and opportunities that might be faced by the UK military with a horizon of 2030. It covers seven dimensions, one of which is S&T (other are Physical, Social, Economic, Legal, Political and Military). The work outlines possible directions of future development, risks, and potential “shocks” (wild cards) in each area. After workshops that examined the defence and security implications of trends and drivers for each dimension, detailed research in key areas was commissioned and a “Shock” analysis conducted: material has been synthesised, discussed with other government departments, and made available in hypertext versions on the website and a CD-ROM. While much of the lead in developing futures and Foresight methods came initially from military studies, there has been limited cross-over between military and civilian futures

10 Material derived from JDCC website at: http://www.jdcc-strategictrends.org/index.asp

7

work for most of the past few decades. The public availability of much of the material in Strategic Trends indicates that there can be much richer interaction between civilian and defence foresight – and those risks and dangers can be handled systematically and seriously.

A second example from the UK is within the third round of the Office of Science and Technology’s Foresight programme, which is dedicated to projects that can identify potential opportunities for the economy or society from new science and technology; or consider how future science and technology could address key future challenges for society.11 Unlike the early rounds of UK Foresight, the third round has worked by examining specific subareas of S&T (or areas for application of S&T) and two that have attracted considerable media interest are Flood and coastal defence (results and action plan published April 2004) and Cybertrust and crime (results published June 2004). As with other projects here, each project has a sponsor Minister, lasts between 9-18 months, and works with a mixture of in-house team work and external consultancy (including academics and, in the second area, consultancy firms – in this case RAND Europe). The aim has been to generate high-quality overviews of the issue, and a vision of how the UK could be successful in meeting the challenges that are raised. Each project should have longer-term impacts, establishing networks who will take the recommendations forward. These projects typically involve an initial seminar, numerous workshops (and perhaps conferences, study trips, etc.), commissioned reviews, and use of various “futures” techniques to avoid being locked into extrapolative expectations. (For example, there are very different applications of scenario methods employed in the Flood and coastal defence and the Cybertrust and crime projects.)

The third example can be treated more briefly. The Economic and Social Research Council decided that social and economic research into the field of genomics was essential, given the salience of the field and the high level of controversy around GMO agriculture (and concerns being expressed about the “genetic divide”, etc.). The ensuing programmes/centres of research were informed by a set of futures studies including scenario workshops in which a structured set of scenarios were developed and discussed (these included “business as usual” scenarios, as well as ones where things go much better or worse, and one in which quite different patterns of development are followed), and on this basis a group of social scientists and practitioners sought to identify priorities for social science research in the field.12 Many of these priorities were associated with emerging risks associated with genomics.

2.2 A Foresight Process

We envisage using the tools that have been applied and further developed in Foresight processes to examining the security dangers raised in Joy’s article and the subsequent debates. Whereas a few years ago such a suggestion might have appeared outlandish, the wide use of Foresight methods in S&T decision-making over recent years suggests that there are reserves of expertise and goodwill that can drawn on.

This could involve establishing a framework along the following lines:

11 Quoted from http://www.foresight.gov.uk where there is the following interesting comment: “carrying out science-based futures work is part of the process of embedding quality science in strategic development more generally.”12 Reports available at http://les.man.ac.uk/prest and http://www.altfutures.org. A set of papers on this project is published in the journal Foresight vol 4 no 4 (2004).

8

A Steering Group should be set up, with senior representation from sponsoring and implementing bodies (in government, scientific academies, etc.), and embedding expertise in the conduct and use of Foresight. Such high-level individuals will need to be supported by an effective secretariat. The Steering Group will be responsible for overall planning, synthesis, etc.

Panels will be required to implement Foresight processes and produce analyses in a number of specific areas. These could, for example, be the three technology fields discussed by Joy (together with an “other” category?) The Panels will meet frequently, and organise various study activities, as well as producing reports on their areas and proposing actions that need to be undertaken. Panels can and should play an important role in implementing their recommendations. Panel members must collectively have a wide spectrum of knowledge concerning the technologies and risks (to provide a broader analysis with greater legitimacy and less chance of “capture” by interest groups).They need to be open-minded and creative team workers, who can relate to each other as experts, rather than as interest group representatives. Panels also require support in the sense of secretarial and technical assistance (e.g. for note-taking, preparation of schedules, processing data, preparing materials for presentation, locating sources of evidence and expertise, etc.).

The sorts of tools and techniques we might anticipate the Panels to utilise include:o Conventional studies scoping the topic, based on literature reviews and expert

interviews.o Large-scale elicitation of expert opinion through Delphi or similar methods. In most

Foresight programmes, the focus has been on the date of fruition of areas of S&T, and the scope for social and economic benefits. A simple step would be to consider the areas where various types of security concern might be raised – accidents, sabotage, malicious use of various types and modalities. A more “goals” oriented Delphi approach could be adopted, where the exploration would also focus on various ways in which such threats might be averted or reduced.

o Workshops of various kinds, including scenario workshops which would examine the ways in which different trajectories of development – toward more or less security, for instance – could evolve. These workshops would need to examine possible actions that need to be taken, as well as indicating areas where research is needed, indicators and early warnings of danger, and so on.

o Possibly the use of gaming and simulation tools in which the co evolution of strategies by different parties could be explored. For instance, can methods be used to trace the acquisition of specific instruments or materials – and what efforts would be likely to be employed to avoid or subvert such methods?

o There might well be cross-Panel working groups looking in depth at topics such as scientific codes of conduct.

The Panels and Steering Group will need to report on results, outlining where priorities for action are. This should be undertaken as part of a sustained dialogue aimed at raising public awareness of the issues confronted, trade-offs that may need to be made, and the reliability of various information sources.

The big issue will be to find powerful sponsors prepared to back such a programme. One strategy might be to add such a specific Foresight activity on to an existing or ongoing Foresight programme. In cases where there are disparate Foresight activities underway in a country, rather than a single central programme, it might make sense to appoint a coordinating team to pull out and work on the security risk issues emerging from these heterogeneous activities.

9

3. Conclusions

The gist of the argument in this paper is that if we are to address the dangers of the knowledge-based economy, we will need to make good use of the tools of the knowledge-based economy. Foresight techniques are widely seen as central tools in S&T policy.13 They can be brought to bear on security issues, and the argument is that it is time that they were used to address some of the biggest issues of our time.

The proposal certainly has its difficulties. The politicization of debate over several fields of S&T may make many key parties reluctant to engage in sustained public analysis of risks, for fear of providing more ammunition to campaigning groups that are already a thorn in their flesh. Policymakers are understandably reluctant to be seen to be outlining problems that are on the horizon – especially ones that are technically complex and hard to provide simple sound bites for – while they can be accused of avoiding immediate issues. And possibly we will find that there are security issues that are extremely difficult to address within current political, regulatory, and scientific frameworks.

But are any of these problems worse than those that will follow from an abdication of responsibility for the future? The future needs us, and Foresight may be one way in which we can meet this need.

13 See, for example, A.Tűbke, K. Ducatel, J. Gavigan and P. Moncada-Paternò-Castello (eds), 2002, Strategic Policy Intelligence: current trends, the state of play and perspectives, IPTS Technical Report Series, EUR 20137 EN, IPTS, Seville; and M Keenan, I Miles, Jari Koi-Ova 2003 Handbook of Knowledge Society Foresight European Foundation, Dublin, available at:http://www.eurofound.eu.int/transversal/foresight.htm

10