From Foresight to Impact? The 2030 Future of Work Scenarios

Martin Rhisiart a, Eckhard Störmer b and Cornelia Daheim c

a Faculty of Business and Society, University of South Wales, Llantwit Road, Pontypridd CF37 1DL, Wales, UKb Z_punkt GmbH, Anna-Schneider-Steig 2, 50678 Köln, Germanyc Future Impacts Consulting, Hermülheimer Str. 67, 50969 Köln, Germany

Abstract

Several factors have been identified as important in generating policy-related impact from foresight work, including an effective communications strategy, engagement with relevant stakeholders, creating partnerships, and alignment with the policy-making agenda. The aim of this paper is to explore the use and impact of a scenarios project on the future of work in the UK, undertaken by the UK Commission for Employment and Skills (Future of Work: Jobs and Skills in 2030). The paper presents the results of a post-project study, where data from UKCES has been collected in two rounds in the first two years after the conclusion of the project. The evidence indicates that it has been an impactful project on many levels. Eight factors linked to foresight impact have been identified. The two most critical of these factors are the role of the ‘foresight frontman’, a high-profile and esteemed individual who is able to engage audiences widely and effectively; and a multi-channel communications strategy that encompasses diverse aspects such as professional design, visualisation and social media. Foresight impact is considered as a key design issue for projects ab initio; dynamic and iterative engagement processes support co-production and facilitate impact prospects.

KeywordsForesight; scenarios; impact; engagement

Highlights An adaptable framework for assessing foresight impact is proposed. Eight factors that account for foresight impact are identified. A ‘foresight frontman’ is identified as a key facilitator of impact.

1. Introduction

The topic of foresight impact has attracted increasing attention in recent years. Various studies have identified success factors for government foresight (Calof and Smith, 2010), offered insights on longitudinal case studies (Rhisiart and Jones-Evans, 2015) and evaluated national foresight programmes (Georghiou and Keenan, 2006; Daim et al., 2009). However,

1

there is still a comparative lack of evaluation of foresight exercises (Poteralska and Sacio-Szymańska, 2013) even though it can support learning and improved policy-making (van der Steen and Van der Duin, 2012). The aim of this paper is to explore the use and impact of a foresight scenarios project undertaken on the future of work and skills by the UK Commission for Employment and Skills (UKCES): Future of Work: Jobs and Skills in 2030. The study was delivered for UKCES by a team that included the authors of this paper. The paper is guided by the following questions. What makes a scenarios exercise impactful? How are the results of a scenarios project used? How do organisations track the use and impact of foresight work?

As a topic, ‘the future of work’, has generated volumes of literature over many decades, and has prompted broader societal discussions. The emergence of the information age and computerisation stimulated much debate in previous decades on the nature, organisation and meaning of work.

…it was during the 1970s…that the familiar scenery of our working lives began to show visible changes. The large employment organizations which had been daytime houses for so many people all their lives began to decline…Jobs began to be a scarce commodity, and work started to mean other things besides the conventional full-time job. Second and third careers, moonlighting and the (informal) economy became part of our language as did the chip and the video – all new words to herald new ways. The old patterns were breaking down; new patterns were forming.

(Handy, 1984: ix)

Handy’s references to the breaking down of old patterns and the formation of new patterns are redolent of the Schumpeterian process of creative destruction, in which entrepreneurs seize on the opportunities afforded by technological and other changes to disrupt the status quo. Zuboff’s (1988) reference to the ‘smart machine’ appears prophetic to contemporary society in the 21st century where ‘smart’ is an epithet for ubiquitous and technologically advanced digital devices. The interest in the future of work remains very strong today. From a technology perspective, there is a fascinating and important contemporary debate on the potential role of artificial intelligence, machine learning and robotics in transforming work, economies and societies (Brynjolffson and MacAfee, 2014; Frey and Osborne, 2013; Ross, 2016); automation or computerisation has taken on a new meaning. Even though technology has a major impact in generating change, the future of work is broader than technology. A wider socio-economic and systemic perspective is needed.

The paper will briefly set the context and outline the main steps and processes of the Future of Work 2030 project. Most of the paper is dedicated to exploring how the foresight results have been used following the launch of the main outputs. Post-project interviews have been conducted with UKCES to track the use and impact of the scenarios produced. Two rounds of post-project interviews have been conducted over 18 months since the end of the project. The paper assesses the types of impact that the Future of Work 2030 has had to date – and the factors that are perceived as critical to impact. The assessment indicates that it has been a highly impactful project. Section 2 reviews the literature around impact of scenarios and

2

foresight in a policy-making setting. The main context of the study and its activities are presented in section 3, followed by the methodology used in section 4. The results are presented in section 5, and discussed in section 6. Concluding thoughts are presented in section 7.

2. The use and impact of scenarios in policy making: a review of the literature

Conceptions of the future structure the decision-making processes of the present. The way in which we use the future has a major influence on the possibilities and options that are revealed to us, both inside and outside government. Path dependencies have material effects for decision-making but also – and perhaps more importantly – set the cognitive frames for actors in these environments. Assumptions and norms generated from past events and conditions can have a powerful influence over the type of future that is ‘available’. There has been a trenchant critique of ‘static’ models in the Strategy field (Schoemaker, 1990; Stacey, 1995; 2007; Brown and Eisenhardt, 1997), particularly where organisations have a ‘precommitment’ to a course of action and the main task is optimization (Lane and Maxfield, 1996).

Futures activity within policy-making habitually encounters the constraints that are imposed when there is a premature closing of thinking and choices. Policy-makers need assistance in how they use the future: to emancipate them from probabilistic thinking, and to re-think decision-making in the present (Miller, 2016). In the absence of this creativity, and advanced knowledge, policy-making (unwittingly) reproduces and projects the present into the future, thereby colonising it. It neglects the creativity and potential that is emergent in the present. Policy-making is both situated in and seeks to consciously influence complex socio-economic systems. After all, if it were simple and linear, ‘wicked problems’ wouldn’t be wicked. In many ways, the goal for policy-makers – and those that seek to support policy-makers – is to reflect more faithfully complexity in decision-making, and to forge ways of adapting knowledge from complexity science – including notions of emergence and adaptive capability – into the practice of policy-making. The study of complexity and complex adaptive systems has been an important dimension within futures studies for several decades (Linstone, 2011). The development of the futures field is said to be undergoing changes from ‘forecasting to anticipatory action learning’ and from ‘reductionist to complex’ (Inayatullah, 2002: 295). It is well acknowledged that decision-makers ultimately have to simplify and narrow down to certain choices. However, these choices can be revealed and enriched by adopting an open, anticipatory systems approach (Miller and Poli, 2010) that embraces the creativity and potential of a non-deterministic Universe (Miller, 2016; Miller, 2007).

Scenario exercises may have a range of tangible and intangible benefits, for example in supporting strategic decision-making, setting priorities, challenging assumptions and promoting learning. The most common reason cited for conducting scenarios exercises is to support strategic decision-making (Varum and Melo, 2010). Within policy-making, the decision-making process is informed by the multiple streams of agenda-setting: of ideas,

3

politics and perceived problems (Kingdon, 2002). Assessing the impact of scenarios on decision-making and policy-making is not a simple task considering the multiplicity of factors, layers and processes involved. These include – but are not limited to – individual and collective learning processes, intra-organisational and inter-organisational aspects, and the influence of ideas, politics and perceived problems in agenda-setting. Impact has been cited as one of the three main areas to be assessed in evaluating foresight activities; the other two are quality and success (Van der Steen and van der Duin, 2012).

Scenarios and other foresight methods form part of the strategic palette of tools that governments use in designing and implementing policies and programmes. In some cases, foresight units within Government may coordinate cross-cutting initiatives that involve multiple Departments and partners in the process. As with other areas of policy-related research and analysis, there have been various initiatives to share good practices and case studies, and develop guides for foresight implementation such as the European Foresight Platform1. Some of these initiatives focus on the application of scenario and related techniques (e.g. FOR-LEARN Online foresight guide2). Recently, attention has been increasingly focused on the key ingredients for governments in maximizing the value of foresight work. With sustained interest and cumulative experiences of using foresight in policy-related work, the quest to learn what works in the design and implementation of foresight is a sign both of its maturity and the expectation that applied (futures) research and analysis should be able to demonstrate its value. Concern with delivering impactful foresight work has also started to move upstream in the design and deliberations over foresight projects. Commissioners of scenarios and foresight studies are thinking ex-ante of impact issues. Contractors and researchers are increasingly sensitized to the impact agenda – beyond the technical accomplishment of the project. A recent example of this trend can be found in the decision of one European Union agency (EU OSHA) to commission a (pre-) study to understand success factors for foresight work. This was planned as a precursor to undertaking a major foresight project (Cox et al., 2015).

There are several possible interpretations of what constitutes ‘success’ in policy-foresight work. These include, for example, achieving stated objectives; stimulating collaboration to address a tricky problem; engaging with target groups, and so on. Whilst some are concerned with achieving tangible goals (e.g. development of a new, formal strategy through scenarios), others can be concerned with intangibles (e.g. forming new networks, changing culture etc). At a prosaic level, much of the theoretical and practical literature in this area is directed towards understanding what works, how and why. Policy makers are increasingly looking beyond merely the technical competence and quality of foresight work taken. There are growing expectations that investments in foresight will deliver greater dividends. One national example of this change can be found in the UK. Over the years, the foresight programme has received positive evaluations (e.g. Gheorgiou et al, 2006). More recently, however, one parliamentary enquiry into horizon scanning activity found that the

1 http://www.foresight-platform.eu2 http://forlearn.jrc.ec.europa.eu/guide/0_home/index.htm

4

high quality of foresight programmes did not necessarily translate into a commensurate level of impact (Science and Technology Select Committee, 2014).

Discussions on impact lead us back to the question of the purpose and value of foresight. Haegeman et al (2010) identify six functions of foresight (or future-oriented technology analysis (FTA)) for policy making: informing policy; facilitating policy implementation; embedding participation in policy-making; supporting policy definition; reconfiguring the policy system; and having a symbolic function. Other authors have similarly focused on the key functions of: generating information and aggregate knowledge to inform policy; advising policy through a process of shaping and interpretation; and facilitation of learning and capacity building by bringing together key actors (Havas et al, 2010; Da Costa et al 2008; Eriksson and Weber, 2008).

Johnston (2012) offers a ‘foresight impact schema’ consisting of four elements: raising awareness of the need and value of foresight amongst policy makers to address future challenges; informing – providing inputs – both conceptual and empirical to inform decision-making; enabling – developing the capacity to deal with uncertainty; and influencing – shaping policy and other outputs. Johnston cites the guidance given by the Economic and Social Research Council (UK) on evaluating the impact of research. They point to three types of impact: the instrumental (for example in influencing the development of particular policies); conceptual – the understanding and framing of issues; and capacity, enhancing the individual and collective skills and knowledge to address these issues) (ESRC, 2009). Calof and Smith (2010) suggest eight key success factors for governments in using foresight for policy-related work:

Focus on a clearly identified client. Establish a clear link between foresight and today’s policy agenda. Nurture direct links to senior policy-makers. Create strong public-private partnerships. Develop and employ methodologies and skills that are not always used in other

departments. Ensure a clear communication strategy. Integrate stakeholders into foresight programs. Take advantage of the existence of, or create a national-local academic receptor and

training capacity.

Evaluation and impact of foresight work also link to the question of power interests and constituencies of opinion. Policy foresight work requires an embedded reflexivity and criticality to bring out the voices and views of citizens with alternative perspectives to those of the status quo (Ahlqvist and Rhisiart, 2015). Indeed the socio-technical context of assessing impact has changed, particularly in the shift to more open, participatory and distributed approaches to conducting foresight work. These are driven or enabled by societal demands, social media and other digital technologies. With more distributed, networked modes of conducting foresight work – which might engage with hundreds or thousands of participants – the task of understanding impact becomes more complex. However, the use of these technologies can enable broader participation and enhance the quality of the

5

process and increase transparency in foresight work (Raford, 2015; Keller and Von der Gracht, 2014; Hilbert et al., 2009). It is interesting to juxtapose here the debates on public engagement in the field of responsible research and innovation (RRI), which broadly calls for the co-development of innovation by a range of stakeholders to embed ethics and social responsibility within the process. One of the proposed dimensions of RRI proposed by von Schomberg (2013) is deliberative feedback mechanisms for allowing feedback between stakeholders and policymakers. The orientation of RRI towards technology (and innovation) ‘with’ and ‘for’ rather ‘in’ society (Owen et al., 2012) is an interesting consideration for national foresight programmes.

Much of the foresight evaluation work is undertaken on an ex-post basis – within a relatively short time period after the main activities have been concluded. Many of the examples from foresight evaluation literature tend to fall into this category (e.g. Daim et al, 2009; Georghiou, 1996; van der Meulen et al., 2003; Popper et al., 2010). This has become a pattern in some Government foresight programmes, where evaluations are conducted systematically one year after the project has been concluded. These tend to be formal and programmatic exercises and collect valuable information on the success of projects and the impact they may have had. However, it may take many years to achieve impact; a change in policy or direction often is brought about in the time span of a decade rather than a year (Sabatier and Jenkins-Smith, 1999). This is a challenge for assessing the impact of foresight programmes. Time horizons are clearly important in considering impact and some studies have sought to track longer term impacts of foresight and scenarios projects – over 20+years (Rhisiart and Jones-Evans, 2015). It is one thing to assess the impact of foresight project outputs over the short term. It is another to assess longer term outcomes over a period of several years. Turnover in personnel and changes in institutions are two practical challenges that make the collection of data more problematic. At a deeper level is the attribution of specified factors for particular outcomes. Even where foresight programmes have a direct application (Cuhls, 2015), outcomes over the longer term may be the consequence of multiple streams of activity. With this type of longitudinal approach, a longer term impact perspective may consist more of retrospective sensemaking and developing coherent and credible narratives. Miles (2012) characterizes foresight programmes as ‘extended service encounters’ between foresight practitioners, sponsors and other stakeholders, and makes the case for more dynamic evaluation of these iterative and complex interactions.

Creating a link between foresight and today’s policy agenda (Calof and Smith, 2010) has been identified, as noted above, as a key factor in successful foresight work within government. There are several ways in which foresight results can be brought to the attention of key decision-makers. Within the field of science, technology and innovation(STI) policy, institutions and structures that support high-level policy-making can significantly influence this process. For example, both South Korea and Japan have high-level councils that directly use and absorb results from the foresight work done by national bodies; the transfer of results can be applied directly (direct plans) or indirectly (diffusing knowledge more broadly to inform policy-making) (Cuhls et al, 2015). For STI policy in the European Union, Cuhls et al. (2015) identify four sets of recommendations for transferring foresight

6

results to decision makers: the growing knowledge that exists on foresight processes and results should be diffused through various channels to the policy-making community; embedding foresight work in the policy process through a resourced, centralised hub; connecting the EU foresight national and international communities; and better education and training to support the development of foresight culture. Embedding foresight can be facilitated by ‘Promoters’ – an individual with deep knowledge that is influential and visible in the policy system. At a formal level, the organisational architecture can be more enabling through establishing high-level councils or committees at the national level – as seen in Finland, South Korea and Japan; these can directly channel results into decision-making. Cross-country analysis has been useful in identifying rich insights and practices in countries with significant experience of foresight in government (Kuosa, 2012; Cuhls, 2015).

To summarise, a common set of (1) types of impact, and (2) tools for impact can be identified from the literature. The types are broader and generally softer or intangible, whilst the tools are more practice-oriented and can lend themselves to planning-based activity. For example, a formal, written communications strategy (tool) can be developed to support the type of desired impact. The types identified are: informing decision making with knowledge and insights – instrumental; facilitation of learning amongst networks of individuals and organisations; building capacity to undertake future-oriented work; and raising awareness and enabling reframing – cognitive and conceptual. The specific tools used to support these desired impacts are: communications strategy; engagement with/integration of stakeholders within the foresight process; creating partnerships; and focusing on policy-making agenda – key themes and key decision-makers.

The aim of this paper is to assess the use and impact of foresight scenarios projects after they have been formally concluded – using the 2030 future or work study. This review informs the development of three research questions for the present paper:

How have the Future of Work Study results been used? What has been the impact of the Future of Work study? Which key factors have contributed to success or impact?

Before developing these themes further, the following section provides more information and context on the Future of Work 2030 study.

3. The 2030 Future of Work Project: a brief overview

The UK Commission for Employment and Skills (UKCES – or UK Commission) commissioned a study to explore the future of work and skills, whose main aim was to “enable key groups in the UK labour market to position themselves effectively by building their understanding of key emerging trends and the implications for the future of jobs and skills for the medium to the long term (up to 2030).” The UK Commission is ‘a publicly funded, industry-led organisation that offers guidance on skills and employment issues in the UK’. It has summarised its four main objectives as follows (UKCES, 2014a)

Lead the debate with industry to drive better outcomes for skills, jobs and growth

7

Work with industrial partnerships and wider networks to push forward employer ownership of skills

Test out employer-led innovation to address persistent skills challenges

Help businesses realise the potential of their people

The key activities of UKCES include the provision of labour market intelligence, research, evaluation and policy analysis. It has carried out two main forward-looking research activities: quantitative modelling of changes in the composition of skills across economic sectors and occupational roles (see Wilson et al., 2014 for the most recent iteration) and (qualitative) foresight research.

Here we present the methodological approach for deriving the results from the Future of Work foresight study.

The core questions that the study aimed to answer included:

What are the main assumptions regarding the future global economic development and performance of UK jobs and skills?

What are the potential future disruptions? Which key factors are driving the development of jobs and skills in the UK? Which

uncertainties do they feature? What are possible and plausible pictures of the future of UK jobs and skills, and how

might they develop? What are the scenarios’ implications for the key actors in the field of UK jobs and

skills? What could these actors’ strategies look like? How robust are the strategies when

applied to different futures?

A programme of research was designed and delivered (2013-14), which featured a triangulation of methods: analysis of trends, drivers and disruptions; interviews with senior UK and international leaders; and the development of four scenarios for work and Skills in the UK. Two sets of consultative workshops were held with UKCES staff, and invited experts and stakeholders, to assist in the validation and sensemaking of the emerging results.

Literature Review

A systematic literature review was undertaken of more than 300 publications related to the future of jobs and skills published in the preceding four years. The literature scanning involved academic literature and grey literature. A STEEP framework (social, technological, economic, environmental and political) was adopted to provide a 360° view of work, skills and the labour market. The scan started from a global perspective (e.g. global economic and environmental trends) before zooming into more specific issues at the UK level. All publications were read and evidence extracted on trends and disruptions. The plausibility of trends and disruptions was initially assessed at this stage. For the trends, plausibility was taken to mean perceived certainty about their occurrence. For the disruptions, plausibility was taken to mean that their occurrence is conceivable under certain conditions. From this

8

scanning, a range of potentially significant themes and trends were identified for further exploration and validation with expert interviewees.

Interviews with experts

The emerging themes and trends identified in the literature review were probed in interviews with 24 experts (national and international) from a range of backgrounds (thought leaders, senior business leaders, trade union representatives, education and training providers, representatives from voluntary organisations, and policy makers). The purposeful sampling approach was used to elicit the views of a range of stakeholders and individuals on the future of work. Interviewees are anonymous but Table X provides a breakdown of the profiles of these by category.

Table 1 Expert interviewees

Interviewee category Interviewee profile examples Number of interviewees in category

Global thought leaders – senior experts from outside the UK

University Professors with special knowledge of work and the labour market

3

UK Thought Leaders – experts from the UK

University Professors and Leaders of Research Centres

5

Global Senior Business Leaders

Directors of Multi-National Companies

2

UK Senior Business Leaders Senior Leaders of UK Businesses across different industry sectors

6

UK Trades Unions Leaders of Trades Unions organisations

2

UK Voluntary organisation Leader of national third sector organisation

1

Policy makers (government and agencies)

Leaders of national agencies, local government and central government senior officials

3

Education and training providers

Senior leaders in the education and training field

2

Total 24

Interviewees were sent a 5-page briefing document in advance of the interview. The document outlined definitions of key terms and provided a list of key trends and disruptions, with brief descriptions.

9

The semi-structured interviews, which lasted one hour on average, explored the trends and disruptions identified in the literature review. Interviewees were encouraged to elaborate on the topics that were particularly relevant to them and their stakeholder groups. Interviewees were asked to note any other major themes that were not already identified by the study to date. The interviews enabled the research team to validate the strength and significance of themes, to enrich understanding, and to explore additional perspectives and assumptions.

Compilation of Trends and Disruptions Reports

From the scanning and validation with interviewees, 13 major trends and 10 potential disruptions for the labour market were further developed and published in reports (Störmer et al., 2014a; Störmer et al., 2014b).

Figure 1 Key Trends for Work and Skills

Figure 2 Potential Disruptions for Work and Skills

10

Key factors analysis

An initial list of influencing factors was used to generate a set of the main underlying drivers – key factors – for jobs and skills to 2030. Twelve key drivers were identified. The following criteria were used for selecting the most influential factors: strength of impact, degree of uncertainty and also the degree of influence on other factors. An Uncertainty-Impact-Analysis was undertaken to support this process to evaluate how certain or uncertain is the development of a factor and how strong is its impact on the field under investigation. The method identifies the factors that are neither important nor uncertain. The identified disruptions were used to make valid assumptions on the degree of uncertainty.

Developing projections for key factors

Potential future developments – projections – were developed for the 12 key factors. For each factor, 3 to 4 alternative projections were generated, supported by the insights derived from the literature analysis and expert interviews.

Scenario generation

The first step of the scenario generation method was to undertake consistency analysis of the projections for each factor. ‘Bundling’ projections produces a set of ‘raw’ scenarios. Software-supported consistency analysis was used to identify potential conflicts and synergies between the projections. This analysis identifies consistent and plausible “matches” between projections. A Consistency Analysis assesses the “match” between all projections, based on the question “Could projection X coexist with projection Y?” In this relatively complex process, all pairs of projections of different key factors are given ‘consistency values’ which describe their compatibility3. The result was a list of all possible and plausible combinations of projections.

These raw scenarios were then further developed, with one scenario representing a trend-based “business-as-usual” scenario and three scenarios featuring disruptive developments. The four scenarios were further developed with insights gained during the literature review and expert interviews. In addition, creative vignettes were developed for each scenario to illustrate potential changes and conditions.

Table 2 Four Scenarios for the Future of Work 2030: brief overview

Scenario Brief description and headline featuresForced Flexibility (business-as-usual)

Greater business flexibility and incremental innovation lead to modest growth in the economy - but this flexibility often results in less opportunity and weakened job security for the low skilled

Volatile world economy after emerging from a difficult financial recovery process between 2010s to mid 2020s

3 Parmenides EIDOS® software uses a scale of -3 (highly inconsistent) to 3 (highly consistent), with 0 defining a non-existent relationship between the two projections

11

Higher flexibility of work and employment contracts due to the necessity to adapt to market volatility

Hourglass shaped labour market - increasing competition faced by low skilled workers, and the hollowing out of the middle of the workforce

Security of employment remains highly important for individuals - but work life balance and autonomy are gaining importance

Wide variety of education and training channels to choose from to update skills - for those that can afford it, or are backed by employer financing

The Great Divide Despite robust growth driven by strong high-tech industries, a two-tiered, divided society has emerged, reinforcing the economic position of the ‘haves’ and ‘have nots’

Innovative, high tech companies are the flag bearers of the new economy – and are redefining industrial growth

It is boom time for London and South East England, other areas of the UK are trailing behind

Growing inequality between workers – of earnings and opportunity Technology and liberalisation is opening up a more marketised

approach to learning and skills deliverySkills Activism Technological innovation drives the automation of white-collar work and

brings large-scale job losses and political pressure, leading to an extensive government-led skills programme

A sharp leap forward in IT innovation automates medium and highly skilled (and well-paid) professional work leading to significant disruption to traditional professions

Driven by necessity and government support, an active localism emerges as a response to the ‘IT revolution’

Government proactively provides employment regulation and skills support – with the ‘battered middle’ in mind – but it’s a long, tough road

A project-based economy develops; health and social sector and micropreneurism also offer good opportunities for job creation

For the fortunate people with the right skill set the labour market offers many opportunities

Innovation Adaptation

In a stagnant economy, improved productivity is achieved through a rigorous implementation of Information and Communications Technology (ICT) solutions

Turbulent international environment with faltering trade Domestic blues as the UK and Europe experience a decade of stagnant

or, at best, low growth prior to 2030 As the economy struggles, the pathway to progress is productive

efficiency – through ICT implementation Mobile and virtual workers satisfy employers’ skills needs – at the

right price Significant increase in online education and skills provision

Implications and action needs from the scenarios

Strategic implications were derived for each scenario for the four key stakeholder groups (employers, education and training providers, policy makers, and individuals) and for 7 economic sectors (business and professional services, manufacturing, construction, health and social care, logistics and retail, education and training, and creative and digital). To test and enrich the implications, a one-day conference was held with 29 high-level expert contributors from the UK, representing different stakeholder groups.

12

Launch Conference

The launch conference was attended by approximately 150 participants from business, policy-making organisations, trade unions, and education and training. A Government Minister provided an address at the conference.

4. Methodology

Following the launch of the project in March 2014, two rounds of interviews have been conducted with the commissioning organization, UKCES, to track the use and impact of the scenarios study. The first round of interviews was conducted 8 months after the launch of the final report. The second round was conducted 20 months after the launch of the study i.e. 12 months after the first round of interviews. Interviews were conducted with the research commissioning team at UKCES. In arranging interviews, the aim of tracking and assessing the impact of the study was explained. The UKCES were keen to support the study and gain further insight as to the use of foresight outputs. In preparation for the interviews, the UKCES staff collected further data on the use and impact of the 2030 Future of Work study from colleagues in the organization and from external stakeholders.

A semi-structured interview tool was used for the interviews. Detailed notes were taken by the interviewers and these were shared with the interviewees to ensure that data had been captured accurately.

5. Results

This section presents the results of the two rounds of interviews conducted with the UKCES commissioning team. First, the main uses and impact areas of the study are identified. This includes the reporting of some of the evidence already collected by UKCES staff. Second, the main factors that are attributed with generating those impacts are presented. For both, the data represents the views and perceptions of the staff. In addition to their knowledge and understanding of the organisation’s activities, the staff are embedded within multi-sector networks and communities (policy, business and education and training), which affords a wider perspective on the impact of the scenarios.

5.1 Main impact areas

The 2030 Future of Work Study generated impacts across a wide range of areas. This has been wider than has been the case with other or ‘usual’ reports published by UKCES. The 2030 study is the most downloaded of all UKCES reports, indicating the wider appeal and

13

dissemination of the study. The outputs have been used across a variety and settings and for different audiences.

5.1.1 Policy and Government: it has been used by the cross-Government group on the Future of Work, with the specific focus on development and design of Government Work and services – it has gained a lot of traction in UK Government. This group is being coordinated by the Department for Work and Pensions, working with other Departments such as the Department for Business, Innovation and Skills, and the Cabinet Office. They had already started looking at the topic when UKCES published the Future of Work 2030 study – but the UKCES study has given a lot of impetus to this cross-Government initiative. It is described ‘part of the fundamental knowledge base’ of the cross-Government work. The initiative of the cross-Government group is still ongoing therefore it is not possible to determine at this stage how – and to what extent – the UKCES study will influence its outcomes. Additionally, there has been a high-level briefing of the Prime Minister’s Office on the Future of Work 2030 Study. There has been positive feedback from the Scottish Government, which was interested in undertaking a supplementary future of work analysis for Scotland. An outcome to these deliberations is pending.

Study results were also included in leadership skills for digital technologies initiatives of the EU Commission, as well as into a EU foresight study on future safety and health at work issues. Furthermore, the approach and insights were discussed in the foresight community, e.g. at the EU FTA 2014 conference in Brussels and at the Singapore Risk Assessment and Horizon Scanning conference 2015.

5.1.2 Employers and businesses: in comparison with the situation for Government and policy-makers, the dissemination to employers and businesses has been more limited. There have been requests for briefings and more information from important industries, such as retail and construction. Large, individual employers have responded positively to the study but there is less evidence on wider, direct impact. In particular, HR managers have shown a high level of interest in the study. Results have been communicated in HR and major career trade magazines in UK, Germany and Australia (see also media section below). UKCES staff have been invited as key note speakers at talent /HR manager conferences in UK and Italy. Popular dissemination of the results of the study has been channelled through social media, news and other outlets; it is more difficult to track the impact of this type of exposure to the study’s results. One of the lessons noted for future studies on the future of work was to include a more explicit programme of dissemination and engagement for businesses. This could involve more business partners to endorse the study and engage with employers.

5.1.3 Education and training providers: The Future of Work 2030 study has featured strongly in keynote presentations at important conferences within the sector; examples include the University Alliance Summit 2015, and the Association of Colleges Conference on digital skills (integrating the qualitative foresight results into quantitative labour market intelligence), at a presentation at the “2020 Vision Conference” of head teachers in 2015, and at an adult learning policy initiative.

14

Presentations have been given by UKCES at future of works seminars organised by the Universities of Oxford and of Salford. At least one professor who was involved in the study as an expert included elements of the study in his university teaching.

Generally, there has been more interest from Colleges than from Universities. There seems to be more interest in careers than curriculum development.

5.1.4 Career professionals and institutions: The study was presented at the CDI (Career Development Institution) annual conference and a workshop with the National Careers Service was held as part of the careers advisors continuous professional development opportunities programme. The career perspective of future jobs and skills has been used as a stimulus for career professionals.

5.1.5 Individuals: It’s quite difficult to evaluate the impact on individuals. It has had impact with careers professionals so this may be more indirect, through an approach of cascading information down.

There has been significant engagement through social media and other digital tools, including the Your Future Jobs Quiz. This is a light-hearted but thought-provoking attempt to engage individuals (particularly young people), to illustrate the potential profiles of the jobs of tomorrow, and the skills and knowledge that they might require. Further products have been developed by UKCES to address individuals with foresight job and skills insights – the the “Careers of the Future” brochure in particular used the results of the study to identify the top jobs of the future, mixing qualitative sector foresight insights from the study with quantitative labour market forecasts.

5.1.6 Media coverage: there has generally been a high level of media coverage of the Future of Work 2030 study, particularly during the launch conference. It was featured in news articles within the Financial Times, The Independent and and City AM – amongst others. It was also covered in print and on screen by BBC News. Some of the major trade magazines in the field, including the Chartered Institute for Personnel and Development (CIPD) and HR Magazine discussed the study’s results in their outlets. A review of the media coverage of the study after its launch was collected by UKCES.4

Further evidence of the reach and ‘success’ of the foresight study can be gleaned from the number of downloads of the main reports and page rankings on the main Internet search engines (in the first year after launch, it was the number one result – or amongst the top few results – for searches on ‘the future of work’ on Google).

5.1.7 Internal organizational impact: the study has had some significant positive effects internally within UKCES in developing foresight awareness and capacity around the future of work.

“People have greater confidence in speaking more widely about the future of work. The Summary Report and Slide pack have been widely used across the organisation.

4 https://storify.com/ukces/the-future-of-work-jobs-and-skills-in-2030

15

Within all our main publications – the Future of Work study has been integral and has directly impacted on our thinking, and recommendations that have been made in other reports.

It provides firmer foundations that we can draw on (UKCES Research Commissioning Staff).”

Within UKCES, it has influenced policy and programmes in a number of tangible ways. It has influenced the development of Careers of the Future (UKCES, 2014b) and its review of high level Science Technology Engineering and Maths (STEM) skills requirement in the UK labour market (UKCES, 2015). It also directly informed the Growth through People Report, the assessment of UK skills published by UKCES and co-signed by the Confederation of British Industry (main employers’ association) and the Trades Union Congress (UKCES, 2014c).

5.1.8 Sector skills councils: Sector skills councils have been galvanised by the results and at least one institution, the Construction Industry Training Board, commissioned a sector- specific follow up study (CITB, 2015).

5.1.9 International reach: UKCES has had enquiries and hosted delegations from other countries interested in the Future of Work 2030 study, including South Korea, Philippines, and Estonia. The foresight work was presented at a British Council event with participants from over 80 nations on “Bringing the learning home”. UKCES has been invited to other countries – New Zealand, Singapore, InterAmerican Development Bank in Argentina (upcoming) – to discuss the Future of Work 2030 Study.

16

5.2 Factors that have generated impact

Analysis of the two rounds of post-project interviews with UKCES staff reveals eight factors as critical in helping to generate impacts from the study. These have been discussed with – and validated by – UKCES staff.

5.2.1 Attractiveness and accessibility of outputs – including visuals:

What attracted people to it was the fact that it is not only a standard research or policy report, but also included a 20-page brochure condensing the content to a more digestible level. This was accessible to a broader audience compared with a 200-page research report. The scenarios have been visualized by designers. The visuals are a core element to communicate the pictures of the UK’s future of work and help to convey the essence of the scenario content at a glance. The utilization of visuals had already been adopted in the course of the project as an important communicative approach in the stakeholder workshop. Additionally, the scenarios are described in a very tangible way with scenario vignettes that help to convey potential future work situations in alternative ways.

The format of the result presentations, consisting of a 20- page brochure, a slidepack with the key results (both with engaging content in terms of punchy messages and design) and a detailed evidence based research report, was a key success factor. This allowed the insights to be made available in different depth for researchers and policy advisors, for labour market specialists but also for an interested audience beyond technical labour market specialists.

5.2.2 Quality of the research is a prerequisite: the quality of the research has to be very high, otherwise there is no sustainable impact or interest.

The 2030 study is regarded as comprehensive, with detailed analysis on trends, disruptions and scenarios, including globally relevant aspects. The comprehensive nature of the study is seen as significant – all encompassing, with a global focus, taking account of different trends. The 2030 study offered an evidence-based analysis on the direction of travel. Beyond a detailed analysis of global and national foresight studies and market projections, global experts in the field of the future of work contributed to the study through interviews. The global perspective enriches the national perspective and contributes to the global interdependences of the topic. National specifics are the key focus of the foresight study, including a look at some specific implications for regional development and industry sectors.

The study is seen as offering a ‘sober, balanced view that looks at all relevant issues’ (UKCES Commissioning Staff), not only focusing on, for example, technology developments. Some other reports may be more sensationalist. The Future of Work 2030 study has more ‘credibility because of its scientific respectability’ and balanced view.

5.2.3 Support of the Commissioners: Commissioners are top-level, well known leaders and practitioners that officially represent the multiple stakeholder groups of UKCES. Commissioners were included in the project via interviews, one commissioner (Trade Union

17

leader) hosted the stakeholder workshop and another (business executive) acted as the “figurehead” of the study, with a foreword and the presentation at the launch event. As a pre-requisite, they took a positive view of the study and saw that it made sense from a business perspective. As leading figures in industry, trades unions and other sectors, the Commissioners served as ambassadors and reinforced the prominence and credibility of the study.

5.2.4 Engagement and inclusion of stakeholders in the process: there was a high level of engagement with stakeholders throughout the study through interviews and stakeholder workshops – the latter designed to share and probe emerging results (scenarios and their strategic implications).

“Engaging at an early stage is critical – the interviews had impact. Getting the most out of workshops is tricky – it’s quite challenging to get participants’ heads around content in a short space of time” (UKCES Commissioning Staff).

Interviews and workshops enabled contributions and the integration of key stakeholders in the topic. Not only did this allow the study to access the knowledge and insights of the stakeholders, it also triggered future thinking and offered ‘multipliers’ for the study.

5.2.5 Use of media and social media: the launch of the report attracted a great deal of interest from high profile, mainstream news and media outlets, including the Financial Times, The Independent and BBC News. The study was discussed widely in social media and many commentators and bloggers reviewed and provided their own interpretations of the future of work study5. This included providing commentary and implications on particular sectors and interests. There was similar commentary internationally on the study and its implications for work and employment6. It’s interesting to note the longevity of social media coverage – with interest, links and commentary provided two years after the study was launched7.

For particular groups, UKCES launched specific social media activities like an online quiz for addressing a younger audience.

“The social media campaign and quiz were very successful for reaching younger audience. There has been a lot of traffic to the online quiz” (UKCES Commissioning Staff).

5.2.6 Launch event: the launch event – a conference with around 150 participants – generated a lot of interest for the study. The person who presented the study was one of the UKCES Commissioners. Having a senior figurehead for this role is considered as an important

5 One example, which looks at the implications for rural areas: https://rosiealexander.wordpress.com/2015/02/15/work-in-2030-implications-for-rural-areas/ 6 For example, the Italian business newspaper Il Sole 24 Ore discussed the study on their ‘nòva’ section dedicated to science, technology and innovation: http://adapt.nova100.ilsole24ore.com/2014/11/17/in-viaggio-verso-il-2030-parte-i-nuovi-sviluppi-digitali/ accessed 30 March 20167 Twitter search “ukces future of work”, accessed 30 March 2016

18

factor in the success and impact of the launch event. The launch event attracted many business representatives, politicians, education and training providers as well as the media (together with PR activities). Therefore it was a lever to reach the stakeholder groups and media and thus start the strategic dialogue with these groups.

5.2.7 Active personal contribution to the dissemination: The key lever of keeping the strategic dialogue ongoing is the very personal engagement of the UKCES staff to initiate discussions and present and discuss the foresight insights, and thus trigger an ongoing engagement. With the strong national connections to the stakeholder groups the UKCES has a good starting point to push this discussion – which is at the core of its mission.

5.2.8 Timing

In this case, the cross-Government group on the Future of Work had just started to work when the study was published. Although beyond the control of UKCES, this timing was a key factor in facilitating impact.

6. DiscussionThe evidence gathered in the two years that have elapsed since the launch of the Future of Work 2030 report suggests that the study can justifiably be considered as impactful. These results will be discussed using the desired impact types (informing decision-making, facilitation of learning, building capacity to undertake future-oriented work, and raising awareness and enabling reframing) and tools (communications strategy, engagement with stakeholders, creating partnerships and focusing the policy-making agenda) discussed and summarised in section 2. To recap, by desired impact types, we mean the broad types or dimensions that foresight activities often seek to influence. By tools, we mean the particular practices that serve to meet desired objectives and impacts in the form of tangible plans and activities.

6.1 Impact concepts and categoriesInforming decision making with knowledge and insights – instrumental: UKCES had a broad rather than a narrow purpose in undertaking the study. It was not designed to facilitate decision-making in a specific, narrow sense (e.g. for example, the development of a particular Policy). What emerges from this analysis, rather, is the use of knowledge, empirical data and insight to assist decision-making across a number of the organisation’s work streams – on careers, STEM skills and so on. There was a broad instrumental focus but the evidence suggested that the outputs have been valuable across several areas.

Facilitation of learning amongst networks of individuals and organisations: Exchanging the challenges of skills and the development of solutions of skills programmes between stakeholders from different sectors and different field of activities (e.g. HR and education provider) helps to get a deeper understanding of necessary activities. The stakeholder workshop was used to discuss future needs by understanding limitations and opportunities of today’s mechanisms and initiating cross-sector learning and thinking.

19

Building capacity to undertake future-oriented work: it was reported that the Future of Work 2030 Study – and the accessibility of its outputs – helped to engender confidence amongst staff within UKCES to use foresight work. “People have greater confidence in speaking more widely about the future of work”.

Raising awareness and enabling reframing – cognitive and conceptual: the development of disruptive scenarios – and their strategic implications – provided a cognitive jolt to staff (and stakeholders). This was particularly evidenced in the discussions around the radical automation scenario (Skills Activism), where the jobs of a large proportion of the population would be replaced by artificial intelligence, machine learning and robotics.

6.2 MechanismsCommunications strategy: the communications approach was clear and timely. It addresses latent policy debates on shaping future work and skills policies and also critical public debates about the quality of future jobs, e.g. zero-hour contract jobs, which were discussed in the media quite a while. The launch of the report attracted a great deal of interest from high profile, mainstream news and media outlets, as noted above.

Engagement with/integration of stakeholders within the foresight processIncluding stakeholders in the project is highly important. Engaging with stakeholders has been recognised as a key factor in generating impact from foresight work (Calof and Smith, 2012; Cox et al., 2015; Havas et al., 2010; Harper, 2013; Johnston, 2012; Meissner, 2012). Increasingly, engagement is seen as an on-going, iterative process where different actors are involved in co-production alongside the commissioning organisation and the project delivery group (Miles, 2012). The experiences of the Future of Work 2030 study reinforce this need to design engagement into the foresight process on an ongoing basis – rather than something that is ‘bolted on’ after the final words of the report have been written. What emerged as particularly significant in the case of this study was the role of senior leaders from different sectors as the outward-facing spokespeople and champions (who were Commissioners of UKCES). A trade union leader facilitated the main workshop with representatives from a range of different sectors and organisations. A senior business executive presented the study’s results at the main launch conference and became the public face and voice, engaging with industry, policy makers and the media. At the interface of the project with the broader community, the role of this ‘foresight frontman’ (or frontperson/ frontpeople) was a key factor in generating impact. By ‘frontman’ we mean – in a positive sense – a person who represents an organisation to the wider world, communicating its message effectively.8 This has some resonance with the concept of a foresight Promoter (Cuhls, 2015). The esteem and prominence of these figures greatly facilitated the process of generating awareness, debate and – ultimately – impact from the Future of Work 2030 study.

8 A frontman is also the lead singer of a pop or rock group. Although this is not our main conception in this instance, the attention usually channelled towards him (or her for a frontperson) is a somewhat useful analogy.

20

Creating partnershipsThe issue of partnerships is interesting within the Future of Work study. The social partnership model is embedded within the governance and direction of UKCES –involving business, trade unions and the public sector. As noted in relation to engagement with stakeholders, these were critical in stimulating broader engagement amongst key constituencies. One of the lessons drawn from the experience was around engaging with employers, particularly small and medium-sized enterprises (SMEs), which are traditionally harder to reach. Alternative partnership models with SME intermediary bodies could be a way of addressing this in future exercises.

Focusing on policy-making agenda – key themes and key decision-makersThe Future of Work 2030 study was not instrumentally directed at developing a specific policy. The cross-Government group on the Future of Work had just started its work when the study was published. In this sense, the timing was slightly fortuitous but the early engagement had primed a range of relevant actors. The Future of Work 2030 study both shaped and was embedded within the political and policy discourse around future challenges on work, employment and skills.

Table 3 summarises the mechanisms identified for the 2030 study based on the framework set out earlier in Table 1.

Table 3 Foresight impact categories and mechanisms: the Future of Work 2030 StudyMechanisms 2030 StudyCommunications strategy Multi-media and multi-channel communications

strategy with a variety of media-compatible productsEngagement with/integration of stakeholders within the foresight process

Involving stakeholders on three levels: via interviews, workshops and in conferences

Creating partnerships Embedded social partnerships within Commissioner structure; future need for partnership with SME intermediary body

Focusing on policy-making agenda – key themes and key decision-makers

Element of serendipity but contributing to and aligning with key political discourse is important

(New)‘Foresight frontman’: influential leaders to be voices and image of the foresight project

6.3 Designing impactful foresight projects: reviewing the project experienceThe 2030 Future of Work study was a relatively large exercise. Whilst the project brief was intentionally broad, exploring futures for work and skills at a macro level, it was also

21

supposed to take into consideration the more nuanced perspectives of key stakeholder groups (employers, policy-makers, education and training providers, and individuals) and seven economic sectors. Within the project’s design and implementation, engaging with these different stakeholders and sector representatives was done on a sampling basis (e.g. for the interviews) and on a cross-sectional basis (e.g. for the one-day conference). One of the key strategic priorities for UKCES is to enable an employer-led skills agenda. Engaging employers in this agenda – particularly small to medium-sized enterprises – is commonly regarded as a challenge. Within our study, it is easier to identify foresight impact for policy-makers than it is for employers. Learning from this experience, we would consider an amendment or addition to the approach, engaging employers more specifically through bespoke workshops and social media tools.

On the whole, the approach in developing a portfolio of outputs and engagement tools appeared relatively effective in generating results and engaging different stakeholders in the process.

7. Conclusion

Robustness and credibility of foresight outputs are essential to achieve policy-related impact. Increasingly, these are taken as given when policy organisations are looking for greater value and impact from their investment in foresight activities. The data collected in the 2-year period since the conclusion of the project indicate that the Future of Work 2030 study has been impactful. Key design choices and processes, notably the quality and variety of outputs for different stakeholders (including visualisations) – i.e. the “readability” of the study for non-experts, the engagement with stakeholders during and after the project, and innovative social media campaigns have all been important factors in generating the attributed impact. Engagement with stakeholders has been widely recognised as a key factor in generating impact from foresight work. This is certainly reinforced by the experience of the Future of Work 2030 study. This iterative process started relatively early in the course of the project – during interviews, and continued with workshops and a conference. This iterative model of co-production was considered effective and highly suitable for working with diverse stakeholder groups.

Whilst several factors contributed to the impact generated, two were perceived as particularly critical. The first is the involvement of a high-level foresight frontman – a senior business executive – from the outset. Whilst broader literature and experiences point to the need for good engagement, paying attention specifically to a foresight frontman could be worthwhile for other foresight activities. The second is the need for the communications strategy and many related efforts along a variety of channels (against the backdrop of high quality research and results). Foresight results need to be solidly-based but also attractive for communication – for both technical experts and non-technical audiences. Visuals, brochures for a non-technical audience, slide decks and social media are helpful. Even with the same quality of results, without these two critical factors/ mechanisms, the study would probably not have achieved a fraction of the impact that it has had. This means that, for

22

generating impact pathways, a greater proportion of effort and attention needs to be allocated to these in foresight projects than has traditionally been the case.

It also emerges that impacts are different along the awareness curve of topics. In our case, the foresight work addressed national skills policies at a pre-action oriented stage of a cross-governmental working group that took up the insights - and might transfer them into policy proposals for Government. In other examples it addresses more latent discussions that lead to an increasing awareness of the topic, but no short-term activities.

The results presented in this paper refer to one case study and the extent to which the conclusions can be generalised is debateable. The conclusions are more applicable to scenario projects that have similar features – which have a variety of stakeholders. The Millennium Project is currently running a series of national workshops for its 2050 Global Work/Technology Scenarios (Millennium Project, 2016). Whilst this is not the type of ‘self-contained’ project described in this paper, it could adopt and adapt the mechanisms that appear to have been particularly influential in the UK Future of Work 2030 project (for example in designing a communications strategy and engaging with different stakeholders). The questionnaire used by the authors for the 2030 study in this paper could also be used by others in seeking to understand the use and impact of foresight projects after they have been completed. To achieve a broader picture of impacts generated, it would be advisable to monitor not only impacts from the point of view of the commissioner or customer, but the other target groups that the outputs are designed to reach. This could mean, for example, running feedback/review rounds in a structured way with a selection of stakeholders.

This paper has discussed the use and impact of foresight results over a period of two years following the completion of the project. Following the second round of post-project reviews, it was announced that the commissioning organisation, UKCES, would be wound up by 2017. The staff who were interviewed for this paper have subsequently left the organisation. This illustrates some of the practical challenges of assessing use and impact of foresight results, particularly over the medium to long term. Methodologically, this stimulates thinking on future research on impact assessment of foresight projects. One approach would be to conduct discourse analysis of relevant documentation and literature on a topic that has been the focus of major foresight activities. Taking the example of the foresight project, Tackling obesities: future choices, completed in 2007, discourse analysis could be undertaken on parliamentary documents and debates, Government statements, policy and strategy documents of interested stakeholders to track citations to the original foresight project, and also changes in norms and views expressed. Discourse analysis has been used in innovation policy in studies on public engagement (Pieczka and Escobar, 2012) and parliamentary discourse (Perren and Sapsed, 2013). Although many foresight exercises carry out ex-post evaluations, it is unclear how much work is undertaken to assess use and impact over a longer time frame. This is also a topic for future research – which would identify how different researchers have approached the task.

23

Foresight impact remains a challenging and complex issue, often relying on serendipity as well as good judgement. However, knowledge on effective mechanisms and foresight impact pathways will help to guide practitioners and commissioners in adopting appropriate study design elements.

AcknowledgementsThe authors would like to acknowledge the kind support of the staff and Commissioners of UKCES. We are particularly grateful to Peter Glover and Helen Beck for their continued assistance.

Funding: this paper draws on research funded by the UK Commission for Employment and Skills. This funding is gratefully acknowledged.

ReferencesAhlqvist, T. and Rhisiart, M. (2015) ‘Emerging pathways for critical futures research: Changing contexts and impacts of social theory’, Futures Vol. 71 pp.91-104

Brown, S. L., & Eisenhardt, K. M. (1997). The art of continuous change: Linking complexity theory and time-paced evolution in relentlessly shifting organizations. Administrative science quarterly, 1-34.

Brynjolfsson, E. and McAfee, A. (2014) The Second Machine Age. Work, Progress, and Prosperity in a Time of Brilliant Technologies, New York: WW Norton & Co

Calof, J., and Smith, J. (2010) 'Critical success factors for government-led foresight', Science and Public Policy, 37 (1), 31-40

CITB (2015) Construction 2030 and Beyond: The Future of Jobs and Skills in the UK Construction Sector, Construction Industry Training Board, London

Cox, A., Swift, S. and Rhisiart, M. (2015) Success factors for achieving policy impact in foresight studies, European Risk Observatory, European Agency for Safety and Health at Work, December 2015

Cuhls, K. (2015) Lessons for policy-making from Foresight in Non-European Countries, European Commission, Directorate-General for Research and Innovation

Da Costa, O., Warnke, P., Cagnin, C. and Scapolo, F. (2008), ‘‘The impact of foresight on policy-making: insights from the FORLEARN mutual learning process’’, Technology Analysis and Strategic Management, Vol. 20 3, May, pp. 369-87.

Daim, T., Basoglu, N., Dursun, O., Saritas, O., and Gerdsri, P. (2009). ‘A comprehensive review of Turkish technology foresight project’. Foresight, 11(1), 21-42

24

Eriksson, E. A., and Weber, K. M. (2008). Adaptive foresight: navigating the complex landscape of policy strategies. Technological Forecasting and Social Change, 75(4), 462-482.

ESRC (2009), ‘‘Taking stock: a summary of ESRC’s work to evaluate the impact of research on policy and practice’’, February, available at: www.esrc.ac.uk/impacts-and-findings/impact-assessment/developing-impact-evaluation.aspx

Frey, C.B. and Osborne, M. (2013) ‘The Future of Employment: How Susceptible are Jobs to Computerisation’, Oxford University

Georghiou, L. (1996). The UK technology foresight programme. Futures, 28(4), 359-377.

Georghiou, L., and Keenan, M. (2006). ‘Evaluation of national foresight activities: Assessing rationale, process and impact’. Technological Forecasting and Social Change. 73(7): 761-777

Georghiou, L., Keenan, M., Miles, I. and Cameron, H, (2006) An evaluation of the UK Foresight Programme: final report. University of Manchester

Haegeman, K., Harper, J. C., & Johnston, R. (2010). Introduction to a special section: Impacts and implications of future-oriented technology analysis for policy and decision-making. Science and Public Policy, 37(1), 3-6.

Handy, C. (1984) The Future of Work: A Guide to a Changing Society, Oxford: Basil Blackwell

Harper, J.C. (2013) Impact of Technology Foresight, Nesta Working Paper 13/16 November 2013

Havas, A., Schartinger, D. and Weber, M. (2010), ‘‘The impact of foresight on innovation policy-making: recent experiences and future perspectives’’, Research Evaluation, Vol. 19 2, June, pp. 91-104.

Hilbert, M., Miles, I., and Othmer, J. (2009). Foresight tools for participative policy-making in inter-governmental processes in developing countries: Lessons learned from the eLAC Policy Priorities Delphi. Technological Forecasting and Social Change, 76(7), 880-896.

Inayatullah, S. (2002). Reductionism or layered complexity? The futures of futures studies. Futures, 34(3), 295-302.

Johnston, R. (2012),"Developing the capacity to assess the impact of foresight", Foresight, Vol. 14 Iss 1 pp. 56 – 68

Keller, J., and Von der Gracht, H. A. (2014). The influence of information and communication technology (ICT) on future foresight processes—Results from a Delphi survey. Technological Forecasting and Social Change, 85, 81-92.

25

Kingdon, J. W. (2002), Agendas, alternatives, and public policies. New Jersey: Longman

Kuosa, T. (2016). The evolution of strategic foresight: navigating public policy making. Routledge.

Lane, D., & Maxfield, R. (1996). Strategy under complexity: Fostering generative relationships. Long Range Planning, 29(2), 215-231.

Linstone, H. A. (2011). Three eras of technology foresight. Technovation, 31(2), 69-76.

Meissner, D. (2012). Results and impact of national Foresight-studies. Futures, 44(10), 905-913

Miles, I. (2012). Dynamic foresight evaluation. foresight, 14(1), 69-81

Millennium Project (2016) 2050 Global Work/Technology Scenarios (unpublished), Washington DC

Miller R. (2016) ‘Recent Developments in Thinking about the Future: An Overview for Policymakers’, in, Neng W.W.: Adaptive Governance for a Changing World, pp. 187-199

Miller, R. (2007). Futures literacy: A hybrid strategic scenario method. Futures, 39(4), 341-362.

Miller, R., & Poli, R. (2010). Anticipatory systems and the philosophical foundations of futures studies. foresight, 12(3).

Owen, R., Macnaghten, P., & Stilgoe, J. (2012). Responsible research and innovation: From science in society to science for society, with society. Science and Public Policy, 39(6), 751-760.

Perren, L., & Sapsed, J. (2013). Innovation as politics: The rise and reshaping of innovation in UK parliamentary discourse 1960–2005. Research Policy, 42(10), 1815-1828.

Pieczka, M., & Escobar, O. (2012). Dialogue and science: Innovation in policy-making and the discourse of public engagement in the UK. Science and Public Policy, 40 (1), 113-126

Popper R., Georghiou L., Miles, I. and Keenan M. (2010) Evaluating Foresight: Fully-Fledged Evaluation of the Colombian Technology Foresight Programme (CTFP), Cali: Universidad del Valle

Poteralska, B. and Sacio-Szymańska, A. (2013). ‘Evaluation of technology foresight projects’. European Journal of Futures Research, 2:26

26

Raford, N. (2015) ‘Online foresight platforms: Evidence for their impact on scenario planning and strategic foresight’, Technological Forecasting and Social Change, Volume 97, August 2015, Pages 65-76

Rhisiart, M., and Jones-Evans, D. (2016). The impact of foresight on entrepreneurship: The Wales 2010 case study. Technological Forecasting and Social Change, 102, 112-119.

Ross, A. (2016) The Industries of the Future. How the Next 10 Years of Innovation Will Transform Our Lives at Work and Home, London: Simon & Schuster

Sabatier, P. A. and Jenkins-Smith, H. C. (1999). The advocacy coalition framework: An assessment. Theories of the policy process, 118, 188.

Science and Technology Committee (2014) Government horizon scanning, HC 703, London: The Stationery Office

Schoemaker, P. J. (1990). Strategy, complexity, and economic rent. Management science, 36(10), 1178-1192.

Stacey, R. D. (1995). The science of complexity: An alternative perspective for strategic change processes. Strategic management journal, 16(6), 477-495.

Stacey, R. D. (2007). Strategic management and organisational dynamics: The challenge of complexity to ways of thinking about organisations. Pearson education.

Störmer, E., Patscha, C., Prendergast, J., Daheim, C., Rhisiart, M., Glover, P., & Beck, H. (2014a). The future of work: jobs and skills in 2030, Evidence Report 84, UK Commission for Employment and Skills

Störmer, E., Patscha, C., Prendergast, J., Daheim, C., Rhisiart, M., Glover, P., & Beck, H. (2014b). Trends and Disruptions Report, UK Commission for Employment and Skills

UK Commission for Employment and Skills (UKCES) (2014a) A Guide to the UK Commission for Employment and Skills, Autumn 2014

UK Commission for Employment and Skills (UKCES) (2014b) Careers of the Future, London, 10 December 2014

UK Commission for Employment and Skills (UKCES) (2014c) Growth Through People: a statement on skills in the UK, UKCES, London, 25 November 2014

www.gov.uk/government/publications/high-level-stem-skills-requirements-in-the-uk-labour-market

27

UK Commission for Employment and Skills (UKCES) (2015) High level STEM skills requirements in the UK labour market, UKCES, London, 10 July 2015

https://www.gov.uk/government/publications/high-level-stem-skills-requirements-in-the-uk-labour-market

Van der Meulen, B., De Wilt, J., and Rutten, H. (2003). Developing futures for agriculture in the Netherlands: A systematic exploration of the strategic value of foresight. Journal of Forecasting, 22(2 3), 219-233.‐

van der Steen, M., and Van der Duin, P. (2012). ‘Learning ahead of time: how evaluation of foresight may add to increased trust, organisational learning and future oriented policy and strategy’. Futures, 44(5), 487-493

Varum, C.A. and Melo, C., (2010). ‘Directions in scenario planning literature – Areview of the past decades’, Futures 42 (4), 355–396

Wilson, R., Beaven, R., May-Gillings, M., Hay, G., and Stevens, J. (2014) Working Futures 2012-2022 Evidence Report 83, UK Commission for Employment and Skills, March 2014

Zuboff, S. (1988). In the age of the smart machine: The future of work and power. Basic Books.

28

Appendix 1: Post-project review interview questions

Round #1 Questions, November 2014

1. What were the key impact / success areas of our project?

2. What is specific / ‘special’ about the impacts generated by this study (as compared to others)?

3. What were the critical levers for generating impact?

4. What problems and challenges did you see with respect to generating impacts? What would you do differently next time?

5. Which different audiences did the study have an impact on?

6. Where and by whom was / is the study being used?

7. Any other reflections with respect to impacts of the study?

Round#2 Questions, November 2015

1. What has happened in the last 12 months – dissemination, use and impact of the study?

2. What are the reasons for the strong interest in the work? What are criteria for success?

3. What has been the use/impact amongst the four stakeholder groups: policy-makers; employers; education and training providers; individuals?

4. What are your reflections now on engagement, methods and approach?

5. What has made the Foresight study impactful?

29

30