Interim evaluation & assessment of future options for ... · Interim evaluation & assessment of future options for ... society and the citizen in Europe"3 in ... Interim evaluation

December 2012

Interim evaluation & assessment of future options for Science in Society Actions

Interim evaluation

Final Report - Annexes

Interim evaluation & assessment of future options for Science in Society Actions i

Table of Contents

Annex A - Study Terms of Reference 1!

Annex B - Questionnaire for SaS/SiS project participants 16!

Annex C - Questionnaire for FP7 Cooperation programme coordinators 23!

Annex D - List of interviewees 26!

Interim evaluation & assessment of future options for Science in Society Actions 1

Annex A - Study Terms of Reference

A.1 General Context

A.1.1 The overall purpose and justification of the study

These are the Terms of Reference of a Call for Tender aiming at awarding a contract to carry out a study on the demand of the European Commission, Directorate General Research, Directorate L: Science, Economy and Society.

The implementation of the European Research Area (ERA)1 and its related strategies require an information base that allows the stakeholders (governments, funding agencies, interested parties) to develop their actions whilst being aware of the context of those strategies, to be able to benchmark their performance against each-other, and to be able to collectively set targets and evaluate progress.

In the context of the 7th Framework Programme (FP7) the 2010 edition of the Capacities Work Programme: Science in Society2, under area 5.4 'Strategic Activities', therefore includes a study on "Interim and ex-ante impact assessments of Science in Society Actions".

The overall objective of the tender is to provide

• for one part key information on the interim results and impact of the science in society projects funded under FP7 as part of the Capacities Programme (including the transition period from the 6th Framework Programme (FP6) to FP7 as well as a sample of activities addressing specific Science in Society related aspects within parts of the Cooperation Programme);

• for the other part an assessment of future options for science in society actions on EU level which will be used as a key input to the preparation of the remaining FP7 Science in Society Work Programmes and the planning of Science in Society actions in the longer term, under a possible future 8th Framework.

A.1.2 Background

Overall, the Science in Society (SiS) Work Programme has developed in three phases:

• The first experimental phase began in 2002 where the Science and Society Programme was a new initiative at European level undertaken under FP6. Its main objectives were formulated in the Commission Working Document "Science, society and the citizen in Europe"3 in 2000 as well as the Science and Society Action Plan4, adopted by the Commission in 2001, and came with the acknowledgement that the relationship between science and society is fundamental to research policy, and will be a key factor in implementing the Lisbon strategy. The programme was partly built on good practice examples in the then EU-15 Member States. It was implemented through a broad range of activities and has established a European context for examining societal issues in research and development.

• With the beginning of FP7 in 2007 there has been a second phase of mobilisation of civil society actors establishing a conceptual shift from Science and Society (S&S) in FP6 towards Science in Society (SiS) in FP7. A reference framework has been set for an inclusive perspective on research in its wider societal and policy context. The overall aim of the Science in Society programme is to contribute to the implementation of the European Research Area and to help building a democratic knowledge-based society. It intends do this by stimulating a harmonious integration of scientific and technological endeavour in Europe, and by encouraging broader

1 http://ec.europa.eu/research/era/index_en.html 2 ftp://ftp.cordis.europa.eu/pub/fp7/docs/wp/capacities/sis/s_wp_201001_en.pdf 3 ftp://ftp.cordis.europa.eu/pub/rtd2002/docs/ss_en.pdf 4 http://ec.europa.eu/research/science-society/pdf/ss_ap_en.pdf


public engagement ('downstream' and 'upstream') in scientific research, in particular, through involvement of civil society in debating and shaping the research agenda.

• Since 2008, a third phase has been entered with the Ljubljana process and a new approach towards ERA governance and actions linked to societal Challenges. The 2020 Vision for the European Research Area (ERA) adopted by the Council in December 2008 underlines that the ERA "is firmly rooted in society and responsive to its needs and ambitions in pursuit of sustainable development", confirming the orientation of the Science in Society Work Programme. Building on this new approach, the Science in Society Work programme will take a two-fold approach in the future. Responding to the Lund Declaration, the first component provides a vertical connection to ERA developments by prioritising large-scale structured actions to promote deeper and more systemic collaboration between a wide range of actors around societal Challenges, the Joint Programming Initiative and other broader EU policy initiatives. This is the aim of the MML initiative (Mobilisation and Mutual Learning Actions5). The second component comprises horizontally-oriented actions to address specific SIS issues relevant to the ERA and which underpin the knowledge triangle (education-innovation-research) as a whole.

The beginning of one phase does not necessarily signal the end of another, rather it means an increase in capacity which helps anchor "science in society issues" within research policy and the societal challenges which it must address.

A.2 Description of the activity to be evaluated

Within the Specific Programme: "Capacities"6 the main aim of the Science in Society area is "with a view to building an open, effective and democratic European Knowledge society" to "stimulate the harmonious integration of scientific and technological endeavour and associated research policies in the European social web, by encouraging at European scale reflection and debate on science and technology, and their relation with the whole spectrum of society and culture". The Science in Society Programme will be implemented through:

• policy-related actions and research supported directly from this theme;

• cooperation between Member States, identifying common goals, and reinforcing national practices, in the spirit of the open method of coordination;

• promoting, supporting and monitoring the uptake and impact of "Science in Society" issues in other parts of the Framework Programme. The overall coordination of issues related to Science in Society both across the Framework Programme and within other relevant Community activities (e.g. relating to education and culture) will be ensured by this theme.

Within these three defined Action lines of the Science in Society Programme the following activities are specified:

First action line: A more dynamic governance of the science and society relationship

• Strengthening and improving the European science system

• Broader engagement to anticipate and clarify political, societal and ethical issues

• Better understanding of the place of science and technology (S&T) in society

• The evolving role of universities

Second action line: Strengthening potential, broadening horizons

5 See Science in Society Work Programme 2010, page 3;6-7 ftp://ftp.cordis.europa.eu/pub/fp7/docs/wp/capacities/sis/s_wp_201001_en.pdf 6 COUNCIL DECISION of 19 December 2006 on the Specific Programme: "Capacities" implementing the Seventh

Framework Programme of the European Community for research, technological development and demonstration activities (2007 to 2013) - (2006/974/EC)

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2006:400:0299:0367:EN:pdf


• Gender and research

• Young people and science

Third action line: Science and society communicate

In more detail the aims of implementing actions within the Programme are7

• render more inclusive and transparent the mechanisms for access to, and validation of the expertise necessary to underpin more robust policies,

• set landmarks for an ethically sound research endeavour in the light of fundamental rights,

• allow Europe to play a more active role on the world stage, in the debate and promotion of shared values, equal opportunities and societal dialogue,

• bridge the gap between those who have a scientific education and those who do not (and in this context support informal and formal science and maths education), promote a taste for scientific culture in the direct neighbourhood of all citizens (calling upon cities, regions, foundations, science centres, museums, civil society organisations, etc.),

• encourage a societal dialogue on research policy, and stimulate civil society organisations to become more involved in research activities,

• explore ways to improve governance of the European research and innovation system,

• provide an image of science and researchers which is meaningful to all, especially to young people,

• promote the participation and progress of women in scientific careers and better use their professional and scientific talents for the benefit of all (including a more gender aware approach to EU-funded research and technological development),

• and renew science communication, favouring modern means to achieve higher impact, helping scientists to work closely with media professionals.

The study shall evaluate the impacts of actions addressing Science in Society issues against these initial goals (respectively the ones set out for FP6 for the FP6 projects to be analysed). In addition to the Specific Programme Science in Society, selected areas of the Cooperation programme, namely Information Society and Technologies, Nanotechnologies, Health, Food, Agriculture and Fisheries, Biotechnology (only the Biotechnology part) and Environment shall be analysed on programme as well as project level. The selection of projects, programme areas etc. in this context has to be made in cooperation with Directorate L of DG Research which will consult its Steering Group in this context.

With view to the part of the study assessing future options, the study shall provide evidence for the intervention on EU level and develop scenarios and options for the implementation of Science in Society actions beyond FP7. The study will have to follow the guidelines for impact assessment within the European Commission and will have to encompass the modules and steps outlined in chapter 3.2.

A.3 Study objectives, evaluation questions, tasks and scope

The call for tender comprises one study that will have two distinctive parts to be covered by one contractor:

• Part 1: Interim evaluation of Science in Society actions funded in the last part of FP6 and FP7 so far (including recommendations for the implementation of the final years of FP7)

• Part 2: Assessment of future options for Science in Society activities on EU-level

7 COUNCIL DECISION of 19 December 2006 on the Specific Programme: "Capacities" implementing the Seventh Framework Programme of the European Community for research, technological development and demonstration activities (2007 to 2013) (2006/974/EC)

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2006:400:0299:0367:EN:pdf page 340 ff.


A.3.1 Part 1: Interim evaluation of science in society actions within the transition phase from FP6 to FP7 and within FP7 so far

Objectives

The study shall evaluate the impact of actions, financed in the final phase of the Science and Society programme in FP6 (72 projects with a start date from 01.03.2006 on) and the Science in Society part of the 'Capacities' Specific Programme so far in FP7 (Calls 2007/2008, 2009 and 2010)and assess in how far the general objectives set out have been achieved on programme as well as project level, also in terms of progress made towards achieving the social dimension of the European Research Area. This part of the study will follow up and complement the findings of the FP6 Mid-term Assessment of Science and Society actions 2002-20068 in order to assess the transition from FP6 to FP79.

In addition, a sample of activities addressing specific SiS-related aspects (e.g. ethics, gender, governance, risk assessment, public engagement etc.) within other parts of the Cooperation Programme, namely the Environment Programme, the Information & communication technologies Programme and the Nano-sciences, nanotechnologies, materials & new production technologies Programme will be included in the scope of the evaluation, in followup to the "embedding S&S in FP6" study.

Context and rationale

The interim evaluation10 shall cover the years 2006-2010, a period during which the European research landscape in relation to Science in Society actions has changed significantly, including:

• A significant increase of EU Budget allocation to FP7 and to SiS activities.

• The latest landmarks of a renewed re-invigorated ERA which focus on the societal and multi-stakeholder dimension of research and innovation and underpin both the current SiS actions and the future relevant developments in the short/medium term.

• An increased research component within the SiS programme.

• A stronger focus on Public Engagement of Science and role of Civil Society Organisations (CSO).

• Actions to promote a more gender-aware Human Resources management in public research sector, involving also the main social actors

• Actions to promote a more gender aware scientific research and technological development

• More explicit support for informal and formal science and maths education

The contractor will have to seek coordination with the Commission Service for any parallel or evaluation/assessment activities going on or planned to avoid potential duplication of work.

Description of the work

The analysis shall be based on the overall rationale of Part 5 of the Specific Programme" structuring the European Research Area" in FP6 (Science and Society) as well as of Part 5 of the Specific Programme "Capacities" in FP7 (Science in Society Programme) and in particular the three action lines and the strategic activities thereof in order to evaluate the results of research and policy activities under the S&S/SiS theme between 2006 – 2010. The analysis should include quantitative elements where appropriate and feasible as well as a qualitative assessment. The qualitative

8 For a summary of links to relevant studies to be consulted please see chapter 3.1.5 9 For the assessment of the transition period and the evolvement of the Programme and objectives from FP6 to FP 7 it is also

necessary to refer to the initial goals of the Science and Society Programme "COUNCIL DECISION of 30 September 2002 adopting a specific programme for research, technological development and demonstration: ‘structuring the European Research Area’ (2002–2006) - (2002/835/EC)in FP 6

10 General information on EC practises in the area of evaluation can be found at http://ec.europa.eu/dgs/secretariat_general/evaluation/index_en.htm For examples of evaluation exercises within DG Research please see http://ec.europa.eu/research/evaluations/index_en.cfm?pg=fp7


evaluation of the transition from FP6 to FP7 - with the aim of advancing the programme from a collection of individual issues and projects to a more strategic approach with intensified links to policy development/making - and the degree of effective implementation of the objectives of FP7 is of crucial importance in this context.

Analysis and deriving recommendations will touch upon different levels and involve a number of issues or questions from which at least the following will have to be considered:

Programme Level

Evolution of the Programme

- What is the relevance of the goals set out for the S&S/SiS activities in comparison with other activities taking place on Member state level and within academia, industry, education etc.? - Have the goals set been realistic? - Is the design of the programme clearly structured, and are the objectives set out clear enough? - Are there sufficient tools foreseen to monitor and evaluate the progress, results and impacts of the activities in different areas? - Has the Programme over time succeeded in its ambition to develop less but bigger, more strategic projects with more pronounced policy links and impacts? - Has a diversification of contexts and partnerships beyond academic circles (especially industry, CSOs, cities) taken place? - Has the introduction of new mechanisms (especially MML) within the Programme succeeded in significantly increasing stakeholder participation? Did they change the type of involvement of nonresearchers in the SiS programme? - Are the elements of novelty introduced by FP7 (broader focus, increased research activities, transversal focus) efficient with respect to reaching their intended objectives? - What is the progress with view to dissemination and awareness? - Are there different approaches with view to instruments, funding and content priorities that could deliver better results?

Programme Coverage

- Do the funded projects cover the whole spectrum of Science S&S/SiS objectives? - What is the Progress made to increase participation from "new" 8 Member States? - Is there evidence for progress in quantitative and qualitative terms to increase international focus of the programme?

Impact of the Programme

- How far has the S&S/SiS programme achieved its general objectives? - What is the impact of the FP6/FP7 S&S/SiS Programme on scientific advice and decision-making processes at different levels (EU, national, regional, local)? - How policy-relevant are the topics taken up in the Work programmes 2007 – 2010 for the ERA? - What is the impact on policy development, including on multi-level policy? - Does the S&S/SiS programme play a role in shaping the ERA? If yes, in which ways and how can the impact of SiS programme in policy developments and shaping ERA be improved? - What has been the impact of Science in Society activities in comparison with other Programmes (SSH Programme as a reference) when it comes to instruments as well as the Programme as a whole? - Did the programme or projects within the programme have an impact with view to the integration and involvement of stakeholders? Did this have an impact for example on the relation with policy making, public accountability or attitudes towards S&S/SiS issues? (e.g. outcomes Eurobarometer) - What has been the impact of the Programme on the SiS communities and landscape in the Member States and Associated Countries? - Are there specific parts of Europe that the Programme "helped" more than others / where the impacts of the Programme have been stronger?

Tools and approaches used

- Is there an adequate consistency between the activities funded and the initial goals set out by the Programme that is to say did the tools and approaches used help to reach the objectives? - Is there evidence of the hypothesis made by the MASIS Expert Group of a European Model of S&S/SiS?

Budget / Funding

- Is the budget adequate in relation to the overall objectives and the individually funded actions? - How efficient, relevant, appropriate and sustainable is the funding level of SiS actions in comparison with comparable national and international activities?


Project Level11

Relevance/ Results/ Impact

- Has the project achieved its specific objectives? - What is the added value compared to the issues requested in the Work Programme Topic? - What is the relevance and significance of these project outcomes for the objectives of the Science and Society Programme in FP6/FP7? - Is there evidence of impact on stakeholders outside the partnership? (Degree of impact; relevance of the impact; on what stakeholders) - Does the project have a potential future impact on public policy, regulation or practice etc.? - Within the sample of projects analysed are there any that have shown outstanding or path breaking advancements with view to new ways of undertaking or governing research activities (stakeholder involvement, participatory processes, impact on policy, indicator development etc.)? - Within the samples of projects are there any best-practice examples of initiatives, actions or cross-thematic partnerships with other parts of the Framework Programme?

European added value

- Does the project go beyond state of the art, can innovative approaches e.g. related to research, the involvement of stakeholders, the dissemination of results etc. be observed? - Do the project outcomes demonstrate European added value?

Dissemination - Are the project outcomes disseminated to all relevant stakeholders? - Is the project disseminated sufficiently within the Commission?

Sustainability - Is there evidence of sustainability (further use of the acquired knowledge, further co-operation, publications etc.) beyond the funding period?

“Embedding” SiS activities within other parts of FP7

Coverage - To which degree (including quantitative elements where possible) are SiS related activities implemented within the Thematic Programmes of the FP7 Cooperation Programme (on programme- as well as project level) and which are the issues most frequently tackled?

Nature of the integration

- Has progress been made in quantitative and qualitative terms with view to the integration of SiS aspects within FP7 (compared to the findings of earlier analysis)? - What is the quality and nature of the integration of SiS issues within other parts of the Cooperation Programme? - How has the new funding scheme research for the Benefit of specific groups – CSOs been used by the different Programmes analysed and with what results (types of stakeholders involved, changes in the relationship / collaboration between research organisations and CSOs)

Connection between Member State / Associated Country level and EU level12

Comparison between national and EU activities

- What is the nature / what are the patterns of SiS activities (programmes, single projects) in different countries and are there any issues that are more frequently / strategically tackled than others?

Impacts on Member State level

- Is there a visible impact of the SiS Work Programmes on policy development in SiS at national level? - Are the new developments for increased integrated SiS efforts through e.g. MMLs, in pace with national and sectoral capacities across Europe? - Can any serious overlaps of activities between EU and MS level be observed that challenge the impact and relevance of the EU-level activities?

Conclusions and recommendations

This part of the study will provide an interim evaluation regarding the achievements of objectives of Science and Society / Science in Society actions. It will thereby - analyse the performance on programme as well as project level - provide conclusions and recommendations on a general programme level as well as related to the different Action lines of the Programme

11 Within the analysis of individual projects the focus in the context of this study lies on the contribution of projects to the overall programme goals. Individual project management or methodology issues are in general not part of the study.

12 In support of answering these questions the contractor will be able to get access to the country reports that will be conducted in 2010 within the MASIS project


• suggest issues, tools and activities to be implemented or strengthened with view to future SiS Work Programmes within the remaining FP7 and beyond

• comment on difficulties arising in the implementation of SiS activities / the SiS Programme on various levels and give recommendations on how they can be overcome.

The contractor must seek evidence, among other sources he/she will describe in the offer, in studies, analyses and earlier reports available across Science and society / Science in Society themes, for example from

• the Mid-Term Assessment 200613,

• the MASIS report "Challenging Futures of Science in Society"14

• the study "Integrating Science in Society issues in scientific research - Main findings of the study on the integration of Science and Society issues in the 6th Framework Programme"15.

For the analysis on Programme level the Specific Programmes of FP616 and FP717 as well as the different Work Programmes18 have to be taken into account.

In addition specific reports related to sub-areas of the Science and Society / Science in Society Programme are available e.g. the Rocard Report on science Education19, various studies and reports on gender issues20, the Globalization of Ethics report and the Public Engagement report. For the envisaged actions for the rest of FP7 one additional reference document will be the "Indicative Roadmap for SiS actions 2011-2013" which will be provided by Directorate L once completed.

On a more general level the ERAWATCH project21, which provides information on European, national and regional research policies, actors, and programmes in the EU and beyond could be of interest, as well as selected surveys within the EUROBAROMETER22.

Finally the Science in Society Website provides information and an overview on these issues as well23.

A.3.2 Part 2: Assessment of future options for Science in Society actions on EU level

Objectives

Objectives, options and potential impacts of SiS actions on EU-level in a future setting mainly beyond FP7 shall be assessed, including a prospective analysis of different policy options in order to contribute to the preparation of the relevant Commission proposals.

It will also be part of the task of the contractor to give contributions and recommendations for an appropriate set of indicators to be used for future assessments of the diverse impacts of Science in Society activities.

Context and rationale

Being in place for one and a half Framework Programmes, the S&S/SiS Programme is still young compared to a number of other parts of FP7. Nevertheless it is at a stage where due to its budget

13 http://ec.europa.eu/research/science-society/document_library/pdf_06/1-mta-report-22032007_en.pdf 14 ftp://ftp.cordis.europa.eu/pub/fp7/sis/docs/sis_masis_report_en.pdf 15www.ec.europa.eu/research/science-society/document_library/pdf_06/integrating-sis-issues-in-research-

mainfindings_en.pdf 16 ftp://ftp.cordis.europa.eu/pub/documents_r5/natdir0000029/s_1884005_20021106_142921_6ERL021884en.pdf 17 http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2006:400:0299:0367:EN:PDF 18 http://cordis.europa.eu/fp7/find-doc_en.html (for FP7) and http://cordis.europa.eu/fp6/sp2_wp.htm (for FP6) 19 http://ec.europa.eu/research/science-society/index.cfm?fuseaction=public.topic&id=1100 20 http://ec.europa.eu/research/science-society/index.cfm?fuseaction=public.topic&id=27 21 http://cordis.europa.eu/erawatch/index.cfm 22 http://ec.europa.eu/public_opinion/index_en.htm 23 http://ec.europa.eu/research/science-society/index.cfm?fuseaction=public.topic&id=781


increase from FP6 to FP7 and its strategic evolvement it is important to assess how such a programme respectively its actions can be best implemented on EU level in a future setting beyond FP7.

The new political agendas of the European Union (EU 2020 strategy, European Research Area, Lisbon Treaty, Political Guidelines of President Barroso) are having an impact on the work of DG Research and future Science in Society actions have to be assessed against this background and may have specific relations to issues like: smart, sustainable and inclusive growth (Europe 2020 strategy); a research and innovation revolution for a knowledge society, advancing people's Europe and enhancing dialogue and information (Barroso guidelines).

Description of work

According to the specific guidelines for impact assessment within the European Commission24 as well as the specific guidelines for ex ante evaluations25 this part of the study will have to encompass the following steps and modules, also taking into account the results obtained from the first part of the study - the interim evaluation of activities and projects implemented so far:

Defining the scope and level of the assessment

The Science in Society Programme has first been introduced within FP6 (from 2002 on) and has since then experienced a budgetary as well as strategic amplification. In addition the Treaty of Lisbon has entered into force and the Europe 2020 Strategy as well as reinforced efforts for implementing the European Research Area are important factors in shaping research policies and implementing strategies. In view of the achievements so far within SiS, but more prominently concentrating on these overall developments the future prospects for Science in Society activities shall be assessed. The results of the study will feed into the more general ex-ante impact assessment for Framework Programme 8.

Gathering information and consulting stakeholders

Since the Science in Society Programme includes a number of different activities that are potentially very heterogeneous, the potential stakeholders of the programme are quite diverse as well. It will be essential to find a mechanism of consultation that respects this characteristic and it may be possible to limit the consultation to specific stakeholder groups if this is duly justified and brings sufficient value within the context of the analysis.

Problem definition

Europe is undergoing profound changes that bring new challenges for science in society, not least because it has been given a significant role in the development of EU in the Vision 2020 of the European Research Area. The social dimension is a challenge and a key dimension of ERA (all along with the effective development, management and deployment of knowledge and research) and should translate into concrete objectives and actions to address the needs of its citizens and the issues of public interest which are in play. Science influences society but society also has a growing influence on science in different ways. Citizens have an increasing stake in the European Research Area and in science in Europe in general. Scientific institutions and practices of science become more and more re-contextualised in society, in an ongoing process, with overlapping and contested partial transformation. Policy makers emphasize the link between science, innovation and quality of life, but the political dimension, i.e. contributions to relevant debates, and the cultural and intellectual dimensions are also important. The revival of excellence and autonomy of science as a goal goes in parallel with calls for increased relevance and democratization of science. These overlapping and contested transformations, and the emerging trends and tensions call for dynamic governance, for productive interactions between a wide range of stakeholders and for an open debate about the question “What is the adequate place of science in society and – equally important - how is society positioning itself towards science?” For the development, continuation and implementation of concrete actions - including a European-level programme - this also means to (re-)evaluate "What are the main issues and gaps in the relationship between science and society", their relevance and potential for action? In order to raise the political profile and general impact of the programme "Which should be the core

24 http://ec.europa.eu/governance/impact/commission_guidelines/docs/iag_2009_en.pdf 25 http://ec.europa.eu/dgs/secretariat_general/evaluation/docs/ex_ante_guide_2001_en.pdf


issues and activities a future programme should concentrate on"? "What is the adequate level of resources and what are the appropriate tools to tackle these issues?" in the light of the experiences acquired so far.

What are the policy objectives?

While in FP6 the main goal of the Science and Society programme has been the "development of structural links between institutions and activities concerned with the dialogue between the scientific community and society at large", this goal has broadened in FP7 to also integrate the aspect of policy making and deepening the integration of science into society: "with a view to building an effective and democratic European knowledge-based society, the aim is to stimulate the harmonious integration of scientific and technological endeavour and associated research policies into European society". Originally the programme has taken its inspiration from the perceived deficits in the interaction/communication between science and society stating that "Science and Society action draws its inspiration from the fact that Europeans feel under-informed about science and research, youngsters are turning away from scientific studies and careers and people misunderstand – and sometimes mistrust – scientists and their activities". Turning towards a more formative perspective in FP7 the Science in Society initiative aims to "encourage Europe-wide reflection and debate on science and technology and their relation with society and culture. This action line is based on the rationale that the ability of European societies to develop themselves in a positive and sustainable way depends, to a large extent, on their capacity to create and exploit knowledge and to innovate". This has over time led to the development of a number of diverse and only partly related areas, making it difficult to give the Programme a distinctive profile. Aiming at more coherence, impact and branding of the programme what kind of effects can be expected from keeping diversification / moving towards more focus and a concentration on core issues?

What are the policy options?

It is expected that the assessment of future options for Science in Society Actions on EU level will, starting from the baseline option, cover at least the scenarios indicated below. The development of additional scenarios or combination of scenarios by the contractor is strongly encouraged. Each scenario considered should also be analysed in terms of its pros and cons and take into consideration wider benefits and impact: 1. Baseline option - a dedicated programme for SiS activities Including the lessons learnt so far, this option would aim at stabilising the programme, providing a certain sense of continuity for the affected communities as well as in strategic terms, focusing on an enhanced visibility and more prominent structuring role. Pertinence of the approach in view of emerging trends and cutting edge-issues (MASIS report) and introduction of new approaches as appropriate. 2. Redistribution of the Programme This option is referring to a segmentation of the Programme in different parts with an effort to placing the single parts where sensed as most appropriate: 1) Specific research centred issues could be implemented within the SSH Programme, using synergies and complementarities the two Programmes, especially with view to the orientation towards societal challenges. 2) Large scale horizontal issues concerning all parts of the Framework Programme could be treated horizontally (similar to the issue of internationalisation within FP7). This could for example be the areas of gender, open access and ethics. In addition core activities covering distinctive SiS themes would be covered, concentrating on the new processes and ways research is performed (interdisciplinarity, society as a stakeholder in research, issues of risk and trust). 3) The issue of Science education could be, for instance, handed over to the Directorate General for Education and Culture of the European Commission. 3. SiS activities with a strong policy pillar Such an option would aim at having the whole SiS Programme implemented as a horizontal activity comparable to the current configuration of running the international cooperation function in FP7. Beside a strong coordination function across all other thematic areas this option would include a more prominent and pronounced partnership with the Member States e.g. via flagship initiatives like MASIS. It would aim at increasing the effectiveness and coherence of national and Community policies and


improve the impact of SiS actions. 4. "Embedding" approach This option would aim at a complete horizontal as well as thematic integration of SiS activities within the different programme parts of a future Framework Programme; meaning that these issues would be tackled separately and individually by the different programme parts and no specific SiS programme would be implemented. Would such integration promote trans-disciplinary approaches to enhance interaction with natural as well as technical sciences? How could synergies between the programmes be achieved and monitoring across the programmes established? Is there a realistic scenario (e.g. via developing thematic modules for gender, communication, ethics etc.) that could raise the awareness for and integration of SiS actions in these areas? 5. No activities on EU level This option would try to assess the impacts in case no further coordinated activities on EU level at all would be implemented in the Science in Society area beyond FP7. In a first phase the contractor would develop a sufficient number of generally described scenarios or a combination of scenarios that cover the widest range of possible options. After seeking agreement with the responsible Commission Services, a limited number of likely options would be analysed in detail by the contractor in a second phase. For the discarded options a short justification will be given. For the work of this part of the study access will be given to the MASIS national reports in order to assess the nature as well as possible best practise examples of SiS programmes and activities within the Member States.

Assessing likely economic, social and environmental impacts

To compare and rank the different policy options they will have to be assessed against their perceived impacts. Key analytical steps in this procedure will include • Identify (direct and indirect) economic, social and environmental impacts and how they occur (causality). • Identify who is affected and in what way. • Assess the impacts against the baseline in qualitative, quantitative and monetary terms. If quantification is not possible explain why. • Identify and assess administrative burden/simplification benefits (or provide a justification if this is not done). • Consider the risks and uncertainties in the policy choices. For a more detailed instruction please refer to the European Commission Impact assessment Guidelines, 15 January 2009; SEC(2009) 92 www.ec.europa.eu/governance/impact/commission_guidelines/docs/iag_2009_en.pdf

How do the options compare?

In most cases there are alternative ways for achieving an objective. Alternative approaches may be identified at the level of intervention strategies, instruments, channels of intervention and levels of intervention. An analysis of alternative delivery mechanisms will identify what options are available and compare them on the basis of chosen criteria. This should be done in order to: – ensure that the instruments chosen for the implementation of the intervention are the most appropriate ones (in terms of effectiveness, efficiency or other chosen criteria), and to – demonstrate to decision-makers why the proposed approach should be considered the "best possible" means to achieve the ends. The relevance of these options shall be assessed and weighed according to the rules and methodology foreseen in the guidelines for preparing ex ante evaluations. Each should be assessed: on its own merits, against the baseline scenario and other options, in the light of the findings of the studies set out in chapter 3.5.1 in the light of the broader literature. How effective are the different options in relation to the objectives, how efficient are the options in achieving the objectives and how coherent are the options with view to overarching EU objectives, strategies and priorities?


With view the instruments implemented (e.g. large scale research projects, CSO, Coordination Actions, MML, "cooperative" research involving non-researchers) – what are the most effective funding instruments when it comes to delivering appropriate results in the SiS area, is there evidence that specific parts of the programme call for a specific set of instruments? Presentation of a summary overview of all positive and negative economic, social, and environmental impacts for the options analysed in detail including the respective quantitative and qualitative evidence to ensure maximum transparency. Ranking of the options (in form of a clear comparison table) according to their performance in achieving the defined policy objectives and with view to the balance of positive and negative impacts associated with the preferred option and possible alternatives present the results.

Arrangements for future monitoring and evaluation

With view to the analysis made so far and the options developed and especially related to the defined objectives it will be important to identify concrete monitoring and evaluation provisions, including a specific set of indicators. Questions to be addressed include: • What kind of data is needed to be able to monitor the progress of the actions implemented? • Is all the information needed readily available and/or how great is the administrative burden in obtaining them? • What will the monitoring data and evaluation findings be used for? • To what extent do monitoring/evaluation structures already exist? Does new capacity need to be put in place? • Is the baseline situation sufficiently well known or will further data collection be necessary once the proposal has been adopted? • Who are the key actors in providing and using such information? (e.g. the Commission, Member States, intermediaries such as Agencies, operators/beneficiaries, etc.)? • In general terms, what will be the roles of these actors? How will information be shared and eventually aggregated? • What will be the additional use for gathering this information?

Conclusions and recommendations

Based on the analysis the contractor shall provide

1. The assessment of a set of different policy options for implementing Science in Society actions in a context beyond FP7.

2. A set of indicators to be used for future impact assessments of science in society activities.

For the studies to be consulted in the context of the analysis, please refer to chapter 3.1.4.

A.3.3 Methodology

On a general level the study must be designed and conducted in a way that the results are supported by evidence and rigorous analysis. The tenderer will present in its offer the methodology to be used for the evaluation and how different approaches could be combined best. To cover the issues and questions described, some specific modules however shall be integrated:

Part 1 Interim evaluation of Science in Society Actions

Taking the relevant documents e.g. the Specific Programmes and Work Programmes as a basis, the tenderer will have to provide a summary of the intervention logic of the Programme and fine-tune it with the responsible Commission Services in the context of the kick-off meeting.

In addition to extensive desk research the following modules have to be considered:

• for tracking the development of the programme and its participants: a network analysis;

• for the projects funded within the S&S/SiS Programme: an adequate sample of projects has to be selected for a more detailed analysis, including their self-assessment of achievements. It is expected that this sample covers at least 25% of the funded projects and contains a balanced distribution with view to FP6 and FP7 projects as well as the different areas of activities within


S&S and SiS activities. Whether this is done via surveys and complementary interviews or other forms of data collection shall be reasoned by the tenderer;

• for the analysis of the integration of SiS aspects in other parts of the Cooperation Programme: a more general analysis on the level of the Specific Programme, Work Programmes and Calls should be combined with a Case study approach describing specific projects in these areas.

Part 2 Assessment of future options for Science in Society actions on EU-level

Based on the results of the interim evaluation and taking into account the relevant documents available this part of the study will focus on the assessment of different policy options. It is expected that in this context

• The development of policy options will be supported by prospective analysis, helping to identify important issues and elements for a future programme

• A consultation (e.g. via questionnaire) of a representative sample of key actors from the following communities will take place: researchers, public authorities, industry, Civil society organisation and media related to SiS;

• An impact analysis of the policy option will have to include a cost-benefit analysis, cost effectiveness analysis taking into account the specifics of public investment; an assessment of potential economic, social and environmental impacts will complement the analysis; where possible quantifications should be made – in cases where this is not possible or proportionate (which has to be explained) the analysis can draw on qualitative descriptions and analysis.

• Benchmarks will have to be established against which the performance of future activities can be measured. This will lead to the development of a set of indicators suitable to assess the impact of different activities in the context of SiS.

An appropriate geographical as well as expert and project diversification has to be sought for all steps of the analysis.

A.3.4 Data

Data Sources

The tender must specify the data sources that would be appropriate for inclusion in the study and describe how they shall be used. In addition information on the concrete terms of access to that data has to be given (e.g. envisaged costs, time-frame, nature and quality of the data). In cases where the tenderer foresees problems obtaining the necessary data, for example, because they might not be directly available, incomplete, contained in private databases or for any other reason, the tenderer must mention this and suggest how these problems could be overcome and what alternative ways of analysis are foreseen.

The contractor will be authorised to use the data related to FP7 projects from the CORDA database26 of DG RTD. In addition data available from the MASIS project, monitoring national activities in the area of Science in Society will be made accessible to the contractor.

Available Information

Information on research activities, the European Research Area, the Framework Programmes and the specific research programmes is available on the CORDIS web site at the following address: http://www.cordis.lu/en/home.html, and on the EUROPA web site at: http://europa.eu.int/pol/rd/index_en.htm, http://europa.eu.int/comm/research/index_en.html

The FP7 monitoring reports provide information on a yearly basis on several indicators related with the implementation of FP7 http://ec.europa.eu/research/index.cfm?pg=reports.

26 CORDA, the common research data warehouse, is the Framework Programmes' (FP) central repository of data collected and/or derived during the course of FP implementation. It is managed by RTD A6.02 as the system owner and RTD R4 as the IT system supplier. This portal is a first attempt to organize the wealth of information contained in CORDA, facilitate user access to this information, and provide user-friendly reporting tools on FP implementation and management including data quality. The information included in the system relate to the submissions in a call for proposals, the evaluation, ranking and decision of the received proposals. CORDA is currently running as an open "beta-test" version.


The site that presents FP7 contributions to the objectives of the renewed sustainable development strategy provides information of the activities carried out by the different themes responding to these objectives http://ec.europa.eu/research/sd/index_en.cfm.

Licences

The tenderer has to prove that the correct licenses to use the databases that will be used to do the work are at the tenderer's disposal. The tenderers are reminded that it is their responsibility to ensure that they hold the correct licences (direct licence or through a service provider such as a specific host or university) to perform and disseminate the work. The data obtained in the context of the envisaged contract will be in detailed form for Commission internal use and in aggregated form for external publications.

A.3.5 Working language

All deliverables and reports have to be provided in English.

A.3.6 Duration of the contract or period of execution of the contract

The duration for the tasks shall be 10 months from the date of signature of the contract by the last of the contracting parties.

A.4 Deliverables and meetings

A.4.1 Deliverables

Deliverable 1 (D1) At the latest at the end of month 1 after signature of the contract by the last of the contracting parties

The contractor shall present an Inception report (no more than 10 pages) outlining a detailed description of the assessment strategy, including the final approach to methodology, and practical steps to be followed in the implementation of the methodology as proposed in the tender. The Commission will approve the Inception report or make comments within 15 calendar days. In the last case, the contractor shall have 15 calendar days to submit a revised version.


The contractor shall present a first Intermediate report based on the results obtained so far. This report shall include information on the approach, the methodology implemented, the work schedule and possible deviations, problems and next steps etc. Specifically this report shall contain first results of Part 1 of the study (interim evaluation) and preparatory steps for Part 2 of the study (assessment of future options). The first intermediate report shall be not more than 30 pages of written text, excluding the annexes. The Commission will approve the first Intermediate report or make comments within 20 calendar days. In the last case, the contractor shall have 15 calendar days to submit a revised version.


Part 1: Interim evaluation of Science in Society actions within the transition phase from FP6 to FP7 and FP7 The Draft Final report shall be presented to the Commission taking into account the Commissions remarks to the Intermediate report. The contractor must provide a study of maximum 70 pages plus Annexes. The structure of the Draft Final report shall be worked out according to the following rough structure: Executive Summary (3 pages) 1. Frame of reference for evaluating Science in Society actions 2. Analysis of the transition phase between FP6 and FP7 3. Assessment of achievements in individual thematic areas 4. Analysis of Science in Society activities within the Cooperation Programme 5. Overall assessment of the Science in Society Programme 6. Conclusions and recommendations. Annexes I. Detailed description of tasks, work organisation and methodology; II. Detailed presentation of the data, statistics and analysis of FP6/FP7 Science in Society actions. The Commission will approve or make comments to the Draft final report within 20 calendar days. In the last case, the contractor shall have 15 calendar days to submit a revised version.



The contractor shall present a second Intermediate report based on the results obtained so far with view to the assessment of future options for science in society actions on EU level; This report shall include information on the approach, the methodology implemented, the work schedule and possible deviations, problems and next steps etc. The second Intermediate report shall be not more than 30 pages of written text, excluding the annexes. The Commission will approve the second Intermediate report or make comments within 20 calendar days. In the last case, the contractor shall have 15 calendar days to submit a revised version.


Part 2: Assessment of future options for Science in Society actions on EU-level The Draft final report shall be presented to the Commission taking into account the Commissions remarks to the Intermediate report. The contractor must provide a study of maximum 70 pages plus Annexes. The structure of the Draft Final report shall be worked out according to the following rough structure: Executive Summary (3 pages) 1. Scope and level of analysis of the assessment of future options for science in society actions on EU level 2. Problem definition and justification of the intervention for the EC 3. Policy objectives 4. Policy options 5. Likely economic, social and environmental impacts 6. Comparison of the options 7. Alternative delivery mechanisms 8. Set of indicators for assessing Science in Society actions in the future 9. Conclusions and recommendations. Annexes I. Detailed description of tasks, work organisation and methodology; II. Detailed presentation of the data and data gathering process (including potential consultation processes) statistics and analysis underpinning the assessment of impacts; III. Detailed description of a set of indicators to be used in future evaluation and assessment exercises for Science in society actions The Commission will approve or make comments to the Draft final report within 20 calendar days. In the last case, the contractor shall have 15 calendar days to submit a revised version.


All quantitative data and other factual information collected under this contract will be delivered in the form of a Facts Database, including the accompanying queries to estimate the indicators, as specified in task 5. The data will be the property of the Commission. The Commission will approve the Database or make comments within 20 calendar days. In the last case, the contractor shall have 15 calendar days to submit a revised version.


The contractor will prepare a Presentation of the results of the study, (in PowerPoint or equivalent software), with explanatory notes and will present the results of the evaluation study to the Commission in Brussels in a dedicated 1/2 day meeting. The Commission will approve the Presentation of the results of the study or make comments within 20 days. In the last case, the contractor shall have 15 days to submit a revised version.


Taking into account the Commission's comments, the Final report shall be delivered to the Commission in 3 bound copies in the form proposed in the tender, as well as in electronic format (word and pdf – or equivalent software) suitable for paper and web dissemination. The Commission will approve the Final report or make comments within 20 calendar days. In the last case, the contractor shall have 15 calendar days to submit a revised version.


All documents under the contract are to be submitted by the contractor in English.

All reports, documents and supporting information must be made available in electronic format such as Microsoft Word (2003), Powerpoint (2003), Access (2003) and Excel (2003) (or equivalent) for the Microsoft Windows XP operating system as agreed with the Commission Services.

A.4.2 Meetings with Commission services

An "ad-hoc Steering Group" comprising Members of different Commission Services will accompany and follow closely the implementation and progress of the studies. It will be composed of Commission staff with DG Research, Innovation and Science. It will monitor the preparation, progress and results of the study and meet with the contractor according to the indicative schedule presented below.

In drawing up the tender, the tenderer needs to bear in mind that the contractor is expected to take part in 4 (four) meetings which will take place on Commission premises in Brussels and at the expenses of the contractor:

• M1: Two weeks after the signature of the contract by the last contracting party, a Kick-off meeting of the contractor will be held on Commission premises in Brussels. The Contractor will prepare the agenda and supporting documents and send it to the Commission at least 5 calendar days before the meeting. The contractor will deliver to the Commission the minutes not later than 5 calendar days after the meeting. This meeting will be organised to fine-tune the evaluation approach and methodology to be used and to prepare the work to be carried out.

• M2: At the end of month 3, the contractor shall meet Commission representatives in Brussels for the first Intermediate Meeting to discuss progress achieved so far, intermediate results obtained and the steps planned for the continuation of the work. The meeting will give an opportunity to fine tune the approach with the contractor, assess the effectiveness of the methods applied and settle for the details of the remaining work. The agenda and supporting documents (including the interim report) shall be presented by the contractor not later than 10 working days before the meeting. The minutes shall be written by the contractor and presented to the Commission not later than 5 working days after the meeting.

• M3: At the end of month 7, the contractor shall meet Commission representatives in Brussels for the second Intermediate Meeting; goals and organisation as in M2

• M4: At the end of month 9, the Final Meeting (maximum 0,5 -1 day) with the Commission and, as appropriate, other external stakeholders will he held in Brussels to present the Draft final report. The agenda and supporting documents (including the Draft final report) will be submitted by the Contractor to the Commission at least 10 working days before the meeting.

The contractor will deliver the minutes at the latest after 5 working days of the meeting.

Timelines for Deliverables, Meetings and Payments for each of the lots

Deliverables (D), Meetings (M), and Payments (P) Deadline (Month)

M1: Kick-off meeting with the Commission in Brussels D1: Inception report

1

D2: First Intermediate report M2: First Intermediate meeting

End of month 3

P1: First payment after approval of 1st Intermediate report by the EC 4

D3: Draft final report Part 1 of the study (interim evaluation) End of month 5

D4: Second Intermediate report M3: Second Intermediate meeting

End of month 7

P2: Interim payment after approval of 2nd Intermediate report by the EC 8

D5: Draft final report Part 2 of the study (assessment of future options) D6: Facts Data base D7: Presentation of the study results M4: Presentation of the study to the EC

9

D8: Submission of the final report 10

P3: Payment of balance


Annex B - Questionnaire for SaS/SiS project participants

B.1 Introduction

This questionnaire is aimed at all participants in the Science in Society (SiS) actions of the European Union’s Sixth and Seventh Framework Programmes (FP6/7).

The data collected will form an integral part of an interim evaluation and assessment of future options for Science in Society actions, which is being undertaken on behalf of the European Commission's Directorate General for Research and Innovation (DG RTD).

This is a voluntary exercise. However, we hope that all participants will find the time to look at the questionnaire and to answer as many of the questions as possible. Thank you in advance for your participation and input to this study.

Please click the 'next' button below to start the survey.

When answering the questions we are asking respondents to represent the views of their research group or organisation as appropriate.

The survey consists of 34 questions and we estimate that it will take around 20 minutes to complete. All individual answers and comments will be treated as strictly confidential and non-attributable. Your answers will be saved automatically, and you can leave the questionnaire at any time and return to it later via the URL contained in the email that we sent you.

Please note that in parallel to this survey, Fraunhofer ISI is undertaking a public consultation process to identify future options for the Science in Society programme. You can contribute to this process by clicking on the link at the end of this questionnaire.

B.2 Basic information about you and your project

A number of questions in this survey ask about the Science in Society project you participated in.

If you have participated in more than one Science in Society project, please answer the questions in relation to the project that you feel is most is most significant in terms of its (potential) wider benefits and impacts.

1. Please enter the title or acronym of your project (just one):

2. Has this project finished?

• Yes – project completed • No – project ongoing

3. Which of the following options best describes your organisation: (tick one) (9)

• A research organisation • A non-research organisation

4. Would you describe your organisation as a Civil Society Organisation*? (7)

• Yes • No • Don’t know

* Civil Society Organisations (CSOs) are defined as organisations that are non-governmental, not-for-profit, not representing commercial interests, and that pursue a common purpose for the public interest. They are responsible for articulating the opinions of various social spheres, and include environmental groups, minority groups, consumer representatives and patient organisations, etc.

B.3 Partnership arrangements and roles

5. Please indicate the number of partners within the project:

• The total number of partners: • Number that you had collaborated with before the project: (7) • Number that you are likely to collaborate with again after the project: (41)


6. To what extent has (or will) your organisation play a role in the following aspects of the project: (no role, minor role, major role) (9)

• Defining the objectives of the project • Defining the content and scope of the project description of work • Defining the size and membership of the consortium • Carrying out the activities • Transferring or disseminating the results of the project • Exploiting the results of the project

7. Have (or will) you be involved in the project mainly as: (please select one)

• A research performer? • A knowledge provider? • A user of results? • A combination of the above? (9)

B.4 Project approach

8. To what extent has (or will) the project make advances in relation to new ways of undertaking or governing research activities in the following areas? (Not at all, minor advances, major advances) (35)

• Stakeholder involvement • Participatory processes • Impact on policy • Indicator development

9. To what extent would you say that this project has (or will) involve innovative approaches to the following: (not at all, to a small extent, to a large extent) (37)

• To research • To a multi-disciplinary approaches • To the involvement of different types of partners • To the involvement of stakeholders • To the dissemination of results • Other innovative elements (please specify)

10. To what extent would you say that the scope of your project: (not at all, to a small extent, to a large extent)

• Goes beyond a purely academic focus or context? (7) • Has an international focus? (15)

B.5 Dissemination and outreach

11. Which of the following dissemination methods/activities have taken place (or are planned) for the project: (tick all that apply) (11)

• Published articles in academic journals • Published articles for a general audience • Book (chapters) • Press releases • Media briefings • Articles in press / radio / TV • Other video / audio • Online information • Events (workshop, conference, information day, etc.) • Other dissemination methods (please specify)

12. To what extent have (or will) the project results be disseminated to the following: (not at all, to a small extent, to a large extent) (11, 40, 39)

• Within the European Commission? • To the general public? • To non-academic audiences? • Internationally?


B.6 Users and beneficiaries

13. Which of the following groups do you consider to be the main intended ‘users’ or beneficiaries of the project and its results? (33, 41)

• Universities and research organisations external to the project • Private industry • Public administration and / or public service organisations • Policymaking and regulatory bodies • User or consumer organisations • Civil Society Organisations (CSOs) • National research programmes • International research programmes • Other research programmes • The media • Citizens • Pupils, teachers or schools • Museums • Other fora and consortia • Other types of organisations (please specify)

14. To what extent has (or will) the project have an impact on stakeholders outside of the project partnership? (not at all, to a small extent, to a large extent) (33)

• Please detail the main external stakeholders and briefly explain the actual or expected benefits:

B.7 Impact on SaS Objectives (FP6 Version)

15. To what extent has the project (or is it expected to) contribute towards the following overall goals and objectives of the programme? (not at all, to a small extent, to a large extent) (2, 16, 32)

• Bringing research closer to society

! Increasing the extent to which policy meets society's needs

! Increasing the extent to which policy making is soundly based on science

! Greater account being taken of the relationship between innovation and social change

! Greater account being taken of the concerns of civil society

• Ensuring responsible research and the application of science

! Ensuring scientific progress is in harmony with ethical principles

! Deepening the understanding of ethical issues

! Addressing issues of scientific uncertainty, risk governance and the precautionary principle

! Better policy making

• Increasing science-society dialogue

! Increasing public awareness of science and its impacts

! Increasing dialogue between citizens and the scientific community

! Raising awareness amongst scientists of the concerns / interests of citizens

! Promoting young people's interest in science

! Improving science education

! Increasing the uptake of scientific careers

• Boosting gender equality in research

! Stimulating the participation of women in science

! Fostering integration of the gender dimension in research

B.8 Impact on SiS Objectives (FP7 Version)

16. To what extent has the project (or is it expected to) contribute towards the following overall goals and objectives of the programme? (not at all, to a small extent, to a large extent) (2, 16, 32)

• Strengthening and improving the European science system


! Better access to expertise to underpin policies

! Improved use of scientific expertise in policy making

! Increased trust and self-regulation in the scientific community

! Increased public debate on information dissemination

• Broadening engagement to anticipate and clarify political, societal and ethical issues

! Broader engagement on science-related questions

! Better conditions for informed debate on ethics and science

! Increased discussion of societal aspects within the research community

• Communication channels for the public / policy makers to engage with science / scientists

! Increased cooperation between scientists and the media

! More reliable / timely scientific information provided to the media

! Increased European dimension at public science events

! Improved promotion of science by audio-visual means

! Improved promotion of excellent trans-national science prizes

! Increased understanding between science and wider audiences

• Boosting gender equality in research

! A strengthened role for women in research or research bodies

! An enhanced gender dimension in research

! The mainstreaming of gender in research policy or programmes

• Improved understanding of the place of Science and Technology in society

! The formation of networks to structure research and debates

• Encouraging young people into science

! Attracting people into scientific careers

! Fostering links across generations

! Raising the level of scientific literacy

! Adapting science teaching to young audiences

! Supporting science teaching

! Developing links between schools and professional life

! Bringing together scientists and aspiring scientists

• Supporting university reforms

! Improved conditions for efficient university research

! The establishment structured partnerships within business

! Reinforced knowledge sharing between universities and society

B.9 Other impacts

17. To what extent has the project (or is it expected to): (not at all, to a small extent, to a large extent)

• Achieve its specific ‘project’ objectives? (30) • Contribute towards the issues requested in the Work Programme and call? (31)

18. Please explain any benefits of the project that have been realised (or are expected) in the following specific areas:

• Scientific advice and decision-making processes at different geographical levels? (17) • Policy (development) in SiS at different geographical levels or multiple levels? (6, 19) • SiS communities and landscapes at national level? (24) • Shaping the European Research Area (ERA)? (20)


B.10 Conclusions on the project

19. Please indicate the extent to which the following factors has had a negative impact on project progress or the exploitation of the results of your project (negative impact on progress, negative impact on exploitation, no problems experienced):

• Insufficient levels of funding (28) • Lack of interest or commitment among partners • Insufficient knowledge or competence among partners • Insufficient levels of human resource • Unclear objectives or poor project planning • Technical difficulties • Problems managing a large team across several countries • Slow or bureaucratic Commission processes / procedures • Shifting objectives or priorities within the project team • Other (please specify)…

20. Please indicate the scale of positive impact the project has had (or is expected to have) on your own organisation in terms of each of the following types of benefit: (no impact, small impact, large impact) (41)

• Improved understanding of specific issues / problems • Improved internal knowledge and capabilities • Improved relationships and networks • Improved ability or capacity to conduct R&D • Enhanced reputation and image • Improved competitive position

21. From your perspective, how have the costs and benefits of participation in the project compared: (-3 costs outweigh benefits to +3 benefits outweigh costs)

B.11 Counterfactual

22. To what extent do you believe that the activities undertaken through the project would have taken place in any form without FP funding? (1, 29)

• Not at all • To a small extent • To a large extent

23. If the project would have gone ahead in some form anyway, would this have been:

• With reduced funds • With less satisfactory outputs • With lower impacts • With fewer partners • With fewer ‘non-academic’ partners (7) • With less ambitious objectives • With national partners only (i.e. not European) • The project would have taken longer

24. Are you aware of another body or programme that might have funded this project (or something very similar)? Please provide brief details: (1, 49)

B.12 Sustainability and the Future

25. Beyond the FP funding period: (yes, no) (41, 34)

• Is there a strategy for continuation of the work of the project? • Will further use be made of the acquired knowledge? • Will the partnership continue? • Will the project continue to have an impact? • Will follow-on activities take place?

26. Do you expect participation in the project to improve your organisation’s prospects for any of the following (not at all, to a small extent, to a large extent)? (17)


• Conducting follow-on research projects in-house • Participating in national research projects • Participating in FP funded SiS projects • Participating in research projects funded in other Commission programmes or initiatives • Participating in other international research projects

B.13 Instruments and mechanisms

27. Did the instrument (e.g. Collaborative project, Co-ordination action, etc.) used for your project: (not at all, to a small extent, to a large extent)

• Provide a suitable platform for the work being undertaken? • Provide a suitable platform for the preferred size and shape of the consortium? • Encourage you to participate in the programme? (8) • Encourage a wider group of stakeholders to be involved? (8) • Encourage a different type of involvement of non-researchers? (9)

28. To what extent do you feel that the instrument was appropriate to the focus and objectives of the project? (26)

• Not at all • To a small extent • To a large extent

29. Please explain why another instrument would have been more appropriate for the project (12)

30. To what extent was the level of funding for the project appropriate for the achievement of the stated objectives? (not at all appropriate, somewhat appropriate, wholly appropriate) (28)

B.14 The SiS programme

31. To what extent would you agree with the following statements about the SiS programme: (not at all, to a small extent, to a large extent)

• The goals set for the programme are realistic? (2) • The SiS programme is clearly structured? (3) • The objectives of the programme are set out clearly? (4) • The programme budget is suitable for the tasks envisaged? (28)

32. To what extent did the following factors positively influence your organisation’s willingness to participate in the SiS programme? (not at all, to a small extent, to a large extent)

• The scale of the project (6) • The involvement of non-academic partners in the consortium (7) • The breadth of focus of the research (10) • The transversal focus (10) • The funding available (28) • The European dimension • The international dimension • The research instruments available • The duration of the project • The project consortium • The participation rules • The rules for funding

B.15 Programme coverage

33. If you also have experience or knowledge of the Science and Society programme in FP6 / Science in Society programme in FP7, do you believe that the changes introduced in FP7 SiS: (yes, no, don’t know)

• Enable more strategic projects (6) • Enable projects with more pronounced policy links and impacts (6) • Have a positive influence on your organisation’s ability or willingness to participate? (8) • Better aligned with national and sectoral capacities across Europe (48)


34. Are you aware of any duplication of effort in SiS research activities between EU and national level that might seriously challenge the impact and relevance of EU-level activities? Explain (1, 49)

35. Do you have any suggestions for topics that you believe should be given (more) attention in FP7 and beyond in the SiS area? (1, 12, 13)

36. Please provide any additional comments you may have relating to the FP7 Science in Society programme. Where possible please provide specific suggestions as to how the programme could be strengthened in the future.

B.16 Thank you

Thank you for taking the time to complete this questionnaire. Your answers have been saved automatically and you can click 'done' below to leave.

Please note that as part of the interim evaluation and assessment of SiS actions, Fraunhofer ISI is undertaking a public consultation exercise (a Delphi survey) aimed at identifying future options for the Science in Society programme. The consultation process is open until Monday 9th April and you can contribute via the following link: Delphi Survey


Annex C - Questionnaire for FP7 Cooperation programme coordinators

C.1 Introduction

This questionnaire is aimed at all coordinators of Cooperation Programme actions in the European Union’s Seventh Framework Programme (FP7). It focuses on the extent to which Science in Society activities and principles (e.g. relating to ethics, gender, engagement, etc.) have been embedded and tackled within research projects.

The data collected will form an integral part of an interim evaluation and assessment of future options for Science in Society actions, which is being undertaken on behalf of the European Commission's Directorate General for Research and Innovation (DG Research).

This is a voluntary exercise. However, we hope that all participants will find the time to look at the questionnaire and to answer as many of the questions as possible. Thank you in advance for your participation and input to this study.

Please click the 'next' button below to start the survey.

When answering the questions we are asking respondents to represent the views of their research group or organisation as appropriate. Respondents may answer from their own personal perspective if they feel unable to talk on behalf of their organisation or research group.

The survey consists of 15 questions and we estimate that it will take around 10-15 minutes to complete. All individual answers and comments will be treated as strictly confidential and non-attributable. Your answers will be saved automatically, and you can leave the questionnaire at any time and return to it later via the URL contained in the email that we sent you. If you are unable or do not wish to answer any of the questions please leave these blank and move on to the next question.

C.2 Preliminary questions

A number of questions in this survey ask about the FP7 Cooperation programme project in which you were the coordinator.

If you were the coordinator in more than one FP7 Cooperation programme project, please answer the questions in relation to just one of these projects (entering the name of this project below).

37. Please enter the title or acronym of your project (just one):

38. Has this project finished?

• Yes – project completed • No – project ongoing

There was an obligation placed on all projects funded under the FP7 cooperation programme to consider issues relating to science and society as a part of their project design (e.g. issues relating to ethics, gender, engagement, education and training).

39. Were you aware of this obligation?

• Yes – go to ‘obligations’ section • No – go to ‘report on societal implications’ section

C.3 Obligations

40. To what extent would you agree with the following statements about the science and society obligations placed on projects in the Cooperation Programme: (not at all, to a small extent, to a large extent)

• They are clear • They are meaningful • They are legitimate • They are flexible


41. To what extent did these obligations lead to you taking greater consideration of each of the following issues (above and beyond what would have happened anyway): (no change, some extra consideration, much greater consideration)

• Ethical issues • Gender issues • Science, education, training and career issues • Communication issues (engaging with the public, use and dissemination) • Legal issues • Political issues • The participation of Civil Society Organisations (CSOs) • The involvement of citizens

42. What was the main aspect of your project that changed as a result of the obligations?

C.4 Report on societal implications

43. In relation to your project, have you previously completed an online ‘project final report’, including the section ‘report on societal implications’?

• Yes • No • Don’t know

44. If ‘No’, why not?

• I was not aware of this report • I am aware of the report, but have not been asked to complete it • The project is ongoing and I do not need to complete the report yet • It is not a requirement for this project • Other (please specify)

C.5 Integration issues

45. How effective do you believe your project has been in integrating the following aspects within the project: (not at all effective, somewhat effective, very effective)


46. How much (including the value of time spent, as well as paid-out costs) do you estimate your project (once it is completed) will have spent on considering and dealing with each of the following: (!) [please enter a ‘0’ for no spend, and leave the cell blank where the figure is unknown]:


C.6 Benefits

47. To what extent did consideration of the following issues lead to benefits for your project? (not at all, to a small extent, to a large extent)

• Ethical issues


• Gender issues • Science, education, training and career issues • Communication issues (engaging with the public, use and dissemination) • Legal issues • Political issues • The participation of Civil Society Organisations (CSOs) • The involvement of citizens

48. What has been the main benefit to the project from consideration of the SiS issues mentioned above? Please specify:

49. Has consideration of SiS issues changed your perspectives in any way? Please explain how?

50. Have there been any drawbacks from considering these issues? Please specify:

C.7 Conclusions (Q42-44)

51. To what extent do you believe it was important to consider the following issues as a part of any research project: (not at all important, slightly important, very important)


C.8 Thank you

Thank you for taking the time to complete this questionnaire. Your answers have been saved automatically and you can click 'done' below to leave.


Annex D - List of interviewees

D.1 Stakeholder Interviews (53)

D.1.1 Commission officials responsible for the SiS programme (7)

Name Job Title Organisation

Mr Peteris Zilgalvis Head of Unit for ICT for Health European Commission

Mr Gilles Laroche Head of Unit European Commission

Ms D. Lagiou Research Programme Officer European Commission

Mr P. Galiay Policy Officer European Commission

Ms V. Willis-Mazzichi Head of Sector European Commission

Mr I. Karatzas Head of Sector European Commission

Mr Johannes Klumper Ex policy officer European Commission

D.1.2 Commission officials responsible for Cooperation programme areas (7)

Name Job Title Organisation Programme Area

Alessio Vassarotti Policy officer DG Research DDG 3 – Unit E.1 Food, agriculture and fisheries, biotechnology

Barbara Kerstiens Head of sector - public health and health services research

DG Research DDG3 - Unit F.3 Health

Mihaela Williams Project Officer - EU policies DG Research DDG 3 – Unit H.1 Transport

Manuela Soares Director DG Research DDG 3 – Unit I Environment

Mats Ljungqvist Policy Officer - Project Officer DG ENTR – Unit H.2 Space

Eva-maria Engdahl Policy Officer DG ENTR – Unit H.3 Security

Nicholas Deliyanakis

Deputy Head of Unit Head of Unit - Policy and Strategy acting

DG Research DDG3 – Unit G.1 Nanosciences, nanotech, materials and new production techniques

D.1.3 Commission officials responsible for related policy areas (3)

Name Job Title Organisation

Octavio Quintana Trias Director DG RTD, Directorate B

James Gavigan Head of Unit for ERA DG RTD, Directorate B

Jean-Caude Burgelman Head of Unit for Relations with stakeholders (ERIAB) DG RTD, Directorate C

D.1.4 Advisory / expert group members that have advised on SaS/SiS programme design (4)

Name Job Title Organisation Country

Mariachiara Tallacchini Professor Law Faculty, Catholic University of Piacenza Italy

Andrew Stirling Professor University of Sussex United Kingdom

Suzanne De Cheveigne Directrice de recherche, CNRS CNRS France

Annette Williams Director of Governance UKRC UKRC United Kingdom


D.1.5 National representatives (30)


Ylli Pango NCP Akti (Albanian Agency for Research, Technology and Innovation) Albania

Michalis Tzatzanis NCP FFG (Austrian Research Promotion Agency) Austria

Nina Sober NCP University of Sarajevo Bosnia and Herzegovina

Ivailo Dimitrov NCP Bulgarian Academy of Sciences Bulgaria

Constantina Makri NCP Research Promotion Foundation Cyprus

Michal Pacvo! NCP Technology Centre of the Academy of Sciences of the Czech Republic Czech Republic

John D. Olsen NCP The Faroese Research Council Faroe Islands

Risto Alatarvas NCP Akademy of Finland Finland

Sophie Tocreau NCP Ministère de l'Enseignement Supérieur et de la Recherche France

Armin Grunwald Professor/director Institute for Technology Assessment and Systems Analysis (ITAS), Technology Assessment Office of the German Bundestag

Germany

Jan Mørch Pedersen NCP Greenland Business A/S u.s. Greenland

Orsolya Tóth NCP / PC National Innovation Office Hungary

Adalheidur Jonsdottir NCP RANNÍS Iceland

John Denari NCP Irish Research Council for Science, Engineering and Technology Ireland

Frank A. Heeb NCP Office (ministry) of Economic Affairs Liechtenstein

Agne Paleviciute NCP Research Council of Lithuania Lithuania

Jean-Michel Ludwig NCP LUXINNOVATION EIG - National Agency for Innovation and Research Luxembourg

Anthea Fabri NCP Malta Council for Science and Technology Malta

Lidia Virtan NCP Academy of Sciences Moldavia

Yvonne Schaap-Koenen PC Representative Ministry of Education, Culture and Science (OCW) Netherlands

Gunnar Bolstad NCP Research Council of Norway Norway

Ricardo Migueis NCP CIÊNCIA VIVA - National Agency for scientific Culture and Technology Portugal

Nada Milosevic NCP Ministry of Science Serbia

Kristína Jen!íková NCP Neulogy, a.s. Slovak Republic

Radojka Vercko NCP Ministry of Higher Education, Science and Technology Slovenia

Regina Schneider NCP Euresearch (Swiss support structure) Switzerland

Seda Göksu NCP TUBITAK Turkey

Alexa Mills NCP Economic and Social Research Council United Kingdom

Chloe Sheppard Special Projects Manager Wellcome Trust United Kingdom

Kerry Leslie Strategy Unit RCUK United Kingdom

D.1.6 Other interviews (2)


Marcos Pére Project partner in CASC project - to discuss incentives for public authorities to participate in SiS

Museos Cientificos Coruneses Spain

Malcolm Harbour MEP, Vice Chair - STOA European Parliament United Kingdom


D.2 Case Study Interviews (62)

Project Acronym Interviewee name Affiliation Country

2WAYS Peter Rebernik EUSCEA (European Science Events Association) Austria

ACUMEN Paul Wouters Universiteit Leiden Netherlands

ALACS Dirk Tänzler Universität Konstanz Germany

CIT-PART Erich Griessler INSTITUT FUER HOEHERE STUDIEN UND WISSENSCHAFTLICHE FORSCHUNG Austria

CIVI.NET Claudia Sattler Leibniz-Centre for Agricultural Landscape Research (ZALF) Germany

COBRA Jay Mistry Royal Holloway, University of London UK

COMBIOSERVE Emily Caruso Global Diversity Foundation United Kingdom

COMET-LA María Del Mar Delgado Serrano Universidad de Córdoba Spain

CRÊPE Les Levidow Open University United Kingdom

EARTHWAKE Raymond Seltz Euroscience France

EARTHWAKE Andrew Millington Omni Communications Ltd United Kingdom

ECB Alexa Joyce European Schoolnet Belgium

EPOCH Ruud ter Meulen Uni of Bristol United Kingdom

EPWS Claudine Hermann European Platform of Women Scientists (EPWS) France

ESCITY Vladimir de Semir Pompeu Fabra University Spain

ESCITY Gema Revuelta Pompeu Fabra University Spain

ESOF Raphaela Kitson-Pantano ESOF France

ESTABLISH Eilish McLoughlin Dublin City University Ireland

ESTABLISH Sarah Brady Dublin City University Ireland

ETHICSWEB Dirk Lanzerath Rhein. Fr-Wilh University Bonn Germany

EUCYS 2011 Pekka Pellinen Finnish Association of Graduate Engineers (TEK) Finland

EUIMA John Smith European University Association Belgium

EURECNET Dirk Lanzerath Rhein. Fr-Wilh University Bonn Germany

FIBONACCI Xavier Person Fondation La main à la pâte France

FIBONACCI Janick Rajoharison Fondation La main à la pâte France

GAP2 Steven Mackinson CEFAS, The Secretary of State forn Environment, Food and Rural Affairs United Kingdom

GAP2 Tracy Maxwell CEFAS, The Secretary of State forn Environment, Food and Rural Affairs United Kingdom

GAP2 Michael Clarke CEFAS, The Secretary of State forn Environment, Food and Rural Affairs United Kingdom

GENEBANC Kris Dierickx Katholic Univ Leuven Belgium

GENIS LAB Barbara de Micheli Fondazione Giaocomo Brodolini (FGB) Italy

GENSET Elizabeth Pollitzer Portia Ltd United Kingdom

GEO FAIR TRADE Juan Manuel Mora Rey Cahmbre de Commerce et d'Industrie du Gers France

GEST Miltos Ladikas Univ of Central Lancashire United Kingdom

LINDAU NOBEL/LIN10 Wolfgang Huang

Executive Secretariat, Council for the Lindau Nobel Laureate Meetings Germany

MASIS Niels Meijlgaard Aarhus University Denmark

NANOCAP Pieter van Broekhuizen IVAM, Univ of Amsterdam Netherlands

NANOCODE Elvio Mantovani Ass Ital per la Ricera Indus. Italy

NECOBELAC Paola de Castro Istituto Superiore di Sanità Italy

PACITA Lars Klüver Danish Board of Technology Denmark

PANGEO Ren Capes FUGRO NPA Ltd United Kingdom

PASSO Silvia Gaggi Instituto di studi per l'integrazione dei sistemi (ISIS) Italy


Project Acronym Interviewee name Affiliation Country

PATS Leon Hempel Techn Univ Berlin Germany

PERARES Henk Mulder University of Groningen Netherlands

PLACES Antonio Gomes da Costa Ecsite - The European Network of Science Centres and Museums Belgium

PRESCIENT Michael Friedewalt Fraunhofer ISI Germany

PRIMAS Katja Maaß Pädagogische Hochschule Freiburg Germany

S-TEAM Peter Gray - United Kingdom

SET-ROUTES Julia Willingale-Theune European Molecular Biology Laboratory (EMBL) (affiliation at the time the project was carried out) Germany

SHE FIGURES Síle O'Dorchai Department of Applied Economics (Universite Libre de Bruxelles) Belgium

SHEMERA Maria Caprile Elola-Olaso

Centre for European Initiatives and Research in the Mediterranean (CIREM) Spain

SIAMPI Jack Spaapen KNAW (Royal Netherlands Academy of Arts and Sciences) Netherlands

SIS CATALYST Tricia Jenkins The University of Liverpool United Kingdom

SISOB Beatriz Barros University of Malaga Spain

SOAP Salvatore Mele CERN Switzerland

TANDEMPLUS IDEA Carmen Leicht-Scholten

Rheinisch-Westfaelische Technische Hochschule (RWTH) Aachen Germany

TECH-CLINIC SST José Pedro Soutinho INOVAMAIS - Sevicos de consultadoria em inovacao tecnologica s.a Portugal

TWIST Sheena Laursen Experimentarium (Centre for Promoting Natural Science and Modern Technology) Denmark

ULAB Javier Lopez Universidad Politecnica de Madrid Spain

UPDATE Päivi Fadjukoff Jyvanskylan Yliopisto Finland

WHIST Giovanna Declich Assembly of Women for Development and the Struggle against Social Exclusion (ASDO) Italy

WiST Pierre Bismuth Schlumberger United States

WiST Claartje Vinkenburg Amsterdam Center for Career Research (VU University Amsterdam) Netherlands

technopolis |group| United Kingdom 3 Pavilion Buildings Brighton BN1 1EE United Kingdom T +44 1273 204320 F +44 1273 747299 E [email protected] www.technopolis-group.com

Documents

Interim evaluation & assessment of future options for ... · Interim evaluation & assessment of future options for ... society and the citizen in Europe"3 in ... Interim evaluation