The Evaluation Partnership - European Commission · Evaluation of Information activities of the ERA -MORE network Final Report The Evaluation Partnership Limited and Deloitte December

The Evaluation Partnership

European Commission Directorate-General Research Evaluation of Information activities of the ERA -MORE network Final Report December, 2008

Evaluation of Information activities of the ERA-MORE network Final Report The Evaluation Partnership Limited and Deloitte December 2008

Page 2

Evaluation of Information activities of the ERA -MORE network Final Report December, 2008

Submitted to: European Commission Directorate General Research Unit C.4. B-1049 Brussels Belgium Submitted by:

The Evaluation Partnership Limited (TEP) and Deloitte

Business Address: TEP

Rosedale House, Rosedale Road Richmond - Surrey TW9 2SZ United Kingdom

Deloitte

Berkenlaan 8c 1831 Diegem Belgium


Page 3

Disclaimer

The views and comments expressed in this text are the responsibility of TEP and Deloitte and do not necessarily reflect the opinion of the European Commission.


Page 4

Acknowledgements

This assignment was conducted by a team of TEP and Deloitte, headed by John Watson and Luc Chalsège and with the support of Lydia Da Silva Gaspar, Laura Delgado Garcia, Mirja Gutheil, Vanessa Ludden, Wim Palmers and Ben Ward.

The production of this report would not have been possible without the efforts of the many interviewees that we met during our fieldwork. The authors would like to express their gratitude to all of them.

Finally, the evaluation team w ould like to thank the many Commission officials who have been helpful and co-operative in providing information and feedback during the course of the assignment.


Page 5

TABLE OF CONTENTS

1.0 EXECUTIVE S UMMARY................................ ................................ ................................ ............................... 10

1.1 INTRODUCTION ................................ ................................ ................................ ................................ ................. 10 1.2 KEY CONCLUSIONS AND RECOMMENDATIONS ................................ ................................ ............................... 12 CONCLUSIONS ON THE EFFECTIVENESS OF THE MOBILITY CENTRES ................................ ................................ .......... 12 RECOMMENDATIONS ON TH E EFFECTIVENESS OF THE MOBILITY CENTRES................................ ................................ 13 CONCLUSIONS ON THE DISSEMINATION OF INFORMATION AND PROMOTION OF THE SERVICES TO TARGET GROUPS 15 RECOMMENDATIONS ON THE DISSEMINATION OF INFORMATION AND PROMOTION OF THE SERVICES TO TARGET GROUPS................................ ................................ ................................ ................................ ................................ ........... 16 CONCLUSIONS ON THE NATIONAL RESEARCHER’S MOBILITY PORTALS ................................ ................................ .... 17 RECOMMENDATIONS ON TH E NATIONAL RESEARCHER’S MOBILITY PORTALS................................ .......................... 17 CONCLUSIONS ON THE REPORTING................................ ................................ ................................ ................................ 18 RECOMMENDATIONS ON TH E REPORTING ................................ ................................ ................................ ..................... 19 CONCLUSIONS ON THE LINKS ESTABLISHED BETWEEN MOCS AND NATIONAL/REGIONAL/LOCAL AUTHORITIES AND SHARED INFORMATION O N FREQUENTLY ENCOUNTERED PROBLEMS IN THE CONTEXT OF NATIONAL/REGIONAL/LOCAL MOBILITY ................................ ................................ ................................ ....................... 20 RECOMMENDATIONS ON THE LINKS ESTABLISHED BETWEEN MOCS AND NATIONAL/REGIONAL/LOCAL AUTHORITIES AND SHARED INFORMATION ON FREQUENTLY ENCOUNTERED PROBLEMS IN THE CONTEXT OF NATIONAL/REGIONAL/LOCAL MOBILITY ................................ ................................ ................................ ....................... 21 CONCLUSIONS ON THE EXPERTISE AND ASSISTANCE PROVIDED BY MOCS THROUGH BHOS IN MOBILITY POLICY ISSUES TO THE NATIONAL/REGIONAL/LOCAL AUTHORITIES AND THE EUROPEAN COMMISSION IN CLOSE COOPERATION WITH THE STEERING GROUP FOR THE HUMAN RESOURCES AND MOBILITY OF RESEARCHERS ........ 23 RECOMMENDATIONS ON TH E EXPERTISE AND ASSISTANCE PROVIDED BY MOCS THROUGH BHOS IN MOBILITY POLICY ISSUES TO THE NATIONAL/REGIONAL/LOCAL AUTHORITIES AND THE EUROPEAN COMMISSION IN CLOSE COOPERATION WITH THE STEERING GROUP FOR THE HUMAN RESOURCES AND MOBILITY OF RESEARCHERS ........ 24 CONCLUSIONS ON SUSTAINABILITY ................................ ................................ ................................ .............................. 25 RECOMMENDATIONS ON SUSTAINABILITY ................................ ................................ ................................ .................... 25

2.0 INTRODUCTION ................................ ................................ ................................ ................................ ............. 26

3.0 SCOPE AND OBJECTIVES OF THE EVALUATION ................................ ................................ ............. 27

4.0 CONTEXT ................................ ................................ ................................ ................................ .......................... 29

4.1 BACKGROUND ................................ ................................ ................................ ................................ .................. 29 4.2 ERA-MORE NETWORK ................................ ................................ ................................ ................................ ... 31

4.2.1 Objectives ................................ ................................ ................................ ................................ ............................. 31 4.2.2 Mission ................................ ................................ ................................ ................................ ................................ ..32 4.2.3 Structure o f the ERA -MORE network ................................ ................................ ................................ .................32 4.2.4 Tasks................................ ................................ ................................ ................................ ................................ ......34 4.2.5 National Researcher´s Mobility Portal ................................ ................................ ................................ ...............35 4.2.6 Target Group ................................ ................................ ................................ ................................ ........................ 36 4.2.7 EC Support ................................ ................................ ................................ ................................ ........................... 36

4.3 SUSTAINABILITY OF THE NETWORK ................................ ................................ ................................ ................ 37

5.0 APPROACH AND METHODO LOGY................................ ................................ ................................ .......... 38

5.1 EVALUATION DESIGN ................................ ................................ ................................ ................................ ....... 38 5.2 EVALUATION TOOLS AND TECHNIQUES................................ ................................ ................................ ........... 38

5.2.1 Desk Review ................................ ................................ ................................ ................................ .......................... 38 5.2.2 Online survey ................................ ................................ ................................ ................................ ........................ 40 5.2.3 Interviews with stakeholders ................................ ................................ ................................ ............................... 46

5.3 OVERALL PLANNING OF THE EVALUATION PROJECT ................................ ................................ ...................... 47


Page 6

6.0 EVALUATION QUESTION: HAVE THE MOBILITY C ENTRES (M OCS) REACHED THEIR OBJECTIVES IN TERMS OF PROVIDING COMPREH ENSIVE AND UP -TO-DATE INFORMATION AND PERSONALISED ASS ISTANCE TO THE RESEA RCHERS IN MATTERS RE LATED TO MOBILITY? ................................ ................................ ................................ ................................ ................................ ..... 48

6.1 FINDINGS ................................ ................................ ................................ ................................ .......................... 48 6.1.1 Assessment of the alignment of the objectives of each MoC with the operational objectives as stated in the Declaration of commitment and effectiveness of the MoCs in providing comprehensi ve and up -to-date information and personalised assistance to the researchers in matters related to mobility ................................ ........48 6.1.2 Assessment of the mechanisms and means put in place by the MoCs to ensure t he delivery of comprehensive and up -to-date information and personalised information to the researchers ................................ ...59 6.1.3 Assessment of the relevance of the services and assistance provided to the rese archers ............................... 62

6.2 CONCLUSIONS ................................ ................................ ................................ ................................ .................. 66 6.3 RECOMMENDATIONS ................................ ................................ ................................ ................................ ........ 67

7.0 EVALUAT ION QUESTION: HAVE T HE MOBILITY CENTRES BEEN EFFECTIVE IN DISSEMINATING THE IN FORMATION AND PROMOT ING THEIR SERVICES T O THEIR MAIN TARGET GROUPS? HAVE THEY BEEN EFFECTIVE IN CREATING A NETWOR K OF USERS OF THEIR SERVICES? ................................ ................................ ................................ ................................ ....................... 69

7.1 FINDINGS ................................ ................................ ................................ ................................ .......................... 70 7.2 CONCLUSIONS ................................ ................................ ................................ ................................ .................. 77 7.3 RECOMMENDATIONS ................................ ................................ ................................ ................................ ........ 78

8.0 EVALUATION QUESTION: HAVE THE NATIONAL M OBILITY PORTALS PROV IDED COMPREHENSIVE AND UP -TO-DATE INFORMATION AND HAVE THEY BEEN EFFE CTIVELY MAINTAINED BY THE BH OS AND/OR MOCS? ................................ ................................ ................................ .. 79


9.0 EVALUATION QUESTION: HAVE THE BRIDGEHEAD ORGANISATIONS AND T HE MOBILITY CENTRES REP ORTED THEIR ACTIVITI ES AND RESULTS BACK TO THE COMMISSION IN A STRU CTURED AND EFFECTIVE WAY?................................ ................................ ......... 86


10.0 EVALUATION QUESTION: H AVE THE MOBILITY CEN TRES ESTABLISHED A L INK WITH NATIONAL/REGIONAL/LO CAL AUTHORITIES AND SHARED INFORMATION O N FREQUENTLY ENCOUNTERED PROBLEMS BY RESEARCHERS IN T HE CONTEXT OF NATIONAL/REGIONAL/LO CAL MOBILITY? ................................ ................................ ................................ ..... 97

10.1 FINDINGS ................................ ................................ ................................ ................................ .......................... 98 10.2 CONCLUSIONS ................................ ................................ ................................ ................................ ................ 104 10.3 RECOMMENDATIONS ................................ ................................ ................................ ................................ ...... 105

11.0 EVALUATION QUESTION: HAVE THE MOBILITY C ENTRES, THROUGH THEI R BRIDGEHEAD ORGANISAT IONS PROVIDED EXPERT ISE AND ASSISTANCE I N MOBILITY POLICY ISSUES TO THE NATIONAL/REGIONAL/L OCAL AUTHORITIES AS WELL TO THE EUROPEAN COMMISSION, IN CL OSE COOPERATION WITH THE STEERING GROUP FOR THE HUMAN RESOURCES AND MOBILITY OF RESEARCH ERS? ................................ ................................ ...... 106

11.1 FINDINGS ................................ ................................ ................................ ................................ ........................ 106 11.2 CONCLUSIONS ................................ ................................ ................................ ................................ ................ 110 11.3 RECOMMENDATIONS ................................ ................................ ................................ ................................ ...... 111


Page 7

LIST OF TABLES

Table 1 - Achievements of the mob ility and career strategies 2000 -2007 ................................ ..........31

Table 2 - Approach followed for the set up of Mobility Centres at national level .......................... 33

Table 3 - Overview of information and assistance services provided September 2006 -December 2007 ................................ ................................ ................................ ................................ ................................ ...35

Table 4 - Visits & pages viewed -NRMPs 2006 -2007 ................................ ................................ ...............36

Table 5 - Researchers’ online survey – Number of responses and % of total replies per category of questions ................................ ................................ ................................ ................................ ..................... 41

Table 6 - Job providers’ online survey – Number of responses a nd % of total replies per category of questions ................................ ................................ ................................ ................................ ....44

Table 7 - Distribution by category of interviewees ................................ ................................ ................. 46

Table 8 - List of train ing sessions carried out by the European Commission (2004 – 2007) ..........62

Table 9 - Structure of the countries sampled during the period 2004 – 2007 ................................ ....64

Table 10 - Overview of channels and tools used in fieldwork countries ................................ ...........71

Table 11 - ERA-MORE network - Reporting 2004 -2007................................ ................................ .........86

Table 12 - Data collection – BHOs and MoCs average response - in % of total number of BHOs and MoCs................................ ................................ ................................ ................................ ......................... 91


Page 8

LIST OF FIGURES

Figure 1 - Researchers’ onl ine survey - Breakdown of respondents by continent – in % of total replies ................................ ................................ ................................ ................................ ............................... 41

Figure 2 - Researchers’ online survey - Breakdown of number of replies by country – 15 most represented countries ................................ ................................ ................................ ................................ ....42

Figure 3 - Researchers’ online survey - Respondents’ profile (gender, age, type, working institution) – in % of total replies ................................ ................................ ................................ ...............43

Figure 4 – Job providers’ online survey - Breakdown of respondents by continent – in % of total replies ................................ ................................ ................................ ................................ ............................... 44

Figure 5 - Job providers’ online survey - Breakdown of number of replies by count ry – 15 most represented countries ................................ ................................ ................................ ................................ ....45

Figure 6 - Job providers ’ online survey - Respondents’ profile (type of organisation) – in % of total replies ................................ ................................ ................................ ................................ ...................... 45

Figure 7 – Online survey - Opinion on the best ways for the ERA -MORE Network to provide mobile researchers with personalised assistance and services – in % of total responses to the question................................ ................................ ................................ ................................ ............................ 51

Figure 8 - Researchers’ online survey - Opinion on the quality of services delivered to them – in % of total responses to the question ................................ ................................ ................................ ...........52

Figure 9 - Job providers’ onl ine survey - Opinion on the quality of services delivered to them – in % of total responses to the question ................................ ................................ ................................ ......53

Figure 10 – Online surveys - Respondents’ opinion on average speed of answering q uestions – in % of total responses to the question ................................ ................................ ................................ ......53

Figure 11 – Data collection - Type of requests for the period 2007 – in number of requests .........56

Figure 12 – Researchers’ online survey - Assessment of researchers’ satisfaction with the information provided on specific issues – in % of total responses to the question ......................... 57

Figure 13 – Job providers’ online survey - Assessment of job providers’ satisfaction with the information provided on specific issues – in % of total responses to the question ......................... 58

Figure 14 – Online surveys - Percentage of services delivered indirectly – in % of total responses to the question ................................ ................................ ................................ ................................ ................60


Page 9

Figure 15 – Online surveys - Assessment of overall users’ satisfaction of services d elivered to them when redirected to other organisations – in % of total responses to the question ................60

Figure 16 – Online surveys - Respondents perception of the relevance of the services and information provided – in % of total responses to the question ................................ .......................... 63

Figure 17 – Online surveys - Respondents perception on the appropriateness of the location of the MoCs – in % of total responses to the question ................................ ................................ ................65

Figure 18 - Overview target audiences reached ................................ ................................ ...................... 73

Figure 19 - Extent to which researchers are still in contac t with an ERA -MORE Mobility Centre................................ ................................ ................................ ................................ ................................ ............76

Figure 20 – Extent to which researchers have consulted a country -specific Researcher's Mobility Portal................................ ................................ ................................ ................................ ................................ .80

Figure 21 - Researchers’ opinion of the extent to which the information provided by the National Researcher's Mobility Portal they used most often matched their need for information................................ ................................ ................................ ................................ ................................ ............81

Figure 22 - Researchers’ opinion on the following statements concerning the information provided by the National Researcher's Mobility Portal they used most often ................................ 81

Figure 23 – Job providers’ opinion on the following statement concerning the information provided by the National Researcher's Mobility Portal they used most often ................................ 82


Page 10

1.0 EXECUTIVE SUMMARY

1.1 Introduction The creation of a network of Mobility Cen tres was proposed as part of the Mobility Strategy for the European Research Area 1. The European Commission saw the creation of a European network of Mobility Centres as a “direct response to the many problems encountered by researchers when undertaking a mobility experience” 2. In addition, National Researchers Mobility Portals were developed as it was seen as essential for the successful implementation of the Mobility Strategy at both national and European level.

The mission of the ERA -MORE3 network is to “provide comprehensive and up -to-date information and personalised assistance to all researchers and their families seeking advice in issues relating to their move across borders. ERA -MORE Mobility Centres provide information and assistance in matters su ch as entry conditions, visas, work permits , recognition of diplomas, job opportunities, salaries, taxation, pension rights, health care, social security, accommodation, day care, schooling, language courses, access to the culture of the host country and intellectual property rights ”4. The objective of the network is to improve the overall environment of researchers in Europe in coordination with the tasks undertaken under the European Researcher´s Mobility Portal 5.

In terms of structure, the network has be en built on existing structures and in absence of relevant existing structures or in case of insufficient service level, then new structures were encouraged. The structures of national networks vary. Some countries have one single Mobility Centre/Bridgehead organisation whereas others have several Mobility Centres at local level in addition to one or several Bridgehead organisations. Furthermore, some national mobility networks have national sub -networks including information offices within research institu tions or universities.

The selection of the members was coordinated by the European Commission in close cooperation with the Research ministries of the countries.

The working plan of 2002 “Towards a European Network of Mobility Centres”6 defines that all members of the Network at whatever operational level are required to provide comprehensive and up-to-date information and personalised assistance to mobile researchers and their families. Mobile researchers are defined to include both incoming and outgoin g researchers both working in the public and private sector. This task could be fulfilled either directly (by providing direct proximity assistance to the person (s) requesting help) or indirectly (by directing

1 COM (2003) 226 final, 30.04.2003. 2 Towards a European network of mobility centres, EC, DG RTD, Directorate D - European Research Area: the human factor, Mobility policy, April 2002. 3 The ERA-MORE Network was rebranded EURAXESS Services Network in June 2008. 4 Towards a European network of mobility centres, EC, DG RTD, Directorate D - European Research Area: the human factor, Mobility policy, April 2002. 5 Declaration of commitment by the members of the European Network of Mobility Centres, 2004-2007. 6 The European Commission, DG RTD, Towards a European Network of Mobility Centres 2002.


Page 11

the person(s) concerned to the closest specia lised centre, at whatever operational level, which may ensure the required service).

The scope of this evaluation covered the European Commission co -funding period from 2004 -2007. As from 2008 no further co -funding from a pan European level was foreseen f or the continuation and maintenance of the national ERA -MORE networks 7.

The aim of this study was to find out to what extent the ERA -MORE network has been successful in its activities and effective in reaching its objectives. The main objective of the evaluation study was to assess the effectiveness of the Mobility Centres (MoCs).

Looking to the effectiveness 8 of the Mobility Centres, the evaluator focused on the following evaluation questions:

• Have the Mobility Centres reached their objectives in terms o f providing comprehensive and up-to-date information and personalised assistance to the researchers in matters related to mobility? (taking into account the direct and the indirect approaches)

• Have the Mobility Centres been effective in disseminating the i nformation and promoting their services to their main target groups, i.e. the researchers’ community, the universities, the research centres and the industry and the multipliers of information in the research community? Have they been effective in creating a network of users of their services?

• Have the Mobility Centres and/or Bridgehead Organisations been effective in the maintenance of the National Mobility Portal?

• Have the Bridgehead Organisations and the Mobility Centres reported their activities and results back to the Commission in a structured and effective way?

Besides these questions covering the objectives and activities of the ERA -MORE network as defined for the period of time considered for this evaluation, the evaluator also looked for additional information relative to the following questions:

• Have the Mobility Centres established a link with the National / Regional / Local Authorities and shared information on frequently encountered problems by researchers in the context of National / Regional / Local mobility?

• Have the Mobility Centres, through their Bridgehead Organisations provided expertise and assistance in mobility policy issues to the National / Regional / Local Authorities as well to the European Commission, in close cooperation with the Steering Group for the Human Resources and Mobility of Researchers?

7 Only funding for trans-European network activities is foreseen. 8Effectiveness is understood by the evaluator as: how far have the intervention’s effects contributed to achieving its specific and general objectives? Has the intervention achieved its objectives and does it show an ability to solve problems and provide added value?


Page 12

1.2 Key Conclusions and Recommendations

Effectiveness of the Mobility Centres

Conclusions on the effectiveness of the Mobility Centres

• The MoCs have a good understanding of their operatio nal objectives, being the provision of comprehensive and up -to-date information and personalised assistance to researchers and their families in matters related to mobility.

• The MoCs understood their target audience as being the researchers’ community in g eneral and third parties acting on behalf of researchers (supervisors, hosting researchers, different departments of University, private companies, etc.).

• The level of the MoCs’ achievement of the operational objectives highly depends on their understanding of their role and the scope of the services to be provided. The MoCs’ understanding of the personalised and customised dimension of the provision of information and assistance varies depending on the country . Indeed, MoCs differ in extent to which they deliver assistance to researchers (for instance, by helping the researcher with the negotiation of residential lease, legal assistance, translation of documents or doctor appointments). To deliver personalised and customised information and assistance, MoCs favour the use of channels such as e -mails and face -to-face contacts to ensure comprehension of stakeholders’ views and needs.

• During the period 2004 – 2007, the ERA -MORE network has realised effectively its operational objectives in terms of providing co mprehensive and up -to-date information and personalised assistance to the researchers and their families in matters related to mobility. Generally speaking, the main services provided show a good degree of coherence amongst the MoCs across the countries vi sited. As described above , there are some differences in the extent and intensity of services provision .

• The services provided by the MoCs are likely to be sustained after the EC funding period. However, the end of the EC funding period might have an impac t on the delivery of services by MoCs. In those countries where the national funding is lower than the EC funding received or uncertain, the national ERA -MORE networks will likely to be maintained but no further developments will be introduced.

• There is a difference in the audience actually reached depending on the role the MoCs were assigned to within their specific national structure. For some of the MoCs, actions targeting the Industry were pursued. However, few countries visited established an effective cooperation with this target group.

• In the majority of cases, the information provided by the MoCs during the period 2004 – 2007 has matched the users’ needs. To some extent, t he MoCs have implemented mechanisms for seeking feedback from users and data or performance monitoring in relation to signposting, the profile of researchers, researchers’ needs, potential gaps and expectations etc.


Page 13

• There was a great level of cooperation within individual national ERA -MORE networks among MoCs which strengthens the se nse of being part of a network. However, little formal cooperation has been established with other national ERA-MORE networks or multipliers except through informal agreements based on personal contacts within the same country. Little to no signposting occ urred at transnational level. If indeed such transnational signposting occur red, it was most often a referral to other members of the ERA -MORE network and not to another network or organisation abroad. Nevertheless, a call for proposal for transnational pr ojects amongst ERA-MORE members was launched in May 2007 focused on stimulating cooperation between ERA -MORE members on operational and strategic issues .

• The MoCs received appropriate support by DG RTD in terms of training sessions that would ensure on the one hand knowledge of the information to be provided and on the other hand the access to networking and exchange opportunities. The topics of the training sessions were mainly related to the development of soft skills . However, the MoCs expressed a need to be further trained on specific topics related to European regulation in the research area .

• The comprehensiveness and accuracy of the information was also ensured in most of the cases by national training sessions on technical and regula tory issues, organ ised by BHOs.

Recommendations on the effectiveness of the Mobility Centres

EUROPEAN COMMISSION – DG RTD

• Provide further guidance Since the scope of the services to be delivered has been further defined in the new Declaration of commitment, DG RTD should provide further guidance in terms of the extent and intensity of the services to be provided.

A welcome pack could be produced by DG RTD for new employees of the network to ensure that all national ERA -MORE members have written guidelines on their operati onal objectives and assist in the handover process to new joiners.

In addition, DG RTD in consultation with the members of the ERA -MORE networks should clearly define the extent to which the MoCs should target Industry by performing a pan -European assessme nt of the specific service needs of the sector.

• Encourage the national ERA -MORE networks to further monitor activities

DG RTD should encourage the national ERA -MORE networks to carry out surveys aiming at identifying potential gaps, needs, target audience s and expectations to ensure user satisfaction and appropriate services. MoCs should be encouraged to seek periodic feedback from users regarding dissemination and promotion activities. Large -scale evaluations of the networks’ activities on a pan -European level should be continued in order to monitor whether all activities carried out effectively reached the intended target groups and met their needs.


Page 14

• Continue training sessions, conferences and events Training sessions, conferences and events at European l evel should be continued in the future as these are seen to have a strong leverage to improve the transnational cooperation of the national ERA -MORE networks. On the one hand, involving national networks in activities organised by the European Commission w ould strengthen the exchange of best practice and knowledge, and reinforce the sense of belonging to a European Network. On the other hand, strong transnational cooperation between national networks would increase and strengthen the signposting activities. Besides involving the national ERA -MORE networks, these activities should target international organisations active in promoting mobility and research in order to widen transnational cooperation between members of the ERA-MORE network to international key players in Research.

The topics of the training sessions organised by DG RTD should continue to focus on issues related to European initiatives such as new European Regulation, etc. Additionally it could be envisaged to organise certain sessions to specif ic sessions, for example thematic or regional.

NATIONAL ERA-MORE NETWORK MEMBERS

• Improve the cooperation with other organisations

The national ERA -MORE networks should improve the cooperation with other organisations active in the area of research and mobility through formal and/or informal agreements, common events or projects at national level. This would likely improve information circulation and increase the effectiveness in delivering the services and assistance. The national ERA -MORE network shoul d clearly define an approach to reach Industry by performing country specific assessments of the specific needs of the sector within a pan -European approach coordinated by the European Commission.


Page 15

Effectiveness in disseminating the information and pr omoting the

services to target groups

Conclusions on the dissemination of information and promotion of the services to target groups

• Most countries do not have a national c ommunication strategy for the ERA -MORE network in place.

• Only a few of the countries visited had carried out an initial study of their target audiences’ needs before developing their communication activities. However, t here is evidence that suggests that varying efforts have been undertaken to disseminate information and promotion of se rvices to the main target groups.

• The level of awareness of the existence of the ERA -MORE networks and the services it provides appear to be rather low amongst the target groups. Th e target groups best reached (in decreasing order) are u niversities’ key su pport staff dealing with researchers, supervisors /hosting professors and the researchers themselves. The industry target group (companies, etc) seems to remain largely unaware of the existence of the network. As the previous ERA-MORE evaluation 9 also suggests, many of these target groups are not aware of the ERA-MORE network as such, because most MoCs are embedded in the university system and most researchers associate their services with the international offices within universities or HR department of the university.

• National ERA-MORE networks have promoted the network to some extent through events relating to EU funding (Framework Programmes), scientific conferences.

• MoCs see the BHOs as being supportive of their information and dissemination activities.

• The MoCs visited principally perceive DG RTD as a source of generic promotional material.

• There is a low level of awareness among the target groups of the information and services offered by the MoCs. Those who have made use of the information and service s provided by the MoCs might be aware of the network, but most researchers and administrative and support staff in the universities do not tend to feel part of a European network.

• Since the end of the EC co -funding period, dissemination activities of the network have continued, though sometimes to a less frequent degree. Moreover, the interviews indicated that the activity of information and dissemination is likely to be affected by a lack of funds in the future.

9 “Evaluation of Communication and Information activities relative to ERA-MORE and the European Researcher’s Mobility Portal”. The Evaluation Partnership and Deloitte, October 2007.


Page 16

Recommendations on the dissemination of information and promotion of the services to target groups


• Encourage the national ERA -MORE networks to develop a communication strategy

BHOs should be encouraged to put together a communication strategy that could be applied throughout the national ERA -MORE networks by the MoCs to ensure effective communication, consistency and transparency in all ERA -MORE countries. A more strategic, integrated approach should be followed regarding dissemination of information and the promotion of the network services to the target audiences, rather than undertaking ad hoc activities. The exchange of best practice should be encouraged at all levels within the network in terms of channels used for dissemination of information and maximising all t he opportunities where promotion of activities can be pursued.

• Attention to impact of the end of co -funding

Attention has to be paid by DG RTD to the impact of the end of EC co -funding on the dissemination activities performed by the ERA -MORE networks.


• Develop a communication strategy MoCs should be encouraged to seek periodic feedback from users regarding dissemination and promotion activities and act upon this where possible in order to better meet the needs of their target audience. Large -scale evaluations of the networks’ activities on a pan European level should be continued in order to monitor whether all activities carried out effectively reached the intended target groups and met their needs.

Information and di ssemination activities could be made more personal by, for example, organising events with testimonials from mobile researchers, giving researchers the possibility to meet MoCs and other researchers, etc.

• Encourage exchange of best practice

The exchange o f best practice should be encouraged at all levels within the network in terms of the channels used for the dissemination of information (e.g. by giving presentations on the network at events organised by third parties including organisations and companies which target relevant audience) and maximising all the opportunities where the promotion of activities can be pursued (e.g. by ERA -MORE network members distributing information packages to


Page 17

researchers upon their arrival in -country.

National Researcher´s Mobility Portals

Conclusions on the National Researcher’s Mobility Portals

• NRMPs are recognised by all members of the ERA -MORE network as being an important communication tool . However, the updating of the content of the NRMPs is not subject to formal procedures in most countries . Moreover, information from the ERMP is not always systematically transposed to the NRMPs.

• Despite some continued discrepancies in terms of format and content, there is evidence to suggest that the implementation of the NRMPs w as carried out to a satisfactory extent, with timeframes usually being met and the majority of information required being made available to researchers.

• Reaching complementarity between the ERMP and the NRMPs was shown to be challenging, with various NRMPs having different formats and set -ups than the ERMP. According to the ERA -MORE network members this could be due to different needs, as well as to their understanding of certain elements being mandatory or not.

Recommendations on the National Researcher ’s Mobility Portals


• Assist in the adaptation of the new branding requirements

DG RTD should assist the national ERA -MORE network in the adaptation of the new branding requirements. A pragmatic approach should be applied by DG RTD in terms of compliance with timeframe, functionalities and layout of the NRMPs.


Page 18


• Continue to actively promote and use the NRMPs

The members of national ERA -MORE networks should continue to actively promote and use the NRMPs as they are key in providing up -to-date and comprehensive information to a wider public of users. Moreover, formal rules and procedures should be followed in order to regularly update the content of the NRMPs.

All national ERA -MORE networks should be encouraged to establish formal rules and procedures to regularly update the information provided on the NRMPs. For those counties in which this is not yet/no longer the case a (external or internal) webmaster should be appointed to follow up an c oordinate the information update process.

The information provided on most of the NRMPs could be further enriched and/or made more interactive by providing personal testimonials of researchers, for example, by asking researchers to write a brief paragraph of their experience in the country or looking to create a space for exchange of personal experiences via a chatroom or blog.

Effectiveness of the reporting

Conclusions on the reporting

• The BHOs followed a structured reporting approach, as per the FP6 reporting guidelines provided to them . The data collection initiative launched in 2006 is considered as a step forward in gathering information on the services provided by the network. The Quality Working Group is seen to have been instrumental in imp roving the process of data collection. However, still in 2008 , such data collection did not yet provide reliable and accurate information on the activities performed by the network, as there are gaps in terms of number of responses, quality and accuracy of the data submitted.

• In some cases, the annual reporting according to the FP6 reporting template did not give a representative view of the national ERA -MORE networks’ activities performed. As the main rationale of the FP6 reporting template was reporting on the implementation of a project, being the establishment of the national network, the reporting template did not take into account providing information on the specific activities performed by the MoCs.

• The data collection initiative is a step forward in gathering information that shows the added value of the network. However, still in 2008 some issues in terms of number of responses, quality and accuracy were identified.

• The soft reporting initiative, despite the initial interest amongst BHOs and perce ived added value in terms of providing information on the ERA -MORE national networks and its


Page 19

activities saw a low response rate and thus, was an initiative carried out only for 2007 period.

• The Quality Working Group has been instrumental in the reporting p rocess in terms of improving the process of data collection.

• The BHOs have reported more frequently the statistical data through the extranet than the MoCs.

Recommendations on the reporting


• Further guidance on the reporting

Incentives to BHOs/MoCs could be foreseen in order to ensure more timely and accurate reporting. For example by: linking reports’ submission to the number of seats made available at European Training sessions; providing feedback on the information provid ed in the reports to BHOs; or by producing a newsletter containing examples of good practice from the information provided in the reports, information on reporting etc.

Guidance on reporting (soft reporting and data collection) should continue to be given at European level to account for changes in staff .

A handbook could be developed by DG RTD within the Quality Working Group, providing guidelines for reporting. This handbook could be put on the extranet, making it visible to users by including it on the homepage.

Further training on data collection and soft reporting should be given to network members in order to ensure a common understanding and acceptance of the added value of such reporting.

• Perform quality checks

Quality check on the reported data should be rigorously performed at EC level .

QUALITY WORKING GROUP

• Overview reporting process

The Quality Working Group should continue to oversee the reporting process in terms of quality check -ups and recommendations, especially following the end of t he EC co-funding period. In addition, the specificities of the network should be reflected in the soft reporting.

The complexity and quantity of reporting should be carefully monitored in order to ensure a


Page 20

high response rate.


• Further guidance on the reporting At national level, BHOs and MoCs should ensure that the guidance given at European level on reporting (soft reporting and data collection) is communicated to all ERA -MORE members.

• Perform quality checks

Quality checks need to be rigorously performed not only at EC level but, as well, at BHO level of MoC/LoCPs’ reporting.

Links established between MoCs and National/Regional/Local Authorities and shared information on frequently encountered

problems in the context of national/regional/local mobility

Conclusions on the Links established between MoCs and National/Regional/Local Authorities and shared information on frequently encountered problems in the context of national/regional/local mobility • There is a great variety in the degree of interaction between the ERA -MORE national

networks and the authorities. In some countries regular meetings take place between the network and authorities, whereas in most countries such interactions are on an ad hoc basis and more informal in nature.

• Generally speaking all BHOs have established links with authorities, but with a varying extent and for differing purposes. The key differentiators that determine the degree of interaction between the BHOs and the authorities in their country are the involvement of the authorities in the development of the network, the role of the hosting institution, but also the activities performed by those working at the BHO and their personal network.

• MoCs show a lesser degree of interaction with a uthorities than the BHOs. There are however differences between the national networks and between MoCs within a single national network as well. The reasons for this relate to differences in local or regional authorities’ degree of involvement with the Mo C, differences in the target groups reached by MoCs from one county to the next, differences in task allocations to some MoCs within a country, the organisation and set -up of the network, but also the participation of MoC members in wider initiatives.


Page 21

• Whereas some interactions with authorities lead to the exchange of information on frequently encountered problems concerning researchers’ mobility , other interactions do not. Quite a number of these interactions are limited to having the authorities as guest speakers at research -related events, etc. Besides this, in all countries visited there is a top -down information flow from the authorities to the network during the regular or ad hoc meetings with the network members (being most often the BHO). Such info rmation exchange mainly deals with the communication of changes in policy or in administrative practices relevant for the research domain. But in several countries the network also provides concrete information about obstacles for mobility to the authoriti es, questions received, statistics, results of surveys, etc (bottom -up). Finally, there have been some meetings and exchanges of an interactive nature concerning the sustainability of the network.

• Depending on their level of interaction with the authoriti es, the network members have different expectations regarding what support could be provided by DG RTD to enhance links with authorities. Those networks with good working relationship with the authorities do not see a need for DG RTD to assist them in suc h a country-specific process. Those networks that are still struggling with establishing good working relationships with their authorities would appreciate more active support from DG RTD by e.g. highlighting the added value and importance of the ERA -MORE network to their authorities.

• The support provided by BHOs to create links between the MoCs and authorities is mainly related to the organisation of events, training sessions, etc. As in most countries the BHO is the liaison between the MoCs and the auth orities, most MoCs do not have recommendations for their BHOs on how to enhance their links with the authorities.

• The exchange of best practice and lessons learned on how to establish links with authorities between national networks has mainly occurred t hrough the 2006 workshop organised in Athens by DG RTD on this topic. However, a few bilateral exchanges were also identified.

Recommendations on the Links established between MoCs and National/Regional/Local Authorities and shared information on frequently encountered problems in the context of national/regional/local mobility

EUROPEAN COMMISSION -DG RTD

• Specific support to national ERA -MORE networks regarding relationship with authorities

Specific support could be given by DG RTD to those national netwo rks that would like to receive additional assistance in building up a working relationship with the authorities in their country. It is important that such assistance takes into account the capacities and organisation of the national network, as well as t he culture and expectations of national authorities.


Page 22


• Strengthen and develop information flow with authorities National networks that have not established a two -way information flow with authorities should actively try to identify those topics on which the authorities could be interested in cooperating with the network. The regular encounters with authorities (during conferences, etc.) should be more pro-actively used to start a more active cooperation with the authori ties.

When defining the key competencies of BHOs, the BHO should pay attention to certain soft skills such as networking capacity. Also when defining training needs, these types of skills should be taken into account.

BHOs could seek to actively involve t hose MoCs interested in building up relationships with authorities. This may be done by the exchange of best practices between the BHO and MoCs on how to create links with authorities. This may particularly apply to those countries with a regional state st ructure in which the MoC may play a role in liaising with regional authorities.

• Strengthen the European dimension of the information flow

The existing information provision from the BHOs to the authorities could be taken to the next level if this informa tion were placed within a wider European context. Information about the situation in the country (e.g. volume of foreign researchers, number of questions, duration for application processes, etc) could be presented to the authorities complemented with information on the situation in other European countries. This way the European span of the network could be used to a full extent. Providing such information would further increase the added value of the national ERA -MORE networks as information providers t o their authorities.


Page 23

Expertise and assistance provided by MoCs through BHOs in mobility

policy issues to the National/Regional/Local Authorities and the European Commission in close cooperation with the Steering Group

for the Human Resources and Mobi lity of Researchers

Conclusions on the expertise and assistance provided by MoCs through BHOs in mobility policy issues to the National/Regional/Local Authorities and the European Commission in close cooperation with the Steering Group for the Human Reso urces and Mobility of Researchers

• In most countries visited, the national ERA-MORE networks do not feel as having the formal task of contributing to the national policy -making process concerning mobility issues. In those countries where indeed such assis tance exists, it mostly takes the shape of informal contact and/or the ad hoc provision of information relevant to the new policy development process.

• Most of the BHOs as well as most of t he national authorities are not very keen on the national ERA -MORE networks having a formal task of assisting the national authorities in policy-making. BHOs are wary of the need for additional resources to perform this task, as well as the (perceived) risk of losing neutrality when getting involved in policy debates.

• The national ERA -MORE networks were involved in initiatives at both the policy level and the level of administrative proce dures, mostly through the participation in the stakeholder consultation process, the exchange of draft policy discussion papers and dra ft legislation, but also on the more practical level through the exchange of draft application forms, the determination of needed documents for certain application dossiers, etc.

• The extent to which such participation has been of a significant nature at th e national level depends on several factors, such as: the level of importance of researchers’ mobility for the country’s policy makers; the volume of services provided by the network and the accuracy of information that could be provided; the extent to the authorities involve stakeholders in policy-making; and the general involvement of the hosting institution in such processes.

• The extent to which the national ERA -MORE networks contributed to policy -making at the European level is generally perceived as no t being of a significant nature. However, both network members and national authorities provided evidence of significant contributions by certain national networks to the transposition of the European Directive on Scientific Visa into national legislation . The training received by DG RTD on this topic is considered to have facilitated this role to a significant extent.

• The depth of interaction between the SG HRM and the national ERA -MORE networks depends on the country. In a few of the countries visited, the SG HRM representative is


Page 24

closely involved in the functioning of the network, allowing for a direct link between both. In most cases the interaction with the national networks takes place through consultations of the SG HRM national representative with stakeholders in his/her country. The frequency of these consultations varies depending on the country’s national context and moreover, on the national representative to the SG HRM.

Recommendations on the expertise and assistance provided by MoCs through BHOs in mobility policy issues to the National/Regional/Local Authorities and the European Commission in close cooperation with the Steering Group for the Human Resources and Mobility of Researchers


• Strengthen the link with the SG HRM

The extent to which the national ERA -MORE networks should be involved in national policy -making and changes in administrative practices concerning mobility should be discussed in the SG HRM. Whereas the new Declaration of commitment does foresee the national ERA -MORE networks to assist their national and local authorities in mobility policy issues, there is a need for openness and flexibility in the extent to which such involvement is feasible in the respective countries. The link between the nat ional ERA-MORE networks and the SG HRM could be further strengthened by planning regular meetings between the SG HRM representative and the network representatives to provide feedback on each other’s tasks and activities.

NATIONAL ERA-MORE NETWORK MEMBE RS

• Strengthen the link with the SG HRM

The link between the national ERA -MORE networks and the SG HRM could be further strengthened by planning regular meetings between the SG HRM representative and the network representatives to provide feedback on each other’s tasks and activities.

If a decision is made to try to further involve the network in the policy -making process, the BHOs should allocate time and resources to their participation in policy development and in administrative simplification in their work plan.

BHOs should actively document concrete cases of obstacles to mobility, as well as, whenever possible, produce reliable statistical data. It is those networks that are able to provide such information that have had most impact on policy develop ment and the simplification of administrative procedures. As also recommended above, it could be


Page 25

envisaged to further enrich such information by presenting it within a pan -European context, allowing authorities to situate their country in comparison with other European countries.

It is important that the BHOs will continue to be actively informed and trained on all new European initiatives in the field of mobility. This way the BHOs will be able to consult authorities on these matters if required.

The national ERA-MORE networks should be known as experts in the field by the policy -makers and as such consulted regarding mobility issues.

Sustainability Conclusions on sustainability

• In terms of sustainability, most of authorities encountered during the fieldwork have all expressed their commitment to the continuation of the network and the delivery of its services. In the majori ty of the countries visited the continued funding of the network has been arranged after the Commission’s co -funding came to an end. In most instances the future funding of the national ERA -MORE networks represents however only a fraction of the amount co -financed by DG RTD during 2004 -2007. There is therefore a widespread request from the national ERA -MORE networks to DG RTD to continue its financial support, as well as its support through the organisation of cross -European meetings, training sessions and events.

Recommendations on sustainability


• Pay attention on the sustainability of the ERA -MORE Network

Following the end of EC co -funding, attention has to be paid by DG RTD on the sustainability of dissemination of informati on and promotion of activities and NRMPs maintenance and update of content performed by the national ERA -MORE networks.


Page 26

2.0 INTRODUCTION

The aim of this evaluation was to provide the European Commission with an evaluation of the Information activities of the ERA-MORE network. The main objective of the evaluation study is to assess the effectiveness of the Mobility Centres (MOCs).

This project took place under the Framework Contract RTD -C5-2005-I&C for the Assessment of the impact of information and communi cation policy products.

The evaluation study ran from April 2008 to October 2008. This document constitutes the Final Report. It consists of the following sections:

• Section 1: executive summary;

• Section 2: introduction to the evaluation;

• Section 3: scope and objectives of the evaluation;

• Section 4: an overall description of the context of the evaluation;

• Section 5: a Methodology section presenting the overall approach and methodology, and the methodological tools and techniques used during the course of the evaluation;

• Sections 6 to 11: these sections present the evaluation questions in terms of findings, conclusions and recommendations. They provide some findings and the conclusions and recommendations based on the evidence gathered during the course of the evaluation .

Additionally, the Annexes to this report (contained in a separate document) present the following:

• Annex 1: Analytical Framework on which the evaluation was based;

• Annex 2 to 5: Interview guides (Bridgehead Organisation representatives, Mobi lity Centre representatives, National Authority representatives, stakeholders) ;

• Annex 6 to 7 : Online survey questionnaires (Researchers and job providers);

• Annex 8: List of interviewees in EU Member States and at EU level.


Page 27

3.0 Scope and objectives of the eva luation

The aim of this study was to find out to what extent the ERA -MORE network has been successful in its activities and effective in reaching its objectives. The main objective of the evaluation study was to assess the effectiveness of the Mobility Cen tres (MoCs).

The main specific operational objective of the MoCs is to provide comprehensive and up -to-date information and personalised assistance to all researchers seeking advice and their families in all matters relating to their mobility experiences. This task will be fulfilled either directly, by providing direct information and proximity assistance to the person(s) requesting help, or, indirectly, by directing the person(s) concerned to the closest specialised centre, at whatever operational level, w hich may ensure the required service.

Looking to the effectiveness 10 of the Mobility Centres, the evaluator focused on the following evaluation questions:

• Have the Mobility Centres reached their objectives in terms of providing comprehensive and up-to-date information and personalised assistance to the researchers in matters related to mobility? (taking into account the direct and the indirect approaches)

• Have the Mobility Centres been effective in disseminating the information and promoting their services t o their main target groups, i.e. the researchers’ community, the universities, the research centres and the industry and the multipliers of information in the research community? Have they been effective in creating a network of users of their services?

• Have the Mobility Centres and/or Bridgehead Organisations been effective in the maintenance of the National Mobility Portal?

• Have the Bridgehead Organisations and the Mobility Centres reported their activities and results back to the Commission in a structur ed and effective way?

Besides these questions covering the objectives and activities of the ERA -MORE network as defined for the period of time considered for this evaluation, the evaluator also looked for additional information relative to the following qu estions:

• Have the Mobility Centres established a link with the National / Regional / Local Authorities and shared information on frequently encountered problems by researchers in the context of National / Regional / Local mobility?

• Have the Mobility Centre s, through their Bridgehead Organisations provided expertise and assistance in mobility policy issues to the National / Regional / Local Authorities as well to the European Commission, in close cooperation with the Steering Group for the Human Resources and Mobility of Researchers?

The last two questions were relative to objectives and activities that have been defined in the new approach. These questions could not therefore be considered as “evaluation” questions

10Effectiveness is understood by the evaluator as: how far have the intervention’s effects contributed to achieving its specific and general objectives? Has the intervention achieved its objectives and does it show an ability to solve problems and provide added value?


Page 28

relative to the effectiveness of the Networ k. Nevertheless these questions were considered as they can provide useful information for the next contracting period.

Besides the above -mentioned evaluation questions, the evaluators have also looked at the sustainability of the network after the EC co -funding period. This deals with the continuation of the existence of the network, as well as the continuation of the services provided, the dissemination activities and the maintenance of the NRMP.

In order to provide answers to this main evaluation questi ons, judgment criteria and indicators were defined in the analytical framework presented in annex 1.

Outcomes focused on the positive and negative aspects, as well as challenges and concrete suggestions.

As far as the scope of the study is concerned, the f ollowing elements have been taken into account:

• the Researchers’ Community include researchers equally within Universities, Research Centres and Industry;

• the focus of the study is on the Information activities and services delivered by the ERA -MORE network (Bridgehead Organisations and Mobility Centres);

• a sample of 10 countries has been defined for the purpose of this study:

o Czech Republic;

o Estonia;

o Greece;

o Ireland;

o Lithuania;

o The Netherlands;

o Poland;

o Romania;

o Spain;

o Sweden.


Page 29

4.0 CONTEXT

4.1 Background

The Communication proposing the creation of a E uropean Research Area (ERA)11, adopted in January 2000, emphasised the need for more abundant and mobile human resources. Thus, more use of mobility was recommended, both at national and European level, as mobility is an “instrument for the transfer of scientific knowledge” 12. Moreover, mobility of researchers between the academic world and the business world should be encouraged and developed according to the Communication. In addition, close cooperation between Member States (MS) and the European Commission (EC) in bringing together the administrative departments of research organisations and researchers was recommended.

A series of objectives and a timetable on mobility of researchers was set by the Lisbon European Cou ncil on 23-24 March 2000. The Council, MS and the EC were asked to work together towards removing obstacles to the mobility of researchers by 2002 and attract and retain high-quality research talent in Europe. Several actions were pursued in order to work towards these objectives, such as the invitation of the Research Council 13 to the Member States and the EC to cooperate, as well as the set up of a High -Level Expert Group on Improving Mobility of Researchers that would assist in the preparation of proposal s for improving the mobility of researchers 14.

In 2001, the Communication “A mobility strategy for the European Research Area” 15, introduces a strategy that aims at creating a favourable environment for the mobility of researchers in the ERA.

The Mobility S trategy is implemented by the Commission in close cooperation with the Member States ,, the candidate countries and the countries associated with the Framework Programme for designing actions aiming to eliminate obstacles to mobility of researchers and at t he implementation of other actions necessary to encourage mobility at large.

Thus, in order to ensure this cooperation, a Steering Group on Human Resource and Mobility (SG HRM) 16 was created in January 2002. The SG HRM is composed of representatives of national Research Ministries. Their main tasks are to monitor -in close cooperation with the EC - the implementation of the activities foreseen by the Mobility Strategy, and –whenever applicable- to enhance coordination at the national level with relevant mini stries.

The implementation of the Mobility Strategy as set in the 2001 Communication was pursued through two types of actions, which dealt with:

• establishing the dynamics required to set up and develop a favourable environment for mobile researchers throug hout their career;

11 COM(2000)6 final, 18.01.2000. 12 Op.cit. 13 Resolution of 15 June 2000: OJ C 205, 19.07.2000. 14 The High-Level Expert Group on Improving Mobility of Researchers presented a report in April 2001. 15 COM (2001) 331 Final: A Mobility Strategy for the European Research Area, 20.06.2001. 16 It was then, in 2002, referred to as the “Mobility Steering Group”.


Page 30

• provide financial support required to reach a critical mass of mobile researchers within the ERA.

The first type of actions is composed of the following actions 17:

• Improving the information regarding mobility through:

o information to resea rchers and better dissemination of vacancies;

o statistics.

• Improving the provision of practical assistance to researchers through:

o Mobility Centres;

o Ombudsmen;

o recruitment methods.

• Qualitative issues through:

o Inter-ministerial meetings on mobility;

o exchange of best practices;

o benchmarking;

o “Quality charter”.

• Legal improvements through:

o Admission, access to employment, social security and taxation.

Further measures have been followed since the 2001 Communication, such as the Communication on 2003 “Researchers in the European Research Area: one profession, multiple careers” 18, the 3% Action Plan 19 of the EC and the Green Paper on the ERA in April 2007.

Moreover, a number of achievements have been made following the different initiatives and actions in the framewo rk of ERA.

17 COM (2001) 331 final, 20.06.2001. 18 COM (2003) 436 final, 18.07.2003. 19 COM(2003) 226 final, 30.04.2003.


Page 31

Table 1 - Achievements of the mobility and career strategies 2000 -2007

The launch of the Researchers' Mobility Portal in 200 3* to provide information on fellowships/grants, research job vacancies (more than 4,800 jobs po sted in 2007) and practical information when moving .

The set up of the ERA -MORE network in 2004 to assist researchers in all matters relating to legal issues, social security, health and taxes, daily life and family support (200 information/ assistance centres in 32 countries) .

The adoption of the "Scientific Visa" package, in 2005, to facilitate administrative procedures for third country researchers entering the European Community. It includes a Council Directive 2005/71/EC (12 October 2005) and two Reco mmendations: the 2005/761/EC on short -term visas and the 2005/762/EC on long -term admission (more than three months). The Directive has been transposed in 15 Member States .

The adoption in 2005 of a Commission Recommendation on the European Charter & Code which aims at giving the same rights and obligations wherever researchers may work in Europe (signed by over 200 organisations, representing over 800 institutions in 23 countries ).

The launch of ERA -link in the US in 200 6* to keep the contact with European researchers active abroad (more than 3,000 EU researchers registered), the launch of ERA -link in Japan in 2008 (more than 500 EU researchers) .

Awareness initiatives (The Researchers' Night events involved 100,000 people in 21 countries in 2006).

The FP6 Marie-Curie actions have been the equivalent of the Erasmus programme for researchers: Approximately 70,000 researchers have benefited so far from EC funding under this dedicated mobility action .

Source: EC, SEC (2008) 1911

*Dates updated as per DG RTD feedback on 31/10/2008 .

4.2 ERA-MORE network 4.2.1 Objectives As part of the Mobility Strategy for the European Research Area, it was proposed to create a network of Mobility Centres 20. These Mobility Centres should “ assist national and foreign researchers in dealing w ith legal and administrative matters and provide training facilities to officials dealing with mobility, particularly within universities ”. Within the tasks proposed, we can find: delivery of practical information on accommodation, day –care or education f or children,

20 COM(2003) 226 final, 30.04.2003, p.10.


Page 32

advice on job opportunities for the accompanying partner. In terms of the structure, it is indicated that these Centres would whenever possible not replace existing structures.

Therefore, the EC saw t he creation of a Europe an network of Mobili ty Centres as a “direct response to the many problems encountered by researchers when undertaking a mobility experience ”21.

The Mobility Centres are seen as a direct response to improve the provision of practical assistance to researchers, for which there is a clear need as Europe has a deficit in welcoming foreign researchers 22.

Mobility obstacles relate not specifically to one moment in the life span of a researcher but to different life stages and range from legal to administrative, social, cultural and p ractical obstacles as well as the obstacles related to a European dimension in research careers and to inter sectorial mobility 23.

The Mobility Centres should offer researchers a practical and harmonised instrument, facilitating their mobility experience an d their arrival and installation in the host country.

The mission and structure of the network were agreed in 2002 (Working Plan “Towards a Network of Mobility Centres” adopted in April 2002) and expanded in 2003. In June 2004 the European network of Mobil ity Centres was created and referred to as the ERA -MORE network (European Research Area -MObile REsearchers).

4.2.2 Mission The ERA-MORE network contributes to the establishment of the European Research Area, the achievement of the Lisbon strategy objectives for jobs and growth and the target of spending 3% of GDP for research by 2010.

The mission of ERA -MORE is to “ provide comprehensive and up -to-date information and personalised assistance to all researchers and their families seeking advice in issues relating t o their move across borders. ERA -MORE Mobility Centres provide information and assistance in matters such as entry conditions, visas, work permits, recognition of diplomas, job opportunities, salaries, taxation, pension rights, health care, social security , accommodation, day care, schooling, language courses, access to the culture of the host country and intellectual property rights”24. The objective of the network is to improve the overall environment of researchers in Europe in coordination with the tasks undertaken under the European Researcher´s Mobility Portal25.

4.2.3 Structure of the ERA -MORE network In terms of structure, as indicated in the Communication: “A mobility strategy for the ERA”, the network has been built on existing structures. In absence of re levant existing structures or in case of insufficient service level, new structures were encouraged.

21 Towards a European network of mobility centres, EC, DG RTD, Directorate D - European Research Area: the human factor, Mobility policy, April 2002. 22 Ibid, Op. Cit. 23 As identified and grouped in the High-Level Expert Group on Improving Mobility of Researcher. 24 Towards a European network of mobility centres, EC, DG RTD, Directorate D - European Research Area: the human factor, Mobility policy, April 2002. 25 Declaration of commitment by the members of the European Network of Mobility Centres, 2004-2007.


Page 33

The selection of the members was coordinated by the EC in close cooperation with the Research Ministr ies of the countries .

In terms of organisation two typ es of network participants were foreseen:

• Mobility Centres (MoCs) providing information and offer assistance to researchers and their family members in all matters relating to their mobility experience.

The 2002 Work Plan defined that the Mobility Centres ´ selection in the participating countries would ensure a “ sound geographic balance and pay due attention to the sectoral experience acquired by specific organisations in areas of particular interest to researchers (social security, pensions, taxation, ent ry conditions, etc.) ”26.

The Mobility Centres committed to the ERA -MORE mission by signing a Declaration of commitment.

• Bridgehead organisations (BHOs) being responsible for coordinating and managing the network at national level and having contacts with th e national authorities and other organisations and liaising with the European Commission. The BHO is the contact for expertise within the framework of the ERA -MORE network. BHOs represent their country when attending BHO meetings and Annual conferences. Th e BHOs provides data and statistics on the national network.

As regards the setting -up of Mobility Centres at national level, three approaches were adopted: Table 2 - Approach followed for the set up of Mobility Centres at nationa l level

Type of National Approach Countries

Creating one Mobility Centre as a single entry point, backed up where necessary by a national mobility network

Cyprus, Czech Republic, Denmark, Finland, Germany, Ireland, Iceland, Lithuania, Latvia, Malta, Norway, Romania, Sweden, Turkey, United Kingdom

Creating several Mobility Centres, each complemented by a network of national or regional correspondents

Austria, France, Hungary, Italy, Slovakia

Creating several decentralised Mobility Centres as a network at national level

Belgium, Bulgaria, Estonia, Greece, Israel, Netherlands, Poland, Portugal, Slovenia, Spain

Source: European Commission, DG RTD 27

26 Towards a European network of mobility centres, EC, DG RTD, Directorate D - European Research Area: the human factor, Mobility policy, April 2002. 27 Commission Staff Working Paper. Second Implementation Report “ A Mobility Strategy for the Euroepan Research Area”, SEC(2004) 412, 1.04.2004.


Page 34

Based on the existence of different approaches of the network and the emphasis on working through existing organ isations when implementing the ERA -MORE network, it is clear that the European network is providing a heterogeneous picture. Moreover, throughout the years the network has expanded and introduced a new type of actors, as from 2008 the (new) Declaration of commitment includes the Local Contact Points (LoCPs) as network members.

The LoCPs offer support on matters at a local level and should have face -to-face contact with the researchers and/or the families.

4.2.4 Tasks As stated in the Declaration of commitment (2004-2007), the members of the ERA -MORE network are required to provide comprehensive and up -to-date information and personalised assistance to mobile researchers and their families. This task could be fulfilled :

• either directly, by providing direct proximi ty assistance to the person(s) requesting help, or

• indirectly, by directing the person(s) concerned to the closest specialised centre, at whatever operational level, which may ensure the required service.

The general mission 28 of all members of the Networ k includes information and proximit y assistance in the following matters such as :

o entry conditions;

o visas;

o work permits;

o recognition of diplomas;

o job opportunities;

o salaries;

o taxation;

o pension rights;

o health care;

o social security;

o accommodation;

o day care;

o schooling;

o language courses;

o access to the culture of the host country;

o other.

28 Declaration of commitment by the members of the European network of Mobility Centres (2004-2007).


Page 35

In terms of reach the ERA -MORE network assisted 55, 608 researchers from September 2006 to December 2007, replied to 98,656 questions in the matters described above. According to the data collection, there has been a steady increase in the number of questions answered and researchers assisted by the ERA -MORE network since September 2006 to December 2007 as shown in the table below. Table 3 - Overview of information and assistance services provided September 2006-December 200729

Overview of information and assistance services provided

01/09/2006-31/12/2006

01/01/2007 - 30/06/2007

01/07/2007 - 31/12/2007

Speed of answering in days 2.31 1.69 2

Number of q uestions 22,946 37,651 38,059

Number of researchers assisted

17,175 18,828 19,605

Source: the European Commission, DG RTD

4.2.5 National Researcher´s Mobility Portal National Researcher’s Mobility Portals (NRMP) were developed complementing the European Researchers' Mobility Portal (ERMP).

The ERMP, launched in 2003 was made available on http://europa.eu.int/eracareers . The ERMP was aimed at improving access to adequate information on available programmes and opportunities as well as on questions related to entry conditions, access to employment, social security rights, taxation, or cultural aspects of the host country. The ERMP is a shared initiative between the EC and the participating countries 30.

The ERMP is seen as a Single Entry Point for:

• Researchers looking to advance their career in another European country ;

• Organisations wishing to recruit the most talented European and non -European researchers .

The ERMP31 would provide the following services;

• general information about research grants and fellowship programmes at EU, international, national or regional level;

• available research opportunities and job offers published by the different actors of the European Research Community (Universities, Industry, Founda tions, etc);

29 The data collection started with a first pilot exercise in September 2006 and was consolidated as from 2007. The evaluator notes that this information has to be treated with caution as there are problems with the comparability and accuracy of the data (different processes and practices in collecting and reporting the data). 30 Commission Staff Working Paper. Implementation Report 2004 “ A Mobility Strategy for the European Research Area” and “Researchers in the ERA: one profession, multiple careers”, SEC(2005) 474, 6.04.2005. 31 Commission Staff Working Paper. First Implementation Report “A Mobility Strategy for the European Research Area”, SEC (2003) 146, 4.02.2003.


Page 36

• information about administrative and legal issues (conditions of entry, social security and tax schemes etc) as well as about cultural/intercultural and family related aspects;

• tailored and customised help -desk function through the European Network of Mobility Centres;

• general information about research policy issues relevant for the career development of a researcher; and,

• other services of particular interest for mobile researchers.

The ERMP is complemented with National Researcher´s Mobi lity Portals. In 2006, there were 31 NRMPs32 fully operational and integrated within the ERMP.

The development of the NRMPs was seen as essential for the successful implementation of the Mobility Strategy at both national and European level. The NRMPs have been set up to complement information provided by the ERMP. In doing so, they all fall under a specific scope: to provide references to job and funding opportunities at national level, and to provide information about national regulations and procedure as well as about administrative and cultural issues relevant for mobile researchers 33.

In terms of reach, the table below shows, as per data collected from the BHOs in each country, the number of unique visitors and page views in total for the NRMPs. Table 4 - Visits & pages viewed-NRMPs 2006-2007

Overview-NRMP visits and page views

Question 01/09/2006-31/12/2006

01/01/2007 - 30/06/2007

01/07/2007 - 31/12/2007

Unique visitors 34 of NRMP 395,862 720,314 360,405

Number of page views 35 1,549,957 4,516,287 1,416,533 Source: the European Commission, DG RTD

4.2.6 Target Group The key target groups of the ERA-MORE network as defined in the Declaration of commitment (2004-2007) are the researchers and the researchers ’ families.

4.2.7 EC Support The EC provided support for a specific programme “Structuring the ERA” under the Sixth Framework Programme (FP6) with a view to co -financing the setting up of the network at

32 In 2006, as reported in the Implementation Report, SEC(2007) 1074, there were thirty-one national Researcher’s Mobility Portals already available: AT, BE, BG, CH, CY, CZ, DE, DK, EE, EL, ES, FI, FR, HU, IE, IL, IS, IT, LT, LV, MT, NL, NO, PL, PT, SE, SI, SK, TR, UK. 33 Declaration of commitment for the maintenance of the NRMP. 34 Unique visitor- refers to the number of visitors who visit a site during a certain period of reporting time (hours, days, one month) and “unique visitors” can make more than one visit during a period, they will in fact be counted only one time. 35 “Page views” refers to all complete pages displayed. Images, graphics and all other page components as well as pages accessed for indexing by search engines, robots and spiders etc are not counted as page views.


Page 37

national level. The EC invited all participating countries to submit proposals and 30 count ries applied for co -financing36. The projects should last between two and three years.

The ERA-MORE network was implemented in the shape of national projects co-financed by FP6 Specific Support Actions. The se projects should last for maximum three years. The national networks were co-funded by the EC at the level of €200,000 per country.

Besides the budget provided under FP6 for co -financing the ERA -MORE network at national level, the EC agreed to provide support for the animation and awareness raising activities of the network. The EC support in this area included: setting up of an Extranet, designing and dissemination of promotional material, organisation of training sessions, BHO meetings, Annual Conferences and the coordination of Working Groups on Quality, Promotion and Annual Conference37.

4.3 Sustainability of the network As from 2008 no further EU funding is foreseen for the continuation and maintenance of the national ERA -MORE networks 38. Thus, the most challenging task for the future of the ERA -MORE network after the EC funding is establishing ERA -MORE as a permanent structure 39.

The sustainability of the network is an issue that requires commitment and investment at both the European and national level. In 2008 the EC continued investing in maintaining the Extranet, organising training sessions, meetings and Annual Conferences, supporting transnational projects, creati ng promotional material , enlarging the network towards new participating countries and promoting the exchange of best practice s. In addition, the ERA –MORE network is now known as “Euraxess services”40 as part of the umbrella brand, Euraxess - Researchers in motion launched in June 2008.

At national level, models for the sustainable operation of ERA -MORE Mobility Centres still have to be developed in some of the participating countries .

Discussions have taken place at different levels, at European and national level.

At European level, the issue of sustainability was brought in a number of meetings at BHO and MoC level; in pa rticular it was the topic of a w orkshop in 2006 41 as well as a t SG HRM level. Discussions during the SG HRM suggest that the added value of ERA -MORE was recognised and that there is a willingness of the countries to continue its service provision in all countries.

36 Austria, Belgium, Bulgaria, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Iceland, Israel, Ireland, Italy, Hungary, Lithuania, Latvia, Malta, Netherlands, Norway, Poland, Portugal, Romania, Slovakia, Slovenia, Spain, Sweden, Turkey, United Kingdom. 37 A specific evaluation on the Communication and Information activities of the ERA-MORE network was delivered by the consultants to the EC in October 2007. 38 Only funding for trans-European network activities is foreseen. 39 Commission Staff Working Paper, Mobility of Researchers and Career Development, Implementation Report 2007. 40 http://ec.europa.eu/euraxess/index_en.cfm?l1=0&l2=2 41 Workshop on Athens, Greece, September 2006. During the workshop, ERA-MORE members discussed, together with policy representatives, ideas and concepts for making the Mobility Centres sustainable.


Page 38

5.0 APPROACH AND METHODOLOGY

This section presents the evaluation approach and methodology . In order to reply to the evaluation questions , as presented in section 2, a specific methodology was designed.

5.1 Evaluation design During the evaluation process we relied on the following main sources of information:

• desk research;

• quantitative data collected through web based surveys targeted at the Researchers’ Community and job providers (universities, research centres, enterprises, foundations that are interested in finding an international audience of talented researche rs);

• qualitative data collected amongst stakeholders, National Authorities, Bridgehead Organisations and Mobility Centres;

More specifically the fieldwork activities consisted of:

• face-to-face interviews with a selection of stakeholders, National Authorit ies, Bridgehead and Mobility Centres at country level (within the 10 selected countries)

• interviews with key EU level stakeholders.

5.2 Evaluation tools and techniques The main tools and techniques that we used are further detailed below. The combination of tools allowed us to draw conclusions based on facts and pe rceptions from the interviewees and surveys.

5.2.1 Desk Review The evaluation team gathered and revised a wealth of information that was provided from a number of sources, mainly being the European Commis sion (including access to the Extranet of the Euraxess portal 42) and the different National ERA-MORE networks43.

The background documentation revised concerning the Researchers’ Mobility Strategy included notably:

• COM(2000) 6 Final Towards a European Researc h Area, 18.01.2000 ;

• COM (2001) 331 Final A Mobility Strategy for the European Research Area , 20.06.2001;

• High Level Expert Group on Improving the Mobility of Researchers, Final report , 4.04.2001;

• Towards a European network of mobility centres , EC, DG RTD, Directorate D - European Research Area: th e human factor, Mobility policy;

42 The ERA-MORE initiative was officially re-branded during the current evaluation as Euraxess. The evaluator refers to the network in this evaluation as ERA-MORE as the period of evaluation is between 2004 and 2007. Nevertheless, the extranet is referred to as Euraxess extranet. 43 The ERA-MORE Network is intended as the network of Mobility Centres created at pan European level. For the sake of clarity, the term “National ERA-MORE networks” refers in this report to the networks set up at national level including the BHO(s), the MoC(s), LoCP(s) and any other actors involved.


Page 39

• SEC(2003) 146 Commission Staff Working Paper First implementation report on a mobility strategy for the European Research Area , 4.02.2003;

• COM (2003) 436 final Researchers in the European Research Area: one profession, multiple careers , 18.07.2003;

• COM(2003) 226 final 3% Action Plan , 30.04.2003;

• SEC(2004) 412 Commission Staff Working Paper Second implementation report on a mobility strategy for the European Research Area , 1.04.2004;

• Declaration of commitment by the members of the European network of Mobility Centres (ERA-MORE) 2004 – 2007;

• Declaration of commitment by the members of the European network of Mobility Centres (ERA-MORE) 2008;

• Commitment of quality assurance for the res earcher's mobility portal 2004.

The reports revised related to the sample of participating countries included notably:

• periodic reports (delivered annually at the end of each reporting period) : including Activity and Management reports;

• final reports 44 (delivered at the end of the project) : including Activity and Management reports and the Audit Certificate;

• periodic project reviews;

• final project review 45.

The documents revised also included those posted on the extranet “EURAXESS”, notably:

• Documents related to Annual conferences;

• BHO meeting minutes;

• Documents related to events;

• Data collection through the Extranet;

• Reporting guidance;

• Success stories;

• Survey and statistics;

• Documents related to training sessions;

• Working Groups reports;

• Workshop minutes.

44 The periodic and final reports were submitted by the BHO(s) of each country 45 The periodic and final reports were reviewed by an external reviewer


Page 40

5.2.2 Online survey The evaluation team conducted two online surveys targeted at:

• the researchers’ community (the potential users of the services delivered by the Network); links to the web -based questionnaire have been asked to be posted on the European Researcher’s Mobility Portal as well as on the National Mobility Portals;

• the universities, research centres and industry (the “providers” of information for the job vacancies on the p ortals); each Bridgehead Organis ation in each country have been requested to sen d an e-mail to the “providers” registered in their databases, with a link to the web-based questionnaire.

The surveys have been hosted on a dedicated site to allow responses to feed directly into a survey software tool.

The majority of questions have been built in the format of closed questions, which ask respondents their level of agreement with a series of statements. A five -point rating scale is used to generate statistics which indicate the degree of agreement or disagreement of respondents. This appro ach provides much greater insights to provide a specific indication of their view rather than a ‘black and white’ yes/no tick -box approach, which provides very limited scope to understand respondents’ views. Whilst closed questions guide respondents to consider key issues and questions that the EC would like to answer, one or two open questions have been introduced to allow additional spontaneous feedback. The survey also included a series of profile questions including function, age, gender, economic profi le and nationality. Some of these questions were optional as in a few cases respondents do not want to provide this type of information. However, experience shows that most respondents are happy to provide profile information.

The web based survey was open for four weeks from 1 st September to 1 st October 2008.

Researchers’ web -based survey

3,468 respondents completed the survey. The web -based survey was divided in four categories of questions aiming at:

• collecting quantitative data about the services and as sistance provided by the MoCs;

• collecting quantitative data about the use of European Researcher ’s Mobility Portal (ERMP);

• collecting quantitative data about the use of National Researcher ’s Mobility Portals (NRMP);

• identifying respondents’ profile. Althou gh those questions were optional, there is a high rate of responses (more than 85% out of the total number of responses).

The table below gives indication on the proportion of respondents who:

• have used the services and assistance provided by the MoC s;

• have searched for research opportunities and/or created a CV on the ERMP;

• have consulted a country -specific Researcher’s Mobility Portal.


Page 41

Table 5 - Researchers’ online survey – Number of responses and % of total replies per category of questions

Category of question Number of responses

Response rate in % of total replies to this question

Researchers having used the services offered by an ERA-MORE Mobility Centre 458 13%

Researchers having searched for research (job/fellowship and gr ants) opportunities and/or created a CV on the European Researcher's Mobility Portal

2,005 58%

Researchers having consulted a country -specific Researcher's Mobility Portal 1,111 32%

Source: Online survey for researchers

In terms of profile of the researc hers, respondents were asked to indicate the country in which they are currently working. The figures below present the distribution of respondents between continents taking into account the countries that have more than 1 respondent as well as the 15 most represented countries (71% of the total amount of responses). Figure 1 - Researchers’ online survey - Breakdown of respondents by continent – in % of total replies

Europe83%

Asia6%

North America4%

Africa4%

Latin America2% Oceania

1%

Source: online survey of researchers


Page 42

Figure 2 - Researchers’ online survey - Breakdown of number of replies by country – 15 most represented countries

82

94

98

102

119

127

130

140

149

163

169

171

209

233

399

0 50 100 150 200 250 300 350 400

INDIA

GREECE

POLAND

UNITED STATES

AUSTRIA

UNITED KINGDOM

ROMANIA

BELGIUM

NETHERLANDS

PORTUGAL

GERMANY

FRANCE

TURKEY

SPAIN

ITALY

Source: online survey for researchers

Regarding respondents’ profile, the results of the online survey indicate that respondents are mainly young researchers working within academic institutions.


Page 43

Figure 3 - Researchers’ online survey - Respondents’ profile (gender, age, type, working institution) – in % of total replies

Male57%

Female43%

<254%

25-3560%

36-4526%

46-558%

>553%

Researcher84%

Supporting staff4%

Partner/family member of a researcher

1%

Other12%

Academic institution (e.g.

university)64%

Public research

institute/organisation (other

than university)

20%

Private research

organisation (e.g.

company)10%

Unemployed / retired

3%Other3%

Source: online survey of researchers


Page 44

Job providers’ web -based survey

192 respondents completed the survey. The web -based survey was divided into the same categories of questions than the one targeting researchers.

The table below gives indication on the proportion of respondents who:

• have been in contact with an ERA -MORE MoC to obtain practical information;

• have used the recruitment tool on the ERMP;

• have consulted a country -specific Researcher’s Mobility Portal. Table 6 - Job providers’ online survey – Number of responses and % of total replies per category of questions

Category of question Response rate Response rate in %

Job providers having been in contact with an ERA -MORE Mobility Centre to obtain practical information 44 23% Job providers having used the online recr uitment tool on the ERMP 86 45% Job providers having consulted a country -specific Researcher's Mobility Portal 60 31%

Source: Online survey of job providers

In terms of respondents’ profile, participants were asked to indicate the country in which their organisation is located. The figures below present the repartition of respondents between continents taking into account the countries that have more than one respondent as well as the 15 most represented countries (85% of the total amount of responses). Figure 4 – Job providers’ online survey - Breakdown of respondents by continent – in % of total replies

Europe96%

Asia1%

North America1%

Africa0%

Latin America3%

Oceania1%

Source: online survey of job providers


Page 45

Figure 5 - Job providers’ online survey - Breakdown of number of replies by country – 15 most represented countries

4

4

5

5

5

8

9

9

10

10

11

12

13

23

32

0 5 10 15 20 25 30 35

SWITZERLAND

IRELAND

UNITED KINGDOM

SWEDEN

TURKEY

NETHERLANDS

FINLAND

POLAND

BELGIUM

GERMANY

SPAIN

PORTUGAL

ITALY

FRANCE

AUSTRIA


Regarding respondents’ profile, the survey indicates that the respondents are mainly representatives of academic institutions or public research institutes. Figure 6 - Job providers ’ online survey - Respondents’ profile (type of organisation) – in % of total replies

Academic institution (e.g.

university)54%

Public research institute other than university

23%

Private research organisation

(e.g. company)15%

Other (start-up, foundation, non-

profit association,…)

8%



Page 46

5.2.3 Interviews with stakeholders Interviews are a key source of primary information f or this kind of evaluation. Three kinds of interviews were carried out.

• Face-to-face interviews with DG RTD officials (3);

Interviews with DG RTD officials were carried out before the fieldwork.

The interviews were carried out in parallel with the desk re search and aimed at deepening the overall understanding of the activities and services delivered by the network. It took place at an early stage of the project, in April and May 2007 ;

• Face-to-face interviews carried out in the ten Member States included in the sample were conducted in July - September 2008;

• Phone interviews carried out with EU level stakeholders (4).

The evaluation team in agreement with the European Commission established the number of interviews foreseen in each Member State as follows:

o 100 interviews in total in 10 selected countries 46:

§ Interviews with BHOs and MoCs;

§ Interviews with National Authorities representatives;

§ Interviews of key stakeholders including:

• Research Community representatives (university, staff responsible for Mobilit y, etc.);

• Researchers (incoming and outgoing researchers);

• Industry representatives.

The relevant stakeholders to be interviewed were identified with the support of the BHOs in each country sampled.

The breakdown hereunder detai ls the interviewees by cate gory. Table 7 - Distribution by category of interviewees

Category Number of interviewees Bridgehead Organisations 10 Mobility Centres 33 Local Contact Points 11 Stakeholders 31 National Authorities 21 Total 106

See Annex 8 for a detailed list of interviews.

The interview guides focused on both qualitative and quantitative aspects. Quantitative in this sense means that we looked for existing quantitative information available to feed the indicators

46 The number of interviews by category of interviewees has differed in function of the selected Member State.


Page 47

defined in the Analytical Fr amework. The sources of the quantitative information were the European Commission itself, existing information at EU level and finally databases and studies available at Member State level.

Interviews took place in English, French, Dutch and Spanish.

5.3 Overall planning of the evaluation project

The evaluation took place in three key phases:

• Preparation, Structuring and Inception Phase that took place from week 18, 2008 to week 22, 2008;

• Data Gathering and Analysis Phase that took place from week 22, 2008 to week 40, 2008;

• Analysis and Recommendations Phase that took place from week 36, 2008 to week 47 , 2008.


Page 48

6.0 Evaluation Question: Have the Mobility Centres (MoCs) reached their objectives in terms of providing comprehensive and up -to-date information and persona lised assistance to the researchers in matters related to mobility?

The main objective of this evaluation question is to assess the effectiveness of the (MoCs) in reaching their operational objectives.

We understood the evaluation question to cover the following main issues related to:

• the alignment of the objectives of each MoC with the operational objectives as stated in the Declaration of commitment and the effectiveness of the MoCs in providing comprehensive and up-to-date information and personalised a ssistance to the researchers in matters related to mobility ;

• the assessment of the mechanisms and means put in place by the MoCs to ensure the delivery of comprehensive and up -to-date information and personalised information to the researchers (direct and indirect approach) ;

• the assessment of the relevance of the services and assistance provided to the researchers.

For the sake of clarity, we have structured the following sub -sections based on these main issues and we analysed them in relation to the judgem ent criteria we formulated in the analytical framework (see annex 1), namely:

• the correspondence between MoCs operational objectives and services delivered (direct and indirect approach) and the extent to which the MoCs provided information and assistance in the issues to be covered as defined by the EC and those that are considered most relevant to users;

• the extent to which the MoCs were regularly trained in order to provide up -to-date information to researchers and the extent to which MoCs develop synerg ies with other networks/specialised centres/multipliers in order to optimise service delivery;

• the correspondence between services delivered and known demand and the extent to which the geographical coverage is comprehensive (location and concentration) an d appropriate to the researchers’ needs.

6.1 Findings 6.1.1 Assessment of the alignment of the objectives of each MoC with the

operational objectives as stated in the Declaration of commitment and effectiveness of the MoCs in providing comprehensive and up -to-date information and personalised assistance to the researchers in matters related to mobility

This first sub -section aims at analysing to what extent the services delivered were in line with the operational objectives of the MoCs. The main specific operational objectives of the MoCs were to provide comprehensive and up -to-date information and personalised assistance to all


Page 49

researchers seeking advice and their families in all matters relating to their mobility experiences. This task might be fulfilled either dir ectly, by providing direct information and proximity assistance to the person(s) requesting help, or, indirectly, by directing the person(s) concerned to the closest specialised centre, at whatever operational level, which may ensure the required service 47.

As stated above, our analysis is based on two judgment criteria;

• the correspondence between MoCs’ operational objectives and services delivered (direct and indirect assistance); and,

• the extent to which the MoCs provided information and assistance in the issues to be covered as defined by the EC and those that are considered most relevant to users.

Correspondence between MoCs operational objectives and services delivered (direct and indirect assistance)

To form our judgment, we looked at:

• The MoCs’ understanding of the ir overall objectives ;

• The MoCs’ understanding of the target audience;

• The MoCs’ understanding of the channels to be used to reach the target audience;

• Users’ assessment of the quality of the services delivered.

The evaluators then formulate overall conclusions on the correspondence between MoC s’ operational objectives and services delivered (direct and indirect approach).

Understanding of the overall objectives of the MoCs

Reaching the basic operational objectives of the network highly depends on the perception that the MoCs’ staff have of it. Therefore interviewees were asked to give their own definition of this objective.

In general, interviewees insisted on the fact that one of the objectives of the network in general terms and of the MoCs i n particular is to be a ‘one -stop shop’ for research -related matters where researchers can seek responses to their queries. The requests received by MoCs tend to relate primarily to either legal regulatory and administrative matters (work contracts and ent ry conditions, salaries, taxation, etc.) or to practical assistance in the fields of housing, job opportunities for the accompanying partner, day care system, education, language courses, introduction to the culture of the host country, etc. In some of the countries visited, the MoCs are also considered to be central points that inform the community of researchers, as well as universities, research institutes and industries, about the European funds and grants available in the area of research.

In addition to this general statement, interviewees highlighted the role of the MoCs, through their BHOs, in identifying and relaying the obstacles to mobility in research both at national and European levels. Moreover, interviewees pointed out that the network acts a s a liaison with national authorities dealing with research issues related to mobility 48.

47 Declaration of commitment for the period 2004-2007. 48 This specific issue will be further analysed in section 9.


Page 50

All stakeholders 49 interviewed in the countries visited shared this understanding of the mission of the MoCs. However, when asked to give their perception of the main o bjective of the network, stakeholders indicated that the main mission of the network is to identify mobility obstacles in order to inform policy decision -makers. In addition, stakeholders indicated that MoCs have also the operational task of facilitating r esearchers’ mobility by providing information and assistance to them and by guiding them through the hosting country. In two countries sampled, stakeholders specifically highlighted the importance of the information concerning job opportunities, European g rants and partnerships in the area of research. Understanding of the target audience

According to interviews with MoCs and BHOs, the MoCs target the researchers’ community in general and third parties acting on behalf of researchers (supervisors, hosting r esearchers, different departments of University, private companies, etc.). However, interviews indicate that the target audience reached by MoCs varies depending on the services each MoC delivers:

• the MoC acts as a Mobility Centre delivering strictly the s ervices as defined in the Declaration of commitment and thus the audiences actually reached are the researchers or third parties acting on behalf of the researchers;

• the MoC acts as a national or regional hub and thus the audience actually reached is not the researchers themselves but third parties acting on behalf of researchers;

• the MoC acts as a Mobility Centre and provides additional tasks as agreed with the BHO. Thus, aside from reaching researchers and third parties acting on behalf of researchers, a specific audience is targeted. In one case, the audience targeted is targeting a national authority and in another case, a MoC is targeting target only industry.

While cooperation with academia seems to be obvious due to the location of numerous Mobility Centres in universities, only a few MoCs are specialised in serving industry needs. In a number of countries further developments to reach the Industry sector are being pursued 50. However, in most of the countries visited, interviewees highlighted that there seems to be no need for the industrial sector to use the MoCs’ services, as they are already well organised to welcome incoming researchers. In addition, the issues of competition are declared an obstacle to involving industry.

In some of the countries sa mpled, BHOs and MoCs agreed that there is a need for further action in order to involve the industry, including more proactive communication campaigns targeting private companies. Some examples of how MoCs tried to involve industry include ‘piggy-backing’ on existing initiatives (including events organised by the Industry) and locating MoCs within the Chambers of Commerce.

Interviewees agreed that when the information was provided, very few companies were willing to find out more about European funds. Ther e was no evidence of information and assistance provided to researchers going to the industry sector, with the exception of one of the visited countries.

49 Cf section 4.2.3 50 The dissemination of information towards this target audience will be analysed in the next evaluation question.


Page 51

Understanding of the channels to be used to reach the target audience

According to interviews with M oCs, the services and assistance to researchers are provided by direct contact which is considered highly important in order to gain proximity with them. Specifically, proximity is considered crucial to reach the objective of providing personalised assistance and customised information. According to interviewees, those close contacts ensure to match the specific needs and requests of each individual user. The personalised and customised dimension of the provision of information and assistance vary depending on the country. Indeed, some of the interviews with MoCs showed that they differ in the extent to which they delivered assistance to researchers (for instance, by helping the researcher with the negotiation of residential lease, legal assistance, translat ion of documents or doctor appointments).

Stakeholders considered that although the first contact with the MoCs often takes place by email, it is very important thereafter for the researchers to be able to have closer contact and thus face-to-face contact. Stakeholders distinguished clearly between the provision of information and personalised assistance. The channels to be used for both approaches are quite different depending on the nature of the need. On the one hand, looking for information is seen as a first proactive step and as such, interviewees pointed out the importance of having the information available on the Internet and the possibility of asking for further information by email. On the other hand, personalised assistance is seen as a second st ep where direct face -to-face contacts are considered essential.

The online survey shows the same findings. Respondents (researchers and job providers ) were asked to indicate the best ways for the ERA -MORE Network to provide mobile researchers with personalised assistance and services. Although no distinction was made between the provision of information and delivery of assistance, the figure below shows that Internet and E -mails are the most appropriate ways of providing information, followed by face -to-face contact. Figure 7 – Online survey - Opinion on the best ways for the ERA -MORE Network to provide mobile researchers with personalised assistance and services – in % of total responses to the question

Internet37%

E-mails39%

Telephone7%

Fax1%

Face-to-face contact

14%

Internet44%

E-mails38%

Telephone5%

Fax0%

Face-to-face contact

13%

Researchers Job providers

Source: Online surveys


Page 52

It is important to note that there is no statistical information available on the profile of the users of the services and the channels used to reach them. The performance indicators uploaded on the, then, ERA-MORE Extranet onl y indicate the number of researchers actually reached.

Users’ assessment of the quality of the services delivered As per online survey of researchers , the figure below shows that in general researchers that have used the services of a MoC considered the se rvices received to be of good quality. A large majority of respondents, 93% either fully or partly agreed that the information provided was up -to-date, while 85% considered it as complete. In addition, 88% of the respondents fully or partly agreed with the statement that the assistance received was personalised.

Figure 8 - Researchers’ online survey - Opinion on the quality of services delivered to them – in % of total responses to the question

56%

50%

47%

37%

38%

38%

3%

7%

9%

4%

6%

5%

0% 20% 40% 60% 80% 100%

The ERA -MORE Mobility Centre provided me with up -to-date information

The ERA -MORE Mobility Centre provided me with personalised assistance

The ERA -MORE Mobility Centre provided me with complete information

I fully agree

I partly agree

I partly disagree

I fully disagree

Source: Online survey of researcher s

As per online survey of job providers , when asked the same question on quality of services to respondents , it can be seen that their responses are comparable to those of the researchers. A large majority of respondents, 96%, either fully or partly agreed that the information provided to their organisation was up -to-date while 100% considered it complete. Moreover, 91% of the respondents either fully or partly agreed that the assistance received was personalised.


Page 53

Figure 9 - Job providers’ online survey - Opinion on the quality of services delivered to them – in % of total responses to the question

81%

65%

65%

15%

35%

26%

4%

0%

9%

0% 20% 40% 60% 80% 100%

The ERA-MORE Mobility Centre provided our organisation with information that was up -to-date.

The ERA-MORE Mobility Centre provided our organisation with information that was complete.

The ERA-MORE Mobility Centre provided our organisation with relevant information on funding opportunities (e.g. Marie

Curie actions open to industry)

I fully agree

I partly agree

I partly disagree


According to the operational objectives, MoCs have to ensure that requests are answered within an appropriate timeframe. Based on the data submitted by the MoCs through the Extranet, the speed of answering questions is, on average less than three days (from one to five days).

A question on the job providers and researchers online survey looked at whether the information was received in due time. The figure below shows that 78% of the researchers and 100% of the job providers fully or partly agreed with the statement that the information was provided in due time. Figure 10 – Online surveys - Respondents’ opinion on average speed of answering questions – in % of total responses to the question

I fully agree40%

I partly agree38%

I partly disagree

14%

I fully disagree9%

I fully agree85%

I partly agree15%


Source: Online surveys of researchers and job providers


Page 54

However, it is also important to note that t here is no consolidated data collection on the quality of services apart from the feedback given every six months by MoCs on the speed of answering requests, despite the fact that there has been an attempt to widen the qualitative data to be collected51.

In conclusion the correspondence between MoCs operational objectives and services delivered is met to a large extent:

• Regarding the operational objectives of the MoCs, it is clear that the members of the network understand them as defined in the Declaration of commitment . It is important to note that MoCs in the countries sampled have the same understanding and provide the same level of information.

• Regarding the target audience, the MoCs understand the target audience as defined in the Declaration of commit ment. However, there is a difference between the target audience and the audience actually reached depending on the specific role of the MoC in each national network. Moreover, during the period 2004 – 2007, the network of MoCs was encouraged to involve the Industry52. The findings clearly demonstrate that industry is not reached homogenously by all the MoCs in different EU countries.

• Regarding the channels to be used, the MoCs understand the personal and customised dimension of the provision of information and assistance as being close to the researchers. It is important to note that they favour the use of channels such emails and face -to-face contacts in order to ensure the services correspond to stakeholders’ views and needs.

• Regarding the quality assessme nt of the services delivered, the information provided to users was perceived as comprehensive and up -to-date and the assistance was personalised. MoCs have reached the objective of responding to the researchers within an appropriate timeframe.

Extent to which the MoCs provided information and assistance in the issues to be covered as defined by the EC and those that are considered most relevant to users

The second judgment criterion which seeks to assess the correspondence between the objectives and the s ervices delivered, relates more specifically to the type of information delivered. Regarding the provision of information and assistance, the matters to be covered as defined in the operational objectives are:

• entry conditions;

• visas;

• work permits;

• recognition of diplomas;

• job opportunities;

• salaries;

• taxation;

• pension rights;

51 Cfr Working Group on Quality 23/05/2006. 52 Cfr Working Group on Industry involvement.


Page 55

• health care;

• social security;

• accommodation;

• day care;

• schooling;

• language courses;

• access to the culture of the host country;

• other.

Our interviews indicated that in general all the MoCs in a national network provided all the information as per the categories above. In addition to informing the information seeker, in a number of countries sampled, the MoCs accompanied the researchers to appointments with third parties due to language issues. Examples include helping the researchers with the negotiation of tenancy agreements, providing legal assistance, translating documents or attending doctor’s appointments.

Thus, the provision of information and assistance by MoCs to users was to a l arge extent achieved. However, interviewees pointed out that the end of the EC funding period might have an impact on the delivery of services by MoCs. The issue of sustainability is a key matter that could affect the service delivery.

Interviews with BHOs indicated that the national ERA -MORE networks are likely to be sustained. In most of the cases, national funding will be ensured. However the amount of the funding varies from country to country. Among the countries visited by the evaluators, there are a couple of cases where the future funding of the network still remains unclear. In two of the countries visited, the funding provided by the national authorities went over and above the EC -funding during 2004 -2007, though in one case the MoC has taken on a dditional tasks and responsibilities. It is important to note that in those countries where the national funding is lower than the EC funding received or uncertain, interviewees pointed out that the national ERA -MORE network will be maintained but no further developments will be introduced .

Based on the interviews with MoCs, the most requested information by users is on: entry conditions, visas, work permits, taxation, social security and pension rights, followed by job opportunities, language courses, acco mmodation, child care and schooling. The statistical information available for 2007 corroborates this information gleaned during the interviews conducted in the ten countries where fieldwork was carried out. The figures reported for the period of 2007 are presented below as indicative data.


Page 56

Figure 11 – Data collection - Type of requests for the period 2007 – in number of requests

4333

2728

2231 20821762

1337 1287 1214907 754

519 465 444 373 215 189

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

Source: European Commission, DG RTD

It is important to note that the statistical data presented above is indicative. It is also unclear what the “other” category covers. There is little qualitative data on the services delivered; only three countries out of the ten visited carried out user satisfaction surveys.

Respondents to the surveys were asked to ass ess the accuracy of the information provided on specific issues. The figure below shows that the researchers were satisfied with the information provided.

As per online survey of r esearchers , respondents ranked the information on entry conditions, job opportunities and visas as the most reliable information received, while taxation and pension rights appear at the bottom of the list.


Page 57

Figure 12 – Researchers’ online survey - Assessment of researchers’ satisfaction with the information provided on specific issues – in % of total responses to the question

53%

52%

47%

46%

44%

44%

42%

41%

40%

39%

37%

34%

32%

32%

32%

39%

36%

32%

32%

38%

31%

31%

34%

35%

41%

38%

48%

39%

31%

28%

6%

7%

17%

11%

15%

12%

18%

10%

13%

14%

10%

14%

17%

26%

29%

1%

5%

3%

11%

4%

14%

9%

15%

12%

6%

15%

5%

12%

11%

10%

0% 20% 40% 60% 80% 100%

Entry conditions

Job opportunities

Visas

Health care

Recognition of diplomas

Language courses

Culture of the host country

Accommodation

Social security

Work permits

Schooling

Salaries

Day care

Taxation

Pension rights

I fully agree

I partly agree

I partly disagree

I fully disagree

Source: Online survey of researchers

As per online survey of job providers , the respondents were generally satisfied with the information provided . This being said, the ranking of each individual category of information is slightly different for job providers. The figure below clearly illustrates that the the information on legal and administrative issues is considered accurate, while the information relating to practical information is ranked at the bottom of the list.


Page 58

Figure 13 – Job providers’ online survey - Assessment of job providers’ satisfaction with the information provided on specific issues – in % of total responses to the question

81%

73%

71%

65%

62%

56%

56%

50%

50%

46%

45%

43%

38%

22%

17%

10%

27%

24%

29%

23%

44%

38%

42%

39%

38%

36%

50%

46%

56%

58%

10%

5%

6%

15%

6%

8%

11%

15%

18%

7%

15%

22%

17% 8%

0% 20% 40% 60% 80% 100%

Work permits

Entry conditions

Visas

Social security

Salaries

Taxation

Health care

Pension rights

Job opportunities

Culture of the host country

Schooling

Recognition of diplomas

Language courses

Day care

Accommodation

I fully agree

I partly agree

I partly disagree

I fully disagree


In addition to the data collected through the online surveys, interviews with stakeholders indicated that the legal, regulatory, and administrative matters were the main obstacles to mobility. Interviewees pointed out that the ma in obstacle that mobile researchers face is the administrative burden related to legal matters (obtaining a visa, dealing with social security and pension rights, etc.). This was attributed mainly to the difficulty in understanding the national specificities in these matters and thus the high importance of getting accurate information.

The data presented above indicates that although the MoCs provided information on the categories de fined in the operational objectives , it appears that there is sometimes a d ifference in understanding of the scope of the assistance to be provided to researchers. In four out of ten countries visited, interviewees expressed clearly the need to define more precisely the scope of the level of assistance that the MoCs need to provi de to researchers 53.

When looking at the overall satisfaction coming from the surveys (researchers and job providers), it is interesting to highlight that the respondents were generally satisfied and the information provided is perceived as accurate.

It is also important to highlight that there is no consolidated data collection on the user’s satisfaction though some attempts have been made by some MoCs to monitor user satisfaction.

53 It is important to note that the new Declaration of commitment further defines the missions of the ERA-MORE network. Moreover, the document defines the role, rights and benefits as well as obligations and duties of the MoCs. It was not the case in the Declaration of commitment for the period 2004 – 2007.


Page 59

In conclusion the correspondence between the services to be delivered and t he services actually delivered is met to a large extent:

• Regarding the type of information provided by the MoCs, in general all the MoCs in a national ERA -MORE network provided all the information as per the categories defined in the operational objectives .

• Regarding the most requested information, MoCs’ users more often sought information on entry conditions, visas, work permits, taxation, social security, and pension rights, followed by job opportunities, language courses, accommodation, child care and sc hooling.

• Regarding the information provided, it is perceived as being accurate and satisfactory . The main obstacles to mobility are assigned to the misunderstanding of the legislation and administrative burdens .

6.1.2 Assessment of the mechanisms and means put i n place by the MoCs to ensure the delivery of comprehensive and up -to-date information and personalised information to the researchers

This second sub -section aims at analysing the mechanisms and means put in place by the MoCs to ensure the delivery of com prehensive and up -to-date information and personalised information to the researchers.

Our judgment criteria to address this specific issue relates to:

• the extent to which the MoCs were able to create synergies with other networks/specialised centres/multipliers in order to optimise service delivery; and,

• the extent to which the MoCs were regularly trained in order to provide up -to-date information to researchers.

Extent to which the MoCs were able to create synergies with other networks/specialised centres/multipliers in order to optimise service delivery

Based on interviews with MoCs, although there is a good level of cooperation between ERA -MORE members in the same country, little cooperation has been established with other networks or multipliers in that country.

Interviews also revealed that when transnational signposting occurs, it is most often a referral to other members of the ERA -MORE network compared to other network or organisation abroad.

The online survey results 54 revealed that 25% of the respo ndents (online survey of researchers ) had received indirect assistance, meaning that the MoC they were in contact with redirected them to the closest specialised centre, at whatever operational level, to ensure the required service, while the same applied to 18% of job providers (online survey of job providers ).

54 471 researchers and 44 job providers replied to this question.


Page 60

Figure 14 – Online surveys - Percentage of services delivered indirectly – in % of total responses to the question

Yes25%

No75%

Yes18%

No82%


Source: Online surve ys of researchers and job providers

No clear trend could be highlighted from the online surveys in terms of organisations to which researchers and job providers were signposted. Nevertheless, interviews with MoCs showed that signposting occurred to a large extent within members of the ERA -MORE network in the same country.

Respondents (researchers and job providers ) to the surveys were asked to assess their overall satisfaction with the services delivered to them when redirected to other organisations. The figure below shows that, in general, both researchers and job providers were satisfied with the services and assistance provided by the organisation the MoCs redirected them to. Figure 15 – Online surveys - Assessment of overall use rs’ satisfaction of services delivered to them when redirected to other organisations – in % of total responses to the question

I fully agree38%

I partly agree47%

I partly disagree

11%

I fully disagree

4%

I fully agree29%

I partly agree57%

I partly disagree

14%

I fully disagree0%


Source: Online survey of researcher and job providers


Page 61

Based on the data presented above, the findings indicate that the direct approach is the approach most often used, whereas the indirect approach is employed to a lesser degree. It is not clear to the evaluator the degree to which synergies have been created with other national ERA -MORE networks. Nevertheless, a call for proposal for transnational projects amongst ERA -MORE members was launched in May 2007 55 focused on stimulat ing cooperation between ERA -MORE members on operational and strategic issues .

It could be argued, based on the findings, that synergies with other organisations at both national and international level have not really been developed, despite the existence of informal agreements that have been created based essentially on personal contacts.

Based on the data collected through the online surveys, it is clear that when respondents were redirected to other organisations, they were satisfied with the services provided.

Extent to which the MoCs were regularly trained in order to provide up -to-date information to researchers

Delivering up-to-date information to researchers depends on a number of factors, one of which is the level of knowledge of the MoCs on the specific issues covered by the MoC. Therefore, continuous training is needed to keep the people involved in the MoCs as know ledgeable as possible in the different areas. As stated in the work plan created in 2002 56, “training courses for mobility centres’ personnel will be organised whenever required in order to develop their acquaintance with the technical and/or content requir ements pertaining to their active participation in the Network, and to enhance the opportunities offered by their access to the information and services offered by the latter” .

All interviewees highly appreciated the support the European Commission offered in the area of training. However, interviewees pointed out that the limited number of places offered was a problem, especially in larger networks.

When analysing the list of training sessions for the period 2004 – 2007, it is clear that the themes of the training sessions are related mainly to the development of soft skills (for instance “Intercultural Communication” – 2005). Interviewees also expressed a need to be trained on specific topics related to European regulation in the research area (visa, taxat ion, etc.) as the one held in 2005 about social security (“Legal issues and C&C” – 2005).

Training sessions were highly appreciated, despite concerns raised by some of the interviewees in relation to the content which was sometimes perceived as too generi c. Some interviewees stated that they were in need of more technical information. Interviewees also highlighted the importance of training sessions, in addition to the annual conferences, at European level in order to maintain the network and the exchange of knowledge between the people involved in the ERA-MORE network. Indeed, the perception was that without the European Commission’s coordination of these sessions, the countries could not afford to organise such training at an international level and this would lead to the loss of networking opportunities at European level.

55 Call under the Seventh Framework Programme (FP7).PEOPLE 2007-5-3 ERAMORE. 56 Towards a European network of mobility centres, EC, DG RTD, Directorate D - European Research Area: the human factor, Mobility policy


Page 62

Table 8 - List of training sessions carried out by the European Commission (2004 – 200757)

Year Training session Number of attendees 2004 European Portal and CIR CA (June) N/A 2004 European Portal and CIRCA (October) N/A 2005 External Communication and Media Relations 17 2005 Intercultural Communication 25 2005 Legal issues and C&C 55 2006 Evaluation N/A 2006 Extranet N/A

Source: European Commission, DG RTD - Extranet

Interviewees indicated that, in the future, the content of training sessions should focus on new European issues. There should be information provided by the EC in new developments at EU level. Interviewees also suggested having training session s focused on specific themes and/or regions.

Regarding the use of Extranet to share training content, interviewees indicated that they do not use the tool very often, though they considered it to be good and effective. Interviewees assessed this tool as ap propriate in theory, but it seems that in practice it is not used very much.

There were also training sessions carried out at national level , be it with different time intervals between the countries. Those sessions were organised by the BHOs. In general, the topics of the sessions focused more on specific issues relating to the issues covered by the services and assistance delivered by the MoCs. For instance sessions on visa, entry conditions, and social security were given to MoC staff. In many of the cou ntries visited, representatives from national authorities were often invited to facilitate the sessions. MoCs staff also indicated that they were given by the BHO an initial training session to help them start running their MoC.

6.1.3 Assessment of the relevance of the services and assistance provided to the researchers

This third and final sub -section to this evaluation question aims at analysing the appropriateness of the services and assistance provided to the researchers. To do so, the evaluators have formed judgment criteria related on the one hand on the correspondence between the services delivered and the known demand and, on the other hand, on the extent to which the geographical coverage is comprehensive (location and concentration) and appropriate to th e researchers’ needs.

Correspondence between services delivered and known demand

MoC’s interviews showed that, in general terms, no study had been carried out on the researchers’ needs in the countries sampled. The evaluation tried, but unsuccessfully, to collect data from the countries sampled on the demand coming from the researchers’ community and/or third parties. Very few of them undertook user satisfaction surveys which makes it

57 In the BHO meeting held in 2006, the action plan indicated two training session for 2007 but the evaluators did not find any information about those sessions in the Extranet.


Page 63

difficult to assess the relevance of the services and assistance provided in terms of answering the researchers’ needs.

However, interviews with the users of the services indicated that, in general terms, the provision of services matched their needs for information. In addition, researchers that had been in contact with MoCs c onsidered the services provided to be very important as they allowed researchers to focus on their work rather than on administrative issues. In addition, having a central point where all the information was available was also considered to be very valuabl e. Again, the MoCs are appreciated as a one -stop shop for information regarding mobility issues.

The researchers’ view is also supported by the answers to the survey. Respondents (researchers and job providers ) having benefited from the services and assist ance provided by the ERA-MORE network were asked whether the information provided matched their need for information/services. The figure below shows that 82% of the researchers considered that the services and information provided responded to their needs while 88% of the job providers were satisfied. Figure 16 – Online surveys - Respondents perception of the relevance of the services and information provided – in % of total responses to the question

Yes82%

No18%

Yes87%

No13%


Source: Online surveys of researchers and job providers

To conclude on the above, it can be said that the information and services provided by the MoCs are appropriate to researchers’ needs. However, it is important to highlight the MoCs’ lack of information on researchers’ needs as few of the countries sampled carried out specific studies at national level. There is a few evidence that MoCs focused on specific topics based on an analysis of most frequent requests or evaluation of researchers’ needs and expectations.

Extent to which the geographical coverage is comprehensive (location and concentration) and appropriate to the researchers’ needs

The perception of the interviewees on the geographical coverage of the network in their respective country has to be seen within the context of the approach taken when implementing the network.

The table below summarises the approach followed by the countries sampled to structure their national network.


Page 64

Table 9 - Structure of the countr ies sampled during the period 2004 – 2007

Structure of the MoCs

One Bridgehead and one MoC with a network of contacts Sweden, Ireland, Czech Republic , Lithuania

Two or more mobility centre/Bridgehead organisation with a network of contacts

The Netherland s

Two or more bridgehead organisation and several MoCs Estonia

One bridgehead organisation and several mobility centres Greece, Poland, Romania

One Bridgehead organisation and several MoCs and a network of contacts

Spain

Source: Data collection (interv iews with BHOs)

In all but one of the countries visited, MoC staff perceived the geographical coverage as sufficient and appropriate in order to reach researchers’ needs. However, in some cases, interviews with MoCs indicated that the coverage in their country could be further improved by adding LoCPs at regional level in support of the MoCs. According to these interviewees ‘going local’ was seen as necessary in order to reach as many researchers as possible.

Regarding the hosting institutions of the MoCs, interviewees indicated that in the majority of cases, universities or research institutes host the MoCs. In general, the network was supported by existing foreign offices, international liaison offices or organisations already involved in researcher mobili ty-related issue. However, interviewees also pointed out that in the case of one MoC based in one university but covering multiple universities, the competition between universities should be taken into account and might affect the delivery of services and assistance. In this particular case, and to avoid creating obstacles due to competition between universities, interviewees highlighted the importance of hosting the MoCs in an organisation or structure not related to a single university.

According to inte rviewees, the MoCs are strategically located to reach the researchers’ community within the academic sector rather than those researchers working in the private sector.

Respondents (researchers and job providers ) to the online surveys were asked to assess the appropriateness of the location of the MoCs they were in contact with. The figure below shows that 88% of the researchers either fully or partly agreed that the location was appropriate to their needs, while 90% of job providers either fully or partly agreed that the location matched their needs.


Page 65

Figure 17 – Online surveys - Respondents perception on the appropriateness of the location of the MoCs – in % of total responses to the question

I fully agree60%

I partly agree28%

I partly disagree

7%

I fully disagree5%

I fully agree55%

I partly agree35%

I partly disagree

10%

I fully disagree0%


Source: Online surveys

However, it has to be noted, as previously indicated, that most of the respondents (researchers and job providers ) come from the academic sector.

To conclude, the geographical coverage of the individual national networks is perceived as appropriate in most of the countries visited. It is important to note that interviewees highlighted the need to be local in order to reach the researchers’ community even when there is only one MoC in the country.

Despite acknowledging that the structure of the national networks defers from country to country, it is important to note that in general the MoCs are located within universities or public research institutes. These hosting organisations are considered as appropriate to the target audience actual ly reached; being the researchers working in the academia. It is clear thus that the location of the MoCs focuses on specific target audience and would therefore be less appropriate to others.

Further improvement should be done to involve the Industry sect or. The issue appears to come from the low level of awareness of the ERA -MORE network amongst this group as well as the competition climate Industry conveys. It seems that within the industry sector, companies have departments that are organised and prepar ed to face specifically the researcher’s issues, this department being usually the administrative department.


Page 66

6.2 Conclusions Based on the findings described above, the main conclusions are:

• The MoCs have a good understanding of their operational objectiv es, being the provision of comprehensive and up-to-date information and personalised assistance to researchers and their families in matters related to mobility.

• The MoCs understood their target audience as being the researchers’ community in general and third parties acting on behalf of researchers (supervisors, hosting researchers, different departments of University, private companies, etc.).

• The level of the MoCs’ achievement of the operational objectives highly depends on their understanding of their r ole and the scope of the services to be provided. The MoCs’ understanding of the personalised and customised dimension of the provision of information and assistance varies depending on the country . Indeed, MoCs differ in extent to which they deliver assistance to researchers (for instance, by helping the researcher with the negotiation of residential lease, legal assistance, translation of documents or doctor appointments). To deliver personalised and customised information and assistance, MoCs favour the use of channels such as e-mails and face -to-face contacts to ensure comprehension of stakeholders’ views and needs.

• During the period 2004 – 2007, the ERA -MORE network has realised effectively its operational objectives in terms of providing comprehensive and up-to-date information and personalised assistance to the researchers and their families in matters related to mobility. Generally speaking, the main services provided show a good degree of coherence amongst the MoCs across the countries visited. As described above, there are some differences in the extent and intensity of services provision .

• The services provided by the MoCs are likely to be sustained after the EC funding period. However, the end of the EC funding period might have an impact on the del ivery of services by MoCs. In those countries where the national funding is lower than the EC funding received or uncertain, the national ERA -MORE networks will likely to be maintained but no further developments will be introduced.

• There is a difference i n the audience actually reached depending on the role the MoCs were assigned to within their specific national structure. For some of the MoCs, actions targeting the Industry were pursued. However, few countries visited established an effective cooperation with this target group.

• In the majority of cases, the information provided by the MoCs during the period 2004 – 2007 has matched the users’ needs. To some extent, t he MoCs have implemented mechanisms for seeking feedback from users and data or performance monitoring in relation to signposting, the profile of researchers, researchers’ needs, potential gaps and expectations etc.

• There was a great level of cooperation within individual national ERA -MORE networks among MoCs which strengthens the sense of being part of a network. However, little formal cooperation has been established with other national ERA-MORE networks or multipliers except through informal agreements based on personal contacts within the same country. Little to no signposting occur red at transnational level. If indeed such transnational signposting occurred, it was most often a referral to other members of the ERA -MORE network and not to another network or organisation abroad. Nevertheless, a call for


Page 67

proposal for transnational projects amongst ERA-MORE members was launched in May 2007 focused on stimulating cooperation between ERA -MORE members on operational and strategic issues .

• The MoCs received appropriate support by DG RTD in terms of training sessions that would ensure on the one hand kn owledge of the information to be provided and on the other hand the access to networking and exchange opportunities. The topics of the training sessions were mainly related to the development of soft skills . However, the MoCs expressed a need to be further trained on specific topics related to European regulation in the research area .

• The comprehensiveness and accuracy of the information was also ensured in most of the cases by national training sessions on technical and regula tory issues, organised by BHOs .

6.3 Recommendations Based on these conclusions, the main recommendations are:


• Since the scope of the services to be delivered has been further defined in the new Declaration of commitment , DG RTD should provide further guidance to the ERA-MORE network in order to ensure its commitment to the new Declaration of commitment ..

• A welcome pack could be produced by DG RTD for new employees of the network to ensure that all national ERA -MORE members have written guidelines on their ope rational objectives and assist in the handover process to new joiners .

• DG RTD should encourage the national ERA -MORE networks to carry out surveys aim ed at identifying potential gaps, needs, target audiences and expectations to ensure user satisfaction and appropriate services .

• In addition, DG RTD , in consultation with the members of the ERA -MORE network should clearly define the extent to which the MoCs should target industry by performing a pan-European assessment of the specific service needs of th is sector.

• Training sessions, conferences and events at European level should be continued in the future as these are seen as having a strong leverage to improve the transnational cooperation of the national ERA-MORE networks. On the one hand, involving nationa l networks in activities organised by the European Commission would strengthen the exchange of best practice and knowledge, and reinforce the sense of belonging to a European Network. On the other hand, strong trans national cooperation between national networks would increase and strengthen the signposting activities and thus increase the effectiveness of the delivery of services and assistance. In addition, besides involving the national networks, these activities should target international organisations active in promoting mobility in research in order to widen trans national cooperation between members of the ERA -MORE network to international key players in Research.

• The topics of the training and exchange of information sessions organised by DG RTD should focus on specific topics that can be tackled at European level, such as new European Regulation, etc. National training sessions should cover issues specific to each country specificities.


Page 68


• The national ERA -MORE networks should improve cooperation with other organisations active in the area of research and mobility through formal and informal agreements, common events or projects at national level. This would improve information circulation and increase the effectiveness in delivering the services and assistance. As well, this would likely improve the delivery of services by releasing availabilities of resources involved in the MoCs and by ensuring accuracy and update of information provided .

• The national ERA-MORE network should clearly define an approach to reach industry by performing country specific assessments of the specific needs of the sector within a pan European approach coordinated by the European Commission .


Page 69

7.0 Evaluation Question: Have the Mobility Centres been e ffective in disseminating the information and promoting their services to their main target groups? Have they been effective in creating a network of users of their services?

The main objective of this evaluation question is to assess the Mobility Centres’ effectiveness in reaching their different target groups through the information and services they provide, as well as their effectiveness in creating a network of users for their services.

While incoming and outgoing researchers are the main target group of the Mobility Centres, they also try to raise awareness of the existence of the network among University staff (i.e. research supervisors, administration staff etc. who have links with researchers) as well as industry and private companies (who may act a s job providers), by disseminating information on the network and by promoting their services. A more detailed description of the intended, as well as the target groups actually reached, is presented in a paragraph below. In addition, by disseminating thei r information and promoting their services, the MoCs will have sought to create a network of users for their services, but the extent to which the members of this network interact and actually feel part of a network may vary from country to country.

The evaluation question was understood as covering the following main issues related to:

• the effectiveness of the MoCs in disseminating the information and promoting their services;

• the effectiveness of the MoCs in reaching their target audiences with these acti vities, i.e. the researchers’ community, the universities, the research centres, the industry, the multipliers of information in the research community;

• the effectiveness of the MoC’s in creating a community of users of their services.

The following sub -sections were structured and analysed on the basis of the main judgment criteria set out in the Analytical Framework (Annex 1), such as:

• Existence of National Communication Strategies for the MoC Network;

• Correspondence between known demand for information a nd extent to which MoCs disseminate information and promote their services;

• Extent to which channels and tools used and audiences reached are appropriate to the needs of the specific audiences being targeted;

• Extent of coherence between intended target aud iences and actual audiences reached;

• Existence of best practice in disseminating information and promoting services;

• Extent to which BHOs and DG RTD provide MoCs with the adequate level of support to ensure an effective dissemination and promotion of servi ces;

• Extent to which BHOs and DG RTD provide the support perceived as necessary for creating a community of users;

• Extent to which there is a community of users of services provided by the MoCs;

• Coherence of services provided across participating countries ;

• Extent to which the network continues its activities after the end of the EC co -funding.


Page 70

In order to assess the effectiveness of the MoCs in disseminating the information and promoting their services to the main target groups, the evaluators gathered inf ormation through face -to-face interviews with officials from the BHOs, MoCs and National Authorities in each participating Member State as well as through interviews with a range of key stakeholders, among them several researchers, to gather their opinions and views of the effectiveness of the MoCs. In addition, online surveys for researchers as well as for job providers were launched in order to ask them about their perceptions of the information and services provided by the ERA -MORE network.

7.1 Findings Existence of a National Communication Strategy for the MoCs Network

Few countries seem to have a national communication s trategy for the ERA -MORE network in place. Most of the representatives of MoCs interviewed were not aware of an overarching strategy, but rather indicated that they had to develop their own strategy for communicating their information and services. This may explain the different approaches and methods taken relative to communication in the individual MoCs, as well as the variety of activitie s carried out across each country. For example, some MoCs have a very active approach and try to come up with as many different communication activities as possible, i.e. by organising conferences for their target groups, information days, round tables wit h researchers etc., while other MoCs rely on more “traditional” and well -known activities, such as publications and leaflets.

Correspondence between known demand for information and extent to which MoCs disseminate information and promote their services

Very few of the MoCs interviewed seem to have carried out a study of the needs for information and services of their target groups before developing the individual communication activities. If evaluations were carried out, they were usually not conducted on a large scale or in a structured way for the whole country, but rather as one -off cases by individual MoCs. The information and services that are effectively provided to the target audiences were therefore mainly developed from scratch, without having coll ected evidence that they were really fulfilling the needs of the target audiences.

However, in several cases since the initial implementation of the service, MoCs interviewed have carried out periodic activity -focussed studies of their target audiences’ ne eds. For example, one country launched surveys to ask users for feedback on the quality of the NRMPs and the possibility to provide feedback on the NRMPs was offered in seven out of ten NRMPs assessed. Moreover, in a range of participating countries , feedback possibilities were provided to participants after events. The feedback helped the representatives of the MoCs to refine and change some of these communication activities and tools in order to target them more specifically to the needs of the audiences. However, this process takes time and many MoC representatives stated that additional funding would be needed in order to produce new tools and provide more services to better meet the information needs of the target audience, as per their feedback.

That said, evidence suggests that there are few tools in place to measure the satisfaction of users with their national networks as a whole. Only a few MoCs have carried out national surveys of users’ satisfaction which focus on their activities as a whole. It a lso remains unclear


Page 71

how far the findings of these surveys have been used to improve the situation and if the feedback was taken into account.

Information gathered during the interviews in the different countries also revealed that each national network has a different approach for developing and using promotional material. This can be explained by the fact that the Commission has tended to take a ‘hands -off’ approach in terms of how communication activities under ERA -MORE have been implemented in the participating countries, focusing mainly in producing centrally promotion materials and organising information activities for national ERA -MORE coordinators and a number of policy makers, as was concluded in the previous evaluation of the ERA -MORE network 58. The promotional material mainly used across participating countries includes publications, emails and newsletters. However, the success of all these activities is highly dependent on the number of staff and their availability in each MoC, their motivation and expertise in this area, as well as the financial means available for the activities.

Extent to which channels and tools used and audiences reached are appropriate to the needs of the specific audiences being targeted.

The BHOs and MoCs use a variety of cha nnels and tools in order to reach specific target audiences.

The table below gives an overview of the channels and tools used in the countries included in the fieldwork: Table 10 - Overview of channels and tools used in fieldwork co untries

Channels Tools

Email contact NRMPs

Events Publications (i.e. Researcher guides)

Face-to-face contacts (i.e. help desk etc.) Brochures

Telephone Promotional material (i.e. pens, business cards, stickers, posters etc.)

Network meetings

Training sessions on how to use the NRMPs

Source: TEP/Deloitte

Interviews with the BHOs and MoCs in the ten countries visited for this evaluation showed that the main tool and channel used to target the needs of the specific audiences are the internet via the NRMPs and e-mails, respectively. The two means complement each other in that they help the BHOs and MoCs provide their target groups with quick access to more generic information via the NRMPs and a response to a specific request by e -mail. A number of BHO s and MoCs

58 “Evaluation of Communication and Information activities relative to ERA-MORE and the European Researcher’s Mobility Portal”. The Evaluation Partnership and Deloitte, October 2007.


Page 72

interviewed also stated that they try to reach their target groups through face -to-face contacts, for example during specific events they have organised. Both using electronic means, as well as interacting with the target groups face -to-face, were perceived by the BHOs and MoCs as being appropriate for meeting the target audiences’ needs.

As indicated in section 5, the researchers’ survey results suggest that the use of these means is appropriate, e -mail being the preferred channel. Respondents w ere asked to rate a range of possible means for the ERA -MORE network to provide mobile researchers with personalised assistance and services. The mean chosen by the highest proportion of respondents (84%) was e-mail, while 80% of respondents also believed the internet would be an appropriate way to provide assistance and services. Almost 30% of respondents perceived face -to-face contacts as being an effective way of providing assistance.

Job providers concurred, indicating in the online survey that the be st channels for the ERA -MORE network to provide assistance and services are via internet and e -mails. Significantly fewer respondents mentioned face -to-face contacts (23%), while very few job providers indicated telephone contacts as the best way to provid e assistance and services (10%).

Furthermore, since both the researchers and job providers stated a preference in the surveys for receiving information about the network via the Internet, and the BHOs and MoCs stated during the interviews that they conside r their NRMPs to be a priority (even though frequent updates remain a challenge, as shown in section 7), it can be said that the use of the Internet by the BHOs and MoCs is an appropriate channel to meet the needs of the specific target audiences.

By including job search opportunities on their websites, seven out of the ten countries visited are providing researchers with a service that they value.

Finally, participants of the focus groups conducted for the previous evaluation 59, indicated that colleagues and networks were two of the most important sources of information when moving abroad. While the BHOs and MoCs try to make available as much information as possible through emails and the NRMPs, the current evaluation’s findings show that not many NRMPs are interactive, offering chatrooms or blogs, and few MoCs offer opportunities to meet researchers from other institutes through events. These two findings combined suggest that offering more opportunities for interaction among researchers in both the virtua l and real world would be a good means to better meet the needs of the target audiences.

Extent of coherence between intended target audiences and actual audiences reached

All interviewees in the countries visited by the evaluation team indicated three mai n intended target groups for the network: a) Incoming researchers; b) Outgoing researcher; c) Multipliers (i.e. supervisors, different University departments, private companies and the industry).

However, based on interviews with stakeholders, there is evi dence to suggest that there is a lack of coherence between the intended audiences and the target audiences actually reached in the participating countries. There seem to be different levels of awareness among the target audiences, as highlighted by the gra ph below.


Page 73

Figure 18 - Overview target audiences reached

Source: Result from the analysis

The target group best reached by the networks’ information and services is the universities’ key support staff (within areas such as, human r esources and research support) who deal with incoming and outgoing researchers. This was also found to be the case during the previous evaluation of the ERA -MORE network60 which concludes that: “ The focus of national ERA -MORE coordinators has primarily been on involving administrative staff at universities, and depending on them as multipliers to get the relevant information to the researchers. ”

Moreover, in line with the findings presented in section 5, according to the interviews carried out during the fieldwork, the supervisors of researchers were to some extent aware of the network, though not to the same extent as the universities’ key support staff.

Researchers only knew about the ERA -MORE network to a limited extent. This finding can be corroborated by the evidence gathered during the previous evaluation of the ERA -MORE network61 which found that awareness of the ERMP among the researcher community was low, but that awareness levels were higher for the ERMP than for the ERA -MORE network itself . Based on the interviews carried out during the fieldwork, it appears that incoming researchers are generally more aware of the network than outgoing researchers.

60 “Evaluation of Communication and Information activities relative to ERA-MORE and the European Researcher’s Mobility Portal”. The Evaluation Partnership and Deloitte, October 2007. 61 “Evaluation of Communication and Information activities relative to ERA-MORE and the European Researcher’s Mobility Portal”. The Evaluation Partnership and Deloitte, October 2007.

University key support staff

Supervisors (of hosting departments)

Private companies and Industry

Researchers (incoming and outgoing)

Best-reached target group

Least-reached target group


Page 74

Interestingly, private companies and industry were indicated by interviewees to have been the least reached target groups in all countries. There seems to be a lack of cooperation between the network and these entities as private companies tend to rely on their own networks and exchange programmes rather than on university facilities. A couple of MoCs a cknowledged that more work needed to be done in this area and that they planned to build relationships with industry in the future with the support of the local authorities.

Extent of best practice in disseminating information and promoting services

While each individual MoC seems to disseminate information and services to a different extent, the evaluation team could identify some examples of best practices which allowed the MoCs to effectively reach the intended target audiences. The interviews carried ou t as part of the fieldwork suggested , across the board , that examples of best practice include the organisation of seminars and conferences for different target audiences, such as incoming and outgoing researchers, but also for administrative staff at univ ersities and research institutions. During these events the target groups receive highly relevant information and can ask questions to experts, including relevant national authorities, as well as, in some cases, meeting members of the network.

Other examples of best practice pointed out by interviewees were the production of a welcome -pack for incoming researchers which is handed to them on their first day at the new research institution, or the organisation of “international cafés” or language courses wher e foreign researchers can meet. In one instance, a BHO stated that it found it best to try to promote the network during relevant events rather than trying to organise stand alone events themselves, so that relevant target audience or a critical mass would be reached. Finally, on one NRMP assessed, testimonials of researchers’ individual experiences in -country were included on the website, providing a more personal element to the website.

Extent to which BhOs and DG RTD provide MoCs with the adequate level of support to ensure an effective dissemination and promotion of services.

Most staff at the MoCs indicated that they did not have a lot of interaction with DG RTD itself. The role of DG RTD was mainly perceived as providing financing and producing a range of rather general information support tools, including brochures, pens, etc. Additionally, DG RTD organises a number of annual conferences and workshops for the ERA -MORE members which interviewees said provides a good platform for the exchange of informat ion and best practice, as well as networking.

Even though the MoCs receive some publications from the DG RTD, a couple of national networks would consider further financial support from DG RTD necessary in order to better promote and effectively dissemina te their information and services. In a number of the countries visited by the evaluators, it has not been possible for them to update certain promotional material since the end of the EC co -funding, as the financial means provided by the national authorities to the networks is less than what was provided by the EC. As such, these national networks only carry out the principal tasks necessary to disseminating information and promoting their services.

Concerning the Extranet the feedback is mixed. Both posi tive and negative feedback w ere provided by the respondents, on both the functionalities and the frequency of use. While some perceived the Extranet as having improved since its creation and as a good tool for facilitating contacts between the users, other s stated that they only used the Extranet to upload their data.


Page 75

Interviewees tended to concur that the main support to the MoCs is provided by the BHOs. They are the main contact point for the MoCs and also act as an intermediary between the national netwo rks and DG Research. The BHOs are usually the institutions which coordinate and organise the network in the countries. They most often produce the national “Guide for Researchers”62, which was considered one of the most important publications by the majorit y of MoCs. There seems to be a very good relationship between the BHOs in the different countries and the MoCs, and the overall support received was perceived as being adequate.

Extent to which BHOs and DG RTD provide the support perceived as necessary for creating a community of users.

As indicated in section 5, according to the online survey results, researchers and job providers were satisfied with the quality of services provided 63.

DG RTD and BHOs within the Quality Working Group have suggested questio nnaires to follow up on the users satisfaction , but fieldwork showed that this recommendation was not followed in a systematic manner in all countries.

During interviews MoCs stated that researcher´s satisfaction was an important aspect of their service delivery in all countries. Interviews with MoCs show that researchers come back to the MoCs as long as they require assistance but once they have been provided this assistance the services of the MoCs are no longer required. Moreover, some MoCs argued that further training and/or information could be given to them so that they can assist better their users, for example in issues that are pan -European (i.e.new legislation).

Therefore, it could be argued that BHOs and DG RTD have taken actions to support the M oCs in assisting their users , but there is no evidence on the extent to which the support received has assisted the MoCs in creating a community of users.

Extent to which there is a community of users of services provided by the MoCs.

It is important to cl arify what exactly is meant by a ‘community of users of the services provided by the MoCs’. In a first instance, this can be defined in broad terms as a generalised network among the research community of users of the services provided by the MoCs. Since the network is not well known among researchers and the research environment in broad terms 64, the extent to which there is a community of service users in this sense of the term is very limited. While the survey and interviews conducted as part of the field work suggested that users of the information and services provided by the MoCs are generally very satisfied, the interviews with researchers and administrative and support staff in universities suggested that they usually do not feel part of a European net work and are mainly bound to their university networks or research groups. This finding is also supported by the conclusions drawn during the previous evaluation of the ERA -MORE network 65. During both studies, it was found that it is generally questionable to what extent users are aware of the existence of the ERA -MORE network as a European initiative, as the MoCs are generally embedded in the university system, which makes it difficult for external users to realise that they are part of a European network.

62 In one instance the production of this guide was tasked to one particular MOC. 63 Section 5, Online Survey, question on users´ assessment of the quality of services delivered. 64 “Evaluation of Communication and Information activities relative to ERA-MORE and the European Researcher’s Mobility Portal”. The Evaluation Partnership and Deloitte, October 2007. 65 “Evaluation of Communication and Information activities relative to ERA-MORE and the European Researcher’s Mobility Portal”. The Evaluation Partnership and Deloitte, October 2007.


Page 76

However, if one focuses on those researchers that are aware of the services provided by MoCs, there is evidence that those researchers who were in contact with the MoCs remain in touch thereafter. As per the online survey of researchers , the findings show that 65% of respondents who used the information and services of an MoC previously are still in touch after their initial contact. This suggests that researchers who are aware of the services provided by MoCs will contact the MoCs with further questions if they have any. In this sense a ‘community of users of the MoCs services’ has been created, though on a smaller scale to that defined above due to the limited levels of awareness of the services provided by MoCs among the wider research community.

Figure 19 - Extent to which researchers are still in contact with an ERA -MORE Mobility Centre

Yes65%

No35%


Coherence of services provided across participating countries.

There seems to be a broad level of cohere nce in the basic level of services provided by the MoCs across participating countries in terms of dissemination and promotion. The key information and services provided across the network include the production of the “Guide for Researchers”, brochures, n ewsletters, emails and the NRMPs. However, as mentioned above, certain countries visited tend to be more extensive and creative in the services they provide in terms of dissemination and promotion than others. This seems to depend on the number of staff working in the MoCs, their experience in carrying out information and dissemination activities and the financial means available to them.

In addition, all the BHOs organise training sessions for the rest of the national network. These sessions enable the par ticipants to meet staff of other MoCs and to exchange examples of best practice.

Extent to which the network has continued its dissemination activities since the end of the EC co-funding.

As described above, in most countries visited , the funding of the ne twork will be or has already been taken over, at least in part, by national authorities after the end of the EC co -funding period. Although there are a few exceptions on this, the funding offered by national authorities to support the national ERA -MORE network is usually lower than the co -funding provided by the EC during 2004 -2007.


Page 77

At the time of the writing , it remained unclear what impact lower funding levels would have on future information dissemination activities by the different national ERA -MORE networks. Thus far, the dissemination activities of the network have continued in the countries visited, be it sometimes to a less frequent degree. In some cases interviewees stated that they did not have sufficient funds to update certain promotional mat erials in the future, e.g. the publishing of a new version of the Researcher’s Guide to the country. In some countries , the members of the national ERA -MORE network have also been asked to provide support to the MoCs dissemination activities, such as by p roviding cost -free venues for dissemination activities or by financially contributing for copies of the Researcher’s Guide to the country.


• Most countries do not have a national communication strategy for the ERA -MORE network in place.

• Only a few of the countries visited had carried out an initial study of their target audiences’ needs before developing their communication activities. However, t here is evidence that suggests that varying efforts have been undertaken to disseminate information and promotion of services to the main target groups.

• The level of awareness of the existence of the ERA -MORE networks and the services it provides appear to be rather low amongst the target groups. The target groups best reached (in decreasing order) are u niversities’ key support staff dealing with researchers, supervisors /hosting professors and the researchers themselves. The industry target group (companies, etc) seems to remain largely unawa re of the existence of the network. As the previous ERA-MORE evaluation 66 also suggests, many of these target groups are not aware of the ERA -MORE network as such, because most MoCs are embedded in the university system and most researchers associate their services with the international offices within universities or HR department of the university .

• National ERA-MORE networks have promoted the network to some extent through events relating to EU funding (Framework Programmes), scientific conferences.

• MoCs see the BHOs as being supportive of their information and dissemination activities.

• The MoCs visited principally perceive DG RTD as a source of generic promotional material.

• There is a low level of awareness among the target groups of the information an d services offered by the MoCs. Those who have made use of the information and services provided by the MoCs might be aware of the network, but most researchers and administrative and support staff in the universities do not tend to feel part of a European ne twork.

• Since the end of the EC co -funding period, dissemination activities of the network have continued, though sometimes to a less frequent degree. Moreover, the interviews indicated that the activity of information and dissemination is likely to be aff ected by a lack of funds in the future.

66 “Evaluation of Communication and Information activities relative to ERA-MORE and the European Researcher’s Mobility Portal”. The Evaluation Partnership and Deloitte,October 2007.


Page 78



• BHOs should be encouraged to put together a c ommunication strategy that could be applied throughout the network by the MoCs to ensure effective communication, consistency and transparency in the participating countries. A more strategic, integrated approach should be followed regarding the dissemination of information and promotion of the network ser vices to given target audiences, rather than undertaking ad hoc activities. MoCs should be encouraged to take a more coherent approach targeted at specific audiences’ needs.

• Attention has to be paid by DG RTD to the impact of the end of EC co -funding on the dissemination activities performed by the ERA -MORE networks.


• MoCs should be encouraged to seek periodic feedback from users regarding dissemination and promotion activities and act upon this where possible in order to better meet the needs of their target audience. Large -scale evaluations of the networks’ activities on a pan European level should be continued in order to monitor whether all activities carried out effectively reached the intended target groups and met their needs.

• The exchange of best practice should be encouraged at all levels within the network in terms of the channels used for the dissemination of information ( e.g. by giving presentations on the network at events organised by third parties including organisations an d companies which target relevant audience) and maximising all the opportunities where the promotion of activities can be pursued (e.g. by ERA-MORE network members distributing information packages to researchers upon their arrival in-country)

• Information and dissemination activities could be made more personal by , for example, organising events with testimonials from mobile researchers, giving researchers the possibility to meet MoCs and other researchers , etc.


Page 79

8.0 Evaluation Question: Have the National Mobility Portals provided comprehensive and up-to-date information and have they been effectively maintained by the BHOs and/or MoCs?

The main objective of this evaluation question is to assess the effectiveness of the National Researcher’s Mobility Portals in reaching their operational objectives as well as the effectiveness of the MoCs and BHOs in the maintenance of the NRMPs.

The ERMP and the NRMP are tools that were developed to help researchers overcome numerous obstacles of an administrative, cultural and linguistic nature. They intend to support researchers in identifying training and job opportunities, as well as helping them to solve more practical problems (i.e. questions related to housing, social security etc.) throughout Europe by providing comprehe nsive and up -to-date information. While the EC is responsible for the running and updating of the ERMP, the BHOs in the individual countries take care of the maintenance of the NRMPs.

This evaluation question was understood as covering the following main i ssues:

• the alignment of the objectives of each NRMP with its operational objectives;

• the effectiveness of the MoCs and/or BHOs in the maintenance of the NRMPs.

The sub-sections presented below have been structured and analysed on the basis of the judgment criteria as set out in the Analytical Framework (see annex 1), including:

• Correspondence between operational objectives and delivery;

• Extent to which NRMPs use the available information from the European Researcher’s Mobility Portal;

• Extent to which the si gnature of the Commitment of Quality Assurance for the NRMPs is followed by signatories;

• Extent to which the NRMPs have been implemented as foreseen (timeframe and its contents, i.e. public, private area and back office);

• Complementarity amongst ERMP and N RMPs;

• Correspondence of tools developed by EC and usage of those tools by the organisation in charge of updating the NRMPs;

• Extent to which the services have continued since the end of the EC co -funding.

8.1 Findings Correspondence between operational objecti ve and delivery

Even though the NRMPs were developed as a tool to help researchers find training and job opportunities as well as solving administrative problems, the findings from the survey for researchers show that 68% of respondents have never consulte d a country-specific Researcher Mobility Portal, while 57% of respondents stated that they have searched for research (job/fellowship and grants) opportunities and/or created a CV on the ERMP.


Page 80

Figure 20 – Extent to which researche rs have consulted a country -specific Researcher's Mobility Portal

Yes32%

No68%


Those users who did access the NRMPs were generally positive about the information they received on the website. Almost 82% of respondents to the rese archer survey thought that the information provided by the NRMP they used most often matched their need for information , and 86% of respondents did not have any recommendations on how to improve the NRMPs. In addition, 32% of respondents to the researcher survey fully agreed and 47% partly agreed that the navigation of the portal they had visited was user friendly, while 20% fully agreed and 40% partly agreed that information about national regulations, administrative and cultural issues was easy to find. A total of 22% of respondents to the survey for researchers fully agreed and 39% partly agreed that the information about national regulations, administrative and cultural issues provided by the NRMP they used most often was up -to-date.


Page 81

Figure 21 - Researchers’ opinion of the extent to which the information provided by the National Researcher's Mobility Portal they used most often matched their need for information

Yes82%

No18%


Figure 22 - Researchers’ opinion on the following statements concerning the information provided by the National Researcher's Mobility Portal they used most often

33%

21%

49%

43%

13%

27%

6%

9%

0% 20% 40% 60% 80% 100%

The navigation was user friendly

The information about national regulations, administrative and cultural issues is easy to find

I fully agree

I partly agree

I partly disagree

I fully disagree


Moreover, the findings of the job provider survey show that 80% of respondents thought that the information provided by the NRMP they used most often matched their organisation’s need for information. In addition, 34% respondents to this survey fully agreed and 43% partly agreed that the information provided by the NRMP they used the most was sufficiently up -to-date.


Page 82

Figure 23 – Job providers’ opinion on the following statement concerning the information provided by the National Researcher's Mobility Portal they used most often

36% 46% 12% 6%

0% 20% 40% 60% 80% 100%

Generally speaking, the information provided is sufficiently up -to-date

I fully agree

I partly agree

I partly disagree

I fully disagree

Source: Online survey for job providers

Extent to which NRMPs use the available information from the European Researcher’s Mobility Portal

In general, the NRMPs differ in the extent to which they make use of the available information from the ERMP. Even thou gh the NRMPs as a tool are considered a priority by most national ERA-MORE networks, interviews with representatives of the BHOs gave evidence that the individual NRMPs are updated at differing frequencies in each country, largely depending on the availabi lity of staff to carry out the task. It seems that in many cases there are no formal procedures for updates in place, unless the maintenance of the NRMP is allocated to a specific webmaster or outsourced to a private company. This often leads to the NRMPs not accessing and using the information on the ERMP as often as they could.

Extent to which the signature of the Commitment of Quality Assurance for the NRMPs is followed by signatories.

Evidence collected suggests that the signature of the Commitment of Quality Assurance 67 for the NRMPs is followed to large extent by the signatories.

In terms of interoperability , the majority of NRMPs assessed as part of this evaluation have the ERA-MORE logo on their websites (seven out of ten NRMPs assessed), and all NRM Ps

67 When signing the Commitment of Quality Assurance for the Researchers Mobility Portal, signatories commit to working towards the widest possible access to the resources of relevant information for mobile researchers through internet based web sources. The document lists a number of commitments that signatories officially agree with when signing, such as interoperability, i.e. of formats; ownership of the information provided; assurance of the quality of information provided; accessibility of the information to all researchers; principle of non-discrimination; confidentiality of the users; the common ownership of the project, meaning the exchange of information and the sharing of experiences.


Page 83

assessed have a link to the ERMP. Seven out of ten NRMPs assessed provide a link to the other European MoCs and eight have a link to the European database for job vacancies. Seven out of ten NRMPs assessed also have a national job vacancies section, whi le six NRMPs provide a free text search option. However, seven of the ten assessed NRMPs have broken links, which indicates that the websites are not as regularly checked or updated as they could be.

The majority of NRMPs assessed by the evaluators comply to a large extent with the commitment to quality assurance . Most NRMPs assessed (six out of ten) are accurate and present a disclaimer, but only five NRMPs provide evidence of the date of the last update. The evaluators found that nine of the ten NRMPs assessed offered an appropriate level of information to users and all NRMPs assessed for this study used an impartial formulation of the information.

The NRMPs assessed are also generally perceived as being user -friendly (this is also shown by the findings of the survey for researchers - see above), providing accessible and easy -to-find information (see also the findings of the surveys for researchers and job providers above). Information that was lacking in several cases included the explanation of acronyms ( five out of ten NRMPs did not explain them), and eight of the ten NRMPs assessed did not provide a print -friendly option. The vast majority of NRMPs assessed (nine out of ten) had complete coordinates, while eight NRMPs made a clear distinction between bas ic and complementary information on the websites.

In terms of the principle of non-discrimination , all ten NRMPs evaluated formulate the information in a non -discriminatory way.

Finally, as regards confidentiality , the interviews with BHOs suggest that only in few cases is confidentiality actually taken into consideration, though permission to use information provided by researchers is requested on occasion. In one instance confidentiality was said not to represent an issue as the data on researchers was n ot stored, while in another , researchers were asked to accept that their details be stored in a database when registering with the NRMP.

Extent to which the NRMPs have been implemented as foreseen (timeframe and its contents, i.e. public, private area and back office)

In the overwhelming majority of cases, the NRMPs were set up within the scheduled timeframe, despite some ongoing discrepancies in terms of templates, content and logo, as highlighted in the section below.

Based on the evaluators’ assessment of the NRMPs, most websites provide the required information to researchers, i.e. information on entry conditions, visas, residence & work permits, taxation, social and health insurance, recognition of diplomas, pension, accommodation, day care and schooli ng, language and culture, job opportunities, fellowships and salaries. However, in some cases the evaluators noted that some of this information was missing, mainly on the recognition of diplomas, but in individual cases also on pensions, taxation and day care/schooling.

In all countries the NRMPs were created from scratch as a completely new tool by the BHOs. While some BHOs used in -house IT staff for the development of the portal, others employed an external company for the set -up. The way the information was gathered and compiled by the BHOs for the websites varied greatly from country to country, it ranged from the BHO signposting to external websites, to the national authorities or MoCs providing relevant inputs directly to the BHOs.


Page 84

Complementarity am ongst European Research Mobility Portal and NRMPs. / Correspondence of tools developed by EC and usage of those tools by the organisation in charge of updating the NRMPs.

Evidence suggests that there are some issues in terms of the correspondence between t he tools developed by DG RTD and the usage of those tools by the BHOs. Some of the NRMPs have a different format compared with that of the ERMP. Some representatives of the BHOs stated that when the NRMPs were initially set up, DG RTD did not make it manda tory to follow the ERMP format. As the ERMP and the NRMP have differing purposes (in that the NRMPs provide country -specific information on a variety of issues including visas, work permits etc., whereas the ERMP does not provide such information), severa l countries chose to have their own template and functionalities developed for their NRMP.

Besides the general format, some NRMPs also do not feature the ERA -MORE logo (three out of the ten assessed),

Representatives of the BHOs generally recognised the overall added value of the re -branding of the European and national websites, but in several countries visited , flexibility on the format and functionalities of the NRMP was requested.

In one case it was also pointed out that the job opportunities templat e on the NRMP was perceived to be complicated to fill in, in the sense that there were too many specific fields which made it very prescriptive, and should be revised; the BHO has not been able to simplify it as it follows the template provided by DG RTD.

Extent to which the services have continued since the end of the EC co -funding

The NRMPs have continued to exist after the EC co -funding stopped in all 10 countries visited for this evaluation, pointing to the sustainability and importance attributed by th e national ERA-MORE networks of the NRMPs as a tool. However, interviews with representatives of the BHOs showed that in some countries, the NRMPs are less frequently updated than before, as there is a lack of funding to employ staff for this task.


• NRMPs are recognised by all members of the ERA -MORE network as being an important communication tool . However, the updat ing of the content of the NRMPs is not subject to formal procedures in most countries . Moreover, information from the ERMP is not always systematically transposed to the NRMPs.

• Despite some continued discrepancies in terms of format and content, there is evidence to suggest that the implementation of the NRMPs was car ried out to a satisfactory extent, with timeframes usually being met and the majority of information required being made available to researchers.

• Reaching complementarity between the ERMP and the NRMPs was shown to be challenging, with various NRMPs havin g different formats and set -ups than the ERMP. According to the ERA -MORE network members this could be due to different needs, as well as to their understanding of certain elements being mandatory or not.


Page 85

8.3 Recommendations Based on these conclusions, the m ain recommendations are:


• DG RTD should assist the national ERA -MORE network in adapting the NRMPs to the new branding requirements. To some extent, the users should be able to recognise the vi sual identity of the ERA -MORE netw ork across the countries. However a pragmatic approach should be applied by DG RTD in terms of compliance with the set timeframe, functionalities and layout.


• The members of national ERA -MORE networks should continue to a ctively promote the NRMPs as they are key in providing up -to-date and comprehensive information to a wider public of users during all the activities carry out when promoting the ERA -MORE network.

• All national ERA -MORE network should be encouraged to estab lish formal rules and procedures to regularly update the info rmation provided on the NRMPS. For those countries in which this is not yet/no longer the case a (external or internal) webmaster should be appointed to follow up and coordinate the information u pdate process.

• The information provided on most of the NRMPs could be further enriched and/or made more interactive by providing personal testimonials of researchers, for example, by asking researchers to write a brief paragraph of their experience in -country or looking to create a space for exchange of personal experiences via a chatroom or blog.


Page 86

9.0 Evaluation Question: Have the Bridgehead Organisations and the Mobility Centres reported their activities and results back to the Commission in a structured and effective way?

This sub-section aims at answering the question in the Analytical Framewo rk: “Have the Bridgehead Organisations and the Mobility Centres reported their activities and results back to the Commission in a structured and effective way?”

The key issue in answering this question relates to the level of correspondence between the agreed reporting and the actual reporting from Bridgehead Organisations and Mobility Centres to the European Commission.

The sub-sections presented below have been struc tured and analysed on the basis of the judgment criteria as set out in the analytical framework (see annex 1), including:

• Extent to which the BHOs reported the activities of the network;

• Extent to which the reports fulfil the European Commission´s requirem ents;

• Extent to which the reports fulfil the European Commission´s needs.

9.1 Findings The overview of the reporting produced by the ERA -MORE network for the period 2004 to 2007 is presented in the table below. Table 11 - ERA-MORE network - Reporting 2004-2007

Type of reporting

Responsible for collecting information

Start date

End date Short description of the reporting

Frequency

FP6 Reporting

BHO Start of FP6 contract (Start of EC-funding)

End of FP6 contract (End of EC-funding)

FP6 reporting consists of a number of reports that were agreed with the EC. These reports monitor on the process of the project and are linked to payments. The reports are submitted by the BHO to the EC.

Annually and at the end of project

Data Collection

BHO Sept. 2006

Ongoing BHO- reports on the NRMP: • page views; • unique visitors The data is uploaded on the Extranet.

Every six months

Data Collection

MoC Sept. 2006

Ongoing MoC- reports on : • number of

questions received

Every six months


Page 87

Type of reporting

Responsible for collecting information

Start date

End date Short description of the reporting

Frequency

in total; • number of

questions per category;

• number of researchers assisted;

• average speed of answering a question (in days).

The data is uploaded on the Extranet.

Soft Reporting

BHO 2007 Reporting carried out only for 2007 due to low response rate.

Reporting of the overall situation of the ERA -MORE national network, including latest developments of the network in the specific country and activities.

Annually.

Source: TEP / Deloitte

Thus, in terms of reporting during the EC funding period, the main reports are the FP6 reports and the data collection.

Moreover, there were other channels of reporting activities through exchange of information either by email, phone or events. The EC and BHOs interviewed, explained that there was a good level of cooperation and informal contacts, as well as formal contacts were pursued whenever necessary. It could be argued that through the years the working relationship has strengthened due to the close contacts they have. In particular, the Quality Working Group 68 has been a forum of discussion for a number of issues, including reporting the activities of the ERA-MORE network.

After the EC funding period, once the FP6 contract has come to an end, the FP6 reporting is replaced by another type of reporting called “soft reporting” 69 (additional informat ion background and activities per country). Therefore, after the EC funding period there is the soft reporting and data collection.

68 The Quality Working Group is composed of, on average, 15 people that meet four times a year for one day to discuss issues previously discussed at a BHO meeting in a more operational manner. 69 Soft Reporting only introduced once the EC funding comes to an end, as the annual reports would be no longer submitted to the EC. Thus, a need for some type of reporting aside from data collection was deemed necessary by EC, BHO and Quality Working Group. Discussions on the format of soft reporting start before the EC funding finished.


Page 88

FP6 Reporting

The ERA-MORE networks were funded as Specific Support Action 70 (SSA) projects under the thematic priority/ac tion: Human resources and Mobility within the Sixth Framework Programme (FP6). Therefore, the reporting followed the FP6 reporting guidelines. Each project had an average duration of three years and the project was divided into reporting periods which had the following duration:

• P1: from month 1 to month 12;

• P2: from month 13 to month 24;

• P3: from month 25 to the last month of the project.

Within their contract, each SSA indicated the number of reports to be covered per project. General guidelines on report ing in FP6 and specifically regarding the SSAs were posted on the Extranet of the then ERA -MORE portal 71.

The BHO was in charge of the reporting, as the BHO was the “coordinator” in each contract.

The EC negotiated with each project coordinator the exact number of reports and there were usually two main reporting periods and a last reporting period (third reporting period) combined in some cases with the Final Report. Moreover, payments were linked to the approval of annual reports.

As indicated on the gu idance notes for Reporting on FP6 for Specific Support Actions 72, we can find the following types of reporting:

Periodic reports: these reports were submitted at the end of each reporting period and include;

• the periodic activity report, which includes a Publishable executive summary and an updated plan for using and disseminating knowledge;

• the periodic management report, the financial statements (Form C) from each contractor and a summary financial report consolidating the costs of the contractors;

• the periodic report on the distribution of the Community’s contribution;

• the interim science and society reporting questionnaire (online report -to be completed once at the end of first reporting period);

• the interim reporting questionnaire on workforce statis tics (online report -to be completed once at the end of first reporting period);

• the interim socio -economic reporting questionnaire (online report -to be completed once at the end of first reporting period);

• supplementary reports, as specified in the annexes to the contract and the project deliverables that refer to those deliverables specified in the Work Plan (which becomes Part B of the contract). These reports were submitted annually.

70 For further information on the FP6 Instrument: SSA, please go to http://cordis.europa.eu/fp6/instr_ssa.htm 71 ERA-MORE was rebranded Euraxess: researchers in motion, which was officially launched on 24 June 2008. 72 Detailed information on the content of each report can be obtained from: Project reporting in FP6 Guidance notes for Integrated Projects, Networks of Excellence, Specific Targeted Research or Innovation Projects, Coordination Actions, Specific Support Actions, Co-operative Research Projects and Collective Research Project, Version 1, October 2004. The document can be downloaded from the Euraxess Extranet.


Page 89

Then at the end of the project, the final report was delivered. The EC aiming at simplifying the process combined, whenever possible, the periodic report for the last reporting period with the final report produced at the end of the project.

In addition to the Final Report, the following reports have to be filled in at the e nd of the project:

• a publishable final activity report;

• final plan for using and disseminating the knowledge;

• final management report;

• final report on the distribution of the Community’s contribution;

• final science and society reporting questionnaire;

• final reporting questionnaire on workforce statistics;

• final socio-economic reporting questionnaire;

• any supplementary final report and other data requested by the Commission.

The BHO would be responsible for reporting to the EC on the FP6 report. The proce ss for review of the FP6 reports is that an external reviewer evaluated the annual reports (and at the end of the project the final report) and if pertinent compared them with previous evaluations and/or annual reports. Then, the EC sent a letter with over all recommendations and the review report to the BHO.

Data collection

Based on the recommendations of the Quality Working Group and approved by the Bled Annual Conference73 a pilot data gathering action was launched. This pilot data gathering targeted BHOs and all ERA-MORE members and covered the period from September to December 2006.

It was agreed that the first pilot data collection would cover three months. Then, as from January 2007, the following periods for the data gathering were of six months.

The data collection will consist of indicators/data gathered from the BHOs and MoCs.

The indicators requested from the BHOs were:

• For the NRMP:

o Number of unique visitors;

o Number of page views.

The indicators/dat a requested from all ERA -MORE members 74 were:

• Number of cases of researchers (also family members and organisations) assisted in the capacity of a Mobility Centre;

• Number and type of question based on the list of tasks in the Declaration of commitment ;

73 The Bled Annual Conference took place in November 2005. 74 An Information Note was sent by the European Commission to the attention of all ERA-MORE Mobility Centres, regarding the data collection on 29/08/2006.


Page 90

• Average speed of answering in days from receptio n of the request to the first answer.

Regarding data collection, during the EC funding period, the BHO was responsible for the NRMP data and the MoCs were responsible for gathering the data related to researchers assisted, number and type of question in wh ich they assisted and average speed in the first answer provided to a request.

Soft Reporting

This initiative was proposed in the Quality Working Group with the purpose of providing a descriptive picture of the national ERA -MORE network following the end o f the FP6 reporting period once contracts were finished. The BHOs agreed to this reporting initiative. The format of the soft reporting would be a couple of A -4 pages describing the situation of the national ERA -MORE network and its initiatives in the resp ective year. Soft reporting initiative started in 2007 and would be an annual activity. Nevertheless, the soft reporting was not continued in 2008 due to a low response rate (only 11 out of 30 BHOs provided the soft reporting for 2007).

Extent to which the BHOs reported the activities of the network

Reporting during 2004 -2007

The response for the FP6 reports produced by the BHOs, based on the EC internal administration follow up documents, was of 100% response rate 75. The same cannot be said about data collection.

The response to the data collection exercise, as from September 2006 to December 2007 is, as follows:

• For the first pilot data collection from September to December 2006, there were 133 Mobility Centres from 30 countries that provided data and the BHOs provided information on national portals for 26 countries 76.

• For the first period of 2007 data collection (from January to June 2007) there were 120 out of 167 registered centres (BHOs and MoCs) that submitted their data 77.

• For the second period of 200 7 data collection (from July to December 2007) there were 125 out of 164 registered centres (BHOs and MoCs) that submitted their data 78.

Concerns have been raised at a number of meetings about the data collection. For example, it was agreed by all particip ants to the 14 th meeting of the BHOs, that this information collected did not provide a complete picture. Then, during the February 2008 Quality Working Group meeting the EC was deeply concerned as only 50% of the MoCs and BHOs had provided their input by 20 February 2008.

For the period January 2008 to June 2008, we see that there has been a slight increase in the total number of centres that reported their activities, as there were 145 organisations that replied out of the 169 that exist.

75 The evaluator is aware that some extensions to contracts have happened, therefore reporting in these cases is not yet concluded. 76 As reported by the Quality Working Group power point presentation dated 12/06/2007. 77 As indicated on the minutes of the14th meeting of the ERA-MORE Bridgehead Organisations, Brussels, 28/11/ 2007. 78 Information provided by the European Commission on 16/10/2008.


Page 91

The interviews in the different visited countries reveal that most of the weight in the FP6 reporting was placed under the responsibility of the BHOs. Most of the BHOs interviewed found the FP6 reporting for ERA -MORE cumbersome. Moreover, MoCs interviewed showed almost no interest in the annual reports submitted to the EC by the BHO. Whereas, MoCs provided the evaluator with feedback about the data collection and seemed more interested on it. Their feedback in terms of the data collection is due to the fact that it is th e responsibility of each MoC to input the figures, as previously described, on the Extranet every six months.

The MoCs interviewed seemed to see positively the data collection. However, some concerns were raised based on the adequacy of the data collected. For example, a number of MoCs in Spain coordinate a regional network as their main activity and this task cannot be reflected in the actual reporting. Therefore, to these MoCs the data collection exercise does not reflect their actual activities.

In the same line, concerns were raised during the Quality Working Group of 24 April 2007, in particular in terms of “the appropriateness of compiling one single overview table given different national situations” 79.

As the FP6 reporting was linked to payments, as described above, there was a 100% response rate. Whereas, data collection response varies as per table below. Table 12 - Data collec tion – BHOs and MoCs average response - in % of total number of BHOs and MoCs

Reporting Period Average response - BHO

Average response - MoCs

1/01/2007 to 30/06/2007 92% 75%

1/07/2007 to 31/12/2007 85% 77%

1/01/2008 to 30/06/2008 93% 91% Source: European Commission, DG RTD

It can be seen in the table above that, in terms of data collection there has been a steady increase in the number of MoC´s responses from 2007 to 2008. 80 Moreover, it is clear that the percentage of BHOs data collection response is higher than that of the MoCs. The evaluator found no reason for such a difference during the intervie ws. Moreover, it has to be noted that the LoCPs did not provide information in 2007 nor in the first half of 2008. For 2004 -2007, this can be explained by the fact that LoCPs had not signed the Declaration of commitment , whereas as from 2008, we find the L oCPs as members of the network in the Declaration of commitment.

Therefore, we could argue that the response rate to calls for data collection has increased since 2007. Interviewees argued that some issues with the Extranet (being a new tool) could have had an impact on the number of respondents in a first stage. It is not clear the extent to which data collection will continue from 2008 onwards, especially as some of the network members are not sure about their future due to some funding concerns raised du ring the interviews. This

79 Quality Working Group Report 24/04/2007. 80 This evaluation covers the period 2004-2007 but as there was already some data available for the first semester of 2008 it is as well presented.


Page 92

being said, the first results for the period January to June 2008 are positive in terms of a high response rate.

It is clear that data collection is an important action that the then ERA -MORE network tried to build upon during the EC funding period. The importance of data collection is even higher following the end of the EC funding period, in order to see the performance of a network that needs to continue showing its added value to stakeholders, national authorities, the European Commission and to the network itself. In addition, soft reporting becomes a crucial tool to report on the state of the national network from the BHO to the European Commission as the annual reporting only covered the EC funding period.

Extent to which the reports fulfil European Commission requirements

In terms of data collection, interviewees raised some concerns regarding the accuracy and homogeneous understanding of what each category means in the different countries. The Quality Working Group has rais ed concerns on the quality of the data collection and recommended that BHOs run a quality check when the data is entered in the Extranet. It is not clear to the evaluator how and if this quality check was made on the information uploaded on the extranet by the MoCs. Moreover, despite the fact that the delivery of training sessions related to issues such as the Extranet, as indicated in the evaluation question in section 581, it is not clear to the evaluator how much time and resources were dedicated at Europ ean level, nor at National level, to the training of the MoCs regarding reporting(data collection).

In terms of soft reporting, the fact that only 11 out of 30 BHOs reported and the difference in terms of information provided (quality and quantity) drove the European Commission to require this type of report only for 2007.

In terms of the number of BHOs/MoCs that are uploading their data on the extranet, there have been numerous discussions. The EC suggested creating incentives in relation to the provisio n of data so that all organisations would input their data. Specifically, the EC suggested “to link access to networking activities with the readiness to provide data” 82. Therefore, according to interviews with EC and a number of MoCs, we can argue that one of the main concerns, not only to the EC but as previously mentioned to the Quality Working Group, is the quality of reporting.

Overall, it can then be said that to a certain extent the reports fulfilled the EC’s requirements for the period 2004 -2007 and whenever gaps were identified either in the FP6 reports or in the data collection follow -up actions were taken. The period following the EC funding will show how the requirements of the EC in terms of reporting are met despite the end of EC funding.

Extent to which the reports fulfill the European Commission’s needs

FP6 Reporting

The FP6 reporting made by the BHOs was used by the European Commission to monitor the progress of the projects. However, according to a number of EC and BHOs interviews, the information provided in these reports did not reflect the MoC´s activities. Therefore, this

81 Evaluation Question: Have the Mobility Centres reached their objectives in terms of providing comprehensive and up-to-date information and personalised assistance to the researchers in matters related to mobility? 82 Discussions regarding data collection took place in many Quality Working Groups. The EC suggestion on incentives was provided in the Quality Working Group that took place on 24/04/2007.


Page 93

reporting did not meet the needs of the European Commission in terms of getting a picture of the network activities per country.

Data collection

In an attempt to meet t he EC‘s needs, the Quality Working Group worked on the definition of performance indicators that could give a picture of the network activities as per Declaration of commitment. It can be argued that with the inclusion of the data collection to the FP6 rep orting made by BHOs, the needs of the EC were better met. Moreover, the Quality Working Group has included a soft reporting as from the end of the EC funding period in order to assure some reporting aside from the statistics.

The data collection was launch ed to get information from ERA -MORE network members, which means that there is no data gathered from those countries that have LoCPs. It is only since the 2008 Declaration of commitment that LoCPs are officially recognised as members of the network. Theref ore, one of the shortcomings of the data collection from 2007 is that no information has been included from those countries that have no MoCs but LoCPs.

It is clear from the MoC interviews that there are a number of gaps in terms of the data provided and usefulness of it. MoCs argued that the information submitted in the Extranet did not accurately reflect their objectives. This perception is, as indicated above, shared by the Quality Working Group Members.

Moreover, gaps were identified by the EC on the r eporting and looked into at the Quality Working Group. These issues raised include the following ones:

• No data reported in a number of countries at BHO level;

• Missing data for National Portals;

• Incoherent information given (either at BHO or MoC level).

It is clear that the EC has been actively involved in the reporting process and how the Quality Working Group has proposed different actions to improve the information collected in line with the needs expressed by the EC.

Other reporting

The EC has adapted t he reporting to the needs of the network. A clear example is the evaluation questionnaire prepared within the Quality Working Group following a request from the BHOs. This evaluation questionnaire falls within the issues of sustainability and the need to have reliable information about the performance of the network. It was agreed by all members of the Quality Working Group that “ only if the added value of ERA -MORE can be proved, requests for funding and support at different levels are likely to succeed ”83. However, a number of MoCs interviewed were not too keen on this initiative. The main reason being that their financial means do not provide enough time to be allocated to more feedback collection but rather to concentrate on their main objectives.

National reporting

Interviews with BHOs and MoCs in the participating countries revealed that in a number of cases there is some further reporting to national authorities mainly due to the fact that a number of national networks received some national funding eit her during the EC funding period and/or

83 Quality Working Group Report, 23/05/2006.


Page 94

in some cases afterwards. The different ways of reporting would go for example, from sending the data collection information/statistics to producing an abridged version of the annual reports or providing specific i nformation in terms of flow of researchers . Some of the BHOs and MoCs in a number of countries thought that their added value could rely in their access to some key information related to mobility of researchers.

Therefore, it can be argued that the EC, i n collaboration with the ERA -MORE network members, is making a continuous effort in order to match the reporting activities with the needs of the EC. Issues such as quality control and accuracy on the information have to be carefully monitored and in addit ion, the objectives of the different members of the network and different national contexts have to be taken into consideration within the reporting exercise.

To conclude, it can be argued that the BHOs and MoCs have to a great extent reported to the EC in a structured way thanks to the existing and new templates and guidelines . The reporting exercise has evolved alongside the network, as previously described. There are still some gaps in terms of accuracy, the number of responses and more importantly quali ty. Therefore, the effectiveness of the members of the network in terms of reporting can and should be improved. The information from the data collection is of great value and usefulness, but provided its quality and accuracy are improved. It can be said t hat to a large extent the BHOs and MoCs have reported their activities and results back to the EC in a structured and effective way. The role of the Quality Working Group has proved to be key in the pursuit of effective reporting, as guidelines and consens us on the information collected and its quantity was discussed. Moreover, effective reporting could assist the network in the area of sustainability in terms of showing its added value to national authorities and other stakeholders. It is, however, not cle ar to the evaluator whether the data collection has proved a useful exercise to a certain degree in the different countries and in the different type of MoCs that exist. That said, it can be suggested, following interviews with BHOs, MoCs that some of the information collected could be key in terms of the network added value within the research community in the area of the mobility of researchers.


• The BHOs rigorously followed the FP6 reporting guidelines in terms of the reports to be provided as per their contract , due to the fact that they are a contractual obligation and payments are li nked to the reports’ acceptance.

• In some cases, the annual reporting according to the FP6 repo rting template did not give a representative view of the national ERA -MORE networks’ activities performed. As the main rationale of the FP6 reporting template was reporting on the implementation of a project, being the establishment of the national networ k, the reporting template did not take into account providing information on the specific activities performed by the MoCs.

• The data collection initiative is a step forward in gathering information that shows the added value of the network . However, st ill in 2008 some issues in terms of number of responses, quality and accuracy have been identified.

• The soft reporting initiative, despite the initial interest amongst BHOs and perceived added value in terms of providing information on the ERA -MORE national n etworks and its


Page 95

activities saw a low response rate and thus, was an initiative carried out only for 2007 period.

• The Quality Working Group has been instrumental in the reporting process in terms of improving the process of data collection.

• The BHOs have reported more frequently the statistical data through the extranet than the MoCs.



• Incentives to BHOs/MoCs could be foreseen in order to ensure more timely and accurate reporting. For example by: linking reports’ submission to the number of seats made available at European Training sessions; providing feedback on the information provided in the reports to BHOs; or by producing a newsletter containing exampl es of good practice from the information provided in the reports, information on reporting etc. Guidance on reporting (soft reporting and data collection) should continue to be given at European level to account for changes in staff .

• A handbook could be developed by DG RTD within the Quality Working Group, providing guidelines for reporting. This handbook could be put on the extranet, making it visible to users by including it on the homepage .

• Quality check on the reported data should be rigorously performe d at EC level .

• Further training on data collection and soft reporting should be given to network members in order to ensure a common understanding and acceptance of the added value of such reporting.

QUALITY WORKING GROUP • The Quality Working Group should continue to oversee the reporting process in terms of

quality check-ups and recommendations, especially following the end of the EC co -funding period. In addition, the specificities of the network should be reflected in the soft reporting.

• The complexity and quantity of reporting should be carefully monitored in order to ensure a high response rate .


Page 96


• At national level, BHOs and MoCs should ensure that t he guidance given at European level on reporting (soft reporting and data collection) is communicated to all ERA -MORE members.

• Quality checks need to be rigorously performed not only at EC level but as well at BHO level.


Page 97

10.0 Evaluation Question: Have the Mobility Centres established a link with National/Regional/Local Author ities and shared information on frequently encountered problems by researchers in the context of National/Regional/Local mobility?

This evaluation question aims at answering the degree to which the ERA -MORE network (BHOs and MoCs) has been able to establis h links with authorities in the respective countries, being at local, regional and national level. The activity of establishing contact with authorities was not an explicit objective mentioned in the Declaration of commitment signed in 2004 by the member organisations and did not appear as a standard work package defined by most of the national networks. In the new Declaration of commitment this objective has however been added. This evaluation question can therefore not be considered as an evaluation ques tion relative to the effectiveness of the network during the period 2004 -2007. Nevertheless this question is part of the evaluation report as it is considered to provide useful information for the next contracting period.

The question was understood as cov ering the following main issues related to:

• the links established between the network members and the authorities in their countries;

• the information that was exchanged between the network members and the authorities;

• the support in establishing links to a uthorities as provided by the EC, BHOs and the mutual exchange of best practices.

In particular, these issues will be dealt with by looking at the following judgment criteria:

• The extent to which the MoCs and BHOs have established links with relevant auth orities in the context of researcher’s mobility;

• The extent to which MoCs and BHOs have exchanged information with these authorities;

• The extent to which the European Commission and the BHOs have provided support to the network to establish links with thes e relevant authorities;

• The extent to which an exchange of best practices and lessons learned on how to establish links with authorities has taken place between the members of the network.

The degree to which such links with authorities have contributed to influencing policy makers will be analysed in the next research question.


Page 98

10.1 Findings Extent to which the MoCs and BHOs have established links with relevant authorities

The activity of establishing links between the ERA -MORE network and the authorities in a country was not described systematically in the annual Activity Reports of all countries and neither in the Implementation Reports. This analysis is therefore based on in -depth interviews performed in the 10 countries 84 selected for this evaluation study. The interviews with BHOs, MoCs and authorities of these countries have provided a coherent and consistent picture regarding this topic. The following trends appeared from the analysis:

The links and interactions of the national networks with authorities in their country vary greatly between the countries. Concerning the type and frequency of the interactions , the following findings emerged:

• In three of the visited countries regular formal meetings took place between representatives of the national ERA -MORE network and authorities of the country. These regular formal meetings may be through the membership of an authority in the board of the network and/or in the form of pre -set meetings with representatives of the network on regular time intervals.

• All countries visited mentioned ad hoc meetings between representatives of the network and authorities. Depending on the case, such ad hoc meetings are taking place upon request of the authorities or upon request of representatives of the national ERA -MORE network. In all countries ad hoc meetings were reported at the end of the funding period to discuss the sustainability of the network. In general the comment should be made that ad hoc meetings may deal with other topics than researcher’s mobility in the strict s ense, as members of the ERA -MORE network often perform additional tasks beside those of the network. Therefore, such ad hoc meetings may also deal with the evaluation of the quality of research and education in the country, the criteria for allocation of r esearch funds, participation in Framework Programme projects, etc. Through these meetings relevant contacts are made which can later serve to discuss topics more closely related to the ERA -MORE services.

• In all countries visited there are informal contacts between key members of the network (being an individual or an organisation) and the authorities. Such interactions take place by e-mail, phone and informal encounters at conferences, events and working meetings. For the same reason mentioned above these c ontacts may deal with other topics than those covered by ERA-MORE in the strict sense.

• In most countries visited there is also the exchange of information/documentation on paper or in digital form between the network and the authorities. Examples are digit al newsletters, annual reporting from the network, a copy of the Researcher’s Guide to the country, etc. In one country the network has had a study produced on the main obstacles for mobility of researchers in the country which was subsequently presented a nd sent to the authorities. The type of information exchanged will be further described below.

Overall, the extent to which the network has established links with authorities differs between BHOs and MoCs of the same national network.

84 See the description of methodology for this evaluation study.


Page 99

• Generally speaki ng all BHOs have established links with authorities in their country, all be it to a varying extent and for differing purposes. This type of activity is typically performed to a lesser degree by the MoCs than by the BHOs. However there are exceptions on this, as will be described below in the section discussing the main findings concerning the MoCs.

• If countries have different layers of authorities (e.g. at regional level), the links with authorities on the national level are most often taken care of by the BHOs, whereas MoCs (may) have established contact with authorities at a regional or local level. For example in one country, some MoCs have established close links with the regional authorities, wh ile the BHO has contact with the authorities on the nat ional level.

Concerning the BHOs , fieldwork has shown that they have established different degrees of interaction with authorities in their countries. The key differentiators that determine the degree of interaction between the BHO and authorities in the ir country85 are as follows:

• National authority involvement in the network: In all countries visited the people interviewed have indicated that the authorities have been involved in the initial set -up of the network. After the initial set -up phase authorit ies of some countries have closely followed up the activities of the network (e.g. through a representative in the board of the network, through meetings discussing activities performed, etc.), whereas in other countries the authorities chose not to closel y follow up on the network’s activities. Those countries in which the authorities have provided financial contributions to the network during the period 2004 -2007 have seen more regular formal meetings with the authorities than other countries. All countries indicated that meetings with authorities have taken place to discuss the future functioning of the network at the end of the funding period.

• Role of the hosting institution: When asked about contacts with the authorities, several interviewees referre d to the general contacts between the hosting institution of the BHO and the authorities. The fact that several BHOs are hosted in a public administration makes that interaction between the BHO and other authorities active in the domain of research occur o n a regular basis. In other countries BHOs are hosted in non -governmental or semi -governmental institutions with close links with the authorities. If this is indeed the case, the BHO is likely to have contact with authorities in that country on a regular basis as well.

• Other activities/tasks performed by people working at the BHO: Often people working for the BHO have more tasks and responsibilities than only those related to ERA -MORE. If this is indeed the case, these tasks are often related to other top ics concerning research and innovation in the country. While performing these tasks, BHO members enter into contact with authorities to discuss these matters.

• Personal relationships: During the interviews it appeared that another key factor influencing the degree of interaction with national authorities is the extent of the personal network of the individual interviewed. The extent of the personal network depends on the individual’s job history, networking skills and character fit with counterparts at t he authority.

As described above, MoCs tend to have less interaction with authorities than the BHOs do. There are however exceptions to this finding. In some countries the MoCs have more contact with authorities than in other countries. And even with som e national networks there are differences between the MoCs. The reasons for this differentiated pattern are as follows:

85 The evaluator is conscious that there is a need to be attentive to relating the degree of interaction between authorities automatically and influence on these authorities, as will be described further below.


Page 100

• Degree to which the authorities are involved in activities performed by the national network: As it is also the case for BHOs, the leve l of involvement of authorities in activities of the MoC has an influence on the frequency of interactions. For example, some regional authorities in a country visited provide financial support to the MoC and have regular contact with them. In other regi ons in the country this is not the case. But also the presence of authorities on conferences, training sessions, etc. organi sed by the network or a particular MoC differs greatly and has an impact on the degree of interaction between the authorities and the MoCs.

• Target groups reached by MoC: As the target groups reached by MoCs differ between the different national networks, their interaction with authorities logically differ as well. Some countries have MoCs that support researchers country -wide, where as in other countries MoCs deal with researchers in a particular region, and in yet another group of countries MoCs are merely serving mobile researchers within their own hosting institution. Depending on the target groups a MoC is reaching, links with au thorities may or may not exist. MoCs only serving mobile researchers within their own hosting institution do not tend to have frequent interactions with authorities.

• Part of task distribution between MoCs in a country or not: Besides the general differenc es between countries on what tasks should be performed by MoCs, there are also differences between MoCs within the same national network. In some countries certain MoCs have taken additional tasks besides from providing information to researchers. In one of the countries visited, for example, a specific MoC takes care of most relations with national authorities. This is due to the high concentration of foreign researchers in the region covered by the MoC, but also to its geographical proximity to these au thorities.

• Organisation and set -up of the network: Some national networks have set up a country -wide helpdesk86 for the network to contact if there are questions on e.g. regulations. In other countries such a helpdesk does not exist. MoCs will therefore contact people at the authorities directly with such questions.

• MoC members’ individual participation in other initiatives: Some individuals in the MoCs are more involved in other initiatives in their region or country which allows them to create an informal network with authorities. This personal network may subsequently serve to exchange information or asking questions on e.g. new legislative initiatives.

Extent to which the MoCs and BHOs have exchanged information with the authorities

Not all interac tions between the network and the authorities automatically lead to the exchange of information. It is therefore analysed to what extent the MoCs and BHOs have used their contacts with authorities to exchange information with these authorities on frequent ly encountered problems regarding researcher’s mobility 87. Again this extent differs considerably between the countries. The following findings occurred from the fieldwork:

86 The Helpdesk, in one of the national ERA-MORE networks, is installed in one of the BHOs which has a few employees dedicated to answer the questions received from the network. A “frequently asked questions” document is updated regularly and circulated amongst the national network. 87 In this research question only the extent of information exchange will be analysed. The extent to which this information exchange has lead to influencing policy makers within the country will be analysed in the next research question.


Page 101

• A number of interactions with authorities described by the national networks do not really lead to the exchange of information on frequently encountered problems concerning researcher’s mobility in the country. Often authorities are invited as guest speakers on events aimed at informing third parties, such as students, researchers, univ ersity staff about the activities of the network or about more generic research -oriented topics such as for example, the launch of new Framework Programmes.

• Interviewees in all countries but one have described varying levels of top -down information exchange from authorities to the national ERA -MORE network. Depending on the topic or country, such top -down88 information exchange may be initiated by the authorities themselves or by the network members. Such information exchange mainly deals with communicating changes in policy or in administrative practices relevant to the wider research domain and/or researchers’ mobility in particular. Depending on the structure of the network and the nature of the information, such information may be provided by the authorities to the BHO which subsequently spreads the information to the MoCs or to the MoCs directly (e.g. by a training course on a new legislation given by the authorities to the MoCs). Several authorities have stated that they feel that communicating mat ters via the national ERA-MORE network significantly facilitates the communication process to the member organisations.

• Besides this, several countries also have a certain level of bottom -up information exchange between the national ERA -MORE network and au thorities. Such bottom -up information exchange process takes different shapes and deals with different types of information:

o Firstly, there is the exchange of information concerning the activities performed by the network. Such information is shared in the shape of activity reports 89, statistics (number of foreign researchers served, number and type of questions received), the exchange of documentation produced for users of the network, etc. This type of information is in some cases reported on a regula r basis, whereas in other cases this is provided on an ad hoc basis. In one of the national ERA -MORE networks such statistical information was, for example, provided when new legislation was being prepared. It should also be noted that such information is not always reported at the national level. In another national ERA-MORE network , certain MoCs report their activities to regional authorities as well.

o Secondly, there is the exchange of information regarding problems that occur with national legislation an d/or national administrative rules applicable to the 14 domains in which the ERA -MORE network provides information. This may also take the form of reports or surveys on most frequently encountered obstacles for mobility in the country. A more in -depth ana lysis of such information exchange is described in the following research question.

88 A “top-down” information flow in this context is understood as the authorities passing on information to the national ERA-MORE networks. A “bottom-up” information flow is a flow in which the national ERA-MORE networks are sharing information with the authorities. The occurrence of a top-down versus bottom-up information flow can take place regardless of which party requests the information flow. 89 Such activity reports most often differ from the reporting of the national network to the European Commission. Only one country uses this reporting to inform the national authorities. Another country provides a significantly reduced version of this reporting. Other forms of activity reporting may deal with the actions of the national ERA-MORE network only, whereas in other cases this is incorporated into the annual activity report of the BHO’s hosting institution.


Page 102

Other types of information exchange can be categorised as being neither top -down nor bottom-up but of interactive nature. Examples of such information exchanges are:

o Authorities and the network sometimes mutually work on certain matters, such as the dissemination of information on certain legislations. There have been examples of the authorities voluntarily proofreading information that will be posted on the NRMP or that will be published in leaflets and/or foreign researchers’ guides of the country. Most often this is legal information on entry conditions, taxation, etc. In some cases the authorities have provided themselves ready -to-use texts that were subsequently p ublished on the website or in publications. Alternatively in some countries the MoCs/BHOs sometimes also voluntarily read discussion documents and draft legislation bearing in mind their implications on mobile researchers. This will be further analysed i n the following research question.

o There were also meetings regarding specific topics such as the sustainability of the network after the end of the EC funding period. As discussed previously in this report, such discussions have not yet resulted in concr ete solutions for three of the countries visited.

Extent to which the European Commission and BHOs have provided support to the network to establish links with authorities

Concerning the support from the European Commission to the network, the following findings appeared from the analysis:

• The European Commission has focused its support in this matter mainly on the BHOs. This support was for example made concrete in a workshop for BHOs on “Cooperation models with policy makers” that was organi sed in 2006. The workshop was composed of presentations from several BHO representatives explaining their interactions with policy makers in the country, as well as a discussion with policy representatives on ideas and practical recommendations to enhance cooperatio n with authorities.

• The general feedback from the ERA -MORE network members on the EC support is as follows:

o BHOs have varying opinions on whether or not the European Commission should have taken particular actions to support them at all in building up re lationships with authorities in their country. Roughly half of the BHOs encountered in the countries visited hold the opinion that establishing such links is country -specific and that there is no real added value for the Commission to intervene in this pr ocess. The other half of the BHOs does however think that the Commission could contribute in facilitating the dialogue with national authorities.

o Regardless of whether the Commission should support the BHOs in creating contact with authorities, the gener al perception is that the EC has supported the BHOs appropriately in this matter during the period 2004 -2007. Several BHOs mentioned that the training sessions organized on new European initiatives (e.g. the European Directive on the Scientific Visa) have contributed to improving relationships with authorities. Having followed this training, several BHO members stated that national authorities have contacted them to provide explanation on the Directive. This has in some cases increased the visibility of t he network to the authorities. Some BHOs however reported that they would have appreciated a more active support from the Commission in this matter for example, by highlighting the importance and added


Page 103

value of the ERA -MORE network to their authorities. T hese BHOs feel that such support from DG RTD would have helped in establishing links with their authorities. Most of these BHOs also mentioned the fact that the change in name of the network into ‘Euraxess’ has not facilitated their work of creating aware ness of the existence of the network to the authorities as these authorities had to be explained again what this network is and what it does.

o Fieldwork clearly indicates that BHOs generally agree on the need to keep providing training sessions on European initiatives by DG RTD, as well as to keep developing initiatives at the European level to remove barriers for researcher’s mobility (e.g. barriers related to pension schemes). One interviewee described this message as “keeping the topic on the agenda”. Al so the suggestion was made in several countries that regional cooperation (e.g. support mobility in border regions) could use some specific attention from the Commission in the future.

o When asking the authorities on their perception of the support from the European Commission is perceived, authorities in general do not think that there is a need for specific actions from DG RTD to support establishing such links. Authorities refer to the differences in national contexts and the varying willingness and need s of authorities to build up such close links. Several interviewees pointed out that excellent relationship exists with the network. As one interviewee pointed out “in this country close relations have been developed with the network over a number of years. Support from the EC in this matter would be equivalent to teaching us how to talk!” Other authorities stressed the fact that it is up to the authorities themselves to decide with which actors they prefer to collaborate closely and with which ones they prefer not to do so.

Concerning the support of the BHOs to create links between the MoCs and authorities, the following findings resulted from the fieldwork:

• In most countries the BHO is the contact link between the MoCs and the authorities. If key topics are to be addressed with authorities, the BHO will most likely take up this issue – be it sometimes with the involvement of a MoC. Several MoCs questioned the added value of themselves establishing direct contact with authorities.

• BHOs mainly facilit ate contacts between MoCs and authorities by organi sing events, training sessions, etc on which guest speakers and/or representatives of authorities are present. Depending on the organisation of the national network, subsequently direct contacts may occur between the MoCs and the authorities.

• No significant recommendations were made by the MoCs to the BHOs to enhance their links with authorities.

The extent to which an exchange of best practices and lessons learned on how to establish links with authorit ies has taken place between the members of the network

The extent to which best practices were exchanged on how to establish links with national authorities between different national networks has been rather limited. However, some examples could be ident ified:

• The most apparent exchange of such experience occurred on a workshop organised by DG RTD in Athens in September 2006, which has been described above.


Page 104

• Besides this, there have been some bilateral exchanges of best practices between the BHOs of some countries. For example, within the framework of the Greek -French Science and Technology cooperation, a workshop was organised in Athens in June 2006 covering amongst others the topics of networking, policy issues and viability of the networks. The workshop was attended by members of the Greek and French ERA -MORE networks, several Greek ministries as well as the French embassy in Athens.


• There is a great variety in the degree of interaction between the ERA -MORE national networks and the authorities. In some countries regular meetings take place between the network and authorities, whereas in most countries such interactions are on an ad hoc basis and more informal in nature.

• Generally speaking all BHOs have established links with authorities, but with a varying extent and for differing purposes. The key differentiators that determine the degree of interaction between the BHOs and the authorities in their country are the invol vement of the authorities in the development of the network, the role of the hosting institution, but also the activities performed by those working at the BHO and their personal network.

• MoCs show a lesser degree of interaction with authorities than the B HOs. There are however differences between the national networks and between MoCs within a single national network as well. The reasons for this relate to differences in local or regional authorities’ degree of involvement with the MoC, differences in the target groups reached by MoCs from one county to the next, differences in task allocations to some MoCs within a country, the organisation and set -up of the network, but also the participation of MoC members in wider initiatives.

• Whereas some interactions with authorities lead to the exchange of information on frequently encountered problems concerning researchers’ mobility , other interactions do not. Quite a number of these interactions are limited to having the authorities as guest speakers at research -related events, etc. Besides this, in all countries visited there is a top -down information flow from the authorities to the network during the regular or ad hoc meetings with the network members (being most often the BHO). Such information exchange main ly deals with the communication of changes in policy or in administrative practices relevant for the research domain. But in several countries the network also provides concrete information about obstacles for mobility to the authorities, questions received, statistics, results of surveys, etc (bottom -up). Finally, there have been some meetings and exchanges of an interactive nature concerning the sustainability of the network.

• Depending on their level of interaction with the authorities, the network membe rs have different expectations regarding what support could be provided by DG RTD to enhance links with authorities. Those networks with good working relationship with the authorities do not see a need for DG RTD to assist them in such a country -specific process. Those networks that are still struggling with establishing good working relationships with their authorities would appreciate more active support from DG RTD by e.g. highlighting the added value and importance of the ERA -MORE network to their aut horities.


Page 105

• The support provided by BHOs to create links between the MoCs and authorities is mainly related to the organisation of events, training sessions, etc. As in most countries the BHO is the liaison between the MoCs and the authorities, most MoCs do not have recommendations for their BHOs on how to enhance their links with the authorities.

• The exchange of best practice and lessons learned on how to establish links with authorities between national networks has mainly occurred through the 2006 workshop organised in Athens by DG RTD on this topic. However, a few bilateral exchanges were also identified.


EUROPEAN COMMISSION -DG RTD

• Specific support could be given by DG RTD to t hose national networks that would like to receive additional assistance in building up a working relationship with the authorities in their country. It is important that such assistance takes into account the capacities and organisation of the national ne twork, as well as the culture and expectations of national authorities.


• National networks that have not established a two -way information flow with authorities should actively try to identify those topics on which the au thorities could be interested in cooperating with the network. The regular encounters with authorities (during conferences, etc.) should be more pro -actively used to start a more active cooperation with the authorities.

• The existing information provision from the BHOs to the authorities could be taken to the next level if this information were placed within a wider European context. Information about the situation in the country (e.g. volume of foreign researchers, number of questions, duration for appli cation processes, etc) could be presented to the authorities complemented with information on the situation in other European countries. This way the European span of the network could be used to a full extent. Providing such information would further increase the added value of the national ERA -MORE networks as information providers to their authorities.

• When defining the key competencies of BHOs, the BHO should pay attention to certain soft skills such as networking capacity. Also when defining training needs, these types of skills should be taken into account.

• BHOs could seek to actively involve those MoCs interested in building up relationships with authorities. This may be done by the exchange of best practices between the BHO and MoCs on how to creat e links with authorities. This may particularly apply to those countries with a regional state structure in which the MoC may play a role in liaising with regional authorities.


Page 106

11.0 Evaluation Question: Have the Mobility Centres, through their Bridgehead Organisations provided expertise and assistance in mobility policy issues to the National/Regional/Local Authorities as well to the European Commission, in close cooperation with the Steering Group for the Human Resources and Mobility of Researchers?

The previous evaluation question dealt with the existence of interactions between the national networks and their authorities and what information was exchanged during those interactions. This evaluation question aims at analysing to what degree such interactions and information exchanges have lead to participation or contributions of the network in policy discussions. As it was the case with the previous evaluation question, this activity was not an explicit objective mentioned in the Declaration of commitment signed by the member organisations in 2004. This evaluation question can therefore not be considered as an evaluation question relative to the effectiveness of the network during the period 2004 -2007. Nevertheless this question is part of the evaluation report as it is considered to provide useful information for the next contracting period.

The question was understood as covering the following main issues related to:

• the assistance of the network members to the authorities concerning mobility policy issues, both at national and at European level ;

• the role of the Steering Group Human Resources and Mobility (SG HRM) in liaising between the network and the EC

More specifically this evaluation question deals with the following judgment criteria :

• The extent to wh ich MoCs (through BHOs) have provided assistance in mobility policy issues to the National/Regional/Local Authorities in their country.

• The extent to which Mo C (through BHOs) have provided assistance in mobility policy issues to the EC.

• The extent to which the Steering Group Human Resources and Mobility (SG HRM) plays a role in liaising between the Mo Cs/BHOs on the one hand and national authorities and the EC on the other hand.

11.1 Findings The extent to which MoCs (through BHOs) have provided assistance in mo bility policy issues to the National/Regional/Local Authorities in their country.

The extent to which the national networks have provided assistance in mobility policy issues to the authorities in their country has been investigated by interviews with the network, but also by interviews with the authorities themselves. The following trends came out of these interviews:

• In all of the countries visited during fieldwork, the ERA -MORE network is not seen as having the formal task of assisting the national polic y-making process regarding mobility issues.


Page 107

• Neither all BHOs nor the national authorities are particularly keen on a more formal role of the network in national policy development. Several BHOs pointed out the lack of resources to play such a role. Besid es this, some BHOs also stated that they prefer the network to remain neutral and not to engage in policy debates. But also the authorities in some countries are not keen on seeing this as a task for the network. In several countries the authorities clea rly stated that the network is free to share information with the authorities, but that policy-making is the exclusive competence of the authorities and that it is up to the authorities to decide who they want to involve in this process. The ERA -MORE network may be consulted in this process, but it is not automatically considered as the sole or main stakeholder in the research domain.

• Despite the general findings described above, several cases were identified in which the national ERA-MORE networks have been able to have an impact on mobility issues in their country-at least to some degree. A distinction is to be made between impact at policy level as compared to impact at the level of administrative practices.

o On the policy level, some BHOs and authoritie s mentioned the exchange of draft legislation and policy discussion papers with a (potential) impact on researcher’s mobility in their country. In some countries BHOs are involved in the stakeholders’ consultation phase of policy development. Also the re gular reporting of concrete problems that mobile researchers face in the country has, according to some BHOs, contributed to the launch of new policy initiatives. An example mentioned was the reduced social security contributions for researchers i n one of the countr ies visited. In several countries the ERA -MORE network was also closely involved in the transposition of the so-called European Directive on the Scientific Visa 90 into national legislation.

o Besides impact on the policy level, the national network s have in some cases also had influence on the administrative level. Some authorities have sent draft application forms (e.g. of immigration services) to the national ERA -MORE network for feedback. Also the feasibility of which type of administrative doc uments can be requested for application dossiers has been discussed with authorities and has subsequently been modified in some countries.

Whether or not the national ERA -MORE networks have contributed during the process of policy-making or changing admin istrative procedures, highly depends on several factors, such as:

• The level of importance of researchers’ mobility for the country’s policy makers. In some countries national authorities have developed their own initiatives to facilitate the mobility of researchers. If the political willingness exists to work on this issue, national networks find it easier to be heard. Several BHOs mentioned that European initiatives such as the European Directive on the Scientific Visa have put the topic on the agenda for the first time in their country. Most networks mentioning how difficult it is to interact with policy makers have mentioned this factor as most important.

90 Council Directive 2005/71/EC of 12 October 2005 on a specific procedure for admitting third-country nationals for the purposes of scientific research


Page 108

• The volume of services provided by the network and the accuracy of information provided. Several authorities mentioned that they are only interested in engaging with networks that can give a concrete and concise view of existing shortcomings in legislation and administrative practices. National networks that serve relatively large volumes of foreign researchers and who have concrete and detailed information on the implications of certain rules and legislation are listened to more attentively than others. As one authority put it: “ we are looking for concrete cases, as well as solid facts and figures. Politicians cannot be convinced with vague terms. They want concrete examples from the field ”.

• The extent to which involving stakeholders in policy-making is part of the national culture. In some countries policy makers have a tradition of involving key stakeholders when developing policy, whereas in other countri es this is much less the case. It speaks for itself that in the latter case it is much more difficult for the network to influence the policy-making process.

• General involvement of the hosting in stitution of the BHO in policy development. As described above, the nature of the BHO’s hosting institution has an impact on the interactions of the BHO with national authorities and policy makers in general. If the BHO is for example hosted in another pu blic authority or in an organisation that is active in discussing policy actions with the authorities, it is more likely to contribute significantly in the policy-making process than if this is not the case.

The extent to which Mo C (through BHOs) have pro vided assistance in mobility policy issues to the EC.

Concerning the extent to which the national networks have provided assistance in mobility policy issues at the European level , the following can be concluded:

• The two key Communications (COM (2001) 331 final of 20.06.2001 and COM (2003) 436 final of 18.07.2003) and the 3% Action Plan ( COM (2003) 226 final of 30.04.2003 ) aimed at enhancing a mobile career for researchers in the ERA were adopted before the launch of the network.

• Concerning the development of the three European instruments (the Directive and two Recommendations 91) that constitute the ‘Scientific Visa Package’, most national networks were still in the process of setting up the network and declared not to have had significant contributions to t hese processes 92. Also the impact on the Green Paper on the ERA, the European Charter for Researchers and the Code of Conduct for Researchers’ Recruitment is estimated to be low to not existent. Most BHO members have not attempted to influence policy makers on these matters.

• Once approved, the European Directive on the Scientific Visa has had to be transposed into national legislation 93. It appears from the interviews that in this process several national ERA-MORE networks have played a role with a certai n impact . In some countries BHOs have contributed to this process in one or several of the following ways:

o Ensuring that the transposition of the European Directive into national legislation is put (and kept) on the national agenda. In several countries the national ERA -MORE

91 Directive on scientific visa and recommendations on long-term admission and short-term visa. 92 These instruments were adopted in September and October 2005. 93 The evaluators are aware that the support to the transposition of a European Directive can be considered as support to national policy-making and European policy-making at the same time.


Page 109

networks have repeatedly stressed the foreseen timeline for implementation of the Directive into national legislation. Some authorities have stated that it was thanks to the BHO’s continuous awareness raising on the Directive that t he country has been able to implement it into national legislation in time. This is mostly, but not exclusively, the case for countries in which no national policies were yet developed to facilitate researcher’s mobility.

o Counseling role regarding the Dir ective. In several countries BHO members have been solicited to provide further explanation to national policy makers on the contents of the Directive. As training had been provided by DG RTD on the new Directive, BHOs were able to provide such further i nsight. Given the familiarity of certain network members with national legislation, also the transposition of the Directive into national legislation has in several cases been counseled by the BHO.

o Wider information dissemination to research organisations. As the ERA -MORE network is composed of research organisations in the country, it has used its network meetings and information channels to inform these research organisations of the existence and contents of the European Directive. Some authorities ha ve stated that they feel such information provision has significantly reduced (potential) resistance of the research organisations against the changes in the national legislation.

o Implementation and follow -up. In a few countries the network has actively contributed to the implementation of the new legislation, as well as to the (planned) ex-post evaluation of the implementation. One of the BHOs interviewed has for example provided feedback on the new application forms before their launch, it has trained the network organisations on the modalities of the new legislation, and will be involved in the evaluation of the implementation scheduled for the end of 2008.

The extent to which the Steering Group Human Resources and Mobility (SG HRM) plays a role in liaising between the M oCs/BHOs on the one hand and national authorities and the EC on the other hand.

The Steering Group Human Resources and Mobility (SG HRM) is an element of the Open Method of Coordination (OMC) encouraged by the European Council of Lisbon for developing human resources in research. The SG HRM follows up the measures set out in the 2001 and 2003 Communications 94 and in the 3% Action Plan 95. Concerning the extent to which the SG HRM has played a role in liaising between the network and the Eu ropean Commission, it appears that:

• The depth of interaction depends on the country. In one of the countr ies visited, for example, the BHO used to be the country’s representative, allowing a direct liaison function between the national network and the Euro pean level. In another country, the country’s representative in the SG HRM was also member of the national ERA -MORE network’s board for a certain period of time. In another country, t he representative is the head of the BHO’s hosting institution, which all ows regular interaction with the national ERA -MORE network as well. In most countries however the representative of the SG HRM does not have a direct involvement in the national ERA -MORE network but does consult several

94 COM (2001) 331 final of 20.06.2001 and COM (2003) 436 final of 18.07.2003 95 COM (2003) 226 final of 30.04.2003


Page 110

actors within the country when prepa ring discussions on certain topics. Depending on the topic, the ERA-MORE network may be involved in these consultations. Whether or not this involvement is systematic or on an ad hoc basis depends from country to country and even from the individual repre sentative.

• As the SG HRM discusses topics on a policy level, the exchange of information between the national ERA -MORE network and the SG HRM representative is mostly an exchange of views on initiatives such as the European Charter for Researchers and th e Code of Conduct for Researchers’ Recruitment, and much less on operational topics. Generally speaking, the national ERA -MORE networks see the SG HRM as a high -level political discussion forum which develops useful initiatives concerning researchers’ mob ility. Most of the BHOs interviewed do however not seek to actively put topics on the SG HRM agenda but limit themselves to discussing matters on an informal basis. Regardless of this, the SG HRM has discussed at several occasions the functioning of the ERA-MORE network and the topic of sustainability of the network after the end of the funding period. The discussions at the SG HRM showed a clear recognition of the added value of the ERA -MORE network and the willingness to continue the network in all coun tries.


• In most countries visited, the national ERA-MORE networks do not feel as having the formal task of contributing to the national policy -making process concerning mobilit y issues. In those countries where indeed such assistance exists, it mostly takes the shape of informal contact and/or the ad hoc provision of information relevant to the new policy development process.

• Most of the BHOs as well as most of t he national authorities are not very keen on the national ERA -MORE networks having a formal task of assisting the national authorities in policy-making. BHOs are wary of the need for additional resources to perform this task, as well as the (perceived) risk of losing n eutrality when getting involved in policy debates.

• The national ERA -MORE networks were involved in initiatives at both the policy level and the level of administrative proce dures, mostly through the participation in the stakeholder consultation process, t he exchange of draft policy discussion papers and draft legislation, but also on the more practical level through the exchange of draft application forms, the determination of needed documents for certain application dossiers, etc.

• The extent to which such participation has been of a significant nature at the national level depends on several factors, such as: the level of importance of researchers’ mobility for the country’s policy makers; the volume of services provided by the network and the accuracy of information that could be provided; the extent to the authorities involve stakeholders in policy-making; and the general involvement of the hosting institution in such processes.

• The extent to which the national ERA -MORE networks contributed to policy -making at the European level is generally perceived as not being of a significant nature. However, both network members and national authorities provided evidence of significant contributions by certain national networks to the transposition of the European D irective on Scientific Visa into national legislation. The training received by DG RTD on this topic is considered to have facilitated this role to a significant extent.


Page 111

• The depth of interaction between the SG HRM and the national ERA -MORE networks depends on the country. In a few of the countries visited, the SG HRM representative is closely involved in the functioning of the network, allowing for a direct link between both. In most cases the interaction with the national networks takes place through con sultations of the SG HRM national representative with stakeholders in his/her country. The frequency of these consultations varies depending on the country’s national context and moreover, on the national representative to the SG HRM.



• The extent to which the national ERA-MORE networks should be involved in national policy-making and change s in administrative practices concerning mobility should be discussed in the SG HRM. Whereas the new Declaration of commitment does foresee the national ERA -MORE networks to assist their national and local authorities in mobility policy issues, there is a need for openness and flexibility on whether or not such in volvement is feasible in the respective countries. The national ERA-MORE networks should be known as expert s in the field by the policy -makers and as such consulted regarding mobility issues.

• If a decision is made to try to further involve the network in t he policy-making process, the BHOs should allocate time and resources to their participation in policy development and in administrative simplification in their work plan.

• BHOs should actively document concrete cases of obstacles to mobility, as well as, whenever possible, produce reliable statistical data. It is those networks that are able to provide such information that have had most impact on policy development and the simplification of administrative procedures . As also recommended above, it could be envisaged to further enrich such information by presenting it within a pan-European context, allowing authorities to situate the ir country in comparison with other European countries .

• It is important that the BHOs will continue to be actively informed and trained on all new European initiatives in the field of mobility. This way the BHOs will be able to consult authorities on these matters if required.

• The link between the national ERA -MORE networks and the SG HRM could be further strengthened by plannin g regular meetings between the SG HRM representative and the network representatives to provide feedback on each other’s tasks and activities.