Planning and running a regional evaluation system:

the Madrid experience

Beatriz PresmanesCoordinadora de Programas

The Madrid R&D System

R&D expenditure 1,7% of GDP50.000 R&D workers 2% of working

population15 universitiesMore than 50 research centersMore than 20 interface

organizations32% of public research centres31% of international publications

Regional Framework for R&D&I

Implementing a territorial Intelligence SystemGaining profit for regional intellectual capital valueCreating and capturing knowledgeSharing and linking interest

REGIONAL PLAN FOR SCIENTIFIC RESEARCH AND TECHNOLOGICAL INNOVATION

Creatingknowledge

Economical value

building linkseconomical support

EVALUATION SCHEME

Three levels for evaluation:REGION

Regional Innovation System Mapping science and technology capabilitiesMeasuring science and technology production

PROGRAMMERegional R&D Programme PRICITForesight activity linkTechnology Watch link

INDIVIDUAL GRANTProject evaluation

A regional dynamic vision of our capabilities constraints and trends

Gathering information trough the madrimasd network in collaboration with academic partners

Long term project funded by CM under agreement with relevant research groups

Three projects:Regional Innovation System

Mapping science and technology capabilities

Measuring science and technology production

www.madrimasd.org/indicadores/

Evaluation of the Regional Innovation System –First step

analysing science, technology and innovation indicators

www.madrimasd.org/informacion/documentos

Prof. M. Buesa et al UCM

Evaluation of the Regional Innovation System-second step

to determine regional innovation capacity and its temporal evolution31 adjusted variables analysed, 5 year period, four group of variables:

regional and productive environmentUniversityPublic administrationenterprises

Developping a regional

innovation index (IAIF)

Prof. M. Buesa et al UCM

Evaluation of the Regional Innovation System: some results

Innovative regional capacity according regional innovation index IAIF

"Intellectual Capital Project"

Main targets:

To develop a versatile tool adaptable to the needs and characteristics of R&D institutions

elaborate dynamic indicators concerning intellectual capital

create an operative tool

useful through the madri+d platform

Prof. E. Bueno and P. Morcillo UAM

Intellectual Capital Project: Intelect model

An integrated an evolutive model

HUMAN CAPITAL STRUCTURAL CAPITAL RELATIONAL CAPITAL

Present Future

Prof. E. Bueno and P. Morcillo UAM

Intellectual Capital Project: the architecture of the map

NODES

KEY WORDS

RESEARCH & TECHNOLOGICALPRODUCTION

RESEARCH GROUPSTOPICS & PERSONNEL

IDENTIFICATION

KNOWLEDGE CREATION

Prof. E. Bueno and P. Morcillo UAM

Scientific and Technological Production of the Community of Madrid

Since 1994 updated every yearBibliometric indicators International and National DatabasesAll disciplinesScientific publications and patents Institutional and research groups performanceSectoral analysis by science domainMeasuring science technology production trendsRegional comparative analysis-relative national and

international position

Prof. I. Gómez-Caridad et al CINDOC

PROGRAMME EVALUATION

Sectoral studies of different PRICIT actions

Comparative analysis against National R&D Plan

Passed performance: regional, national and international

Whole analysis under Foresight action: Three kind of discussion groups

Wider participation through internet

Selection criteria PRICIT evaluationPRICIT evaluation

TECHNOLOGICAL TRENDS ANALYSIS

TECHNOLOGICAL TRENDS ANALYSIS

Key Technologies- Science-Technology basisKey Technologies- Science-Technology basis

WARNINGS INDICATORS RECOMENDED ACTIONSWARNINGS INDICATORS RECOMENDED ACTIONS

Attractiveness and relative position

S/T CapabilitiesS/T Capabilities

Industrial & social demand

Industrial & social demand

Regional Programme for Research –IV PRICIT (2004-2007)

FORESIGHT LINK

R&D Regional Programme - PRICIT

Technology watch link :Innovation Circles

The aim is to set up and develop technological watching and economic intelligence routines and to promote co-operation between companies and research groups in order to facilitate technology transfer.

Circles already existing are: Agrofood, Biotechnology, Energy, Materials and Production Technologies, Environment, Microsystems and Nanotechnologies and Information Technologies

GRANT EVALUATION PROCESS “present”

R&D PROPOSAL

R&D PROPOSAL

EVALUATION CRITERIA AND S&T PRIORITIES

EVALUATION CRITERIA AND S&T PRIORITIES

SCOPE OF THE CALL-

THEMATIC CLASSIFICATION

THEMATIC CLASSIFICATION

EV

ALU

ATIO

N S

CIE

NTIF

IC C

O-O

RD

INA

TO

RS

EXTERNAL EVALUATION ANEP

RELEVANCEPOTENCIAL IMPACT

SCIENTIFIC QUALITY

INTERNAL EVALUATIÓN DGR -CM

FINALEVALUATION REPORT

GENERAL DIRECTORATE FOR RESEARCH

FINAL SELECTION

GENERAL DIRECTORATE FOR RESEARCH

EVALUATION PROCESS “future needs and problems to solve”

The attenuation effect of ex -ante evaluation on planning

Combining excellence and innovation success

Ad hoc criteria linked to strategic goals

Gaining transparency maintainig confidentiality

More diverse and open vision in the evaluation process

How to cope with complementarity national, european one

Funding science supply instead social demand

Balance scientific production and technological transfer activities

Need to continuous integrated monitoring against deliverables

Formative evaluation, visible evaluation committees

Integrated vision, interactive process

Need to measure impacts of regional policy