Upload
colette-butler
View
20
Download
1
Embed Size (px)
DESCRIPTION
SAI’s role in development and use of key indicators for R&D evaluation. Timo Oksanen 3.4.2012. About SAI’s role in indicator development. Depending on the national mandates, the SAI’s role can be active or passive – or something in between - PowerPoint PPT Presentation
Citation preview
2
About SAI’s role in indicator development
Depending on the national mandates, the SAI’s role can be active or passive – or something in between
However, an active role in indicator development can endanger SAI’s independency and objectiveness
The NAO of Finland has not participated in Finland’s KNI development
Therefore, we have kept an outsider’s view to Finnish KNI-system
Findicator: The place to find up-to-date information on:
demographic developments in Finland
international crisis management
public expenditure
obesity among Finns
income differences
fish catches
Available to all at www.findicator.fi
3
Findicator is a comprehensive databank
4
Includes approximately 100 indicators of social progress
Up-to-date and relevant information on important societal issues
Indicators selected in consultation with user groups and data providers
Grouped thematically and by policy issue in line with Government Programme
5
R&D key indicators
In the Finnish KNI collection (findicator.fi), there are only two indicators that measure R&D outcome and effort:
Time series of patent applications
Time series of R&D expenditures, by sector
6
Patent applications in Finland, 1972 - 2010
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
1970 1975 1980 1985 1990 1995 2000 2005 2010
nu
mb
er o
f p
aten
t ap
pli
cati
on
s Foreign
Finnish private
Finnish businesses
7
R&D expenditures by sector, 1971 - 2010
0
1000
2000
3000
4000
5000
6000
7000
8000
1971
1973
1975
1977
1979
1981
1983
1985
1987
1989
1991
1993
1995
1997
1999
2001
2003
2005
2007
2009
mil
lio
n e
uro
s
universities
government+npo
business
8
On the basis of our R&D audits it can be said that…
1. A lot of work has been done (Slides 7-9)
http://www.tekes.fi/u/Better_results_more_value.pdf
2. Some things are just happening (slide 10)
3. A lot of things has still to be done….(slides 11-18). To put it shortly:
”Systemic change is a highly topical issue both in Finland and internationally. Climate change, aging of the population, advancing sustainable development as well as structural transformations in the economy and the related need to find new growth areas are current grand challenges. Responding to these challenges is not possible through individual small-scale reforms but calls for system-level changes”. http://www.tekes.fi/u/systeemisen_muutoksen_haasteet.pdf
9
Indicators related to economy and economic renewal
Phenomenom Indicators
National prosperity GDP per capita
Overall productivity of the economy Total Factor Productivity TFP
Productivity renewal indicator
Foreign Direct Investments Share of Foreign Direct Investments per GDP
Strengthening of intangible assets Share of new innovative products and services from business turnover.
Volume and share of intangible investments
Position in global value-networks Exports of knowledge-intensive sectors
Continuous improvement of competitiveness
Development of turnover in knowledge intensive sectors (or alternatively in KI jobs)
Collaboration, networks and knowledge flows
Share of public and private organisations having collaborated in innovation projects
Capability to innovate Development of patenting, registered trademarks and designs (EPO / USPTO / TRIAD)
Investments in R&D&I Share of R&D&I expenditure in business turnover
Government direct & indirect support to business R&D
Foreign direct investments in Finnish R&D&I
Human resources for R&D&I Availability of highly educated workforce
General conditions and incentives for R&D&I
GDP share of VC investments at different growth stages
9
10
Indicators related to well-being
Phenomenom Indicators
Health and quality of life Life expectancy
Well being in working life Share of 25 to 64-year-olds very or fairly satisfied with their current job
Healthy and safe living environment No indicator selected yet
New knowledge and competence associated with well-being
Scientific publications among the top 10% most cited publications worldwide as % of total scientific publications of the country
Innovations and systemic changes supporting well-being
New products, processes, services and social innovations
High-quality and innovative well being services Productivity of the social and health services of municipalities and federations of municipalities
Quality and extent of R&D&I activities directed towards well-being
Share of public organisations involved in health and well-being related R&D&I activities
Interorganisational collaboration related to well-being in value networks and the strengthening of flows of know-how
Mobility of researchers in the fields of health and well-being
R&D&I investments on well being Private and public R&D expenditure on well-being, health care and working life
Knowledge and human resources No indicator at present
Supportive operational environment Health and social care costs10
11
Indicators related to knowledge, education and culturePhenomenom Indicators
Competences and opportunities for life-long learning Education level of population
Education and active citizenship Interest in science, research and technology
Active and diverse cultural life Value added in the cultural sector
Openness, diversity and networks Share of foreign nationals in the human resources of science and technology
The quality and efficiency of the educational system OECD international student assessment - PISAThe quality and efficiency of higher education and research
Scientific publications within 10% of the most cited publications worldwide as % of total scientific publications of the country
Knowledge as a resource for the economy and society Patent applications by institutes of higher education and public R&D-institutes
Citizen participation Participation of population aged 18+ in lifelong learningActive and vital cultural life No indicator selected at present
Internationalisation and openness in research activities No indicator selected at present
Scientific research and education Share of doctors of the Human Resources in Science and Technology
Disseminating research information to citizens and the use of society
No indicator selected at present
Research and innovation activities related to culture No indicator selected at present
International mobility and cooperation in research Researcher mobility (inwards and outwards)
Investments in competences and human resources Investments in R&D activities in the public sector and in the higher education sector
Investments in competences and human resources Research personnel's share of workforce
Investments in general education and adult education Costs from adult education
Investments in the culture related to research and innovation
The Government R&D funding based on societal objective: culture
Investments in international cooperation and networking The share R&D expenditure from abroad in the Higher Education and Government sectors
11
12
Levels of the implementation
12
• Visual level = Graphics and visualisation of indicators
• Activity level = e.g. availability of the information behind the indicators
• Technical level = platform and connections, technical implementation of administration
• Definition level = Detailed indicator definition
• Data level = sourcing and updating of data
• Administrative level covers all the levels above defining the responsibilities and funding
Visual level
Activity level
Technical level
Data level
Adm
inistrative level
Data
Indicators
Definition level
13
Concluding remarks (1)
There is a clear need for discussion on research and innovation and the related societal development and challenges
The framework should allow for the decision makers to quickly assess the Finnish research and innovation development against the main international trends and key benchmark nations
The question of phenomena selection contains a political aspect
The set of indicators (including the impacts chains) needs continuous update on the basis of newest knowledge
As well, it should adapt to the changes in societal priorities and goals
International cooperation is needed to make existing and forthcoming efforts comparable
More knowledge needs to be gained on analyzing the impacts of research and innovation on well-being and the society 13
14
Concluding remarks (2)
Factors that need to be considered particularly when setting target values to indicators include:
Accumulation factor: how much emphasis is put on the current level of standing, what has been achieved and built over the past years or decades
Improvement and development factor: as compared to relation to peers or selected benchmarks
Systemic factor: how much emphasis is given to the existence and functioning of the research and innovation system (or ecosystem) as a whole, instead of its elements alone.
Global factor: how much emphasis is given to Finnish position and impact in global value chains, EU frameworks or in addressing global challenges, as compared to our success and impact within the country
Relevance factor: how well is the current research and innovation activity tuned for, prepared or adaptive to address the societal challenges, such as ageing of population, economic recession, etc.
14
15
A step to wider scope: where are we now?(Fred Gault: Social impacts of the development of the STI Indicators, 2011)
Indicator development is a dynamic activity. The OECD Innovation Strategy included a Measurement Agenda which is now being implemented. It includes intentions to:
i. Improve the measurement of broader innovation and its link to macro-economic performance;
ii. Invest in high-quality and more comprehensive data infrastructure to measure the determinants and impacts of innovation;
iii. Recognize the role of innovation in the public sector and promote its measurement;
iv. Promote the design of new statistical methods and interdisciplinary approaches to data collection; and,
v. Promote the measurement of social goals and social impacts of innovation”.
16
Special issues…(1)
Policy impacts (Gault, 2011)
As the indicators expand and policy makers recognize that innovation is not an isolated event, more attention is being given to the framework conditions and the policy mix that helps the system to work better. As more micro data analysis is done, the important result that the propensity to innovate in firms is higher than the propensity to do R&D will have more influence on policy.
17
Special issues…(2)
Funding of Finnish universities 2012 The model comprises in three main parts: education, research, and other education and science policy objectives. A total of 75% of the core funding will be allocated on the basis of a formula for education and research, of which 41% is based on educational factors and 34% on research factors. The remaining 25% of the core funding is based on education and science policy objectives.
Education-based funding criteria (as % of core funding): Master’s degrees awarded by the university (15 %), Bachelor’s degrees (9 %), the number of students completing a minimum of 55 credits (11 %, of which 3 % based on data produced by the student feedback system from 2015), credits completed in open university and non-degree studies (2 %), the number of degrees awarded to foreigners by the university (1 %), incoming and outgoing international student exchanges in the university (2 %) and the number of job-holding graduates (1 %).
18
… universities…(3).
The research-based funding criteria: doctoral degrees awarded (9 %), publications (13 %, of which 10 % international refereed publications and 3% other scientific publications - from 2015 the number of the Finnish Publication Forum classification levels 2 and 3 publications instead of international refereed publications and the number of level 1 publications instead of other scientific publications); competed research funding (9 %, of which 3 % international competed research funding and 6% other competed research funding); doctoral degrees awarded to foreigners (1 %); and foreign teaching and research personnel (2 %).
19
SAI’s role in development…?
Attribution/contribution/independence/capacity-building/resources:
So far the NAO of Finland has not participated in Finland’s KNI development
The Future: it is difficult or even impossible to separate the impact of SAI from other inputs and activities.
20
The Role of the SAI with the STI-indicators? i. besserwisser?
ii. Facilitator?
iii. Risk Manager?
iv. Measurement Technician?
v. Capacity
Builder?
(workshops 2011,
brainstorms 2012)
vi. Problem solver?
vi. Link between
R&D actors
and desicion
making?
- from measurement of results for contributor of the knowledge management of the whole R&D
- from peaces to the whole (the life cycle of the knowledge and knowledge management)
- from analyses to synthesises,
- from outputs to processes and capabilities of actors
- from mechanical synthesises to real problem solving
- …by co-operation with the R&D actors
The last steps at 2012: Findicators: An official network to coordinate the development of indicators (1)
to make propositions about developing the services and infrastructures of indicators (and knowledge management) in the government of Finland
to follow-up international development in the field
to co-operate with other developmental projects in this field, etc.
21
The spesific lines of developing R&D indicators in near future 2012 (2)
http://www.tekes.fi/u/Better_results_more_value.pdf
The Academy of Finland and Tekes will report until September 2012 to the Research and Innovation Council of Finland about the main lines of implementing the Framework (slides 9-11 above) (resources, administrative responsibilities and task, pilots (focusing single indicators) etc.
are politics able to utilise this kind of information?
how? when? why? where? how much it costs?
22