Innovation Measurement

Keith SmithImperial College London/TIK Oslo

Why do we need data?

• Economy-wide data enables a structural, generalisable view to emerge

• It allows us to explore the properties of a system as a whole

• It helps us to identify where the really relevant questions are

The background issues

Historically, 3 sources of data:• R&D• Patents• Bibliometric

Each has more or less serious problems as innovation indicators

Problems with existing indicators

• All have problems with their conceptual and definitional bases

• Two are by-products of legal or institutional processes – patent law or academic publishing conventions

• None focus directly on innovation

Research and Development Data

• Collected by survey, procedures formalised in OECD ‘Frascati Manual’ (1968)

• Collects data on expenditure on R&D, personnel employed (in FTEs), types of research (basic, strategic, applied, experimental), object (by field)

• Monitored by OECD NESTI working party

R&D Indicators

• The most common indicator: ‘R&D Intensity’ • R&D Intensity = R&D/GDP or R&D/GVA ratio• Countries and firms can be ranked using this

ratio• It is often used as a policy target (Norway –

target to reach OECD average for R&D/GDP; EU target ‘to reach 3%’)

Problems with R&D intensity indicator

• The overall indicator reflects not only R&D effort but also the industrial structure of the country

• If the country is heavily based on low R&D industries, then the aggregate indicator will be low even if the country is relatively R&D intensive – so the aggregate intensity indicator is misleading as in terms of country efforts (Norway has low R&D/GDP even though it is relatively high in many industries)

R&D and high tech sectors

• The OECD uses R&D to distinguish between technology intensity of industries

• High tech= >4% R&D/GVA ratio• Medium tech = between 1 and 4 %• Low tech = <1%

But this only indicates R&D performance, it does not reflect use of science, non-R&D inputs, technology flows etc. By this criterion food is a low tech sector, when actually it is strongly science using.

Patents

• A patent is a grant of monopoly use of a discovery, usually for a period of 17 years

• The discovery must be an advance in the state of the art, and non-obvious

• Problems: patents are only rarely taken into use. Their economic value usually varies enormously. Very few firms patent. Research shows that patenting is not a strong method of appropriation.

Bibliometric data

• Data on scientific publication and citations (publications from ‘World of Science’, citations from Science Citation Index)

• Widely collected and widely available by field• ‘High Impact’ publications are in the top 1

percent of highly cited publications• Can map relative national performance, filed

changes, international collaboration• Can indicate surprising changes in world patterns

Innovation indicators

• Emerged in 1980s as researcher-driven exercises in France, Germany, USA, Italy, Scandinavia

• Development of OECD ‘Innovation manual’ (the ‘Oslo Manual’) in early 1990s

• First Community Innovation Survey 1992

The Community Innovation Survey

Covers:

• Direct outputs of innovation – sales from new and technologically changed products

• Inputs – R&D, design, marketing, training, acquisition of licences etc

• Collaboration – partners and locations• Sources of information• Incentives and Obstacles

CIS

• Now implemented six times, currently every two years

• Funded and overseen by European Commission (Eurostat in Luxembourg)

• Frequently revised by R&D and Innovation working party – covers sampling and collection methodologies

• Also collected in Canada, Australia, China, India, Brazil, Russia, South Africa.

Main CIS results – what did we learn

• Innovation drives growth – the CDM model• Much weaker role of R&D than expected• Pervasiveness of innovation – especially in

‘low tech’ sectors• Asymmetry in innovation performance • Central role of collaboration• Characteristics of highly innovating firms

(distributed across all sectors)

Publications using CIS

1994 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011

0

50

100

150

200

250

300

350

400

450

0

10

20

30

40

50

60

70

80

90

100

4 9 14 20 27 34 5081

100140

165 182211

261

316

366

427

Academic papers (in English) using Community Innovation Survey data (1994-2011)

Cumulative ...

Publication year

Nu

mb

er

of

stu

die

s

Publications and versions of CIS

0

5

10

15

20

25

30

35

40

1994 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011Nu

mb

er o

f ac

adem

ic p

aper

s

Publication year

Use of each CIS version over time

CIS 1

CIS 2

CIS 3

CIS 4

CIS 2006

CIS 2008