November 17, 2008
John MacCollEuropean Director, RLG
Partnership, OCLC ResearchPresentation to RLG UK
PartnershipUniversity of Leeds, 18
September 2009
Libraries in Research Assessment: Implications from a Study of Five Countries
Research questionsResearch questions
• What are the key characteristics of ‘high-intervention’ and ‘low-intervention’ research assessment regimes?
• What are the advantages and disadvantages of the systems of research assessment in the countries studied?
• What are the main activities undertaken by universities in data-gathering in support of research assessment?
• What are the roles which libraries play in data-gathering within their institutions in support of research assessment?
• To what extent are libraries developing systems which achieve their institutional commitment to research assessment but also go further, taking account of possible future changing circumstances, or of the wider need for institutional self-assessment?
• In regimes in which institutional research self-assessment has a significant role, what part do libraries play?
• What roles do libraries play in relation to bibliometrics?• What evidence is there of library management of access to non-standard research
outputs?• Where might universities – and libraries – rationalise their efforts through collaborative
activity, within countries and internationally? • What evidence of good or best practice can be identified within research libraries in
support of national or institutional agendas?
?
Number of HEIs per country (Webometrics)Number of HEIs per country (Webometrics)
UK – 236Netherlands – 154Ireland – 50Denmark – 137Australia – 82(USA – 3328)
Number of universities in Top 200Number of universities in Top 200
THE Top 200 Universities 2004 2008
UK 30 29
Netherlands 5 11
Ireland 1 2
Denmark 3 3
Australia 14 9
USA 62 58
Number of universities in Top 50Number of universities in Top 50
THE Top 50 Universities 2004 2008
UK 8 8
Netherlands 0 0
Ireland 0 1
Denmark 0 1
Australia 6 6
USA 20 20
Gross domestic expenditure on R&D as % of GDP (OECD)Gross domestic expenditure on R&D as % of GDP (OECD)
UK expenditure on QR - £1.6b p.a.?
Transitional timeTransitional time
Regime characteristicsRegime characteristics
Link to funding UK
Self-assessment-based NL
Unit-led NL
Includes publication venue rankings DK, AU
Bibliometrics-led DK
National research administration system NL
?
Publication venue ranking systemsPublication venue ranking systems
• Danish Bibliometric Research Indicator (BRI) System• Modelled on Norwegian: points allocated for publication
in different ‘publication channels’• Eg 5 points for a monograph in Level 2; 8 points for a
monograph in Level 1• But: all Danish publishers were classed as Level 1!
Same tendency within Australian journals.• A Level 1 journal article attracts 3 points; a Level 2 only 1• A Level 1 anthology attracts 2 points; a Level 2 only 0.5• PhDs – 5 points• Has advantage of not being tied to ISI or Scopus• So can much better accommodate humanities• But libraries feel obliged to buy at least all of the top-ranked
journals and publications
Library roles in bibliometric regimesLibrary roles in bibliometric regimes
Researcher fractionalisationResearcher fractionalisation
NetherlandsNetherlands
• Even though Dutch system is ‘non-competitive’, administrators complain about the excessive burden. Mainly because Dutch system is so fine-grained (down to research group level).
• So they are trying to make the internal assessments more light-touch• Some complain that lack of link to funding means that top research talent is not
attracted to the Netherlands• Government gradually building more research themes to encourage
specialisation (in competitive grants awards), so inhibiting academic freedom• Some faculties ‘bribe’ academics to publish in high-impact journals! So the
absence of a link to funding on a national level does not remove game-playing. Indeed, it may lead to the other side of the dual support regime behaving even more competitively.
• ‘Tenure is achievable relatively easily, so there is a need for a stable funding regime to support the existing structure of universities.’ Even if academic jobs are safe, who wants to work in a department with no funding?
• Dutch university system quite decentralised. Many faculty libraries are still the main libraries, and the university library therefore takes little role in research assessment.
Choose: stability, or excellence?Choose: stability, or excellence?
IrelandIreland
• Irony in Ireland is that it has a quite cohesive, well-integrated infrastructure for research management, but no national system of assessment.
?
The accidental success of the IRThe accidental success of the IR
AustraliaAustralia
• ‘There is a general sense of Australia’s distance from world research centres’
• So – partly about national status. Or – perhaps more, about researchers’ fear of being overlooked because of being in Australia
• Creative artists are happy at recognition. And in Denmark (journals) and UK (peer review) too, there is a sense that a national system recognises them, and therefore that the status quo – even without an assessment system – is prejudiced against them
Negative impact on scholarshipNegative impact on scholarship
The ‘central library’The ‘central library’
Could we say this about the UK?Could we say this about the UK?
Fascinating peer-review idea! What does it mean?Fascinating peer-review idea! What does it mean?
?
Reluctance to involve librariesReluctance to involve libraries
Library leaders must engage institutionallyLibrary leaders must engage institutionally
Disciplinary divisionsDisciplinary divisions
Discipline sensitivityDiscipline sensitivity
Disciplinary differences (a RIM theme)Disciplinary differences (a RIM theme)
ValuesValues
?
Key findingsKey findings
• Libraries with repositories have had most involvement in their institutions’ assessment submissions
• Australia furthest ahead there: largely because of government funding for repositories
• The freedom to research what you want is fast disappearing everywhere – and in countries like the UK and Australia, it has been reducing for years. However, governments don’t want to lose it altogether. What is the right balance?
• In most countries, in most institutions, the library senior management role is operational rather than strategic (UK, Australia, Denmark)
• In the Netherlands, ironically, libraries are closely involved at the strategic level – but not in relation to research assessment
Directions for research libraries?Directions for research libraries?
1. Help make the existing processes more efficient. It’s in the library’s interest for its institution to do well in research assessment exercises. Individual institutional libraries want to learn about best practice in order to create and embed new workflows as cost-effectively as possible.
2. Aim to provide more data to the mix that might help the system to become fairer. Are there ways in which the library community can arrive at its own metrics (eg libcitations; MESUR)?
3. Develop neutral role into a strategic asset. The library should be a source of interdisciplinary, international advice to its institution, and to national and international bodies, as we work through a transition towards fairer assessment methods in an increasingly competitive environment.
?
Thank YouThank YouJohn [email protected] Research
St Andrews University Library (Gareth JM Saunders)