Upload
sally-hort
View
217
Download
2
Embed Size (px)
Citation preview
October 24, 2005 ISPM academic staff meeting 1
Research Assessment Exercise
Medical Faculty, University of Bern
Matthias Egger
October 24, 2005 ISPM academic staff meeting 2
Menu
• Research Assessment Exercises (RAE)
• Important bibliometric indices– The Impact Factor (IF)– The Maximum Rank (MR)
• Methodology of Bern RAE
• ISPM publications 2002-2004
• Results of 2003 RAE
• Conclusions
October 24, 2005 ISPM academic staff meeting 3
Rationale for RAE
• Improve quality of research
• Foster accountability and transparency
• Promote strategic thinking and focus
• Guide resource allocation– UK and other countries– Basel– Bern?
October 24, 2005 ISPM academic staff meeting 4
Bibliometric indices
October 24, 2005 ISPM academic staff meeting 5
Journal impact factor
• The number of citations in one year (for example 2002) to items published in a specific journal in the two preceding years (2000 and 2001) divided by the total number of citable articles (original articles and reviews, but not other items) published in this two-year period.
• Journal specific measure
October 24, 2005 ISPM academic staff meeting 6
Journal citation reports (JCR)
• There are two editions– Science JCR– Social Science JCR
• Journals are grouped in subject categories– Typical impact factors vary between categories
• The same journal may appear in more than one subject category
• Updated every year
October 24, 2005 ISPM academic staff meeting 7
Subject categories: examples
• Social Science– PUBLIC, ENVIRONMENTAL & OCCUPATIONAL
HEALTH
– HEALTH POLICY & SERVICES
– SOCIAL SCIENCES, BIOMEDICAL
– PSYCHOLOGY, APPLIED
– PSYCHOLOGY, SOCIAL
– SOCIOLOGY
– WOMEN'S STUDIES
October 24, 2005 ISPM academic staff meeting 8
Maximum Rank (MR)• Measure of the standing of a journal within a
given field (subject category)
• Controls for differences in impact factors across subject categories
• Defined as: IF-Rank of Journal -1Rank in Subject Category = 1 - ------------------------------ Number of Journals
MR = Maximum of all ranks, if the journal is listed in several subject cateogories
October 24, 2005 ISPM academic staff meeting 9
Top Journals (MR=1)
• New England Journal of Medicine (IF=38.57)
• Value in Health (IF=3.66)
• Womens Health Issues (IF=1.352)
October 24, 2005 ISPM academic staff meeting 10
MR: Examples
• British Medical Journal
• Social Science & Medicine
• Statistics in Medicine
• J Adolescent Health
• Sex Transm Inf
• http://wok.consortium.ch/
October 24, 2005 ISPM academic staff meeting 11
Social Science & MedicineIF=2.088 (2004)
• PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (Social Science Edition)5/60 MR=0.933
• SOCIAL SCIENCES, BIOMEDICAL2/26 MR=0.961
October 24, 2005 ISPM academic staff meeting 12
Statistics in MedicineIF=1.389 (2004)
• PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH46/93 MR=0.516
• MEDICAL INFORMATICS6/18 MR=0.720
• STATISTICS & PROBABILITY13/77 MR=0.844
October 24, 2005 ISPM academic staff meeting 13
J Adolescent HealthIF=1.571 (2004)
• PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH(Science Edition)46/93 MR=0.580
• PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH(Social Science Edition)12/60 MR=0.817
October 24, 2005 ISPM academic staff meeting 14
Sex Transm InfIF=1.181 (2002)IF=2.204 (2004)
• 2002DERMATOLOGY & VENEREAL DISEASES7/40 MR=0.850
• 2004INFECTIOUS DISEASES19/41 MR=0.561
October 24, 2005 ISPM academic staff meeting 15
RAE 2003: Methods
October 24, 2005 ISPM academic staff meeting 16
Bern RAE 2003
• Separate evaluation of two groups– Clinics– Institutes (n=21)
• Three ranks based on– Number of all original articles / Stellenpunkt– Average MR of in-house original articles– Competitive grant income spent / Stellenpunkt
• Global ranking based on sum of three ranks
October 24, 2005 ISPM academic staff meeting 17
Definitions
• Original articles– Pubmed-indexed original articles– www.pubmed.org
• Original articles in-house – first and/or last author from ISPM
• Original articles collaborations
• Other publications
October 24, 2005 ISPM academic staff meeting 18
• Institut für Infektionskrankheiten • Institut für Pathologie • Institut für Klinische Pharmakologie • Institut für Rechtsmedizin • Institut für Sozial- und Präventivmedizin • Fakultäre Instanz für Hausarztmedizin • Universitätsspital-Bibliothek• Institut für Medizinische Lehre • Kollegiale Instanz für Komplementärmedizin• Departement Klinische Forschung • Institut für Anatomie • Institut für Physiologie• Institut für Biochemie und Molekularbiologie • Theodor-Kocher-Institut • Institut für Medizingeschichte • Institut für Pathophysiologie • Institut für Pharmakologie • Institut für chirurgische Technologien und Biomechanik• Institut für Evaluative Forschung in orthopädischer Chirurgie• Institut für Dental- und Skelettbiologie • Zahnmedizinische Kliniken
21 I
nsti
tute
s
October 24, 2005 ISPM academic staff meeting 19
ISPM publications 2002-2004
• 188 publications listed on website
• 100 original articles indexed in Pubmed
• 92 articles with impact factor
• 57 in-house articles, 52 with impact factor
• 56 journals
• 28 subject categories
October 24, 2005 ISPM academic staff meeting 20
Sex Transm Inf 7 Lancet 6Soz Praeventivmed 6BMJ 5Int J Epidemiol 5Clin Oral Implants Res 3Eur J Public Health 3Eur Respir J 3Int J STD & AIDS 3JAMA 3Swiss Med Wkly 3AIDS 2Eur J Clin Nutr 2Eur Spine J 2Gesundheitswesen 2J Clin Epidemiol 2J Infect Dis 2Rheumatology 2
Stat Med 2AIDS Res Human Retrovir 1Acta Paediatr 1Age Ageinig 1Aids Care 1Alcohol 1Am J Epidemiol 1Am J Public Health 1Am J Respir Crit Care Med 1Anesthesiology 1Arch Int Med 1BMC Med Res Methodol 1Clin Chim Acta 1Clin Sci 1Clin Ther 1 Contr Clin Trials 1 Diagnostica 1 Epidemiology 1
Jour
nals
October 24, 2005 ISPM academic staff meeting 21
Eur J Cancer 1 Eur J Epidemiol 1 Familial Cancer 1 HIV Med 1 Health Psychol 1 Health Technol Assess 1 Heart 1 Hepatology 1 Human and Ecol Risk Assess 1 Int J Hyg Environ Health 1 Int J Techno Assess Health Care 1 J Clin Endocrin Metabol 1 J Epidemiol Community Health 1 J Occup Environm Med 1 J Public Health 1 P Natl Acad Sci USA 1 Pest Manag Sci 1 Resp Res 1 Scan J Work Environ Health 1 Schweiz Monatsschr Zahnmed 1
Jour
nals
October 24, 2005 ISPM academic staff meeting 22
In-house publications in journals indexed in Social Science Edition
• Eur J Public Health (AGF 2, EpStat 1)
• Gesundheitswesen (Epstat 1)
October 24, 2005 ISPM academic staff meeting 23
PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH 25 MEDICINE, GENERAL & INTERNAL 18INFECTIOUS DISEASES 10RESPIRATORY SYSTEM 4DENTISTRY, ORAL SURGERY & MEDICINE 3DERMATOLOGY & VENEREAL DISEASES 3NUTRITION & DIETETICS 2ORTHOPEDICS 2PHARMACOLOGY & PHARMACY 2PSYCHOLOGY, CLINICAL 2RHEUMATOLOGY 2STATISTICS & PROBABILITY 2VIROLOGY 2AGRONOMY 1ANESTHESIOLOGY 1CARDIAC & CARDIOVASCULAR SYSTEMS 1CRITICAL CARE MEDICINE 1ENDOCRINOLOGY & METABOLISM 1
Sub
ject
cat
egor
ies
October 24, 2005 ISPM academic staff meeting 24
ENVIRONMENTAL SCIENCES 1GASTROENTEROLOGY & HEPATOLOGY 1GERIATRICS & GERONTOLOGY 1HEALTH CARE SCIENCES & SERVICES 1MEDICAL LABORATORY TECHNOLOGY 1MEDICINE, RESEARCH & EXPERIMENTAL 1MULTIDISCIPLINARY SCIENCES 1ONCOLOGY 1PEDIATRICS 1PSYCHOLOGY, CLINICAL 1TOXICOLOGY 1
Sub
ject
cat
egor
ies
October 24, 2005 ISPM academic staff meeting 25
010
2030
Fre
quen
cy
0 5 10 15 20 25impactfactor
October 24, 2005 ISPM academic staff meeting 26
05
1015
Fre
quen
cy
.2 .4 .6 .8 1maxrank
October 24, 2005 ISPM academic staff meeting 27
2002 2003 2004 2002-04
AGF 4 8 5 17
EpStat 21 23 39 83
ISPM 25 31 44 100
Number of original articles
October 24, 2005 ISPM academic staff meeting 28
2002 2003 2004 2002-04
AGF 0.147(n=1)
0.392(n=5)
0.647(n=1)
0.393(n=7)
EpStat 0.842(n=11)
0.681(n=13)
0.705(n=21)
0.731(n=45)
ISPM 0.784(n=12)
0.600(n=18)
0.702(n=22)
0.695(n=52)
Average MR in-house original articles
October 24, 2005 ISPM academic staff meeting 29
RAE 2003: Results
October 24, 2005 ISPM academic staff meeting 30
RAE 2003 ISPM All 21 units
Staff points (Stellenpunkte, SP)
1694 1722
No of original publications
Number / SP
31
0.0183
19.7
0.0114
MR of inhouse publications 0.684 0.819
Competitive grant income / year / SP 762 1366
4
14
3
October 24, 2005 ISPM academic staff meeting 31
Core funding of ISPM
• 1694 staff points, corresponding to ~2.1 Mio SFr/year
• 125,000 SFr / year for running costs
• Rent of offices, maintenance
October 24, 2005 ISPM academic staff meeting 32
Current distribution of staff points
By administrative unit• Director’s office 250 (15%)• Central services 271 (16%)• AGF 559 (33%)• EpStat 614 (36%)
By research unit• AGF 559 (41%)• Epstat+ME 806 (59%)
October 24, 2005 ISPM academic staff meeting 33
Conclusions• We are doing very well in terms of the number
of original articles
• We are doing less well in terms of the Maximum Rank of our in-house publications
• Our output is very heterogeneous– Lack of focus?– Reflection of consultancies with University
Hospital and others?
October 24, 2005 ISPM academic staff meeting 34
Conclusions• EpStat contributes substantially more RAE-
relevant output than AGF• There are very few in-house publications in
journals indexed in Social Science Edition• There are many journals that rank highly in the
Social Science Edition and that may be of interest to us
• We need to make sure that the faculty appropriately considers Social Science journals
October 24, 2005 ISPM academic staff meeting 35
High ranking journals indexed in Social Science Edition
Addiction (1.0) Soc Health Illness (0.922)Aids Care (0.846) Social Sci Med (0.961)Am J Health Prom (0.900) Sociol Health Ill (0.922)Archives Sex Behav (0.892) Women Health Issues (1.0)Demography (0.941)Health Psychology (0.928)Health Services Research (0.842)J Adolescent Health (0.817)J Health Soc Behav (0.782)J Soc Policy (0.928)Medical Care (0.921)Perspect Sex Repro Health (1.0)Psychosom Med (0.930)
October 24, 2005 ISPM academic staff meeting 36
Conclusions• We should generally try to publish our work in
journals that rank high in relevant subject categories (MR>0.7)
• Everyone at ISPM should know these journals for her/his field
• We may want to exclude original publications with very low MRs from the list we submit to the faculty
• We should use our core funding (staff points) wisely, taking into account the RAE (every staff point counts)