ContentsContents
• Part I: Getting Published
• Part II: Being Measured
• Part III: Real World Outputs from Liaison Psychiatry
NHS Power -2004 StatsNHS Power -2004 StatsAcademics• 494 clinical lecturers (down 17% on 2003)• 1500 Readers / SnR Lecturers (down 11% on 2000)• 1146 Professors (up 10% on 2000)
• 3113 FTE clinical academics in all specialties
• 302 SnR academics in Psychiatry • 40 lecturers in Psychiatry
NHS Power -2004 StatsNHS Power -2004 StatsAcademics• 494 clinical lecturers (down 17% on 2003)• 1500 Readers / SnR Lecturers (down 11% on 2000)• 1146 Professors (up 10% on 2000)• 302 SnR academics in Psychiatry • 40 lecturers in Psychiatry • 3113 FTE clinical academics in all specialtiesNHS Staff• 117,000 NHS doctors (up 5% on 2000)• 34,000 GPs (up 2%)• 30,600 Consultants (up 5%)• 397,000 Nurses (up 2.5%)• 129,000 Allied health professionals (up 4%)• 1,071,462 FTE qualified staff• 3,500 NHS Consultant Psychiatrists vs 350 academic psychiatrists
38,411
30,650
2,294
5,467
41,697
16,823
20,601
4,273
6,888
1,048
3,4762,364
34,085
0
5000
10000
15000
20000
25000
30000
35000
40000
45000Care
er grad
es C
onsult
ant
Ass
ociate
Specia
list
Staf
f grad
eDocto
rs in tra
ining
Reg
istrar
group 2
Senio
r House
Offic
er
Hous
e Offic
er an
d FY1Other
grades
Hosp
ital P
ractiti
oner
Clin
ical A
ssist
ant O
ther Staf
f
GPs
Source: DOH Workforce Stats
618
4,355
392670
6,726
1,413
1,902
2,411
926
3,555
1,928
5,754
0
1,000
2,000
3,000
4,000
5,000
6,000
7,000
8,000
Acciden
t & em
ergenc
y
Anaesth
etics
Clinica
l onco
logyDen
tal Gro
up
General
med
icine g
roup 2
Obstet
rics &
gynae
cology
Paedia
tric gr
oup
Patholog
y gro
up
Public H
ealth
Psych
iatry gr
oup
Radiolog
y gro
upSurg
ical g
roup
Source: DOH Workforce Stats
Consultants by Specialty
ProductivityProductivity
• Academics– 3000 x 5 papers per year– N= 15,000
• NHS Medical Staff– 120,000 x 1 paper every 5 years!– N= 24,000
• NHS AHP– 500,000 x 1 paper every 100 years– N= 5,000
• Ratio of Outputs = 15,000 to 29,000 (1:2)
ProductivityProductivity
• Academics– 3000 x 5 papers per year– N= 15,000
• NHS Medical Staff– 120,000 x 1 paper every 5 years!– N= 24,000
• NHS AHP– 500,000 x 1 paper every 100 years– N= 5,000
• Ratio of Outputs = 15,000 to 29,000 (1:2)
Typical Journal Statistics (eg BMJ)Typical Journal Statistics (eg BMJ)• 7000 research papers received• 1000 rejected by one editor immediately
– further 4500 rejected by editors after reading
– Only 25% (1500) sent to reviewers
– Only 10% reach weekly manuscript meeting • (with Editor, a clinicians and a statistician)
– 350 research articles after revisions (and up to 6 peer reviews)
• Only 3% (1990-2000) to 6% (2000+) accepted by BMJ=> time!
Conventional Publication Cycle – 24moConventional Publication Cycle – 24mo
Step1
Paper Completed
Step 2
Present to colleagues (informal PR)
Step 3
Present at conference
Step 4
Submit to journal
Step 6
Peer
Review 1
Step 7
Revision 1
Step 8
Peer Review 2
Step 10
Proof
T0 T+1mo
Step 12
Readers letters (public PR)
T+3mo T+6mo
Step 11
Restricted Publication
Step 9
Revision 2
Step 5
Editor Agrees to Send for Review
T+7mo T+10mo T+12mo T+14mo
T+15mo T+18mo T+24mo
Open Publication Cycle – 12moOpen Publication Cycle – 12mo
Step1
Paper Completed
Step 2
Present to colleagues (informal PR)
Step 3
Present at conference
Step 4
Submit to online journal
Step 7 Peer Review 1
Step 6 Public comments
Step 8
Revision
T0 T+1mo
Step 11
Readers letters (post-publication)
T+3mo T+5mo
Step 10
Open Access Publication
Step 9
Online proof
Step 5
Editor Publishes “discussion paper”
T+6mo T+7mo T+9mo
T+11mo T+12mo
=> Good news2
How Many?How Many?• Hospital Specialist
• 75 journals in psychiatry • + 50 in neurology• + 50 in medicine• + 25 in misc. fields
– 2400 issues per year– 24,000 specialty articles per year
– 6.5 journals per day
– 65 articles per day
The Publishing LandscapeThe Publishing Landscape
10,000s+Not generally peer-reviewed but highly targetted readership
PulseMagazine titles
Direct ReadershipScopeExample
100s+Not peer-reviewed, not widely read, never cites
NewsletterInternal titles
1000sPrimary care focus, can be low scientific impact if jnl is new
PCPPrimary care titles
Sub-specialists on disease1000s
Specific disease area/therapy class, low impact
J of ECTSub-specialty titles
Specialists (mainly secondary care)10,000s-00,000s
Covering specialty as whole, high impact (IF3+)
BJPsychSpecialty titles
Generalist1000,000s?
Generalist, very high impact (IF5+), closely peer reviewed
The LancetGeneralist titles
Some Obscure Psychiatry TitlesSome Obscure Psychiatry Titles
STRESS AND HEALTH0.721
INT JJOURNAL PSYCHIATRY IN MEDICINE0.733
ARCH PSYCHIAT NURS0.734
NORDIC J PSYCHIAT0.752
J PSYCHOSOMATIC OBST & GYN0.951
BEHAVIORAL MEDICINE1
NEUROPSYCHIATRIE1.051
PERSPECT PSYCHIATR CARE1.062
JOURNAL OF ECT1.176
PSYCHIATRY CLINICAL NEUROSCIENCE1.181
ACTA NEUROPSYCHIATRY1.206
SUBST USE MISUSE1.229
AGING MENT HEALTH1.264
PSYCHOPATHOLOGY1.441
NEUROCASE1.505
WORLD J BIOL PSYCHIATRY1.691
Overview of Peer ReviewOverview of Peer Review
Grading Accept
Revise
Reject
Peer Review Filter
Published Article
Send elsewhere
Reject
Qualitative Grade
Editorial Filter
Quantitative Grade
Outcome
Comments to Author
Author Filter
Article Appraisal
Where to Submit
Bias from Hidden ReviewersBias from Hidden Reviewers
AuthorAuthor
Friend
Unknown
Unknown
Enemy
Revision
Rejection
Enemy
+
+
--
-
Acceptance
See Maddox J. Conflicts of interest declared [news]. Nature 1992; 360: 205; Locke S. Fraud in medicine [editorial]. Br Med J 1988; 296: 376–7.
What Makes a Good Reviewer?What Makes a Good Reviewer?• A reviewer was less than 40 years old
• From a top academic institution
• Well known to the editor choosing the reviewer
• Author blinded to the identity of the manuscript's authors
• Then the probability of good review was 87% vs 7%
Evans AT, McNutt RA, Fletcher SW, Fletcher RH. The characteristics of peer reviewers who produce good-quality reviews. J Gen Intern Med. 1993 Aug;8(8):422-8
Open & Blind Review & SubmissionOpen & Blind Review & Submission
• Does revealing reviewers’ identity influence outcome? – Open vs blind peer review
• Does revealing Authors’ identity influence outcome?– Open vs blind submission
• http://www.slideshare.net/ajmitchell
Problems with AuthorsProblems with Authors• Multiple publication
• Ghost authorship
• Gift authorship
• Conflicts of Interest
• Fraud / Plagarism
Ghost AuthorshipGhost Authorship• An Inquiry by the UK House of Commons Health Select
Committee into the Influence of the Pharmaceutical Industry October 2004– 50% of therapeutics articles may be ghost written
• Flanagin A et al (1998) Prevalence of Articles With Honorary Authors and Ghost Authors in Peer-Reviewed Medical Journals JAMA 280:222-224
• Of 809 articles– 19% had evidence of honorary (gift) authors– 11% had evidence of ghost authors– 2% had evidence of both
How Many Authors?How Many Authors?• Deep Impact: Observations from a Worldwide Earth-Based CampaignK. J. Meech, N. Ageorges, M. F. A'Hearn, C. Arpigny, A. Ates, J. Aycock, S. Bagnulo, J. Bailey, R. Barber,
L. Barrera, R. Barrena, J. M. Bauer, M. J. S. Belton, F. Bensch, B. Bhattacharya, N. Biver, G. Blake, D. Bockelée-Morvan, H. Boehnhardt, B. P. Bonev, T. Bonev, M. W. Buie, M. G. Burton, H. M. Butner, R. Cabanac, R. Campbell, H. Campins, M. T. Capria, T. Carroll, F. Chaffee, S. B. Charnley, R. Cleis, A. Coates, A. Cochran, P. Colom, A. Conrad, I. M. Coulson, J. Crovisier, J. deBuizer, R. Dekany, J. de Léon, N. Dello Russo, A. Delsanti, M. DiSanti, J. Drummond, L. Dundon, P. B. Etzel, T. L. Farnham, P. Feldman, Y. R. Fernández, M. D. Filipovic, S. Fisher, A. Fitzsimmons, D. Fong, R. Fugate, H. Fujiwara, T. Fujiyoshi, R. Furusho, T. Fuse, E. Gibb, O. Groussin, S. Gulkis, M. Gurwell, E. Hadamcik, O. Hainaut, D. Harker, D. Harrington, M. Harwit, S. Hasegawa, C. W. Hergenrother, P. Hirst, K. Hodapp, M. Honda, E. S. Howell, D. Hutsemékers, D. Iono, W.-H. Ip, W. Jackson, E. Jehin, Z. J. Jiang, G. H. Jones, P. A. Jones, T. Kadono, U. W. Kamath, H. U. Käufl, T. Kasuga, H. Kawakita, M. S. Kelley, F. Kerber, M. Kidger, D. Kinoshita, M. Knight, L. Lara, S. M. Larson, S. Lederer, C.-F. Lee, A. C. Levasseur-Regourd, J. Y. Li, Q.-S. Li, J. Licandro, Z.-Y. Lin, C. M. Lisse, G. LoCurto, A. J. Lovell, S. C. Lowry, J. Lyke, D. Lynch, J. Ma, K. Magee-Sauer, G. Maheswar, J. Manfroid, O. Marco, P. Martin, G. Melnick, S. Miller, T. Miyata, G. H. Moriarty-Schieven, N. Moskovitz, B. E. A. Mueller, M. J. Mumma, S. Muneer, D. A. Neufeld, T. Ootsubo, D. Osip, S. K. Pandea, E. Pantin, R. Paterno-Mahler, B. Patten, B. E. Penprase, A. Peck, G. Petitas, N. Pinilla-Alonso, J. Pittichova, E. Pompei, T. P. Prabhu, C. Qi, R. Rao, H. Rauer, H. Reitsema, S. D. Rodgers, P. Rodriguez, R. Ruane, G. Ruch, W. Rujopakarn, D. K. Sahu, S. Sako, I. Sakon, N. Samarasinha, J. M. Sarkissian, I. Saviane, M. Schirmer, P. Schultz, R. Schulz, P. Seitzer, T. Sekiguchi, F. Selman, M. Serra-Ricart, R. Sharp, R. L. Snell, C. Snodgrass, T. Stallard, G. Stecklein, C. Sterken, J. A. Stüwe, S. Sugita, M. Sumner, N. Suntzeff, R. Swaters, S. Takakuwa, N. Takato, J. Thomas-Osip, E. Thompson, A. T. Tokunaga, G. P. Tozzi, H. Tran, M. Troy, C. Trujillo, J. Van Cleve, R. Vasundhara, R. Vazquez, F. Vilas, G. Villanueva, K. von Braun, P. Vora, R. J. Wainscoat, K. Walsh, J. Watanabe, H. A. Weaver, W. Weaver, M. Weiler, P. R. Weissman, W. F. Welsh, D. Wilner, S. Wolk, M. Womack, D. Wooden, L. M. Woodney, C. Woodward, Z.-Y. Wu, J.-H. Wu, T. Yamashita, B. Yang, Y.-B. Yang, S. Yokogawa, A. C. Zook, A. Zauderer, X. Zhao, X. Zhou, and J.-M. ZucconiScience 14 October 2005: 265-269.
Part II: Being MeasuredPart II: Being Measured
Impact Factors
Journals vs papers vs
individuals
H-Index
Dissemination
How to Grade Research…of an AuthorHow to Grade Research…of an Author
10% top 10 journals
70% peer reviewed journals
20% other material (eg chapters)
100% Listed on Pubmed
75% Listed on Pubmed90% Listed on Scopus/WoK
5% Listed on Pubmed15% Listed on Scopus/WoK
How to Grade Research…of an AuthorHow to Grade Research…of an Author
10% top 10 journals
70% peer reviewed journals
20% other material
Impact(citations)
The Dreaded “Impact Factor”The Dreaded “Impact Factor”
Q. How do you know what published research is important?
Q. How do you decide what research to read?
What is the Impact factor?What is the Impact factor?
• Journal IF• A = citations in the last 2 years in journal• B = number of articles Impact Factor = A/B
• Article IF• A = Total citations of article• B = Age of article Impact Factor = A/B
• Personal IF• A = citations in the last 2 years by individual• B = number of articles Impact Factor = A/B
What is the Impact factor?What is the Impact factor?
• Journal IF• A = citations in the last 2 years in journal• B = number of articles Impact Factor = A/B
• Article IF• A = Total citations of article• B = Age of article Impact Factor = A/B
• Personal IF• A = citations in the last 2 years by individual• B = number of articles Impact Factor = A/B
Eugene GarfieldEugene Garfield
• Citation Databases– Science citation index (CD)– BIDS– ISI Science Direct– Web of Science– Web of Knowledge– Scopus
0
0.5
1
1.5
2
2.5
3
1988 1990 1993 1994 1995 1996 1997 1998 1999 2000 2001
CM
AJ'
s im
pact
fact
or
Hoey J, Todkill AM. CMAJ 2003;168:287-8 [online appendix]
One Journal’s Impact Factor Trend
http://www.americanscientist.org/my_amsci/restricted.aspx?act=pdf&id=3263000957901
Increase in Scientific Output
Importance of ReviewsImportance of Reviews• WHITTINGTON, C. J., KENDALL,T., FONAGY, P., COTTRELL, D.,
COTGROVE, A. & BODDINGTON, E. (2004) Selective serotonin reuptake inhibitors in childhood depression: systematic review of published versus unpublished data. Lancet, 363, 1341-1345.
• Lancet Paper of the Year 2004
• BACKGROUND: Questions concerning the safety of selective serotonin reuptake inhibitors (SSRIs) in the treatment of depression in children led us to compare and contrast published and unpublished data on the risks and benefits of these drugs. METHODS: We did a meta-analysis of data from randomised controlled trials that evaluated an SSRI versus placebo in participants aged 5-18 years and that were published in a peer-reviewed journal or were unpublished and included in a review by the Committee on Safety of Medicines. The following outcomes were included: remission, response to treatment, depressive symptom scores, serious adverse events, suicide-related behaviours, and discontinuation of treatment because of adverse events. FINDINGS: Data for two published trials suggest that fluoxetinehas a favourable risk-benefit profile, and unpublished data lend support to this finding. Published results from one trial of paroxetine and two trials of sertraline suggest equivocal or weak positive risk-benefit profiles. However, in both cases, addition of unpublished data indicates that risks outweigh benefits. Data from unpublished trials of citalopram and venlafaxine show unfavourable risk-benefit profiles. INTERPRETATION: Published data suggest a favourable risk-benefit profile for some SSRIs; however, addition of unpublished data indicates that risks could outweigh benefits of these drugs (except fluoxetine) to treat depression in children and young people. Clinical guideline development and clinical decisions about treatment are largely dependent on an evidence base published in peer-reviewed journals. Non-publication of trials, for whatever reason, or the omission of important data from published trials, can lead to erroneous recommendations for treatment. Greater openness and transparency with respect to all intervention studies is needed.
Part III: Real World Research OutputsPart III: Real World Research Outputs
Journal outputs
Top 10 Psychiatric papers
Top 10 Authors
IF and Submission RatioIF and Submission Ratio
• The IF can be considered inversely proportional to the chances of acceptance
BMJ Manuscript Processing
5000 scanned by 1 editor
2500 sentfor review
1000 Immediately
rejected
6000 received per year
Poorly writtenObvious flawsToo Obscure
Hand writtenSilly mistakesWrong journal
100 accepted
2500Rejected by
editor
1000 Rejected by reviewer /
editor
Methodological concerns
Not interesting
400 rejected
1500 sent to hanging
committees
500 discussed
100 accepted400 rejected500 discussed
100 accepted400 rejected500 discussed
300 accepted per year(6 per week + 2 short
reports)
Acceptance Rate = 1 in 20
Liaison JournalsLiaison Journals
>10.0300.733823INT J PSYCHIAT MED0.733
9.2131479BEHAV MED1
3.4451.206201ACTA NEUROPSYCHIATR1.206
7.91571.8596203J PSYCHOSOM RES1.859
7.8552.0532396J NEUROPSYCH CLIN N2.053
3.31782.0951937EPILEPSY BEHAV2.095
7832.1171952GEN HOSP PSYCHIAT2.117
7.5802.1992259PSYCHOSOMATICS2.199
Cited Half LifeArticlesImpact FactorTotal CitesAbbreviated Journal TitleImpact Factor
Individual PapersIndividual Papers
Q. What are the “most important” papers in psychiatry?
Q. How many times do you think the top papers are cited?
Personal “Impact Factor”Personal “Impact Factor”
Q. Who are the most influential authors in psychiatry?
Q Can influence by measured?
GoldGeneral PsychiatryASZIGMOND
GoldOld Age PsychiatryKMCWILSON
GoldGeneral PsychiatryJLTREASURE
GoldGeneral PsychiatryGSTRATHDEE
GoldGeneral PsychiatryTESENSKY
GoldGeneral PsychiatrySPRIEBE
GoldGeneral PsychiatrySO'CONNOR
GoldGeneral PsychiatryASLEE
GoldGeneral PsychiatryPBJONES
GoldGeneral PsychiatrySHUNTER
GoldC & A PsychiatryJWHILL
GoldForensic PsychiatryDHGRUBIN
GoldC & A PsychiatryRGOODMAN
GoldGeneral PsychiatryEAGILVARRY
GoldGeneral PsychiatryASDAVID
GoldC & A PsychiatryPFBOLTON
GoldGeneral PsychiatryDBHUGRA
GoldGeneral PsychiatryLJAPPLEBY
PlatinumC & A PsychiatryRJWWILLIAMS
PlatinumGeneral PsychiatryGJTHORNICROFT
PlatinumGeneral PsychiatryMJOWEN
PlatinumOld Age PsychiatryIGMCKEITH
PlatinumGeneral PsychiatryJHLACEY
PlatinumGeneral PsychiatryGMGOODWIN
PlatinumGeneral PsychiatryCJAFAIRBURN
PlatinumC & A PsychiatrySMBAILEY
LevelSPECIALTYINITIALSSURNAME
Measuring Individuals ResearchMeasuring Individuals ResearchOld Methods (counts)• Subjective top 4 publications (RAE)• Subjective top 10 publications (CEA form D)• Total publications count• Total Peer reviewed publications
New Methods (IFs)• Total citations• Citations per publication• Citations per year (last 2 years)• No. publications with same number of citations (H-index)• Citations per year since qualification
Output MeasuredOutput Measured
12h-index
4.17Average Citations per Item
325Sum of the Times Cited
78Results found:
Publication Count in Liaison (Anonymous)Publication Count in Liaison (Anonymous)
464
292
243
203183 183
134114
87 78 77 69 67 6142 40
26 21 18 17 10
0
50
100
150
200
250
300
350
400
450
500
Academ
icAca
demic
Academ
icAca
demic
Academ
icAca
demic
Academ
icAca
demic
Academ
icNHS
Academ
icAca
demic
Academ
icNHS
Academ
icAca
demic
Academ
icNHSNHSNHSNHS
Citation Count by Liaison (anonymous)Citation Count by Liaison (anonymous)
0
2,000
4,000
6,000
8,000
10,000
12,000
Academ
icAca
demic
Academ
icAca
demic
Academ
icAca
demic
Academ
icAca
demic
Academ
icAca
demic
Academ
icAca
demic
Academ
icAca
demic
NHSNHS
Academ
icNHSNHSNHSNHS
2 Yr Citation Count by Liaison (anonymous)2 Yr Citation Count by Liaison (anonymous)
2040
1600
1020960
830 820
610 580520
390
250195 180 175 160
85 85 30 12 5 30
500
1000
1500
2000
2500
Academ
icAca
demic
Academ
icAca
demic
Academ
icAca
demic
Academ
icAca
demic
Academ
icAca
demic
Academ
icAca
demic
Academ
icNHS
Academ
icNHS
Academ
icNHSNHSNHSNHS
H-Index by Liaison Authors (anonymous)H-Index by Liaison Authors (anonymous)
54
50
4038
34
28 2725
22
18
14 13 13 12 12 12
7 6 5 4 3
0
10
20
30
40
50
60
Academ
icAca
demic
Academ
icAca
demic
Academ
icAca
demic
Academ
icAca
demic
Academ
icAca
demic
Academ
icAca
demic
NHSAca
demic
Academ
icNHS
Academ
icNHSNHSNHSNHS
Predictors of Journal AcceptancePredictors of Journal Acceptance• 1. Submitting the paper!• 2. Proofing and high quality submission• 3. Co-authorship• 4. Choosing a low impact journal• 5. Improving and re-sending after rejection• 6. Selling in the Cover letter• 7. Speaking with the editor• 8. Getting an invited paper
Predictors of High Impact CitationPredictors of High Impact Citation• 1. Publishing in a high impact journal
Lawrence S. Online or invisible. Nature 2001;411:521http://external.nj.nec.com/~lawrence/papers/online-nature01/
Citations vs probability article is online
A comment from Rennie Drummond deputy editor JAMAA comment from Rennie Drummond deputy editor JAMA
There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive…..for a paper to end up in print.
What is the Cost of Research?What is the Cost of Research?
• CATIE (Clinical Antipsychotic Trials of Intervention Effectiveness)– National Institute of Mental Health (NIMH) clinical trial cost $67 million, with 1460
participants enrolled = $45,000 per patient for one year or $123 per day
• Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT) – The National Heart, Lung, and Blood Institute (NHLBI) funding for Fiscal Years
(FY) 1993-2004 was $83,170,059, and 42,418 participants enrolled => $2,000 per patient
• Diabetes Prevention Program (DPP) Clinical Trial – Funding: FY 1994-2002 was $176 million and 3,234 participants enrolled (45% were
minorities). => $54,000 per participant
Pubmed vs Wok vs ScopusPubmed vs Wok vs ScopusPubmed• 5,500 journals• 18 million articles
Scopus • 16,000• 36 million articles
WOK• 23,000 journals • 40 million articles