Research and Publication: The light and the dark side

Simon Finfer sfinfer@georgeinstitute.org.au

Royal North Shore Hospital

@icuresearch

Disclosures

http://www.nejm.org/doi/suppl/10.1056/NEJMra1208705/suppl_file/nejmra1208705_disclosures.pdf

or see @icuresearch

What will I talk about? •  Research

–  Why do it? Gattinoni vs. Finfer –  Testable vs. untestable hypotheses –  “Industrial plant” –  How to do it well – collaborators and industry –  How to mess it up

•  Publication –  What do journals want –  Is peer review a good idea –  Could I make it all up and get away with it? –  Things you won’t believe (or at least shouldn’t happen) –  Don’t take “No” for an answer –  Protect yourself –  What’s next …………… ?

Eminence based medicine

Evidence based medicine

Charisma based medicine?

Am I qualified? •  Research

–  Lead SAFE, NICE SUGAR, –  MC – RENAL, CHEST, ADRENAL ETC. –  Grant reviews – NHMRC, NZHRC, CIHR, NIHR (UK),

MRC (UK), ICF, etc –  SC – PROWESS SHOCK, ACCESS, EUROTHERM

•  Publication –  Only 120 papers but….. –  20% (24) in NEJM, JAMA, Lancet or BMJ –  Editor OTCC, section editor OTM, Guest Editor NEJM

•  What can’t I tell you? –  I can’t tell you that …………

Why do we do research? •  We do it for

– Power

– Money

– SEX

But seriously…….

Why do we do research?

– To understand

– To change behaviour – To improve patients’

outcomes

•  How much certainty do we want? •  Few treatments produce dramatic effects

•  Most claims for large benefits prove evanescent

Strict control of random error •  large numbers of deaths or other relevant outcome And Bias •  Strict randomisation with allocation concealment •  Appropriate analysis, with no unduly data-dependent

emphasis on specific parts of the overall evidence

TBI Research

13 13

IIT-TC – Eligible but not included

Eligible but not included (16 Trials - 5320 participants) Other  IIT  trials  (n=16)   Country   Sample  size  UNABLE  provide  data  (n=4)          Ma  2008   China   510  Yu  2005   China   54  He  2007   China   188  Fernandez  2005   USA   207  CollaboraEon  denied  (n=2)          Yang  2009   China   240  Yan  2009   China   356  Withdrew  (n=2)          Van  den  Berghe  2001   Belgium   1,548  Van  den  Berghe  2006   Belgium   1,200  No  response  (n=8)          Grey  2004   USA   61  He  2008   China   122  Taslimi  2009   Iran   129  Lacherade  2008-­‐2010   France   440  Benito  2008   Spain   28  De  Azevedo  2007   Brazil   48  Stecher  2006   Slovenia   n/a  Desai  2012   USA   189  

So, how to do it well …….

Build a team and collaborate… even with industry

“can’t be done ……” “never been done ……..”

“pick a patient group – think small ……” “can’t be blinded…….”

The University of Sydney Faculty of Medicine

Royal North Shore Hospital Northern Sydney Area Health Service

and

Institute for International Health

Study Treatments: 4% albumin (ALBUMEX®) or saline Manufactured by CSL Limited, Melbourne, Australia.

Study fluid distribution and tracking

SAFE – Belfast 2005

Study Design •  Packaging and distribution

Data management

SAFE Study: Actual v Target Recruitment

0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

8,000

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69

Week of Study

Num

ber

Rec

ruite

d

Actual Recruitment Target Recruitment

It’s your data, don’t compromise!

V

How to mess it up ……..

How to mess it up? •  Launch into it unprepared….

–  Without knowing incidence, outcome, realistic effect or minimal clinically important effect

•  Choose an untestable hypothesis –  Dopamine / Frusemide to reverse early AKI

•  Work with people seeking to prove something –  “we are not interested in funding you because your

proposed study is not designed to show our product in a favourable light”

–  “can we arrange for these two hospitals to be randomised to the same group?”

–  “can we analyze the data in all three ways and publish the analysis that shows our product is better?”

•  Do your sample size calculation backwards …….

Who is behind it?

Who is behind it??

Who is behind it??

Who is behind it??

Who is behind it??

Research…….. Converting b*llsh*t into FF points.

Writing and publication ……..

What do journals want?

•  Advertising revenue •  Advertising revenue •  Advertising revenue …..

What do journals want?

•  Revenue from reprints •  Revenue from reprints •  Revenue from reprints

What do journals want?

•  High quality papers or •  Papers that will be bought in the thousands by

company marketing departments or •  Papers that will be widely cited because they will

generate ……

•  Advertising revenue …….. •  And publication fees

Is peer review a good thing?

Is peer review a good thing?

Reviewer 1 – the results appear to be robust Reviewer 2 – this is a very important study Reviewer 3 – the authors should be commended on a well thought out and executed clinical trial Reviewer 4 – effects of HES on renal function is a never ending story Reviewer 5 – “this is not a randomised trial, there was no allocation concealment, the study was not blinded, I doubt one can draw meaningful inferences from these data with any acceptable degree of certainty.”

Reviewer 3

Reviewer 3

Too many journals?

•  Editor - “We can’t ask them to change it all …..

•  Finfer - “Well, it’s the wrong design, the wrong outcome, the wrong analysis. It’s meaningless. You can’t publish it.”

•  Editor – “It’s better than 90% of the stuff I’m sent, if I don’t publish this I may as well close the journal down”

•  Finfer – “Sounds like a good idea”

SURPRISE!!

SURPRISE!!

Some good news

Cochrane albumin Review: Reactions from industry

Manufacturers allocated $2.2 million: •  To prepare literature reviews supporting the use of

albumin, to be sent to leading regulatory agencies

•  To prepare and disseminate a Cochrane critique dossier

•  To establish a medical advisory panel to write articles supporting the use of albumin

Who is behind it??

§  Issues?

§  Unblinded

§  Marked baseline imbalance

§  Insufficient data re resuscitation and renal function

§  Stopped for futility but reports “positive” findings

§  Changing P values

§  Senior industry (HES) employee on trial advisory board

What’s next?

Goodness

My view of the world ….