Clinical trials and pitfalls in planning a

research project

Dr. D. W. GreenConsultant AnaesthetistKing's College Hospital

Denmark HillLondon SE5 9RS

with grateful thanks to Professor Alan Aitkenhead

• insufficient information

• poor research

• inadequate sample sizeno power analysis

no confidence intervals

• biased

• confounding factors e.g. mixed sexes for PONV

• vague end points e.g. not clearly defined severity of pain

• straying from hypothesis

Seven deadly scientific sins

Laboratory structure/activity analysisAnimal does it work in animals ?

is it toxic ?

Human volunteers

• Phase 1 .... Is it toxic ?• Phase 2 .... Does it work ?• Phase 3 .... Does it work better

than existing drugs ? • Phase 4 .... Post marketing surveillance

What's it like in the real world ?

New Drugs: Types of study

Background

• has it been done before?

• is it worth doing?

clinical scientific essential step

• has anything similar been done before?

• methods used by others?

Protocol

Introduction

• background information

• justification

why, what gap will it fill, what benefits

• succinct

don’t miss out relevant info

Methodology: Ethics and consent

• Crucial

• Declaration of Helsinki

• benefit to patients

• benefit to society

• Information to patientspurpose, what it involvespotential benefits, ability to

withdrawrisks and disadvantages

without prejudice

• children and incompetent adults

Selection of patients

• Age

•efficacy and current disease

• ASA status

• Sex

• pharmacokinetics, dynamics• e.g. PONV

• Type of surgery

•applicability and availability

• Ability to give consent e.g. ICU

• Pregnancy

Designs

• prospective vs retrospective• open vs blind (double or single)• randomisation

acceptable methods eg envelopes opened after entering the trial• use of placebo

ethics and other treatments• block design

blocks of patients: analyse after each block to enable

one to stop when results are available• stratification• sequential analysis

Pitfalls

• Funding salaries drugs, equipment and investigations e.g. NHS costs

• statistics and data collection design

• time …. how long do we go on for?

• negative result … do (should) we publish?

• contradictory results vs other studies

• statistical and clinical effects

• rival investigators

Assessment and measurements

• which techniquesvalidity, accuracy, objective, analysis

• which observer

blinded, nurses, how many make measurement, are they trained

• how oftenscience, statistics, practicality over

long periods, placebo effect of frequent assessments • number of variables, fewer the better • availability of test e.g. troponin T

Documentation

• Ethics committee approval

• patient information

• data collection formsdata type, storage, security,

confidentiality, safety

• consent forms

Disproving the null hypothesis

• The ‘null’ hypothesis is that there is no difference between the treatments

• a probability value ‘p’ tells you how often the difference between the treatments could have occurred by chance.

• p < 0.05 is 1 in 20 or less (statistically significant)

• p < 0.01 is 1 in 100 or less (highly statistically significant)

Disproving the null hypothesis

• Type I error is where a difference is shown which could have occurred by chance

• 1 in 20 trials will show a difference where none exists if ‘p’ is reported at the 0.05 level

• multiple subgroup analysis in a trial may also give subgroup treatment differences

• a statistically significant result is more likely to be reported!

Disproving the null hypothesis

• Type II error is showing no difference where one actually exists

• almost always due to insufficient numbers

• can mask beneficial treatment effects

• BUT! if trial is large enough it may produce a statistically significant effect where the clinical significance is marginal

Size of study

• Power of study to show a difference in Rx

(e.g. 70% chance of demonstrating a 15% difference with a p < 0.05))

• able disprove the null hypotheses with minimal or no Type II error

• may require pilot to determine treatment differences

• requires large numbers if differences are small or if great variability in treatment outcomes

• lower power (smaller numbers) may be acceptable if outcome is important (e.g. leukaemia)

Assessment of population size

15% of patients die within one year of admission to hospital for suspected myocardial infarction. Preventing 1/3rd of these deaths would be a major advance. Roughly, how many patients are needed for a clinical trial if doctors want to be 90% sure that a difference between treatments as large as the prevention of 1/3rd of deaths will not be missed at the p < 0.05 level?

Presentation of results

• Significance: clinical versus statistical

• p values

• confidence intervals (95%) (+/- 2 SE)

• risk reduction (relative and absolute)

• numbers needed to treat

• odds ratios

Measures of risk reduction

• Relative risk reduction …. Is it meaningful?

• Headline “50% reduction in mortality”– if normal mortality is 50/100 this is great (25)– if normal mortality is 1/100 … (1 in 200)

• Number needed to treat is better measure– reciprocal of risk reduction e.g. 4 in first (25/100)– 200 in the second (0.5/100)

• If cost of treatment is £10,000 ………. !!

Number needed to treat

• Control event rate is 9 cases in 30 (0.3)• Experimental event rate is 1 case in 29 (0.033)

Then, NNT = 1/(CER - EER)

= 1/(0.3-0.033)

= 4• This method corrects for relative and absolute

risk by relating to the control event rate

Number needed to treat

• Diabetic neuropathy 6.5 year prospective trial– 9.6% developed DN (conventional)– 2.8% developed DN (intensive treatment)

• Relative risk reduction = (9.6-2.8)/9.6 = 71%

• Absolute risk reduction = 9.6-2.8 = 6.8%

• Number needed to treat = 1/.068 = 15 people for 6.5 years to prevent one case of DN

Odds ratios

• OR are used where it is difficult to calculate the relative risk e.g. case control studies

• A value greater than 1 assumes increased risk

• Confidence intervals (95%) will give the overall picture (e.g. if CI crosses 1 then the result may not be significant

Odds ratio calculation

• Calculated as the ratio of the results of the control group divided by the experimental group

• (9/21) divided by (1/29) = 0.08

• The relationship between OR and NNT is not linear and is very confusing … even to statisticians!

Evidence based medicine

The process of systematically

finding, appraising and using

contemporaneous research findings

as a basis for clinical decisions

Evidence based medicine

• Accurate identification of the clinical question to be investigated

• a search of the literature to select relevant articles

• evaluation of the evidence

• implementation of the findings into clinical practise