View
218
Download
1
Category
Tags:
Preview:
Citation preview
Parts of the Research Study
Title, Abstract, Methodology, Results, Discussion
Even before the Title...
Where is study published? Respected journal? Is journal in same field as the research
study? Is journal peer reviewed? Was paper revised? Is is published in a journal of like
content?
Title
First potential source of biasIt should not state any conclusionsIt should reflect the actual content as
clearly and concisely as possibleIt should be consistent with the
abstract and summary
AuthorsEach author should have participated sufficiently in the work represented by the article to take public responsibility for the content. Conception or design, analysis or
interpretation of data Drafting the article or revising it for
critically important content, or in final approval
Participation only in data collection doesn’t qualify for authorship
Authors
Persons who contributed intellectually, but whose contributions do not justify authorship may be named separately.
Authors should list credentials for carrying out research.
Conflicts of interest should be noted.55% of articles today have multiple
authors
About Authors...
Reputable?Independent from drug company?Affiliated with research institutions?No conflicts of interest or bias?If funded by Drug Company, it should
be declared as suchReader bias…-- too much weight
given for credentials, big names, etc.
Abstract
Purpose: provide a brief summary of the research to help the reader determine if the article is worth reading in its entirety.
Some are structured abstracts and some are have restrictions about number of words used.
You cannot form a critical opinion of the study’s validity without reading the whole article.
Introduction
Contains the specific problem which exists
Rationale for the study with background material (review of literature)
Identifies the purpose of study = study objective- stated as a study hypothesis.
Should not contain bias or any results.
Introduction Should Include:
Statement of the importance of anticipated results from the study
Reasons for doing drug efficacy studies: there is no other effective treatment for cond. This drug is potentially superior to other
drugs Due to low SE, this would be a better choice Cost savings Pharmaceutical properties- tolerance, safety
Purpose of Study/Study Objective
Should be explicitly stated--you shouldn’t have to infer purpose
Is/are objective(s) reasonable?Are there too many objectives to be
answered in a single study?Will results measure the study
objective, ie. Are there valid endpoint measurements?
Study Objective
Describes anticipated relationships between factors to be studied
Specific and reasonable enough to study
Define clearly and exactly what the investigators are going to do
Relevant to what the investigators would like to determine
Stated as null or alternative hypothesis
Null Hypothesis
This assumes that there is no relationship between the factors to be studied and the outcome.
Is assumed to be true until proven otherwise.
Stated as: “There is NO difference between products”, or, “Both products are equal”
Alternative Hypothesis
Assumes that there is a relationship between the factor to be studied and the outcomes.
Two types of alternative hypothesis: one tailed: indicated the direction of the
relationship between the factor to be studied and the outcome
two tailed: indicates there is a relationship between the factor and outcome but doesn’t state the direction
Examples of Hypothesis
Null: Pravastatin is equivalent to Simvastatin in terms of lowering of cholesterol.
One tailed: Pravastatin is more effective than Simvastatin in lowering cholesterol.
Two tailed: Pravastatin and Simvastatin differ in their efficacy to lower cholesterol.
Methodology
Written so study could be repeated from the investigator’s description.
Includes design, patient selection criteria, sample size, inclusion/exclusion criteria, randomization, controls, blinding, etc.
Determines internal validity of study
Study Design
Study design guides evaluation methods:RCT: methods of treatment assignments,
blinding and controlsLongitudinal: duration of the follow upCrossover study: use and details of
washout periodRetrospective: methods to avoid recall
bias should be included
Validity
Related to precision and accuracyInternal validity: adequacy of the
study as a whole. Relies on study design, bias, and
random variationExternal validity: can results be
extrapolated to other settings Relies on inclusion/exclusion criteria
Internal Validity
A study has internal validity if the following have been done properly: Study design
controls, blinding
Methods of patient selection sample size, random sampling, inclusion/exclusion
criteria, external validity
Randomization Outcomes and endpoint measurements Statistical analysis
External Validity
External validity is determined by: inclusion criteria --
Are the study participants like your patient population, ie. Elderly, diabetic, CHF, etc.
exclusion criteria --Who is not included in study, ie. Diabetics,
elderly, CHF, etc.
both criteria are used to determine if results can be extrapolated to other settings.
Homogeneous Groups
Study groups are closely related in terms of important clinical characteristics or disease attributes
The more homogenous, the easier to identify and quantify the effects which a drug exerts
Increases the internal validity of the study.
Heterogenous groups
Patients differ in one or more identifiable clinical characteristics of the disease or condition being treated.
Acceptable when there aren’t enough patients who meet some narrowly defined inclusion criteria.
Acceptable when the results of the study won’t be affected by the differences
Inclusion Criteria
Characteristics patients must have to be eligible for participation in study
Homogeneous groups preferred--easiest to identify and quantify effects and increases internal validity of study.
Heterogeneous groups okay when results won’t be affected by the differences.
Exclusion Criteria
Characteristics which prohibit the patient from participating in the study
Examples: presence of other disease states, severity of disease, other medications/therapies affecting study results, patient safety, ethics, compliance.
Exclusion criteria helps ensure the study sample is homogenous.
Patient Selection Criteria
How many patients did they have in the study?
Is this number appropriate for the study design?
Does the study population represent the population from which it is drawn?
Was random sampling truly done?
Sample Size
Determined during initial planning stages of study
Need enough subjects to allow for significant differences between treatment groups to be detected statistically.
Need to balance statistical concerns with subject availability, cost, time constraints
Sample size considerations
RCT’s with small number of subjects may not be adequate to determine long term toxicity.
Other study designs may be needed based on the study sample size.
Sample size factors
Alpha or level of significance: the probability of obtaining a false positive result -- indicated as the “p-value”.
Beta: probability of false negative result--indicated as the “power”.
Delta: amount of difference that one wants to detect between groups
variance or standard deviation needed
Sample Size Factors
Investigator sets 4 factor levels, goes to table (or program) and selects appropriate sample size.
Very rough minimum, 30 patients needed for parallel study, 15 needed for crossover
Increasing sample size beyond certain point can lead to wasteful time and money--law of diminishing returns
Random Sampling
Selection of population into the study Each member of the population has the
same opportunity to be selected into the study.
Each is selected independently of anyone else.
Non-Random Sampling techniques to beware of
Consecutive non-random sampling: accept every patient who meets study criteria until a certain number is reached.
Convenience non-random sampling: select patients from a population which is easily or readily accessible.
Systematic non-random sampling: Every nth person is selected for study inclusion
Controls
What are investigators comparing the study drug/test to? Active control Placebo control No control Historical control
Active Control
Study drug is compared to another drug
Tells only “relative” efficacy Is study drug more, less or of equal
efficacy to comparison drug
Placebo Control
An inactive medicine without pharmacological effect.
Same dosage form and routeIt will contain small amount of sugar,
lactose or other inert substance which has no therapeutic action.
Can tell “actual” efficacyMinimizes bias, controls confounders
Ligation of Mammary Artery Trials
1940’s, double blinded, placebo controlled trial (sham operation vs. actual operation)
saved lives of many high risk patients from going through risky surgery which was not effective.
No Treatment Control
Refers to a group of patients in a study who do not receive any study drug or placebo
Tells “actual” efficacy Ethical concerns arise for placebo
and no treatment control groupsSalk polio vaccine trials in 1950’s
Historical Control
Utilizes a group of patients from who data have previously been collected.
Uses: effectiveness of surgical procedures, rare diseases, oncology studies
Disadvantages: inability to determine if control group was truly comparable, esp. when disease/condition can change over time.
BlindingOpen label
Both investigator and patient know treatment
Single blind Investigator knows who is receiving which
treatment, but patients don’t know what they are receiving.
Double blind Neither investigator and patient know
treatment
Keeping the study blinded
Make placebo look like active drug“Double Dummy”-- patients take 2
drugs each-- one placebo and one study drug.
RPh often involved with studies-- we keep investigators and patients blinded.
Unblinding can occur
Randomization
Refers to: assignment of patients to a treatment
group in a parallel or time series design Assignment of the order of treatments
in a crossover designPurpose of randomization in
assignment to groups -reduces bias, keeps groups balanced
Simple Randomization
Random numbers tablePulling names out of a hat
Systematic Randomization
Selecting a treatment group in which every “nth” person is selected for a treatment group
Acceptable if the starting point for selection (random sampling) is determined properly (randomly).
Block Randomization
Useful when using small numbers of patients
Ensures equal number of patients are randomized to each treatment group.
Cluster Randomization
The population is divided into natural groupings (geographical locations) and a random sample is selected from each group. A multicenter study across the U.S. All
lpatients from SE are divided into treatment and placebo groups, all from NE are divided, etc.
Stratified Randomization
Patients are assigned to subgroups, called strata, based on important characteristics called confounding factors. Then a separate randomization schedule for each stratum is chosen.
Useful when confounding factors will have large effect, and when small sample size
Non-Random Assignment
Potential Bias increasedMay use hospital admission numbers,
phone numbers, SS#, days of the week patients joined the study, etc.
Tendency to show larger treatment effects and increase the risk of false positive results
Results are difficult to evaluate--need multivariate modeling in statistical analysis
Outcome Measurements
Do measured endpoints match objective endpoints?
Are they measured correctly?Is statistical analysis done?
By independent investigator?
Results
Clearly presented and accurately reflect the study hypothesis.
Summary of study groups all patients should be accounted for reasons for missing data explained why drop-outs occurred Is length of study appropriate for study
objective?
Patient/Subject Drop-out Drop-outs change balance of “groups”Reasons for drop-outs can impact
results: Non compliance with study protocol development of side effects lack of efficacy subject was found not to meet inclusion
criteria developed another condition which
interfered Unavailable for follow up
How to Handle Data from Drop-outs
Intent-to-treat method: all data from all patients are included in analysis, regardless of whether or not their treatment was modified in any way
Exclusion of subjects method: patients are excluded from analysis if their treatment was modified in any way.
Intent-To-Treat
Advantage: reflects normal or actual clinical practice for a drug, in which patients are often started on a drug and later have their therapy altered.
Disadvantage: If large numbers of patients drop out or have therapy altered, the true efficacy of the drug itself will be obscured
Intent to Treat measurements:
Intent to treat method takes drop out patients and measures their scores by: A. Their last score or measurement at
the time they dropped out B. The average for the entire group C. The worst score or measurement for
the group.
Exclusion of Subjects
Advantage: The true efficacy of a drug in the regimen outlined in the study can be better determined.
Disadvantage: Patients can drop out of a study for reasons that can affect the usefulness of a drug in practice-- this method will not always reflect the actual clinical usefulness of a drug
Missing DataPatient completes study but one or more of their data measurements are missing
The greater the number of variables to measure in a study, the greater chance that certain data points will be missing
Missing data points are not significant if: only a small percentage of data points are
missing missing points occur by chance rather
than by a single factor
Options for Handling Missing Data
Dropping patients with incomplete data from study
Submitting the mean of the other scores or mathematically estimating the value for the missing point
Excluding the missing points for analysis
Types of Data
Raw data: actual measurements obtained
Derived data: measurements which have had some manipulations
Summary data: results which represent the combine data for all patients
Types of DataDerived data should be accompanied
by the raw data it was prepared from to allow for interpretation.
Summary data should be accompanied by the individual data to allow you to: fully evaluate how well it represents all
the patients determine if appropriate statistical
analysis performed repeat calculation
Outcomes Reported
Data presented must be: complete clear missing data must be explained and
accounted forResults section: determine whether a
study has fulfilled its objectives and proven or disproven its hypothesis
Tables and Graphs
ClearAccurateNot misleadingSimple
Things to watch for when analyzing the data:
Graphs with skewed vertical axis or no zero point
misleading line graphsTruncated bar graphsPercentagesColumns and rows not equaling 100%Sample size inflated
Percentages
Listed as: percent cure rate percent response rate percent of patients achieving desired
outcomePercent change can be misleading
without knowing baseline value.Exact amount of change most valuable
Sample Size
Artificially inflating sample size when repeated observations of a particular parameter are made and the author considers the total number of observations, and not the number of patients to be the sample size.
Discussion/Summary
Form your own conclusions before reading the Discussion/Summary
Watch out for persuasive languageWatch out for downplay of conflicting
evidenceStudy objectives have to be
consistent with results
What to watch out for in the Discussion/Summary
Author bias, reader bias Investigator interpretation of percentage
change or degree of change relative to control.
Biased citation or related publications Cause and effect relationship Errors in explaining a “non-significant p-
value” Statistical significance vs. clinical significance Quality of life vs. death as endpoints Inappropriate conclusions
Recommended