Designing Clinical Research

Anita S. Kablinger, M.D.

Associate Professor

Departments of Psychiatry

and Psychopharmacology

Chapter 2: Conceiving the Research Question

Research question:

•Addresses uncertainty

•Resolved by measurements

Successful Research Questions Require:

scholarship, experience, mentor

• Mastering the Literature

-Published literature

-Attend meetings

-Relationships with professionals • Knowledge of New Ideas and Techniques

• Imagination

FINER Research Questions

• Feasible Number of subjects, technical expertise, affordable in time and

money, manageable

• Interesting to the investigator • Novel Confirms/ refutes/ extends previous findings, provides new findings

• Ethical No unacceptable physical risks or invasion of privacy

• Relevant Scientific knowledge, clinical and health policy, future research

Early Stage Outline

• Researcher clarifies plan

• Researcher discovers specific problems

-Is research question finer ?

• Allows colleagues’ reactions

Chapter 3: Choosing the Study Subjects

• Study sample must be affordable in

time and money

• Study sample must represent the

target population

Inclusion Criteria

• Includes main characteristics of target population

• What factors are important to the research question?

• How should these factors be defined?

• Include demographic, clinical, geographic, and temporal characteristics

Exclusion Criteria

• Characteristics that may interfere with the data or randomization of the study

• Excessive exclusion may degeneralize the study

Clinical vs. Community Populations

• Clinical are often already hospitalized so study subjects are inexpensive, available, and easy to recruit

BUT

• Their condition may be severe or extraordinary, so the data may be distorted and not representative of the population

Convenience Samples• Subjects meet entry criteria

• Subjects are accessible

• Samples often selected consecutively

• Minimizes voluntarism – Volunteers are often healthier than the

general population, so data may be distorted

Probability Samples• Guarantee that every member in the

population has a specific chance of selection for the study

• Types:

-Simple random

-Stratified random

-Cluster

-Systematic

2 Goals in Recruiting Study Subjects

• Representative

Response rate affects validity of study

Non-response subjects are often different than those that respond – Disease may be the cause of non-response

• Size

Monitor recruitment progress

Note when/ why potential subjects are lost

Chapter 7Designing an Observational

Study: Cohort Studies

• Cohort studies take place over time

• Descriptive vs. Analytic

• Prospective vs. Retrospective

Prospective Cohort Studies

• Strengths – Measures predictor before outcome occurs (Less bias, more accurate)

• Weaknesses – Expensive and inefficient for studying rare

outcomes– Conditions present with symptoms before

diagnosis

Retrospective Cohort Studies

• Strengths – Measures predictor before outcome occurs

(Less bias, more accurate)– Less costly and time-consuming than

perspective studies

• Weaknesses – Investigator has limited control over sampling

design or data collection

Nested Case-Control Study

• Used for predictor variables that are expensive to measure and can be assessed at end of study

• Subjects from completed cohort study -Cases: Part of cohort that developed outcome -Controls: Part of cohort without outcome *Matching is optional but an unmatched design is preferable

Nested Case-Cohort Studies

• Random sample selected from original cohort regardless of outcomes

• Advantages:

-Same control group for different studies

-Information on risk factor prevalence

Nested Studies• Strengths

– Costly measurements are available – Variables are collected before outcome– Reduces bias from fatal cases and use of

various populations

• Weaknesses – Observed association from confounded

variable– Silent preclinical disease

Multiple Cohort Studies• 2 different subject samples based on level

of predictor variable• May compare cohort study outcomes with

census data

• Strengths – Feasible approach for studying rare exposure or

potential hazards

• Weaknesses – Cohorts may differ– Data may be imprecise, incomplete, nonexistent

**Following the Entire Cohort

• Exclude those planning to move/ difficult to reach

• Obtain information for difficult follow up

-Physician, close friends, SSN, Medicare #

• Periodic contact and repeated follow-up efforts

Chapter 8: Designing an Observational Study: Cross-sectional and

Case-control Studies

• Cross-sectional

• Case-control

• Bias

Cross Sectional Study

• Simultaneous measurements

• No follow up

• Cause and effect inferred

• Predictor and outcome designated

Case Control Study

• One sample from cases

(With outcome)

• One sample from controls

(No outcome)

• Compares levels of predictor in cases vs. controls

Prevalence vs. Incidence

• Cross sectional studies provide prevalence (at given time) – Relative prevalence: Outcome prevalence by

level of predictor

• Cohort studies provide incidence – Over period of time

Cross Sectional Studies

• Strengths -Fast and inexpensive

-No loss from follow-up

• Weaknesses -Difficult to determine cause and effect

Serial Survey

• Series of cross sectional studies

• Inferences can be drawn but a single group is not followed over time

• May be used in cohort study to prevent learning effect on data

Case-Control Studies

• Used to compare risk factor prevalence– How often do predictor variables lead to disease?

• Cases (with disease) vs. controls (without)– Specified number of each, so incidence or

prevalence of disease not determined

• Retrospective so may contain bias

Case-Control Studies

• Rare diseases

• Diseases with long latency

• Retrospective so can look at various predictor variables to determine cause of new outbreak

Case-Control Study Bias

• Control and case separate sampling

-Sampling bias

• Retrospective predictor variable measurements

-Differential measurement bias

Sampling Bias

Not all affected are available for study -Undiagnosed, misdiagnosed, dead

SOLUTIONS:1. Hospital- or clinic-based controls2. Matching3. Population-based sample4. Two+ control groups

Differential Measurement Bias

Imperfect recall

SOLUTIONS:

1. Data recorded before outcome

2. Blinding

Chapter 10 Designing an Experiment:

Clinical Trials I

• Treatment applied• Effect on outcome observed• Causality demonstrated• Mature research questions

Selecting Participants• Entry criteria:

Rate of outcome

Treatment effectiveness

Recruitable

Follow up

Generalizability

Compliance

• Sample size and recruitment planning

Measuring Baseline Variables

• Fewer variables may be more efficient

• Spend time and money wisely

• Tracking information

• Participant description

• Measure risk factor and subgroup defining variables

• Material banks

• Initial outcome variable

Randomizing random assignment to interventions

• Treatment assignment must be random

• Blocked

• Stratified blocked– Guarantees even distribution of strong predictor in

small sample

Applying Intervention

• Blinding -Co-intervention effects

-Biased outcome assessment

• Choosing the intervention-Effectiveness, safety, blindness, generalizability, combinations

• Choosing the control-The control treatment should mimic the active treatment

-Ethical co-intervention

-Equivalence trials

Chapter 11 Designing an Experiment:

Clinical Trials II• Maximize follow up and adherence• Measuring outcome• Analyzing results• Monitoring clinical trials • Vulnerable populations• Research misconduct

Maximize Follow up and Adherence

• Tolerable drug

• Daily dosage

• Pill counts and drug screening

• Easy, convenient, and interesting studies (“participant friendly”)

Clinical Trials

• Clinical relevance should be balanced with feasibility and cost

• Measurable variables: Outcome (Clinical) Risk of outcome (Surrogate)

Measuring Outcome

• Outcomes should be accurate and precise

• Continuous variables preferred over dichotomous

• Include outcome measures to detect adverse effects

(See FDA website “Good Clinical Practices”)

Analyzing Results

• Dichotomous: chi squared

• Continuous: t test

• Intention to treat– Cross-overs covered, prevents bias, may

underestimate results (results are conservative)

– Preferred over per protocol

• Per protocol analyzes only evaluable – May differ from those that drop

Subgroup Analysis• Comparing randomized subjects of trials

• Randomization measurements should be determined before treatment

• Post randomization factors should not be considered

• Subgroup size is problematic – May be too small to detect differences

• Different findings among subgroups

Monitoring Clinical Trials• Prevent harmful intervention

– Risks are greater than benefits

• Provide beneficial intervention• Discontinue intervention when research question

becomes unanswerable• Before study begins, create guidelines and

procedures for monitoring• Monitor recruitment, adherence, randomization,

blinding, follow up, outcome, adverse effects, potential confounders

• Who will monitor and how often• Statistical monitoring methods

Committee Monitoring

• Intervention and continuation decisions should balance ethical responsibility with advancement of medical knowledge

• Committee should be composed of physicians, participant advocates, biostatisticians, and clinical trial experts with no connection to study

Options to Randomized Blinded Trials

• Factorial Design

• Randomization of Matched Pairs

• Group or Cluster Randomization

Factorial Design

• Answers two separate research questions

• Efficient and cost effective

• Interaction b/t treatments and outcomes

• Useful for studying two unrelated questions

Randomization of Matched Pairs

• Subject pairs with matching factors

(ex. age, sex)• Contrast treatment and control in two parts of

the same individual at same time

(ex. one eye is treated, other serves as control)

Group or Cluster Randomization

• Assigning by group (ex. family, sports team)

• Answers questions about public health programs

• More feasible and cost effective than individual assignments

• Complicated analysis

Within Group Designs

• May be used in time-series studies• Time series studies: Participants are their

own control for evaluating treatment effects– Disadvantage: Lack of concurrent control

group– Efficacy is optimistic b/c learning effects,

regression to the mean, secular trends– Repeatedly starting and stopping therapy to

establish treatment effects

Cross-over Design

• Each person is their own control so requires fewer participants

• Due to carry over effects: Study duration doubled Complex analysis and interpretation

– Carry over effects: Influence of intervention outcome after treatment has stopped

– Use washout period to diminish carry over effect

• Use when limited number of subjects and unproblematic carry over effects

FDA Approval• “Good Clinical Practices” on FDA website • Preclinical involves animals• Phase I: Unblended and uncontrolled with small

number of human volunteers• Phase II: Small random blind trials of diff doses• Phase III: Large trials to test hypothesis• Phase IV: Large studies after drug approval

Study Implementation

Anita S. Kablinger MDAssociate Professor

Departments of Psychiatry and Pharmacology

Outline of Presentation

• Addressing Ethical Issues

• Designing Questionnaires and Data Collection Instruments

• Data Management

• Implementing the Study

• Community and International Studies

• Writing and Funding a Research Proposal

Chapter 14: Addressing Ethical Issues

• Ethical Principles• IRB • Informed Consent • Scientific Misconduct• Conflict of Interest• Authorship• Confidentiality

Ethical Principles

• Respect for persons- Informed consent, confidentiality

• Beneficence- Sound research design, risks are acceptable in comparison to benefits

• Justice- Vulnerable populations should not be targeted, equitable access to the benefits of research

Federal Regulations Ensure Protection of Subjects (OHRP)

Institutional Review Board Approval• Minimal risks

• Reasonable risks when compared to possible benefits

• Equitable participant selection

• Informed consent

• Confidentiality

IRB• Lack research expertise

– Research design, scientific merit, protocol adherence

• Investigator must uphold ethical research

• IRB reviews include: – Exemptions: surveys or observations, existing

records, normal educational practices– Expedited: minimal risk, minor changes in

approved research

Informed Consent• Nature of research project• Study procedures• Risks, benefits, and alternatives Including medical, psychosocial, economic realms

• Confidentiality• Voluntary participation assurance• Comprehensible consent forms• Subjects who lack decision-making capacity

Vulnerable Populations

• Avoid vulnerable populations unless problem under study is especially prevalent within that specific population

• Vulnerable populations should not be used unless necessary– These populations involve additional informed

consent requirements– Children, prisoners, pregnant women, fetuses,

and embryos, people with impaired decision-making capability

Research Misconduct

• Includes fabrication, falsification, plagiarism

• May lead to incorrect results

• Undermines public confidence and support

• Investigators should have access to all data, statistical analysis, and publishing rights– Sponsor should not veto or censor publication

• Conflicts of interest

Conflicts of Interest

• Blinded studies and peer review prevent bias in conflicts of interest

• Any conflicting interests should be disclosed and situations leading to conflicts of interest should be avoided

• Dual roles for clinician-investigator and financial conflict of interests may impair objectivity of study and undermine public trust

Dual Roles • Physicians should always uphold patient

welfare

• Data and safety monitoring boards that make study termination decisions should not include researchers

• Randomized blinded studies without controls should have equitable protocols for both groups in the study

• Principle of Nonmaleficence: Effective therapies should be provided to controls

Authorship

• Criteria for authorship to avoid questionable contribution and responsibility – Substantial contributions– Drafting or revising the article– Give final approval of article

• Honorary and ghost authorships

Chapter 15: Designing Questionnaires and Data Collection

Instruments

• Questionnaires• Interviews

Questionnaire Format• Questionnaires should include purpose of

study and how data will be used

• Instructions on answering should be provided– Include instructions throughout where format changes

• Wording: Clear, simple, neutralVisual analog scales

• Branching scales

Questionnaire Questions• Questions should be mutually exclusive

• Avoid assumptions in questions

• Questions should cover one concept (avoid “and,” “or”)

• Include “other” or “none of the above” if questions are not exhaustive

• Group questions from same subject groups together and include heading

More Questions• Use series of questions with scores to measure

abstract concepts• In questions with lists, use “yes” or “no” so each

option requires a response• Open-ended questions: Better for determining

concept understanding but qualitative analysis is time consuming and subjective

• Close ended questions: More standardized, quicker and easier to analyze but may influence participants’ answers

Questionnaires: Additional Information

• Emotionally neutral questions at beginning, sensitive ones in middle, personal at end – Use introductions for questions about undesirable

behavior

• Set the time frame and quantify responses – Determine if average or extremes in behavior is more

important for study– Recent brief time frames increase recall but

responses may not be typical– Longer time frames make recall difficult and bias

toward recent behavior may result

Questionnaire Editing• Check for internal consistency to

guarantee acceptable summing or averaging of scores

1) Draft made from focus group interviews

2) Critical review of draft

3) Pretest new questions or questionnaires

Developing Instruments for Study

• List variables to be collected and measured in study• Existing measures for each variable• Best to not modify unless necessary

– Changes prevent comparison to previous studies with same instrument

• Make draft• Revise to simplify and clarify

– Check for length and ambiguity– Shorten to exclude any unnecessary variables

(Think ahead to data analysis and presentation)• Pretest and validate

– Face validity, gold standard, predictive validity

Questionnaires vs. Interviews

• Questionnaires: – Efficient for simple questions, less expensive and time

consuming for research staff, standardized– Less likely to be completed by people of limited literacy or lower

education so results may be biased

• Interviews: – Complicated questions, more costly and time-consuming, less

uniform, answers may be influenced by interviewer or participant/ interviewer relationship

– Should be as standardized as possible• Wording, nonverbal signals, tone of voice, probing

• Both are subject to imperfect memory• Both are affected by tendency to give socially acceptable

answers

Chapter 16: Steps in Data Management

• Define each variable• Set up the study database • Test data management procedures before

the study begins• Enter the data-identify and correct errors• Back up the dataset regularly• Create a dataset for analysis• Archive and store database and results

Chapter 17: Implementing the Study

• Pretesting (pilot studies)

• Less stimulating than design or analysis

• Either can lead to a change in protocol once study begins

• Quality control measures: train the team, certify the team, regular meetings, performance review

Chapter 18: Community Studies

• Research outside the academic setting designed to meet the needs of the community– Answers questions of local or regional

importance– May be more generalizable– Can help local economy and self-sufficiency

Chapter 19: Writing and Funding a Research Proposal

• Protocol: Detailed written plan of study• Proposal: Written document to receive funding

– Contains the protocol, the budget and other administrative and supporting documentation

Proposal Writing Guidelines

• Decide where proposal will be sent• Follow specific guidelines, requirements, process of each

agency• Find model proposal specific to agency of interest • Written criticisms of successful and unsuccessful

proposals submitted to agency • Contact scientific administrators to review proposal draft • Organize team, assign leader

– Organization and delegation– Authority/ accountability chart including all members of research

team• Timetable and periodic meetings

– Research plan and timetable– Work from outline

• Review, pretest, and revise repeatedly

Proposal Elements• Title, abstract, table of contents• Budget with budget justification, biosketches of

investigators, resources– Rebudgeting usually acceptable, increased funding may not be

• Specific aims, significance, preliminary studies, and previous work of investigators– What has been accomplished, what are the problems, what needs to be

done in the field under study

• Methods, statistical section, timetable– Special attention to methods because they will be scrutinized

• Ethical issues, consultants, references and appendices

Scientific Methods• Give special attention to this section

• Serves as manual for future studies

• Should contain table of contents

• Should contain general overview – Diagram may be helpful

• Includes subjects and measurements

• Includes pretest plans, data management, quality control

Methods Sections

• Overview of design (time frame and nature of control)

• Study subjects (selection criteria, recruitment plans)

• Measurements (intervention, outcome variables)• Pretest plans• Statistical issues• Quality control• Timetable and organizational chart

More Scientific MethodsStatistical Section

• Analysis plans

• Null hypothesis, statistical test, sample size, power estimate

• Involve statistician in writing or editing this section

Human Subjects Section• Address ethical issues• Include children, women, minorities involved in study• Risk and benefit presentation• Obtaining informed consent• Use and value of each consultant, with letter of consent

and biosketch• Arrangements with collaborating institutes with letters of

agreement addressed to investigator• References• Appendices as needed for technical and supporting

material

Funded Research• Government (NIH)

– Funding is general and determined by peer review– Summary statement of criticisms and comments– Resubmission should begin with quotes from summary

statement with corresponding responses and proposal changes• Private nonprofit (Foundations)

– Funding is specific and determined by executive process• Private for profit (Drug companies, Med equipment suppliers)

– Establish right to publish• Intramural resources

– Local research funds from the universities for their own researchers

Acknowledgments

• Hulley, Stephen, et. al. Designing Clinical Research, 2ND ed. Lippincott Williams & Wilkins. Philadelphia. 2001