Turning a Good Idea Into a Great Research Project:

How to get StartedJames R Miner MDResearch DirectorDepartment of Emergency MedicineHennepin County Medical Center

John H. Burton, MDChairDepartment of Emergency MedicineCarillion Clinic

Its easy to think of good research questions in Emergency Medicine

Ideas: Research Questions

Usually emerge based on the findings of previous work

“Although a fresh perspective can sometimes be useful by allowing a creative person to conceive new approaches to old problems, lack of experience is largely an impediment” Steven Hulley MD

Origins of the Research Question

Scholarship is an important part of good research

Research that does not recognize previous work in the area that applies to is very likely to be poorly done

The mastery of a subject also entails participating in meetings and building relationships with experts in the field

Mastering the literature

Have a skeptical attitude about prevailing beliefs

Remember that the application of new technologies often generates new insights and questions about familiar clinical problems

Attend conferences and listen to things you don’t agree with

Being Alert to New Ideas

Carefully Observe your patients Teach Be Creative in your approach to old

questions Have Tenacity: Don’t give up on things

that “can’t be done”

Keep the Imagination Roaming

Everyone in Academic EM can claim slides #4-6

FINER◦ Feasible◦ Interesting◦ Novel◦ Ethical◦ Relevant

Characteristics of a Good Research Question

Number of subjects Technical expertise Available time and money Manageable in scope

Feasible

To the investigator◦ Is this a logical and important step in building a

career or in getting to the truth of an important scientific matter?

To everyone else◦ Will anyone care to read this if its written up?

Interesting

Can be determined by reviewing the literature and the CRISP database

Confirmatory studies are only necessary◦ To determine if a new methodology can be

reproduced◦ to measure external validity in a different setting◦ to avoid weaknesses noted in previous studies

NOVEL

This is a whole other lecture, but usually you don’t need a degree in ethics to tell

Ethical

Most important aspect How will the various possible outcomes of a

study advance knowledge, guide further research, or change clinical practice

Relevant

Idea Summary

1. Start with a good idea2. Ask if anyone else cares and ‘who will publish it?’3. Research the idea exhaustively4. Say ‘NO’ to unpublishable ideas

Be Warren Buffett:Choose Your

Investments/Ideas Wisely

Methods

The question is not feasible◦ Too broad

Narrow the question◦ Not enough Subjects available

Expand inclusion criteria Decrease the sample size you need

◦ Methods beyond the skills of the investigator Collaborate Review the literature Learn the skills

◦ Too expensive Decrease sample size and simplify measurements

Developing the Research Question and Study Plan

Not Novel◦ Modify the research question

Not Ethical◦ Modify the research question

Not relevant◦ You can’t fix this

Too Vague◦ Write the research question first◦ Stick to the question

Developing the Research Question

Many studies have more than one question One question needs to drive the study

◦ Multiple questions don’t work for statistics as they are designed today

◦ If you ask 20 random questions with random data and use the same test on all of them, one will be statistically significant

Primary and Secondary Questions

The 4 elements of Well-built clinical Questions (PICO)◦ Patient or problem◦ Intervention (independent variable)◦ Comparison (measurement device)◦ Outcome (dependent variable)

What is the Question?

What is the best treatment for acute asthma exacerbations?

A research question

Is continuous albuterol better than intermittent albuterol for the treatment of acute asthma exacerbations?

What is the Study?

Is continuous albuterol better than intermittent albuterol for the treatment of adult ED patients with acute asthma exacerbations?

Who are the study subjects

Does continuous nebulized albuterol (10 mg/hr) results in better outcomes than intermittent nebulized albuterol (2.5 mg q 30 min) for the treatment of ED adult patients (18 – 40yrs.) with acute asthma exacerbation

What is the intervention?

Does continuous nebulized albuterol (10 mg/hr) result in better outcomes, in terms of increased PEFR and admission rates at four hours, than intermittent nebulized albuterol (2.5mg q 30 min) for the treatment of ED adult patients (18 – 40 yrs) with an acute asthma exacerbation of less than 8 hours and presenting with a PEFR of <200, after one 2.5 mg albuterol treatment?

What is the comparison and outcome?

Cheaper is better! It will take years to establish funding:

◦ Department Discretionary Funds◦ Hospital/University Internal Grants/Funds◦ Local Organizations/Foundations◦ National Foundations◦ Government/NIH

Funding

Don’t be an Emergency Physician…,

FOCUS

Methods Summary

1. Hypothesis, hypothesis, hypothesis2. Don’t study things you can’t study3. Smaller/Tighter is better

Focus

Research Design

Know the basic types Know the pros and cons of each Look to the literature for good examples of

each Use reference texts

Research Design

Research Question Study Plan Actual Study

Truth in the Universe

Truth in the Study

Findings in the Study

Design Implementation

inferenceinference

errors

Errors in research design cause errors in the application of information from the study to reality (inference)

Research Design

Appropriate Designs yield information that can be inferred to clinical practice

Inappropriate designs yield data

Conclusion

Design SummaryYou are a person of many great ideas…

Unfortunately, it’s not practical for all your great ideas to get published….

Maybe ONE every 2 years?

Hail Mary Passes are Rarely Caught

Study Execution

Clinical research is always a form of a “natural” experiment as it involves trying to make measurements in the “real world”

Clinical research is always imperfect and limited by the lack of full control over study subjects

The ED environment is particularly difficult and presents unique challenges

Practical Aspects of EM Research

Lack of control over patients how present Lack of control over patient care resources Less time for staff to correctly apply I/X

criteria Usually less staff knowledge about the

specific study subject area

Unique Aspects of EM Research

Conception is easier than birth The “birth” is often problematic if not planned There is usually a honeymoon period when the

project has the affection of the staff Enthusiasm quickly wanes After 6 months no one remembers its name 2 years later the data forms languish in a drawer

The Natural History of a Research Project

Question should be interesting and relevant to the staff

Modify the question until its feasible◦ Realistic numbers of patients to answer the

question◦ Adequate resources and time to complete

Don’t try to get more data than is necessary to answer the study question◦ When you ask for more you often get less

The Study Conception Process

Field test the project before it is implemented

Don’t start the study until its ready (call it a pilot study first)

Get most of the work done at the beginning Remind everyone frequently Keep the machinery working Embrace Failure

Recommendations

Investigators lead by example Modify the protocol as needed Eliminate data or steps that are not working

as planned Send out notices when the study has ended Publish the results – that’s the next lecture Its easier to repeat a project with

improvements than publish a bad study

Recommendations

Collaborate with others who have similar interests

Study Execution Summary1. Try to push your

study DOWNHILL2. Capitalize on success 3. Keep the momentum going4. Admit failure

If a senior investigator yells at you when you present your research you’re onto something good

Its amazing what you can get done when you don’t worry about whose going to get the credit for it

You’re a CLINICAL researcher, don’t let funding stand in your way

Final Thoughts

Thank You CORD!