View
312
Download
2
Category
Preview:
Citation preview
Dr Samira Alsenany
Dr SA 2012
Purpose of this seminar The purpose of this seminar is to provide
a brief overview of the critical appraisal process.
A critical review must identify the strengths and weaknesses in a piece of research and this should be carried out in a systematic manner (Eachus, 2003)
Dr SA 2012
Critical Appraisal“Critical appraisal is the process of
systematically examining research evidence to assess its validity, results, and relevance before using it to inform a decision” (Hill and Spittlehouse, 2001, p.1).
Dr SA 2012
Critical Appraisal Critical appraisal is an essential step in the
process of putting research into practice. Asking questions about an article’s research methodology, scrutinizing its data collection and analysis methods, and evaluating how its findings are presented will help you to determine whether that article’s conclusions should influence practical decision-making.
Dr SA 2012
When should you Critically Appraise?
• Conducting literature reviews for grant proposals.
• Evaluating the effectiveness, costs, and benefits of health programs.
• Establishing new health programs.
• Implementing policies; and Public health decision making, especially at the senior management level.
Dr SA 2012
What is criticalappraisal?
Critical appraisal is an essential part of evidence-based.
Critical appraisal allows us to make sense of research evidence and thus begins to close the gap between research and practice.
The most appropriate design for studying the effectiveness of a specific intervention or treatment.
Dr SA 2012
Appraising systematicreviews
Once you have conducted a literature search and obtained full text articles, you can begin the critical appraisal process.
Consider the following questions to assess the quality of the article
Dr SA 2012
Questions to AskWhat kind of reports do I want?
Is this article relevant to my issue and setting?
How much detail do I need?
How comprehensive do I need to be?
How far back should I search?
Dr SA 2012
Questions to AskIs it of interest?Why was it done? How was it done?What has been found?What are the implications?What else is of interest?
Dr SA 2012
Critically Appraise What You Read.Simple checklists enable the useful information to be identified.
Dr SA 2012
ChecklistsChecklists for particular types of
literature are a quick and easy way of learning critical appraisal.
They all have 3 stages:Basic questions.Essential appraisal. Detailed appraisal
Dr SA 2012
Critical Appraisal: Notes and Checklists
Available at: http://www.sign.ac.uk/methodology/checklists.html
Scottish Guidelines Network (SIGN) has developed a number of critical appraisal tools in support of their creation of evidence-based guidelines. These appraisal tools used by SIGN to conduct systematic reviews have been made available for public use on their website, including methodology evaluation checklists for each of the following 6 types of studies: systematic reviews/meta analyses, RCTs, cohort studies, case-control studies, diagnostic studies, and economic evaluations.
Dr SA 2012
Questions to AskThe abstract
Use the information found in the abstract to answer the questions below:
• Are your issues discussed there?• What are the main findings of the research?• Do you want to know more after reading the abstract?• Was the research done in a similar setting to yours?• Does it address a related question? (Even research
that covers your issue indirectly can be useful.)• Are there reasons to doubt the findings without
reading the whole article?
Dr SA 2012
Questions to Ask
You may conclude that the study is not reliable merely by reading the abstract.
If this is the case, move on to another article! Critical
Dr SA 2012
Questions to AskWhy was it done?
IntroductionShould a clear statement of the purpose of
the study.Focus on the time, place and circumstances of the
population and setting studied.How similar or different is the study population or setting to
yours? Is a difference likely to matter for the issue at hand?The absence of such statements can imply that the authors
had no clear idea of what they were trying to find out.
Or they didn’t find anything but wanted to publish!
Dr SA 2012
Questions to AskHow was it done?
Brief but should include enough detail to enable one to judge quality.
Must include who was studied and how they were recruited.
Basic demographics must be there. How the data was collected (interviews, focus
groups, questionnaires, surveys, observations, etc.).
Methods
Dr SA 2012
Questions to Ask Results
Three broad issues need to be considered the results of the study :
A Are the results of the study valid?B What are the results?C Will the results help locally
The major challenge.
What do I think this really means?
Dr SA 2012
Questions to AskResults
What has it found?The data should be there statistics.Is the sample large enough to produce significant
results? Is the effect size clinically or operationally relevant?
Are the aims in the introduction addressed in the results?
Look for illogical sequences, bland statements of results.
Flaws and inconsistencies.All research has some flaws , the
impact of the flaws need to assessed.Dr SA 2012
Questions to AskHow good are the measures?• Do the measures accurately reflect what the
researcher was trying to measure(validity)?• How clear and appropriate are these
measures? Too broad? Too narrow?• Are they actual measures or proxy measures?• Are the measures well established in either
prior research or through pilot testing?
Dr SA 2012
Questions to Ask Discussion
The discussion section is more detailed and precise than the abstract, and will explain:
A review of how these results compare or contrast with prior research? If this report found something different from.
The limitations of the research and possible implications which are not mentioned in
the abstract.
Dr SA 2012
Questions to Ask implications
What are the implications?Abstract / discussion.
The whole use of research is how far the results can be generalised.
All authors will tend to think their work is more important than the rest of us!
What is new here? What does it mean for health care? Is it relevant to my speciality?
Dr SA 2012
Recommended Reading
The following 3 reading series were published in core medical journals and contain in-depth
information on critical appraisal and evidence-based practice.
From the British Medical Journal (BMJ): How to Read a Paper: The Basics of Evidence-Based
MedicineAvailable at: http://www.bmj.com/collections/read.dtl Evidence-Based Nursing NotebookAvailable at: http://ebn.bmj.com/cgi/collection/notebookFrom the Journal of the American Medical Association (JAMA): User Guides to Medical LiteratureAvailable at: http://www.shef.ac.uk/scharr/ir/userg.html
Dr SA 2012
Online Critical Appraisal ToolsThe Centre for Methods and Tools (http://www.nccmt.ca/) is currently developing a
critical appraisal framework that is specifically tailored topublic health research. This guide will be the definitive critical appraisal resource for
public health practitioners in Canada.• AGREE InstrumentAvailable at: http://www.agreetrust.org/instrument.htm The purpose of the Appraisal of Guidelines Research & Evaluation (AGREE)
Instrumentis to provide a framework for assessing the quality of clinical practice guidelines. Itoriginates from an international collaboration of researchers and policy makers who
worktogether to improve the quality and effectiveness of clinical practice guidelines byestablishing a shared framework for their development, reporting and assessment.• A Beginner’s Guide to Judging Research StudiesAvailable at http://www.cihr-irsc.gc.ca/e/34192.htmlThis is a succinct guide to critical appraisal written by John Frank, Scientific Director atthe Canadian Institutes of Health Research. Although this article focuses on makingsense of media reports of new health research, the astute questions Frank poses can beapplied directly to research studies themselves.
Dr SA 2012
Online Critical Appraisal Tools• CASP Critical Appraisal ToolsAvailable at: http://www.phru.nhs.uk/Pages/PHD/resources.htmThe UK-based Critical Appraisal Skills Programme (CASP), a division
of the NHS’s Public Health Resources Unit, has developed a customized critical appraisal checklist for each common type of research study. Direct links to each of the 6 appraisal tools are
listed below: • Qualitative Research http://www.chsrf.ca/kte_docs/casp_qualitative_tool.pdf • Review Articles (including Systematic Reviews) http://www.phru.nhs.uk/Doc_Links/S.Reviews%20Appraisal
%20Tool.pdf • Case Control Studieshttp://www.phru.nhs.uk/Doc_Links/Case%20Control
%2011%20Questions.pdf
Dr SA 2012
Thanks
Dr SA 2012
Recommended