The Literature Review in the Masters Dissertation

Roberta Sammut

The role of research reviews

What is research? ‘The systematic investigation to develop theories, establish

evidence and solve problems’ (Gough et al 2012 p.1)

Research can focus on: The creation of new knowledge through primary studies Creation of knowledge on the basis of previous research

Knowledge should be cumulative (Oakley 2012)

‘If I have seen further it is by standing on the

shoulders of giants’ –Isaac Newton

Why are reviews needed

‘Research information is like small jigsaw puzzle pieces in a box, where there are several pictures, several duplicates and several missing pieces’ (Sheldon 1998)

Individual studies use different methods, are of different quality and may present contradictory findings

We cannot give too much importance to one individual study

Why do you need to carry out a literature review?

Needed for identifying: Areas of uncertainty Where reality may be different

to what is believed Where more research is

needed How research in the area has

been carried out – strengths and limitations

The main theories and issues on your topic and critique of these

What distinguishes a good quality literature review?

Appropriate breadth and depth Rigour and consistency Clarity and brevity Effective analysis and synthesis Use of the literature to justify:

The particular approach to the topic The selection of methods That your research contributes something new

Changing expectations at a postgraduate level

What is expected of a literature review at undergraduate level Familiarity with a topic Skills to be able to carry out a search on the

subject Knowledge on appropriate referencing style and

an ability to create accurate bibliography The ability to summarise key ideas and some

critical awareness

Changing expectations at a postgraduate level (Hart 2007)

The content of the literature review at undergraduate level Descriptive and focused on the topic Includes the main current papers on the topic Analyses the papers on the topic in terms of

different arguments presented and different results

The expectations at Masters level (Hart 2007)

An increase in the scope, breadth and depth of the literature search

Application of relevant literature from across other disciplines

Competence in reading research

The literature review of the Masters dissertation (Hart 2007)

The literature review is a major component of your dissertation

Analytical – evaluating current ideas on the topic Summative – providing a comprehensive overview

of what is known, what the gaps are Covers methodological issues in relation to different

research techniques Includes discussion of theoretical issues relevant to

the study

Your literature review in context

Your thesis must form a coherent whole Your literature review should be clearly linked

to: Your justification for carrying out the study Your aims and objectives Your choice of research design The methods used to collect data Your discussion of the results Your conclusions and recommendations

Starting out: what type of review is appropriate to your work?

Traditional Review (Gough 2004) Journalistic Review (Greenhalgh 1997) Narrative Review (Macdonald 2003)

Usually broader in focus Do not address specific question Not necessarily comprehensive in literature included Do not state reasons for inclusion of papers Not structured in approach to searching for literature and

evaluation of quality Systematic Review

‘the shift in emphasis from the art of writing a review to the science of reviewing the evidence’ (Milne and Chambers 1993)

Famous example of possible different outcomes for systematic vs. traditional reviews (Petticrew and Roberts 2006)

Linus Pauling (1974) Well-known physician and Nobel prize laureate Carried out review on effect of Vitamin C on

prevention of colds Conclusions:

High dose of Vit C prevents colds People should consume 100 times dose of Vitamin C

than currently being consumed

Famous example of possible different outcomes for systematic vs. traditional reviews (Petticrew and Roberts 2006)

Douglas et al (2004) Systematic review of papers published during the

time of Pauling’s review Conclusions:

High doses of Vitamin C do not prevent colds Can reduce the duration of the cold by a few days Pauling did not include 15 relevant articles

What is a systematic review?

‘A review of research literature using systematic and explicit, accountable methods’ (Gough 2012)

The key characteristics of a systematic review are: Rigor: use of systematic methods to answer set research

question Transparency: every step is described; nothing left to

reader’s imagination Replicability: a second researcher should arrive at the

same conclusions (Oakley 2012)

Features of systematic reviews: Rigor (Oakley 2012)

The methods used are designed to ensure rigor in the process being used and are predetermined Comprehensiveness in the search used to avoid excluding

relevant research e.g. grey literature which could lead to publication bias

Specific criteria for the inclusion or exclusion of studies – to avoid leaving out unfavourable results

Use of more than one researcher to search literature, decide on inclusion and exclusion of studies, appraise studies

Conclusions are based on the most rigorous studies

Features of systematic reviews: Transparency (Oakley 2012)

Systematic reviews must be clear about: The question the review is designed to answer The suitability of the methods chosen How the studies were identified Why some studies were included and others not How judgements were made about the value of

particular studies in answering the research questions

The conclusions which are reached in relation to policy and practice

Features of systematic reviews: Replicability (Oakley 2012)

A systematic review should provide a clear explanation of all steps taken in the review process

This should allow another researcher to repeat the study

If the review was carried out rigorously, then the results of the second review should be the same

Because procedures used are described, the review can be updated

Diversity of systematic reviews

A systematic review is a secondary research study

Questions and methods used in systematic reviews reflect those of the primary research studies Share the same theoretical assumptions Share the same approach

The key steps of a systematic review (Gough 2012)

Review initiation: Formation of review team; engagement of stakeholders

Preparation of a protocol: review question, conceptual framework and methodology

Search strategy: search and screen literature on the basis of eligibility criteria

Mapping: identifying and describing relevant research papers Appraising: critically critiquing the research papers using

systematic methods (quality appraisal criteria) Synthesis: Putting together the results of the review into a

coherent whole, creating something new (using conceptual framework and quality judgements)

Using reviews (interpret and communicate findings with stakeholders)

All decisions/methods used are explained and justified

One species; many breedsSystematic reviews may differ on the basis

of the: Nature of the research question (Oakley

2012) What we want to find out:

What works What people want What people consider to be appropriate

The breadth and depth of the research question

E.g. ‘What is known about the barriers to and facilitators of healthy eating and physical activity in young people?’ vs.

‘Is CBT more effective than Health Education in producing weight loss in young people between 14 and 16 years of age?

Step 2: develop a search strategy Clearly identify your review question PICO framework:

Population (P), Intervention (I) or Exposure (E), Comparison (C), Outcomes (O), Time (T)

SPICE framework: Setting – where? Perspective – for whom? Intervention – what? Comparison – compared with what? Evaluation – with what result?

Step 2: Develop a search strategy

Identify the relevant databases: e.g. CINAHL, MedLine, PsychInfo, AgeLine etc.

Consider the advantages/disadvantages of running combined searches

Identify the keywords which you should use to access relevant research papers – use thesaurus, MeSH terms

Plan out Boolean phrases, truncation and wild cards Identify any limiters to your search – with

justification

Step 3: Develop inclusion/exclusion criteria

Your review should not include every possible paper on your topic

The papers you include should be directly relevant

Develop inclusion/exclusion criteria on the basis of your review question

Step 4: Develop review management tools

Keep track of what you are doing on a daily basis: use a diary

Use bibliographic software/files on databases/excel sheets to keep track of articles you exclude with reasons for exclusion

Develop an information extraction sheet to consistently extract the same type of data from each paper

Step 5: Use the PRISMA flowchart

PRISMA 2009 Flow Diagram

Records identified through database searching

(n = )

Scre

enin

g In

clud

ed

Elig

ibili

ty

Iden

tifica

tion

Additional records identified through other sources

(n = )

Records after duplicates removed (n = )

Records screened (n = )

Records excluded (n = )

Full-text articles assessed for eligibility

(n = )

Full-text articles excluded, with reasons

(n = )

Studies included in qualitative synthesis

(n = )

Studies included in quantitative synthesis

(meta-analysis) (n = )

Step 6: Appraise your studies

Weight of evidence framework (Gough 2007). Three dimensions: Quality of execution of the study ‘soundness’ Appropriateness of the study design and analysis

for addressing the research question How well matched the study is to the focus of the

review

Step 6: Appraise your studies Use a Checklist or scale to systematically examine main

methodological aspects of each study Less likely that methodological problems will be missed More than one checklist may be needed if mixed methods are

used

Multitude of appraisal tools available: Downs and Black – randomized and nonrandomized studies Cowley – comparative studies Newcastle-Ottawa Scale – nonrandomised studies Critical Appraisal Skills Programme

There may not be a suitable checklist available – you may need to adapt a checklist or develop a new one

Step 6: Appraise your studies

When choosing an appraisal tool consider: Checklist chosen must be suitable for design of

studies to be included in the review Whether the appraisal tool has been previously

tested or not for validity and reliability You may need to use more than one appraisal

tools if mixed methods are included

Step 7: Decide on how you are going to use the information from the appraisal As a threshold to include/exclude studies ‘Weight’ the studies qualitatively, when summarizing the

results e.g. high, intermediate, low quality ‘Weight’ the studies quantitatively - low scoring studies

contribute less to the final summary effect size estimate Describe the quality and relevance of each study for the

reader to arrive at own conclusions Carry out a sensitivity analysis – effect of

including/excluding studies of lower quality on the results Recommendations for future research in terms of

methods

Step 7: Writing up Prepare a plan of your review

Introduction History of the topic – including assumptions and definitions

from other researchers Theoretical background Address each of your research objectives by summarising

research Conclusion

Identify how the data you extracted will be synthesised: Meta analysis Narrative synthesis

What are the key sources?

What are the major issues and debates

about the topic?

What are the political

standpoints?What are the origins and

definitions of the topic?

How is knowledge on

the topic structured and

organised?

What are the main questions and problems

that have been addressed to

date?

What are the epistemologica

l and ontological

grounds for the discipline?

What are the key theories, concepts and

ideas?

Literature search and

review on your topic

Questions to ask yourself when writing up (Hart 2007, p. 14)

How have approaches to these questions increased our understanding and knowledge?

In summary: key issues for success Perseverance and diligence! Justification for the topic of your research and your choice of

approach Avoid communicating personal opinions and views and don’t

present facts without sufficient evidence Learn how to reference properly – invest in a training programme

on the use of bibliographic software Learn how to use search databases Befriend your librarian! Keep records of your ongoing work to prevent panic later on! Be charitable to others’ work whilst at the same time evaluating it! Remember – there is no such thing as a perfect review!

Recommended Reading List Bettany-Saltikov, J.B. (2012) How to do a systematic literature review in nursing.

Open University Press, England

Gough, D., Olivers, S. and Thomas, J. (2012) An introduction to systematic reviews. Sage, London

Greenhalgh, T. (2010) 4th ed. How to read a paper Wiley-Blackwell, Oxford

Hart C. (2007) Doing a literature review: releasing the social science research imagination. Sage, London

Hart C. (2001) Doing a literature search. Sage, London

Petticrew, M. and Roberts H. (2006) Systematic reviews in the social sciences Blackwell publishing, U.S.A.

Rudestam, K.E. and Newton R.R. (2007) 3rd ed. Surviving your dissertation Sage, London