WRITING RESEARCH PROPOSALS & PUBLICATIONS

CILIP IL Group and LIRG

Getting started

Dr Alison Brettle

• To help LIS practitioners get started in research or evaluation projects (and then write them up)– What is research? What is evaluation?– What are the challenges and practical

considerations?– When to use quantitative and qualitative approaches– How to plan a project

• No ..isms or ..ologies or paradigms!

Aims

• Defining – which questions are suitable?

• Designing – what methods can I use?

• Doing – how do I do it?• Describing – how do I tell

people about it?

4 D’s

10 good reasons to engage in research?

Context: why research?

• Discover causes • Understand activities• Understand behaviours• Explore perceptions• Predict future trends• ‘Blue skies’ – exploratory research• Testing hypotheses• Evaluating impact

Aims of research ...

Research Evaluation

“The systematic investigation into and study of materials and sources in order to establish facts and reach new conclusions” (Oxford Dictionaries, http://oxforddictionaries.com/definition/english/research)

“An enquiry” (Robson, 2002)

“A study with a distinctive purpose; it is not a new or a different research strategy”

“Often to assess the effects and effectiveness of something” (Robson, 2002)

Research or evaluation?

Successful Unsuccessful

Activity and involvement Expedience

Convergence Method or technique

Intuition Motivation by publication, money or funding

Theory Lack of theory

Real world value (Robson, 2002 – adapted from Campbell, 1982)

Characteristics of research

Generic categories of research purpose

• Exploratory• Descriptive• Evaluative• Predictive• Explanatory• Developmental

To find out if user needs are met

To improve a service

To assess the outcomes of the service

To find out how a service is operating

To assess the efficiency of a service

To understand why a service works (or doesn’t)

What should the focus be?

How can we make the service better?

Does the service meet its planned goals?

What happens in the service?

How do the costs compare with the benefits?

Are we reaching the right group?

What happens to users as a result?

Is the service operating as planned?

Is it more or less efficient than other services?

Are we providing what users need?

Is it worth continuing?

(Adapted from Robson, 2000)

Some purposes of evaluation

• Deciding the focus• Developing the questions• Choosing a strategy• Selecting the method(s)• Arranging the practicalities• Collecting the data• Analysing the findings• Reporting what you have found

What is involved?

What skills do I need?

Everyday life skills for research

• Reading• Listening• Watching• Choosing• Questioning

SummarisingOrganisingWritingPresentingReflecting

• Research methodology – the approach or perspective taken to do the research

• Research design – how you do the research• Research methods – the tools you use to do

the research

A note on terminology

Getting started……Defining

Why do you need a research question?

• Offers direction throughout the study

• Guides the search strategy and choice of data collection method

• Suggests the format of likely answers

Start with a topic• What are you interested in?• What problems are there at work?• What issues are topical?• What do we know little about?• What do people disagree about?• What do influential people want to

know about?• What did your mum tell you to do?

???

??

?

?

Can the topic be researched?

• What is pain?

• Why are people anxious?

• Is Coke better than Pepsi?

• Why do girls get pregnant?

• Should we return to the moon?

• Can teenagers live on only noodles and beer?

• Why do patients not take their drugs properly?

Back to your topic, then…

Is it too broad (or too narrow)?

Is it researchable?

Is it worthwhile?

Does is still grab you?

Focus of questions – and studies

• What? (What is happening?) Exploratory

• What? (What has happened?) Evaluative

• What? (What will happen?) Predictive

• What? (What caused it?) Experimental

• Why? (Why is this happening?) Explanatory

• How? (How could things be different?) AR

• How? (How many?) Survey

Defining!• Explanatory – Do doctors find

things quicker after being taught search skills?

• Experimental – What works best – face to face or online teaching?

• Exploratory – What are students’ experiences of information literacy training?

• Descriptive - What are users perceptions about information literacy training?

Components of a research question

• Concise and direct

• Understandable

• A researchable problem

• Focused on central issues

• Multiple questions?

• Sub-questions?

Activity 1 - • In small groups

• What are the key issues about information literacy?

• What questions could be asked?

• …are they what, why or how questions?

– …what sorts of answers might they prompt?

• It may help to think of a problem

• Why do a literature review?

Using the literature

Why…• Demonstrates your subject knowledge• Improves your writing skills• Contextualises your research• Helps direct your research• Helps formulate your research questions• Can’t reinvent wheels – justify your original contribution• Provides material for comparison in later discussion sections

The literature can help justify/discuss

• Whether your findings confirm those of other studies• Whether your findings extend other studies• Whether your findings break new ground• Whether your work raises issues about the methodological

choices used by other studies• Whether your work challenges existing theoretical

approaches to your subject

Your literature review should be

• A coherent synthesis of existing research which – Demonstrates the context of your work– Involves thematic lines of argument round the research question– Demonstrates trends in how the topic has been treated by other

researchers– Makes links to the themes of your study– Shows a clear gap where your study fits in

What does this involve?• Assessing the value of the literature at a number of levels

– Individual papers – eg significant material– Collections or groupings of papers

• Emphasising the limitations of existing knowledge– Identifying the gaps – promote the value of your research– Justify the contribution of your study

What doesn’t it involve?• Lists of references• Long descriptions/summaries of other studies• Inaccurate citations• Illogical flow• Overuse of quotations and just paraphrasing of other’s

work• etc

Another way of thinking about it• Literature review as a map of the

field and its debates• Purposes of a literature review:

-> identify and summarize key paradigms and arguments

in your field -> position

yourself in relation to these arguments to set up your

own argument

Can learn from (or do a) Systematic Review

A review of all the literature on a particular topic, which has been systematically identified, appraised and summarised giving a summary answer.

What is a systematic review?• An overview of primary research

studies conducted according to explicit and reproducible methodology

• A rigorous method of summarising research evidence

• Shows what we know and don’t know about a topic area

• Provides evidence of effectiveness (or not) by summarising and appraising relevant evidence

Systematic review process• Define/focus the question• Develop a protocol• Search the literature (possibly 2 stages scoping and actual

searches)• Refine the inclusion/exclusion criteria• Assess the studies (data extraction tools, objective manner)• Combine the results of the studies to produce conclusion• Place findings in context – quality and heterogeniety of studies,

applicability of findings

Your protocol

• Plan what you are going to do in the review

• Set out the background and objectives

• Outline the resources you will use• Establish inclusion/exclusion

criteria• How will data be extracted (what

will be extracted)• How will you synthesise literature

• Keeps you on track and focussed

How will you refine the inclusion/exclusion criteria?

• Tighter the criteria =– less papers to review

• BUT – will your review draw any

meaningful conclusions?– Will it cover all relevant

perspectives• Focussing the question v quality

of studies

Know what you want to find out????

• Think about your structure before you start writing

• Use a tool so that you record the same information about each of the study

• Make sure it captures the elements that you wish to write about in your final report

• Don’t just cherry pick the bits you like

• Quality of studies – what evidence are you going to include? How are you going to assess the quality

• LIS specific or adapt one from healthcare– http://

nettingtheevidence.pbworks.com/w/page/11403006/Critical%20Appraisal%20Checklists

– HCPRDU tools• http://usir.salford.ac.uk/13070/

• LIS specific systematic reviews– http://lis-systematic-reviews.wikispaces.com/Welco

me

Critical appraisal and SRs in LIS

Quantitative v Qualitative?

What is quantitative research?• Objective approach, neutral• Scientific? Experimental? Non-

experimental– Introduces a change and collects

data about effects– Specifiy design, collect data about

effects• Fixed (Robson)

– Set out what you are going to do and how you are going to do it

– Follows well established procedures• Samples, variables,

measurement, control, confounders

What is quantitative research?• …allows you to count things• …may try to prove things• Answers “what” or “how” questions

– Questionnaires (to collect numerical data),

– Usage figures– Web logs

• Understanding behaviour or perceptions or views

• Less structured • Obtaining meaning

• “Why” questions or “how things are perceived

• In reality – mixed methods are often used

What is qualitative research?

Methods: summary

What? Why? How?

• QUANTITATIVE• Counting/measuring• Large samples (often)• Questionnaires• Analysis of statistics,

weblogs

• QUALITATIVE• Understanding of

behaviour• Perceptions• Exploratory research• Interviews, focus groups• Observation

Some Worked Examples• What database is best for searching on the topic

of severe mental illness? (Counting – Quantitative, simple descriptive statistics)

• Teaching online is as good as teaching face to face (Proving – Quantitative, quasi experimental or experimental)

• Which is best? Mediated searches or teaching users to find information? (Mixed – simple descriptive statistics, inferential statistics and thematic analysis of users views confirmed with focus groups)

• Do doctors receive training in searching?• Do doctors use the methods they were taught in training?• Do doctors find relevant material?• How often do doctors need to search using online

databases?• How frequently do doctors use online databases?• What do doctors think about the training they receive?• Do doctors percieve that the training they receive equips them to

be evidence based practitioners?• Does the training enable doctors to be better evidence based

practitioners?

What are doctors experiences of searching?

Defining Tips!• Make sure your question isn’t

too big/broad/narrow• Make sure your objectives are

smaller than your aims• Think – what exactly is it that

you wish to find out. Does your question really reflect this?

• Don’t build in assumptions– What are the benefits of

teaching doctors to search?• The clearer the question – the

easier it is to find the answer!

Activity 2: Defining• Think of a question that

you could answer by a research or evaluation study

• It may help to think of a problem!

• You may want to break it down into objectives that will allow you to collect data to answer the question

• Does it lend itself to quantitative, qualitative or mixed methods

Designing• What methods are you going to use to collect the data to

answer your question?– Involves asking questions of people, systems or texts– Involves testing? A hypothesis?– Could use questionnaires or data you already collect or other

measure or test or “experiment”

• Who (what) is your population?• Sampling – random, representative, purposive,

theoretical, snowball?• Valid/trustworthiness – does it do what it says on the tin?• Reliable – does it do it consistently/accurately?• Ethics – is your approach ethical ?• Bias and confounders - can you avoid them,

or account for them?

• Need to be considered at all stages:– Formulating questions– Gathering data (sampling, informed consent)– Analysing data (anonymity, confidentiality)– Writing up (reliability, accuracy)– Dissemination

• Working with particular groups– ‘hard to reach’– Children– NHS ethics

Research ethics

• As part of the degree course nurses need to learn IL skills – to help them through assessments (obtain their degree) and to ensure competency in professional practice (evidence based)

• What are the ethical issues involved?

An experimental study of information literacy training to pre-registration nurses (testing online training v

traditional teaching)

An experimental study of information literacy training to pre-registration nurses (testing online training v

traditional teaching)• Training session(s) linked to assessed work

Two training sessions – 4 tests– Ensure that both methods cover the learning outcomes in the same

way– Ensure both groups receive the same information at the same time– Ensure that the cohort wasn’t being tested for anything else– Ensure that test didn’t affect their assessment or its results– Students were able to opt out of intervention group and/or not have

their “test” results included in the study– University ethics procedure/school procedure– Student names/numbers not used (analysed by group not individual)– Crossover design

Design – Tips!• Look for examples of similar

studies – can you use/adapt the approach

• Has someone else developed a tool you can use?

• Make it feasible and manageable

• Be pragmatic - be as rigorous as you can whilst being aware of the limitations

• Make sure it is appropriate for the question

• Write a research proposal

Activity 4: Designing• Go back to your question• How would you design

your study to answer it?– What are your aims and

objectives?– What data do you need to

collect?– How are you going to

collect it?– Who are you going to

collect it from?– What

tools/methods/approaches are you going to use?

Doing• How am I going to collect the

data/information?– Eg online or paper questionnaires,

interviews, observations, recording, transcribing

• How am I going to analyse the data I collect?– Excel, SPSS, Descriptive statistics,

Inferential statistics, content analysis

• Do I need any help? (You may want to seek this at the design stage)

• Do I have the right skills?• Do I have enough resources?

Describing• What should I write about and how should I do

it?– Need to explain what you have done and how

you did it. – Need to present your study in a way that is

meaningful for the particular audience.– Need to think about style

• Where should I write about it?– Project report– Newsletters– Journals – Evidence Based Library and

Information Practice, Library and Information Research, subject specific

– Posters – conferences or study days– Presentations – internal, conferences

• If you are not going to do anything with it – why do the research?

Describing - Tips• Think – what are the key messages for

this audience• What is the best way of presenting the

data?• Can the audience understand what I’ve

done?• Can the audience work out if it is valid?

Reliable?• Have I explained and addressed the

limitations• Don’t “hide” results• Have I “answered” my

objectives/research question?