View
319
Download
0
Category
Preview:
DESCRIPTION
This presentation was developed for participants on a PgDip programme - the remit was to offer insight into focus groups as a potential method of data collection for their research project.
Citation preview
Using focus groups for evaluating learner experiences
Tünde Varga-Atkins - eLearning Unit, CLLLuciane Vieira de Mello Rigden – School of Life Sciences
University of Liverpool6th June 2013
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
Outline1. Why focus group?2. What is a focus group? 3. Focus groups in context of other methods4. Viewpoints: focus groups as a – Commissioner (stakeholder)– Observer– Facilitators
5. Focus group analysis and reporting6. Alternatives / enhancements:– Nominal Group Technique– Visual and any other methodsCreative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
Your experience
• Have you had experiences of focus group, either as:– Commissioner?– Facilitator?– Observer?– Participant?
What is your research question?Appropriate/Not to explore in a
focus group?
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
1. Why focus groups?
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
Rationale for focus groups
“When people gather to talk about something, their contributions and understandings will be
enriched by the group dynamic.” (Cousins 2009)
“To display and discuss differences in a group. ...” (Lewis 2003)
Focus groups are for sharing and comparing … and remembering
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
2. What is a focus group?
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
Focus group process
Neutral facilitatorSize: 6-8 (4-12) participants
Incentive
Sample selection!
Ethics: informed consent and reporting
Audio recorder & other resources or stimulus material
1-2 hours
an open group discussion with a focus
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
What is a focus group?
Study skills
International student experienceFocus:
Question(s):
English language Peer
support
Can you tell me about the support that you have received on campus?
A focused but open discussion with a group of 6-8 participants, moderated by a neutral facilitator usually lasting 1-2 hours. The facilitator usually has a topic (or question) guide which may or may not be covered in sequence.
Introduction Closure
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
The evaluation/research cycle* 1. Set out purpose
2. Agree what questions
to ask
3. Agree on method
(focus group)
5. Compiling draft report
6. Draft report sent to students for confirmation
10. Action & feedback to
students
7. Production of final report
4. Conduct of focus groups
8. Circulation of report to staff
9. Presentation and discussion
of report
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
*of not just focus groups but any participant consultation!
3. Focus groups in context of other methods
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
Some possible combinations
Exploring depth
survey
survey
focus group(s)
focus group(s)
Generate ideas
focus group(s)
a
b
c
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
Methods: from individual to group
Surveys
Focus groups
Interviews
Researcher present
individual
group
Researcher not present
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
Methods: from individual to group
Surveys
Focus groups
Interviews
Researcher present
individual
group
Researcher not present
How many hours do you spend studying?*
What makes you study (more)?
Can you tell me a bit more about what you do when you study?
Idea development
aided by others?
Participant motivation
Bias by others?
Shared experience?
* BUT: in a focus group setting, participants can have a discussion about what constitutes study, which may make them adjust their own understanding of what study is.Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
Surveys Focus groups Interviews
Bias? Useful when you don’t want others to bias your participant
Potential of ‘group bias’ – participants say what they think you or other participants want to hear
Useful when you don’t want others to bias your participant
Shared experience? Participant relies on own experience – may be more difficult to recall.
Useful to talk about shared experiences. ‘You bounce off ideas from one another’.
Useful when you want to explore individual experiences.
Idea development aided by others?
Relies on participant able to formulate their ideas without prompts.
FGs are useful when the development of ideas and understanding is aided by other group members.
FGs are useful when the development of ideas and understanding is aided by other group members.
Motivation/Enjoyment ? Non-response and quality of response as a problem.
Participants generally enjoy the process.* They learn something new.
Participants generally enjoy the process. They learn something new.
BUT: feasibility and practicality may be the largest factors(of getting volunteers, organising them together etc.)
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
Sample questions from a UoL curriculum evaluation focus group
• 1.What aspects of the X best facilitated your learning?
• 2. What aspects of the X inhibited your learning ?• 3. What aspects of the X caused the most
stress/anxiety and did this stress/anxiety worsen or lessen as the X progressed?
• 4. Optional specific questions.• 5. Can you suggest one (two) different ways in
which the X could be improved?
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
4. ViewpointsFocus groups as a:
commissioner
observerfacilitator
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
As a commissioner of a focus group• Experiences of Dr Mello (Lu):
– Asking the right questions• After a questionnaire: realisation of non-properly covered aspects guided question
formulation• Questions were discussed with the facilitators (but I would do it more extensively
next time….)
– Coverage of questions• Some really short answers from students• Group dynamic
– What is important? • Have clear what you want• Take the first one as an experience
– Expectations • If you have an open mind, you will surprise yourself!
As an observer of a focus group
• Experiences of Dr Mello (Lu):– What is going on?
• Got to know the questions just before meeting the students
– Benefits• Better appreciation & understanding of the whole process• Confidence to plan another focus group• How I learned from the students!!!
– Challenges• Hard to just observe! • Have a good chat with the facilitator in advance
As a facilitator of a focus group
• Design• Introduction, scene setting• Questioning techniques• Managing group dynamics• Managing schedule / topics / coverage
Slides supporting focus group process (facilitation)
The schedule of a Focus Group
1. Introduction = setting scene and establishing trust:– purpose – ground rules (conformity / convergence)– ethics – reporting
1. Introduction
2. The ‘Focus’ = questions
3. Closure
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
2. The ‘Focus’ = aka questionsLow vs high moderation
2 questions 6 questions: exploring international student experience on campus
6 questions
(Cousins 2009, p63)
Note: Participants do not normally see the questions – the discussion is good if it seems to flow naturally.
These will not necessarily be covered in order.
Anatomy of a good focus group question
with ready probes
(Cousins 2009, p63)
Preparation and good question (and prompts, probes) design is very important!
English language
Peer support
Probing for range
Probing for depth
Study skills
Can you say something
about further support..?
Open question
5. Analysis and reporting
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
Analysis
TranscriptReport
Analytical process
Various decisions and challenges!• Emphasis – key points• Focus• Agreement• Majority vs single voices
Research question dependent
6. Alternative methods
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
It doesn’t have to be just words!Tasks e.g. projection techniques
Source: http://bit.ly/JZRz1T
Visual techniquesStimulus
Source http://bit.ly/Mc0kSu
Bringing as many stimulus material related to the topic as possible to help participants recall e.g. bring assignment samples for a discussion on feedback.
Asking participants to take photos, draw diagrams or drawings to elicit their experiences e.g. ask them to draw a timeline of their time at university
Asking participants to undertake an activity, e.g. using projective techniques or write down their top five priority in the given topic etc.
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
The Nominal Group Technique • Good decision-making technique with no need for
transcription!• Brings in a quantitative element through the ranking stages.
• More useful for evaluation and less useful for ‘full’ research.– See references for more details.
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
The Nominal Group Technique: an alternative focus group method
Surveys
Focus groups
Interviews
individual
group
Nominal Group Technique
The Nominal Group Technique is a useful technique that is structured in a way that although it is a group session, it does focus on individual experiences and opinions rather than group consensus. Participants at various points in the session are asked to give their individual opinions which are ranked at the end.
Summary• Focus groups are useful when:
– exploring a shared experience.– participants (and the research!) benefits from the development of
understanding and ideas in a group environment.– group bias or conformity is not likely/important.
• Focus groups:– need to be facilitated by a neutral facilitator.– produce quality of the output (if focused and run well).– can be enriched by various techniques (visuals, stimulus) or tasks.
• Analysis involves various decisions on what/how to report.• A useful alternative is the Nominal Group Technique.
– Just writing up a combined ‘Nominal Focus Group’ as a useful method.
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
References
• Focus groups– Cousin, G. (2009). Focus Group Research. IN:
Researching learning in higher education : an introduction to contemporary methods and approaches. London: Routledge, pp.51-69.
– Ritchie, J., & Lewis, J. (2003). Qualitative research practice : a guide for social science students and researchers. London; Thousand Oaks, Calif.: Sage Publications.
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
References
• Nominal Group Technique– Practical guide: Varga-Atkins, T., with contributions from Bunyan, N;
McIsaac, J; Fewtrell J. (2011) The Nominal Group Technique: a practical guide for facilitators. Written for the ELESIG Small Grants Scheme. Liverpool: University of Liverpool. October. Version 1.0. Available at http://slidesha.re/s5KPUr
– Project report: Varga-Atkins, T. with contributions from Bunyan, N; McIsaac, J and Fewtrell, R. (2011) Using the nominal group technique with clickers to research student experiences of e-learning. Project Report written for the ELESIG Small Grants Scheme. Liverpool: University of Liverpool. Available from slideshare.net at http://slidesha.re/sc8gwT
Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
Recommended