Educational Educational Action ResearchAction Research

Todd TwymanTodd Twyman

Summer 2011Summer 2011

Week 2Week 2

Qualitative ResearchQualitative Research

Goal:Goal: to understand deeply to understand deeply Intense and/or prolonged contact Intense and/or prolonged contact

with life situationwith life situation Approach:Approach: Interpret as you go; keep Interpret as you go; keep

questions open so you don’t miss new questions open so you don’t miss new insights that lead to discovery, be insights that lead to discovery, be open to emerging themes as you go.open to emerging themes as you go.

Qualitative DataQualitative Data

In-depth In-depth view of the phenomena.view of the phenomena. Must be Must be situatedsituated in context. in context. Multiple data sourcesMultiple data sources are are

important.important.

How will you gather your How will you gather your data?data?

ObservationObservation Interview / focus groupsInterview / focus groups Extant data sources / document Extant data sources / document

analysisanalysis Survey with open-ended questionsSurvey with open-ended questions

Field ObservationField Observation

Take clear notes.Take clear notes. Include both observations of what you Include both observations of what you

see AND reactions / reflections as you see AND reactions / reflections as you go.go.

Conduct “member checks” to be sure Conduct “member checks” to be sure your interpretation makes sense.your interpretation makes sense.

Try to remain outside the situation; Try to remain outside the situation; write down what you observe NOT write down what you observe NOT what you interpret!what you interpret!

Observation vs Observation vs InterpretationInterpretation

Observation: Captures what you see, Observation: Captures what you see, avoiding interpreting.avoiding interpreting. I see a woman with grey hair walking I see a woman with grey hair walking

slowly across the quad.slowly across the quad. Interpretation: Imbues what you see Interpretation: Imbues what you see

with additional meaning/judgments.with additional meaning/judgments. I see an old, tired woman limping I see an old, tired woman limping

across the quad.across the quad.

Field ObservationField Observation

Observations Reflections

Time What I saw What I think about this…

3 Sources of Data 3 Sources of Data Dependent on Dependent on

Appropriate QuestionsAppropriate Questions InterviewsInterviews Focus GroupsFocus Groups Open-Ended Survey QuestionsOpen-Ended Survey Questions

Writing Good QuestionsWriting Good Questions

Avoid questions that begin with the Avoid questions that begin with the word “why”.word “why”.

Avoid questions that can be answered Avoid questions that can be answered with a “yes” or “no” response.with a “yes” or “no” response.

Avoid ‘tipping your hand’ through the Avoid ‘tipping your hand’ through the wording of your questions -- try to wording of your questions -- try to remain neutral in how you ask them. remain neutral in how you ask them. (both semantically and through tone).(both semantically and through tone).

InterviewsInterviews

Be prepared (recording device / paper Be prepared (recording device / paper with pencils; list of questions; on time).with pencils; list of questions; on time).

Concentrate on helping the person you Concentrate on helping the person you are interviewing feel at ease.are interviewing feel at ease.

Ask open-ended questions that allow the Ask open-ended questions that allow the person to provide you with rich insight.person to provide you with rich insight.

Ask follow-up questions to probe more Ask follow-up questions to probe more deeply into topics as they arise.deeply into topics as they arise.

InterviewsInterviews

Include information to establish Include information to establish context (setting, atmosphere, context (setting, atmosphere, emotions -- if any).emotions -- if any).

Check to be sure you captured their Check to be sure you captured their meaning (member checking).meaning (member checking).

Consider what is Consider what is notnot said as well as said as well as what what isis said, and said, and howhow it is said. it is said.

Have your interview protocol ready in Have your interview protocol ready in advance.advance.

Example Interview Example Interview (Todd)(Todd)

1. Describe the ESL population at your school.How many?At what grade levels?New Comer/Developing/Transitional?Language backgrounds? Cultural/SES backgrounds?

3. Can you tell me about the goals for the ESL Program?Timeframe for transitioning?NC/D/T?L1 use?Family interaction?

Focus GroupsFocus Groups

Make sure you have introduced Make sure you have introduced participants.participants.

Have a facilitator AND a note-taker.Have a facilitator AND a note-taker. Solicit opinions from all participants.Solicit opinions from all participants. Ensure all participants are treated Ensure all participants are treated

respectfully.respectfully. Consider your groupings carefully.Consider your groupings carefully.

Focus GroupsFocus Groups

Like interviews, have your focus group Like interviews, have your focus group protocols ready to go ahead of time. protocols ready to go ahead of time.

Provide participants with an Provide participants with an abbreviated list of questions; retain the abbreviated list of questions; retain the full protocol to guide you in case you full protocol to guide you in case you get nervous or distracted.get nervous or distracted.

Send ‘Thank You’ cards (more personal Send ‘Thank You’ cards (more personal than email) to all participants.than email) to all participants.

Extant Data Sources / Extant Data Sources / Document AnalysisDocument Analysis

Read with an open mind.Read with an open mind. Look for recurring themes as well as Look for recurring themes as well as

those that occur rarely.those that occur rarely. Be creative in searching out Be creative in searching out

appropriate documentation.appropriate documentation. Student work.Student work. School newsletters.School newsletters. Minutes from School Board meetings.Minutes from School Board meetings.

Survey (open-ended Survey (open-ended questions)questions)

Ask questions that will prompt Ask questions that will prompt sufficiently detailed responses.sufficiently detailed responses.

Focus on Focus on whatwhat is said, is said, howhow it is said, it is said, and what is and what is notnot said. said.

Consider following up especially Consider following up especially intriguing responses with an intriguing responses with an interview.interview.

How will you make sure How will you make sure your data are reliable?your data are reliable?

What sort of training will observers/ What sort of training will observers/ scorers/ raters receive?scorers/ raters receive?

How will you determine if your How will you determine if your observers/ scorers/ raters are observers/ scorers/ raters are performing reliably?performing reliably?

How will you check for how your How will you check for how your own biases / life experiences might own biases / life experiences might affect your results?affect your results?

If you’re going to be If you’re going to be interviewinginterviewing

Have you:Have you: Practiced with your recording Practiced with your recording

devices?devices? Thought out the types of follow-Thought out the types of follow-

up questions you might want to up questions you might want to ask?ask?

Had someone check to be sure Had someone check to be sure you are not ‘leading’ the people you are not ‘leading’ the people you are interviewing?you are interviewing?

Validity in Qualitative Validity in Qualitative StudiesStudies

Use multiple sources of data Use multiple sources of data (triangulation)(triangulation)

Have analyses / interpretations checked Have analyses / interpretations checked by participants (member checking)by participants (member checking)

Have more than one person conduct Have more than one person conduct analyses; check for common analyses; check for common interpretations between those engaged interpretations between those engaged in analysisin analysis Use ‘double blind’ techniques for all Use ‘double blind’ techniques for all

analysesanalyses

Practicing Field Practicing Field ObservationsObservations

Observation: Go someplace with Observation: Go someplace with people where you can do a 10 minute people where you can do a 10 minute observation. Try to be an ‘impartial observation. Try to be an ‘impartial observer’.observer’.

Take field notes Take field notes (you’ll be sharing these)(you’ll be sharing these).. Pay particular attention to the flow of Pay particular attention to the flow of

human traffic. Who is there? How are human traffic. Who is there? How are they interacting? Try to capture the they interacting? Try to capture the experience with rich description.experience with rich description.

Practice InterviewingPractice Interviewing Focus on drawing out your partner’s thoughts / Focus on drawing out your partner’s thoughts /

feelings on what he/she observed outsidefeelings on what he/she observed outside. . Remember to: Remember to:

Use open-ended questions. Use open-ended questions. Use your best active listening techniques to Use your best active listening techniques to

encourage your partner to explore his/her encourage your partner to explore his/her thoughts deeply. thoughts deeply.

Go beyond the surface, using follow-up Go beyond the surface, using follow-up questions to get at your partner’s underlying questions to get at your partner’s underlying beliefs of which he/she may not even be beliefs of which he/she may not even be aware. Can you identify certain perspectives?aware. Can you identify certain perspectives?

Practice InterviewingPractice Interviewing

Step 1Step 1 Write your Interview Protocol Write your Interview Protocol

Step 2Step 2 InterviewInterview

Step 3Step 3 DebriefDebrief

Debrief InterviewingDebrief Interviewing

Describe how it felt to be Describe how it felt to be interviewed. Were there things the interviewed. Were there things the person interviewing you did / could person interviewing you did / could have done to make the interaction have done to make the interaction more positive?more positive?

Short discussion of interviewing Short discussion of interviewing experience.experience.

Writing Open-Ended Writing Open-Ended Survey QuestionsSurvey Questions

Working with a partner, brainstorm Working with a partner, brainstorm some possible open-ended survey some possible open-ended survey questions you could use to gather questions you could use to gather information about your topic.information about your topic.

Meet in groups of 4, discuss and Meet in groups of 4, discuss and critique each others’ questions.critique each others’ questions.

What ‘rules of the road’ should folks What ‘rules of the road’ should folks keep in mind as they write such keep in mind as they write such questions?questions?