Upload
katherine-bates
View
218
Download
0
Tags:
Embed Size (px)
Citation preview
QUALITATIVE EVALUATION OVERVIEW
Kim KoesterDirector of Qualitative ResearchAIDS Policy Research CenterCenter for AIDS Prevention StudiesUCSF
Objectives
To teach you to understand the application of and methods associated with qualitative evaluation
To provide an overview a peek “under the hood” of the qualitative analytic process.
Outline of talk
1. Explain qualitative research process1. Methods2. Procedures 3. Sampling4. Analysis
2. Demystify the analytic activities 1. Briefly describe different ways to approach qualitative
data analysis.3. Dig in to practical issues
1. Talk about resources required2. Touch on issues that must be addressed in writing up
data for publication
Questions for you
What comes to mind when you think of qualitative methods?
Who plans to incorporate qualitative methods into your evaluation plan for this class?
Has anyone participated in a qualitative research project, as either an informant or as a member of a research team?
What is Qualitative Research?
Qualitative research is an approach to scientific inquiry that relies on more
naturalistic, humanistic and interactive processes. The methods are primarily
language based, with data in the form of words rather than numbers.
Joan S. Ash and Kenneth P. Guappone 2007
Qualitative Inquiry Areas of Emphasis Emphasizes a holistic perspective
Strive to understand a program and situation as a whole
Emphasizes the importance of understanding the meanings of human behavior and the social-cultural context of social interaction.
Emphasizes the emic perspective
Qualitative researchers empathize and identify with
the people they study in order to understand how those
people see things. Taylor and Bogdan
Qualitative Methods
Geared towards exploration, discovery, inductive logic
Defined in comparison to deductive research that begins with an hypothesis.
Comparing Qualitative & Quantitative Methods
Inductive Small n Purposeful sample Open-ended
questions, geared towards discovery
Seeking holistic understanding
Not generalizable Investigates how or
why
Deductive Large n Random sample Closed-ended
questions, predetermined measures
Individual variables Generalizable Investigates how
many when, where
Qualitative Techniques
In-depth interviewing (IDI) Key informant interviews Semi-structured interviews Narrative interviews
Life history interviews Sexual history interviews Critical incident interviews
Focus groups or Discussion groups Ethnography
Participant observation – shadowing/tag alongs
Selecting the method
The method you select depends entirely on the research question.
Give me an example and we’ll talk through the best method.
What methods used in the articles you read for today?
Sampling
Sample size in qualitative research vary from project to project; “there are no rules for sample size”
Purposive or purposeful sampling is the most common strategy in applied research/evaluation.
Purposeful sampling relies on the selection of information-rich cases; information rich cases allow learning a great deal about the issues that are of central importance to the evaluation
Patton delineated 16 different types of sampling frames
Designing Interview Guides
The purpose of the interview guide is to list the questions or areas of inquiry to be approached in the interview setting. Semi-structured interviews In-depth interviews
Example Interview Guide
DOMAIN: PERSONAL HISTORY/LIFE CIRCUMSTANCES INTENT: The intent of this section is to learn about
what has happened since the person was released and about how they are doing, in general.
QUESTIONS:Tell me how you have been meeting your day-to-day survival
needs, for example, housing, money, food, etc.?Tell me about your experiences re-connecting with friends and
family since you got out of prison.
Qualitative Analysis
Qualitative analysis is the process by which you turn your data into findings.
What are you doing when you are analyzing data? Reducing the sheer volume of raw data Separating out trivia from significance Identifying significant patterns
Qualitative Analysis
Driven by goals and aims of the evaluation Describe program Elucidate program dynamics Identify patterns Confirm or disconfirm quantitative findings
Key analytic activities
Thinking Reading Noticing Reacting Learning Producing insights Reading Sharing & discussing
•Refining insights•Reading •Labeling •Producing insights •Labeling•Reading•Parsing•Assembling•Understanding•Interpreting •Reading •And so on
Phenomenology
What is the meaning, structure, and essence of the lived experience of this phenomenon for this person or group of people?
Goal: to elucidate meaning of lived experience
Narrative Analysis
What does this narrative or story reveal about the person and world from which it came? How can this narrative be interpreted so that it provides an understanding of and illuminates the life and culture that created it? Comparing stories and cases to create meaning
Goal: to interpret stories to reveal cultural patterns
Grounded Theory
What theory emerges from a systematic comparative analysis and is grounded in fieldwork so as to explain what has been and is observed? Theoretical sampling Theoretical saturation Open and axial coding
Goal: to generate explanatory theory from social processes
Best uses: unstructured in-depth interviews, structured data sets with rich, lengthy narratives
Framework Analysis
Commonly used in applied policy research, this approach involves a set of distinct, yet highly interconnected analytic stages.
Familiarization w/data Identify thematic framework Indexing or coding Charting Mapping and Interpretation
Goal: to follow a transparent analytic process Best uses: interdisciplinary, team-based
research, in depth interviews and focus group data
Content Analysis
• Involves tagging a set of texts or other artifacts with codes that are derived from theory or from prior knowledge and then analyzing the distribution of the codes, usually statistically
• Once codes are developed, they become the unit of analysis rather than the utterances to which codes are assigned
• Goal: begin with deductive application of a priori codes, but remain open to inductive revelations.
• Best uses: Structured in-depth interviews, focus group
Software
Qualitative software packages exist to facilitate the organization of qualitative data.
Dedoose single user $(15 per month)
Atlas.ti single user $1800 Multi-user (5) - $6700
Nvivo single user - $1300 Multi-user (5) - $5200
Simply observing and interviewing do not ensure that the research is qualitative; the qualitative researcher must also interpret the beliefs and behaviors of participants.
Valerie Janesick
Disposition Ideas
Characteristics of analysts: Significant amount of patience and
tolerance for reading and re-reading data Able to sit with ambiguity Non-linear orientation Creative