Upload
phillip-hamilton
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
DATA ANALYSIS AND INTERPRETATION
Allison Nichols, Ed.D.
Evaluation Specialist, West Virginia University
Martha Garton, M.A.
Grant County, WV, Extension Educator
Evaluation Community of Practice Webinar
October 20, 2010
IS YOUR DATA MOSTLY TEXT FROM . . .?
Interviews Focus groups Meeting minutes Sign-in sheets Emails and other correspondence Evidence of completed projects such as a new
curriculum Media outlets such as newspaper articles, TV and
radio clips, Internet sites and Internet hits
CHOOSING A METHOD
If the data is text or words, you will probably choose a qualitative method.
Content analysis Look for reoccurring themes Categorize themes into larger and smaller themes Look for the number of times a theme is mentioned Illustrate findings with quotes
Unlike quantitative analysis, findings cannot be generalized beyond the group being studied.
CHOOSING A METHOD
If your data consists of numbers, you will probably choose a quantitative method to analyze. Most commonly used are frequencies
Frequencies are summary data of the number of attendees, meetings, workshops,
dissemination methods, items that are disseminated.
the number of survey respondents who choose a particular answer.
Descriptive data including Mean (average) Mode (most common answer) Median (answer right in the middle)
POLL
How comfortable are you analyzing quantitative data? A = not at all comfortable B = somewhat comfortable C = very comfortable
How comfortable are you analyzing qualitative data? A = not at all comfortable B = somewhat comfortable C = very comfortable
STATISTICAL TESTS T-tests – if you want to measure the difference between
the mean for answers pre- and post-intervention. There are two types: unmatched tests matched tests
Chi Square – if you want to look at the difference between categories of people, i.e. men and women
Regression – if you want to look at the association of one variable with another variable; does one go up when the other goes down? i.e. are respondents more likely to react positively the longer they have been in the program.
Factor analysis – when you want to see if questions are correlated with each other and thus fall within factors that can be named.
COMBINING STATISTICAL AND QUALITATIVE DATA Strong evaluation plans combine statistical
and qualitative data. Example 1
If by using statistical tests such as Chi Squares, we know that women are more likely to adopt healthy behaviors than men, but ---
We don’t know why - - We can use qualitative methodologies such as
focus groups and interviews to understand why. Example 2
If we conduct focus groups and know ways that participants benefit from a program
We can conduct a survey to learn how many program participants benefit in a particular way.
WHAT TO DO WITH THE DATA ANALYSIS
What are the important findings?
What are the implications of the findings?
What are the recommendations?
CASE STUDY: EXAMPLE FROM WV 4-H CAMPING
The National 4-H Camping Consortium created a series of tools for evaluating camp. They included
Three logic models General camping Life skills at camp Camp Context – Essential Elements of Youth
Development Two corresponding questionnaires
Life Skills Camp Context
EVALUATION PROTOCOL
West Virginia University Extension has used both questionnaires since 2007: In its 60 summer residential camps (county and
state) With 2,000 – 3,000 youth each summer
FACTORS/DOMAINS USED IN THE ANALYSIS
Life skills developed at camp Accepting Self and Others Accomplishing Goals Taking Responsibility
Essential Elements provided in camp setting Opportunity to Build a Relationship with a
Caring Adult Opportunity for Independent Learning and
Mastery Emotionally Safe and Inclusive Environment Physically Safe Environment
DATA SUMMARIES
Each camp director/Extension agent receives the following information
Frequencies for all questions Descriptive statistics for all questions Descriptive statistics for each of the factors:
life skills and essential elements All of the above for the whole state
PURPOSE OF SUMMARIES
Each camp director receives directions on how to use the statistics for: Improving programs Reporting to stakeholders Writing narratives for the faculty file
INSTRUCTIONS
The directions state that camp directors should: Discuss significant findings for their camp in the
current year Compare findings for their camp with findings
from previous years and discuss improvements Compare findings for their camp with findings
from the whole state and discuss why their camp is different and how they should adjust their programming to meet needs of their county
INTERESTING FINDINGS
For the past several years we have found that boys feel significantly less safe (both emotionally and physically) than girls at camp.
Girls have consistently more positive responses on all items.
As the number of years at camp and the number of years in 4-H increase, responses on all item are more positive.
COMBINING QUANTITATIVE DATA WITH QUALITATIVE DATA
To answer the question “why do boys feel less safe, we held three focus groups at state camp this year (Older Member Camp)
After analyzing the data from the focus group, we plan to revamp the questions and conduct focus groups (using counselors as facilitators) at county-based camps next summer.
COUNTY-BASED FACULTY MEMBER (EXTENSION AGENT) WILL DISCUSS HER EXPERIENCE THE 4-H CAMP EVALUATION PROCESS