Research Process

Step One – Conceptualization of Objectives

Step Two – Measurement of Objectives

Step Three – Determine Sampling Technique

Step Four – Determine Data Collection Design

Step Five – Collect and Analyze Data

Step Six – Develop Graphs and Charts to Present Data.

Step Seven – Write a Report – Reporting Results

Review of Evaluation Research

Proposition

Conceptualization

Intervention

Observable Changes

Dependent Variable

Independent Variable

Identify Independent and Dependent VariablesIndependent

Intervention Strategies

DependentChange Resulting from Intervention

Develop Reliable and Valid Indicators of these MeasurementsReliable – Indicator provides consistent

measurement across time

Valid – Indicator provides accurate measurement

Outcome Measurements

Hypothesis

Example - Conceptualization

Use of fertiliz

er

Greater yield

of crops

Independent Variable

Dependent Variable

Concept Indicator

Operationalization of Concepts:Identifying Indicator

Use of Fertilizer

# of bags of

fertilizer used per acre of crop

Concept Indicator

Operationalization of Concepts;Identifying Indicator

Crop Yield

# of bales of hay per acres

Indicator Indicator

State Relationships Between Indicators

# of bags of

fertilizer per acre

Increases

# of bales of hay per acres

Non-Random Sample – for Descriptive StatisticsConvenience SampleSnowball SampleTheoretical Sample

Random Sample – For Inferential Statistics

Selecting a Sampling Technique

Non-Random SampleConvenience Sample – Select units that are

convenient (i.e., the nearest fields of crops)Snowball Sample – Have one farmer refer you

to another who will refer you to another etc.

Theoretical Sample – Your theory states that this fertilizer only works for innovative farmers so you select only innovative farmers as your experimental group.

Examples- Non-Random Sampling Techniques

Random SampleThis type of sample should be used when

You want to publish in a peer-reviewed journalYou want to generalize to the population

Every unit (field or farmer) in your population has an equal probability of being selected for your study.

Random Sampling Technique

Random Sample

Make a list of all the farmers in your county who plant this crop.

Assign a number to each farmerPlace numbers in a bin/hat and blindly draw

out the number of farmers you need for your study.

Example - Random Sampling Technique

QualitativeCase Studies

Gather detailed information from one or a small group of individuals.

Intensive Interviews/Focus GroupsIn-depth Understanding of SubjectsDisadvantage – Bias of interviewer can impact interpretation of

results.

Participant ObservationWatch ongoing processDisadvantage – Hawthorne Effect – People act differently when

they know they are being observed

Content AnalysisStudy materials objects (e.g., content of fields themselves)

Selecting a Data Collection Technique

QualitativeCase Studies - Tell Farmer Brown’s story about his experience with fertilizer.

Intensive Interviews/Focus GroupsSit down with individual farmers and asked open-ended questions,

or sit down with a group of farmers and “focus” the open-ended questions on fertilizer and crop yield.

Participant ObservationSpend a summer as a farmer who uses fertilizer and live amongst

farmers who use fertilizer.

Content AnalysisStudy the content of fields that have been fertilized – measure

amount of grain/hay grown per square inch etc.

Examples of Data Collection Technique s

QuantitativeLaboratory Experiment

Study experimental and control groups in a laboratory situation

Field TrialsStructure a experiment out in the field/community

Surveys

Construct questionnaires and mail/read to farmers.

SecondaryUse information collected by someone else

Selecting a Data Collection Technique

QuantitativeLaboratory Experiment

Plant small plots of land in a laboratory. Half of them would be fertilized and the other half would not.

Field TrialsSelect farms that are fertilized and compare those to ones that

are not.

SurveysUse survey questions to ask farmers how satisfied they are with

fertilizer and to report how much it has improved their crops.

SecondaryFind old records that contain information about fertilizers and

crop yield

Examples of Data Collection Techniques

Field TrialsData is collect literally “out in the field” or the

community

Laboratory ExperimentData is collected in a laboratory setting

Different Types of ExperimentsOne shot post-testOne group pre- and post-testClassical Experimental design

Experimental and Control Groups – Pre- and Post-tests

Experimental Designs

One Shot Post-Test

Weeks 1 and 2 are The Baseline

Intervention Begins Week 3

0

1

2

3

4

5

6

Week 1 week 2 Week 3 Week 4 Week 5 Week 6

One Group Pre and Post

Year 1 and 2 are baselines (before fertilizer); year 3 was group yield after fertilizer was used

0

1

2

3

4

5

6

Average Profit of Crop Yield in $1,000 Before and After Fertilizer was Used

Year 1 and 2 are crop yields before fertilizer; year 3 is crop yield after fertilizer

One Group Pre and Post Tests

Field Trials Using Classical Experimental Design

Years 1 and 2 are the baseline years

0

1

2

3

4

5

6

Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 Year 8

ExperimentalGroupControl Group

SurveysSelf reported attitudes and behaviorDevelop survey instrument (see power point on

the development on survey instrument – have link here)Mail out surveyFace to face surveyOn-line surveyTelephone survey

Collect dataAnalyze data

Surveys

Descriptive StatisticsMeanMedianModeStandard Deviation

Inferential Statistics (Relationships)Statistical SignificanceChi Square

Data Analysis

Should Include the Following:IntroductionLiterature ReviewMethods SectionResultsDiscussion – Summary and ConclusionReferences/Appendix

Data Presentation

Dr. Carol AlbrechtAssessment Specialist USU Extcarol.albrecht@usu.edu(979) 777-2421

Questions or Comments, Contact: