Purpose of the Research Process

By Using the Scientific Process to Evaluate Your Programs, You Avoid the Following Errors: Overgeneralizations Selective Observation Illogical Reasoning Resistance to Change Adherence to Authority

Definition of Terms Overgeneralization

Assuming that what is true for one or two cases is true for all

Selective Observation Choosing to look only at things that are in line with our

preferences/beliefs

Illogical Reasoning Jumping to conclusions based on illogical reasoning

Adherence to Authority Believing the person making the claim has the knowledge

Resistance to Change Reluctance to change our ideas in light of new information

Ego based commitment Devotion to tradition

Errors In Reasoning Overgeneralization – She observed only two cats – both of

whom were previously confined indoors. On the basis of these two observations she generalized to all cats.

Dear Ann,I just moved my two cats to a house in the country.

When I lived in the city, I felt guilty because I never let my cats outside. When we arrived in the country I noticed several cats outside, so I threw open my back door to let my cats roam – free at last! My cats cautiously went to the door and looked outside for a while and then returned to the living room and lay down. I’ve realized that people should not feel guilty if they keep their cats inside – even when cats have the opportunity, they don’t want to play outside.

Errors In Reasoning Selective Observation – She observed the cats only once –

and focused only on her cats.

Dear Ann,I just moved my two cats to a house in the country.

When I lived in the city, I felt guilty because I never let my cats outside. When we arrived in the country I noticed several cats outside, so I threw open my back door to let my cats roam – free at last! My cats cautiously went to the door and looked outside for a while and then returned to the living room and lay down. I’ve realized that people should not feel guilty if they keep their cats inside – even when cats have the opportunity, they don’t want to play outside.

Errors In Reasoning Illogical Reasoning – She assumed that others feel guilty

about keeping cats indoors, and that cats are motivated by emotions.

Dear Ann,I just moved my two cats to a house in the country.

When I lived in the city, I felt guilty because I never let my cats outside. When we arrived in the country I noticed several cats outside, so I threw open my back door to let my cats roam – free at last! My cats cautiously went to the door and looked outside for a while and then returned to the living room and lay down. I’ve realized that people should not feel guilty if they keep their cats inside – even when cats have the opportunity, they don’t want to play outside.

Errors In Reasoning Resistance to Change – She was quick to conclude that

she had no need to change her approach to cats.

Dear Ann,I just moved my two cats to a house in the country.

When I lived in the city, I felt guilty because I never let my cats outside. When we arrived in the country I noticed several cats outside, so I threw open my back door to let my cats roam – free at last! My cats cautiously went to the door and looked outside for a while and then returned to the living room and lay down. I’ve realized that people should not feel guilty if they keep their cats inside – even when cats have the opportunity, they don’t want to play outside.

Errors In Reasoning Adherence to Authority – She was writing to Ann as an

“expert” to validate/support her conclusion.

Dear Ann,I just moved my two cats to a house in the country.

When I lived in the city, I felt guilty because I never let my cats outside. When we arrived in the country I noticed several cats outside, so I threw open my back door to let my cats roam – free at last! My cats cautiously went to the door and looked outside for a while and then returned to the living room and lay down. I’ve realized that people should not feel guilty if they keep their cats inside – even when cats have the opportunity, they don’t want to play outside.

Exercise These same kinds of errors lead to the

following conclusions:

Affirmative action results in reverse discrimination

People are poor because they are lazy

Politically conservative people are prejudiced

Affirmative Action Results in Reverse Discrimination

Overgeneralization – Because they read about one or two cases in the newspaper, they believe it is common

Selective Observation – They focus only on these examples and not on evidence that could discount their claims

Illogical Reasoning – They assume that affirmative action legislation is the problem and not erroneous implementation of the legislation by individuals

Resistance to Change – They discount empirical evidence that this statement is not true because they had a negative personal experience related to affirmative action (i.e., a relative who experienced reverse discrimination)

Adherence to Authority – They refer to a popular political figure who has made this statement

People are Poor Because They are Lazy

Overgeneralization – Because they observed one or two people who “took advantage of the system” they assume this is the same with all people who are poor

Selective Perception – They focus only on those who are lazy and don’t consider all of the working poor

Illogical Reasoning – They assume that there are jobs available for people who aren’t working

Resistance to Change – They haven’t been generous to people in poverty, so they want to believe they deserved it. OR they have been raised to believe you can “be anything you want to be”

Adherence to Authority – They refer to popular political leaders who have made this claim

Politically Conservative People are Prejudiced

Overgeneralization – May know one or several people who are prejudiced and conservative

Selective Perception – Discount evidence that conservative people may be very involved in volunteer work, etc. Only focus the on fact that they don’t vote to support social programs

Illogical Reasoning – Don’t realize that people can have some

prejudices, but not others Also assume that lack of support of social programs means they are prejudiced.

Resistance to Change – Have made disparaging remarks in past, and so don’t want to admit they, themselves, may be prejudiced – prejudiced against conservative people

Adherence to Authority – Refer to political leaders or people in Social Work profession who have made this claim

What can we do to AVOID these Errors?

Know the research!

Make claims based on empirical evidence

Know how to evaluate information/research Distinguish good research from poor research

Avoid “never” and “always” Understand that research is based on the idea that we

claim to know what MOST people will do MOST of the time! Recognize there will be exceptions and realize behavior is

generally explained by multiple factors

Review of Terms Overgeneralization

Assuming that what is true for one or two cases is true for all

Selective Observation Choosing to look only at things that are in line with our

preferences/beliefs

Illogical Reasoning Jumping to conclusions based on illogical reasoning

Adherence to Authority Believing the person making the claim has the knowledge

Resistance to Change Reluctance to change our ideas in light of new information

Ego based commitment Devotion to tradition

If You Read It in the Newspaper, is it TRUE?

Or Should You Just Make Paper Airplanes?

Social Scientific Approach to Gaining Knowledge

Use random sample to study individuals/groups

Use explicit criteria for determining cause

Measure and sample phenomena systematically Use scientific method

Use evidence that can be examined and critiqued by others (repetition)

Random Sample Every case in the population must have

an equal probability of being selected for study group

Sample groups must be of adequate size

If you do not have a random sample then you can determine what CAN happen, but not what TENDS to happen

Criteria for Determining Cause X and Y must be related

Change in independent or causal variable will mean change in the dependent or effect variable

X (independent variable) must come before Y (dependent variable)

There is no Z variable (other factor) explaining the relationship between X and Y

Example – X and Y Related Example 1

X is gender (male versus female) Y is grade point average Mean grade point average is higher for

females than males Thus gender and GPA are related

Example – X and Y Related Example 1

X is amount of time student studies Y is score on exams Students who study more have higher scores

on exams Thus amount of time students study (x) and

scores on exams (y) are related

X Must Come Before YIt’s not always easy to know

Which comes first….

The chicken or the egg

Watching violence on TV or being violent

Education or income

Testosterone or aggression

There is no Z (other) variable that explains the relationship

between X and Y Is damage caused by firemen?

Is shoplifting caused by large hands?

Does race explain criminal behavior?

Does height influence IQ?

These Variables are Related

As the number of firemen increases so does the amount of damage

People with larger hands are more likely to shoplift

African American males are more likely to commit crimes than Anglo American males

Taller second graders have higher IQ scores than shorter second graders

But these relationships are explained by a Z (or third) Variable

The relationship between the amount of damage and the number of firemen is explained by size of fire--These two are related because they are both related to size of fire

As size of fire increases so does amount of damage As size of fire increases so does number of firemen called

The relationship between hand size and the monetary value of things that are shoplifted is explained by gender

Men have larger hands than women and men are more likely than women to shoplift items of greater monetary value.

BUT these relationships are explained by a Z (or third) Variable

The relationship between race and criminal behavior is explained by socioeconomic factors (income, family structure, etc.)

The relationship between height and IQ is explained by their relationship to age

In second grade, older kids are taller and older kids have higher IQs

Use The Scientific Method - The Scientific Method Contains Rules that Indicate

HOW You Collect Data

Identify a research question and do a literature search

Identify and operationalize concepts What you want to study

Identify population and draw sample Who you want to study

Collect data How you want to study them/it

Analyze data How you will study them/it

Write a report and present information Carefully document this process so someone else could repeat it—The more time

an experiment is repeated, the more valid and reliable it becomes

Evidence Based Practice Empowers Extension Employees and Protects The Integrity of Their

Programs Using the best current research evidence allows us to assess:

Accuracy of assessment tools Effectiveness of different interventions

Research helps program developers identify: Client values (i.e., prefer individual vs. group interventions) Community values (i.e., understanding of what interventions

are acceptable) Client Circumstances (i.e., voluntary or court ordered)

Research expertise results in program expertise Skilled extension agents can integrate past experience and

skills with “sound” empirical evidence

Extension Workers Using Research

To influence policy makers Guide policy development Influence distribution of resources

To assess intervention strategies Make judgments about appropriate treatments

To implement and evaluate programs

To identify characteristics of current clients

To communicate with clients – give results

Extension workers can use research:

Different Types of Research Can be Used for Different Purposes

Descriptive Research Describe the sample, program, etc.

Exploratory Research Explore unique program situation(s)

Explanatory Research Explain WHY people behave in certain ways under

certain conditions

Evaluation Research Evaluate current programs

Resources

Engel, Rafael J. and Russell K. Schutt—The Practice of Research in Social Work

Dr. Carol Albrecht Assessment Specialist USU Ext carol.albrecht@usu.edu (979) 777-2421