CHAPTER 8

How good is the evidence: Personal Observation, Research Studies, and

Analogies?

PER SON AL O BSERVATIO N AS EVIDENCE

Observations do not give us “pure” observations

What we “see” and report are filtered through a set of values, biases, attitudes and expectations

There is a need for you to determine whether there are good reasons to rely on such reports

The most reliable reports will be based on recent observations made by several people observing under optimal conditions who have no apparent strong expectations or biases related to the event being observed

RESEARCH STUDIES AS EVIDENCEResearch Studies

• Form of authority that relies on observation and carries special weight

• How dependable are the findings?• We turn to the scientific method as an important

guide for determining facts because the relationships among events in our world are very complex and because humans are fallible in their observations and theories about these events

So what does the scientific method do & what are it’s characteristics?

THE SCIENTIFIC METHOD

Attempts to avoid many of the build-in biases in our observations of the world and in our intuition and common sense

How does it do this? The following are the characteristics of the scientific method that make it unique and special …

THE SCIENTIFIC M ETHO D CHAR ACTER IST ICS

1.Publically Verifiable Data• the scientific method seeks info in the form of publically verifiable

data- that is – data obtained under conditions such that other qualified people can make similar observations and get the same results (replicate)

2.Control – the use of special procedures to reduce error in observations and in the interpretation of research findings (i.e. controlling bias – double blind?)

3.Precision in Language – precise and consistent in it’s use of language (use of operational definitions)

When conducted well, scientific research is one of our best sources of evident because it emphasizes verifiability, control and precision

RESEARCH FINDINGS ARE SOMETIMES

UNDEPENDABLEWhen communicators appeal to research as a source of evidence,

you should remember the following:

1.Research Varies Greatly in Quality (research, like anything, varies from good to bad)

2.Research Findings Contradict One Another (go with those that have been replicated)

3.Research Findings Do Not Prove Conclusions (at best they support conclusions because researchers interpret the results & different people can interpret different ways)

RESEARCH FINDINGS ARE SOMETIMES UNDEPENDABLE

4. Regardless of how objective a scientific report may seem, important subjective elements are always involved

• Researchers have expectations, attitudes, values, and needs that bias the questions they ask- the way they conduct research etc.

5. Writers often distort or simplify research conclusions

RESEARCH FINDINGS ARE SOMETIMES UNDEPENDABLE

6. Research “facts” change over time (once accepted – now refuted)

7. Research is sometimes more artificial then we think; in order to achieve control, research often looses it’s real world quality

8. The need for financial gain, status, security, and other factors can affect research outcomes (remember; researchers are just PEOPLE)

RESEARCH FINDINGS ARE SOMETIMES UNDEPENDABLE

Con* Research varies in quality

and artificiality*Findings Contradict

*Findings only support conclusions

*Research can be distoredPro

* Research subject to verifiability

*Use of control *Research is Precise &

consistent in use of language

IMPOSSIBLE CERTAINTY FALLACY

When critically evaluating research conclusions, be wary of the reasoning error of demanding certainty in some conclusion when some uncertainty is to be expected but that does not negate the conclusion

Fallacy Impossible Certainty

Assuming that a research conclusion should be rejected if it is not absolutely certain

CLU ES FO R EVALUATIN G RESEARCH STUDIES

1.What is the quality of the source of the report?

2.Does the report detail any special strengths of the research?

3.How recently was the research conducted, and are there any reasons to believe that the findings might have changed over time?

4.Have the study’s findings been replicated by other studies?

5.How selective has the communicator been in choosing studies?

CLU ES FO R EVALUATIN G RESEARCH STUDIES

6. Is there any evidence of strong-sense critical thinking?

7. Is there any reason for someone to have distorted the research?

8. Are conditions in the research artificial and therefore distorted?

9. How far can we generalize, given the research sample?

10. Are there any biases or distortions in the surveys, questionnaires, ratings, or other measurements that the researcher uses?

GEN ER AL IZ IN G FR OM THE RESEAR CH S AMPL E

Sampling – the process of selecting events or persons to study

Why sample?

Because researchers can never study all people and events about which they want to generalize, they choose a sample – You need to keep several important considerations in mind when evaluating a research sample

EVALUATING A RE SE ARCH SAM PLE

1.Sample must be large enough to justify the generalization or conclusion

2.The sample must possess as much breadth, or diversity, as the types of events about which conclusions are drawn

3.The more random the sample, the better (equal chance of sampling)

Note: we can generalize only to people and events that are like those that we have studies in the research

WATCH SURVEYS AND QUESTIONNAIRES

You can not assume that survey responses accurately reflect true attitudes

Ask yourself the following:1. Were the questions ambiguous in their

nature? (How were they worded?)2. Were they answered honestly?

ANALOGIES AS EVIDENCEAnalogies involve comparisons and rely on resemblance as a major form of evidence

I.E. “We know a lot about something in our world (X), and another event of interest in one or more respects, then they will probably be alike in other respects as well.”

Argument by Analogy- an argument that uses a well-known similarity between two things as the basis for a conclusion about a relatively unknown characteristic of one of those things

TO EVALUATE ANAL O GIES FO CUS ON TW O FACTO RS

1.The ways the two things being compared are similar and different

2.The relevance of the similarities and the differences

Strong analogies will be the ones in which the two things we compare possess relevant similarities and lack relevant differences

Relevant similarities and differences are the ones that directly relate to the underlying principle illustrated by the analogy

GENERATE YOUR OWN ANALOGIES

1.Identify some important features of what you are studying

2.Try to identify other situations with which you are familiar that have some similar features. Brainstorm – Try to imagine diverse situations

3.Try to determine whether the familiar situation can provide you with some insights about the unfamiliar situation

FALLACY: FAULTY ANALOGY

The fallacy of the faulty analogy occurs when an analogy is proposed in which there are important relevant dissimilarities

In short, analogies that trick or deceive us fit our definition of a reasoning fallacy; such deception is call the faulty analogy fallacy