VALIDITY vs. RELIABILITYby: Ivan Prasetya

Because of measuring the social phenomena is not easy like measuring the physical symptom and because there are a lot of MEASUREMENT ERROR in measuring social phenomena, so we should do VALIDITY & RELIABILITY TESTING.

VALIDITY vs. RELIABILITY

• Measurement VALIDITY:How well an empirical indicator and the conceptual definition of the construct that the indicator is suppose to measure ‘fit’ together.

• Measurement RELIABILITY:The independability or consistency of the measure of a variable.

Different tools are usedfor measuring different things

A tape measure is used tomeasure length or distance.

A thermometer is used tomeasure temperature.

A stopwatch is used tomeasure time.

If we want to measure weight/mass,we can use these tools:

Digital scale have the highest VALIDITY.

(1 litre = 1 kg)

However, even though a digital scale has the highest validity, but if the subject is not

cooperative, the test result will not be accurate:

If you keep jumping around whilemeasuring your weight, you won'tget an accurate result of your bodyweight.

And if you carry extra loads whilestepping on a digital scale, youwon't get an accurate result ofyour body weight either.

High Validity, Low Reliability

VALIDITY vs. RELIABILITY

Low Reliability,High Reliability, High Reliability,Low Validity Low Validity High Validity

GOODNESS OF MEASURES

GOODNESS OF DATA

RELIABILITY (Accuracy in

measurement)

VALIDITY (Are we measuring

the right thing?)

STABILITY

CONSISTENCY

TEST-RETEST RELIABILITY

PARALLEL-FORM RELIABILITY

INTERITEM CONSISTENCY RELIABILITY

SPLIT –HALF RELIABILITY

LOGICAL VALIDITY (CONTENT)

CONGRUENT VALIDITY (CONSTRUCT))

CRITERION-RELATED VALIDITY

FACE VALIDITYPREDICTIVE

CONCURENT

CONVERGENT

DISCRIMINANT

RELIABILITY

• Indicates the extent to which the measure is without bias (error free) and hence offers consistent measurement across time and across the various items in the instrument.

• In other words, the reliability of measure indicates the stability and consistency with which the instrument measures the concept and helps to assess the “goodness” of measure.

STABILITY OF MEASURES

TEST-RETEST RELIABILITY

The Reliability coefficientobtained with a

repetitionof the same measure on

asecond occasion.

PARALLEL-FORM RELIABILITY

When responses on twocomparable sets ofmeasures tapping the

sameconstruct are highlycorrelated.

TEST-RETEST RELIABILITY

If infrastructure in the country is not good, your company will cancel your decision to do Foreign Direct Investment (FDI) in that following country.

Answer (now): strongly disagree

Answer (20 days later): agree

Low Reliability

PARALLEL-FORM RELIABILITY

Do you think that Susi Similikiti is beautiful?Answer: YES

Do you think that Tukul’s wife is beautiful?Answer: NO

Fact: Susi Similikiti is Tukul’s wife.

Low Reliability

CONSISTENCY

INTERITEM CONSISTENCY

This the test of theconsistency of

respondents’answer to all the items in

themeasure.

SPLIT-HALF RELIABILITY

Reflecting the correlations

between two halves of an

instrument.

PROBLEMS and PITFALLin VALIDITY

VALIDITY in QUESTIONAIRE DESIGN

• How diligent are you?5 = very diligent4 = diligent3 = indifferent2 = lazy1 = very lazy

Because respondents have to evaluate themselves, they tend to choose number (4) and (5), or at least number (3).

Low Validity

To measure the degree of diligent(as a student),

we also can use this questionaire:

• How much time do you spend to learn everyday?• How many books do you read every week?• How often do you go to library every week?

The answer which revealed from questionaire above will tend to be more accurate to measure the degree of diligent.

High Validity

Avoid question that ask two things at once, you won’t know which ‘bit’ people are answering:

Low Validity

Avoid ambiguity:

Apakah kalian sudah membeli buku sejarah demokrasi yang baru?

Apanya yang baru???Bukunya...???Sejarahnya...???Atau...Demokrasinya???

Low Validity

Avoid jargon/abbreviations/slang:

Higher number of current GDP in a country will encourage your company to do more FDI in that country.

Higher number of current Per capita Income in a country will make your company do more FDI in that country.

If the level of corruption in a country increased, your company will reduce their amount of FDI in that country.

Low Validity

Avoid not mutually exclusive options:

I’m 20 years old now.

What age are you? 16-20 20-25 25-30 35-40

Low Validity

Problems and Pitfalls

• Avoid making questionnaire too long

• Typographical / spelling errors

Low Validity

Using SPSS

Analyze >>> Scale >>> ReliabilityScale If item delete >>> Continue >>> OK

If Corrected Item-Total Correlation > r-table:VALID

If Cronbach’s Alpha if item deleted > r-table:RELIABEL

Using SPSS

Item-Total Statistics Scale Mean if Item Deleted Scale Variance if Item Deleted

Corrected Item-Total Correlation

Cronbach's Alpha if Item Deleted

VAR1.1.1 152,7333 249,0299 0,2785 0,9368

VAR1.1.2 153,2667 244,7540 0,3568 0,9366

VAR1.1.3 153,0333 249,5506 0,3113 0,9364

VAR1.1.4 152,9667 246,9989 0,4354 0,9357

VAR1.1.5 153,2000 248,1655 0,3300 0,9364

VAR1.1.6 153,9000 245,6793 0,4432 0,9356

VAR1.1.7 154,0667 246,0644 0,4555 0,9356

VAR1.1.8 154,1667 241,3851 0,5454 0,9349

VAR1.1.9 153,7000 245,7345 0,3929 0,9361

THANK YOU!!!