APPLICATION OF RASCH ANALYSIS IN SOCIAL SCIENCE STUDYsmartdigitalcommunity.utm.my/cite/files/2018/05/Slide-Rasch-Kolokium... · Step Dapatkan Nilai (Dari Raschke Excel) 1. Klik output

APPLICATION OF RASCH ANALYSIS IN SOCIAL SCIENCE STUDY

Mohd Rustam Bin

Faculty of Education, UTM

copyright MohdRustamUTM

APPLICATION OF RASCH ANALYSIS IN SOCIAL SCIENCE STUDY

Bin Mohd Rameli

Faculty of Education, UTM


WHAT IS

Rasch is a model of That estimates That estimates

That predicts response probabilityNothing but a function of Nothing but a function of

Rasch is a model of That uses number right for estimating And count of correct responses for

That relates raw score to And response distribution to

--with no ambiguity


WHAT IS Rasch?

is a model of probabilityThat estimates person abilityThat estimates item difficulty

response probabilityNothing but a function of ability and difficultyNothing but a function of ability and difficulty

is a model of sufficiencyThat uses number right for estimating person abilityAnd count of correct responses for item difficulty

That relates raw score to person ability And response distribution to item difficulty

no ambiguity



Person Fit Analysis

Item Fit Analysis

Summary Statistic (Reiability & Separation)Summary Statistic (Reiability & Separation)

Unidimensionality & Local Dependency

Differential Item Functioning

Rating Scale Calibration

Item-Person Map


Person Polarity:

Application: To check whether response of persons are align with their ability

Person Fit Analysis

Condition:

i) 0.3 < PMC < 0.8 @

ii) 0.4 < PMC < 0.85 @

ii) PMC >0


Application: To check whether response of persons are align

Person Fit Analysis


Person Measure

Application: To check if theremisbehave (abnormal response)

Syarat:

Person Measure

Syarat:

. 0.5<MNSQ<1.5 (Infit/Outfit)

ii. -2<ZSTD<+2 (Infit/Outfit)


there is (are) person(s) who are

Person Measure



Person Fit Analysis


Item Polarity:

Application: To check whether the direction of all items are same with the latent variable

Item Measure (Polarity/Measure)

Condition:

i) 0.3 < PMC < 0.8 @

ii) 0.4 < PMC < 0.85


Application: To check whether the direction of all items are



Item Fit Measure:

Application:

) To check the extend of item measure

ii) To check if there is (are) item(s)


Conditions:

. 0.5<MNSQ<1.5 (Infit/Outfit)

ii. -2<ZSTD<+2 (Infit/Outfit)

iii. There are no two or items in themeasure value (refer to column measure)


measure fit the model measure

item(s) which are misbehave


the same constructs having samemeasure)



Item Fit Analysis


Summary Statistic: Person Separation Index

Person separation• Application: To classify sample/respondents

• Low person separation index valueto discriminate high ability sample

• Implication: More items are needed• Condition: Refer Table 1


Summary Statistic: Person Separation Index

sample/respondents

value: Instrument is not sensitive enoughsample with low ability sample

needed


copyright MohdRustamUTMcopyright MohdRustamUTM

Summary Statistic: Item Separation IndexItem separation• Application: To verify item hierearchy

• Low item separation index value:• Low item separation index value:verify the item hierarchy

• Implication: More sample are needed• Condition: Refer Table 1


Item Separation Index

hierearchy

The sample is not large enough toThe sample is not large enough to

needed



Summary Statistic: Person Reliability Index

Person Reliability:

• Application: are the item/test ableability level

0.9 = 3 or 4 levels. 0.8 = 2 or 3 levels.0.9 = 3 or 4 levels. 0.8 = 2 or 3 levels.

• Application: There is high possibilityhigh measure value is really has higherhas lower measure value

• Condition: Refer Table 1


Person Reliability Index

able to discriminate sample into certain

0.5 = 1 or 2 levels0.5 = 1 or 2 levels

possibility that person who is estimated to havehigher ability as compared to person who


Summary Statistic: Person Reliability

Person Reliability:

Factors affecting person reliability:

i) Sample abilityi) Sample ability

ii) rating scale length

iii) Number of items


Summary Statistic: Person Reliability Index


Item Reliability:

• Application: Low item reliability impliesto locate item on latent variable accurately

Summary Statistic: Item

• Application: There is high possibilityhave high measure value is really hasitem which has lower measure value

• Condition: Refer Table 1


implies the sample is not large enoughaccurately

Item Reliability Index

possibility that item which is estimated tohas higher difficulty as compared to

value


Item Reliability:

Factors affecting item reliability

Summary Statistic: Item Reliability Index

Factors affecting item reliability

i) Item difficulty

ii) Sample size


reliability:

Statistic: Item Reliability Index

reliability:




Application:

i) To ensure that items in the instrument

ii) To ensure that measuring items really

Unidimentionality

iii) To evaluate how much variance been


instrument share the same dimension

really measuring specific objective

Unidimentionality

been measure by instrument


Condition:

i) raw explained by measure ≥40%

ii) unexplained variance for 1st contrast

Unidimentionality

ii) unexplained variance for 1st contrast

(refer Table 1)


contrast ≤15%

Unidimentionality

contrast ≤15%



Local Dependency and

Application:

To ensure that the measuringotherother

To ensure there is no confusionresponse

Condition:

-There is no pair(s) of item(s)coefficient ≥0.7


Dependency and Rasch Measures

items are independent from each

confusion of sample in giving their

correlated with the correlation




Aplikasi:

Untuk menyemak sekiranya terdapatkumpulan responden tertentu

Untuk memastikan pemilihan responUntuk memastikan pemilihan respontidak cenderung/memihak kepada(Contoh: terdapat item yang respondenmemilih sangat tidak setuju tetapicenderung untuk memilih sangat



terdapat item yang bias untuk

respon tertentu bagi item tertenturespon tertentu bagi item tertentukepada sesuatu kumpulan sahaja

responden lelaki cenderung untuktetapi akan responden perempuan

sangat setuju


Differential Item FunctioningSyarat:

-0.5 <Dif Size<0.5

-2 <Dif t < +2

Contoh:Contoh:

Bagi kumpulan lelaki dif size=0.34,

Bagi kumpulan perempuan dif size=

Bagi contoh ini, item pengukuranresponden perempuan



dif t=1.36

size=0.80, dif t=2.35

pengukuran adalah bias bagi kumpulancopyright MohdRustamUTM

Application:

) To ensure the choice of ratingcorrect

i) To ensure the response for each


i) To ensure the response for each

Condition:

. observe average value increase

ii. 1.4<structure calibration<5


rating scale in the instrument is

each scale is distributred equally


each scale is distributred equally

from one scale to other scale





Cara Bina Fail Rasch-SurveyDISSC CP CP

(dissc sipi sipi city square• Double click pada winstep

• Import

• Spss

• Select spss

• Cari fail spss• Cari fail spss

• Copy (item) Paste di bawah item response variables

• Copy (respondent) Paste di bawah person label variables

• Construct file

• Save

• File

• Open

• Cari file rasch

• Double enter

SurveyCP CS-FOC D

city square-free of charge doh)

item response variables

person label variables

Cara Bina Fail Rasch-Dikotomi• Double click pada winstep

• Import

• Excel

• Select excel

• Cari fail excel


• Copy (respondent) Paste di bawah person label variables• Copy (respondent) Paste di bawah person label variables

• Construct file

• Save

• Pada notepad yang terhasil, taipkan seperti

ini

• File

• Open

• Cari file rasch

• Double enter

Dikotomi (MCQ)


person label variablesperson label variables

seperti yang ditunjukkan pada slide berikut

Cara Bina Fail Rasch-DikotomiDikotomi (MCQ)

1. Taipkan ini selepas ayat reported decimal places for

scaling

2. KEY1= taipkan jawapan yang betul ; key for MCQ scoring

Cara Bina Fail Rasch-Partial Credit Model• Double click pada winstep

• Import

• Spss

• Select spss

• Cari fail spss


• Copy (respondent) Paste di bawah person label variables• Copy (respondent) Paste di bawah person label variables

• Construct file

• Save

• Pada notepad yang terhasil, taipkan seperti

ini

• File

• Open

• Cari file rasch

• Double enter

Partial Credit Model


person label variablesperson label variables

seperti yang ditunjukkan pada slide berikut

Cara Bina Fail Rasch-Partial Credit ModelPartial Credit Model

1. Taipkan ini selepas ayat

matches the data

2. B1 hingga E3 adalah item yang2. B1 hingga E3 adalah item yang

dikodkan dalam SPSS

3. Huruf , C dan A pada sebelah

nombor 1, 2 dan 3 mewakili

skala yang digunakan bagi item

tersebut (rujuk nota 4)

4. IVALUE A=4 bermaksud item

yang dikodkan A (nota 3) ialah

item yang skalanya 1 hingga 4

Step Dapatkan Nilai(Dari Rasch ke Excel)

1. Klik output tables

2. Klik person entry

3. Highlightkan dan copy jadual person entry

4. Pada file excel, lebarkan column A

5. Pastekan data yang telah dicopy pada column A5. Pastekan data yang telah dicopy pada column A

6. Klik menu data

7. Klik Text to columns

8. Klik Delimited

9. Klik Next

10. Klik Space

11. Klik Finish

12. Copy column nilai person measure

Nilai Person Measure Excel)

person entry

column Acolumn A

Step Dapatkan Nilai Logit(Rasch Probability Calculator)

1. Buka file rash calculator

2. Copy nilai person logit dan paste pada column C

3. Kenal pasti nilai item measure untuk item 1

(klik output table, pilih item entry, baca pada (klik output table, pilih item entry, baca pada column measure)

1. Taipkan nilai tersebut pada column E dan drag hingga row terakhir person

2. Drag column F hingga row terakhir person

3. Ulang untuk semua item

Logit Setiap Item Probability Calculator)

Copy nilai person logit dan paste pada column C

Kenal pasti nilai item measure untuk item 1

(klik output table, pilih item entry, baca pada (klik output table, pilih item entry, baca pada

Taipkan nilai tersebut pada column E dan drag hingga row terakhir person

Drag column F hingga row terakhir person

POP KUIZ:• Soalan 1: Cikgu Asyraf menguji

Tambahan tahun 2014 bagi melihatBerdasarkan nilai person separationdiperoleh, berapa dan apakah kumpulan

SUMMARY OF 16 MEASURED PERSONSUMMARY OF 16 MEASURED PERSON

-------------------------------------------------------------------------------

| TOTAL MODEL INFIT OUTFIT |

| SCORE COUNT MEASURE ERROR MNSQ ZSTD MNSQ ZSTD |

|-----------------------------------------------------------------------------

| MEAN 80.7 28.0 .93 .32

| S.D. 10.9 .0 1.06 .03 .38 1.5 .37 1.5 |

| MAX. 99.0 28.0 2.91

| MIN. 59.0 28.0 -1.01

|-----------------------------------------------------------------------------

| REAL RMSE .34 TRUE SD 1.01 SEPARATION

|MODEL RMSE .32 TRUE SD 1.02 SEPARATION 3.21 PERSON RELIABILITY .91 |

| S.E. OF PERSON MEAN = .27 |

-------------------------------------------------------------------------------


menguji soalan peperiksaan Matematikmelihat kualiti soalan yang terlibat.

separation dan item-person map yangkumpulan pelajar yang terhasil?

-------------------------------------------------------------------------------



-----------------------------------------------------------------------------|

| MEAN 80.7 28.0 .93 .32 .99 -.1 .95 -.2 |

| S.D. 10.9 .0 1.06 .03 .38 1.5 .37 1.5 |

.38 1.69 2.4 1.69 2.4 |

.29 .38 -3.0 .39 -3.0 |

-----------------------------------------------------------------------------|

SEPARATION 2.37 PERSON RELIABILITY .90 |



-------------------------------------------------------------------------------


PERSON - MAP – ITEM

<more>|<rare>

3 T+

|

|

|

|T

|

2 S+

| VAR00001

| VAR00028

X | VAR00005 VAR00024

X |

XXX |S VAR00004

1 XX M+ VAR00013 VAR00029

XXXX | VAR00010

X | VAR00002 VAR00023 VAR00050

| VAR00038

| VAR00017 VAR00003

| VAR00014

SUMMARY OF 16 MEASURED PERSON

-------------------------------------------------------------------------------



|-----------------------------------------------------------------------------

| MEAN 80.7 28.0 .93 .32

| S.D. 10.9 .0 1.06 .03 .38 1.5 .37 1.5 |

| MAX. 99.0 28.0

| MIN. 59.0 28.0

|-----------------------------------------------------------------------------

| REAL RMSE .34 TRUE SD 1.01



-------------------------------------------------------------------------------| VAR00014

0 +M VAR00009 VAR00053

S| VAR00033

| VAR00032

| VAR00048

|

| VAR00021

-1 +

T|S VAR00018 VAR00027

| VAR00046 VAR00054

|

|

xxxxxxxxxxxxx | VAR00045

-2 xxxxx +

|

|T VAR00025

|

| VAR00039

|

-3 +

<less>|<frequ>

Jawapan: BILANGAN KUMPULAN

PELAJAR

A

B

C

D

-------------------------------------------------------------------------------


SUMMARY OF 16 MEASURED PERSON

-------------------------------------------------------------------------------



-----------------------------------------------------------------------------|

| MEAN 80.7 28.0 .93 .32 .99 -.1 .95 -.2 |

| S.D. 10.9 .0 1.06 .03 .38 1.5 .37 1.5 |

| MAX. 99.0 28.0 2.91 .38 1.69 2.4 1.69 2.4 |

| MIN. 59.0 28.0 -1.01 .29 .38 -3.0 .39 -3.0 |

-----------------------------------------------------------------------------|

| REAL RMSE .34 TRUE SD 1.01 SEPARATION 2.37 PERSON RELIABILITY .90 |



-------------------------------------------------------------------------------

BILANGAN KUMPULAN

PELAJAR

KATEGORI PELAJAR

2 Sangat Lemah dan Lemah

3 Sangat Lemah, Sederhana dan

Baik

2 Lemah dan Baik

3 Sangat Lemah, Sederhana dan

Sangat Baik

-------------------------------------------------------------------------------


Soalan 2: Berdasarkan nilai item separation, item yang terhasil?

SUMMARY OF 28 MEASURED ITEM

-------------------------------------------------------------------------------



|-----------------------------------------------------------------------------

| MEAN 46.1 16.0 .00 .42

| S.D. 6.7 .0 1.18 .04 .32 1.0 .32 1.0 |

| MAX. 59.0 16.0 1.77 .56 1.51 1.5 1.47 1.4 |

| MIN. 35.0 16.0 -2.59 .38 .24

|-----------------------------------------------------------------------------

| REAL RMSE .44 TRUE SD 1.09 SEPARATION 2.46

• i sangat mudah dan sangat sukar

• ii mudah dan sederhana

• iii sangat mudah, sederhana dan sangat

• iv mudah, sederhana dan sukar

A: i dan ii

B: i dan iii

C: ii dan iv

D: ii dan iii

| REAL RMSE .44 TRUE SD 1.09 SEPARATION 2.46

|MODEL RMSE .42 TRUE SD 1.10 SEPARATION 2.61 ITEM RELIABILITY .87 |

| S.E. OF ITEM MEAN = .23 |

-------------------------------------------------------------------------------


item separation, apakah kemungkinan aras kesukaran

-------------------------------------------------------------------------------



-----------------------------------------------------------------------------|

| MEAN 46.1 16.0 .00 .42 .98 .0 .95 -.1 |

| S.D. 6.7 .0 1.18 .04 .32 1.0 .32 1.0 |

.56 1.51 1.5 1.47 1.4 |

.38 .24 -3.0 .26 -2.9 |

-----------------------------------------------------------------------------|

SEPARATION 2.46 ITEM RELIABILITY .86 |

sangat sukar

SEPARATION 2.46 ITEM RELIABILITY .86 |

|MODEL RMSE .42 TRUE SD 1.10 SEPARATION 2.61 ITEM RELIABILITY .87 |

| S.E. OF ITEM MEAN = .23 |

-------------------------------------------------------------------------------


Soalan 3: Cikgu Maslina ingin membangunkan instrumenpelajar bagi aras analisis (A), menilai (N) dan merekaterhasil, apakah item yang mungkin akan digugurkan

----------------------------------------------------------------------------------------------|ENTRY TOTAL TOTAL MODEL| INFIT | OUTFIT |PT|NUMBER SCORE COUNT MEASURE S.E. |MNSQ ZSTD|MNSQ ZSTD|CORR. EXP.| OBS% EXP%| ITEM ||------------------------------------+----------+----------+| 1 35 16 1.77 .38|1.50 1.5|1.47 1.4| .54 .57| 31.3 53.3| A1| 28 36 16 1.77 .38|1.35 1.1|1.38 1.2| .42 .57| 62.5 53.1| C1| 5 37 16 1.48 .38|1.16 .6|1.20 .7| .10 .57| 62.5 52.4| C2 || 24 37 16 1.48 .38|1.12 .5|1.12 .5| .43 .57| 43.8 52.4| C3| 4 39 16 1.19 .38|1.25 .8|1.25 .8| .51 .57| 43.8 50.8| B1 || 13 40 16 1.04 .38|1.26 .9|1.25 .8| .44 .57| 43.8 52.9| B2 || 29 40 16 1.04 .38|1.51 1.5|1.45 1.3| .70 .57| 31.3 52.9| C4 |

• A: C1

• B: C3

• C: B2

• D: A4

| 29 40 16 1.04 .38|1.51 1.5|1.45 1.3| .70 .57| 31.3 52.9| C4 || 10 41 16 .89 .39| .83 -.4| .82 -.4| .67 .56| 62.5 54.4| A4| 2 42 16 .74 .39|1.42 1.2|1.42 1.2| .48 .56| 25.0 56.1| A2 || 50 42 16 .74 .39| .58 -1.4| .57 -1.4| .72 .56| 62.5 56.1| A3| 23 43 16 .59 .39| .73 -.8| .73 -.8| .71 .56| 68.8 56.9| B3| 38 44 16 .59 .40|1.25 .8|1.36 1.1| .35 .56| 68.8 57.4| B4|------------------------------------+----------+----------+| MEAN 46.1 16.0 .00 .42| .98 .0| .95 -.1| | 59.6 58.9| || S.D. 6.7 .0 1.18 .04| .32 1.0| .32 1.0| | 16.7 5.3| |----------------------------------------------------------------------------------------------


instrumen pengukuran diagnosis penguasaan KBATmereka/mencipta (C). Berdasarkan nilai measure

digugurkan?

----------------------------------------------------------------------------------------------MODEL| INFIT | OUTFIT |PT-MEASURE |EXACT MATCH| |

|NUMBER SCORE COUNT MEASURE S.E. |MNSQ ZSTD|MNSQ ZSTD|CORR. EXP.| OBS% EXP%| ITEM |+-----------+-----------+---------|

| 1 35 16 1.77 .38|1.50 1.5|1.47 1.4| .54 .57| 31.3 53.3| A1 || 28 36 16 1.77 .38|1.35 1.1|1.38 1.2| .42 .57| 62.5 53.1| C1 || 5 37 16 1.48 .38|1.16 .6|1.20 .7| .10 .57| 62.5 52.4| C2 || 24 37 16 1.48 .38|1.12 .5|1.12 .5| .43 .57| 43.8 52.4| C3 || 4 39 16 1.19 .38|1.25 .8|1.25 .8| .51 .57| 43.8 50.8| B1 || 13 40 16 1.04 .38|1.26 .9|1.25 .8| .44 .57| 43.8 52.9| B2 || 29 40 16 1.04 .38|1.51 1.5|1.45 1.3| .70 .57| 31.3 52.9| C4 || 29 40 16 1.04 .38|1.51 1.5|1.45 1.3| .70 .57| 31.3 52.9| C4 |

.4| .67 .56| 62.5 54.4| A4 || 2 42 16 .74 .39|1.42 1.2|1.42 1.2| .48 .56| 25.0 56.1| A2 |

1.4| .72 .56| 62.5 56.1| A3 |.8| .71 .56| 68.8 56.9| B3 |

| 38 44 16 .59 .40|1.25 .8|1.36 1.1| .35 .56| 68.8 57.4| B4 |+-----------+-----------+---------|

.1| | 59.6 58.9| || S.D. 6.7 .0 1.18 .04| .32 1.0| .32 1.0| | 16.7 5.3| |----------------------------------------------------------------------------------------------


Soalan 4: Berdasarkan dapatan unidimensionalitymanakah yang lebih jitu

• INSTRUMEN A: KOMPETENSI GURU DALAM MENGAJAR KBAT

• Raw var explain by measure = 45% unexplained

• INSTRUMEN B: KERESAHAN MATEMATIK PELAJAR


• INSTRUMEN D: KOLABORASI IBU BAPA DALAM PEMBELARAN MATEMATIK

• Raw var explain by measure = 38.9% unexplained

• INSTRUMEN C: EFIKASI MATEMATIK PELAJAR



unidimensionality berikut, instrumen

INSTRUMEN A: KOMPETENSI GURU DALAM MENGAJAR KBAT

explain by measure = 45% unexplained var 1st contrast=17%

INSTRUMEN B: KERESAHAN MATEMATIK PELAJAR

explain by measure = 36% unexplained var 1st contrast=6.3%

INSTRUMEN D: KOLABORASI IBU BAPA DALAM PEMBELARAN MATEMATIK

explain by measure = 38.9% unexplained var 1st contrast=4.5%

explain by measure = 58% unexplained var 1st contrast=27%


Soalan 5: Cikgumembawa kepadapembelajaran KBATsiapakah pelajar

ITEM - MAP - PERSON<rare>|<more>

7 +||T |

6 + | | |

5 + 00021P | 00016P |S 00019P 00043P| 00042P

4 + 00032P| 00008P 00035P 00037P 00040P 00048P| 00002P 00003P 00029P| 00005P

3 +| 00010P 00055P|M 00013P 00015P 00022P 00031P 00054P 00056P |M 00013P 00015P 00022P 00031P 00054P 00056P | 00020P 00024P 00049P

2 + 00014P 00041P 00053P 00060P| 00004P 00007P| 00012P 00058P

X T| 00017P 00023P 00044P 00046P1 XXX + 00009P 00036P 00047P

XXXX S|S 00001P 00011P 00039PXX | 00018P 00045P 00051P

XXXXXXXX | 00059P0 XXXXXX M+ 00052P

XXX | 00057PXXXXX |

X S|1 XX +

XX T|T 00063PX |

| 00061P2 + 00064P

<frequ>|<less> copyright MohdRustamUTM

Cikgu Din ingin meneroka apakah faktor-faktor yangkepada penguasaan yang baik dan lemah pelajar dalam

KBAT. Berdasarkan person-map yang terhasil,pelajar yang seharusnya ditemu bual oleh Cikgu Din?


Documents

APPLICATION OF RASCH ANALYSIS IN SOCIAL SCIENCE STUDYsmartdigitalcommunity.utm.my/cite/files/2018/05/Slide-Rasch-Kolokium... · Step Dapatkan Nilai (Dari Raschke Excel) 1. Klik output