1

Using the IDB Analyzer III

Andrés Sandoval-Hernández – IEA DPC

Workshop on using PISA, PIAC, TIMSS & PIRLS, TALIS datasets

Ispra, Italy- June 24-27, 2014

Note: These slides were prepared as part of the IEA training portfolio with the collaboration of IEA staff and resource persons.

2

Table of contents

• Percentiles with PIAAC

• Benchmarks with PIRLS

3

Percentiles

What are the 25th, 50th, 75th and 90th percentiles of the literacy achievement distribution?

4

Percentiles

What is it?

• Calculates the score points that separate a given percentage of the distribution of a variable, by subgroups defined by the grouping variables, with the adequate standard errors

• You can choose the values for percentiles freely, by separating them with a space-digit

• You can choose one analysis variable or one plausible value

5

Percentiles

So, for example, see the “OECD Skills Outlook 2013: First Results from the Survey of Adult Skills” p. 74, Figure 2.4:

http://www.oecd.org/site/piaac/Skills%20volume%201%20%28eng%29--full%20v12--eBook%20%2804%2011%202013%29.pdf

6

Percentiles

7

IDB Analyzer: Percentiles

• Data File: C:\EU_Workshop\Work\PIAAC_Merged.sav

• Analysis Type:

PIAAC (using final full sampling weight)

• Statistic Type: Percentiles

• Grouping Variables: CNTRYID

• Plausible Value: PVLIT1-10

• Weighting variable: SPFWT0 (final full sample weight)

• Percentiles: 5 25 75 95

8

IDB Analyzer: Percentiles

PVPSL1-10

SPFWT0

Percentiles

C:\EU_Workshop\Work\PIAAC_Merged.sav

C:\EU_Workshop\Work\Literacy_Percentiles.*

CNTRYID

25 50 75 90

PIAAC (using final full sampling weight)

9

SPSS: Percentiles

Start SPSS and run syntax

10

Percentiles – SPSS Output

11

List of countries

The 25th, 50th, 75th and 90th percentile of the distribution of the analysis variable (PVLIT1-10)

Percentiles – Excel Output

12

Percentiles – Excel Output

The standard errors of the percentiles

13

Table of contents

• Percentiles with PIAAC

• Benchmarks with PIRLS

14

Research Question: Example

School Emphasis on Academic Success scale (ACBGEAS)

IRT score with an international mean of 10 and international standard deviation of 2

Advanced International Benchmark (625 score points)

What is the average “School emphasis on academic success” scale score for students reaching the advanced international benchmark?

15

IDB Analyzer: Benchmarks

International benchmarks

• Four points on the overall reading scale

16

How to Retrieve Information?

• International benchmarks

• International Report, chapter 2 (625 score points)

17

IDB Analyzer: Benchmarks

School Emphasis on Academic Success

International Report, Exhibit 6.1

Example 1:

Contextual scale

18

Example: Benchmark Analysis Analysis File:

C:\EU_Workshop\Work\PIRLS_merged.*

Analysis Type: Using Student Weights

Statistic Type: Benchmarks

Plausible Value Option: Use PVs

Benchmark Option: Discrete with Analysis Variable(s)

Grouping Variables: IDCNTRY

Analysis Variables: ACBGEAS

Plausible Values: ASRREA01-05

Achievement Benchmark: 625

19

IDB Analyzer: Benchmark Analysis

C:\EU_Workshop\Work\PIRLS_Merged.*

Statistic Type:

Benchmarks

Plausible Value Option:

Use PVs

Analysis Type:

PIRLS (Using Student Weights)

Benchmark Option:

Discrete with Analysis Variable(s)

20

IDB Analyzer: Benchmark Analysis

Grouping Variables:

IDCNTRY

Analysis Variables:

ACBGEAS

Plausible Values: ASRREA01-05

Weight Variable:

TOTWGT

Achievement Benchmarks: 625

21

IDB Analyzer: Benchmark Analysis

Start SPSS

Output Files:

C:\EU_Workshop\Work\ACBGEAS_625.*

22

SPSS: Syntax File (*.sps)

C:\EU_Workshop\Work\ACBGEAS_625.sps

Run All (Ctrl-A and Ctrl-R)

23

SPSS: Output File (*.spv)

• C:\EU_Workshop\Work\ACBGEAS_625.spv

24

SPSS: Output File Total student weight – contains estimates in the population that belong to the group defined by the classification variables (IDCNTRY, ACBGEAS)

List of countries

Categories

Number of students in each group (IDCNTRY, ACBGEAS)

25

Standard error of percentage

Weighted percentage of students for the group defined by the classification variables (IDCNTRY, ACBGEAS)

Weighted mean for the group defined by the classification variables (IDCNTRY, ACBGEAS)

Standard error of the mean

SPSS: Output File

26

Standard deviation of the mean

Standard error of the standard deviation

SPSS: Output File

27

Percent Missing

SPSS: Output File

28

MS Excel: File (*.xls)

• C:\EU_Workshop\Work\ACBGEAS_625.xls

29

Any Questions?

Thank you for your attention!