The literacy divide: territorial differences in the Italian education system Claudio QUINTANO, Rosalia CASTELLANO, Sergio LONGOBARDI University of Naples

The literacy divide: territorial differences in the Italian education system

Claudio QUINTANO, Rosalia CASTELLANO, Sergio LONGOBARDIClaudio QUINTANO, Rosalia CASTELLANO, Sergio LONGOBARDI

University of Naples “Parthenope”University of Naples “Parthenope”

[email protected] [email protected]

[email protected]

Statistical Methods for the analysis of large data-sets University G. d'Annunzio - Chieti-Pescara

September 23, 2009 – September 25, 2009

The literacy divide: territorial differences in the Italian education systemC.QUINTANO, R.CASTELLANO, S.LONGOBARDI 2

Overview

• Investigating the determinants of student achievement

• Highlighting the influence of the TEST-TAKING MOTIVATION on territorial differences

Italian data from the last PISA (Programme for

International Student Assessment) survey edition (2006)

A MULTILEVEL REGRESSION MODEL is applied

This approach is suggested by the hierarchical structure of

the PISA data where students (level-one units) are nested in

schools (level-two units)

DATA

GOAL

METHOD


PISA 2006

400.000 students (21.773 in Italy)

14.300 schools (806 in Italy)

57 CountriesThe survey has

involved

The OECD’s PISA “The OECD’s PISA “Programme for International Programme for International Student AssessmentStudent Assessment” survey is an internationally ” survey is an internationally standardised assessment administered to 15 standardised assessment administered to 15

years old studentsyears old students


The survey assesses the The survey assesses the students’ competencies students’ competencies

in three areasin three areas

Readingliteracy

Scientificliteracy

Mathematical literacy

The OECD collects data on

STUDENT QUESTIONNAIR

E

SCHOOL QUESTIONNAIR

ESCHOOL

CHARACTERISTICS

FAMILY ENVIRONMENT OF

STUDENT

PISA 2006

COGNITIVETEST


International ranking-Reading literacy-

556547

536527

521517

513510508

507507

501501

499498

496495

495494494

492490488

484484483

482479479477

472470

469466

461460

447442440439

417413

410402401401

396393393392

385380

374353

312285

280 300 320 340 360 380 400 420 440 460 480 500 520 540 560 580

CoreaFinlandia

Hong KongCanada

NuovaZelandaIrlanda

AustraliaLiechtenstein

PoloniaPaesi Bassi

SveziaBelgio

EstoniaSvizzeraGiappone

Taiw an-CinaGermania

Regno UnitoDanimarca

SloveniaMacao-Cina

AustriaFranciaIslanda

NorvegiaRep.CecaUngheriaLettonia

LussemburgoCroazia

PortogalloLituania

ItaliaRep.Slovacca

SpagnaGrecia

TurchiaCile

RussiaIsraele

ThailandiaUruguayMessicoBulgaria

GiordaniaSerbia

RomaniaBrasile

IndonesiaMontenegro

ColombiaTunisia

ArgentinaAzerbaijan

QatarKyrgyzstan

ITALY: 469 points

OECD average: 492 points

Italy ranked 33th in reading

among 57 countries


549548

547547

531530

527525525

523522

520520

515513

510506

505504504

502501496495495

492491

490490

486486

480476476

467466

462459

442435

427424

417415

413411

406399

391384

381370370

365318311

280 300 320 340 360 380 400 420 440 460 480 500 520 540 560 580

Taiw an-CinaFinlandia

CoreaHong KongPaesi Bassi

SvizzeraCanada

LiechtensteinMacao-Cina

GiapponeNuovaZelanda

AustraliaBelgio

EstoniaDanimarcaRep.Ceca

IslandaAustria

GermaniaSlovenia

SveziaIrlanda

FranciaPolonia

Regno UnitoRep.Slovacca

UngheriaLussemburgo

NorvegiaLettoniaLituaniaSpagna

AzerbaijanRussia

CroaziaPortogallo

ItaliaGreciaIsraeleSerbia

UruguayTurchia

ThailandiaRomaniaBulgaria

CileMessico

MontenegroIndonesiaGiordaniaArgentina

BrasileColombia

TunisiaQatar

Kyrgyzstan

ITALY:462 points


Italy ranked 38th in

mathematics among 57 countries

International ranking-Mathematical literacy-


563542

534532

531531

530527525

522522

519516

515513

512511

511510

508504

503498496495

493491

490488

488488487

486479

475474473

454438

436434

428424

422421

418412

410393

391390

388386

382349

322

300 320 340 360 380 400 420 440 460 480 500 520 540 560 580

FinlandiaHong Kong

CanadaTaiw an-Cina

EstoniaGiappone

NuovaZelandaAustralia

Paesi BassiCorea

LiechtensteinSlovenia

GermaniaRegno Unito

Rep.CecaSvizzera

AustriaMacao-Cina

BelgioIrlanda

UngheriaSveziaPolonia

DanimarcaFranciaCroaziaIslanda

LettoniaLituania

Rep.SlovaccaSpagna

NorvegiaLussemburgo

RussiaItalia

PortogalloGreciaIsraele

CileSerbia

BulgariaUruguayTurchia

GiordaniaThailandia

RomaniaMontenegro

MessicoIndonesiaArgentina

BrasileColombia

TunisiaAzerbaijan

QatarKyrgyzstan

Italy ranked 31th in Science

among 57 countries

ITALY:475 points


International ranking-Scientific literacy-


The literacy divide

READING MATHEMATICS

SCIENCE

No

rth

Eas

t; 5

06

No

rth

wes

t; 4

94

Cen

tre;

482

So

uth

; 44

3

So

uth

an

d Is

lan

d 4

25

300

350

400

450

500

No

rth

Eas

t; 5

20

No

rth

wes

t; 5

01

Cen

tre;

486

So

uth

; 44

8

So

uth

an

d Is

lan

d 4

25

300

350

400

450

500

No

rth

Eas

t; 5

05

No

rth

wes

t; 4

87

Cen

tre;

467

So

uth

; 44

0

So

uth

an

d Is

lan

d 4

25

300

350

400

450

500


Determinants of learning outcomes

Many studies (Marks, 2006; Korupp et al., 2002) emphasize

the role of socio-economic background for determining

learning outcomes and explaining the territorial differences.

This work aims to asses how much the DIFFERENCES IN

THE TEST-TAKING MOTIVATION “boost” the effect of

socio economic background on the Italian literacy divide


Low stake test

The PISA test is considered as a low stake test since the

students perceive an absence of personal consequences

associated with their test performance.

Without an adequate effort, test performance is likely to

suffer, resulting in the examinee’s test score

underestimating his or her actual level of proficiency

(Wise and De Mars, 2005; Wolf & Smith, 1995; Wolf,

Smith, & Birnbaum, 1995)


Index of student effort

1. the “Test non-response rate” computed on the basis of the number of missing or invalid answers in the PISA cognitive test

An index of student effort is computed on the basis of three variables :

3. the Students’self-report effort in the PISA questionnaire measured on a 10-point scale

2. the “Questionnaire non-response rate” computed on the basis of the number of missing or invalid answers in the PISA student questionnaire (family environment data)

Q

ia

Qi

QiQ

inc N

emI


Effort and performance

80

85

90

95

100

105

110

North West North East Centre South South and islands

Science performance (PISA 2006)

Index of student effort

Variation of the index of student effort in correspondence of the average performance at macro region level. Italy=100

The correlation coefficient between this index and the science performance is equal to 0,553 at national level


A two-level random intercept regression model is adopted

Multilevel approach

Variables at student level

Variables at school level

Error components

εij~ IID-N(0, σ2) U0j~ IID-N(0, τ2)

cov(U0j, εij)= 0

Outcome of ith student of jth school

(Science plausible values)

ijY

m

kkijk x

10

s

ttjt z

1

ij ojU


Student performance

(Science Test score)

Scholastic context-School mean of ESCS-Parents’pressure -Private vs public

Study programmeTechnical, Professional, Vocational, Lower secondary vs Classical studies

Macro areaNorth East, North West, South, South and Islands vs Centre

Scholastic resources-Computers with web-Quality of educational resources-Teacher shoratge

GenderGirls vs boys

Home educational resources

Self confidence in ICT

Hours per week spent on homework

Immigration background1. or 2. generation vs native

LEVEL 2

School j

LEVEL 1:

Student i within School j

Causal structure of multilevel model


Estimation strategy

A block entry approach is adopted (Choen and Choen, 1983)

which consists to the gradual addition of the first and second

level covariates

The process starts with the simplest model, denoted the

empty model, and then progressively adds complexity

introducing school and student variables

In the last model the index of student effort is considered

with 8 school variables and 5 students variables


1 2 3 4 5 6 7 Intercept 460.09*** 466.35*** 468.13*** 495.24*** 511.97*** 514.31*** 499,18***

Student level variables Gender (reference=male) -11.64*** -11.9*** -12.20*** -12.34*** -12.25*** -12,33*** Immigrate (reference=native) -42.44*** -41.78*** -39.98*** -42.45*** -42.47*** -42,41*** Index of Home educational resources 6.72*** 6.73*** 6.79*** 6.73*** 6.74*** 6,81*** Index of self-confidence in ICT 4.71*** 4.71*** 4.69*** 4.69*** 4.69*** 4,67*** Hours per week spent on homework (reference=0-2) 0 -10.52*** -11.21*** -10.86*** -10.95*** -10.88*** -8,86***

02-04 7.19*** 6.77*** 6.47*** 6.62*** 6.55*** 6,14*** >4 9.93*** 9.52*** 9.12*** 9.79*** 9.68*** 9,31***

School level variables Scholastic context

Index of economic, social and cultural status -ESCS- (school average) 91.16*** 60.95*** 34.07*** 36.43*** 31,66*** Parents’ pressure academic standards (reference=low pressure) -0.86*** -1.77*** 1.82*** 4.48*** 1,74*** Private school (reference=public school) -49.17*** -56.17*** -50.58*** -38,43*** Study programme (reference= schools specializing in classical studies or in science education Technical institutes -12.76*** -30.71*** -33.47*** -27,95*** Professional institutes -49.48*** -72.43*** -70.64*** -45,91*** Vocational training -26.64*** -87.01*** -89.63*** -55,09*** Lower secondary school -99.28*** -120.00*** -112.44*** -74,76*** Macro area (reference=Center) North East 42.46*** 36.49*** 27,21*** North West 27.32*** 23.15*** 23,34*** South -31.02*** -31.69*** -16,33*** South and islands -37.78*** -39.14*** -20.55*** Scholastic resources Computers with web 0.08*** 0.06*** Quality of educational resources 0.31*** -1,13*** Teacher shortage 7.74*** 1,77***

Index of student effort- (school average) 56,48***

Block Entry Approach

Student variables

Scholastic context

Study programme

Scholastic resources

School location

Student effort

Empty model


Pre-R2

Proportion reduction in variance or “variance explained” (PRE-R2)

Variance component

EMPTY MODEL

Full model without

effort index

Full model with effort

index

Variance between schools 5,347.98 1,025.56 589.76

Variance within schools 4,674.31 4,470.30 4,474.87

Variance expalined at school level - 80,82% 88,97%


-12,42

-42,41

6,68 5,44

-8,98

6,179,36

-50,00

-40,00

-30,00

-20,00

-10,00

0,00

10,00

20,00

Girls vs Boys Immigrate vsNative

Home educat.Resources(HEDRES)

ICT Self-confidence

Hours forhomeworking 0

vs 0-2

Hours forhomeworking

2-4 vs 0-2

Hours forhomeworking

>4 vs 0-2

Effect of the student variables


Effect of the school variables


• The index of student effort allows to explain a larger amount of variance, indeed after controlling for students motivation the accounted total variance total variance among schools increases from 80% to 89%

• The analysis confirms that the socio-economic contextsocio-economic context plays an important role on the student achievement

• The North-South divide has been overestimated by the PISA test since the score differences are also influenced by the lower effort and engagement of the Southern students

Main findings


36.49

27.223.15 23.34

-31.69

-16.33

-39.14

-20.55

-50

-30

-10

10

30

50

North East vs Center North West vs Center South vs Center South and Islands vsCenter

Model without effort index

Model with effort index

Territorial dummies

With effort index:-25.46%

With effort index:+0.82%




0

20

40

60

80

100

120

140

160

North West North East Centre South South andIslands

Questionnaire non-response rate

Test non-response rate

Collaboration indexes


minmax

min

xx

xxx iiresc

Qinc

Qi II 1

Tinc

Ti II 1

Transformation steps

Rescaling procedure in order to obtain a common scale (0-1) for each indicator

(growing when the non-response rate declines)

COLLABORATION INDEX1)

2)

Documents

The literacy divide: territorial differences in the Italian education system Claudio QUINTANO, Rosalia CASTELLANO, Sergio LONGOBARDI University of Naples