1

The effect of students’ perceptions of the learning

environment on mathematics achievement

Explaining the variance in Flemish TIMSS 2003 data

W. Schelfhout , G. Van Landeghem, A. Van den Broeck, & J. Van Damme, K.U.Leuven

2nd IEA International Research Conference

2

• TIMSS 2003

• Mathematics achievement

• Constructivism

3

TIMSS 2003

4

TIMSS 2003 international

• Flanders: 5213 pupils 8th grade pupils in 276 classes in 148 schools

• Math and science achievement

• Pupils’, teachers’ and principals’ questionnaires

5

TIMSS 2003 Flemish extras

• Parents’ questionnaire

• Additional questions in pupils’, teachers’ and principals’ questionnaires

• Spatial and numerical intelligence test

• Two classes per school

6

A-stream vs. B-stream

Classes Number of schools

Two A 109

One A 10

One B 10

Two B 19

Total 148

A = general B = vocational

7

Mathematics achievement

8

Math achievement

• TIMSS 2003 Rasch score

• 8th grade math achievement in Flanders

• 4908 pupils in 268 classes in 144 schools

• A-stream: 4328 pupils in 224 classes in 119 schools

• B-stream: 580 pupils in 44 classes in 25 schools

9

Math achievement

Basic statistics

N Mean SD

A-stream 4328 152.8 (0.4) 8.3

B-stream 580 137.0 (0.7) 7.2

Variance Components

% pupil % class % school

A-stream 69% 17% 15%

B-stream 80% 2% 18%

10

Intelligence (A-stream)

Basic statistics

N Mean SD

A-stream 4374 52.0 (0.5) 11.6

Variance Components

% pupil % class % school

A-stream 72% 12% 16%

Correlation with math achievement: 0.62

11

Intelligence as a predictor of math achievement (A-stream)

Dependent variable: Math N = 4266 A-stream pupils

Model 1 Model 2

Fixed Intercept 152.8 (0.4) 152.0 (0.2)

Explained variance

INT 0.37 (0.01)

Random 2school 9.9 (2.5) 3.1 (0.9) 68%

2class 11.6 (1.9) 4.0 (0.8) 66%

2pupil 47.1 (1.0) 35.9 (0.8) 24%

Deviance 29023.67 27714.97

12

Constructivist learning environment

13

Measurements

• Pupils’ questionnaire (Flemish part): 33 (4-point) items

• Teachers’ questionnaire (Flemish part): 6 (5-point) items

14

Scales, pupils’ questionnaire• Activation (ACTIV)• Clarity (CLAR)• Authentic (AUTH)• Motivation (MOTIV)• Feedback (FEEDB)• Cooperation (COOP)

• Constructivism (TIMSS 1999) (CP)

15

‘Activation’ scale (11 items, = 0.76)

In the math class … … the teacher asks about relationships between

different parts of the subject material during tasks. (8) … … the teacher gives small clues that help us to find

solutions by ourselves. (22) … … during team work or when I am working on my

own, the teacher inquires after the time I need to solve a problem. (33)

16

‘Clarity’ scale (7 items, = 0.82)

In the math class … … the teacher bears in mind pupils’ remarks when

searching for suitable assignments or practice materials. (3)

… … the teacher keeps the class under control. (9) … … it’s thanks to the teacher’s approach that I

understand the subject matter well. (29)

17

‘Authentic’ scale (3 items, = 0.74)

In the math class … … the teacher gives examples of situations in daily

life where the subject matter can be applied. (1) … each new chapter starts with examples from

daily life that clarify the new subject. (5) …situations are described that can happen in the

real world and that need a mathematical solution. (14)

18

‘Motivation’ scale (4 items, = 0.76)

In the math class … … the teacher makes sure that I get interested in

the subject matter. (2) … the teacher uses an agreeable diversity of

approaches in his/her teaching. (4) … we work in a pleasant manner. (12) … I feel that the subject matter will be useful to me

later. (21)

19

‘Feedback’ scale (3 items, = 0.70)

In the math class … … the teacher explains the solution after an

exercise. (18) … the teacher repeats the subject matter when it is

not properly understood by some pupils. (26) … the teacher clarifies errors in tests. (28)

20

‘Cooperation’ scale (2 items, = 0.74)

In the math class … … we have the opportunity to ask other pupils to

explain their way of solving a problem. (27) … we have the opportunity to discuss our

approach to math problems with other pupils. (32)

21

‘Constructivism’ scale (6 items, = 0.73)

Combines items from the scales Activation (2 items) (15) (33) Clarity (1 item) (3) Authentic (1 item) (5) Cooperation (both items) (27) (32)

22

Scales, pupils’ questionnaireBasic statistics

Scale N Mean SD

Activation 4620 2.56 0.48

Clarity 4690 2.99 0.62

Authentic 4829 2.16 0.74

Motivation 4715 2.38 0.71

Feedback 4855 3.21 0.69

Cooperation 4834 2.22 0.85

CP 4738 2.21 0.63

23

Scales, pupils’ questionnaireVariance components

Scale % pupil % class % school

Activation 87% 6% 7%

Clarity 72% 19% 9%

Authentic 79% 9% 12%

Motivation 79% 17% 4%

Feedback 78% 14% 9%

Cooperation 85% 10% 5%

CP 79% 12% 9%

24

Scale teachers’ questionnaire

• 6 item scale (CT), = 0.74

• Items closely related to CP items

• Range 1 to 5; mean = 3.16; SD = 0.64; N = 256 classes

• Variance components: class 48%, school 52%

25

Class level constructivism variables

8 class level indicators of ‘constructivism’:

• Class means of 7 scales from pupils’ questionnaire

• Scale CT from teachers’ questionnaire

26

Class level constructivism variablesBasic statistics in A-stream

Scale N Mean SD

Activation 227 2.56 0.19

Clarity 227 3.01 0.35

Authentic 227 2.08 0.33

Motivation 227 2.34 0.34

Feedback 227 3.24 0.35

Cooperation 227 2.15 0.36

CP 227 2.15 0.28

CT 216 3.15 0.65

27

Class level constructivism variablesVariance components in A-stream

Scale % class % school

Activation 67% 33%

Clarity 69% 31%

Authentic 69% 31%

Motivation 95% 5%

Feedback 65% 35%

Cooperation 91% 9%

CP 86% 14%

CT 47% 53%

28

Class level constructivism variablesCorrelations in A-stream

CLAR AUTH MOTIV FEEDB COOP CP CT

ACTIV 0.80 0.56 0.72 0.77 0.57 0.78 0.16

CLAR 0.47 0.75 0.88 0.55 0.71 0.16

AUTH 0.59 0.41 0.40 0.69 0.14

MOTIV 0.67 0.60 0.76 0.14

FEEDB 0.52 0.66 0.12

COOP 0.85 0.19

CP 0.19

29

Class level constructivism variablesCorrelations with class mean math achievement (A-stream)

ACTIV 0.12

CLAR 0.14

AUTH -0.16

MOTIV 0.00

FEEDB 0.09

COOP -0.04

CP -0.14

CT 0.12

30

Single predictor modelsExample

Dependent variable: Math N = 4328 A-stream pupils

Model 1 Model 2

Fixed Intercept 152.8 (0.4) 152.8 (0.4)

ACTIV 2.9 (1.7)

Random 2school 9.9 (2.5) 9.6 (2.4)

2class 11.6 (1.9) 11.5 (1.9)

2pupil 47.0 (1.0) 47.0 (1.0)

Deviance 29431.02 29428.04

31

Single predictor modelsSummary

Variable N Coeff. p-value

Activation 4328 2.9 (1.7) 9%

Clarity 4328 1.8 (0.9) 6%

Authentic 4328 -1.5 (1.0) 14%

Motivation 4328 -0.1 (0.9) 89%

Feedback 4328 1.1 (0.9) 25%Cooperation 4328 -0.8 (0.9) 39%

CP 4328 -1.9 (1.1) 10%

CT 4088 0.9 (0.5) 9%

32

Conclusion

• Major intake differences between classes and schools (cf. intelligence)

• Indications of marginally significant effects of some aspects of teaching as perceived by the students: activation and clarity