1

Review of Teaching and Assessment Practices of Ontario English, Ontario French, and Quebec French teachers

on PIRLS and SAIP

Principal Researchers: Renée Forgette-Giroux, PhD Marielle Simon, PhDResearch Assistants: Catherine Turcotte, PhD student

Tracy Ferne, MA student Hyeran Choi, statistical analyst

CESC–SSHRC Symposium 2006

2

Research Questions1. According to the PIRLS 2001 and SAIP 2002

responses to the teacher questionnaire, what is the nature and frequency of use of the classroom teaching and assessment practices used by Ontario English, Ontario French, and Quebec French teachers?

2. Are there significant differences in teaching and assessment practices among these three groups of teachers?

3. What classroom teaching and assessment practices have a significant link with student achievement in reading and writing among the three populations?

3

Methodology: Data Sources

Ontario English

Sample size

Ontario French

Sample size

Quebec French

Sample size

student 2774 student 1530 student 2229

teacher 114 teacher 79 teacher 99

PIRLS (Progress in International Reading Literacy Study) teacher questionnaire administered in 2001

4

Methodology: Data Sources

Ontario English

Sample size

13 yr 16 yr

Ontario French

Sample size

13 yr 16 yr

Quebec French

Sample size

13 yr 16 yr

student 551 392 student 246 192 student 632 526

teacher

school

138 182

69 61

teacher

school

31 45

26 25

teacher

school

142 113

66 58

SAIP (School Achievement Indicators Program) teacher questionnaire administered in 2002

5

Methodology: Statistical Analysis

Descriptive analyses (focus of this presentation)

Exploratory factor analyses

Measure of association using Cramer’s V

Multiple correspondence analyses

Hierarchical linear modelling

6

Research Question 2(focus of this presentation)

Are there significant differences in teaching and assessment practices among the three groups?

Measure of Association using Cramer’s V Teaching practices: 20/42 items significant (PIRLS)

Assessment practices: 9/14 items significant (PIRLS) Teaching and assessment practices and beliefs:

22/62 items significant (SAIP)

Cramer’s V of .10 - .20 are considered weak; those between .20 and .40 moderate (Rea and Parker, 1992).

7

Teaching practices (PIRLS reading) (Cramer’s V = .391) Q13c

Use of long texts with chapters

0

5

10

15

20

25

30

35

40

45

50

EVERYDAY ORALMOSTEVERY

DAY

ONCE ORTWICE A

WEEK

ONCE ORTWICE AMONTH

NEVEROR

ALMOSTNEVER

frequency

perc

en

tag

e Ontario-anglais

Ontario-français

Québec-français

8

Teaching practices (PIRLS reading) (Cramer’s V = .372) Q14

Use of teaching materials for students of different ability levels

0

20

40

60

80

SAMEMATERIAL,SAME

RDG LEVEL

SAMEMATERIAL,DIFFRDG LEVEL,DIFF

SPEED

SAMEMATERIAL,DIFF

RDG LEVEL,SAMESPEED

DIFFMATERIAL,DIFF

RDG LEVEL

frequency

perc

entag

e Ontario-anglais

Ontario-français

Québec-français

9

Assessment practices (PIRLS reading) (Cramer’s V =.32) Q27c

Frequency of use of paragraph-length response about what students have read

0%10%20%30%40%50%60%

AT LEAST ONCEA WEEK

ONCE A WEEKOR TWICE A

MONTH

ONCE ORTWICE A YEAR

NEVER

Frequency

Per

cent

age Ontario-anglais

Ontario-français

Québec-français

10

Assesment practices (PIRLS reading)(Cramer’s V =.31) Q26a

Emphasis placed on diagnostic tests to monitor students' progress in reading

0%10%20%30%40%50%60%70%80%

MAJOR EMPHASIS SOME EMPHASIS LITTLE OR NOEMPHASIS

emphasis

perc

enta

ge Ontario-English

Ontario-French

Quebec-French

11

PIRLS teaching items differing among three populations

Items related to types of texts Ont

En

Ont

Fr

Qc

Fr

Cramer’s V

12a. How often do you use textbooks or a reading series (daily)

29% 44% 65% .24

12f. How often do you use a variety of children’s books – novels, non-fiction (daily)

56% 24% 22% .27

13b How often do you have students read other stories – fiction (daily-weekly)

79% 13% 33% .36

13e How often do you have students read plays (rarely) 45% 65% 91% .29

13f How often do you have students read descriptions or explanations (monthly)

34% 56% 53% .25

13h How often do you have students read charts, diagrams, graphs (monthly)

36% 62% 61% .38

12

PIRLS teaching items differing among three populations

Items related to ability levels Ont

En

Ont

Fr

Qc

Fr

Cramer’s V

11a. How often do you teach reading as a whole-class activity (almost always)

21 % 45% 35% .21

11b. How often do you create same ability groups (never)

10% 20% 36% .25

14 Use of different materials with students at different reading levels (see diagram above)

45% 8% 4% .37

Items related to time on reading Ont

En

Ont

Fr

Qc

Fr

Cramer’s V

7a How much time do you spend on language instruction (7 hrs or more a week)

81% 58% 89% .32

9a Whether planned or not, how much time do you spend on reading instruction (3 hrs or more a week)

90% 65% 90% .33

13

General conclusions from PIRLS reading

Teaching Practices

• Ontario English teachers tend to use a variety of texts in the classroom, whereas both Ontario French and Quebec French teachers tend to use manuals or sequential texts.

• Ontario English teachers tend to differentiate their teaching according to the ability levels of students more than do Ontario and Quebec French teachers.

• Ontario French teachers tend to spend less time on the teaching of reading and reading-related activities.

14

General conclusions from PIRLS reading

Assessment Practices

• Ontario and Quebec French teachers tend to emphasize external and classroom assessments to evaluate students’ reading progress more than Ontario English teachers.

• Ontario and Quebec French teachers tend to favour listening to students read aloud to assess students’ performance in reading more often than Ontario English teachers.

• Ontario English teacher tend to rely on oral questioning, teacher conference, and portfolios more often than Ontario and Quebec French teachers to evaluate students’ progress in reading.

15

Teaching and assessment practices ― SAIP Writing (Cramer’s V = .39) Q14w

Frequency with which students have choice of forms

0

20

40

60

80

100

Rarely Monthly Weekly Daily

Frequency

PercentageOnt En

Ont Fr

Qc Fr

16

Teaching and assessment practices ― SAIP Writing (Cramer’s V =.37) Q14v

Frequency with which students have choice of topics

010203040506070

Rarely Monthly Weekly Daily

Frequency

PercentageOnt En

Ont Fr

Qc Fr

17

SAIP items where French teachers and English teachers differ

Items Ont

En

Ont

Fr

Qc

Fr

Cramer’s V

14k. How often do students write descriptively (monthly)

58 % 77% 77% .30

14e. How often do you hold writing conferences with individuals or small groups (rarely)

25% 60% 68% .29

18h. How often do you ask questions for general discussion (daily)

44% 23% 27% .28

14i. How often do students write on demand (monthly)

45% 35% 36% .25

14s. How often do you work with individual students (monthly)

26% 44% 40% .23

14f.How often do students work on aspects of grammar and syntax (weekly)

39% 49% 49% .20

18

SAIP items where all three populations significantly differ

Items Ont

En

Ont

Fr

Qc

Fr

Cramer’s V

14w. How often do students have a choice of forms (rarely)

31% 65% 88% .39

14v. How often do students have a choice of topics (monthly)

53% 64% 44% .37

14m. How often do students edit each other’s writing (monthly)

55% 43% 33% .35

14r. How often do students use workbooks or worksheets (daily)

16% 31% 47% .28

23c. How often do you assign work on long-term writing projects (monthly)

68% 72% 54% .28

19

SAIP items where all three populations significantly differ (cont’d)

Items Ont

En

Ont

Fr

Qc

Fr

Cramer’s V

13c. Students need to know the basic rules of grammar and syntax (strongly agree)

39% 62% 73% .24

23d. How often do you assign work in groups on long-term writing projects (rarely)

45% 54% 74% .21

14a. How often do students write essays (monthly)

57% 66% 84% .21

14l. How often do students edit their own writing (monthly)

37% 54% 67% .21

18k. How often do students ask questions requiring responses by other students (daily)

52% 41% 28% .20

20

General Conclusions from SAIP Writing

• Ontario and Quebec French teachers tend to emphasize teaching of basic rules of grammar and syntax and to ask students to write short texts on demand with little choice of form and topics.

• Compared to Ontario and Quebec French, Ontario English teachers tend to more often encourage students to write descriptively, to hold writing conferences, and to work with students individually.

• The three populations differ significantly in their use of essay questions, long-term projects, oral questioning, and peer assessment practices to assess writing.