Development of a Math Screening Assessment on a Districtwide Basis

Development of a Math Screening Assessment on a Districtwide Basis

Washington Educational Research AssociationAnnual ConferenceDecember 5-7 2007Mike Jacobsen-Assessment and Curriculum DirectorAndy McGrath-Principal Glacier Middle SchoolWhite River School [email protected]

By The End of This Presentation You Will:

Understand how the district implements a K-10 CBM reading assessment system

Understand how the WRSD developed a math screenerDistrict-wide focusEstablish a committeePilot processFull implementationFall, winter & spring data 06-07Next steps

Basic DefinitionsCBM=Curriculum Based MeasurementDeveloped Initially at University of Minnesota Institute for Research on Learning DisabilitiesMeasures students progress in basic skills using existing curriculumPsychometrically sound

ORF=Oral Reading FluencyWhat is measured is students ability to read out loud, accurately and fluidly

DIBELS=Dynamic Indicators of Basic Early Literacy SkillsResearchers from the University of Oregon coined the phrase

What is CBM?Standard, simple, short duration fluency measures of reading, spelling, written expression and mathematics computationWRSD Reading CBM is very similar to DIBELS with one exceptionWRSD Math Screener is different than DIBELS in math

In reading CBM is oral reading fluency

Measures vital signs of student achievement

Academic thermometer

Big Ideas About CBMExtensive data supporting validity of use as a measure of basic skills

Principle use is in formative evaluation

Sensitive to changes in performance due to instruction

Easy to use within classrooms

Brief

Repeatable

ORF and WASL PERFORMANCE

Fall Words Per Minute

Group

Mean

Level 4

136

Level 3

107

Level 2

70

Winter Words Per Minute

Group

Mean

Level 4

149

Level 3

122

Level 2

87

Spring Words Per Minute

Group

Mean

Level 4

162

Level 3

137

Level 2

103

-Stage, S., & Jacobsen, M. D. (2001) The association between performance on the Washington Assessment of Student Learning and growth in oral reading fluency. School Psychology Review, Vol. 30, No. 3.

ORF and Other Reading Tests1999- 3rd Grade Qualitative Reading Inventory to 3rd Grade ORF=.89

1999-3r Grade ITBS to 3rd Grade ORF=.64

1999-2th Grade Gates-MacGinitie to 2nd Grade ORF=.84

1999-3rd Grade Gates-MacGinitie to 3r Grade ORF=.77

1999-4th Grade Gates-MacGinitie to 4th Grade ORF=.64

1999-5th Grade Gates-MacGinitie to 5th Grade ORF=.86

ORF and WASL Relationships1998- 4th Grade WASL to 5th Grade ORF=.70

1999-4th Grade WASL to 4th Grade ORF=.51

2003-6th Grade ORF to 7th Grade WASL=.68



ORF and WASL Relationships

Chart1

95646

949092

132118121

154134147

ESD

ASD

WRSD

Words Correct Per One Minute

WASL/ORF 98/99

Sheet1

ESDASDWRSD

WASL-195646

WASL-2949092

WASL-3132118121

WASL-4154134147

Sheet1

ESD

ASD

WRSD


WASL/ORF 98/99

Sheet2

Sheet3

CBM ORF/WASL

Sheet1

NameWASLCBM% of Passing WASL>=135 WPM>=120 WPM=135 WPM>=120 WPM

Why Assess Computational Fluency?

Many of the difficulties children have in arithmetic result from not understanding number ideas supposedly learning at an earlier time

Engelhart, Ashlock & Wiebe, 1984

In most cases the precision and fluency in the execution of the skills are the requisite vehicles to convey the conceptual understanding.H. Wu, 1999

White River School District Assessment ProcessImplemented during the 98-99 school year for K-6 Reading6th-8th grade added 20029th/10th grade added 2005

Implemented during the 2006-2007 school year for 1-10 Math screener

Kindergarten students, initial sound fluency, letter names and segmenting phonemes

Grades 1-10 orally read passages from appropriate grade level material

Conducted three times per year during September, January and May

White River School District K-10 CBM Assessment Matrix

Fall

Winter

Spring

Fall

Winter

Spring

Fall

Winter

Spring

Kindergarten Registration

Kindergarten

First Tenth Grade

DIBELS Initial Sound Fluency

DIBELS Letter Naming Fluency

DIBELS Phoneme Seg.

CBM Oral Reading Fluency

Get It, Got It, Go!

Rhyming, Picture Naming

Math Screener-

Computation & Applications

Chart1

530

5689

90111

81107

104128

131149

119140

115132

125134

126140

Fall

Winter


WRSD ORF Fall/Winter 2003-04

Sheet1

FallWinterSpring

First53067

Second5689105

Third90111116

Fourth81107117

Fifth104128141

Sixth131149163

Seventh119140153

Eight115132146

Ninth125134137

Tenth126140142

Sheet1

Fall

Winter

Spring


WRSD ORF Fall/Winter 2003-04

Sheet2

Sheet3

WRSD ORF 2006-2007 Targets

Grade

ORF Targets

ORF Targets

ORF Targets

Fall

Winter

Spring

1st

5

30

70

2nd

55

85

90

3rd

80

110

120

4th

100

110

125

5th

105

130

140

6th

125

150

165

7th

125

140

155

8th

125

140

155

9th

125

140

155

10th

125

140

155

FL

FISF

WL

WISF

WPS

SL

SPS

KG

10

10

35

20

20

50

35

White River School District

K-10 Reading Assessment Flow Chart 8/30/04

All Students

DIBELS

FISF=Fall, Initial Sound Fluency

FL=Fall Letters

WISF=Winter, Initial Sound Fluency

WL=Winter Letters

WPS=Winter Phonemic Segmentation

SL=Spring Letters

SPS=Spring Phonemic Segmentation

Kindergarten

Students less than

Fall:FL=5

FISF=5

WinterWL=15

WISF=10

WPS=10

SpringSL=30

SPS=20

Or by teacher judgement*

Early Reading Intervention-Placement Test

If early literacy difficulties confirmed, Reading Improvement Plan

See Tool Kit

Fall Targets FL=10

FISF=10

Winter Targets WL=35

WISF=20

WPS=20

Spring TargetsSL=50

SPS = 35

Wright Group

K-3 Assessment

Students less than

Fall1

Winter15

Spring 35

Or by teacher judgement

All Students

ORF Fall, Winter, Spring

First Grade

If reading difficulties confirmed, Reading Improvement Plan

See Tool Kit

Fall Target =5

Winter Target =30

Spring Target= 70

All Students ORF

Fall, Winter, Spring

Wright Group

K-3 Assessment

Students less than

Fall35

Winter60

Spring85


Second Grade


See Tool Kit

Fall Target=55

Winter Target=90

Spring Target=105


See Tool Kit

Students less than

Fall55 wpm

Winter68 wpm

Spring89 wpm


All Students CBM


Third Grade

Wright Group

K-3 Assessment

Fall Median WC 73

Winter Median WC 99

Spring Median WC - 109

Analytical Reading Inventory

Students less than

Fall65 wpm

Winter84 wpm

Spring87 wpm


All Students CBM


Fourth Grade


See Tool Kit

Fall Median WC 87

Winter Median WC 112



Students less than

Fall84 wpm

Winter99 wpm

Spring102 wpm


All Students CBM


Fifth Grade


See Tool Kit

Fall Median WC 102




Students less than

Fall97 wpm

Winter111 wpm

Spring117 wpm


Sixth Grade

All Students CBM



See Tool Kit

Fall Median WC 118



Students less than

Fall:93 wpm

Winter:114 wpm

Spring:126 wpm


All Students given CBM Fall, Winter, Spring

Seventh Grade


Gates McGinitie

Gates McGinitie

Gates McGinitie

Students less than

Fall:103 wpm

Winter:116 wpm


Fall MedianWC-126

Winter MedianWC-140

Spring MedianWC-

All Students given CBM


Tenth Grade


Fall MedianWC-125

Winter MedianWC-134

Spring MedianWC-


Students less than

Fall:99 wpm

Winter:113 wpm




Ninth Grade

Fall MedianWC-115

Winter MedianWC-132

Spring MedianWC-146

Gates McGinitie

Students less than

Fall:88 wpm

Winter:107 wpm

Spring:114



Eighth Grade


Fall MedianWC-119

Winter MedianWC-140

Spring MedianWC-153

* Examples of teacher judgement may include the following:

Student scores above the cut off point but teacher still has concerns

Student scores above the cut off point but is a word caller with poor comprehension

Current results dont accurately reflect students ability

1

1


See Tool Kit

Students less than

Fall50

Winter85

Spring95


All Students ORF


Third Grade

Wright Group

K-3 Assessment

Fall Target=80

Winter Target=110

Spring Target=120


Students less than

Fall65

Winter85

Spring95


All Students ORF


Fourth Grade


See Tool Kit

Fall Target=100

Winter Target=110

Spring Target=125


Students less than

Fall85

Winter105

Spring110


All Students ORF


Fifth Grade


See Tool Kit

Fall Target=105

Winter Target=130

Spring Target=140


Students less than

Fall100

Winter130

Spring140


Sixth Grade

All Students ORF



See Tool Kit

Fall Target=125

Winter Target=150

Spring Target=165

Students less than

Fall:93 wpm

Winter:114 wpm

Spring:126 wpm



Seventh Grade


Gates McGinitie

Gates McGinitie

Gates McGinitie

Students less than

Fall:103 wpm

Winter:116 wpm


Fall MedianWC-126

Winter MedianWC-140

Spring MedianWC-



Tenth Grade


Fall MedianWC-125

Winter MedianWC-134

Spring MedianWC-


Students less than

Fall:99 wpm

Winter:113 wpm




Ninth Grade

Fall MedianWC-115

Winter MedianWC-132

Spring MedianWC-146

Gates McGinitie

Students less than

Fall:88 wpm

Winter:107 wpm

Spring:114



Eighth Grade


Fall MedianWC-119

Winter MedianWC-140

Spring MedianWC-153





1

1

All Students given ORF Fall, Winter, Spring

Seventh Grade


Students less than

Fall:100

Winter:115

Spring:125


Gates McGinitie

Students less than

Fall:105

Winter:115

Spring115


Students less than

Fall:105

Winter:115

Spring115


Gates McGinitie

Gates McGinitie

Fall Target=125

Winter Target=140

Spring Target=155

All Students given ORF


Tenth Grade


Fall Target=125

Winter Target=140

Spring Target=155


All Students given ORF


Ninth Grade

Fall Target=125

Winter Target=140

Spring Target=155

Gates McGinitie

Students less than

Fall:100

Winter:115

Spring:115


All Students given ORF Fall, Winter, Spring

Eighth Grade


Fall Target=125

Winter Target=140

Spring Target=155





1

1

Background of Development of the Math Screener: District Learning Improvement PlanningEstablished Fall of 2005Approximately 30 members, teachers, building administrators, central office administrators, parents and school board membersEach building had a stipend position for a teacher who served as DLIP coordinatorMet monthly during the 05/06 school yearThe first meeting was on structure and goals, research on effective schools and role of the district

Background: District Learning Improvement PlanningEstablished Fall of 2005Approximately 30 members, teachers, building administrators, central office administrators, parents and school board membersEach building had a stipend position for a teacher who served as DLIP coordinatorMet monthly during the 05/06 school yearThe first meeting was on structure and goals, research on effective schools and role of the district

Background: District Learning Improvement PlanningThe second meeting focused on district-wide information using the data carousel formatWASL trend data-desegregatedITBSCBMDemographicsSafe and Civil SurveysNine CharacteristicsHealthy Youth SurveySports and Arts program participationCurriculum alignment Professional development

Chart1

15.919.730.634.548.357.761.46561.463.4

7th Math11.712.620.421.115.321.838.749.144.6

10th Math10th Math19.530.826.831.826.129.535.343.6

WRSD 97

WRSD 98

WRSD 99

WRSD 00

WRSD 01

WRSD 02

WRSD 03

WRSD 04

WRSD 05

WRSD 06

% Proficient

WRSD WASL Math Trend

Chart2

69.8

55.4

48.3

48.7

WRSD 06

% Proficient

WRSD WASL Math Trend

Chart3

3rd Math3rd Math3rd Math3rd Math3rd Math3rd Math3rd Math3rd Math3rd Math69.8

15.919.730.634.548.357.761.46561.463.4

5th Math5th Math5th Math5th Math5th Math5th Math5th Math5th Math5th Math55.4


7th Math11.712.620.421.115.321.838.749.144.6


10th Math10th Math19.530.826.831.826.129.535.343.6

WRSD 97

WRSD 98

WRSD 99

WRSD 00

WRSD 01

WRSD 02

WRSD 03

WRSD 04

WRSD 05

WRSD 06

% Proficient

WRSD WASL Math Trend 97-06

Chart4

47.856.753.352.2

69.157.756.855.2

575543.956

64.371.365.439

63.163.361.164.3

61.570.36855.2

59.664.776.963.5

65.667.757.860.6

WRSD 03

WRSD 04

WRSD 05

WRSD 06

WRSD WASL Math Trends 4th 03-06

Chart5

40.255.134

28.43733.4

48.15440.6

37.245.535.2

40.239.445.1

34.946.538.7

44.94646.6

31.738.327.2

WRSD 04

WRSD 05

WRSD 06


Chart6

40.255.134

28.43733.4

48.15440.6

37.245.535.2

40.239.445.1

34.946.538.7

44.94646.6

31.738.327.2

WRSD 04

WRSD 05

WRSD 06


Chart7

43.634.449.328.5

44.432.43644.9

32.726.136.845.8

43.232.752.942.4

45.935.541.133.6

31.133.851.546

38.138.151.850.8

3530.731.942.4

WRSD 03

WRSD 04

WRSD 05

WRSD 06


Chart8

3rd Reading3rd Reading3rd Reading3rd Reading3rd Reading3rd Reading3rd Reading3rd Reading3rd Reading73.7

52.354.661.461.872.571.972.976.982.885

5th Reading5th Reading5th Reading5th Reading5th Reading5th Reading5th Reading5th Reading5th Reading83.6


7th Reading34.728.225.933.740.540.654.467.555.6


10th Reading10th Reading49.455.454.955.644.556.663.385.5

WRSD 97

WRSD 98

WRSD 99

WRSD 00

WRSD 01

WRSD 02

WRSD 03

WRSD 04

WRSD 05

WRSD 06

% Proficient

WRSD WASL Reading Trend 97-06

WRSD 97-07

3rd Math3rd Math3rd Math3rd Math3rd Math3rd Math3rd Math3rd Math3rd Math69.874.3

15.919.730.634.548.357.761.46561.463.456.1

5th Math5th Math5th Math5th Math5th Math5th Math5th Math5th Math5th Math55.464.8


7th Math11.712.620.421.115.321.838.749.144.655.9

8th Math8th Math8th Math8th Math8th Math8th Math8th Math8th Math8th Math48.743.6

10th Math10th Math19.530.826.831.826.129.535.343.639.4

WRSD 97

WRSD 98

WRSD 99

WRSD 00

WRSD 01

WRSD 02

WRSD 03

WRSD 04

WRSD 05

WRSD 06

WRSD 07

WRSD WASL Math 97-07

Sheet1

4th Math7th Math10th Math

WRSD 9715.9

WRSD 9819.711.7

WRSD 9930.612.619.5

WRSD 0034.520.430.8

WRSD 0148.321.126.8

WRSD 0257.715.331.8

WRSD 0361.421.826.1

WRSD 046538.729.5

WRSD 0561.449.135.3

WRSD 0663.444.643.6

3rd Math5th Math6th Math8th Math

WRSD 0669.855.448.348.7

3rd Math4th Math5th Math6th Math7th Math8th Math10th Math

WRSD 9715.9

WRSD 9819.711.7

WRSD 9930.612.619.5

WRSD 0034.520.430.8

WRSD 0148.321.126.8

WRSD 0257.715.331.8

WRSD 0361.421.826.1

WRSD 046538.729.5

WRSD 0561.449.135.3

WRSD 0669.863.455.448.344.648.743.6

WRSD 0774.356.164.84555.943.639.4

4thNumber SenseMeasurementGeometric SenseProbabilityAlgebraic SenceSolves Prob.CommunicatesMakes Con.

WRSD 0347.869.15764.363.161.559.665.6

WRSD 0456.757.75571.363.370.364.767.7

WRSD 0553.356.843.965.461.16876.957.8

WRSD 0652.255.2563964.355.263.560.6

7thNumber SenseMeasurementGeometric SenseProbabilityAlgebraic SenceSolves Prob.CommunicatesMakes Con.

WRSD 0440.228.448.137.240.234.944.931.7

WRSD 0555.1375445.539.446.54638.3

WRSD 063433.440.635.245.138.746.627.2

Number SenseMeasurementGeometric SenseProbabilityAlgebraic SenceSolves Prob.CommunicatesMakes Con.

WRSD 0343.644.432.743.245.931.138.135

WRSD 0434.432.426.132.735.533.838.130.7

WRSD 0549.33636.852.941.151.551.831.9

WRSD 0628.544.945.842.433.64650.842.4

3rd Reading4th Reading5th Reading6th Reading7th Reading8th Reading10th Reading

WRSD 9752.3

WRSD 9854.634.7

WRSD 9961.428.249.4

WRSD 0061.825.955.4

WRSD 0172.533.754.9

WRSD 0271.940.555.6

WRSD 0372.940.644.5

WRSD 0476.954.456.6

WRSD 0582.867.563.3

WRSD 0673.78583.658.855.667.885.5

Sheet2

Sheet3

Background: District Learning Improvement PlanningUsed data from the carousel process to identify three major focus areas:Professional developmentCurriculum alignment-mathMathEach focus area had co-chairsEvery member of the district learning improvement team was on one of the focus area committeesOutcome oriented

Math CommitteeDistrict Math TOSA Kathie Ross and Andy McGrath Co-chaired the Math Committee

Goal:To produce a math assessment that will reliably predict a students success on the WASL (not diagnostic)To produce an assessment that can be given in 20-30 minutes and can be graded in a timely manner without added cost

Math CommitteeTo Achieve This Goal:

We added teachers to the committee from each level primary, intermediate, middle and high schoolCommittee Makeup3 Administrators7 Teachers1 Central Office2 Parents

Math CommitteeASSESSMENT DEVELOPMENTCommittee met for about 2 months discussing the makeup of the assessment:Assessment Structure:20 Total Questions12 Computation8 Applied ProblemsSingle Number Answer

Reading fluency assessment already established in district

Reviewed Fuchs and Fuchs-Monitoring Basic Skills Progress-2nd Ed.

Reviewed Ken Howells et all- Multilevel Academic Skills Inventory-Revised

Next step split subcommittee into three groups:ElementaryMiddle High School

Math Committee-Assessment Cont.The Groups using the GLEs as a guide developed a draft assessment for each grade level

Assessments were brought back to full committee to be discussed and edited

Developed assessments for grades 2 10

Assessment give three times a year in conjunction with reading assessment

Sample Page 4th Grade Computation

Name _______________________________ Date ________________

Score (comp)

Score (app)

4th Grade Fall Math Assessment - Computation

1. 45 + 82 =

Answer ____________

2. 3 x 4 =

Answer ____________

3. 16 - 5 =

Answer ____________

4. 425 + 188 =

Answer ____________

5. 4 x 30 =

Answer ____________

6. 56 - 8 =

Answer ____________

7. 397 + 45 =

Answer ____________

8. 6 x 5 =

Answer ____________

9. 195 - 65 =

Answer ____________

10. $4.54 + $2. 36 =

Answer ____________

11. 6 x 50 =

Answer ____________

12. 43 - 26 =

Answer ____________

Name _______________________________________________ Date ____________________

4th Grade -Fall

Applied Problems

13. Matthew worked 20 hours, 15 hours, and 22 hours during three weeks in June. How many hours did he work altogether?

Answer___________

14. Andy asked his classmates to name their favorite drinks. 5 students voted for milk, 4 students chose water and 8 chose fruit juice. Which drink received the fewest votes?

Answer________

15. Draw a figure with the following attributes:

closed

4 sides

only 2 right angles

16. Which unit of measurement should you use to measure the weight of a penny?

ounce

pound

ton

Answer________

17. The roof of a house needed 140 shingles on one side and 185 on the other. How many shingles were needed altogether?

Answer________

18. Esther made a necklace with 16 beads. Every 4th bead is pink. How many pink beads will she need?

Answer________

19. Mark paid 2 quarters for a candy bar. Ali paid for a drink with 4 dimes. Who spent more money?

Answer________

20. At lunch, Brooke spent $.25 for milk and $.35 for an orange. She gave the cashier $1.00. What was her change?

Answer________

WRSD Math Assessment4th Grade fallRevised 9/7/06

WRSD Math Assessment4th Grade fallRevised 9-7-06

Sample Page 4th Grade Applied Problems

Name _______________________________________________ Date ____________________

4th Grade -Fall

Applied Problems

13. Matthew worked 20 hours, 15 hours, and 22 hours during three weeks in June. How many hours did he work altogether?

Answer___________

14. Andy asked his classmates to name their favorite drinks. 5 students voted for milk, 4 students chose water and 8 chose fruit juice. Which drink received the fewest votes?

Answer________

15. Draw a figure with the following attributes:

closed

4 sides

only 2 right angles

16. Which unit of measurement should you use to measure the weight of a penny?

ounce

pound

ton

Answer________

17. The roof of a house needed 140 shingles on one side and 185 on the other. How many shingles were needed altogether?

Answer________

18. Esther made a necklace with 16 beads. Every 4th bead is pink. How many pink beads will she need?

Answer________

19. Mark paid 2 quarters for a candy bar. Ali paid for a drink with 4 dimes. Who spent more money?

Answer________

20. At lunch, Brooke spent $.25 for milk and $.35 for an orange. She gave the cashier $1.00. What was her change?

Answer________

WRSD Math Assessment4th Grade fallRevised 9/7/06

WRSD Math Assessment4th Grade fallRevised 9-7-06

Pilot ProcessIMPLEMENTATION

An assessment for each grade level completed by April 2005

Piloted last May with volunteer classrooms at least two per grade level

Pilot results to Assessment Office analyze math assessment and reading fluency to see if this would be a good predictor of WASL success

If the assessment proved to be an accurate predictor of WASL success then implement district wide Fall 06

Pilot ProcessManila envelope provided to each pilot teacherDirections for AdministrationInstructions for ScoringStudent Response Sheets Test KeyCopies of student response forms provided to each teacher624 students grades 1-8 participatedStatistically strong relationships with WASL math and spring oral reading fluency demonstrated

Pilot Results

Grade

Total Students

Median Computation

Median Application

Total Comp/Appl

25th PR

Med SCBM

Med WASL

1st

34

10

6

16

14

63

N/a

Predictor

Criteria/Dependent

Correlation

% Contribution

SCBM

Total

.62

38%

SCBM

Applications

.55

31%

SCBM

Computations

.51

26%

Documents

Development of a Math Screening Assessment on a Districtwide Basis