Assessment for Instruction in Assessment for Instruction in Reading in Grades 3-12Reading in Grades 3-12

Joseph Torgesen and Yaacov PetscherJoseph Torgesen and Yaacov Petscher

Florida Center for Reading ResearchFlorida Center for Reading Research

International Dyslexia Association, November, 2009International Dyslexia Association, November, 2009

To describe Florida’s new 3-12 To describe Florida’s new 3-12 assessments for instruction in the assessments for instruction in the context larger issues about formative context larger issues about formative assessmentsassessments

Purpose of this presentation:Purpose of this presentation:

Three questions/issues will be discussedThree questions/issues will be discussed

How does the Fair system fit within the larger How does the Fair system fit within the larger context of a comprehensive formative context of a comprehensive formative assessment plan to improve outcomes in assessment plan to improve outcomes in adolescent literacy?adolescent literacy?

What is the nature of each of the assessments?What is the nature of each of the assessments?

What are the purposes of each of the assessments What are the purposes of each of the assessments within the FAIR system?within the FAIR system?

Torgesen, J. K., & Miller, Torgesen, J. K., & Miller, D. H. (2009). D. H. (2009). Assessments to guide Assessments to guide adolescent literacy adolescent literacy instructioninstruction. Portsmouth, . Portsmouth, NH: RMC Research NH: RMC Research Corporation, Center on Corporation, Center on Instruction.Instruction.

www.centeroninstruction.org

Click on Reading Click on Reading SectionSection

Three important purposes for assessment Three important purposes for assessment in reading with adolescentsin reading with adolescents

1. How successful are schools in helping all 1. How successful are schools in helping all students meet grade level standards in students meet grade level standards in literacy?literacy?

2. Monitor student growth during the year to 2. Monitor student growth during the year to help guide instructional adjustments within help guide instructional adjustments within classrooms or instructional groupsclassrooms or instructional groups

3. Identify students who may need special 3. Identify students who may need special interventions to accelerate growth toward interventions to accelerate growth toward grade level standardsgrade level standards

Three important purposes for assessment Three important purposes for assessment in reading with adolescentsin reading with adolescents

1. How successful are schools in helping all 1. How successful are schools in helping all students meet grade level standards in students meet grade level standards in literacy?literacy?

2. Monitor student growth during the year to 2. Monitor student growth during the year to help guide instructional adjustments within help guide instructional adjustments within classrooms or instructional groupsclassrooms or instructional groups

3. Identify students who may need special 3. Identify students who may need special interventions to accelerate growth toward interventions to accelerate growth toward grade level standardsgrade level standards

The FAIR system for grades 3-12The FAIR system for grades 3-12

Broad Screen/Progress Monitoring Broad Screen/Progress Monitoring Tool Reading Comprehension TaskTool Reading Comprehension Task

3 times a year3 times a year

Targeted Diagnostic Targeted Diagnostic Inventory Inventory

Maze and Word Analysis Maze and Word Analysis TasksTasks Ongoing Ongoing

Progress Progress MonitorinMonitorin

gg

DiagnostiDiagnostic Toolkitc Toolkit

If NecessaryIf Necessary

As neededAs neededAs neededAs needed

Brief Description of tests in the FAIR Brief Description of tests in the FAIR systemsystemBroad Screen/Progress monitoring tool. Is a computer-adaptive, multiple choice test of reading comprehension designed to assess grade level literacy skills in a manner similar to the FCAT.

Maze test – a computer administered maze test to assess reading efficiency and low level comprehension

Word analysis test – a computer-adaptive measure of student knowledge of the phonemic, orthographic, and morphological knowledge necessary for reading words accurately in text

Informal assessment toolkit- Informal diagnostic instruments

Instructional questions the FAIR can Instructional questions the FAIR can answeranswerWhich students may require additional reading instruction (i.e. some type of intervention) because they currently have a low probability of success on the FCAT at their gradelevel? – Reading comprehension testOf students needing some type of intervention, which have relatively serious or pervasive problems with fundamental reading skills? Maze and WA test

Of students needing some type of intervention, which might profit from relatively less intensive interventions that focus primarily on comprehension skills (strategies, knowledge, vocabulary) Maze and WA test

Instructional questions the FAIR can Instructional questions the FAIR can answeranswer

Are students who receive intensive interventions in basic reading skills making acceptable progress in those skills in response to the instruction they are receiving? Maze and WA tests

Are interventions that focus exclusively on improving higher level reading skills effective in increasing students’ reading skills and their probability of success on the FCAT at theend of the year? Reading comprehension testAre students who do not receive interventions making acceptable progress toward grade level standards in reading throughout the year? Reading comprehension test

What specific aspects of basic reading skills or comprehension are students struggling with? Informal assessment toolkit

The FAIR system for grades 3-12The FAIR system for grades 3-12

Broad Screen/Progress Monitoring Broad Screen/Progress Monitoring Tool Reading Comprehension TaskTool Reading Comprehension Task

3 times a year3 times a year

Targeted Diagnostic Targeted Diagnostic Inventory Inventory

Maze and Word Analysis Maze and Word Analysis TasksTasks

If NecessaryIf Necessary

Ongoing Ongoing Progress Progress MonitorinMonitorin

gg

As neededAs needed

DiagnostiDiagnostic Toolkitc Toolkit

As neededAs needed

The Broad Screen/Progress Monitoring The Broad Screen/Progress Monitoring Tool: The RC TaskTool: The RC Task

• 10-30 minute computer-adaptive task of reading 10-30 minute computer-adaptive task of reading comprehensioncomprehension

• A student receives a minimum of one passage and A student receives a minimum of one passage and a maximum of three passages-7 to 9 questions pera maximum of three passages-7 to 9 questions per

• Administered to 3Administered to 3rdrd grade students identified with a grade students identified with a reading deficiency by their districtsreading deficiency by their districts

• Other students may take the assessment at the Other students may take the assessment at the discretion of their districtsdiscretion of their districts

• Predicts student’s FCAT success probability (FSP) Predicts student’s FCAT success probability (FSP)

• In addition to FSP, provides standard scores based In addition to FSP, provides standard scores based on Florida normson Florida norms

• Administered to students performing below grade Administered to students performing below grade level on FCAT in previous year (levels 1 and 2)level on FCAT in previous year (levels 1 and 2)

The Maze TaskThe Maze Task

• Two 3-minute (grade-level) passagesTwo 3-minute (grade-level) passages

• Assesses text reading efficiency (i.e., Assesses text reading efficiency (i.e., reading accuracy and speed, and gist - reading accuracy and speed, and gist - level comprehension)level comprehension)

• Administered 3 times a yearAdministered 3 times a year

• Student responds to embedded cloze Student responds to embedded cloze items within textitems within text

• Provides raw scores and standard scores Provides raw scores and standard scores based on Florida normsbased on Florida norms

TDI: The Word Analysis TaskTDI: The Word Analysis Task

• A 5-15 minute computer-adaptive spelling task A 5-15 minute computer-adaptive spelling task that assesses a student’s understanding of that assesses a student’s understanding of letter/sound correspondence, orthographic letter/sound correspondence, orthographic conventions, and morphologyconventions, and morphology

• Results indicate whether probability of success in Results indicate whether probability of success in reading is hampered by difficulties with word-reading is hampered by difficulties with word-level skills. level skills.

• A student receives a minimum of 5 words and a A student receives a minimum of 5 words and a maximum of 30 wordsmaximum of 30 words

• Provides raw score and standard scores based Provides raw score and standard scores based on Florida normson Florida norms

All FAIR assessments are intended to be All FAIR assessments are intended to be formativeformative (help guide instructional (help guide instructional decisions), rather than decisions), rather than summativesummative (evaluate instructional and learning (evaluate instructional and learning success)success)

A definition from Black and William (2007)A definition from Black and William (2007)

“An assessment is formative to the extent that information from the assessment is fed back within the system and actually used to improve the performance of the system in some way.” (p. 31)

There is a lot of confusion in the literature There is a lot of confusion in the literature about what types of assessments can be about what types of assessments can be called formative, because there are many called formative, because there are many different kinds of instructional decisions to different kinds of instructional decisions to be madebe made

What is assessment for instruction?What is assessment for instruction?

Does this student need to be in an intervention Does this student need to be in an intervention class?class?What kind of intervention class should be What kind of intervention class should be provided?provided?

Does this student need more instruction/practice in Does this student need more instruction/practice in using the prediction strategy to improve using the prediction strategy to improve comprehension?comprehension?What specific aspects of writing does this study need What specific aspects of writing does this study need more instruction in?more instruction in?

Is this student’s current intervention placement Is this student’s current intervention placement strong enough to help him close the gap in strong enough to help him close the gap in reading?reading?

There is a lot of confusion in the literature There is a lot of confusion in the literature about what types of assessments can be about what types of assessments can be called formative, because there are many called formative, because there are many different kinds of instructional decisions to different kinds of instructional decisions to be madebe made

What is assessment for instruction?What is assessment for instruction?

Does this student need to be in an Does this student need to be in an intervention class?intervention class?What kind of intervention class should be What kind of intervention class should be provided?provided?

Does this student need more instruction/practice in Does this student need more instruction/practice in using the prediction strategy to improve using the prediction strategy to improve comprehension?comprehension?What specific aspects of writing does this study need What specific aspects of writing does this study need more instruction in?more instruction in?

Is this student’s current intervention Is this student’s current intervention placement strong enough to help him close placement strong enough to help him close the gap in reading?the gap in reading?

Screening tests– Screening tests– what are initial student abilities in what are initial student abilities in critical areas related to the outcome of interest– critical areas related to the outcome of interest– intervention or no intervention, initial placement, intervention or no intervention, initial placement, etc. etc. RC, Maze, WARC, Maze, WA

What types of formative What types of formative assessments are currently in use?assessments are currently in use?

General outcome progress monitoring/benchmarkGeneral outcome progress monitoring/benchmark – is – is student making acceptable progress in critical student making acceptable progress in critical outcomes? – change class, change teacher, more outcomes? – change class, change teacher, more time, etc. – typically no details about time, etc. – typically no details about whywhy student is student is having problems, having problems, RC, Maze, WARC, Maze, WA

Formal diagnostic assessmentsFormal diagnostic assessments –in depth and reliable –in depth and reliable information about skill and knowledge levels in information about skill and knowledge levels in critical areas – similar to screening tests in types of critical areas – similar to screening tests in types of decisions they informdecisions they inform

Classroom based formative assessments – Classroom based formative assessments – what what strategies is the student using? What are the strategies is the student using? What are the problems with making correct inferences? What problems with making correct inferences? What parts of current objectives has the student parts of current objectives has the student mastered? – information to guide ongoing lesson mastered? – information to guide ongoing lesson planning and instructional adjustments by the planning and instructional adjustments by the teacherteacher

What types of formative What types of formative assessments are currently in use?assessments are currently in use?

Mastery assessmentsMastery assessments within CBM are a more within CBM are a more formalized subset of classroom based formalized subset of classroom based assessments. Typically provided within well assessments. Typically provided within well specified curriculum, and assess mastery of specified curriculum, and assess mastery of steps in complex taskssteps in complex tasks

Informal diagnostic assessmentsInformal diagnostic assessments – what specific – what specific phonics skills has the student mastered, what phonics skills has the student mastered, what kind of questions can they answer from grade kind of questions can they answer from grade level text? – can be used for initial lesson level text? – can be used for initial lesson planning. planning. Informal assessment toolkitInformal assessment toolkit

Evidence/rationale for investment in Evidence/rationale for investment in formative assessmentsformative assessments

“The need for accurate assessments arises because of the enorrmous diversity in the rate of learning and level of literacy skills among adolescents.” (Torgesen & Miller, 2009)

When there is great diversity among students in their rate of learning and level of literacy… “little variation in teaching will always result in great variation in student learning.”

Formative assessments are necessary to help teachers provide instruction that is targeted and paced appropriately

Evidence/rationale for investment in Evidence/rationale for investment in formative assessments – how can formative assessments – how can you tell when you have a good you tell when you have a good system of formative assessment?system of formative assessment?

Traditional ways of evaluating tests include examinations of their reliability and validity

Can you have a reliable and valid test of reading comprehension that does not provide information useful for guiding instruction?

It depends on the instructional question you want to answer–not all tests provide information relevant to all important instructional decisions

Evidence/rationale for investment in Evidence/rationale for investment in formative assessmentsformative assessments

The critical questions for “formative” tests are: 1) what kind of information do they provide, and 2) what is that information useful for?

The most important outcome from formative assessments is improvement in student performance – thus, the ultimate test of their utility can only be made as they are integrated within a system of instructional response.

Evidence/rationale for investment in Evidence/rationale for investment in formative assessmentsformative assessments

“It is impossible to disentangle the impact of formativeassessment from the instruction that follows it. When improved student outcomes are used as the criterion for effective practices, evaluations of classroom-based formative assessment are as much an evaluation of theinstructional adjustments resulting from the assessments as they are of the assessments themselves.” (Torgesen & Miller, 2009)

Evidence/rationale for investment in Evidence/rationale for investment in formative assessmentsformative assessments

Two research summaries of interest

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Educational Assessment: Principals, Policy and Practice, 5, 7–74.A comprehensive review of classroom-based formative assessment. Effect sizes from .4 to .7

Fuchs, L. S., & Fuchs, D. (1986). Effects of systematic formative evaluation: A meta-analysis. Exceptional Children, 53, 199–208.A look at very frequent (2-5 times a week) progress monitoring. ES of .92 when teachers required to follow up in specific ways, .42 when follow-up actions left to teacher judgement

The FAIR system in this context:The FAIR system in this context:

Screening/PM tool (reading comprehension test)– screening and general outcome progress monitoring no more frequent than once a month

Maze test – screening and general outcome progress monitoring no more frequent than once a month

Word analysis test – screening and general outcome progress monitoring no more frequent than three times a year– also can be used diagnostically through error analysis

Informal assessment toolkit- Informal diagnostic instruments

Beginning of yearBeginning of year

Screening Screening and informal and informal diagnosticsdiagnostics

1. General 1. General Screening 2. Screening 2. Targeted Screening Targeted Screening 3. informal 3. informal diagnosticsdiagnostics

Throughout the Throughout the yearyear

Progress Progress AssessmentAssessment

ss

1.1. Classroom Based Classroom Based Formative Assessment Formative Assessment including CBA Mastery including CBA Mastery assessment assessment

2. General Outcome CBM2. General Outcome CBM

End of yearEnd of year

Standards based Standards based accountability accountability

measuresmeasures

End of year End of year “high stakes” “high stakes”

testtest

At this time, the only evidence to support formative assessments At this time, the only evidence to support formative assessments with older students concern classroom based formative with older students concern classroom based formative assessments and very frequent general outcome progress assessments and very frequent general outcome progress monitoring with special needs students.monitoring with special needs students.

QuestionsQuestions//DiscussioDiscussionn