Teaching Reading: Elementary Education Capacity Committee Meeting July 8, 2013 Kathy Pruner Client Relations Director Eric Steinhauer Principal Assessment

Teaching Reading: Elementary Education

Capacity Committee MeetingJuly 8, 2013

Kathy PrunerClient Relations Director

Eric SteinhauerPrincipal Assessment Designer

Copyright © 2013 Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are registered trademarks of Educational Testing Service (ETS). THE PRAXIS SERIES is a trademark of ETS. ETS10083A

• A Reading Test for Ohio

• Testing Ohio Candidates

• Test Development and Validity Evidence

• Test Content and Research Base

• Review of Ohio Passing Score

| 2Copyright © 2013 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are registered trademarks of Educational Testing Service (ETS). PRAXIS and THE PRAXIS SERIES are trademarks of ETS. 22441

Today’s Agenda

Ohio RFP for a Reading Test

• Test selected: Praxis Teaching Reading: Elementary Education

• Candidates: Ohio third grade teachers seeking to meet requirements of ORC 3313.608 Fourth Grade Reading Capabilities (aka Third Grade Reading Guarantee)– Effective July 1, 2014 - earned a passing score on a

rigorous test of principles of scientifically research-based reading instruction

Copyright© 2011 Educational Testing Service. All rights reserved. ETS, the ETS logo and LISENTING. LEARNING. LEADING. are registered trademarks of Educational Testing Service (ETS). THE PRAXIS SERIES is a trademark of ETS.


4Copyright © 2013 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are registered trademarks of Educational Testing Service (ETS). PRAXIS and THE PRAXIS SERIES are trademarks of ETS. 22441

• Designed to measure critical knowledge and skills that elementary teachers will use to support reading development

• 2 ½ hours testing time• Delivered on computer

– Includes questions in two formats:• 90 selected response (S/R) – 75% of the test• 3 constructed response (C/R) - 25% of the test

– Innovative items with audio stimuli and video stimuli, etc.

Teaching Reading: Elementary EducationContent Categories


I. Assessment and Diagnostic Teaching of Reading 27%

II. Reading DevelopmentA. Phonemic Awareness and Oral Language Development B. Phonics and Alphabetic PrincipleC. Word-Analysis Skills and Vocabulary Development

D. Development of Reading Fluency and Reading ComprehensionE. Reading Comprehension Strategies Across Text Types

46%

III. Writing in Support of ReadingA. Interdependence of Reading and Writing DevelopmentB. Reading and Writing as Tools for Inquiry and Research

27%

Teaching Reading: Elementary Education Key Dates


Multi-state standard setting November 2012

Registration begins July 2013

Free test preparation materials available to candidates

July 2013

First administration October 2013

Testing Ohio Candidates


Ohio’s Praxis Website


Registration and Score Reporting

• Registration completed through Praxis website and available 24/7 at http://www.ets.org/praxis/register/

• Test fee is $139• Candidates select school district as score

recipient during test registration process• Scores are reported to school district 15

business days following last day of testing window


http://www.ets.org/praxis/register/

Computer Testing Windows for 2013-2014

• October: 10/7/2013 – 10/19/2013 • December: 12/9/2013 – 12/21/2013• January: 1/13/2014 – 1/25/2014• February: 2/3/2014 – 2/15/2014• April: 4/7/2014 – 4/19/2014• May: 5/12/2014 – 5/24/2014• July: 7/14/2014 – 7/25/2014• August: 8/4/2014 – 8/16/2014



Test Centers for Ohio Candidates

Test Development and Validity Evidence


13

The whole process for teacher tests

1. Job Analysis

2. Creation of Test Specifications

3. Item Creation & Test Assembly

4. Test Administration

5. Item Analysis and Evaluation

6. Scoring

Candidate’s total score is: 170

Decision: PASS

Minimum passing score is: 162

2. Agencies/states set a “cut score”

7. Standard Setting Study

Big picture

14


1. Job Analysis





6. Scoring


Decision: PASS




Big picture

Creating a Rigorous, Research-based Assessment


Domain

•Draft domain created based on National Reading Panel findings, IRA Standards & the CCSS Standards

•Domain revised and confirmed by a National Advisory Committee

•Domain confirmed through a national survey of educators

Specificatio

ns

Test

Job Analysis Report



Domain




Specificatio

ns

•Test Specifications developed—a blueprint used for each form of the test form—by ETS testing experts and the National Advisory Committee

Test

Following a process of Evidence-Centered Design (ECD), the NAC translated the knowledge and skill statements into evidence statements



Domain




Specificatio

ns

•Test Specifications developed—a blueprint used for each form of the test form—by ETS testing experts and the National Advisory Committee

Test

•Test items written and reviewed

•Test forms assembled according to specifications

18


1. Job Analysis





6. Scoring


Decision: PASS




Big picture


Standard Setting

Not Yet Qualified

Qualified

(Still Not Qualified)

Target Candidate(Just Qualified)

Passing Score

Low Score

High Score

Standard Setting Process


The JudgmentsRound 1 (Independent) Discussion & Round 2

The Training

Standard Setting Training Practice

The Just-Qualified Candidate

Breakout Discussions Consensus Definition

The Test

Take the Test Discuss the Content Covered

Standard Setting Technical Report


Test Content and Research Base


Teaching Reading: Elementary Education• Calls for candidates to demonstrate

– Knowledge of how to use diagnostic reading data to differentiate instruction

– Knowledge of instructional strategies to support the development of different literacy skills



– Application of knowledge in situations calling for instructional decision-making

• Analyzing student assessment data• Planning instruction to meet specific student needs• Demonstrating an understanding of instructional

strategies



• 2010 International Reading Association

• Common Core State Standards

– Print Concepts– Phonological Awareness– Phonics and Word Recognition– Fluency– Comprehension – Writing


Technology-enabled Response Formats


Click and drag words from the fifth-grade word wall to answer the questions that follow.

Provide two pairs of homophones from the word wall.



The test categories and content are grounded in scientific

research


Report of the National Reading Panel

Teaching Children to Read

What is the National Reading Panel?


• 1997, initiated by Congress

• Partnership between the Secretary of Education and the NICHD

• Task: Determine the effectiveness of various approaches to teaching children to read. Evaluate readiness for implementation by teachers.

The Process


• Analyzed the 1998 National Research Council report

• 1998 - Series of regional hearings attended by teachers, parents, scientists, students and policymakers

• A rigorous methodology-measureable outcome, experimental design, refereed

The Topics Chosen by the Reading Panel


Phonemic Awareness Instruction

Phonics Instruction

Fluency Vocabulary Instruction

Comprehension Instruction

What were the findings of the National Reading Panel?


A Few Findings from the National Reading Panel:

Children should be explicitly and systematically taught to manipulate phonemes with letters.

Focus on 1 or 2 types of phoneme manipulation rather than multiple types.

Repetition and multiple exposures to vocabulary items are important.



Preventing Reading Difficulties in Young Children

1998 – Board on Behavioral, Cognitive, and Sensory Sciences (BBCSS)

Catherine E. Snow, M. Susan Burns, and Peg Griffin, Editors; Committee on the Prevention of Reading Difficulties in Young Children, National Research Council

Teaching Reading is Rocket Science: What Expert Teachers of Reading Should Know

and Be Able to Do

1999-American Federation of Teachers

Louisa C. Moats

http://sites.nationalacademies.org/DBASSE/BBCSS

Common Themes in Scientifically-Based Reading Research


Sublexical structure of

spoken words

Orthographic system

Frequent practice in reading to

achieve fluency

Exposure to a variety of text

types and complexities

Explicit instruction in

comprehension strategies

Frequent opportunities to read and write

Modification of instruction based on assessment

data

Instruction that explores word relationships

Promoting independent

reading in and out of school

“A critical element for preventing reading difficulties in young children is

the teacher. Central to achieving the goal of primary prevention of reading difficulties is the teacher's knowledge

base and experience…”

Preventing Reading Difficulties in Young Childrenp. 329


Adopting a Passing Score for Ohio

A policy decision to achieve Ohio’s Goals…

Reviewing the Multistate Process

• Two independent panels• 28 educators• Two-day process for each panel• Research-based methodology• Two rounds for judgments

Review of Recommendations

Series4140

145

150

155

160

165

170

175

180

(-1 CSEM)

(-2 CSEMs)

(+2 CSEMs)

RSV

39

(+1 CSEM)

• Score Scale = 100-200. • Multistate panels’ decision was 71 raw score points

out of a possible 107; equivalent to 162 scaled score.

• Ohio ESB decision – set 162 as passing score

Ohio Passing Score Decision

• Ohio ESB voted on June 18, 2013 to adopt 162 as passing score and review Ohio performance data after one year.

• Passing scores can always be revisited to assure it is serving your purposes

Contact Information

Kathy Pruner – [email protected] for Ohio 609-683-2694

Eric Steinhauer – [email protected] Assessment Designer


mailto:[email protected]

mailto:[email protected]

Documents

Teaching Reading: Elementary Education Capacity Committee Meeting July 8, 2013 Kathy Pruner Client Relations Director Eric Steinhauer Principal Assessment