George N. Noell, Ph.D. Louisiana State University and A&M College &

quality

George N. Noell, Ph.D.Louisiana State University and A&M College &

Louisiana Department of Education

Jeanne M. Burns, Ph.D.Louisiana Board of Regents

Office of Special Education ProgramsAnnual Directors’ Conference

July 18, 2011

USING THE LOUISIANA DATA LONGITUDINAL SYSTEM TO

IMPROVE TEACHER PREPARATION

quality

What has Louisiana learned about teacher effectiveness and data systems (2000 – 2011)?

Teacher preparation ISimportant

quality

. . . especially when examining the achievement

of exceptional students.

quality

How does Louisiana know this is true?

• Louisiana can now link teacher preparation programs to the new special education and regular teachers they prepared to the students that the new teachers taught

• Louisiana can provide teacher preparation programs with relevant data pertaining to their effectiveness in preparing new special education and regular teachers

(October 2006) By content areas (Reading, Mathematics, Science, Social Studies, English/language arts)

(March 2011) By grade spans within content areas (Grades 1-5, Grades 4-8, Grades 6-12, & Special Education)

(Fall 2011) By student performance subsets within content areas (Low, Middle, and High)

(Fall 2011) By individual teacher actual student achievement, predicted student achievement, and content standards breakdowns for achievement tests

qualityWhy Louisiana?

• Louisiana has had the political will (1999 – Present) Blue Ribbon Commission for Educational Excellence

(1999 – Present) Commissioners of Higher Education (2000-Present)

(Current: Dr. Jim Purcell; Past: Dr. Sally Clausen & Dr. E. Joseph Savoie)

State Superintendents (2000-Present) (Current Acting: Ollie Tyler; Past: Paul Pastorek & Cecil Picard)

Governors (2000 – 2011) (Current: Bobby Jindal; Past: Kathleen Blanco & Mike Foster)

• Louisiana has had researchers who possess the knowledge and expertise to create a Value-Added Teacher Preparation Assessment and change agents at the State level (Louisiana Board of Regents and Louisiana Department of Education) to implement and support the use of the assessment

quality

Four Levels of Effectiveness forTeacher Preparation Programs in Louisiana

Level 1: Effectiveness of Planning (Redesign of Teacher Preparation Programs)

Level 2: Effectiveness of Implementation (NCATE/TEAC – Comprehensive Assessment System)

Level 3: Effectiveness of Impact (Teacher Preparation Accountability System)

Level 4: Effectiveness of Growth in Student Learning

(Value Added Teacher Preparation Program Assessment)

quality

Redesign of Special Education Programs

• Guidelines and Evaluation – Redesign of Teacher Preparation Programs October 1, 2001 – June 30, 2003 (New Programs: July 1,

2003) Baccalaureate and Alternate: Grades PK-3, 1-5, 4-8, 6-12,

and Mild/Moderate Special Education (Grades 1-12) Evaluation by National Experts

• Guidelines and Evaluation - Redesign of Special Education Programs April 20, 2005 – December 30, 2006 (New Programs: Jan.

1, 2007) Baccalaureate: Early Interventionist (Birth – Five Years)

(Note: Special Education Mild/Moderate Blended Program Withdrawn due to problems with structure)

Alternate : Early Interventionist (Birth – Five Years); Deaf/Hard of Hearing (Grades K-12); Visual Impairments/Blind (Grades K-12); and Significant Disabilities (Grades 1-12);

Advanced Masters – Add on Certification: Deaf/Hard of Hearing; Educational Diagnostician; Gifted; Inclusive Practices; Instructional and Assistive Technology; Learning Disabilities; Mild/Moderate Special Education; Significant Disabilities; and Visual Impairments

Evaluation by National Experts

quality

Redesign of Integrated to MergedSpecial Education Programs

• Guidelines & Evaluation - Redesign of General-Special Education Mild/Moderate: An Integrated to Merged Approach for Grades 1-5, Grades 4-8, & Grades 6-12 August 23, 2009 – June 30, 2010 (New Programs: July 1,

2010) Baccalaureate: General-Special Education Mild/Moderate: An

Integrated to Merged Approach (Grades 1-5; Grades 4-8; and Grades 6-12)

Alternate: General-Special Education Mild/Moderate: An Integrated to Merged Approach (Grades 1-5; Grades 4-8; and Grades 6-12)

All teacher preparation programs were asked to use the document Collaborative Programs in General and Special Teacher Education: An Action Guide for Higher Education and State Policymakers (2007) that was developed by Linda Blanton and Marleen C. Pugach to guide them as they developed their programs

Teams from universities met with national experts at statewide meetings sponsored by the LA Department of Education to develop a better understanding of integrated to merged approaches

Evaluation by National Experts

quality

Four Levels of Effectiveness forTeacher Preparation Programs in Louisiana

Level 1: Effectiveness of Planning (Redesign of Teacher Preparation Programs)

Level 2: Effectiveness of Implementation (NCATE/TEAC – Comprehensive Assessment System)

Level 3: Effectiveness of Impact (Teacher Preparation Accountability System)

Level 4: Effectiveness of Growth in Student Learning

(Value Added Teacher Preparation Program Assessment)

quality

Challenges in Assessing Teacher Preparation

• The challenge of measures– Achievement versus opinions

• Geography• Heterogeneous schools & classes• Data management• Technical issues• The plausible counter factual (Rubin)

quality

Value-Added Teacher PreparationAssessment Model

• Predict achievement of individual students based on prior achievement, demographics, and attendance

• Assess actual student achievement• Link growth of student achievement to new

teachers and teacher preparation programs that taught the new teachers

• Calculate degree to which students taught by new teachers met achievement of similar students taught by experienced teachers

• Act on results

quality

Nesting Structure of Students with Teachers Within Schools

quality Prior

AchievementDemographics

ExpectedAchievement

300

310 Strong Result

Averageoutcome

300

Weak Result

290

Illustration of the Impact of Teacher Preparation on

Average Student Test Performance

Actual Achievement

quality

Tests and Teachers

• Tests - State Achievement Tests (Math, Science,

Social Studies, Reading, & Language Arts - Grades 3-9)

• New Teachers: – 1st and 2nd year teachers with teaching

certificates– Teaching within area of certification

• Experienced Teachers– 3rd or subsequent year teachers with

teaching certificates – Teaching within area of certification

quality

Criteria for Inclusion of Programs in the Assessment

• Inclusion for each content area– Redesigned programs only– 25 or more new teachers in grades 4-9 – Teaching within certification– Remained with student full academic year

• 10 universities and 2 private providers are included in the 2009-2010 results

• 9 universities lacked a sufficient number of new teachers in the content areas to be included – they will be included in the future once they meet the criteria for inclusion

quality

2009-2010 Value-Added Teacher Preparation Assessment Results

EXAMPLE: NORTHWESTERN STATE UNIVERSITYALTERNATE CERTIFICATION PROGRAM (2005-09 Data)

Amount of Growth in Achievement of Grades 4-9 Students Taught by

New Teachers

Science Language Arts

Reading Math Social Studies

Level 1: Growth in achievement GREATER than students taught by EXPERIENCED teachers.

Example:

+3.3Level 2: Growth in achievement COMPARABLE to students taught by EXPERIENCED teachers.

Level 3: Growth in achievement COMPARABLE to students taught by NEW teachers.

Level 4: Growth in achievement BELOW students taught by other NEW teachers.

Level 5: Growth in student achievement SIGNIFICANTLY BELOW students taught by other NEW teachers.

quality

State Research Teams and Universities Using Data to

Create Effective New Teachers

Dr. Vickie Gentry, DeanCollege of Education and

Human DevelopmentNorthwestern State University

Dr. Gerald Carlson, DeanCollege of Education

University of Louisianaat Lafayette

quality

Longitudinal Assessment Results

NORTHWESTERN STATE UNIVERSITYALTERNATE CERTIFICATION PROGRAM

SCIENCE

Amount of Growth in Achievement 2006-07Results

2007-08Results

2008-09Results

2009-10Results


quality



LANGUAGE ARTS

READING

Amount of Growth in Achievement of Grades 4-9 Students Taught by New Teachers

2007-08Results

2008-09Results

2009-10Results


Level 2: Growth in achievement COMPARABLE to students taught by EXPERIENCED teachers.


2007-08Results

2008-09Results

2009-10Results



quality



SOCIAL STUDIES


2006-07Results

2007-08Results

2008-09Results

2009-10Results






quality

Example of Longitudinal Assessment Results


MATHEMATICS


2006-07Results

2007-08Results

2008-09Results

2009-10Results






quality

Example of Grade Span Results (2009-2010 Report)


MATHEMATICS

Mean Effect Estimate for All New Teachers for Mathematics = -3.1

Grade Spans N Effect EstimatesElementary (Grades 1-5) 17 5.9Middle School (Grades 4-8) 7 0.4Secondary (Grades 6-12) 11 -6.4Special Education 27 -6.1Mean Effect Estimate for Northwestern State University for Mathematics (Level 3 Performance Level)

-1.5

CURRENT QUESTION: Are students of new teachers performing equally as well in all mathematical content standard areas on the achievement test (e.g., Number and number relations; algebra; measurement; geometry; data analysis, probability, & discrete math; and patterns, relations, & functions)?

Strength

quality

2009-2010 Value-Added Teacher Preparation Assessment Results

EXAMPLE: UNIVERSITY OF LOUISIANA AT LAFAYETTEALTERNATE CERTIFICATION PROGRAM (2005-09 Data)

Amount of Growth in Achievement of Grades 4-9 Students Taught by

New Teachers

Science Language Arts

Reading Math Social Studies






quality

Example of Grade Span Results (2009-2010 Report)

UNIVERSITY OF LOUISIANA AT LAFAYETTEALTERNATE CERTIFICATION PROGRAM

LANGUAGE ARTS

Mean Effect Estimate for All New Teachers for Language Arts = -2.7

Grade Spans N Effect Estimates

Elementary (Grades 1-5) 58 -6.7

Middle School (Grades 4-8) 8 -4.5

Secondary (Grades 6-12) 25 -1.6

Special Education 9 7.4

Mean Effect Estimate for the University of Louisiana at Lafayette for Language Arts (Level 4 Performance Level)

-5.1

Strength

quality

Example of Content & Achievement Test Breakdown


LANGUAGE ARTS (Grade 4 Achievement Test)

Levels for State Achievement Tests

AverageActual Achievement Levels of Students in Teachers’ Classrooms

Number of New Teachers Whose Actual Student Achievement Scores

were at or above the Predicted Student Achievement Scores

Unsatisfactory 3% 0 out of 1 new teachersApproaching Basic 26% 2 out of 8 new teachersBasic 68% 12 out of 21 new teachersMastery (Proficient) 3% 0 out of 1 new teachersAdvanced --- ---

14 out of 31 new teachers (45%) met their predicted student growth targets for the Grade 4 achievement

test .

quality

Example of Content, Achievement Test, &Content Standard Breakdown


LANGUAGE ARTS (GRADE 4)

Description Write competently

Locate, select, and synthesize

information

Percentage of new teachers whose students scored at or above the state mean on items addressing specific state content standards for language arts on the Grade 4 achievement test

10% 61%

DATA DRIVEN FINDING BY UL-L: Writing competently is a need of students taught by new teachers from UL-L.

quality

Considerations in Making Assessment of

Teacher Preparation Programs Work

• Data quality issues & what do the data mean

• Who taught vs. who tested the student• Modeling issues: who, how, and where?• Is it complicated enough yet or too

complicated?• The time to action in the information

feedback loop

quality

George N. [email protected]

Jeanne M. [email protected]

http://www.regents.la.gov/Academic/TE/Value%20Added.aspx

What do you need to do to make this happen in your states?

Documents

George N. Noell, Ph.D. Louisiana State University and A&M College &