AWSP Leadership Framework
Topics for this session:
• Background and history
• Principal/AP evaluation - current status – recommendations
• Calibration of elements to arrive at summative score for each of the eight criterion – how are districts handling this issue?
• Evidence collection for student growth areas – Criterion 3, 5 and 8
Clarity toward a desired outcome
Focus . . . Never losing sight of your goal
Success is not accidental . . .
COLLABORATION IS KEY!
Collaboration + Teamwork = Growth
• 8 Teacher Evaluation Criteria• 8 Principal Evaluation Criteria• 4-Tiered Rating System
E2SSB 6696(2010)
• 3 Criteria must include student growth• Up to 3 instructional and leadership
frameworks• Experienced teachers and principals cannot be
basic• Evaluations will be used in HR decisions
SB 5895(2012)
• Summative Scoring Formula• Separate analysis of Student Growth
Criteria• Matrix for analyzing summative and student
growth scores• Options for low student growth scores
NCLB Waiver
(Summer 2012)
Shifting thinking from . . . to . . .
Assessment as Autopsy
Assessment as Diagnostic Tools to Improve Leadership & Instruction
Data AccessUtilizing Data for Improving
Instruction
Educator Quality Educator Effectiveness
Reflection for Improved Individual Teaching and Leadership Practice
Reflection for Improved Collective Teaching and
Leadership Practice
1970 2013
Expectations of Practice
1970 2013
Expectations of Practice
Evaluation Systems
1970 2013
Expectations of Practice
Evaluation Systems
Satisfactory
Unsatisfactory
Distinguished
Proficient
Basic
Unsatisfactory
Did I make the grade? How can I grow?
The AWSP
LeadershipFrameworkTo Support Principal Development
Unsatisfactory
Basic
Proficient
Distinguished
Intent of the AWSP Framework
• Promotes a growth model linked to professional development
• Provides for public and professional accountability
• Allows for greater participation in joint goal setting, meaningful discussions, increased collaboration, planning, teamwork and relationship building
• Provides clarity around common language and expectations
Intent of the AWSP Framework
• Is technically, professionally and legally defensible
• Allows for reflection in professional practice
• Leads to improvements in staff and student outcomes
• Addresses variations in school context
• Is valid and reliable
• Moves from evaluation “to them” to evaluation “with them.”
TPEP Criteria Themes
Leadership Framework Presentation
Research Background
AWSP Framework Intent
Research Background on Principal Evaluation
1. Rigorous empirical evidence regarding best practices in principal evaluation is extremely thin with regard to the influence on important school outcomes.
2. The quality of the conduct of principal evaluation may be more important than its content; strong, trusting and collaborative relationships between principals and their district office evaluators is especially critical to the success of the evaluation process.
Research Background on Principal Evaluation
3. Establishing a balance between the formative and summative functions of evaluation appears to result in greater principal and evaluator ownership and motivation regarding the evaluation process.
4. Principal evaluation systems appear to be most effective when they are based upon clear standards and expectations of performance and aligned with key goals and needs of principals, schools, districts and the community.
Davis, S., Kearney, K., Sanders, N., Thomas, C. & Leon, R. (2011). The policies and practices of principal evaluation: Executive summary. San Francisco, CA: WestEd.
AWSP Leadership FrameworkContents
History and Introduction
8 Criteria (Culture, Safety, Data, Curriculum, Evaluation, Resources, Community, Gap)
Reflection ConsiderationsRubricsResearchResources
Certification (ISLLC) and Evaluation (AWSP Criteria) Comparison
Criterion
Examples of
knowledge and skills that apply
to the criterion
Evidenceto assist
placementon therubrics
Context
Criterion 1 Research | Creating a Culture
Criterion 1 Research | Creating a Culture
Use of Student Growth Data Student growth data must be a substantial factor in evaluating
principals for at least three of the eight evaluation criteria.
• Criterion 3 – Planning with Data (3.4)
• Criterion 5 – Improving Instruction (5.2)
• Criterion 8 – Closing the Gap (8.3)
AWSP Leadership FrameworkContents
History and Introduction
8 Criteria (Culture, Safety, Data, Curriculum, Evaluation, Resources, Community, Gap)
Reflection ConsiderationsRubricsResearchResources
Certification (ISLLC) and Evaluation (AWSP Criteria) Comparison
Criterion 1 Rubric | Creating a CultureCreating a school culture that promotes the ongoing improvement of learning and teaching for students and staff.
1.1 Develops and sustains focus on a shared mission and clear vision for improvement of learning/ teaching
1.2 Engages in essential conversations for ongoing improvement
1.3 Facilitates collaborative processes leading toward continuous improvement
1.4 Creates opportunities for shared leadership
ELEMENTSMultimedia
(Video, audio)From the Field (Strategies from
practicing principals)
Professional Development
Forms & Surveys(Templates,
protocols, sample goals)
Books & Research
PRINCIPAL PROFESSIONAL GROWTH PLAN
School name, logo, mission sttatement, etc.
District: School: School Year: Principal/Assistant Principal: Evaluator: Date:
Criterion 1: Creating a school culture that promotes the ongoing improvement of learning and teaching for students and staff.
Element SMART Goal Strategies Evidence and Artifacts
Action Steps/Timeline
Progress Monitoring
Date Comments/Feedback/Revisions
Principal Signature:
Supervisor Signature:
Success at the Core: Resources and AlignmentEvolution of New Policy Mandates: Student Growth Measures
Teacher/ Principal Evaluation & Implementation
InstructionLeadership Modules: Leadership Teams and Quality Instruction Instructional Expertise
Instruction
Teacher Development Strategies: Active and Exploratory Instruction
Ailene M. Baxter, EdD, 2012-2013FJH SIP 2012-2013
“We Use Assessment Data
Keeping Students at the Center
to Improve Instruction
Strengthening the Instructional Core
and Close the Achievement Gap”1
Improved Performance Across ALL Student Groups
AssessmentLeadership Modules: Using Data Effectively Common Formative Assessment
Teacher Development Strategies: Authentic and Student-Centered
Assessment
Achievement GapLeadership Modules: Professional Development
Teacher Development Strategies: Support Structures to Help All
Students Succeed
Success at the Core: Leadership Modules and Teacher Development Strategies
Teachers and Principals MUST have a way of improving and elevating practice through ongoing, authentic, engaging, and collaborative professional learning.
TEACHER DEVELOPMENT STRATEGIESASSESSMENT INSTRUCTION ACHIEVEMENT GAP
Authentic and Student-Centered Assessment Active and Exploratory Instruction Support Structures to Help All Students Succeed• Checking for Understanding • Analyzing Data • Differentiating Instruction• Guiding Self-Assessment • Categorizing Ideas • Scaffolding a Lesson• Reviewing Homework • Conducting Investigations • Using Multiple Approaches• Structuring Peer Assessment • Cooperative Learning • Using Rubrics • Facilitating Student-Centered Discussion
• Hands-On Learning • Promoting Inquiry • Teaching as Facilitation • Using Models • Writing with Detail
1 Dr. Gene Sharratt, AWSP 2012
ROADMAP1. Self-Assessment
2. Goal Setting• Professional• Student Growth
3. First Formal Evidence Gathering
4. Student GrowthCheck-in
5. Second Formal Evidence Gathering
6. Student GrowthCheck-in
7. Summative• Criteria• Student Growth
September
October
November
JanuaryFebruary
April
May
Cashmere School District Principal/Teacher Evaluation Roadmap
1. Self-Assessment
2. Goal Setting• Professional• Student Growth
3. First Formal Evidence Gathering
4. Student GrowthCheck-in
5. Second Formal Evidence Gathering
6. Student GrowthCheck-in
7. Summative• Criteria• Student Growth
September
October
November
JanuaryFebruary
April
May
Comprehensive Evaluation CycleEverett School District | 2012-2013
August Presentation• Strategic Plan• Annual Operating Plan• Work Priorities• State of the School Review
Components• School Improvement Plan Models• Updated State Data Information
State of the School Review• Targeted Data Analysis• School Improvement Actions and Interventions• Developing High Performing Teams• Support Needed and Debriefing• Classroom Visits
Second Formal Visit• Review of Goals (Criterion 3, 5 and 8 Required)• Review of other District Priorities• Review of Teacher Observation
Reports• Classroom Visits• School Safety
First Formal Visit• Self-Assessment against State
Criteria• Goal Setting (Criterion 3, 5 and 8 Required)• Classroom Visits• School Safety
Final Evaluation• Summary of Goals Attainment• Summary of Evidence and
Artifacts• Summary of other District
Priorities
School Improvement Planning Support
August Support Visits• Enrollment• Staffing• Other Needs
Continuous Growth
Reflective Practice
Formative Feedback
AWSP Leadership FrameworkContents
History and Introduction
8 Criteria (Culture, Safety, Data, Curriculum, Evaluation, Resources, Community, Gap)
Reflection ConsiderationsRubricsResearchResources
Certification (ISLLC) and Evaluation (AWSP Criteria) Comparison
Evaluation Summative Scoring Process
Criteria 2
Standards
Criteria 1
Criteria 3
Criteria 4
Criteria 5
Criteria 6
Criteria 7
Criteria 8
Evidence
Frameworks+
Student Growth Rubrics
ObservationArtifactsOther evidence relevant to the frameworks
Student Growth Measures
(From 3 specific criteria)
Summative Rating
State determined process
DistinguishedProficientBasicUnsatisfactory
Student Growth Impact Ratings:Low, Average, High
Criterion Rating
Districtdetermined process
DistinguishedProficientBasicUnsatisfactory
The RAW Score ModelPrincipal Criteria* Indicate Criterion embedded with student growth rubrics
Overall Criterion Scores
Criterion 1: Influence, establish and sustain a school culture conducive to continuous improvement 4
Criterion 2: Lead the development and annual update of a comprehensive safe schools plan 4
*Criterion 3: Lead the development, implementation and evaluation of a data-driven plan for improvement 3
Criterion 4: Assist instructional staff in aligning curriculum, instruction and assessment with state and local goals 4
Criterion 5: Monitor, assist and evaluate staff implementation of the school improvement plan and instruction 3
*Criterion 6: Manage human and fiscal resources to accomplish student achievement goals 4
Criterion 7: Communicate and partner with school community members to promote student learning 4
*Criterion 8: Demonstrate a commitment to closing the achievement gap 3
Total Summative Score 29
Evaluators place principals into preliminary summative rating categories based on score bands. As illustrated above, this principal would receive a preliminary overall summative rating of Distinguished.
8-14 15-21 22-28 29-32
1Unsatisfactory
2Basic
3Proficient
4Distinguished
Evaluation Summative Scoring Process
Criteria 2
Standards
Criteria 1
Criteria 3
Criteria 4
Criteria 5
Criteria 6
Criteria 7
Criteria 8
Evidence
Frameworks+
Student Growth Rubrics
ObservationArtifactsOther evidence relevant to the frameworks
Student Growth Measures
(From 3 specific criteria)
Summative Rating
State determined process
DistinguishedProficientBasicUnsatisfactory
Student Growth Impact Ratings:Low, Average, High
Criterion Rating
Districtdetermined process
DistinguishedProficientBasicUnsatisfactory
Student Growth Rubric and RatingStudent Growth Student Growth* Score Based
on Rubric
Criterion 3.4 2**
Criterion 5.2 2**
Criterion 8.3 1**
Student Growth Score 5
*Must include a minimum of two student growth measures (i.e., state-, district-, school-, and classroom-based measures).
** A student growth score of “1” in any of the student growth rubrics will result in a Low growth rating.
Evaluators place principals into summative rating categories based on score bands. As illustrated below, this principal would receive a low student growth rating
3-5 6-9 10-12Low Average High
Summati
ve Rating
Distinguished Proficient Rating1 Year Growth Plan
Distinguished Rating
ProficientProficient Rating1 Year Growth Plan
Proficient Rating
Basic
Basic Rating
Unsatisfactory Unsatisfactory RatingPlan of Improvement
Low Average High
Impact on Student Learning
Basic Rating1 Year Growth Plan
Defining Key Terms• Student Achievement: The status of subject-matter knowledge, understandings, and skills at one point in time.
• Student Growth (Learning): The growth in subject-matter knowledge, understandings, and skill over time.
Standards
Criteria 1
Criteria 3
Criteria 4
Criteria 5
Criteria 6
Criteria 7
Criteria 8
Evidence
Framework Component
s +
Student Growth Rubrics
(3, 5, 8)
ObservationArtifactsOther evidence relevant to the frameworks
Student Growth Measures
Criterion = Summative
Rating
DistinguishedProficientBasicUnsatisfactory
Criteria 6
One Criterion is chosen and approved by
evaluator
Focused Evaluation Summative Scoring Process
Criterion 1
Criterion 2
Criterion 3
Criterion 4
Criterion 5
Criterion 6
Criterion 7
Criterion 8
Focused Evaluation
Criterion 1
Criterion 2
Criterion 3
Criterion 4
Criterion 5
Criterion 6
Criterion 7
Criterion 8
Focused Evaluation
It is student growth, not student achievement, that is relevant in demonstrating impacts teacher and
principals have on students.
Formal Tests in Core Subjects Only
Knowledge and Learning that can be
Measured
All Classroom Learning
State-based
Tools
District and School-Based
Tools
Classroom-
based To
ols
59
Student Growth DataExamples
• State-Based Tools (with proposed changes)– e.g., MSP, HSPE, EOCs, SMARTER Balanced
Assessment Consortium (SBAC)• District-Based Tools
– e.g., MAP, AIMS Web, SBAC interim• School-Based Tools
– e.g., content area, grade-level or other school team assessments
• Classroom-Based Tools– Applies to all teachers
59
Substantial factor Student growth data must be
a substantial factor in evaluating principals for at
least three of the eight evaluation criteria.P.7 (ESSB 5895 PL)
Multiple MeasuresStudent growth data must be based on multiple measures
that include classroom-based, school-based, district-based, and state-based tools.
P. 8 (ESSB 5895 PL)
Student growth means the change in student
achievement between two points in time.
P.8 (ESSB 5895 PL)
3.4 Planning with data
5.2 Improving Instruction
8.3 Closing the gap
3.4 Planning with data
5.2 Improving Instruction
8.3 Closing the gap
3.4 Planning with data
5.2 Improving Instruction
8.3 Closing the gap
3.4 Planning with data
5.2 Improving Instruction
8.3 Closing the gap
DISTRICT DECISION-MAKING MATRIX
Progress Elementary
Schools
Two Points in TimeLeadership Goals
Hardy Middle School
Success High School
Achievement Alternative School
Substantial factor Student growth data must be a substantial factor in evaluating principals for at least three of the eight evaluation criteria.
P.7 (ESSB 5895 PL)
Multiple MeasuresStudent growth data must be
based on multiple measures that include classroom-based, school-based, district-based, and state-
based tools.P. 8 (ESSB 5895 PL)
Two Points in Time
Student growth means the change in student
achievement between two points in time.
P.8 (ESSB 5895 PL)
3.4 Assists staff to use data to guide, modify and improve classroom teaching and learning.
5.2 Assists staff in developing required student growth plan and identifying valid, reliable sources of evidence of effectiveness.
8.3 provides evidence of growth in student learning.
State-based tools, e.g., MSP, HSPE, EOCs, SMARTER Balanced Assessment Consortium (SBAC)
District-based tools, e.g., MAP, AIMS, Web, SBAC interim
School-based tools., e.g., content area, grade-level or other school team assessments
Classroom-based tools (applies to all teachers)
Dynamic Indicators of Basic Early Literacy Skills (DIBELS)
Lexile Scores
Data Walls : (explanation: http://web.nmusd.us/cms/page_view?d=x&piid=&vpid=1264862614295 )
3.4 Planning with data
5.2 Improving Instruction
8.3 Closing the gap
3.4 Assists staff to use data to guide, modify and improve classroom teaching and learning.
5.2 Assists staff in developing required student growth plan and identifying valid, reliable sources of evidence of effectiveness.
8.3 provides evidence of growth in student learning.
State-based tools, e.g., HSPE, EOCs, WELPA, SMARTER Balanced Assessment Consortium (SBAC)
District-based tools, e.g., MAP, AIMS, Web, SBAC interim
School-based tools., e.g., content area, grade-level or other school team assessments
Classroom-based tools (applies to all teachers)
District Reading/writing scores
Scholastic Reading Inventory (SRI) is a reading assessment program which provides immediate, actionable data on students' reading levels and growth over time.
3.4 By April 1, each common content area team will hold a data talk with a building administrator discussing the results of a common formative or summative assessment. This information will be used to plan interventions and inform the next steps of instruction.
5.2 Monthly, each administrator will individually deliver FINC rate information to every teacher with the goal of achieving a 90% passing rate. Teachers will be supported in achieving this goal by having individual conferences with the administrator, assistance in planning interventions and academic support plans for students who are behind.
8.3 Every six weeks disaggregated FINC rate data will be presented to each content area team. Content teams will develop academic intervention and student supports for each subgroup that is below the 90% targeted passing rate.
DISTRICT DECISION-MAKING MATRIX
Elementary
Schools
Leadership Goals
Middle School
High School
Rater Agreement Background
• The TPEP project has relied heavily on the growing body of research, the framework authors and the practical input from practitioners in the pilot sites to create a “working definition” of rater agreement for the 2012-13 school year.
• The new law requires that evaluators of both teachers and principals “must engage in professional development designed to implement the revised systems and maximize rater agreement.”
Rater Agreement Definition
The extent to which the scores between the raters have consistency and accuracy against predetermined standards. The predetermined standards are the instructional and leadership frameworks and rubrics that define the basis for summative criterion level scores.
Stages of Rater Agreement2-3 Day
Foundational
Training
Ongoing Rater
Agreement Training
Topics for this session: What I hope you heard.
• How research supports the framework• Research base is provided for each criterion and each indicator under the
criteria.
• How Washington’s eight principal evaluation criteria are addressed• Seven of the eight Washington criteria came directly from the AWSP
Framework. • There is now a strong match between the framework and the eight criteria.
• Key strengths of the framework• This is a Washington state framework designed by and for Washington
administrators, responsive to Washington state statutes.
• The role of evidence gathering when putting the framework into practice• The intent behind the framework is professional growth. Evidence related
to the various components will not only assist in assigning scores on the rubric, it will lay the foundation for professional growth.
Recommendations and Suggestions for Effective
Principal Evaluations
Putting the Pieces Together
1. Evaluation should stimulate and guide a principal’s professional development.
2. Evaluation protocols should be aligned with important school and student outcomes (e.g., student achievement and effective instruction).
3. Evaluators should acquire appropriate feedback from multiple stakeholders.
4. Evaluations are enriched and strengthened when evidence is collected through multiple methods (e.g. portfolios, self-assessments, 360-degree feedback, and outcome-based assessments).
5. Evaluation systems should be flexible enough to account for variations in school contexts and environments.
6. Principals should be engaged partners in the process of establishing evaluation goals and objectives and assessing their own performance.
7. The quality of the conduct of principal evaluation may be more important than its content; strong, trusting and collaborative relationships between principals and their district office evaluators is especially critical to the success of the evaluation process.
8. Evaluation procedures and tools should be reliable and valid.
9. Evaluation systems should be based on established standards of administrative practice and on objective and measurable performance objectives.
10. School district leaders should regularly assess the alignment between the district’s principal evaluation system and the critical goals and needs of principals, the schools, the district and the community.
10. School district leaders should regularly assess the alignment between the district’s principal evaluation system and the critical goals and needs of principals, the schools, the district and the community.
2013-14 2014-15 2015-16 2016-17
Provisional TeachersFirst Year
Comprehensive Comprehensive Comprehensive Comprehensive
Provisional TeachersSecond Year Comprehensive Comprehensive Comprehensive Comprehensive
Provisional TeachersThird Year Comprehensive Comprehensive Comprehensive Comprehensive
ProbationaryClassroom Teachers
Comprehensive Comprehensive Comprehensive Comprehensive
Non-Provisional or Non-Probationary Classroom Teachers(4 years of satisfactory evaluations)
25% on Comp75% on Focused
25% on Comp75% on Focused
25% on Comp75% on Focused
25% on Comp
75% on Focused
Four Year Implementation Plan
2013-14 2014-15 2015-16 2016-17
Provisional TeachersFirst Year
17 FTERequired: Comp
15 FTERequired: Comp
10 FTERequired: Comp
15 FTERequired: Comp
Provisional TeachersSecond Year
16 FTERequired: Comp
17 FTERequired: Comp
15 FTERequired: Comp
10 FTERequired: Comp
Provisional TeachersThird Year
8 FTERequired: Comp
16 FTERequired: Comp
17 FTERequired: Comp
15 FTERequired: Comp
Total Provisional 41 FTE 48 FTE 42 FTE 40 FTEProbationaryClassroom Teachers
1 FTERequired: Comp
3 FTERequired: Comp
4 FTERequired: Comp
3 FTERequired: Comp
Non-Provisional or Non-Probationary Classroom Teachers(4 years of satisfactory evaluations)
Total: 378 FTE
Comp: 75Focused: 303
Total: 369 FTE
Comp: 110Focused: 259
Total: 374 FTE
Comp: 125Focused: 249
Total: 377 FTE
Comp: 94Focused: 283
Total teachers on a Comprehensive:
117
161 171 137
Total teachers on a Focused:
303 279 259 293
Sample District: Total Number of Classroom Teachers: 420 Total Student FTE: 8,423
Cougar Greats!