SUPPLEMENT TO PHI DELTA KAPPAN · 6 Kappan Professional Development Discussion Guide November 2012 Topic: Values and beliefs Use the Likert scale to rate the degree to which you agree

SUPPLEMENT TO PHI DELTA KAPPAN

PROFESSIONAL DEVELOPMENT DISCUSSION GUIDEBy Lois Brown Easton

for the November 2012 issue

for the for the November 2012 November 2012 issueissue

* *

Using this guide

This discussion guide is intended to assist Kappan readers who want to use articles in staff meetings or university classroom discussions.

Members of Phi Delta Kappa have permission to make copies of the enclosed activities for use in staff meetings, professional development activities, or university classroom discussions. Please ensure that Phi Delta Kappa and Kappan magazine are credited with this material.

All publications and cartoons in Kappan are copyrighted by PDK International, Inc. and/or by the authors. Multiple copies may not be made without permission.

Send permission requests to [email protected].

Copyright Phi Delta Kappa, 2012. All rights reserved.

kappanmagazine.org V94 N3 Kappan Professional Development Discussion Guide 3

Professional Development Discussion Guide/November 2012

Editor’s Note: The November issue of Phi Delta Kappan focuses on teacher evaluation, a topic that affects every K-12 teacher today and those planning to enter the profession. We are deviating from our usual practice of providing guides for a variety of articles in this issue to focus more broadly on the issue of teacher evaluation.

Every teacher should have a voice in this debate, whether at the local, state, or national level. To be a successful advocate, each educator must be clear about the many facets of this complex issue and clear about his or her own beliefs. We hope this guide will help you prepare to advocate for whatever position you may have.

OVERVIEW OF THE ARTICLES

Educators and policy makers are examining teacher evaluation as a way of making sure students learn to their capacity.

Measures of teacher

effectiveness

Teacher evaluation, which is ultimately about the achievement of students, can be

accomplished in a variety of ways

Measures of student

achievement

Multiplemeasures

Other measures of effectiveness, such

as teacher and parent survey results, evidence of planning

and preparation, measures of classroom

environments, examining assignments and other

artifacts, reviewing teaching portfolios, teacher self-reports

and teaching logs, and interviewing teachers

Observations of teachers by trained,

certifi ed, and monitored observers using

assessment tools

Other ways of measuring achievement, such

as looking directly at student work, evaluating

demonstrations and performances,

interviewing students, observing

Test scores, which may be computed as “value-

added”

Does not include “outside” factors affecting

achievement such as SES

May include contextual factors (school conditions for learning); may

include teacher out-of-classroom professional

responsibilities

include teacher out-of-classroom professional

Measures of teacher

effectiveness

Multiplemeasures

4 Kappan Professional Development Discussion Guide November 2012

KEY POINTS

• States have long relied on degrees and certifications to indicate that teachers are proficient. This has not worked.

• Many educators and policy makers agree that “traditional” evaluations are insufficient; they “often consist of cursory classroom visits by principals who declare nearly every teacher good, or at least competent, even in failing schools where few if any children meet basic educational standards” (New York Times, 2012, p. A22).

• The New Teacher Project which published The Widget Effect: Our National Failure to Acknowledge and Act On Differences in Teacher Effectiveness (Weisberg, Sexton, Mulhern, & Keeling, 2009) demonstrated that “In districts that use binary evaluation ratings (generally “satisfactory” or “unsatisfactory”) more than 99% of teachers receive the satisfactory rating. Districts that use a broader range of rating options do little better; in these districts, 94% of teachers receive one of the top two ratings and fewer than 1% are rated unsatisfactory” (p. 6).

• Furthermore, “When all teachers are rated good or great, those who are truly exceptional cannot be formally identified” (Weisberg et al., 2009, p. 6).

• A 1998 study by the National Bureau of Economic Research showed that a difference in teacher quality could account for at least 7.5% of the achievement differences between schools (Ginn, 2012).

• Put another way, according to the Seattle Times, referring to New York Times columnist Nicholas D. Kristof, “having a bottom 1% teacher has the same impact as a student missing 40% of classes” (Chan, 2012).

• According to a Harvard study, “Students assigned to high-VA [value-added] teachers are more likely to attend college, attend higher-ranked colleges, earn higher salaries, live in higher SES neighborhoods, and save more for retirement. They are also less likely to have children as teenagers” (Chetty, Friedman, & Rockoff, Abstract, p. ii).

• The federal government is getting involved in teacher evaluation by requiring that applicants for programs such as Race to the Top, School Improvement Grants, and waivers for No Child Left Behind include a focus on teacher evaluation.

• Education leaders in 20 states at the district and state levels are working to improve student achievement by ensuring that students have effective teachers (Workman, 2012).

• The September 2012 Chicago teachers’ strike was the “first open rebellion of teachers nationwide over efforts to evaluate, punish and reward them based on their students’ scores on standardized tests of low-level basic skills in math and reading” (Strauss, 2012, p. 1).

DEEPEN YOUR THINKING

Choose one or more of these individual inquiry topics for thinking and writing.

1. How are/were you evaluated as a teacher? To what extent does/did evaluation help make you a better teacher?

2. What policies govern teacher evaluation in your district or state? What practices have resulted from these policies?

3. What issues come to mind as you consider the policies and practices of teacher evaluation?

4. What do you think is the upside of evaluating teachers on student achievement measures? What is the downside? What is your evidence?

5. What is the benefit of evaluating teachers by observing them? What is the downside? What is your evidence?

6. What, if any, outside influences on students, such as poverty, should be factored in to student achievement scores?

7. In what ways, other than through observation, should teachers be evaluated? What artifacts might be most useful? How would these artifacts be evaluated?

8. What, if any, school conditions, such as available resources, should be factored in to measures of teacher effectiveness?

9. What, if any, outside-class professional responsibilities of teachers should be factored in to measures of teacher effectiveness?

10. What are the upsides to multiple measures? What are the downsides?


EXTEND YOUR THOUGHTS THROUGH ACTIVITIES FOR GROUP DISCUSSION

The issue of teacher evaluation is complex. With colleagues, consider one or more of the following topics related to teacher evaluation. Working through one topic at a time, select one or more specific issues related to the topic to discuss. Once you have selected a topic and a specific issue, individually respond to the issue statement, using the following Likert scale.

Completely Somewhat Somewhat Completely disagree disagree Neutral agree agree

1 2 4 5 6

Then, jot down a few notes explaining your rating. When everyone has finished rating the issue statement and noting reasons for their ratings, engage in this simple protocol for having a dialogue about the topic according to the issue you have selected.

If the group is large, you may want to have a facilitator who makes sure the group follows the protocol and adheres to the time limits. You may also want to have a summarizer who takes notes during the protocol in order to capture common understandings in Step 4.

1. Share ratings.

(5-10 minutes) A member of the group posts on a piece of chart paper or white board the ratings each group member gave to the issue.

Group members refrain from commenting on their own or others’ ratings.

2. Explain the extremes.

(5–10 minutes) Those whose ratings are the most extreme (1 or 2, 5 or 6) explain their ratings, not defending them, just clarifying their reasons for rating the issue statement as they did. Others listen silently, perhaps taking notes.

3. Engage in dialogue.

(15 minutes) This is not a high-stakes discussion; no decision is expected at the end of the dialogue. Instead, group members take an inquiry approach to the issue, asking for clarification and seeking to understand each viewpoint. They paraphrase what they have heard and ask about assumptions and mental models. They may probe in order to understand, but not in an aggressive way. Above all, they listen carefully and pause before saying anything.

4. Summarize common understandings.

(5 minutes) One member of the group summarizes shared understandings regarding the issue.

5. Apply understandings.

(5 minutes) If appropriate, the group considers how they will use their common understandings. For example, if their district has a teacher evaluation system, group members might plan to investigate that system in light of the topic and issue, as they understood it through this protocol.


Topic: Values and beliefs

Use the Likert scale to rate the degree to which you agree or disagree with these issue statements.

Issue #1: It is necessary to evaluate teachers in order to increase student achievement.

Issue #2: Having an effective teacher evaluation system is the best way to ensure better student achievement.

Issue #3: Evaluation influences effectiveness.

Issue #4: Anything is better than current/historical teacher evaluation policies and practices.

Issue #5: Educators can’t use poverty or other aspects of students’ lives outside school as excuses for low achievement.

Issue #6: Because teachers are not solely responsible for student achievement (Goe, Bell, & Little, 2008), other factors (such as school resources and family supports) affecting achievement should be factored into teacher evaluation.

Issue #7: School demographics should be considered when evaluating teachers because schools are unique, e.g., an “urban school that has a high proportion of student dropouts” might want a different measure of teacher effectiveness than other schools (Goe, Bell, & Little, 2008, p. 48).

Issue #8: Conditions for teaching and learning (such as resource availability) should be considered when evaluating teachers.

Issue #9: Effective teaching results in more than academic achievement (i.e., socializing students, promoting personal and social development), and growth in these should also be evaluated.

Issue #10: Teacher evaluation is the newest “bandwagon.” Educators and policy makers are “jumping the gun” on this one, without sufficient research and with evaluation systems still in their formative years.

Issue #11: New forms of teacher evaluation should not have critical consequences until research on evaluation measures has proven them valid and reliable.

Issue #12: Just because educators and policy makers have technological advances, such as the statistical tool called a value-added measure, they don’t need to use them (Little, Goe, & Bell, 2009).

Issue #13: Performance standards need to be part of a continuum of expectations (see Linda Darling-Hammond’s article in the November 2012 Kappan) that guide teacher preparation programs, licensure requirements, and mentoring experiences, and are used to determine when a teacher is ready to transition from beginning to professional teacher status (perhaps with tenure) and when that teacher is ready to become a master teacher.

Topic: Purpose(s) of a teacher evaluation system


Issue #1: The overall purpose of teacher evaluation is to make sure that students learn optimally.

Issue #2: The purpose of evaluation systems is to identify and act upon variation among teachers by recognizing and rewarding high-performing teachers, helping “promising, but struggling, young teachers,” and removing “disastrous teachers who have no feel for the profession” (New York Times, 2012, p. A22).

Issue #3: The purpose of teacher evaluation is to improve all teachers by focusing on data and characteristics of effective teaching.

Issue #4: The purpose of teacher evaluation is to determine appropriate compensation.


Issue #5: The purpose of teacher evaluation is to determine status change from probationary to professional to master.

Issue #6: The purpose of teacher evaluation is to ensure focus on all students, including those previously ignored.

Topic: The process of developing teacher evaluation systems


Issue #1: Teachers and principals should be involved in developing, monitoring, and improving teacher evaluation systems in their districts and the state.

Issue #2: The federal government should influence development of teacher evaluation systems in states and districts through incentives and funding for programs.

Issue #3: States need to balance their need for accountability with districts’ needs for autonomy (Goe, Holdheide, & Miller, 2011).

Issue #4: The best state-level evaluation system is one that has certain required parameters for how districts come up with their own systems, as opposed to requiring use of the state evaluation system or districts’ developing their own idiosyncratic systems (Goe et al., 2011).

Issue #5: Unions need to work with states and districts to design evaluation systems that ensure student achievement.

Topic: Choosing the best form(s) of evaluation of teacher effectiveness


Issue #1: Scores on standardized, norm-referenced tests provide the best indicator of teacher effectiveness.

Issue #2: Tests and test preparation cause educators to narrow the curriculum to what’s tested; cut instructional time devoted to history, the sciences, the arts, music, drama, physical education, etc.; focus on lower-level rather than higher-level thinking skills; and reduce focus on noncognitive behaviors.

Issue #3: Tests and test preparation cause students to disengage from a limited curriculum and, if passing a test is required for graduation, may cause students to drop out of school.

Issue #4: The percent of teachers whose students are not tested, estimated at 60% to 70% of teachers (Ginn, 2012) limits the effectiveness of a test-score-based approach to teacher evaluation.

Issue #5: A good way to overcome the problem of teachers whose students are not tested is to report test results at the school level rather than individual level.

Issue #6: Using test scores for teacher evaluation is more cost-effective and less intrusive than other methods.

Issue #7: Using value-added methods for test scores is an effective way to evaluate teachers. Value-added methods are defined by Gary W. Ritter and James V. Shuls in their November 2012 Kappan article as “an effort to calculate the value that each individual teacher (or school) adds to the learning of his or her students during a given time period.” Specifically, value-added measures are based on the premise that “students’ prior achievement on standardized tests can be used to predict their achievement in a specific subject the next year. Whether the student met, exceeded, or failed to reach the predicted score forms the basis for the teacher’s effectiveness score” (Goe et al., 2008, p. 42).


Issue #8: Value-added methods aren’t effective because they can’t account for teachers who haven’t had students a full year, haven’t been teaching a grade level or subject long enough to have value-added scores, or are early elementary teachers who don’t have previous teachers’ scores to estimate their own scores.

Issue #9: Value-added scores aren’t very helpful to teachers because they reveal little or nothing about what teacher practices contributed to the scores.

Issue #10: Test scores aren’t helpful, value-added or not, unless students are randomly assigned to teachers.

Issue #11: Looking directly at student work is the best method of using student achievement as a way to gauge teacher effectiveness.

Issue #12: Teaching effectiveness can be described well enough to evaluate teachers in their classrooms. Valid performance standards have been developed by the National Board for Professional Teaching Standards, the National Comprehensive Center for Teacher Quality Center descriptions, the Interstate New Teacher Assessment and Support Consortium, Charlotte Danielson in A Framework for Learning, and the Measures of Effective Teaching (MET).

Issue #13: It is important for teachers’ observation scores to match student achievement scores (Bill & Melinda Gates Foundation, 2012).

Issue #14: Observations can be both reliable and valid if observers (including principals) are intensively trained, certified, have appropriate observation instruments, have their skills recalibrated after training, have a manageable number of teachers to observe, operate within a culture conducive to observation and feedback, and are monitored.

Issue #15: If principals are to observe teachers, they’ll need a lot of training and district support for focusing their time on observations, perhaps in lieu of other responsibilities.

Issue #16: Principals will need to understand the characteristics of effective teaching at a deep level, be able to “know it when they see it,” and be able to describe teacher behaviors and relate them to the characteristics.

Issue #17: Observations require a lot of time and money, but they’re worth it because the “full dynamic of the classroom” is available to observers (Little et al., 2009, p. 11).

Issue #18: Observations are better than test scores because they help educators determine professional learning needs for individual teachers and for a school as a whole; in fact, teachers can cluster for professional learning around common needs.

Issue #19: Observations work because instruments (rubrics) can be customized for the pedagogies related to the arts, physical education, and other nontested subjects.

Issue #20: Observations work because they can be customized to the context, goals, and purposes of a school (for example, a focus on not letting students drop out of school).

Issue #21: Using other artifacts of teacher effectiveness (see box in the schematic on p. 3) along with observations is the best way to evaluate teaching.

Issue #22: A teacher evaluation system should include consideration of out-of-classroom teacher responsibilities, such as participating in professional learning communities and serving on committees, among the criteria for teacher evaluation.

Issue #23: Multiple measures are the best — some combination of student test scores and evaluating teacher performance in the classroom.

Issue #24: The various components of a multiple-measures teacher evaluation system have to be weighted in terms of what’s most important to the school, district, and state.


Issue #25: A multiple-measures approach is confusing to teachers because it sends mixed messages about what’s really important.

Issue #26: Principals who know student test-score results (value-added or not) are biased when they observe teachers.

Topic: Results of a teacher evaluation system


Issue #1: Student achievement will improve with a teacher evaluation system in place.

Issue #2: Replacing an ineffective teacher with an effective one may not be possible, unless a career spectrum (see Linda Darling-Hammond’s November 2012 Kappan article) is developed that focuses on elements of teaching effectiveness from entry into the teacher preparation program to master-teacher status.

Issue #3: Working to improve an ineffective teacher is better than dismissing one (Chetty et al., 2012).

Issue #4: When teacher evaluation is based on test scores, successful educators could be dismissed because they don’t teach to the test.

Issue #5: Teacher evaluation systems may be demoralizing to teachers and principals if test-based measures require focus on tested subjects and test taking.

Issue #6: Educators may be demoralized if they lack trust in the evaluation system or fear they will be unfairly evaluated.

Issue #7: Working alongside incompetent teachers is demoralizing.

Issue #8: A test-based evaluation system may lead to competition rather than collaboration.

Issue #9: A system based on characteristics of effective teaching can lead to effective professional learning.

Issue #10: Compensation will be more fairly awarded to educators with an evaluation system in place.

Issue #11: Policy makers and educators will be satisfied in terms of teacher quality.

Topic: The effects of culture and history of a school on a teacher evaluation system


Issue #1: A history of distrust, anger, and discontent may make it hard for a teacher evaluation system to work.

Issue #2: Educators in a school must have compatible values and beliefs about teaching and learning for a teacher evaluation system to be effective.

Issue #3: Teachers may want to withhold their best teaching strategies rather than share them with others, especially if they perceive the teacher evaluation system as competitive.

Issue #4: Teachers who aren’t accustomed to sharing their own and students’ work won’t be open to observation and feedback.

Issue #5: The school culture must promote teachers as learners alongside their students.

Issue #6: The culture won’t change if teachers believe the teacher evaluation methods are punitive or ineffective.


References

Bill & Melinda Gates Foundation. (2012, January). Gathering feedback about teaching: Combining high-quality observations with student surveys and

achievement gains. Seattle, WA: Author. www.metproject.org

Chan, S.P. (2012, September 18). Discuss: Should teachers get evaluated based on performance? Seattle Times.

Chetty, R., Friedman, J.N., & Rockoff, J.E. (2012, January). The long-term impacts of teachers: Teacher value-added and student outcomes in

adulthood. Cambridge, MA: National Bureau of Economic Research.

Ginn, J. (2012, September/October). Testing teachers: New evaluations focus on educators’ needs to make them better. Capitol Ideas, 55 (5) 32-33.

www.csg.org

Goe, L., Bell, C., & Little, O. (2008, June). Approaches to evaluating teacher effectiveness: A research synthesis. Washington, DC: National

Comprehensive Center for Teacher Quality. www.tqsource.org

Goe, L., Holdheide, L., & Miller, T. (2011). A practical to designing comprehensive teacher evaluation systems: A tool to assist in the development of

teacher evaluation systems. Washington, DC: National Comprehensive Center for Teacher Quality. www.tqsource.org

Little, O., Goe, L., & Bell, C. (2009, April). A practical guide to evaluating teacher effectiveness. Washington, DC: National Comprehensive Center for

Teacher Quality. www.tqsource.org

New York Times. (2012, September 17). In search of excellent teaching. New York Times, p. A22.

Strauss, V. (2012, September 15). What research really says on teacher evaluation. The Answer Sheet: Washington Post Local.

Weisberg, D., Sexton, S., Mulhern, J., & Keeling, D. (2009). The widget effect: Our national failure to acknowledge and act on differences in teacher

effectiveness. New York, NY: The New Teacher Project.

Workman, E. (2012, March). 2012 State of the State addresses: Governors’ top education issues. Denver, CO: Education Commission of the States.


ApplicationsThis Professional Development Guide was created with the characteristics of adult learners in mind (Tallerico, 2005):

• Active engagement • Relevance to current challenges

• Integration of experience • Learning style variation

• Choice and self-direction

As you think about sharing this article with other adults, how could you fulfill the adult learning needs above?

This Professional Development Guide was created so that readers could apply what they have learned to work in classrooms (Marzano, Pickering, & Pollock, 2001):

• Identifying Similarities and Differences • Summarizing and Note-Taking

• Reinforcing Effort and Providing Recognition • Homework and Practice

• Nonlinguistic Representations • Cooperative Learning

• Setting Objectives and Providing Feedback • Generating and Testing Hypotheses

• Cues, Questions, and Advance Organizers

As you think about sharing this article with classroom teachers, how could you use these strategies with them?

References

Marzano, R.J., Pickering, D., & Pollock, J.E. (2001). Classroom instruction that works: Research-based strategies for increasing student achievement. Alexandria, VA: ASCD.

Tallerico, M. (2005). Supporting and sustaining teachers’ professional development: A principal’s guide (pp. 54-63). Thousand Oaks, CA: Corwin.

About the AuthorLois Brown Easton is a consultant, coach, and author with a particular interest in learning designs — for adults and for students. She retired as director of professional development at Eagle Rock School and Professional Development Center, Estes Park, Colo. From 1992 to 1994, she was director of Re:Learning Systems at the Education Commission of the States (ECS). Re:Learning was a partnership between the Coalition of Essential Schools and ECS. Before that, she served in the Arizona Department of Education in a variety of positions: English/language arts coordinator, director of curriculum and instruction, and director of curriculum and assessment planning.

A middle school English teacher for 15 years, Easton earned her Ph.D. at the University of Arizona. Easton has been a frequent presenter at conferences and a contributor to educational journals.

She was editor and contributor to Powerful Designs for Professional Learning (NSDC, 2004 & 2008). Her other books include:

• The Other Side of Curriculum: Lessons From Learners (Heinemann, 2002);

• Engaging the Disengaged: How Schools Can Help Struggling Students Succeed (Corwin, 2008);

• Protocols for Professional Learning (ASCD, 2009); and

• Professional Learning Communities by Design: Putting the Learning Back Into PLCs (Learning Forward and Corwin, 2011).

Easton lives and works in Arizona. Email her at [email protected].

Documents

SUPPLEMENT TO PHI DELTA KAPPAN · 6 Kappan Professional Development Discussion Guide November 2012 Topic: Values and beliefs Use the Likert scale to rate the degree to which you agree