1

Q &A: Summative Assessments and Value-Added We have been asked many questions about the summative testing and our overall program to improve teacher effectiveness. These questions have come from parents, from teachers, from the Board of Education and from members of the community. We have decided to answer all the questions we received, even in cases of overlap, because we want to provide accurate information about these tests. The summative testing and the pay-for-performance program have generated intense interest and in some cases, anxiety or fear. There is much misinformation about them as well. We are using summative testing to make our schools better. Data from these tests will help us strengthen teaching and school management. It will show us where we are doing a good job and where we can improve, and it will point the way to how we can improve. Some context: Summative testing is one part of our broader program to improve teacher effectiveness. In order to measure teacher effectiveness fairly and accurately, we need to know what students are learning. Right now, we can’t do that in most of the classes and courses we offer – more than 90 percent of them do not have a standard test that would allow us to compare across classes and across schools. In some cases, the CMS summative tests will replace teacher-made exams. In others, summative tests will be the only tests that provide data for comparison for a particular grade or course. In all cases, the summative tests are aligned with the North Carolina Standard Course of Study (NCSCOS). The questions and answers that follow are divided into two broad parts: summative testing and pay for performance. Within the summative group of questions, which is first, we have divided them further into subgroups addressing procedure, rationale, costs and other issues. Summative testing

Q and A: Summative Assessments and Value-Added

1. How will these tests help students? These assessments will be used next year by nearly everyone directly involved in a student’s education. Teachers will use data from the assessments to shape instruction and improve the delivery of the curriculum. The results will help teachers with instructional planning for each student next year, because they will show the gaps and strengths from this year. Curriculum and instruction will use them to create professional development for teachers that is based on teachers’ needs. Principals will use the test results to identify areas of teaching strength and areas that need improvement, which will help in placing teachers within a school. Academic facilitators will use them to identify areas where students need improvement. Parents will be able to see how their children are doing and where more work is needed.

2

Summative tests – development

2. What did the field tests look like? Number of questions? Types of questions? All multiple choice? In kindergarten, first and second grades, the tests were comprised of about 30 student tasks per test. In grades three through 12, the tests were about 60 multiple-choice questions.

3. How will the field tests compare to the summatives at the end of the year? All items on the final summative tests will be on grade level, unlike the field test which had items from adjacent grade levels. The final summatives will have about 10 tasks in kindergarten, first and second grades. In grades three through 12, there will be about 40 multiple-choice questions.

4. Did the tests fit the North Carolina curriculum? The field tests and the summatives are aligned with the North Carolina Standard Course of Study. When the state adopts the Common Core Standards (anticipated in 2012), we will align the tests with those.

5. Could there have been more than one correct answer in the multiple choice questions? Were all correct answers taken into account? What about write-in answers? For multiple-choice questions, there was only one right answer. In kindergarten, first and second grades, teachers were provided a list of correct responses.

6. How many tests in a person’s future will use this “bubble” framework for evaluating the work of an employee’s work or in their accreditation for a particular profession? Not all the field tests were multiple-choice, nor are the summatives. Students can expect to be tested regularly throughout their academic years, including collegiate study and advanced degrees, in multiple-choice format. In addition, many licensing exams for professionals, such as architects and lawyers, often use a multiple-choice format.

7. Why aren’t students involved in the design of the tests and in pay for performance? Students are not involved in the test-item design (the questions) because if they know the questions, they may focus on the answers instead of the content the test should be measuring. However, students are involved in the design because they take the field tests which we use to build the final tests. Student evaluation of teachers is one of the measures we are considering in pay for performance.

8. How similar to the North Carolina End-of-Course (EOC) and End-of-Grade (EOG) tests are the field tests/summatives? The tests are very different. The summatives measure different content areas and focus on those objectives that are viewed as important for further study.

9. How similar to IB or AP tests are the field tests/summatives? The AP summative tests will be very similar to the current AP exams.

3

10. What would we have done differently based on what we have learned from the field tests? In hindsight, we should have been more specific about the test delivery in kindergarten through grade two (this was where much instructional time was lost unnecessarily in some schools). We should have requested oral, rather than written responses, from students in kindergarten, first and second grades for science, math and social studies. Our communications with teachers should have been more explicit and our overall communications efforts should have started earlier to give schools time to prepare and plan.

11. What is the difference between a test designed to evaluate how much a student has learned and one to evaluate how effectively a teacher has taught the subject matter? Which are these? A test that accurately measures student learning can also be used to evaluate teacher effectiveness. These tests are designed to measure student knowledge of course content and the North Carolina Standard Course of Study (NCSCOS). They are designed to also yield proficiency and growth data that can be used to calculate a value-added score for teachers.

12. Are the summatives aligned to the North Carolina Standard Course of Study or the

Common Core Standards? The summatives are aligned with NCSCOS now and will be aligned with the Common Core Standards when the state changes to them.

13. The field test had off-grade level items. Will the actual test have these too? No, the final version will only include on-level questions. Part of field testing the items to ensure alignment to the NCSCOS requires that items are included that are both above and below grade level.

14. Why were religious holidays on the tests? Religious holidays are part of the state’s social studies curriculum at the kindergarten level. They were included in the tests at the appropriate level.

15. Who looked at these tests? Each item has been reviewed by staff in accountability and by staff in curriculum and instruction. This insures appropriate alignment to the curriculum and age-level appropriateness. The field-test forms were proofread by two other staff members prior to printing.

16. When you say you have teachers on the teams developing the testing, as I understand it you have 100 teachers. We have 178 schools in CMS. How is that a representation? That is not even one teacher per school. No one was denied participation in the design teams. We sent personal emails to every teacher in November inviting them to get involved. We held four design-team orientations in December and January, and these were attended by more than 300 teachers. Right now, we have about 100 teachers meeting weekly for one to two hours in seven design teams. The team topics are: value-added; contributions to school and

4

professional learning communities; student survey; classroom observations; student work samples; other ideas, and hard-to staff schools and subjects. We have about that many signed up for next year’s design teams, which will address four broad areas (there may be several teams within a particular area): communication/portals; professional development; weighting the measures, and linking evaluation to compensation. We did not turn anyone away from the design teams. Please note, too, that design teams are not the only way to be involved. They require a significant time investment. That’s why we are also having focus groups of teachers like the one that met last week on value-added. In the focus groups, design team members share their best thinking so far with a wider group of teachers and get their feedback. We want everyone to be involved and remain open to any ideas to make that possible.

17. These tests caused instruction to completely cease. We have solicited feedback from the schools on the logistical aspect of the tests. At some schools, there were challenges with scheduling and administering the tests. But in most cases, there was relatively little disruption for students. Schools that allowed only classroom teachers in kindergarten through second grade to administer the assessments generally had more challenges than schools that used additional staff, suggesting that in the future, staffing flexibility could address many of these challenges.

18. Can you validly measure the arts with a multiple-choice test? For some areas, multiple-choice tests may not be the best measurement. In the fall, we plan to have our new assessments in place for the fine and performing arts. These will not be based primarily on multiple-choice items, but rather on performances that align with course curriculum.

19. The seventh-grade tests had missing graphics and passages that were incomplete. This is not correct information. There were no missing graphics or incomplete passages on seventh-grade science or social studies tests. All the tests have been reviewed again as part of the quality check. Summative tests – rationale

20. If there are tests in every subject do you have a sense that teachers will become more or less creative in the classroom? Does teacher creativity matter? Creativity is important in the classroom. So is making sure that the material being taught aligns with the North Carolina Standard Course of Study. An effective teacher will creatively teach the material required by the North Carolina Standard Course of Study. The summative testing does not stifle teacher creativity; it is instead intended to monitor student progress and also ensure that every class offered in CMS aligns with the standard course of study, as required by law. It will also help us determine if there is equity in our classrooms by allowing us to compare what is being taught in various schools for the same subjects. Access to data on student performance should help all teachers direct their creativity to meet student needs.

21. Is there any support outside CMS for adding these summatives?

5

Yes. During our accreditation review, the accrediting agency AdvancED offered some specific suggestions about the need for more district assessments. Here is what the review recommended: “Develop assessments for all subjects and grade levels to more accurately measure instructional and organizational effectiveness. While CMS continues to develop a very robust assessment and internal accountably system, interviews and review of data indicate that some areas of the curriculum and some grade levels are not adequately monitored and student learning measured through the assessment system.”

22. Are these tests related to Race to the Top funding in North Carolina? North Carolina has made a commitment to link student performance with teacher evaluation. Summative tests such as the ones CMS is designing are needed in order to gather student data to do this, because current state tests cover less than 10 percent of the courses offered.

23. Did we find that the tests were too easy, too hard, or just about right? We are analyzing that data now and will share it when analysis is complete.

24. Is there data that clearly demonstrates that tests stimulate students to learn? A recent study by the journal Science found that students who took tests retained the content covered by the tests more effectively than students who used other methods, such as content mapping or extra study time. In CMS, the content covered by the summatives has not been tested previously. We are using the results of the tests to improve our curriculum delivery. Once we test the content again next year, we will be able to determine if our curriculum changes have improved student learning and make further adjustments if necessary to deliver better results.

25. Can we put a hold on any additional testing this year until we get a better handle on what we are doing? It is possible to do this but not advisable. There will never be a convenient opportunity to do this. Delaying these assessments affects our ability to improve our curriculum delivery and our progress on the goals of our strategic plan for 2014 – and this is not a good outcome for our students.

26. Why are students being tested so much? Time on tests has been subject to some exaggeration. Elementary school students spend 22.5 hours taking tests each year (including the End-of-Grade tests). There are 1,035 hours in the school year. Middle school students spend between 20 hours and 31 hours taking tests out of 1,080 hours in the school year. High school testing times vary widely depending on what Advanced Placement and International Baccalaureate classes a student chooses. We don’t feel that that this is too much testing when considered in the context of more than a thousand hours of instruction each year. Each assessment has a specific purpose intended to help students learn more effectively. The summatives are for the purpose of measuring the student’s knowledge of content. Evaluating student progress against common benchmarks allows us to address both deficiencies and subject mastery far better than we can today.

6

27. Why are we so focused on standardized testing when there are so many documents that

indicate that standardized testing is not good for my child’s education? Much of the work dismissing standardized testing documents the results of inappropriate uses, lack of alignment between the assessments and the curriculum or other systemic issues not directly tied to the tests themselves. The new CMS assessments are being designed following the principles of James Popham, a respected educational researcher. Popham, among many others, disputes the inappropriate use of standardized testing. Moreover, an even larger body of research shows the important role of appropriate standardized testing in improving how well schools teach an established curriculum. Our new summative assessments provide insight into areas of our instruction that until now have lacked objective measures of student achievement. Without these assessments, we do not have a way of monitoring how well we are educating our students and what students are learning in each school.

28. My child already takes many tests, AP, IB, EOC, etc. Why is CMS adding so many duplicate tests? Is CMS aware of the number of tests an individual student takes and how large that number can be? These new assessments do not duplicate existing measures. For example: We added a summative test in science for kindergarten, first, second, third, fourth, sixth and seventh grades – but to minimize time away from instruction, not in fifth and eighth grade because there is a state science test for those grades. We have also compiled a list of the tests and the time each requires. The list has been distributed to principals and it is posted on the CMS website so the public can read it.

29. My child is a high flier, one or more grade levels ahead. Can these tests measure student growth for my child? Yes. The tests are developed to measure a wide variation in student performance. When the results of the 2011-2012 assessments are available, we will begin to develop both the growth model and the value-added model using these results. One additional note: At that time, the model will be adjusted to decrease the likelihood that any teacher is penalized for having high- or low-achieving students – the students at the far edges of the variation.

30. In a time of budget cuts, why are we spending so much money on more tests that inhibit a teacher’s ability to teach and take time away from instruction? It is important for everyone to have a realistic picture of how well a student is learning. We have made summative tests a priority because they will help our students and teachers. They are an investment that will improve teaching and learning. The tests don’t hinder classroom learning; they increase it by helping us focus our instruction. The summatives will help us do that by identifying areas of ineffective teaching and addressing them in two ways. First, we can adjust instruction to strengthen teaching where needed the following year. Second, we can recognize the gaps in an individual child’s learning the previous year and use education strategies to close them. Summative tests – cost

7

31. What is the cost of the summative tests for CMS? The development of the test will cost $1.9 million, with an estimated yearly cost to administer them in subsequent years of about $300,000. This is about $2.19 per student. We believe this is a minimal cost for an academic tool that will help us increase student achievement.

32. Where did the money come from to form these tests in the 2010-2011 budget? The money used for this project came from the 2009-2010 budget. It was end-of-year money. Some of it may have been unused by various departments, some of it came from efficiency savings and some from the hiring freeze in non-instructional positions.

33. How much money is in the 2011-2012 budget for these tests and where will it come from? We have budgeted $300,000 for annual costs of printing, administration and analysis. It is part of the Office of Accountability’s annual budget.

34. In reading the paper it was noted that the funding for this project came from funds not used in the 2009-2010 budget. When I read that I gasped aloud. How could we have let so many teachers go last year, and how and why was class size increased, if this type of funding was available? The money used for this project did come from the 2009-2010 budget. But it was not money that could fund ongoing teacher positions (which can decrease class size) because it was what is called non-recurring money. Some of it may have been unused by various departments, some of it came from efficiency savings and some from the hiring freeze in non-instructional positions. Paying for teachers requires a stable source of funding that will continue year after year – and non-recurring funds by definition do not meet this standard. It is never good financial practice to use fund balance (leftover one-time money) for expenses that occur every year, like positions.

35. What is the cost of reviewing the performance testing? We will use existing staff members to analyze the test results. We do not have a precise figure, although it is included in the $1.9 million one-time cost this year, and the $300,000 annual cost in subsequent years. Summative tests – communication

36. How was information about the field tests communicated to teachers and principals? The field tests were given April 6. Principals were informed of the field tests on Dec. 7. Teachers and others were informed of the field tests Feb. 18 (after the dates were set) via an email that went to all staff. In addition, a webinar was held March 10 for principals covering all aspects of the field test. Test coordinators were informed and also given a webinar training March 24 that covered all aspects of the field test.

37. How did we communicate with parents about the field tests? Each school received a parent letter in both English and Spanish to send home to parents of students participating in the field test in March. The letter outlined the purpose of the test and included a fact sheet.

8

Summative tests – use

38. How will the summatives be used? A baseline for individual student’s knowledge? A baseline for teacher performance? How will that information be gathered from the tests? How will we know it is accurate data? We will use the summative test data to measure student growth and evaluate teacher performance. The field-testing process is designed to ensure the validity and reliability of the summative tests.

39. Do these tests measure proficiency? Growth? Both? They will be used to calculate both measures.

40. What feedback will my child get from these tests to increase learning? Today there is no feedback from EOCs and EOGs. We are aware of the feedback issues around EOCs and EOGs. Our summatives will provide feedback on student level mastery of specific learning objectives linked to the North Carolina Standard Course of Study (and eventually the Common Core Standards) at the student level.

41. What is the difference between these tests and the formative tests? Formative tests are given throughout the year to help teachers plan instruction on a daily or weekly basis. They provide information about what part of the grade-level material students know, as well as what parts they don’t know yet. Summative tests are intended to summarize for teachers and administrators what a student has mastered during the entire year.

42. Will these tests help us assess the value of Bright Beginnings before third grade? Yes. At present, there is no summative testing in kindergarten, first or second grade. Thus, the first test of Bright Beginnings comes three years after the student has left the program. The summatives will allow us to assess Bright Beginnings students’ educational progress much sooner, starting in kindergarten.

43. How will these tests help our teachers improve? By providing information about student learning, gaps in student knowledge and student strengths, teachers will be able to identify successful classroom strategies and less successful ones. This information will help teachers learn from their own performance and that of colleagues. The district will be able to target professional development to address areas needing improvement.

44. Many teachers say that there is not a test that can be made that measures their performance. Why is CMS so certain that it can develop such tests? The tests will measure student performance on the North Carolina Standard Course of Study, the required curriculum for all public schools. Teachers are currently expected to teach this curriculum and improve student academic performance but there is no common

9

measurement. So improvement in student performance is one direct comparative measure of how our teachers are performing.

45. How can these tests yield growth if they are only given at the end of the year like final exams? My child may know all of the information when he walks in the class on day one. We will follow the state model to measure growth. We will look at the summatives given in one year and compare them to the summatives given a year later to calculate growth. Statistical data indicates that this comparison is more reliable than measuring from the beginning of the year to the end of the year.

46. How will the summative test help my child’s teacher? The summative tests will give teachers insight into their classroom practices. Teachers will get feedback showing them if instruction on one academic objective was more effective than another. The tests will also identify teachers who are particularly effective in certain objectives, so that they can share what they are doing with others.

47. How will the results of the summative pilot test be used this year? The pilot results will be used for planning instruction for the coming year but individual student results will not be made available this year. However, in the future individual student results, including areas of strength and weakness, will be made available to teachers and students.

48. What about language immersion programs? We are gathering the data from the field test and plan to discuss the results and options for appropriate inclusion of these students in the assessment system. One possibility is that we may give students in lower grades the math, science and social studies assessments in the immersion language.

49. How can you get good data from the pilot tests if the class isn’t completed yet? The purpose of the field tests is to help us build the summative tests by determining what students are learning. The field tests took into account that students were not yet finished with the school year. Summative tests – procedure

50. Were students – especially K-2 students – adequately prepared for the fact that they would be asked questions for which they were not supposed to know the answer? Preparation for students occurred at each individual school and in each classroom. We have not collected data on how students were prepared.

51. Is there any concern that our best teachers might leave because of this kind of testing? Every teacher wants to know that he or she has been successful in transferring knowledge to students. These tests provide a common measure of that work. Our best teachers know and understand that and are not intimidated by this testing.

10

52. How many students did not take the field tests? Were there more absences than usual during the testing? Our attendance data shows that there were fewer absences than usual during the tests. We compared Wednesday, April 6, the day of the summative testing, with March 30, the Wednesday of the week before. Our data shows that 4.6 percent of our students (6,278 students) were absent April 6, compared to 5.1 percent (6,979) March 30.

53. Do we need proctors? Not necessarily. We allow testing without proctors if the teacher administering the test is not the teacher for that subject or class.

54. Did the administration of these tests take more or less time than expected? In most kindergarten, first- and second-grade classes, the tests took more time than expected. We expected the tests to take about 25 minutes and we set a maximum of 50 minutes. However, in some cases the schools allowed the students to continue for 1.5 hours. We will make sure this does not happen with the final summatives.

55. Based on what we learned from the field tests, is there enough time between now and the time for the summatives to make the necessary changes in the tests themselves as well as in their administration? We should be able to make the adjustments if our analysis shows the test questions were valid and effective. We will have the results of that analysis by April 26.

56. How much teacher-led instructional time was lost because of the testing? Because schools were given flexibility in making testing arrangements, the time lost for instruction varied by school. In some cases, it was as little as 90 minutes; in others, it was as much as 30 hours per teacher. We will be establishing test protocols to prevent such large losses of time in the future.

57. How much EC or ESL instructional time was lost because of the testing? We are still collecting this information and will share it when it becomes available.

58. How much administrator time was lost because of the testing? We are still collecting this information and will share it when it becomes available.

59. How will my child be taught when his teacher is giving another student a test? This issue arises mostly in kindergarten through grade two (in higher grades, students are tested simultaneously). We have given schools the flexibility to arrange the logistics of giving the tests. So practice varies by school. Some schools allow only the classroom teacher to administer the assessments. Others used support staff familiar to the students to give the test to each individual student. This allows the teacher to continue normal classroom instruction for the rest of the class. After the summative field tests, we will review the testing plans to ensure that instruction has not been hindered by the testing process. If it has, we will work with schools to prevent that from occurring again by sharing models that worked well.

11

60. Teachers are teaching to the tests in response to the pressure from the district to raise test scores. How do teachers know what is on the tests? Why does CMS allow this to happen? We do not allow or condone ethics violations related to testing and we employ a variety of methods to discourage and identify violations. Among them: We use proctors in classrooms, we have a trained test coordinator at each school and teachers are moved around during testing. In addition, we will vary the examples used in the assessments each year. This encourages teachers to teach to the curriculum, rather than to the specific examples. Teachers are not allowed to review the test or keep a copy of the test.

61. Will the next test in May (and all following tests) be given by the teacher and take a week to administer? Remember, next year there will be fewer teachers and assistants to give the test so it could potentially take even longer than a week to test. In grades three through 12, each test takes about 90 minutes – so we don’t expect the testing period to last a week. In kindergarten through grade two, each test is designed to take about 15 minutes for most students, which is a total of one hour per child. Schools have flexibility in setting the test schedule and the staffing to support it. During field testing, it was determined that modifications are needed to ensure we meet the allotted time. The tests took longer than we expected in some cases this time because schools allowed students more time than we specified to take the test. We are working to develop a test protocol that will prevent this next time.

62. How will the tests in first, second and third grades be administered next year without teacher assistants? We have given schools flexibility in choosing testing logistics for the pilot testing because the answer to this question may vary from school to school. When the pilot tests are complete, we will review the procedures and make sure that every school can handle the needs of testing.

63. Many parents want to opt out of this non-state-mandated test. Why can't they? There is no law or policy in NC that allows a parent to opt children out of the instructional activities of the school. Assessment is as much a part of the instruction as practicing math problems in class or reading the textbook. These assessments are given to help students learn and should be followed just as the rest of the curriculum requirements are.

64. How much time will the summative tests take? The field tests did take longer than expected for K-2. Staff members have made adjustments to ensure students and teachers can complete the necessary tasks in the time allotted. For a child in kindergarten, first and second grades, they should take a total of an hour for most students. In grades three through 12, each summative should take no longer than 90 minutes unless the student needs an accommodation that extends the test time.

65. Do we have to organize testing sessions like we do for state tests?

12

No. The summative tests for grades three and up are designed to be 90 minutes long. More than one test can be given during a school day. The school can follow its normal schedule in many cases.

66. Why did the tests have all these errors? Teachers have raised issues with approximately 50 of the 3,500 items in the field tests. We are checking these items to be sure they are accurate and making corrections where needed. Our goal is to have no errors; however, 50 out of 3,500 on a pilot test is a relatively low error rate. Part of the purpose of a pilot test is to find areas for improvement.

67. It is immoral to assess children so young, and the state has a law against it. There are no state laws or policies that forbid the assessment of young children in school. In fact, testing of young children is an accepted practice and children are frequently assessed before they ever come to kindergarten.

68. These tests caused instruction to completely cease. We have solicited feedback from the schools on the logistical aspect of the tests. At some schools, there were challenges with scheduling and administering the tests. But in most cases, there was relatively little disruption for students. Schools that allowed only classroom teachers in kindergarten through second grade to administer the assessments generally had more challenges than schools that used additional staff, suggesting that staffing flexibility could address many of these challenges.

69. Parents need to see the assessments so they can prepare their children for them. This would nullify the value of the assessment, which is intended to determine what students have learned in the curriculum. We don’t give out copies of any other tests in advance. Summative tests – Board role

70. How was information conveyed to the Board about the summatives? Summative information was included in the Board Update on Dec. 3 and Dec. 17.

71. Are the tests public documents? Can the Board see them? They are not public because knowledge of the test items could encourage teaching to the test questions and affect the effectiveness of the tests. However, Board members may review any of the final summative test forms as they are created.

72. When will the Board get a report on the data gathered from these tests? We are not able to share this data because reports to the Board are public, and we don’t make field/pilot test data public. All of this year’s testing is considered pilot testing. We will share final summative data, however, at the end of the 2011-2012 school year. Pay for performance questions

13

73. When will the Board vote on the use of these tests in the pay for performance plan?

The Board will not vote on these tests. The summative testing is part of educational practice, not a policy matter.

74. How does a teacher’s evaluation affect student growth? A teacher’s evaluation should reflect how effective the teacher is. Teacher effectiveness is the single biggest school-based factor in student learning – so a teacher’s effectiveness matters. The current evaluation tool does not factor in student learning as a measure of teacher effectiveness. We believe that student learning should be part of a teacher’s evaluation because it is an important measure of effectiveness. In addition, North Carolina has committed to linking teacher evaluations to student outcomes as part of the state Race to the Top work.

75. There has been a lot written about how flawed and imprecise the value-added measure is; why is CMS still using it and what is the district doing to improve its accuracy? CMS is using value-added as only one measure of teacher effectiveness. As part of our commitment to continuous improvement, we are constantly reviewing our value-added measure – including having independent experts provide input – to improve its accuracy. The final measure will not be based on a single year because there is the possibility that one year might not be representative. Instead, it will use a three-year average.

76. If so many districts who have implemented pay for performance failed, why does CMS believe it can do what no one else has successfully done? Some districts have seen poor results, including Nashville, New York and Chicago. Elsewhere, there have been very positive results, such as the Teacher Assessment Project in many states and Little Rock’s Achievement Challenge Pilot Project. Most of the studies so far, regardless of result, tested the idea that short-term bonuses will encourage current teachers to work harder and consequently raise their students’ achievement. We know that many, many teachers are already working as hard as they can for their students. So we are not surprised when these programs are shown to have no impact on student achievement. Instead, we believe that the impact of the new approach to student achievement will come from the long-term effects on the number of people who choose teaching as a profession, who stay in teaching and who are able to improve their practice with better data on their strengths and weaknesses in helping kids.

77. How will CMS gain teachers’ support for pay for performance? We know that teachers will make up their own minds. We are trying to create a teacher-assessment system through the teacher design teams and focus groups that will be a system that everyone can believe in. Engaging teachers in the work, as we are doing with the design teams and focus groups, will help build support for it.

78. If the current teacher evaluations are not working, why not go back and train our principals to evaluate in a manner that would be better suited to the information you are trying to gain? Businesses rely on managers to evaluate their employees; why should our schools be run differently? I would imagine that if you retrain our principals, and tell

14

them their evaluations will be audited that evaluations would change. I suggest, at the very least, to try this first before launching into this new program. We are holding our principals accountable for accurate teacher performance evaluations. This evaluation process is monitored very closely by central office. These are good suggestions, and ones that we have already begun implementing. We believe that the classroom observations on which these evaluations are based are one very important measure of a teacher’s impact on students, but not the only one. The high ratings – both in CMS and nationally, more than 90 percent of teachers are found to be at standard or above – also suggests that the current evaluation process is flawed.

79. Why do you view the pay-for-performance funding as more important than keeping teachers, teacher assistants and support staff? If I had to choose one over the other, I would certainly choose keeping our teachers. I know that pay for performance is a project that is close to your heart and one you believe will work. But many projects, for various reasons, are shelved. In our current economic situation, developers are walking away from projects and in our personal lives we start things and then determine they are not what we expected. Does the pay for performance, in this economic climate, not fall in this category? We cannot postpone our work toward having an effective teacher in each and every classroom. Our work on pay for performance is much more far-reaching than compensation. It is about the way we grow in our understanding of effective teaching and how what we learn will influence our entire approach to recruiting, developing, placing, compensating and retaining teachers. We believe that we cannot afford to delay this work because of the enormous benefit it will provide to our students. Even if pay for performance did not happen, CMS would continue the summative assessment system to continuously inform and support teacher instruction.

80. Have you been in the schools lately to see what this pay-for-performance program is doing to the morale of our teachers? Teachers are not getting raises and are being asked to do more and more each year. The Human Capital Strategies team has visited nearly 150 schools this year to meet with faculties. We have also had extensive engagement with teachers through the design teams. We are aware that this change is stressful to many, but we believe that as the benefits to students become apparent, the teachers will understand the need for this work.

81. Why was the pay-for-performance program implemented and created without teacher and principal input? These are the people who know the most about the information you are trying to gather. Why not use them to assist in this process rather than shut them out during its creation and then force it on them now? And for those teachers that were involved, was their input real, or were they possibly worried about their feedback and the impact it could have on their jobs? In fact, the system has neither been created nor implemented already. We are working on creating it now with teachers. That’s the purpose of the design teams. Teachers involved on the design teams can answer whether their feedback is real. You can see the minutes of their meetings and post questions for them on their wiki sites. They are on the pay-for-performance website, which can be accessed from the CMS home page.

15

82. Have you taken into consideration the reality of executing these new pay-for-

performance tests? For starters, I simply cannot swallow losing more than 40 hours of class time to give more tests. For younger students, have you considered the anxiety this could cause? Children have a hard enough time performing as it is, now you are testing them possibly with their teacher, possibly with someone else, possibly being monitored by a second person in the room, and all of this not knowing what the day holds for that young child on an emotional level. Did their parents recently get divorced? Is the child not feeling well? Is it just a bad day? How can this truly be a valid test with so many variables in play? In no case, except perhaps high school students who take multiple Advanced Placement and International Baccalaureate classes, do CMS students take 40 hours of tests per year. While it is true that some students get anxious about taking tests, we encourage staff to help ease the tension and helping students understand why tests matter. Schools have been given flexibility in setting the logistics for the summative pilot tests, so some students may have someone new in the room administering the tests but others may not. Schools make these judgments based on their particular circumstances and needs. As for the variables at play during a test, the instances you cite could be true for any kind of test. That’s part of the reason that we are not using a single measure of teacher effectiveness. Instead, we plan to use more than one measure and also use multiple years and multiple students. This will limit the impact on a teacher’s evaluation of students who may be having a bad test year.

83. Can you tell me specifically what you are doing differently than Chicago and New York that will make the pay for performance program work here when it has not worked there? How will it not create competition between teachers? How will it not drive teachers away from the profession? How will it improve education if ultimately teachers can simply start teaching to the test? We are not basing our work on the idea that temporary bonuses will encourage existing teachers to work harder, as was the case in Chicago and New York. We believe that many, many teachers are already working as hard as they can. We are not surprised by the recent results from Chicago and New York that found providing financial incentives to teachers did not result in increased student achievement. Our theory of action for this work is that by more tightly aligning compensation with impact on students -- right now, there is no relationship at all -- then more people will enter teaching, those who are effective will stay in the classroom longer and we will be able to support current teachers by being able to give them more nuanced feedback about how they can improve their teaching. It will not create competition among the teachers because we are also measuring team- and school-level contributions to student achievement. One of our teacher design teams is exploring the possibility of also measuring teachers’ contribution to their professional learning communities.

84. Why on the CMS website does it state that teachers will have a vote in this new system, and now you have created a bill that will take that vote away? We said that we would follow the law and the current law requires a vote if our proposal would affect the state salary schedule. It was never a given that our proposal would touch

16

the state salary schedule. Of course, we will follow that law if the proposed bill does not pass. It is important to note that it was never the case that we would hold a vote about whether to move forward with our teacher-effectiveness work.

85. What are the design teams and what are they working on? The design teams are groups of teachers and school-based staff who are working to develop specific parts of teacher-effectiveness measures. At present we have seven design teams meeting right now: value-added; contributions to school and professional learning communities; student survey; classroom observations; student work samples; other ideas, and hard-to staff schools and subjects. When the 2011-2012 school year begins, we will convene additional teams. These will address four broad areas (there may be several teams within a particular area). The areas are: communication/portals; professional development; weighting the measures, and linking evaluation to compensation.

86. How has the public been made aware of the work on teacher effectiveness that CMS is doing? We have used a variety of communications strategies and channels to share information about our work. For instance, on Sept. 28, staff provided a detailed presentation to the Board that offered new information about the Harvard work on the End-of-Course tests in CMS and the days-of-instruction model we are using. On Dec. 14, we provided another Board presentation, this one on National Board certifications in CMS and the design-team process. On Feb. 22, teachers from the design teams gave a Board presentation about their work. We have also created a group of web pages to give information about teacher performance and pay for performance that can be accessed here: http://www.cms.k12.nc.us/cmsdepartments/accountability/payforperformance/Pages/default.aspx. Principals have received regular updates at the monthly principal meetings. We have sent out emails to teachers, principals and others about specific milestones in the work.