2017 NEPF Admin Survey and Focus Group Results · Focus Group Responses Regarding Setting SLGs Cont’d • We know that there was the protocol. We went through that as a district

NEVADA EDUCATOR PERFORMANCE FRAMEWORK

(NEPF) IMPLEMENTATION Survey and Focus Group Preliminary Results –

Administrators

TLC Meeting – September 27, 2017

Focus Group Demographics

Role Group (Region) Focus Groups Total Participants

(approximate)

Administrators (Northeast & Northwest) 6 15

Teachers (Northwest) 7 65

Teachers (Southern & Northeast) 4 33

Total 17 113

Survey Response Rate: 372/1,248* = 29.81% Complete Responses: 259/372 = 69.62%

*The number above does not account for the link to the survey being tweeted on Twitter by multiple people as well as being shared via email by multiple people.

Administrator as Evaluator

(Slides 4 – 19)

What type training(s) did you attend regarding implementation of NEPF? (Check all that apply)

• Answered: 370 Skipped: 2

Of the training(s) you've attended, which one had the most impact on your practice?


Focus Group Responses Regarding Training

• Administrators provided suggestions to improve the quality of training:

o Logistically, they suggested there should be more ongoing regular training; content wise,

o They suggested it is important to pay attention to consistency;

o They would like to have trainings on how things are rated and scored, how to utilize the tools, etc.

• “I think more ongoing training will help. These rubrics and the indicator and standards have a lot of verbiage in them and we always have new staff so just training new staff, new admin, just [is] a tireless cycle”

Did you attend the RPDP Inter-rater Reliability training?


When was your most recent Inter-rater Reliability training?


How have you worked with others to calibrate NEPF observations in an effort to increase Inter-rater Reliability? (Check all that apply)


Focus Group Responses Regarding IRR Training

• “I forget about them...Just to confirm, we watch video segments of numerous lessons...we scored them and then we discuss the evidence on what they did or didn’t do and then coaching points and celebration points.”

• “So we’ve had the inter-rater reliability training but we obviously need more because we’re not there yet.”

At your school, implementation of NEPF Standards and Protocols is positively impacting student learning.


You are able to meaningfully identify performance levels on the NEPF Instructional Standards for Teachers.


You are able to meaningfully identify performance levels on the NEPF Professional Responsibilities Standards for Teachers. • Answered: 270 Skipped: 102

You have read the Nevada Department of Education's 2016-2017 NEPF Protocol document.


On average, how much time per educator did you spend on classroom observations and conferences?


On average, how much time per educator did you spend completing the NEPF forms (outside of classroom observations and conferencing)?


Focus Group Responses Regarding Observation Conferences

• “The pre conferences aren’t as effective obviously as the post in my humble opinion. The reason I say that is we’re held accountable for finding evidence for all the indicators, so we have to look for everything. If there’s something in particular that a teacher has a goal for, that might be something we talk about. Other than that I really feel like I’m tied to collecting data as much as possible on every standard and every indicator”

• “Not to have so many questions that guide those conferences. I mean when we’re talking, 30 minutes max is generally what we have. It’s not possible to get through all of those questions that are tied to nearly all of those standards and indicators. I mean there’s a lot of tools and I think each of us use them all differently”

Focus Group Responses Regarding Observation Conferences Cont’d

• “I think that when you approach the pre-conferences and post-conferences, you do it with a different hat depending on who you're working with..... for the new teachers it kind of gives them an outline and it guides them. For the more veteran teachers it's a hoop that they have to jump through and I don't really think that for teachers who have been in the business for a while- I don't really think that it does much to focus them or help them, or even guide the conversations when I'm with them. Because they know what they need from me and they can ask those questions and we can have that conversation without having that piece of paper...”

Focus Group Responses Regarding the Self -Assessment

• “I think any time they reflect on their craft it’s beneficial for them. I don't know if the tool is so much effective as just the fact it makes their reflection mandatory”

• “And it was so demanding [for] our teacher[s] to try and answer all of those questions… I tried it a couple of times. I just went with my own”

• “It makes them dive in to the document of how they are going to be evaluated. It is not a mystery, it is not a surprise, that is what I tell them”

Administrator as Evaluatee

(Slides 21– 45)

Who conducted your evaluation this year?


You are able to meaningfully apply the NEPF Professional Responsibilities Standards for Administrators.


You are able to meaningfully apply the NEPF Instructional Leadership Standards for Administrators.


You are confident in knowing how to set your Student Learning Goal (SLG).


Focus Group Responses Regarding Setting SLGs

• (Question: are you confident in knowing how to set your student learning goals?)

o Participant: Absolutely not.

o Participant: No. we struggle just as much as the teachers do.

o Participant: We probably struggled more.

o Participant: It’s probably harder. It is.

o Participant: It’s harder as the whole school.

• “I'm confident in setting student learning goals. I'm not confident in really guiding teachers on how to best set their own personal goals.”

• “It was a lot more work than it needed to be. On my level, I think that we have so many other initiatives going on that it was not necessary for me to set a student learning goal for myself, necessarily.”

Focus Group Responses Regarding Setting SLGs Cont’d

• “We know that there was the protocol. We went through that as a district. But I think it was so much to take in so quickly. By the time teachers needed to create those, I think all of our definitions of what rigorous look like and what they needed to focus on varied.”

• “For an art teacher or music teacher…[it] created a lot of anxiety for them because they have to try to figure out how to tie it to reading and math. Our overall school goals was hard.”

You are confident in knowing how to measure progress towards your SLG.


Focus Group Responses Regarding Measuring SLGs

• “We are a K-5 school and we make all of our teachers set SLGs and we do it off of MAPS. We look at the Read By Three legislation and we took the cut score which the district gave us- or not the cut score- the percentage that the district gave us according to reading and at our school because we do have such a high needs population, those numbers were huge. ”

• “Let’s say depending upon the assessment piece that the teacher chose. Some of those SLG conferences were more beneficial than others. But if again they chose MAPS or some sort of an assessment like that, they’re not helpful because you don’t have any real data to look at.”

• “I can speak on ours that we use math data.”

Did you modify your SLG based on your Mid-Cycle Goal Review?


When did you complete the NEPF Self-Assessment?


How long after your Self-Assessment did your immediate supervisor discuss it with you?


Did you use your immediate supervisor's feedback from your Self-Assessment to complete your Goal Setting and Planning Tool?


Focus Group Responses Regarding Self-Assessment

• “My supervisor didn’t really go over my self-evaluation with me.”

• “I think that doing that helped focus on what goals you needed to work on for the school year. So I think that was a good way to start the year.”

• “I was handed the rubric and told where to mark and told to do the self-assessment. But the discussion never included the rubric or where I was, how I was rated where I was. I've asked several times on where I could work on certain areas and it was an effective evaluation to make it highly effective, there have never been any suggestions or any conversation”

Did you participate in a Pre-Observation conference?


How many times did your supervisor conduct a scheduled observation of you this year?


How many times did your supervisor conduct a unscheduled observation of you this year?


On average, how long after your scheduled or unscheduled observation did you participate in a Post-Observation Conference? (Not the Summative Evaluation Conference)


As a result of the observation cycle conferences, to what extent do you agree that feedback positively impacted your Instructional Leadership practice?


Focus Group Responses Regarding Observations

• “I thought my supervisor did a great job. We met at the beginning of the year, we met midway, we discussed, he listened to me, I listened to him, we had a good conversation. It usually lasted between an hour and a half and two hours which is quite a bit of time which is fine.”

• “I believe it was essential. I do feel like I got a lot out of the conversation…”

Focus Group Responses Regarding Observations Cont’d

• “There were no surprises in pre- and post conferences at all. I would say it’s different being the principal probably. You don’t necessarily see your evaluator all the time. So those meetings are more critical in my mind. But then they’ve got to be based on good solid data too.”

• ” Once again I think it's a good conversation point, but me being a Vice Principal I feel like there's just a lot of it that doesn't necessarily pertain to me. I feel like when I was doing my pre and post- or my pre-conference we both kind of sat there and went, well that's not really what I do. “

Using NEPF Standards and Protocols have helped you to identify your areas of growth as an educator.


Using NEPF Standards and Protocols have helped you to identify your areas of strength as an educator.


Focus Group Responses Regarding Changes in Instructional Leadership Practices

• “It helped me stay on course probably more than anything…”

• “I believe it was essential. I do feel like I got a lot out of the conversation…”

• “I think it makes me more aware. I think it makes me more aware of not only instruction but also like the school climate and how kids feel and if I’m being consistent with discipline and making those parent phone calls, good and bad.”

• ” ... this helped me focus on the literature reviews and the research behind what we do and help me become more academic in what I do....NEPF is great. “

The NEPF scores you received are fair.


How does the school or district use NEPF evaluation results? (Check all that apply)


Focus Group Responses Regarding NEPF Overall

• The major responses regarding the success of NEPF are: the program provided common language/talking points to educators; it cultivated more collaborations among teachers within a school; there’s noticeable student growth after the implementation of the program

• The major responses regarding the challenges of NEPF are: it is cumbersome and time-consuming; the program is specially more difficult for new teachers; there is no consistency in implementing the program.

• They also provided additional comments to the NEPF in general, such as: a lot of teachers still view it as punitive thing; and the program does not provide enough instruction to teachers in special education as well as teachers with subjects other than reading and math.

Where does your school reside?


What is your primary position?


What was your status during the 2016-2017 school year?


How long have you worked in education?


At which type of school(s) are you currently an administrator? (Check all that apply)


Documents

2017 NEPF Admin Survey and Focus Group Results · Focus Group Responses Regarding Setting SLGs Cont’d • We know that there was the protocol. We went through that as a district