ASRC FRiday the 13th - CCTA Minutes.pdf

8/20/2019 ASRC FRiday the 13th - CCTA Minutes.pdf

1/23

ASRC Meeting Friday November 13, 2015

Andre: Motion to call meeting to order

Read through agenda for today.

http://www.doa.nc.gov/asrc/documents/10192015/ASRCMTGAGENDAOCT19.pdf

Regarding the remaining timeline, they will have to refine both work group reports, write an

executive summary, following the template Olivia put together for them. They will put together

a draft a package that they all will review, from that package provide any evidence, concerns

and feedback, then they will reconvene to address those issues, to be sure to get the

deliverable to the legislature and the state Board in December. They need to formalize the draft

reports to deliverables by the commissioners voting on the reports that will move forward.

Olivia will report more on this. There is a lot of work that still needs to get done. They will haveto decide today if they need to add extra meeting time, do they need to go to Senators and get

an extension to meet the expectations of the General Assembly? (Personal comment, Senate

Bill 812 expires Dec. 31, since the General Assembly is not in session, I don't know how this can

be accomplished. Any failure to deliver their final report, in my opinion would be used to

negate the efforts of this commission) Today’s meeting will be shorter than usual so the work

groups can meet with their colleagues to help them.

Roll Call: Jeff Isenhour, Bill Cobey, Katie Lemons, Laurie McCullum, Joe Herrera, Tammy Covil,

Andre Peek, Ted Scheik, Jeannie Metcalf, Olivia Oxendine is on her way, Denise Watts is

supposed to call in, and Anne Clark is absent.

Andre asked for approval of minutes from last time, stated that they are posted on website,

(http://www.doa.nc.gov/asrc/documents/10192015/Oct19MinutesoftheMeeting.pdf ) Asked

for a motion to accept minutes. Motion made and 2nd. Accepted as written.

Item B on old business. The status of the draft writing of the final reports.

Ted: States that he is 99% done with his work groups' report. They received a document from

Board of Directors for the NC Council of Teachers of Mathematics (NCCTM) This document had

some points for consideration. Some points were questioning some of the methods used to get

the draft done. Questioned the expertise behind the results that were obtained and frankly

questioned in specific detail some of the conclusions that were drawn on specific standards and

questioned whether their view was consistent and was a standard complete or not. Ted said

their report had some errors and arguable things, just differences of opinions.

http://www.doa.nc.gov/asrc/documents/10192015/ASRCMTGAGENDAOCT19.pdfhttp://www.doa.nc.gov/asrc/documents/10192015/ASRCMTGAGENDAOCT19.pdfhttp://www.doa.nc.gov/asrc/documents/10192015/Oct19MinutesoftheMeeting.pdfhttp://www.doa.nc.gov/asrc/documents/10192015/Oct19MinutesoftheMeeting.pdfhttp://www.doa.nc.gov/asrc/documents/10192015/Oct19MinutesoftheMeeting.pdfhttp://www.doa.nc.gov/asrc/documents/10192015/Oct19MinutesoftheMeeting.pdfhttp://www.doa.nc.gov/asrc/documents/10192015/ASRCMTGAGENDAOCT19.pdf


2/23

Andre: Reiterates (I don't know why he always has to restate what someone has said!) Their

group will review Ted’s report, and come to consensus. Agreed that the work team’s process

has been transparent, that there has been feedback and input throughout the process and they

are taken seriously. Some feedback is complimentary, some critical, they should respect that

and be forthright when communicating with the stakeholders that all input is taken seriouslyand will be evaluated and included in the deliberations. Ted's team deliberated on this

particular feedback and found an area that a change does need to be made and reflected in the

final report, however, there were other things that they have a difference of opinion on. As

Commissioners, in reviewing Teds report, they will have a chance to weigh in and vote on the

final recommendations. There will be consensus on what they believe the final

recommendation will be.

English Language Arts:

Update, Olivia: They are making slow but steady progress on putting together a full draft report.They had conference call meeting last Friday, talked about things they are corralling a lot of

information to incorporate into the final report. There are various sections to their report, all

the information from focus groups, surveys, today's data, going back to state analysis of other

state recommendations (Fla, VA, CALIF, and NC) that were vetted and it is a monumental task.

Lots of data in various places. Tentatively talk about a timeline considering, rather ambitious

and aggressive timeline, asks for a full draft (both Math and ELA) by the first of Dec. Wants this

done by Dec 1, release this to the commission, called a draft, the commission would then have

week to read it, make comments, corrections, additions, deletions, clean it up, and resubmit to

commission by December 9, calling it the final report that is to be read, adopted and approvedon the 18th of Dec. It will require nonstop writing for her and other members of the team.

Ted: Said he is mostly done, will not take today's test score data into account. (My comment:

you could hear the gasp from DPI)

Olivia asks, you will not draw implications from the test data?

Ted said he looked at it and there is only one place he can draw any implication from and that is

in the end of grade and end of course Math data over time, outside that he doesn't see much.

Andre: To be clear, you ARE taking it into account, you just don't see it changes your report.

Ted: If you want me to comment on that and put it in my report I will, I just hadn't thought of

that.


3/23

Andre: For consistency you should, because we want to be consistent between ELA and Math.

The point you are making is you looked through scores and don’t see scores having significant

bearings on the findings you and your team reported.

Ted: Except for possibly for the end of grade data.

Jeff: I don't know that as a commission we have come to full agreement on the standard

recommendations, to him we have not even hashed out some of this so he appreciates what

Olivia said, Doesn't see how they can get this done

Tammy said they have had the opportunity to hash things out, the interim preliminary findings

were released a few months ago, since then, and they have had opportunity to discuss. A lot of

the findings that were released have been discussed and a lot of the gathered information that

Olivia speaks to is a wealth of information that has been gathered and it does support the

preliminary findings, she doesn't see there will be significant disagreement in the report

discussions. She asks Jeff if he disagrees with the preliminary findings and what has been put

forth so far.

Jeff: wants to see all the drafts, then have discussions on those drafts......,

Andre: Hold on a minute, what we are talking about is we need to get a full draft of everything

to each commissioner, then have them go through the full draft and have a chance to weigh in

on the full draft, and turn that into a final set of documents. You will have a chance once again

to voice your opinion on it. We still need to vote on any recommendation to make sure that we

all have a document in the end that we agree to by a majority not that any individual will agree

to everything inside the report, but at least have a quorum.

Olivia: Had problems the entire time even trying to wrap her head around a cogent set of

recommendations because this process is not easy, the nature of the work to do and the way

they have had to get it done has been very piecemeal, if we were to have another life as a

commission, they would probably do this work differently. What she is proposing is what Andre

said, get a full draft to commission By Dec 1, within it will be a full set of recommendations.

Those will include ELA and Math. Some of those recommendations have been talked about for

the last couple of months. Would then have a week to read and consider what is in it, send lots

of feedback, I don't like it, it needs to be re worded, it isn't what I thought it should be, I agreewith this recommendation, etc. This is when they as a commission will have to hunker down

and read the full report. There is tons of information there. We will have to come together and

then get the full draft to commission by 8 or 9 Dec. Between then and the 18th, will have the

time to do a clean report. Commissions will vote and agree or not, deliver report or continue to

work.


4/23

Andre: To Tammy's point, we have all been meeting for a long time and we shouldn't be that

far apart, remember the guiding principles that they agreed upon, clarity and simplicity,

developmentally appropriate, sequentially correct, teacher and LEA flexibility. We need to get

to the point as a commission weighing in, we have the opportunity to voice opinion, as we

refine our draft, what are we missing and how do we fix it. If there is a sticking point, there willbe a vote.

Jeff, wants the draft looked at like that of an elected official, where generally there is a first

second, and sometime third reading of policies under consideration. We don't have a full draft

policy yet. To him a week is not enough time with a draft,

Andre: do you agree with the process that we are submitting, that we submit a full draft for the

commissioners to review.

Jeff, wants a full draft first, give time to read, process, absorb it, then meet, at first reading

where there is feedback as a public body. That’s not where approve this. To him we need at

least 1 or 2 more readings to go over that.

Tammy: We are a commission, not a legislative body, they are making recommendations the

purpose of draft policy is to get feedback from the community which is what they have done all

along, the bulk of the work is summarizing a years’ worth of work that has already been done.

The draft report should not be a surprise.

Jeff: disagrees, says this will possibly lead to policy issues. Legislative policy action by the state

board, so it holds the same amount of weight even if it is it is coming for an independent

commission.

Andre: If this is a concern about having enough time, and they (Jeff) don't think that 8 days is

enough time to review the document and as a body, decide they need more time, then they

need to discuss that, there is no disagreement between the commissioners that they all need to

look at a complete document before they are put in the position to vote on the

recommendations set forth.

Jeannie: As an elected official with a lot of experience dealing with policy, that process in these

cases are to introduce policies that we haven't seen before, this is different, it is not like a first

reading is to learn about it, the second reading to discuss it and the third reading to take action.

We have seen this for over a year so I don't look at it that way. If we have 8 or 9 days to look at

the draft, meet on the 18th to talk about this, they may all agree, we are assuming there will be

contentious sections and parts of the report that they want to tear apart and that is not

necessarily true, he may be anticipating something that may not be reality.


5/23

Jeff: I am not anticipating that, I just want to see a holistic draft.

Jennie: I do too, but I feel like I have a very good handle of what I think the report will look like

as far as the big umbrella picture, I will never say I will know what a specific Kindergarten

standard will be, but I know the 4 points the commission has used as their guiding principles. I

am assuming that is what we will see and I think beneath the core of the guiding principles

there may be some changes to bear to the report as it goes forth.

Tammy; Not going to see anything new on the 18th, no deviance from what has already been

concluded thus far, there is no new evidence to draw from.

Olivia: From a technical point of view in terms of writing the report and the important pieces

of what needs to be in this report on educational on standards, it has become clear to her that

what is missing is the foundational piece, this will not shock anyone, a foundational piece that

lays out the meaning of the standard based on research, what does research say about the

shear unit of a standard, what is a good standard, how do you get to the standard movement in

education. This is more than a page and one half of information that kind of serves as a way to

set the theme of the piece, and that piece is missing and has to be in this report, and will not be

difficult to do.

Ted: I have drafted lots of math standards, and they all have this preface of long feel good

thoughts, to me I don’t think that is where the action in. His interest is what is the standard

and how does it work. It would be nice to have some preamble stuff, but not so much.

Andre: I think that what Olivia is describing, and what we saw in response to the draft reports

is that there is discussion around our charge, and why would we address curriculum instead of

standards for example. The fact of the matter is we are not addressing curriculum, but because

of how the draft was written, there was an assumption that could be made that it was

addressing curriculum number. #2 is that by the way you actually implement your curriculum

does have a bearing on whether or not standards are duplicated, dropped or not pursued. And

of course you have the testing component and we heard from teachers that some of the

standards may not be addressed at all because it may not be tested. There is some implication

that we need to frame out for our audience, here is what we looked at, we looked at standards,

here is what we mean when we say standard, so now the work that you see subsequent to this

definition. Agrees with Ted, this is not a time for a dissertation, on standards but simply a way

to summarize the information as we move forward. I think it is needed because I do see that we

received information that has been forwarded to all the commissioners that stakeholders are

concerned that what they saw initially was not the final report, we need to be very clear on

how we write the report so people don’t draw the wrong conclusion.


6/23

Jeff: doesn’t want to be difficult. We have not had any real work sessions as a full commission

to really go through the document, is it possible to have a work session as a full commission?

Andre: Saying what we have shared between the work groups has been out, everything has

been posted, once we got the draft document and got back together as a commission this

discussion would happen. His point to Jeff is that if they as a body they feel they need more

time, an extension, he and Tammy can pursue that and find out if that is possible. They may

find they are not that far apart. (Personal Comment: Like I said, an extension is not probable

since the General Assembly is not in session, looks to me like Jeff is trying to stall the

commission report so Senate bill 812 would "sunset" with no action taken, actually "die". If Jeff

can keep the report from coming out, he could sabotage the commission’s work and

recommendations over the past year, and keep Common Core intact. I smell a conspiracy. All

the work group meetings were open to all the Commissioners, if he was so concerned about

what was going on in them, why didn't he show up? If he is so concerned in what will be written

in the report, why isn't he helping write the report? What work team did Jeff work on? The

answer is neither)

Jeff: Draw a timeline for me, how do we make that decision?

Andre: When we have the full document

Tammy: Jeff, if you are implying they schedule a work session in the interim, that can be done,

however, coordinating everyone can be difficult. She is totally comfortable with that if it can

give him a better degree of comfort. She wants to work towards a draft report first of Dec. If at

that time they think they need an extension, than start discussions with legislators to see if thatis an option.

Jeff: is it possible that we get a draft document by your date and have a work session at our

next meeting? (Personal comment, stalling again Jeff, you don’t want the final report out on

the 18th, you want a work session on the 18th, when do you think the report could be finalized

with the holidays?)

Andre: I don't think we want to do that only because it is not fair to the legislature for them to

expect a deliverable by the end of December and then a week before Christmas we say sorry,

not going to happen.

Jeff: said the legislature doesn't necessarily do what's best for the people (Personal Comment,

cheap shot Jeff)

Bill Cobey: States that at a minimum, they should get a hold of Representative Horne and

Senator Tillman, to start a discussion about a possible extension, legislature is not in session so


7/23

they can’t change the law and he doesn’t know if government operations committee that

meets next week is empowered to do that sort of thing when the legislature is not in session so

I recommend you have the discussion, I don’t know how they operate.

Tammy: Talked to Tillman today, she had emailed him for guidance on the deliverables. Tillman

indicated they are anticipating a final report in Dec. She asked how did the General Assembly

expect this report to be delivered, do they drop it on his desk? The General Assembly will want

an opportunity to digest what they (the commission) puts forth, then invite the commissioners

to come back and appear before the Joint Education Oversight Committee in Jan of Feb. To

answer questions. The commission could probably work in those 2 months before going to the

LEOC Legislative oversight and make some adjustments to the report if necessary. What is

important is a deliverable on the record, in December.

Andre: I think that’s good, I don’t want to send anything outside our commission that

everybody doesn’t agree with, even if it’s a draft because of danger of any expectation ofchange, would rather be in the position of being in agreement and just having to clean it up.

With that said, we could probably have an extra 4 weeks without running into problems,

Tammy: Technically we could us 18th as a work session, hash out issues, possibly do a

conference call later on to do a more formal approval prior to the Dec. 31 deadline. This

commission expires on Dec 31. We are getting into the holidays

Andre: suggests Tammy and he formalize this point with Senator Tillman and Representative

Horne, to go on record by the end of December, but indicate they need more time to clean up

the final deliverable, but get it done in month of January, so we have it on record in December,

and then work through the schedule. The commission would still meet on Dec. 18th with final

document being prepared. See where it is, assess time that is needed. Doesn't want to put

anything forth that they all don't agree on. (Personal Comment, Andre sure is hard to decipher

today!)

Tammy: Disagrees, states the bill sponsors are anticipating and expecting final report by end of

Dec. Thinks they can deliver, wants to stick with that date, does not anticipate there being any

kind of serious disagreement in direction they are heading, as an elected official, I spent moretime on this than in her role on the school board in New Hanover County. Does feel they can

compile, the information and finish putting it in a document in summary report and forward on

by the deadline.

Andre: Askes what if they are at an impasse.


8/23

Tammy: They may have some disagreement, but this commission is going to be a majority vote I

may disagree with the direction we’re going on a specific recommendation, but if the majority

of the commission agrees, it is what it is. They will have a majority vote, if the majority agrees,

it is what it is. Never get 100% consensus on everything

Jeannie: Speaks to Laurie, Katie Olivia; if any of the commission ever wanted to go to any of the

meetings, they were able to. Would have been welcomed to share the workload. She

appreciates work done on her behalf, and would not want to take the last day of the session to

be the one to come in at the last minute and ask why are they doing this, the people on the

board trusted those that have put on the work teams, they have already hashed this out

beforehand. I trust these guys, and when I see the final report if I don’t feel that way, I will

certainly speak up, but I don’t want to take away from any of the work they have done and

insert herself at this point when she had ample opportunity in the beginning and chose not to.

I think we need to wrap it up. Does not think there will be any surprises when she sees the final

report because of the discussions all along.

(Representative Speciale and Representative Pittman are in attendance today, and Larry

Pittman asked if he could address the commission)

Larry Pittman: gave a draft outline to the commission at J’s request. Stated that he and

Michael were the beginning of all of this, not Tillman and Horne. Stated that they would be glad

to work with the commission on this. I will insert Representative Pittman’s suggestion on what


9/23

he would they expect to see in their report.

Andre: We do recognize you and Representative Speciale have been here throughout the

process and appreciate that and obviously you are the audience that we are charged to answer

to, thank you so much for your input.

Andre: The bottom line, make sure we all agree to this, we do want to wrap up, hold the

timeline and complete the draft by end of December. We will have a document to review, we

will have a vote individually on the content of that document that represents our individual

views, and I think that is what I heard? Is everyone in agreement with that? We are agreeing

that we are not going for extra time past December 31. We will vote on whatever we have.

That says we need to see the document by the first of Dec. to give us the time to read, and get

our individual feedback documented. .

Tammy, with the draft complete, they can pull all the pieces together. When the preliminary

drafts were released it was with the purpose of giving teachers an opportunity for feedback so

we were giving them some information to draw from. There has been allot of people that

looked at the drafts and made comments on incomplete documents they don’t have the

evidence to support the recommendations of findings within them, so you do have to be careful

and be sure that whatever document we put forth as a final draft agreed upon by the

commission is really what we want to put forward I a public way.


10/23

Jeff: All I’m saying with all reason and prudence, all members of this commission have a full

version of this draft, read it holistically, then digest it, then feedback on it, not piecemeal and

fragment aspects of things. That is impractical to do.

Tammy: That is what we are trying to do with the draft document is pull all those pieces

together in a succinct way so it is understood where we are headed with this report.

Olivia: Askes Jeff if he remembers writing his dissertation?

Jeff: Yes, and it took a number of years to finish that document. I don’t have years to go

through this.

Olivia that is what this report is, on steroids.

Andre: To wrap up, by Dec 1, complete draft will be complete it will include entire framework

with glossary, introduction, and summary, will incorporate summary feedback from the survey

and work focus groups. With the Math sections, with the ELA sections. Joe will distribute, work

with Olivia, once done on Dec 1, all of them take time to fully digest so when they get back

together we will have a meeting and discussion that will lead to a vote. If after receiving all of

this information you think there is substantial element that needs to be discussed in order to be

prepared to express your vote, editing, something missing, not understandable, that needs to

be shared as soon as possible, give them an opportunity in advance of that meeting to address

it, do not wait till the 18th to address. Anything else on timeline, on ELA, Olivia you are going to

take the lead to get the definition on standards.

Olivia: that will be nothing compared to the reset of the task.

Andre: Summary reports on accountability Dr. Howard, her bio follows:

http://www.doa.nc.gov/asrc/documents/11132015/TammyHoward_Short_Vita_080315.pdf

She has slides to hit high points her presentation on the data from included reports.

The evidence links:

http://www.doa.nc.gov/asrc/documents/11132015/AcademicStandardsReviewCommissionEO

G-EOGDataRequest10232015.pdf

http://www.doa.nc.gov/asrc/documents/11132015/AP2013-2015.xlsx

http://www.doa.nc.gov/asrc/documents/11132015/ASRC_EOG-EOC_By_Race_10232015.xlsx

http://www.doa.nc.gov/asrc/documents/11132015/HSMathSurveySummaryuntilSept162015.p

df

http://www.doa.nc.gov/asrc/documents/11132015/TammyHoward_Short_Vita_080315.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/TammyHoward_Short_Vita_080315.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/AcademicStandardsReviewCommissionEOG-EOGDataRequest10232015.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/AcademicStandardsReviewCommissionEOG-EOGDataRequest10232015.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/AcademicStandardsReviewCommissionEOG-EOGDataRequest10232015.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/AP2013-2015.xlsxhttp://www.doa.nc.gov/asrc/documents/11132015/AP2013-2015.xlsxhttp://www.doa.nc.gov/asrc/documents/11132015/ASRC_EOG-EOC_By_Race_10232015.xlsxhttp://www.doa.nc.gov/asrc/documents/11132015/ASRC_EOG-EOC_By_Race_10232015.xlsxhttp://www.doa.nc.gov/asrc/documents/11132015/HSMathSurveySummaryuntilSept162015.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/HSMathSurveySummaryuntilSept162015.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/HSMathSurveySummaryuntilSept162015.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/HSMathSurveySummaryuntilSept162015.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/HSMathSurveySummaryuntilSept162015.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/ASRC_EOG-EOC_By_Race_10232015.xlsxhttp://www.doa.nc.gov/asrc/documents/11132015/AP2013-2015.xlsxhttp://www.doa.nc.gov/asrc/documents/11132015/AcademicStandardsReviewCommissionEOG-EOGDataRequest10232015.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/AcademicStandardsReviewCommissionEOG-EOGDataRequest10232015.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/TammyHoward_Short_Vita_080315.pdf


11/23

http://www.doa.nc.gov/asrc/documents/11132015/NAEPALLSUBJECTS-

ALLGRADESSCALESCORESOVERTIMEWITH2015.xls

http://www.doa.nc.gov/asrc/documents/11132015/NAEP2015ELA4.pdf

http://www.doa.nc.gov/asrc/documents/11132015/NAEP2015Math4.pdf

http://www.doa.nc.gov/asrc/documents/11132015/NAEP2015Math8.pdf

http://www.doa.nc.gov/asrc/documents/11132015/NAEP2015Reading8.pdf

http://www.doa.nc.gov/asrc/documents/11132015/SAT2005-2015.xlsx

http://www.doa.nc.gov/asrc/documents/11132015/SAT2005-2015.xlsx

-

Dr. Howard: Did provide the requested reports to the committee, the first piece of data that

was requested was looking at the end of grade, and the end of course tests that assed student

proficiency. Going back from 2005 to 20015 refers to the chart. Looking at the data presented,

there are vertical double thick lines that indicate when content standards changed. The testing

program as it is now began in 1992, 93 since that time there have been 3 previous content

standard changes and when that happens you accept that tests are being developed. The first

step in test development is to align tests to the new content standards you will see the breaks

between 05 and 06 in both reading and math, end of grade and between 12 and 13 as well.

One area that does need to be noted for end of grade in 2008 and 2009 is there is not a double

bar on the chart, but there is an obvious increase in the percent of students that are proficient

that was due to a State Board policy that allowed for retesting after the initial test

administration and the first year that was in place we saw an increase in the percent of the

scores that were proficient. For the end of course, that change took place in 2009, 10, so that

again for algebra 1, math 1, English 1 and English 2 you will see that that jump in 2010 school

year.

The 2nd piece of data that was requested was the National test of educational process often

referred to as NAEP. All the way back to the 1960’s there was something referred to as State

NAEP that is administered through a representative sample of the population within the state

so that we can have state level results to work for NAEP there are not student level results and

there are not district level results or school level results, the results that are state level only. In

the early 2000’s it became the practice through federal law that all states participate in NAEP.

http://www.doa.nc.gov/asrc/documents/11132015/NAEPALLSUBJECTS-ALLGRADESSCALESCORESOVERTIMEWITH2015.xlshttp://www.doa.nc.gov/asrc/documents/11132015/NAEPALLSUBJECTS-ALLGRADESSCALESCORESOVERTIMEWITH2015.xlshttp://www.doa.nc.gov/asrc/documents/11132015/NAEPALLSUBJECTS-ALLGRADESSCALESCORESOVERTIMEWITH2015.xlshttp://www.doa.nc.gov/asrc/documents/11132015/NAEP2015ELA4.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/NAEP2015ELA4.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/NAEP2015Math4.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/NAEP2015Math4.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/NAEP2015Math8.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/NAEP2015Math8.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/NAEP2015Reading8.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/NAEP2015Reading8.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/SAT2005-2015.xlsxhttp://www.doa.nc.gov/asrc/documents/11132015/SAT2005-2015.xlsxhttp://www.doa.nc.gov/asrc/documents/11132015/SAT2005-2015.xlsxhttp://www.doa.nc.gov/asrc/documents/11132015/SAT2005-2015.xlsxhttp://www.doa.nc.gov/asrc/documents/11132015/SAT2005-2015.xlsxhttp://www.doa.nc.gov/asrc/documents/11132015/SAT2005-2015.xlsxhttp://www.doa.nc.gov/asrc/documents/11132015/NAEP2015Reading8.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/NAEP2015Math8.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/NAEP2015Math4.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/NAEP2015ELA4.pdfhttp://www.doa.nc.gov/asrc/documents/11132015/NAEPALLSUBJECTS-ALLGRADESSCALESCORESOVERTIMEWITH2015.xlshttp://www.doa.nc.gov/asrc/documents/11132015/NAEPALLSUBJECTS-ALLGRADESSCALESCORESOVERTIMEWITH2015.xls


12/23

Reading and Math tests are administered in odd numbered years. You can see in data it goes

back to 2005 to 2015, we just released the most recent NAEP results in October, 2 or 3 weeks

ago. Looking at this information in addition to the chart that is provided, we are also providing

the slides looking at each of the grades and the content Areas that were assessed through NAEP.

Of course NAEP is administered at grade 4 and 8 in reading and math for what we call StateNAEP. There are other neap content areas that are assessed, but they are assessed at the

National level and there are not state level results available for those other areas. We’re

looking at reading going back to 1998, NC has participated prior to it being required through

Federal legislation which is why we have results back to 1998 you can see that NC has had a

steady increase particularly looking at 1998 to 2003, a true success for NC. If you follow trend

line, you can see for the most recent release in 2015 that NC was above the nation which was

really good for grade 4 reading. Grade 8 reading again going back to 1998 you can see it’s kind

of flat in some instances there, we had a bit of a bump then it came down and went back up. Of

course course the most recent results showed that NC was below the Nation in reading and

these results were really very similar to results across the nation with respect to 8th grades

reading. Reading at the middle Scholl nationally has been flat for a very long time and there has

been a lot of conversation around this about why is this the case. And part of it is considered to

be that reading is very much a fundamental skill and once that is accomplished then the

challenge then is to increase the complexity of the reading and it gets more challenging. The

students can read even more difficult text and can draw inferences and make conclusions.

Andre: interesting, because we talked about then we had a focus on the whole idea of earlychildhood development and formative years and had someone come in and present to us the

whole concept of getting off to a good start in reading and writing skills, this suggests we were

doing good in 4th grade but somehow that foundation that we began with we were not able to

sustain. Why is that, what is the root cause of that you would point to explain that 10 years of

data.

Howard, one thing that is important to remember not a content expert, doesn't have final

answer. Very complicated answer and involves many factors, I do have an ELA background,

again, not a content expert. L looking at reading, first task is kids can read, as they go through

the years, up into high school, require deeper understanding of for complex tasks. Perhaps we

are reading at a lower level. Reading is reading, and being able to fine tune the skill at a higher

level as you go through your education experience and through your professional experience

than that is the difference, so if we think about the fact that most mass media, that newspapers

are written to a certain grade level to a 6th or 8th grade level, written to a level that the general

public can read and understand, very simple, easy to digest. Short, simple sentences. The long


13/23

term goal of being successful post high school in that emphasis has to be on having more

complex text. What is interesting in the challenge is to find a way to transfer that skill and that

accomplishment from 4th grade as you go up through the grade levels. Of course what we are

really about is seeing that same performance in grade 8 and beyond that we see at grade 4, so

the answer is not that you are better in 4

th

grade and they just don’t do better, the question ishow do we increase student achievement which is the purpose of assessment, is show what the

status is so we can have these conversations.

Andre: What has changed? (My comment, the standards!!!!) Let me jump to the point that I

am hearing from the other commissioners especially the ELA, the importance of writing and it

was pointed out that without attention to writing, it doesn’t challenge them to read and

understand, they don’t have to demonstrate understanding. I was wondering if in your analysis

if you have seen that kind of dynamic where reading is taught instead of understanding. In

business we write Manuals and test our manuals for technology, how to unpack a lap top, how

to set it up, it is written to an 8th grade level. We found that is the best place to be, but is not

helpful to increase efficiency and skills. I was wondering if in your data, you make these

inferences between reading and understanding. There has to be a connection with writing,

they have to demonstrate understanding and what we are challenging the kids with.

Howard: First step of this is to have content standards that do challenge students and the role

of the content standards is to have that in place, to challenge the kids, to increase their

performance. So that when they are measured, not that we just have a measurement of what

is taught, but what is taught increases the performance and measures the achievement so that

students have the ability to learn more and do more. The measurement is all about seeing whatthe students know and are able to do, content standards tell us what we want the students to

know and be able to do, the tests are to see if it was achieved.

Setting the expectation is the first task at hand, your comment about writing is very important,

we did have them for many years. Writing assessments at grade 4, 7 and 10 are not in place

now, it is our understanding that we have a process that we refer to as peer review. They have

a process USDE where they review our assessments to make sure that they are sound and in

line with Federal law, (COMMENT: did you hear her say align to Federal law! There are 3 bills

that prohibit the intrusion into states’ rights) now we are at the point of going back to peer

review and sharing our assessments and will have to see if writing is in the content standards

and if writing has to be measured. Often times states including NC, have not measured writing

because you can’t do that in a multiple choice format. That is expensive and time consuming.

Andre: To Laure and Olivia, a way of input to the ELA work, I would be interested to see as we

focus on ELA recommendations, as we look at these gaps, and be mindful of our charge and the

recommendations of accountability that we are tying your recommendations to.


14/23

Olivia, In being sensitive teachers meeting, some feedback from the surveys, sporadic

comments requesting writing tests, we can certainly include those comments in our report,

however I don’t think that is substantial enough to include in our recommendations. (My

Comment” SAY WHAT!! On one had the kids can’t write, can’t defend an opinion, can’t

comprehend, but we don’t want to include that??!!)

Andre: The only point I am on is we are concerned that some of the formative skills that we

develop in the early grades and we have no figures there, it seems like we are kind of

misleading ourselves because it looks like we’re doing great in the 4th grade and by 8th grade we

regress, did we miss the point in the 4th grade on what the foundation is and not get the skills

needed or is it because you don’t have a progression of continuing progression to develop on

that foundation. Kids forget tools from early. I would like to see that reflected in the report.

Jeff: Look at this data in a diagonal fashion. Says there is progress made at end of grade/course

assessments, students make good progress about the time they scores get better, and theychange the standards and assessment, so those that align with NAEP through the years. We

start to get it right and you change it on us.

Andre: Says I understand the point you are making but when you change the measurements,

but you don't forget how to read just because the standard changed.

Jeannie: Always seems to be a movement to re norm the tests, because the test scores too high

re norm the test, the scores go down. Does that mean the next year the kids got dumber? The

skills scores don’t change, the arbitrary bar of what is considered proficient gets moved, now

they have added a level of proficiency. Levels 3, 4 5 now proficient, so the actual scale scorenumber of proficiency has been lowered and now we have 5 levels. This gets manipulated all

the time. This is manipulation. Is arbitrary. I don’t put much faith in numbers. Look at skills

scores. NAEP always tells us that the scores were not real because our curriculum standards

were not in line with NAEP.

Jeff: Until recently you are saying that we were in alignment with NAEP?

Howard: Several pints to be made here, one is going back to 98, early 2000, NC developed own

content standards. In math it developed those content standards keeping in mind the NAEP

framework and the national mathematics standards. You can see here in math the significantincrease over the years partly because the content standards that were taught in NC were lined

up with the high expectations of NAEP and that showed up very well, NC versus the Nation, NC

was consistently above the nation going back to the early 2000 NAEP is a state level result

assessment, the math standards have very consistently be aligned with those expectations.

Reading and math are very different. Math is specific, reading is reading, as you move up the

scale, reading is expected to be more difficult. An 8th grade reader should be able to read more


15/23

complex material and draw more inferences and conclusions at a deeper level than that of a

4th grader, not to minimize the 4th grade performance, though it is outstanding that in 2015 the

4th graders outperformed the nation. The 2 content areas are very different and teachers

approach it differently in applying those content standards and doing the professional

development on those content standards. They are not the same. So going back to whathappens when we change content standards, the first step in test development is to adopt

content standards. We are in 4th edition of assessments, they do not meet the same

expectations that we are setting for 2015, they continually increase, like a runner, once you go

around the block and not get out of breath, and you go a bit further. A credit to NC when you

look at state assessments compared to NAEP which is considered the Gold Standard, to

national results, and TIMMS, which is the international assessment showing NC at top in the

world in math and science, when you look at that it tells you we are moving the bar forward on

expectations for students in NC students going back to the 1990's. So the scale scores are very

useful, but if you notice on the state chart, when you look at the scale score you can’t compare

the scale score across editions. Look at when we introduced re testing, note in 2008 and 2009

you will see the scales in grade 3 increased from 344 to 345 so that is more than a difference in

a scale score, but if you look at it compared to other years it is more of a jump. There is

renorming and resetting of the scale, when we change standards and assessment we are

changing the expectations on the students based on the current standards. What is expected

and what is measured.

Olivia, comment, thanks her for her presentation on the data and walking them through it.

Looking at 2015, paying attention to math at top of the page, and she knows a lot of things

happen in classroom in the minds of the students and the way teachers teach and prepare

students for testing In this data we will not be able to capture all this in the data. If you look at

grade 3 to 8, the everyday person would look at this data and say we have to be about 49% of

the students in NC grades 3 to 8, to about 64% of the NC students who are not proficient in

Mathematics. That is pretty dramatic. (My comment, I think what she is saying is the spread of

non-proficient student s ranges from 49 to 64% of our student population) An overarching

statement is our students appear not to be doing very well in Mathematics, not posing a

question, but is something to ponder for the commission

Andre: how will our recommendations address this that is what we should be looking as we are

evaluating our work?

Ted talking, we can’t hear a word, he said, but had to do with score deviation between 2008

and 20012 where there was not such a drop in the scores.


16/23

Howard. Responding to what Ted said, that we couldn’t hear, : That is an excellent observation,

what is happening there is the content standards are expecting students at the higher grades, 6

7 and 8 is trying to prepare them as they go into high school. Refers to Dr Curtis to talk,

Dr. Curtis: All based on content standards and testing changes, look at 2012 grades 3 to 8 those

content standards were different requirements per grade level but not necessarily lay in a

foundation issue as you go up to get more rigorous so that more students will be prepared

and successful in high school mathematics. So the goal is to be successful in HS math so they

can be successful in college.

Ted, all previous data shows prior to 1012, there was not much of a change in grade 3 to 8,

now all of a sudden we have this huge drop.

Dr Curtis: These standards require more conception understanding and thinking and they

assume mastery of previous topics.

Ted: Interrupts, did this not happen in previous years,

Dr Curtis: Not necessarily.

Ted: It’s kind of hard for me to believe that in teaching all these things you didn’t require

mastery to move forward?

Dr. Curtis: Were talking about 2 different things, curriculum and assessment.

Ted: I’m talking about the grades and what I see.

Dr Curtis: what would you like me to respond to Dr Scheik?

Ted: Well, I asked why the scores went down going from 3 to 8th grade post common core and

why they did not in 2012 and 2005.

Dr. Curtis: In higher education there are some variables, I can make some conjectures but that

doesn’t mean that they will be accurate, I can tell you that we have had standards changes and

also testing changes. Something you might want to look at, first of all is 2 years of data for any

group is not enough to talk about a trend in data, so how we look at pre common core we have

several years of data.

Ted Interrupts: I beg your pardon, but all years previous to 2012, not much change from grade

3 to 8, all of a sudden there is a drop


17/23

Dr Curtis: We have our assessment items now that she sees and reviews both current forms

and pretest. Speak to the fact that the assessments that they currently have, regardless of the

standards are a much high level of rigor, and our test in Math standards as a whole, not a test

per standard. Look holistically, the rigor of the test has been increased where students are

accountable for multiple standards in one test item. With a full blown implementation ofstandards, right or wrong all at once, when we do that, change lock stock, one time, we will

have will have students with gaps, no matter what the standards are. So the students and

teachers work really hard to close those gaps it gets smaller over time.

Ted: Dr. Curtis, I understand all of that , I mean I can very well see when they introduce new

standards the scores have a significant drop, and it takes, 6 or 7 years to recover to the prior

level, so I think what you are trying tell me is that after CC was invented, we made the tests

harder. Is that what you are telling me?

Dr. Howard: If I could please interject. The tests are aligned to the content standards. The testpart of the alignment, we are talking about saying that an item measures two digit

multiplication, then that is what the standard calls for. The standards are also written to

require a certain level of rigor. So the other piece of the alignment is it is not just measuring

two digit multiplication, it is measuring the rigor. What we are seeing here, in my test and

measurement opinion, is the difference, 2 things, one as the content standards go up they do

require more conceptual understanding. The rationale for that was to prepare the students to

be successful in high school and beyond.

Tammy: to your point though Dr Scheik……….He interrupts

Ted: You are not answering my question. My question is from 2003 to 2015, one of you said

that the tests got harder, boiling it down to plain English you added more to the tests as you

progressed through progressive grades. So the tests got harder, yes or no.

Dr. Howard: Yes, but because is aligned to the content standards. So the decision on how

difficult an item is, is not made just by someone sitting in a room and saying we are going to

make all these items hard that is not how it is made. It’s made in consistency with the adoption

of standards by the State Board of Education. That is a technical quality for test items. I’m

sorry.

Tammy: Aren’t the reformed items supposed to prepare the students for the more rigorous

standards? I think if they were properly prepared, we would not see that dip.

Dr. Howard: You are absolutely right, what we would expect to see as time goes on for that to

occur but if we back map from this year’s current 8th graders, then the 2014-15 8th graders,

who had their first exposure to these content standards in 6th grade, one of the things we are


18/23

watching and we would hope to see is as these kids progress with consistent content standards,

there is time for the instructional practices to catch up with the content standards, and that is

one of the important things about having consistency is to give it time for the instructional

practices to catch up with the content standards so the students become familiar and then to

see those increases, that is why you do see those increases over time. For example when youlook at 2006 to `12, and remembering to factor in re test as well.

Tammy: In grade 6, essentially you have students taking the test that have had 3 previous years

of higher standards, correct? I don’t understand why if they were prepared in previous years

suddenly now they are not prepared. In other words you are saying that in grades 6, 7, and 8

grade you are preparing them for more rigorous high school standards, but shouldn’t the

elementary grades reflect that as well, that doesn’t jive with me and to Dr. Scheiks’ point, this is

consistent for the past 3 years. You would anticipate that over that time frame there would be

steady improvement, but there isn’t.

Jeff: If I remember correctly Dr. Garland told us before that there was a gap at middle

education level and above, so theoretically it would take 10 years to catch up based on the

current standards. My point has always been that it is not fair to teachers, or children in the

way and how we are judging folks across the state. What is the increase in the number of

standards currently accessed? Is there an increase? Does DPI say we are testing this number of

standards now?

Dr. Curtis: Speaking to the test holistically, she would defer to Dr. Howard but I do know that

test items test multiple standards because we look at a test holistically, not item by item.

Jeff: I understand that.

Dr. Curtis: Our last set of standards were more general, they are comparable to what we have

now, not that much of a difference except in the way they are written is different,

Jeff: that being said, are our teachers across the state of NC confident in what students are

going to be assessed on at the end of each year or course, if there is an increase in specificity

are the teachers confident in your opinion? Based on the alignment of these numbers, I don’t

see that confidence in the classrooms.

Dr. Howard, 2 clarifications I think are very important, if you compare the school year 2014 to

15, every grade went up, not a lot, it didn’t go up 5 points, which is what we like to see, right?

But at the state level with such large numbers it is significant. That does not necessarily look

like we want it to look, we would rather see 48.3 go up to 52 then 48.8, but never the less this

level is significant.


19/23

Jeff: It is significant unless you are on the other side of that number as a child and you did not

pass that exam.

Dr. Howard: I agree with that wholeheartedly.

Jeanie Metcalf: If there are 200 standards and we can’t test them all on the test maybe that issomething we should look at. My other comment is about the statement that standards are

driving the tests, and I am so tired of people saying that standards don’t drive curriculum, well,

if the standards drive the test, then standards drive the curriculum.

Andre: We need to get our standards aligned to our tests then it will align to the curriculum as

well. But right now we fail. Standards are one thing but we have to follow through on the

others.

Dr. Howard: Can I clarify the 200? I think it is important to clarify, the comment about having

200 standards. The example that we often get, and you’ll probably agree Dr. Isenhour, is thatteachers want to know how their students do on these tests. They want to know that at what

we call the content standard level. They want to know what percentage of their students

performed well on the items that measured “X” standard. To do that, we would have to have a

very long test with multiple items and that is where I think the comment comes from that it

would take 200 items and a long time to measure at that level to say with confidence that at a

student level, these tests are developed for the school level ideally. So the student level the

entire score on the test is a reliable score that we are unable to get teachers the standard level

information, we do give them groups of standard information because we have more items.

The standard level information would be huge. The example that I give to illustrate this is if wehave a basketball game, and we have a free throw shooter, then we pick some someone

randomly off the bench, he makes the shot, the next time we need a free throw shooter, we

could get him and he could get the shot, or miss the shot, we have no way of knowing what will

happen next time. That is why they get a % of free throws listed across the bottom of the

screen. It is the same with test items. To say that a student has mastered a content standard,

you have to have enough items to say they demonstrated at a student level. The student level

assessment for that type of reporting is very different from a state assessment that is reporting

on the entire test. Also while we have an assessment system that includes formative

assessments, for teachers to have daily in their classroom, to see how students are doing atthat standard level. While we also have benchmark assessments that measure fewer standards

on an assessment rather than the entire years’ breadth of standards. We have to remember

this comprehensive assessment system and the Summative assessment is a snapshot, at the

end of the year to get this kind of data to have state level conversations, to have district level

conversations depending upon the population.


20/23

Jeff: I hear what you’re saying, I just want to make a point of this, and you are certainly

welcome to disagree with me. I think we have created a standardized culture in this state with

the number of assessments and the way we give assessments and we have not created a

diagnostic culture in the state to give feedback to the teachers and the students on how well

they are progressing, because when those summative assessments come up at the end of theyear, the feedback, the data that we get, it's worthless. They are not able to serve kids well

because of this standardized culture. They are just now getting teachers understanding the

system, my concern is that we have kids every year that go through the process and it may not

seem that way to us, but it belittles kids taking the tests, these long drawn out assessments that

they do not do well on. That process in this state needs to be changed dramatically.

Andre: So here’s what I think we need to do, we have talked about this for a while. I am not

satisfied that we have gotten to the point of clarity that we want to achieve just in the point of

what Dr. Sheik was asking and I think what would help us to do that better is if you could come

back to us with a written response in that particular point. Let me be clear on what we are

saying here. In both reading and math, the same thing is being portrayed, which is when you

look, and I get your point about going horizontally on this chart, year to year, that horizontal

year to year is in a particular grade, but one of the things we are concerned about is

progression. So when a student goes from K to 12, we saying that our standards are helping us

to make sure that they are better prepared to get from K -12 and progress into college and

career , that is what the premise is, but when you look at this, vertically you are actually

declining . I don’t want you to explain it to us here now, we’ve tried that.

Dr Curtis: Interrupts, telling him she wants us to look at it diagonally, follow cohorts ofstudents, not vertically.

Andre: That’s what I’m saying is that that is not on this chart, right? I want to see it written

out very clearly , because what I want to see here is why is it that this is happening, I think what

you are arguing is not particularly the way we are looking at it, what you are arguing is that

actually in a certain grade level the students are improving year to year

Dr.Curtis: I’m not arguing that.

Andre: Why is it, Tammy made the point, if that is the case then why is it that they are doing

worse in the other grades. I just want you to come back and be real clear with us on the

dynamics underlined on that. I want to have the discussion here so that is clearly documented.

It is important because once again as we try to address how standards reflect the outcome, it is

not clear right now, based on the questions I am hearing.

Dr. Curtis: Can I share one thing before they close, Look at grade 8 and look all the way across

to 2014, you see 34.6, follow those kids to 2015 and math 1 and you see 48.6, that is the same


21/23

group of students, that is the type of year to year comparisons we should be looking at because

over time those gaps will get smaller. The growth will increase.

Andre: Right, but that is the point you need to give us chart showing us that way that is the

more relevant way for this to be looked at.

Ted I have one more comment, when you changed standards in 2005-6 scores dropped from

around 88 to 63 and it took 7 years to recover. When you changed the standards in 2012, it

dropped form low 80’s to about 45, which is a much bigger drop. My other comment is, and I

don’t want to have any further discussion on this, because we are just going around in circles.

But I want to point out that if you look at any of the grades in 2006 they had a big drop and

had gradually went up, and went up faster than what has happened after Common Core. The

rate of increase is substantially less. Understands drop and recovery time, that is clear, wants

to emphasize recovery seems to be slower with Common Core than otherwise, Can we end this

conversation now.

Andre: You understand what we are looking for. We want to see the trend analysis, the trend

as standards change, and the cohorts as they go through the progression.

Jeff: The cohort we need to look at has not been tested yet, the 2016.

Andre: There is something else going on here, you know this discussion we are having is exactly

what parents are seeing and asking about now. You get a letter telling you that your level 3-4

student is now a level 2 student, and you can’t understand what happened? No explanation,

How did they get dumber that fast? I think the dynamic we are talking about is, it is not bad

news, it’s just that we raised the bar and all of these things that we claim, we can talk about

that here, but we have to communicate that to the parents. (My comment, why would they

want to raise the bar and lower proficiency.......then go to a 10 point grading scale for the

student, and a 15 point scale for the school, smells to me)

Dr. Curtis: You are right about that, we could have done a much better job at the state level,

local level, all over the place. I can say this, we adopted the highest standards.

Andre: I am the parent of a child, and I get those reports.

Dr. Curtis, I understand sir, I am too

Andre: I am just stating that I don’t feel like this has been communicated well.

Dr. Curtis: I was agreeing with you sir, that we could have done a much better job. What I will

say is we adopted very high standards for the students in our state, hoping to make them

college and career ready, because the previous standards were not near as rigorous.


22/23

Andre: I am more on the communication, than the underlying data.

Olivia, I have a comment on the math table. At the top of the sheet of paper it could just as

easily been the ELA table, I have a theory about standards, based on teaching, testing and

learning herself. I think what was going on in 2007, 2008 to 2012, says our standards were

pretty rigorous back then, and if we say that they weren’t, how do we explain that to the public?

I mean how do we say to the citizens of NC that between 2007 and 2012, the students sailed

through because we had easy standards? I theorize that something was going on with the way

the standards were understood by the teachers, they were probably more explicit, I don’t want

to say simple, that is not my term they were simplified, adaptable, teachers had enough hands

on with very succinct, clear refined tuned standards that their confidence level in teaching and

assessing those standards was quite high. I am not here to argue that with you, that is my gut

instinct, there was something right about the standards, I know we are not going to go forward

and say the reason we had 81% proficient level in grade 3 is because we had easy standards.

No! There was something else going on that felt right about those standards, something

worked right then. I only say this because we are about to make some recommendations a

report and look at a vast amount of data collected from teachers from surveys, and we just

have to get the statement about standards right, we really do.

Andre: Any more questions? Thanks Dr. Curtis and Dr. Howard, acknowledges he has given

them more work to do, said even though we have these debates, at the end of the day we are

all North Carolinians, and want what is best and right for the citizens of NC.

New business: appreciates everyone coming today, looking for motion to adjourn.

Public session closed, the 2 work groups recessed to work on their respective reports.


23/23

Documents

ASRC FRiday the 13th - CCTA Minutes.pdf