Running head: CLASSROOM ASSESSMENT PRACTICES · instruction and assessment in response to student needs” (Ontario, 2010, p. 31) In this study, teachers focus on both “Assessment

Running head: CLASSROOM ASSESSMENT PRACTICES

Classroom Assessment Practices

of Early-Career

Ontario Secondary School Teachers

Terry Whitmell

LHA 3042

For: Prof. Jim Ryan

Ontario Institute of Studies in Education

April 2017

CLASSROOM ASSESSMENT PRACTICES 2

Abstract

This paper analyzes qualitative data gathered through interviews with nine early-career

teachers in an Ontario secondary school, and examines the data in the context of the current

Ontario Ministry of Education policy document: Growing Success.

Keywords: assessment, evaluation, feedback, grades, rubrics, Growing Success


Classroom Assessment Practices

of Early-Career

Ontario Secondary School Teachers

In 2010, the Ministry of Education of the Province of Ontario published Growing

Success: Assessment, Evaluation, and Reporting in Ontario Schools. (Ontario Ministry of

Education, 2010)

Implementation was the responsibility of individual school boards, and in 2017, seven

years following publication, there are many aspects of the policy that are not yet consistently

adhered to by the teachers in my school. I have found little research or reporting on the effective

implementation of this policy document, nor on the resultant impact on student success. My own

observations, as a secondary school administrator in four schools, have been that there is great

variation in the levels and depths of understanding of Ontario’s assessment policy among my

teaching staff.

My search of articles in peer-reviewed journals resulted in only three publications that

referenced this policy document, and only one of which referred to its use as a basis for

classroom assessment.

However, the topic of classroom assessment is broad and well-researched. It “occurs at

the intersection of three teaching functions: instruction, classroom management and assessment”

(Brookhart, 2004, p. 430) and so it is relevant to all teachers.

Given the wealth of research that can be found regarding classroom assessment, it is

puzzling that there has been no study to consider the impact that the Ontario Ministry of

Education policy has had on classroom assessment and student achievement in Ontario. Nor has

there been any examination of how effective the policy implementation has been.


Research Question

How do teachers connect their current assessment practices to student learning, and what

is the role that Ontario Ministry of Education policy plays in their practice?

While a comprehensive study, involving all teachers, would be ideal, I considered a

subset of teachers who might be able to respond to questions from a position of interest,

experience and knowledge. I limited my potential pool of interviewees to those who I knew had

experience with the Growing Success policy, and had implemented aspects of the policy into

their teaching and assessment practice. Since I annually interview and hire long-term occasional

teachers to fill positions of teachers on leave, these teachers would already have met this criteria

in order to be hired to their current position. They had also, through the job interview process,

demonstrated that they were reflective practitioners and would be willing to share their practice

with me.

Analysis of the teacher interviews provides a wealth information beyond the scope of the

initial research question. This report addresses only those practices mandated by the current

Ontario Ministry of Education policy, Growing Success, and the tools available to classroom

teachers to manage assessment in their classrooms.

Conceptual Framework

Susan M. Brookhart’s literature review identifies three theoretical bases of classroom

assessment: studies of individual differences, studies of groups and the study of measurement.

(Brookhart, 2004, p. 430)

Individual differences of students are addressed by teachers based on frameworks of

learning and motivation theory, feedback theory, and teacher beliefs and practices. Stiggins’

(1999) work on student motivation, and Black and Wiliam’s (1998) connection between student


motivation and formative assessment, inform today’s assessment practices. Feedback theory

claims that constructive feedback improves student performance (Ross, Rolheiser, and

Hogaboam-Gray, 2002). And teacher beliefs and practices are grounded both in their experiences

as students themselves, and in the training they receive in pre-service programs and in-service

professional development.

Theories about groups address classroom assessment environment theory, social

constructivism, and culture and theory. Since this study is based upon teacher conversations,

rather than classroom observation, these areas are not addressed in this study. Further research

that examines both the classroom environment and the students’ interactions would be necessary

to connect this theoretical basis.

Measurement theory is the basis of much of the assessment work of teachers: considering

validity and reliability, and theory of formative assessment.

Validity and reliability are grounded in the connection between objectives and

performance-based assessment. In this study, teachers discuss how they are able to connect their

objectives, their assessments, and the learning of their students.

Formative assessment is “Assessment for learning”, as opposed to “Assessment as

learning” or “Assessment of learning”. And formative assessment is “used by teachers to monitor

students’ progress toward achieving the overall and specific expectations, so that teachers can

provide timely and specific descriptive feedback to students, scaffold next steps, and differentiate

instruction and assessment in response to student needs” (Ontario, 2010, p. 31) In this study,

teachers focus on both “Assessment for learning” (formative) and “Assessment of learning”

(summative), and share their experiences and challenges as they work to gather data to provide

feedback to students and adjust their instruction to better meet the needs of their students.


Context

The subject school is an urban Ontario secondary school, of approximately 1250 students,

with both English and French Immersion programs. The students are aged 14 to 19, and are

diverse in their ethnic background.

I approached twelve Long-Term Occasional teachers, all of whom are currently teaching

in the secondary school of which I am the Principal.

Nine teachers were available to be interviewed: all are in their first seven years of

teaching, and none have permanent contracts with the school board. They have either completed

their pre-service teacher training after the publication of Growing Success in 2010, or have

participated in induction programs that focused on the document. As well, they each have had

many job interviews in their short careers, since the nature of their work requires that they apply

for and be interviewed for each position and often attend multiple interviews before being

offered a position. During annual job interviews they would be expected to respond to questions

relating to classroom assessment, evaluation and reporting, where the Growing Success policy

document would be referenced. This was predicted to support a thoughtful, considered response

from each teacher to the questions posed.

The teachers have been teaching from one to seven years, and have had long-term

assignments in one to four different schools. They teach Mathematics, Physics, Science, French

Immersion, Core French, Social Science, Business, Music, English, Drama and Visual Arts, with

several of them teaching in more than one discipline. I hired each of them this year to replace

teachers on leave, and so they have all been interviewed by me at least once prior to this process.

Design & Methods


In preparation for the interview, I provided each of the teachers with a set of questions, as a basis

for our conversation:

1. How does your choice of assessments support the learning of your students? - Which do you find most effective? - Is there information you’d like to know, that the assessment tools you have don’t

provide?

2. How do you manage the assessment data that you collect? - Do you collect quantitative data, qualitative data, or both?

3. What tools do you have that assist you when you use your assessment data to report

on student achievement? - Do you use Markbook?

o Is it an effective assessment management tool?

4. What tools do you have that provide students with feedback and information about their achievement that helps them learn?

5. What tools do you wish you had, to improve upon the gathering of assessment data,

analysis of the data, and communicating or reporting of your students’ achievement?

I began the interview with the first question. I then followed the initial response with

probing questions to explore more deeply, with a focus on why and how each teacher plans for

and gathers assessment data, and then how they manage this data in order to provide feedback

and evaluation. Many teachers brought notes to the interview, and all responded fully and

thoughtfully.

I recorded the interviews using a handheld digital recorder and Bleutext, an iPad app.

Bleutext provided a rough transcription, which I then edited for accuracy using the digital

recording. The nine teachers were provided with printed transcripts, and were invited to provide

corrections and additional input.

I transferred the transcripts from Word to Excel, with one sentence per row, numbering

each text cell and identifying each as being either question or response. I then coded each

sentence by theme, and then by subtheme. Sorting then allowed the themes to be expanded or


further refined. I identified themes related to the research question, and filed the remaining data

for future analysis.

Results

The teacher responses could be categorized into five main themes:

1. Choice of assessment tools 2. Data gathering tools 3. Feedback tools 4. Data management and analysis tools 5. Next steps

Choice of Assessment Tools

The process of backwards design (Wiggins, 1998) involves grouping course expectations,

determining what would be evidence of attainment of the expectations, and then planning

instruction to support successful demonstration of competence using the chosen assessment tool.

One teacher explained this by saying: “So the success criteria is obviously going back to the

curriculum: What is my learning goal for today? What do I want my students to achieve?” And

another explained that “I always fall back on a printed curriculum expectation document in my

binder, and I’ll read the strands and I’ll check them off as I go along.”

Both assessment and instruction require that the learner is motivated and engaged, in

order to succeed. Teachers’ choice of assessment tool is made with consideration of the students’

potential engagement, as this will impact their ultimate success.

Growing Success identifies three categories of assessment tools: products, observations,

and conversations. One respondent referred to this: “So how I come up with the mark, I try to do

observations, conversations, product. So it’s not just what they write on the test, or what they

hand in with projects.”

Traditional testing, one of the most common products, presents some challenges:


I don’t use the word ‘tests’. It seems to be very worrisome for the students… and I guess some people have test anxiety…they feel like they can’t do it, and so I try to avoid using that term. I wish we could do (conferencing) for final evaluations for those exams, especially in English. They get to see their exam once it’s been given back, but we can’t really discuss why they came to these conclusions, how they came up with these ideas, and it’s a little bit frustrating for me. One teacher considers the issue of providing unlimited time, or of chunking assessment

into smaller units to support student success as being particularly problematic:

(I feel) it’s not valid, because if you think in life, if you’d go for a driving test, they wouldn’t say you drive this, now you go home, now let’s do the other part. In life there are so many things that are not broken for you just because I want to make them a bit easier for you. Teachers choose to use student products, beyond the traditional pen and paper test:

Certainly just going with traditional quiz and assessments or tests wouldn’t be the way to encourage them to do better, for sure. So, I, obviously you need to take other ways like projects that are interesting, or hands-on projects, that they have to do or… real-life situations…like you’re gonna buy a car; do you understand what’s an interest rate, how they play in your life later on? I give them the choice of how they want to present it. So if you want to make me a comic book, make a comic book. If you want to do a presentation, if you want to write a rap song, you know, play to your strengths.. However you want to deliver, that is entirely up to you. There was a video project, and I thought that was really cool that they were able to look at themselves and listen to themselves speak French, before they handed it in. They had a checklist and rubric, and then I entered the data in Markbook. Teachers utilize student observations, to gather data for both formative feedback and

summative data for evaluation. One stated “That’s something that I’m still trying to work on as a

new teacher: recording, or keeping track of my observations, because it’s not always easy to be

in the middle of… watching them…yes, I’m taking notes”. And another commented on the

strengths of this as an assessment tool: “:I love observation, because I find when you do the

student product, sometimes if you haven’t scaffolded enough, you can get poor results”.


Teachers engage students in conversation, to provide immediate feedback, and to gather

data for evaluation. They explained why this is valuable: “If she wouldn’t speak up, how would I

know? When it’s the quiz time, maybe it’s too late? When it’s the unit test time?” Another said

that “I notice that they’re doing things, that sometimes hinders our conversation for me to stop

and start writing.” In a music class, students “did a pre-performance interview and an after-

performance interview. We talked about this before, you did it then, and I’ve acknowledge it,

and that’s great.” This was seen as a great strength of this tool: “In my ideal work, it would be a

lot more student conferences. But it takes a lot of organization and planning to have the time

where you can do a one-on-one, even for two or three minutes. For 20 kids, which is a small

class size, that’s the full period.”

One teacher reports working with a student who had problem writing, so she offered to

scribe for him. The previous assessment resulted in a mark of 30%, but with scribing he was able

to achieve 75% on the same content.

While teachers confirm that conversation is an excellent method to gauge student

understanding, they are concerned about how students will perform beyond secondary school:

“I’m going to get this kid having 85% and then he’s going to go to university and nobody’s

going to test him that way. How’s he going to cope with real life after that?”

Teachers make use of observation: “I do a lot of anecdotal observations, that I just write

down”.

Differentiating between formative and summative assessments, and allowing products,

observations and conversations to serve both needs, is a challenge for some teachers: “If I find

that they did really well, and they learned a lot from it, I will make it officially, I don’t say

‘count’, but it might be an actual mark, and make that a summative assessment”.


Feedback is only seen as useful if the students actually read or hear the feedback. “I find

that, especially the academic students, they just go straight for the number. One of the things I try

to do, so that the requirements are clear, is create the criteria with them, you know co-construct”.

Data Gathering Tools

Most teachers make use of a Learning Management System to provide course resources,

organize student tasks, and to receive student products.

One teacher finds Desire2Learn supports her students:

I can see exactly when a kid opens a file. I know if half the class was watching that video or not. I think it’s really good, instead of you going home goving over this paper or notes, you can just watch that movie or video in three, four minutes, seven minutes, and it’s a review of what was done yesterday. Google Classroom is a Learning Management System that has features that teachers and

students value, saying: “Google Classroom is something they can do on their phone” and

“Students need to see something written down. So Google Classroom is a good way for doing

that. Everything is just there, and then I can also keep track of who’s submitting what on time,

and who’s submitting it late.” It’s not just text communication, but “I put (audio files) on Google

Classroom, they get an email notification themselves, and it does remain there. And so it’s that

whole building of a portfolio that I talked about”.

Sesame, which is discussed later as an assessment data management tool, is also used by

teachers to record student observations. It is particularly appropriate when used with a teacher’s

personal device: “I was there with my iPad and just going around, selecting the student, writing

down some noes, and taking snaps”.

Paper and pen remains a valuable data gather tool for teachers. In addition to the

traditional process of quizzing and testing, and of collecting student products, teachers make use

of paper to record both observations and conversations. On teacher explained the process as: “In


my binder I have my lists of my students, and then if there is something that I’m observing, I’ll

write it on the top, and then I’ll do the checkmarks.”

Feedback Tools

Many teachers report that their students still prefer feedback in the form of written

comments on paper, saying “The lab reports, they’re covered in ink when I give them back”. The

value of this format is explained: “So rather than communicate it to them in English, because I’m

trying to stay in French, at least if it’s written down, then they can read it, take the time to

understand it, and then come find me and ask question.” There appears to be pressure on teachers

to move away from this traditional format: “Even though I feel like I should be using more

electronic means… I find that I’m still like very pen to paper type person.”, and “Right now the

feedback is still pen and paper”.

Some teachers are making use of electronic tools for feedback, and others express interest

in moving to an electronic format, explaining that “It’s mostly done online, right on the

document that they prepared for me”. The ability of students to retain the feedback, and view it

at any time, is valued: “Being able to give them some sort of thing they could look at

whenever…. And say: these are the areas of need you still have, ‘cause we’re moving on to the

next unit”. It also allows both teacher and student to retain feedback information:

So if they could have an electronic copy of that, that would be really good too. ‘Cause there’s no way that these kids are going to hang on to a grade 10 lab report all the way until first year university.

A colleague annotates digitally, and she can draw, and it shows up in real-time. And when they get it back, not only the physical copy but digital copy, there’s more that is being given back to the student in terms of feedback. An electronic form of feedback, used by one teacher, is auditory annotation. He utilizes

Garageband, and is considering use of Swipe for his next class assignment.


I recorded them, gave them, a rubric themselves, and sent them the recording this weekend and I asked them, evaluate yourself. And actually what I did on top of that is I recorded on top of their recording… a mini-adjudication. Today we’re going to compare our marks, we’re going to compare the feedback.

Several teachers find conferencing to be an effective means of providing and receiving

student feedback: “With my immersion kids it’s usually personal discussions we can have,

whether it’s at the moment or after class”. It’s seen as useful when one assessment has not been

successful: “If they didn’t do well on a certain assessment, is it because they didn’t understand

it? Did they not get enough practice in class? So, after that conversation, I would see what the

students are missing, and then obviously try to alter what I do in the classroom.”

Another group of teachers makes use of online forms for “..self reflection after

assessment. So I have a Google Form where they put their thinking. They choose from the list,

and …spot for one or two sentences to say it in their words. I can see if I have five kids out of

twenty who are telling me the same thing.” Ease of use is a key benefit: “Making a Google form

is perhaps the easiest way, so that way I can actually have it accumulate all into one spot.”

Teachers report using Kahoot (https://getkahoot.com) as a tool to provide feedback to the

students, as well as feedback to inform their instructional decisions from students who might not

volunteer orally in class: “While in Kahoot, you don’t need to speak up. You need to pick out an

answer”. This instruction feedback supports improved planning and instruction for teachers:

We’ll do it in class a couple of days before a review day. I look at the data and say, okay, here are the areas of weakness for the kids in the class. And I can also decide, do I need to go back and cover this again? Because, if no one’s getting it, then obviously they’re not ready to be tested on it. It’s immediate, it’s interactive, they’re getting that feedback right away, and I’m not sitting there marking 30 quizzes. And so we just look at the bars and ‘OK, so everyone’s OK with this type of question’. And then if I see something that’s flawed, I take that moment as a teaching moment and revisit the idea again before we move on to the next question.


I really like Kahoot, because it does that (strand, expectations) for me as we’re playing. You’ve got the bar of: These are the kids that answered this question, these are the number of kids that answered this question, and in my head, I know the strand that goes with. A similar function is ascribed to Mentimeter (www.mentimeter.com) which combines

presentation software with interactive feedback functions. On teacher reported that “I take a poll

at the beginning of class; it’s very engaging.”

Peer feedback is also seen as an important component of assessment. This works well for

group activities: “The next video project they’re doing, they’re creating it in groups, so I’m

hoping that they’re going to give each other feedback as they’re doing it.”

Self-assessment is also valued, and is provided primarily through the use of Learning

Goals and Success Criteria, rubrics, and checklists, “Whether that be using success criteria with a

rubric, or also using just a checklist, or sometimes I’ve combined the two.” Flexibility and

individualization make this a powerful process:

I’m currently using a mix of checklists and rubrics.. Because if a student sees…. ’sometimes makes grammatical errors’, that’s where the checklist comes in, and I’m saying a level four, you didn’t have any grammatical errors, a level three equates to this many check marks, and you have more checkmarks as you go up the levels. Sesame, an assessment management system, is also noted as being a useful feedback

medium. The one teacher who uses this tool regularly, shared that “They have their own

individual code to login, they see their account, they see any rubric that I posted, they’ve seen the

comments that I’ve written down, they’ve seen the pictures that I’ve taken”. Since Sesame also

attaches curriculum expectations, the feedback provided is concrete and actionable by the

students.

All teachers speak to a need for a student portfolio, where students are able to organize

their products, and provide data for self-reflection. Its value is seen in the ability to reflect, over


time, on progress: “I think that would be amazing if all students just had a portfolio, something

that they could track throughout all four years.” A portfolio is able to provide a broader

perspective on student learning:

I think if they were able to… see how much they’ve improved, and how they spoke in grade 9 versus how they speak in grade 12, they’d see a huge difference. I think that would give them the motivation they need to either pursue French later on, or at least finish the program and feel like they were successful. I think it would be a good motivational tool. So it would be great if students could hold on to both the feedback and the assessment.

I photograph a lot of stuff. We also have a portfolio that they create at the beginning of the year, so they can keep all their process work and drawings. So even though their actual product might be gone, all of the process and all of the lead-up work is still available for me to show somebody’s skill level where they’re at and justify anything.

I’ve tried portfolios with my Grade 10’s, because I found that the grade 10 cohort is in such a place in high school where their marks don’t really matter. I let them choose what their best ones were, choose three out of four.

Data Analysis and Reporting Tools

There are a wide range of tools used by the teachers interviewed: Markbook, Sesame,

Student Learning Notebook, Markboard (in chalk.com), Excel, and paper. The decision to

choose one tool is a difficult one for many teachers: “I want to find a good way to keep track of

my observations and conversations in an efficient manner, that doesn’t affect the actual

observations and conversations that are happening”.

All teachers interviewed use Markbook, the assessment management tool provided by the

school board, for some of their assessment recording, and to provide data to determine final

grades. There are many concerns with Markbook’s functionality, including “If you didn’t back

up properly, if you print it out a blended mode compared to the average median… When they

look at a graph it only shows them a line graph, a plot graph.” Concerns, and suggestions for

improvement, we consistent throughout the nine interviews:


I find that Markbook… very hard to navigate. ‘Cause right now I’m using mark sets, so the weighting is always changing. And then when I give marks the next time around, the weightings are going to change. One thing that I find with Markbook, it seems like it’s disconnected form other things. Like it’s got a sole purpose. “It’s not very appealing in some ways. I mean Markbook, it just feels like you’re operating a dinosaur. If it had a section where I could put the learning skills, and maybe just check off boxes for learning skills. And maybe even have the big ideas in the curriculum in it. With our comments you have to refer back to a specific strength or a specific weakness.. And I think if I have that level and then here’s the curriculum document that it matches to, I think it would make writing report cards a lot easier.

However, teachers find the graphing tools in Markbook useful for the students and to

inform their instructional practice:

I love being able to give kids their distribution, and then compare it to the class average as well. If they see on a unit…where people have done pretty well, and they haven’t done as well, they know they can go and talk to their peers.

Consistently, teachers express concerns regarding the choice of marks versus levels, and

on the method used to calculate the marks, observing that “We’re pretty attached to grades”.

There are concerns with the algorithms utilized, both when using levels and with traditional

marks:

I don’t trust the way they calculate marks. I know we put in category weighted average and stuff, and it’s supposed to work out. But like seeing what happens with blended modes, if you forget to switch it over, they can make like a 25 percent difference for some kids.

When considering the use of levels, which then generate anchor marks, one teacher expressed concern: “how do I explain why a kid didn’t get a 78? And they got a 76 instead.” I think they just don’t really understand what a level four is, because there are a range of marks within the whole four, and what’s the difference between a three plus and a four minus? I just fall into the trap of, students like to see specific numbers, but sometimes levels can be beneficial.


Levels sometimes work. But I feel like I then I would use my professional judgement. Like, how do I gauge between a 92 and a 94? Especially when they’re applying to universities. A lot of students still look for that mark to benchmark themselves, to understand where they stand. And it’s hard because these students are in grade nine and I don’t want them to fall into the trap of being ‘marks are the end-all and be-all of your educational career.’ ‘Cause then you’re a slave to a number and that’s not what education’s about. Sometimes levels and marks, the percentages don’t necessarily match up.. I use my professional judgement. I’ll pick and choose. I go into my Markbook and I play around with that. You still have to put in a level, or you still have to put in a numerical value. I think checklists would make it a lot easier, and then room for comments if you have any.

Student Learning Notebook is a web-based app designed by the subject school board,

which allows students to connect their products in Google Classroom. Teachers can then

articulate “over-arching learning goals”, and evaluate the student products to assign levels of

achievement of the goals. There is interest in this as a new tool: “I think it’s the right idea, going

web-based, and not have to rely on Markbook”. However, those who had tried this tool, identify

limitations:

When I’m in the Student Learning Notebook, it takes too long for me to find the student, when I’m doing an observation, and I want to write the note really quick. And I found that almost stressful to try and click the student, try and do the learning skill, and write a little note, so it actually had some meaning to it.

Sesame is a data management tool used by several teachers. It records assessment data,

and allows teachers to view the data. One of its strengths is its ability to connect curriculum

expectations to assessment data. One teacher explains that “I prepare a checklist or rubric on

there, and just so that I remember what I saw, I’ll take snaps and that saves it on the website”.

Another is able to “attach it to the curriculum so that it will ask me what grade, what course, and

there’s the strands and I just take them. It shows me the ones I’ve selected so far.” A strength of

this tool is that “It does give you a breakdown of how students performed in each section. You


could do a rubric, which would be level, you can do a checklist, you can do an ‘out of’, be like

points based. I usually us it for, as a rubric.”

An eclectic approach to assessment and evaluation tools is evident among the group of

teachers. One teacher uses Sesame for observations and conversations, and Markbook for

products. She then uses the data from Sesame, the data from Markbook, the student work on

Google Classroom, and her paper notes, to use her professional judgement to determine a mark

to report.

A tool supported by the Ontario Teacher’s Federation is Markboard, in chalk.com. It is

used by one of the teachers, and appears to have strengths in allowing for connection to

curriculum expectations, to allow for recording of product, observation and conversation, and to

provide excellent feedback tools to students:

What I like about chalk.com is that it also has something called Planboard, so you can start tying in your calendar. It’s a more multi-faceted software program, because the constant logging into this, logging out of this, logging into this, logging out of this… If you could just have more things in one spot, that is the convenience factor. As far as the marking portion, it is easier to view, it is more graphic-based, there is none of that blended mode and average median mode. It’s easy to just change the weighting, it’s easy to omit things as just diagnostic, or not have things weighted within a final mark. But also you can just assign levels. They actually have Chalk.com into Google classroom as an extension. So that to me has future-proofed itself. But in chalk.com they also show bar graphs, they also show pie charts. So for visual learners, they can see visually how they’re doing, how they’re weights, where are they doing well, where are they not doing so well. If a student doesn’t know how well they are doing, or how poorly they are doing, they won’t be able to really move forward. It’s actually all of the curriculum documents, and all the expectations. And so when you assign a lesson plan or any of your marks, you can say this. You know this assessment is tied to grade 9 academic English, strand A, creativity, 3.2 creative process. It helps me look at the whole picture. As I’m doing it, it checks it off. It shows back to the weighting.. that we struggle with the media components.


Despite the availability of many new assessment data management apps and programs,

paper and Excel remain the preferred choice of many teachers, due to their simplicity: “I feel like

something maybe as simple as an Excel spreadsheet… because I’m not as tech savvy as I’d like

to be”. This was seen as useful both to gather qualitative data, and to organize quantitative data

prior to analyzing it in an electronic tool:

I’m trying to actually just have a notebook, which is the kids’ names, and then if something specific happens in class, then I can make qualitative data about them and then hopefully kind of read through that when I’m doing my reports.

I always mark it on paper somewhere. I will document if it’s observation, conversation or product. I’ll mark what it’s out of. I will say what part of TACK it fits under. I have a hard copy with me at all times. Then I’ll go and I use Markbook.

Connecting course expectations to student assessment data is an identified weakness in

most of the tools available to teachers, with the exception of Sesame and Markboard

(chalk.com). Most report using some form of paper chart to organize expectations in the planning

process, and some utilize this chart to track throughout the course. However, of the tools

identified provide them with feedback during the course of which expectations had been

successfully achieved. So, they are missing information that will allow them to adjust their

instruction, and be confident that their students have achieved all course expectations or success

criteria.

Student motivation benefits from a teacher’s effective use of curriculum expectation data.

One teacher explained why it is necessary to track expectations: “You don’t want to be too

repetitive, because once things become redundant it’s a waste of their time and students lose

interest, you get that disengagement.”


Goobric is used by one teacher to design rubrics based upon course expectations. The

teacher is then able to view the distribution, and talk with the students: “were these sort of

expectations met?”

There is frustration with the area of Growing Success that addresses reporting on

Learning Skills, where teachers assess students’ performance as Needing Improvement,

Satisfactory, Good or Excellent. One teacher reported that “when it comes to Growing Success,

it’s a lot more vague. When I’m looking at the exemplars that are given, I can’t really apply it to

every single class that I’m in.” The lack of exemplars, and lack connection to the realities of a

secondary school classroom, were identified as a failing of the document:

They say ‘an organized student will show this’. And it’s maybe a two-liner or three-liner. But my students show organization in many more ways. It feels like instead of backwards design it’s forwards design. That I’m taking what my students are doing in the classroom and seeing if it fits into Growing Success. It is also observed that although all teachers follow the Growing Success policy, and

report on report cards that are common to all students in Ontario, there is little to ensure

consistency from one school to another.

How does one teacher from Peel compare to Dufferin-Peel compare to TDSB? What does an 80 look like across all the boards. How do we come to the conclusions that that student that I have performing at an 80 is the same student that you have performing at an 80? If something can be a little bit more universal through all the school boards, at least in Ontario, that would also help inform teachers, like, we’re on the same page. This is how you’re going to be doing it.

Next Steps

Teachers were asked about what they would wish to have as tools, to assist them to use

their assessment practices to support learning. A common thread is a wish for everything to be in

one place: learning management system, student portfolio, feedback communication, assessment

recording, and mark calculation/reporting. When asked to consider such a tool, one teacher


responded: “Everything’s all in one place? That would be amazing.” Others provided

specifications for the potential tool:

I think that would be amazing if there some sort of checklist. Something online so that we can just track to see what has been done? Where have you checked off the points? Is there anything that has been missed? Or anything that they need to improve on? Especially if there’s something electronic. It would be awesome if attendance was connected to one thing, if Markbook was connected, if a student portfolio was, curriculum, if everything was in one. Everything’s all over the place. The fact that everything could be accessed on one website, I feel like would make it a lot easier. So that you know where to go and how to plan, as well for lessons, and that Markbook is…Like I said, if everything was just into one, I’d be happy. It’s trying to give them as much feedback to be successful later on. And it’s also, for me, administratively, organizing trying to keep myself afloat, and not drown in paper, not drown in all these formats. It’s a really tough battle.

This transparency is what it’s all about. Being able to know how they’re being assessed is not only important to the students but to a lot of parents. And especially if we want to build that community, the collaboration all three ways, we can’t be ‘the classroom only stops here, academically’. I’m more of a ‘checklist, let me do it on paper first and then you have to enter it’ because I’m not in the habit of walking around with a device. But if I could open up my binder and do the exact same thing. You know I’ve got my seating plan, OK today: check, check, check, check. I think that makes it a lot easier. And if you could then put the template down, take a picture of it, and it translates that into the software, that would be amazing. I think if it’s simplified, or if it’s electronic, it’s just a click of a mouse or a click of button on your iPad then I think it makes it a whole lot more doable and you’ll have more people maybe motivated or wanting to use it. I would love to click on a student, say that she achieved that goal, and then have her email send ‘Congratulations, you have shown sufficient evidence for a satisfactory’, or for younger students, an emoji, a super smiley face!

Discussion

I asked teachers to connect their current assessment practices to student learning, and to

describe the role that Ontario Ministry of Education policy plays in driving their practice.


The teachers interviewed connect policy to practice, but are not able to effectively

provide evidence to connect practice to student learning. Instead, they are focused on the

operational and procedural challenges of gathering and assessing student work. They trust that

following Ontario Ministry of Education policy, and utilizing tools provided to them by their

school and school board, will result in improved student learning.

While teachers report organization of their assessment data within the four achievement

chart categories, “more teachers are beginning to organize their grade books around learning

outcomes”. (Schimmer, 2012, p. 117) Tools such as the Student Learning Notebook, Markboard,

and Sesame connect assessment tasks directly to either curriculum expectations or teacher-

designed learning targets or goals. However, the tool most used by teachers, Markbook, has no

ability to do so.

The use of learning goals and success criteria is prescribed in Growing Success, and

“heads the list of formative assessment strategies, no matter how that list is constructed.

(Brookhart, 2016, p. 81)

Teachers utilize a broad range of tools to gather student assessment data. Teachers’

choices of assessment tools demonstrate strong alignment with the conversations, observations

and products, as prescribed in Growing Success. They also consider carefully how they “match

(learning targets) to assessment methods” (Schimmer, 2012, p. 37), selecting from all three to

gather both formative and summative data.

Feedback to students, and feedback from students to inform their teaching, also involves

a wide range of tools. It is either evaluative or descriptive (Schimmer, 2012, p. 49) taking the

form of marks and comments in its evaluative form, but appearing in a variety of media when it


is descriptive. The use of electronic tools to facilitate the feedback process is seen by teachers as

a positive development.

Traditionally, products such as tests, projects, assignments and notebooks are used to

gather data on student achievement. And, “as products are constructed, teachers have

opportunities to observe students’ learning during the learning, not just at the end of learning”.

(Davies & Herbst, 2014, p. 18)

Feedback is a valuable formative tool, but difficult to implement with summative tasks,

especially those at the end of a course. “Even when the tests are well designed and related to the

curriculum, it doesn’t help practice if the results are not released until after the students have left

school”. (Reeves, 2016, p. 97) Descriptive feedback “that students can’t act upon is useless

feedback”. (Schimmer, 2012, p. 51) And “if students do not take action on the feedback, then

there will be no improvement”. (Drapeau, 2014, p. 165)

One means of communication of descriptive feedback is via rubrics, which are “a way of

organizing the descriptive feedback in a way that makes sense to everyone involved”. (O'Connor,

2012, p. 57) The form that these rubrics take, and their connection to data management tools,

present challenges to teachers. When these rubrics are criterion referenced, they can effectively

“be used by students themselves for peer assessment during the learning process and by the

teacher for diagnostic, formative and summative assessment purposes”. (Cooper D. , 2011, p.

127) However, they are often confused with evaluative rating scales (Brookhart, 2013, p. 21)

Electronic tools such as video conferencing (Skype, Google Hangout), audio commentary

(Voxer, Voice Thread, Kaizena) and written commentary (MS Word’s tracking function, Google

Doc comment function) may be added as tools to facilitate conferencing and feedback. (Kallick

& Zmuda, 2017, p. 105)


Use of a portfolio to organize student products and record feedback is a suggestion of

several teachers. A possible portfolio format might begin with a table of contents organized

around a set of expectations…organized by subject areas representing…standards.” (Niguidula,

2010, p. 155) This “collection of work would also provide valuable information to potential

employers, admissions officers, coaches, and mentors about a learner’s demonstrated

achievement”. (Zmuda, 2010, p. 106) Digital portfolios “create virtual time capsule for high

school students, illustrating growth over many school years” and they “say much more about

learning than a GPA ever could”. (Sackstein, 2015, p. 122)

Data management and analysis tools also vary greatly among the nine teachers. They

have, as a group, embraced the concept of professional judgement, and are utilizing more than

just the single mark calculating software tool provided by the school board. They are willing to

utilize professional judgement (Reeves, 2016, p. 79) but need more experience, tools, and

training to exercise their professional judgement with confidence.

Mathematical distortions resulting from the impact of choice of measures of central

tendency, and weighting of assessment items, concern teachers. “The use of a zero within a four-

point level scale is accurate; it demonstrates no work completed. However the use of zero, or any

number between zero and 50, is more problematic in a zero to 100 point scale.” (Reeves, 2016, p.

77) Teachers recognize the limitations of utilizing “average”, and are skeptical of the use of other

measures such as “blended mode”. Weighting of items, with most recent and most consistent

weighted more heavily (Reeves, 2016, p. 81) is desired by teachers, but difficult to confidently

implement using the existing mark calculation tools.

Where rubrics are the tool of choice, “teachers often make errors when calculating the

conversion of rubric scores to percent grades”. (Schimmer, 2012, p. 115) Strategies such as use


of anchor marks, or provision of “points” within each rubric category, are seen as problematic by

teachers.

This pressure to convert all assessment data to grades is associated with the student myth

that “I feel proud of myself only if a receive a good grade” and the observation that “students

only glance at the comments that accompany their grades, even though this explicit feedback

is… a powerful tool for student to use to improve their performance”. (Zmuda, 2010, p. 18) Marc

Prensky asserts that we need only three categories of assessment and evaluation: outstanding,

competent, and not yet competent. (Prensky, 2016, p. 127) This structure would be supportive of

good learning practices, but would not meet the requirements of Ontario Ministry of Education

policy.

Teachers appear to be struggling with the role that grades or marks play, and seem to

understand that grading is a “quasi-measurement, since grades do not possess the characteristics

of true measures.” (Yorke, 2011, p. 1) They describe many different paths they are taking to

determine a valid final mark or grade; on which reports the level of achievement of each student

in their course.

None of the teachers referred to board documents, to support their determination of a

final grade. The school board’s monograph: “Moving from Evidence of Learning to Determining

Meaningful Grades – 9-12”, provides detailed direction on how to work with levels, and then

determine a final grade. (Peel District School Board, 2015) Despite this, none of the subject

teachers mentioned either the resource, or the process.

In practice, however, all teachers demonstrate strong alignment with many of the

principles upon which this document is based. They all utilize practices that align with Ken

O’Connor’s five “Fixes” for inappropriate grade calculations and support learning:


1. Don’t rely only on the mean; consider other measures of central tendency and use professional judgment.

2. Don’t include zeros in grade determination when evidence is missing or as punishment; use alternatives, such as reassessing to determine real achievement, or use I for Incomplete, or Insufficient evidence.

3. Don’t use information from formative assessments and practice to determine grades; use only summative evidence.

4. Don’t summarize evidence accumulated over time when learning is developmental and will grow with time and repeated opportunities; in those instances, emphasize more recent achievement.

5. Don’t leave students out of the grading process. Involve students; they can – and should– play key roles in assessment and grading that promote achievement. (O'Connor, 2012, p. 13)

Effective practices of data management are required to ensure that students are confident

that their record of achievement is accurate. “Before learning occurs, students must feel

confident that they can learn what they need to learn. Part of this confidence comes from a past

record of success.” (Schimmer, 2012, p. 85) When this record is based upon faulty tools or

processes, learners and teachers cannot proceed with confidence. This confidence is missing if

students “don’t know what level of proficiency they’ve achieved because they haven’t been

provided with specific information about what they’ve done well and what needs continued

effort” (Sackstein, 2015, p. 99) Inclusion of tools that connect student data to curriculum

expectations is necessary, since “organizing assessment information by learning outcome enables

teachers to produce a profile of strengths and areas for improvement”. (Cooper & O’Connor,

2008, p. 24) The results of this have been seen when teachers have implemented a standard-

based report card, that includes marks for individual standards. Parents overwhelming prefer this

to the traditional single overall grade. (Swan, Guskey & Jung, 2014, p. 289)

The issues of validity and consistency remain for Ontario teachers; they do not have

effective tools to ensure either. “Grades need to meaningful, but they lose their meaning when

they are either inflated or deflated for reasons not connected to the intended learning.”


(Schimmer, 2012, p. 103) Principals currently have responsibility for this consistency and

validity, and are expected to “ensure that there is a common understanding among all staff about

the process for determining the final grade”. (Ontario Ministry of Education, 2010, p. 39)

Common understanding within one school might be possible, under the leadership of one

Principal, but it does not scale to the board or provincial level.

Conclusion and Future Study

Effective management of assessment data, determination of final grades, and alignment

to policy, all impact student success. “When schools embrace effective grading practices and

terminate toxic grading policies, student performance, motivation, and discipline improve”.

(Reeves, 2016, p. 2)

While teachers have many tools available to them, none of the tools currently in use are

sufficient to ensure that effective grading practices are followed.

Development of a tool that integrates a learning management system, student portfolio,

feedback communication, assessment recording, and mark calculation and reporting, and with

the ability to connect student achievement directly to curriculum expectations, is recommended.

With such a tool, teachers will be better able to plan, instruct, assess, and report on

student learning, and students will be empowered to play an active role, and take ownership and

responsibility for their own learning.


Bibliography

Black, P. & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 139-144.

Brookhart, S. (2013). How to create and use rubrics for formative assessment and

grading. Alexandria, VA, USA: Association for Supervision and Curriculum Development. Brookhart, S. (2016). How to make decisions with different kinds of student assessment

data. Alexandria, VA, USA: ASCD. Brookhart, S. M. (2004, March). Classroom assessment: Tensions and intersections in

theory and practice. Teachers College Record, 106(3), 429-458. Cooper, D. & O’Connor, K. (2008). Communicating student learning: Guidelines for

schools - In consultation with Manitoba Education, Citizenship and Youth. Winnipeg, MB, Canada: Government of Manitoba.

Cooper, D. (2011). Redefining fair: How to plan, assess and grade for excellence in

mixed-ability classrooms. Bloomington, IN, USA: Solution Tree Press. Davies, A. & Herbst, S. (2014). A fresh look at grading and reporting in high schools.

Courtenay, BC, Canada: Sandra Herbst, A. Davies Duncan Holdings Inc. Drapeau, P. (2014). Sparking student creativity: practical ways to promote innovative

thinking and problem solving. Alexandria , VA, USA: Association for Supervision and Curriculum Development.

Kallick, B., & Zmuda, A. (2017). Students at the centre: personalized learning with habits

of mind. Alexandria, VA: Association for Supervision and Curriculum Development.

Niguidula, D. (2010). Digital portfolios and curriculum maps: Linking teacher and student work. In H. H. Jacobs, Curriculum 21: Essential education for a changing world (pp. 153-167). Alexandria, VA, USA: Association for Supervision and Curriculum Development.

O'Connor, K. (2012). Fifteen fixes for broken grades: A repair kit. Toronto, ON, Canada: Pearson Canada, Inc.

Ontario Ministry of Education. (2010). Growing success: Assessment, evaluation, and

reporting in Ontario schools. Queen's Printer of Ontario. Peel District School Board. (2015, March). Moving from evidence of learning to

determining meaningful grades - 9-12. Peel's Growing Success Monograph Series. Mississauga, ON, Canada: Peel District School Board.


Prensky, M. (2016). Education to better their world: unleashing the power of 21st century kids. New York, NY: Teachers College Press.

Reeves, D. (2016). Fast grading: A guide to implementing best practices. Bloomington,

IN, USA: Solution Tree Press. Ross, J. R.-G. (2002). Influences on student cognitions about evaluation. Assessment in

Education, 9, 81-95. Sackstein, S. (2015). Hacking assessment. Cleveland, OH: Times 10 Publications. Schimmer, T. (2012). Ten things that matter from assessment to grading. Toronto, ON,

Canada: Pearson Canada Inc. Stiggins, R. (1999). Assessment, student confidence, and school success. Phi Delta

Kappan, 81, 191-198. Swan, G., Guskey, T, & Jung, L. (2014). Parent’s and teacher’s perceptions of standards-

based and traditional report cards. Educational Assessment, Evaluation and Accountability, 26, 289-299.

Wiggins, G., & McTighe, J. (1998). Understanding by design. Alexandria, Va, USA.

Association for Supervision and Curriculum Development. Yorke, M. (2011). Summative assessment: dealing with the ‘measurement fallacy’.

Studies in Higher Education, 36(3), 251-273. Zmuda, A. (2010). Breaking free from myths about teaching and learning: innovation as

an engine for student success. Alexandria, VA, USA: Association for Supervision and Curriculum Development.

Documents

Running head: CLASSROOM ASSESSMENT PRACTICES · instruction and assessment in response to student needs” (Ontario, 2010, p. 31) In this study, teachers focus on both “Assessment