The Machine Scoring of Essays: Redefining Writing Pedagogy?

Deborah CrusanWright State University

My talk this morning will

focus on one of the programs typically used for classroom writing/writing assessment - MY Access!

define MY Access!

discuss a small study in which a few ELLs used MY Access!

report their attitudes toward MY Access!

wonder about the impact of MY Access! and similar programs on second language writers and writing.

But first, we must remember ~

"For there is nothing either good or bad, thinking makes it so."

~ William Shakespeare (1564-1616), Hamlet, II.ii

My translation: We are (well at least I am) very good at creating dichotomies of good and bad

So just for now, let’s suspend our biases . . .

So . . . What is MY Access?

MY Access! is a “web-based instructional writing product that provides students enrolled in grade 4 through higher education with the opportunity to develop their writing skills within an electronic portfolio-based environment” (Vantage Learning, 2007, p. 1).

MY Access! grades students’ responses to select writing prompts and offers suggestions for improvement to the text in a matter of seconds.

MY Access claims

that their program decreases writing instructors’ grading burden

that the almost instantaneous feedback provided by MY Access! motivates students to revise more.

Problems Warschauer & Grimes (2008) found that

students rarely use programs like MY Access! to revise anything other than surface errors

Teachers can create their own prompts; however, MY Access! will score only those prompts included in the program

In many cases, longer essays receive higher scores; consequently, MY Access! assumes that length is a proxy for fluency.

The study Graduate students worked with ELLs, asking them to

respond three times to two different prompts

Students responded to a survey about their experiences with the program

Additionally, grads interviewed ELLS regarding their attitudes toward MY Access!

For the purposes of this presentation, I’ll highlight comments from six of the students who agreed to be interviewed

Overall, students’ opinions about MY Access! were mixed; students found useful aspects as well as aspects they termed less helpful.

Student attitudes - Hannah

Hannah reported that she was excited about the program, but because she had little computer experience, she had difficulty learning the program. She had problems remembering what to do each time she logged on because she didn’t use the program very often. She found that the MY Access! prompts were not very interesting.

Xiao Xiao admitted that it was very easy

to find and insert information from other sources into his writing – even if it wasn’t his ideas. He found MY Access! feedback useful for making surface corrections – the program pointed them out, so he could correct them – but he wasn’t motivated to revise his work beyond fixing the surface errors.

Heecheon Heecheon believed that he could

get the same kind of information from Microsoft Word. Initially, he liked the instant feedback My Access! provided, but he began to feel overwhelmed as he read through the massive amount of information offered to him by the program.

Mohamed

Mohamed believed that MY Access! would be beneficial in the classroom. He found it to be a great tool to use during the writing process because it gave him a variety of tools and activities to aide in the revision of his writing.

Farin

After writing and submitting his first draft, Farin thought it was scored too low, so he went to the Internet and found some material loosely related to his topic. When he inserted the material into his essay, his score went up even though the essay was not as coherent as it had been when he first wrote it.

Wafa

Wafa felt that MY Access! helped her get into the habit of doing multiple drafts. She liked the editing tools and found some of the feedback very helpful in the revision of her essays.

Consensus Students enjoyed writing with a computer

They liked the immediate feedback provided

However, they often found the feedback overwhelming

Students appreciated help finding grammar errors

Many became bored

Many students did only surface editing, not revision

Several students admitted adding text from the Internet

Students did not use features in MY Access.

Should you use MY Access! in your second

language writing classroom?

It depends!

Teachers need to ask themselves questions about these programs and how their use might impact their pedagogy

However, this technology is here to stay, so . . .

Teachers must understand this technology and strive to keep assessments locally controlled, context-specific, and accessible