30
Susan Bleyle Semire Dikli Georgia Gwinnett College SETESOL 2009

The online writing evaluation system panacea: myth or reality?

Embed Size (px)

DESCRIPTION

The online writing evaluation system panacea: myth or reality?

Citation preview

Page 1: The online writing evaluation system panacea: myth or reality?

Susan BleyleSemire Dikli

Georgia Gwinnett College

SETESOL 2009

Page 2: The online writing evaluation system panacea: myth or reality?

Main purpose What is OWE?

Advantages Disadvantages

Problem statementProcess approach requires a great amount

of time and effort from the teachers.Feedback on writing has been found to be

highly individual and/or situation specific.Many OWE systems were originally

designed for native English speakers, but they were developed and marketed for non-native speakers, as well.

Page 3: The online writing evaluation system panacea: myth or reality?

What are the student perceptions of using an OWE system in a college ESL classroom?

Page 4: The online writing evaluation system panacea: myth or reality?

15 advanced ESL students in a writing course for matriculated studentsprimarily U.S. high school graduates from various linguistic backgrounds

Page 5: The online writing evaluation system panacea: myth or reality?

Student drafts Survey

Page 6: The online writing evaluation system panacea: myth or reality?

Over the course of a semester, students wrote a minimum of two drafts on six prompts3 were written using an OWE system. 3 were written in-class using pen and paper

and revised at home using Microsoft Word. • At the end of the semester, students

completed a survey to ascertain their perceptions of using OWE as compared to the more traditional essay formats

Page 7: The online writing evaluation system panacea: myth or reality?

Survey results are tallied:Planning WritingFeedbackScoringRevising

Page 8: The online writing evaluation system panacea: myth or reality?

Do you plan your essays (e.g. brainstorm, map, outline) before writing?

1. Never 02. Rarely 2 (14.3%)3. Usually 5 (35.7%)4. Always 7 (50%)

Page 9: The online writing evaluation system panacea: myth or reality?

Which do you generally prefer for planning your essays?

1. Criterion 2 (14.3%)2. Paper and pencil/pen 11 (78.6%)3. Other 2 (14.3%)

Page 10: The online writing evaluation system panacea: myth or reality?

Negative comments:“I prefer planning my essay on paper because I can see my ideas in an expanded way.”

“I like to use paper and pen for planning my essay. I feel comfortable writing on the paper better than on computer.”

Neutral comments: “I have never use the “Make a Plan” feature, so I do not know what I don’t like about it.”

Page 11: The online writing evaluation system panacea: myth or reality?

Which do you generally prefer for writing your essays?

1. Criterion 5 (35.7%)2. Paper and pencil/pen 7 (50%)3. Other 4 (28.6%)

Page 12: The online writing evaluation system panacea: myth or reality?

Postive comments:“I like Criterion better because it has spell check and sometimes I'm not sure how to spell words. Also, I like Criterion because I'm saving paper.”

“I prefer both Criterion and on paper because I like to brainstorm on paper before writing it on Criterion.”

Page 13: The online writing evaluation system panacea: myth or reality?

Negative comments:“I like to use paper and pen writing essay better than use Criterion. Because I turn on the computer that make some noise, I cannot concentrate to plan my essay.”

Other:“I like use Microsoft word because it can correct some mistakes and spelling. Furthermore, it also have many kind to choose to make my essays more interesting such as color, frame, art...”

Page 14: The online writing evaluation system panacea: myth or reality?

How helpful do you find the Instructor Pop-Up Notes on Criterion?

1. Not at all helpful 1(7.1%)

2. Not very helpful 1 (7.1%)3. Somewhat helpful 7

(50%)4. Very helpful 6

(42%)

Page 15: The online writing evaluation system panacea: myth or reality?

How helpful do you find the Instructor General Comments on Criterion?

1. Not at all helpful 02. Not very helpful 1 (7.1%)3. Somewhat helpful 5 (35.7%)4. Very helpful 8 (57.1%)

Page 16: The online writing evaluation system panacea: myth or reality?

How helpful do you find the Criterion Trait Feedback (feedback on grammar, mechanics, usage, style, etc.)?

1. Not at all helpful 2 (14.3%)2. Not very helpful 2 (14.3%)3. Somewhat helpful 7 (50%)4. Very helpful 4 (21.4%)

Page 17: The online writing evaluation system panacea: myth or reality?

How do you prefer getting feedback on your essays?

1. On Criterion 5 (38.5%)2. On paper 8 (61.5%)3. Other 2 (15.4%)

Page 18: The online writing evaluation system panacea: myth or reality?

Positive comments:

“i like the criterion feedback and the instructor feedback because both helped alot in rewriting my last draft”

“Because it is easier for me to see my mistakes on Criterion.”

Page 19: The online writing evaluation system panacea: myth or reality?

Negative comments:

“Criterion is not 100% correct. Sometimes, it would give false feedback.”

“not helpfull because it does not read the run on or other misstakes that only a profesor can faind.”

“Either one would work, but on Criterion its not as clear as on paper.”

Page 20: The online writing evaluation system panacea: myth or reality?

How helpful do you find the Criterion Essay Score of 1-6?

1. Not at all helpful 2(14.3%)

2. Not very helpful 3 (21.4%)3. Somewhat helpful 9 (64.3%)4. Very helpful 0

Page 21: The online writing evaluation system panacea: myth or reality?

Comments:

“I like that it helps us become better writers. What I don't like is that sometimes it'll rate the essay good but deep down inside I know it's not the best that i can produce. Also, it pinpoints any error and sometimes there is no error to be fixed.”

“I don't like the way it work with the scoring... But i like that it gives a lor of feed back.”

Page 22: The online writing evaluation system panacea: myth or reality?

Which way do you prefer to revise your essays?

1. On Criterion 4 (33.3%)2. On Paper 5 (41.7%)3. Other 4 (33.3%)

Page 23: The online writing evaluation system panacea: myth or reality?

Positive Comments:

“I like it [Criterion] because I don't have to rewrite my entire essay, I can just correct it.”

“I think it is easier to revise my essay on Criterion because I can see my teacher's grammar comments clearly.”

Page 24: The online writing evaluation system panacea: myth or reality?

Negative Comments:

“I like to use words to revise my essay. Because words format can save a copy and easy to correct or edit. However, Criterion format is not many function, for example, it can not enlarge the character.”

“I like doing it on paper or on the microsoft word make ir easier to read and understand.”

Page 25: The online writing evaluation system panacea: myth or reality?

1. That Criterion be used in the same way in future semesters

2. That Criterion be used in future semesters with some changes (please explain below)

3. That Criterion be made optional in future semesters

4. That Criterion not be used at all in future semesters

5 (45.5%)

3 (27.3%)

3 (27.3%)

1 (9.1%)

Which of the following is your overall recommendation for Criterion:

Page 26: The online writing evaluation system panacea: myth or reality?

Negative comments:

“I think we don't need use criterion. It's nothelp somebody to improve their essays.”

“many student can fail because of thatsoftware”

Page 27: The online writing evaluation system panacea: myth or reality?

Positive comments:

“the criterion is really a good resource, so it has to be use in future too.”

“I think Criterion is a good way to practice writing essay. Sometimes, Criterion feedback is not 100% true, but I can get a hint of how I did on my essays.”

Page 28: The online writing evaluation system panacea: myth or reality?

Suggestions for change:

“Let the students decide if they would like to submit their thruough criterion or paper”

“I think the Criterion format should be changed at least as same as Microsoft word's function. Because Criterion program function is limited. For example, essay cannot post double line in row, enlarge character,underline ,and add different color.”

Page 29: The online writing evaluation system panacea: myth or reality?

For schoolsBudget Classroom resources

For teachersAs a support tool Inform students of limitations

For developersMore user friendlyMore attention to ESL writers