dcronyn.files.wordpress.com · Web viewHow do student and teacher comments compare with one another in terms of quantity ... to learn and practice American written ... methodology

Douglas Cronyn Peer Review in the ESL classroom ENG 732 Kohls

Peer review in the second language classroom

Introduction

This inquiry project informed by research and which in turn informed

pedagogical best practices investigated the role of peer review for ELL. The

literature that I have read indicated several possible fruitful areas of investigation

and reflection for this project. In piloting this mixed method research project, and

through this process discovering the many complex issues around peer review, I

have pursued multiple lines of inquiry. Initially, this study investigated student

attitudes toward peer review, including whether or not geographical or cultural

identity influences these attitudes and perception. Additionally, teacher and student

(peer) comments were studied to determine to what extent they complimented or

were similar to one another..

Guiding questions for this mixed method research project are: What are

student attitudes toward peer review and does geographical or cultural identity

influence these attitudes? How do student and teacher comments compare with one

another in terms of quantity and substance? What are some of the affective concerns

in peer review? In the end, as an instructor in training, what are some of the

practices and perspectives that I will need in making peer review into a worthwhile

experience for students and teachers alike? This final question includes whether or

not students prefer written or oral feedback and whether they had any suggestions

to make their peer review experience better.

Douglas Cronyn Peer Review Literature Review English 732 Professor Kohls 1


In my capacity as instructor in training and novice researcher, various

methodological processes were utilized in this inquiry project. In illuminating these

processes, it is my hope through this transparency of process to burnish my

research skills still in its nascent stage, necessarily exposing some of the possible

groundless assumptions, generalizations, or approaches that needs further scrutiny.

Because of this desire for feedback on methodological practices, the methodology

section may be larger than is commonly done in an academic inquiry project of this

nature.

Context

The context for this project is a first year multilingual composition class at

San Francisco State University (SFSU). According to the class syllabus, the class

“provides multilingual students opportunities to learn and practice American written

rhetoric” through various readings and expository essay responses to these essays. There

were approximately 19 students in the classroom, coming from a diversity of various

Asian, Middle Eastern, European, Central American, North American, and African

countries. The class met two times per week for one hour and fifteen minutes. The

semester was 15 weeks in length. I attended all of the classes save a few. I took copious

notes during these classes, met with my mentor teacher frequently and for good amounts

of time, tutored many of the students, and conducted research about peer review. It

should be noted that a moderately significant amount of class time was devoted to

training for peer review, including a peer review guideline sheet that was distributed by

the teacher before each of the 3 peer review sessions. A copy of the peer review sheet that

was utilized for the peer review session that I scrutinized can be seen in Appendix A.



Peer review defined and theoretically contextualized

“Peer assessment is an arrangement in which individuals consider the

amount, level, value, worth, quality or success of the products or outcomes of

learning of peers of similar status’’ (Topping, 1998, p. 250). Theoretically, there are

four areas from which peer review draws its primary inspiration: “process writing,

collaborative learning, Vygotskian learning theory, and interactionist theories of L2

acquisition” (Hyland, 2006, p. 90) “A common theme in these theories is that

learning takes place, not in an isolated individual mind but among people…in a

society…or within communities of practice“(Lin, 2011, p. 90). Impacting these

theories are new conceptions of multimodal forms of communication that are

increasingly prevalent, such as wiki technology. But, remaining are the fundamental

aspect of students assessing one another’s work, regardless of culture or language.

Peer review in the ESL classroom has many complimentary and related areas

of study, including, in this author’s view, from most to least relatedness: peer review

in EFL classrooms, peer review of same language students in a multiplicity of genres

and disciplines, students tutoring one another in university writing centers

(frequently between 1st language and 2nd language students) (Harris, 2014), and

finally, the peer review process that professional researchers engage in with one

another in reviewing each other’s work, determining its academic value and such,

perhaps even including teachers evaluating or reviewing one another. Any system

which seeks to position workers or subjects as equals, or where feedback can be

seen as bi-directional, could theoretically be conceived as affecting or being affected

by the principles of peer review.



These related ideas and contexts could use further scrutiny, but not in this

inquiry project. Instead, I will note some of the relevant theories and considerations

in peer review, for the most part in ELL peer review, laying the groundwork for

further critique around the particular context that I studied. This study examines a

limited sample size in an educational research context known for multitudes of

variables to be considered. Due to these considerations, this inquiry project must be

viewed as a pilot research project from whence more detailed assumptions can be

drawn for future teaching and researching.

Cultural and practical considerations around peer review

An initial consideration in peer review is the perception by teachers and

students of whether peer review is in fact a viable part of the ESL or EFL classroom.

This debate involves both student and teacher perceptions about peer review and

its appropriateness. Can students fairly evaluate one another competently and

usefully? Is the training and time necessary to implement peer review worth it? Can

certain cultural generalizations and assumptions be made about student attitudes

toward peer review?

Student evaluations of the appropriateness of peer review begins with

fundamental questions of whether they see themselves as being qualified to

measure one another, evaluate one another, especially in a second language

classroom. In instances where students are unfamiliar with peer review, perhaps

coming from classes which are more teacher-centered or deferential to teachers,

some research indicates student resistance to evaluating and interacting with one

another (Ferris & Hedgcock, 2014). According to this schema, students from



collectivist or non-Western oriented cultures and classroom practices, characterized

by a teacher-centered classroom and student non-participation in the classroom, are

at a disadvantage when asked to participate in peer review type activities, as they

must abandon previously established norms of classroom communication (Hyland,

2006)

But, these ideas of Asian students in particular not liking peer review can be

challenged on several fronts, through various studies and alternative cultural

constructs. (Hu, 2010; Hyland, 2006; Miao, Badger, & Zhen, 2006). These

deconstructions perhaps begin with an assault on the very concept of culture,

especially when the “West” defines and essentializes the “East,” and thus transforms

it into something “thinglike, static, homogenous, coherent, and timeless” (Hyland,

2006, p. 49). Hyland (2006) warns us that “the cultural backgrounds that students

bring to group interactions are somewhat opaque” (p. 49) due to, amongst other

issues, the teacher usually not being a member of the sociocultural group being

critiqued, analyzed, constructed.

Affective considerations

Peer review has the potential to deepen classroom divides or bridge them.

Feedback in a general sense can be a significant contributor to teacher and student

motivation (Dorneyi & Csizer, 1998; Pennington, 1995). Interpersonal variables of

“psychological safety, trust, value diversity, and interdependence” may need to be

considered and addressed (van Gennip et al, 2010, p. 282) Steps may need to be

taken so that students have a shared understanding of the task at hand, not letting

personal relations affect positively nor negatively their honest appraisal. “Peers



have a tendency to assess on the basis of issues such as friendship and uniformity”

(Van Gennip, p. 282. 2010). As a teacher, it is important to devise lessons around

peer feedback that encourage fair and frank discussion, building a collaborative

classroom environment. This affective side of peer review could use more study,

especially considering that in a study of L1 peer reviewers, “40% indicated that peer

assessment was an uncomfortable experience” (VanGennip, 2010, p. 281). The

reasons for this need to be understood better and mitigated, perhaps especially so

in an L2 classroom. The question of whether or not students found peer review

difficult, awkward, or uncomfortable was addressed in the free write survey prompt

that I administered to them.

Methodology

For this inquiry project, various methodologies were employed, making this

a “mixed methods study (which) involves the collection of both quantitative and

qualitative study” (Dorneyi, 2007, p. 163). This mixed methods began with extensive

classroom observation as well as research on best practices. Professors were

consulted for suggested resources on peer review. This lead to the formulation of

the survey freewrite that I constructed and administered, which employed both

quantitative and qualitative features. This format was decided upon after consulting

with various professors who were skeptical of the results of solely a survey. With

the help of my mentor teacher, the original survey I made was transformed into a

combination survey and freewrite, a “peer review writing prompt,” which can be

seen in its entirety in the Appendix B.



There is some controversy around the legitimacy of this kind of mixed

methodology surveying. Most of the controversy surrounds the use of the “open-

ended” question in a survey. Says one researcher, “the desire to use open-ended

questions appears to be almost universal in novice researchers, but is usually

rapidly extinguished with experience” (Robson, 1993, p. 243) But, the particularities

of the freewrite survey that I administered to my class are held in higher esteem

when they “contain guidance” such as with “specific open questions (that) ask about

concrete pieces of information, such as facts about the respondent, past activities, or

preferences….short-answer questions are different from ‘essay questions in that they

are worded in such a focused way that the question can be answered succinctly”

(Dorneyi, 2007, p. 107). It is my belief that my questionnaire/survey/freewrite met

just such a criterion. The only question that provoked a longer response was

probably the last one, “in your opinion, how could this peer review process be

improved?” This final position is considered the best place for an open-ended

question (Dorneyi, 2007).

This survey was supplemented by my observing, taking notes, and mingling

during the peer review process. By doing this, I sought to confirm whether or not

students participated in a way similar to their survey responses, whether the

students were engaged or not. My primary concern while doing this and in students

knowing that I was studying peer review was in provoking “the Hawthorne effect,”

where “participants perform differently when they know they are being studied”

(Dorneyi, 2007, p. 53) I am still left to wonder if even the mere act of students

reflecting on their attitudes toward peer in answering my “peer review writing



prompt” could possibly have positively affected their attitudes. I was able to speak

with another student who also is studying peer review in my 732 English class at

SFSU to verify similar results and impressions as the ones that I had, somewhat

mitigating any particular bias.

Besides the interactions with my mentor teacher, observation, and surveying,

I also was granted access to all of the graded papers in the class in an easily

downloaded arena, the SFSU Moodle site, iLearn. Through this site I was able to gain

unfettered access to all of the comments that the teacher made to the students.

Unfortunately, I was not able to gain quite as much of an unfettered access to peer

review comments that students made and I only had a limited window of time in

which to electronically scan student peer reviews. Because of this limited time, I

decided to choose a representative sampling of papers from students of various

cultural backgrounds, genders, and achievement levels in the class. I chose 5

students for this representative sampling.

With these peer reviews, I compared, contrasted, and tabulated the student

comments on these papers with those that the teacher had made, categorizing them

accordingly: rapport building, meaning/content, elaboration, answered prompt,

identified the main idea, surface (grammar) correction, organization. I also noted

whether or not the students and teacher had placed any narrative comments at the

end of the peer review. This tabulation allowed me to both compare student and

teacher comments as well as see distinguishing characteristics of each.



Findings

Through the various class observations, survey, and peer comments, I gained

a grasp of some of the research questions framing this inquiry project. I will first

address the survey and then written comments students made on each other’s

papers.

Survey: In gauging whether or not students found peer review to be useful, I

asked Ss exactly this in the survey. All students responded affirmatively, that it was

useful, many saying it was “very useful” or even “outstanding.” Because these

affirmative responses crossed multiple geographical and cultural identities, it can be

inferred that these identities did not significantly impact their attitudes toward peer

review. Some of these responses included the following (responses were left in their

original state even if they were ungrammatical):

“feedback gives you chance to understand the work from other perspectives

(other than your own point of view)”

“when reading other people’s work makes me think on how I can be as

objective as I am with their work”

“It’s always great to receive a comment not matter it is a positive and

negative one.”

“I would not mind to give my opinion to the person if they’re confused by

their own writing, sometimes they’re too much in their mind and don’t know

how to organize and put the ideas in sentence.”

“Sometimes it’s difficult to make an appointment with a professor.”

“I’m not always sure if my thesis is clear or my explanation is well written.”



“It has helped me get over anxiety of whether I’m matching the standards

that are expected of me.”

“the feedback I receive from peers could not be as useful as the feedback

from teachers I can get during office hour…the problem is that all my

classmates are foreigners, so it’s not infrequent that we have a difficult time

to understand essays we write and give proper feedback for it.”

Another question which I sought to answer was whether students preferred

to receive written or oral feedback. Of the students who answered this question,

slightly more preferred to receive written to oral feedback. The students preferring

written feedback liked that they could refer to it later and was easier to understand.

Alternatively, those who preferred oral feedback did so because they could ask their

reviewers to clarify any questions that they may have. They also felt that oral

feedback was more clear.

A third research question had to do with how students affectively related to

peer review. The survey addressed this question by asking if it was uncomfortable

for them to review or be reviewed and why this might be so. There was some range

in these responses, with varying levels of discomfort. Some of these reasons are as

follows:

“It is not hard for me to review others work but I sometimes feel

uncomfortable when others read mine. Because I don’t want everyone to

know what I wrote.”



“I do not want to say straight in my peer’s face that they have these

mistakes.”

“At first, I was a little bit shy when a person who is not my teacher read my

papers. However, later, I found it was really helpful to me…”

“At first time, I felt little bit embarrassed because my poor writing skills

revealed to them. But when I became who examine the other peer’s writing I

realized that poor writing skills doesn’t matter…So from now on, I don’t feel

awkward anymore.”

“It was my first experience to read somebody else work…hence I was little bit

hesitant to write what I really felt but then when I thought that my work is

also going to be reviewed and I will like a honest feedback on it so that I can

improve myself, hence the other person should also get a honest feedback to

improve his work.”

“other students are too worried about ‘feelings,” and they try to be as much

polite as possible (GREAT) but they end up not being totally objective and

not helping the other student.”

“I admit it could be uncomfortable if got a peer review by someone I don’t

trust. They might only do it because it is in class assignment, which means

they might don’t care why I write.”

Because of these disparate replies to the survey freewrite, it is difficult to

generalize any findings on the affective response that students have to peer review,



though it does seem from the responses that some students may have had initial

misgiving which subsided over time. Additionally, some may have doubted the

authority of their peers, but seemingly not their good intent. It would be interesting

to note how these responses would compare to students engaging in computer-

mediated anonymous peer review.

Finally, in gleaning out other useful information from this survey and also

addressing the idea of peer review groups size, some students did respond that

they felt that the size of the group should be larger and that they should have longer

to review one another’s work. Of the 14 students who took part in the survey

writing prompt, 5 addressed the question of whether the process could be

improved. One respondent “didn’t think there was much to change or to improve.”

Three students suggested that at least two people should read your paper. One

student felt that there should be more time to respond to each other’s work.

Written comments on papers: For this section I compared the number of

student(peer) and teacher comments for an essay that the students wrote about an

article by Abbie Hoffman. These comments were tabulated in Appendix C. As

previously elaborated upon in the methodology section above, the categories for the

comments are: rapport build, meaning/content, elaboration, answers prompt,

identification of main idea, surface error, and organization. Additionally, I added up

how many questions were included in the comments, though the content of these

questions was tabulated with other comment types. I also noted for each of the five

papers whether or not the reviewer had made any narrative type comments at the

end of the expository essay, “end comments.”



Although, as discussed previously in the methods sections, there are various

methods for coding student and teacher responses, I have chosen this system after

some consulting with my esteemed TESOL writing/reading professor. I slightly

altered the classification due to my deciding to quantify teacher comments

according to a later draft of the essay rather than an initial draft that was a journal

type entry. Explanations for these codes, where they are not obvious, are as follows:

Rapport build: this refers to any conversational type of comment made, such

as “Interesting!!! Do you think that society has instilled fear in young people

today so they are passive citizens?”

Meaning/content: this section had the most number of comments, but

involved a broad number of possible comments, including: “clearer thesis

statement needed…don’t just summarize…unclear, and..what is the purpose

of this paragraph?”

Elaboration: this included over and under elaboration as well as the positive

comment, “good example!” The teacher gave 5 “good examples! and the

students gave 16. As noted by the peer review sheet in Appendix A, students

are supposed to make this notation where inspired to do so.

Answer prompt: students needed to answer prompt

Identify main idea: students were supposed to underline the main point of

each paragraph.

Surface: grammatical error. Students were not asked to address this.



Organization: this included comments such as “connect theses to points…this

should go elsewhere, and…begin paragraph with a clear reason/point.” This

also included where students noted an overview or summary.

Conclusion, discussion, and pedagogical implications

As previously elaborated upon, this study needs to primarily be seen as a

pilot project to lay the foundation for possible future studies, including various post-

peer review interviews. These interviews, along with the peer reviews, could shed

some important light on such issues as optimal sizes of peer review groups, good

amounts of time for students to review one another’s work (including perhaps some

days), and how to select peers to review one another’s work. Additionally, more

research comparing computer-mediated and in person peer review would be

interesting and useful, especially in identifying such affective issues as whether

anonymity negatively or positively affects the peer review process and also include

such issues as how increasing teacher oversight over the peer review process could

be beneficial or harmful to the process. It would also be interesting to survey how

students work best according to personality types or gender identification. In

classrooms with both L1 and L2 learners, it would be interesting to see how

different groupings of first language students and ELL compare.

Besides these implications for future studies which could come about as a

result of this pilot, there are other theoretical and pedagogical implications of this

project. The student reflections and feedback in the previous section speak for

themselves for the most part, not needing further elaboration. They have said



clearly enough why they found peer review to be a useful activity. Their affective

responses have also been noted. In the theoretical realm, this study indicated that

cultural identity does not have a strong influence on peer reviewer perceptions of

the usefulness of peer review. All students studied found the peer review useful

regardless of their geographical and cultural identity.

There are numerous pedagogical implications of this study, some of them

coming from the research and others from the observation and survey conducted.

These would include possible changes in peer review groupings, including having at

least two people review everybody’s paper, as was suggested in the research as well

as on some student survey responses as possible ways to improve the experience.

Additionally, based on the research and partially by the survey, it may also be

worthwhile to have students review one another’s work outside of class as well,

giving them more time to adequately review their partner’s paper.

Further study would need to be conducted in order to determine whether

students gave correct feedback and whether or not it was taken up by the peer

being reviewed. Additionally, as a researcher, it would be interesting to code teacher

and student comments according to such criteria as those utilized by Rysdam and

Johnson (2014) of: “corrections, praise, labels of failure, non sequitur comments,

guidance for improvement, and comments that provided an authentic

conversational response to the content” (p. 79). It would be interesting to see how

these comments fluctuated from class to class according to various peer review

systems. Some sort of replication of the ELI Review computer-mediated peer review

system (seen in Appendix D) would also be interesting to instigate, making peer



review a more frequent part of paper development, including suggestions by Reid

(2014) to use peer review for smaller issues such as choosing topics, design, or such,

as well as possibly expanding the process to make peer review into a major

assignment that includes revisions and reflections over an extended period of peer

review. There are many possible avenues for interesting study and practice are to be

had in understanding peer review better and personalizing it to my own

pedagogical and theoretical means. I look forward to engaging this in the years to

come.



References

Dornyei, Z. (2007). Research methods in applied linguistics: Quantitative, qualitative,

and mixed methodologies. Oxford: Oxford University Press.

Dornyei, Z. & Csizer, K. (1998). Ten commandments for motivating language

learners: results of an empirical study. Language Teaching Research, 2,3, (pp.

203-209).

Ferris, D. & Hedgcock, J. (2014). Teaching L2 composition. (pp. 237-279) New York:

Routledge.

Guichon, N. (2009). Training future language teachers to develop online tutors’

competence through reflective analysis. ReCALL, 21 (2), 166-185

Harris, M. (2014). A non-coda: Including writing centered student perspectives for

peer review. Peer pressure, peer power: Theory and practice in peer review

and response for the writing classroom. (pp. 277-288) Southlake, TX:

Fountainhead Press.

Hyland, K. & Hyland F. (2006). Feedback on second language students’ writing.

Language Teach, 39, 83-101.

Knott, J. (2014). Student Perciptions of Eli Review at MSU: 2012-2014. Michigan State

University.

Lin, W. & Yang, S. (2011). Exploring students’ perceptions of integrating Wiki

technology and peer feedback into English writing courses. English Teaching:

Practice and Critique, 10 (2), 88-103.

Liu, J. (2002). Peer response in second language writing classrooms. Ann Arbor:

University of Michigan Press.

Miao, Y., Badger, R., & Zhen, Y. (2006) A comparative study of peer feedback in a



Chinese EFL writing class. Journal of Second Language Writing, 15, 179-200.

Pennington, M.C. (1995). Work Satisfaction, Motivation and Commitment in Teaching

English as a Second Language. ERIC Document ED 404850.

Rahimi, M. (2013). Is training student reviewers worth its while? A study of how

training influences the quality of students feedback and writing. Language

Teaching Research, 17 (1), 67-89.

Reid, S. (2014). Peer review for peer review’s sake: Resituating peer review

pedagogy. Peer pressure, peer power: Theory and practice in peer review and

response for the writing classroom. (pp. 277-288) Southlake, TX:

Fountainhead Press.

Robson, C. (I993). Real World Research: A Resource for Social and Practitioner-

Researchers. Oxford: Blackwell.

Saito, H. (2008). EFL classroom peer assessment: Training effects on rating and

commenting. Language Testing, 25 (4), 553-581.

Topping, K. (1998). Peer assessment between students in colleges and universities.

Review of Educational Research, 68, 249-276.

Van Gennip, N., Segers, M., & Tillema, H. (2010). Peer assessment as a collaborative

learning activity. The role of interpersonal variables and conceptions.

Learning and Instruction, 20, 280-290.



Appendix A“Peer review form”



Appendix B

Peer Review Writing Prompt

Writing professor Bill Hart-Davidson from Michigan State University said, “The most effective way that people learn as writers is from other writers, from their peers, from someone who is in a situation like they’re in, trying to do what they’re trying to do.” Do you agree with this statement? Think about your own experiences, and take 10 minutes to write a response. You may choose to respond to any or all of the following in your response.

Is peer review a useful activity for you? Do you use the feedback you receive from your peers?

Have you done peer review previously (outside of your peer reading experiences in English 209) and was it useful to you?

Is it difficult for you to review others work and have them review yours? Is this awkward or uncomfortable? If so, why do you think so?

Do you prefer written or oral feedback?

In your opinion, how could this peer review process be improved?

Optional: Please circle your geographical/cultural identity (or identities):

African Latino European Asian North American Other (please specify)_______________



Appendix C

“

“Teacher and student comments compared”


Comment types Number of T comments Number of S comments

Rapport build (rb) 1 0

Meaning/content (m/c) 14 5

Elaboration (e) 10 33

Answer prompt (ap) 4 0

Identify main idea (mi) 1 7

Surface 0 3

organization 5 0

Total comments 35 50Questions 6 5End comments 5 2


Appendix D

Eli Review write/revise/review cycle

“Traditional write/review/revise cycle”

Eli Review’s write/review/revise cycle

From: http://elireview.com/content/td/feedback/


http://elireview.com/content/td/feedback/

Documents

dcronyn.files.wordpress.com · Web viewHow do student and teacher comments compare with one another in terms of quantity ... to learn and practice American written ... methodology