Peer Reviewer Training Workshop Shea Wang & Trudi Ohki CAF
Sept. 22, 2014
Slide 2
Workshop Overview Upon completion of this workshop, you will be
able to: Conduct a pre-observation meeting Review teaching
materials Use the narrative, checklist, rating, and data collection
forms for classroom observation Conduct a post-observation debrief
Write a final post-observation report
Slide 3
Activities Overview Practise using the pre-observation meeting
form Practise using the review of teaching materials form Practise
classroom observation skills and techniques Practise using the
narrative, checklist, rating, and data collection forms for
classroom observation Practise post-observation debrief Practice
post-observation report writing
Evaluation at MacEwan Policy C5065 guided by the philosophy
that we: promote a culture of teaching and learning foster
professional development and scholarly activity promote fairness
and transparency timely and accurate feedback comply with
collective agreements undertake as a process involving multiple
stakeholders, and a variety of assessment approaches
Slide 6
Faculty strive for excellence in teaching, which is assessed
by: teaching (including curriculum development) research,
scholarly, or artistic engagement service (collegiate citizenship)
collegiality (professionalism, respect for students & peers,
and ethical behaviour) professional development Evaluation at
MacEwan
Slide 7
C5065 Instructional Faculty Requirements based on appointment
type ContinuingProbationary Sessional/Sessional- extended Teaching
Dossier Peer Review Sessional-extended at renewal (Collective
Agreement 2011-14) Annual Report Student Online Feedback
Performance Review
Slide 8
C5065 Probationary Requirements Teaching Dossier - Annually
Peer Review - Two reviews during each of the first two years (4
total) Annual Report - Annually Student Online Feedback - Every
course during appointment Performance Review - Annually
Slide 9
Introduction What is Peer Review? Scholarly publications review
process to assess quality of the work Evaluation of
classroom/online teaching and teaching materials ASAC position
review committees Any evaluation process where the evaluators are
peers
Slide 10
Peer Review Literature Peer review is appropriate for Course
content, expertise in field of study Course objectives and
materials Appropriate teaching methodology Course organization
Appropriate tests, assignments, evaluation methods Seldin,
1999
Slide 11
Peer Review at MacEwan Teaching effectiveness Classroom visits
Teaching materials ASAC position review committees
Slide 12
Qualifications of Peer Reviewers Criteria developed by FEC
Completion of an appropriate peer review training course and / or
demonstration of peer review experience Minimum 3 years full-time
teaching experience Pedagogical expertise Possess a continuing
appointment Recommendation by Dean / Associate Dean / Director /
senior administrator
Slide 13
Attributes of Peer Reviewers Utilizes effective instructional
practices (demonstrated through dossiers, annual reports, awards,
etc) Demonstrates collegiality Provides constructive, objective
feedback focusing on teaching behaviours and improvement Models
ethical behaviour Appreciates different styles / modes of
teaching
Slide 14
Exceptions For all peer reviewers, should there be a deficit in
meeting the recommended qualifications, the Dean, Associate Dean,
Director, or senior administrator will assess and provide final
approval.
Slide 15
Choosing Peer Reviewers Chairs will support and advise faculty
members in the selection of two peer reviewers to conduct classroom
observations and review of relevant teaching materials. It is not
necessary that peer reviewers be content- area experts. Peer
reviewers can expect to be asked to perform peer reviews outside of
their home department(s).
Slide 16
Defining Effective Teaching Conversations about teaching
(30min) Pair and Share Break into small groups and discuss the
following sentences: I most enjoyed teaching when __________ I knew
I had a problem when __________ Have a member of your group report
on your discussion to the main group *Chickering and Gamson,
1987
Slide 17
Measuring Good Teaching Myths Dr. Fox Expressiveness High
Ratings Reflect Lenient Grading Workload/course difficulty Rank and
experience Gender
https://facultycommons.macewan.ca/headlines/professors-
from-sfu-share-mean-student-evaluations
Slide 18
Measuring Good Teaching
Slide 19
Measuring in context Take 15 min. and complete the following
sentences in your group: I knew my teacher was a good teacher
when/because I knew my teacher was a poor teacher when/because
Slide 20
Literature on Good Teaching Eight dimensions of college
instruction : Course organization & planning Faculty
communication Faculty/student interaction Assignments, exams, &
grading Instructional methods & materials Course outcomes
Student effort & involvement Course difficulty, workload, &
pace (Marsh, 1984 & 1987)
Slide 21
Literature on Good Teaching (2) Encourages student interaction
Encourages student cooperation Active learning Prompt feedback to
students Emphasizes time on task High expectations Respect
(Chickering and Gamson, 1987)
Slide 22
Literature on Good Teaching (3) Good teachers Are good
learners; Display enthusiasm; Know how to modify their teaching
strategies; Encourage learning for understanding; Demonstrate an
ability to transform and extend knowledge, rather than just
transmit it; Set clear goals; Provide high-quality feedback Show
respect for their students (Ramsden, 2003)
Slide 23
Overall Good Teaching Parker Palmer capricious chemistry Good
teaching is ultimately irreducible to a simple technical formula
Characteristics are beyond common descriptors or attributes of a
checklist Solution: multifaceted approach Student feedback, peer
review, self-reflection, and administrative evaluation
Slide 24
Peer Review Activity In your small group, reflect on your past
experiences with giving and receiving peer feedback. Address the
questions: What did you learn from the experience? In what ways it
was a positive/negative experience? Did it influence your
instruction? How did receiving and giving feedback change the way
you thought about teaching and learning?
Slide 25
Peer Observation The two forms of observation: Summative
observation - This observation is an evaluation. It can be done for
administrative or supervisory reports for the purpose of job
retention, promotion, salary increases, etc. Formative observation
- The purpose of a formative peer observation is to help
instructors become better teachers and more knowledgeable
professionals.
Slide 26
Peer Review Process at MacEwan Evaluation Policy outlines two
main areas of teaching assessed by peer reviewers: 1.Teaching
materials - lecture notes, course outlines, tests and exams,
assignments, grading rubrics, etc.; and 2.Instructional delivery -
observation of teaching in formal lectures, as well as other
delivery modes such as labs, studio, clinical, and online/distance
formats
Slide 27
Peer Review Process at MacEwan Five main elements:
Pre-observation meeting Review of teaching materials Classroom
observation Post-observation debrief Post-observation report
Slide 28
Forms Some of the forms available Narrative form Checklist form
Rating form Data Collection (see appendices of workbook)
Slide 29
Pre-Observation Meeting Emotionally, it is important to address
distrust of evaluation, violation of personal style and space,
insecurities about time, and fear of bias (Chism, 2007) Essential
that trust be established - Acknowledge the emotional aspect. Clear
objectives of observation should be outlined - Communicate
expectations Stressed that the process is collaborative - Engage in
active listening skills (Review pre-observation forms in
appendices)
Slide 30
Course Materials Some examples of materials include: Syllabus
Course guides Course packets and assigned texts PowerPoint
Presentations, overheads, BlackBoard, etc Handouts Videos and other
multimedia supplements Project assignment directions and
handouts
Slide 31
Procedure for Evaluation of Course Materials Importantly, we
are not focusing on student achievement measures when reviewing
teaching materials. Rather, we are looking at content, design,
clarity, etc Materials enable reviewers to see an instructor's
philosophy enacted, and can alert reviewers to the areas in which a
given instructor excels Materials can promote discussion about the
overall goals (Review course materials checklist in
appendices)
Slide 32
Classroom Observation Observers should use the same observation
form they should take separate minutes/notes At least two observers
should attend each class observation session At least two visits is
preferable No unannounced visits Discuss with the instructor the
format of the review prior to visit: Where will you sit? Will you
be introduced?
Slide 33
Classroom Observation During the Classroom Visit Be unobtrusive
(basically, remain quiet) Do not ask questions Do not volunteer
answers Do not correct the instructor Be attentive Note specific
examples of observed behaviours (positive & negative)
Slide 34
Classroom Observation Challenges One class may be not
representative Atypical performance (good or bad) Observer may not
have a good understanding of the context Personal biases on what
constitutes good teaching
Slide 35
Some Solutions Attend 2+ classes Video tape class? Focus-group
students after class? Use the data collection form, cite examples
Discuss visit with co-reviewer, debrief together Contextualize the
course reflect back on: Pre-observation meeting Examination of
course materials
Slide 36
Peer Observation Exercise Pre-observation In your small group,
decide on the form you will be using for review (narrative,
checklist, rating) Review form and highlight areas that each
reviewer will be responsible for focusing on and taking notes about
Observation Choose one of the two scenarios to review Reflect on
the class from the following viewpoints: Viewpoint 1: students
Viewpoint 2: instructor
Slide 37
Peer Observation Exercise Compare your list of observer
behaviors with others in your group Be reflective and systematic
through the criteria outlined in your observation form Embed your
observations & conclusions in examples of observed behaviours
Did the instructors behavior appear to be what you might expect in
a formal observation?
Slide 38
Peer Observation Exercise Role Play In your small group,
designate an individual to be the instructor, the reviewer, and a
reporter Break into large groups based on role as instructors &
reviewers & reporters Compare the notes your groups took during
the observation and discuss strategies for approaching the debrief
(and how the characters might react) Break back into your small
group and role play the post-observation debrief meeting
Slide 39
Post Observation Debrief Meet with the instructors immediately
following the classroom visit Ask the instructor to describe how
they felt the class went in relation to the goals, objectives and
expectations outlined in the pre-observation meeting At this time,
the peer reviewers should bring up any immediate concerns that were
highlighted by the observation session. **This is an important
aspect of the review as it can potentially change the observers
perceptions about what they saw and heard.
Slide 40
Post Observation Debrief Procedure Ask the person who taught
how s/he felt the lesson went Reconfirm the observation focus
points for the final report Offer positive comments on what you
think went well Identify any urgent issues Ask questions for
clarification, as needed End on a courteous note Provide an
expected timeline for the final report and final meeting
Slide 41
Post-Observation Exercise Prepare yourself to care watch that
you are not overly forgiving or overly critical (put yourself in
their shoes) Use descriptive language rather than evaluative Be
responsive to the goals outlined in the pre- observation meeting
Check to make sure the instructor has heard your message as you
intended Be forward-looking and positive in your discussion, i.e.
what can be improved over what went wrong
Slide 42
Peer Review Final Report Policy C5065 states Section 4.4.4,
Peer reviewers shall prepare brief written assessments, grounded in
evidence, and submit a copy of reports to the faculty members and
Chairs.
Slide 43
Who Sees the Final Report? Faculty member used to improve
teaching effectiveness, make adjustments as necessary for
application for continuing status Chair and Dean used for
administrative review ASACs used for personnel decisions
Slide 44
Characteristics of a Good Report Reference the goals
established during the pre- observation meeting Provide a written
record of what was observed and what was discussed Consider the
audience Use examples from the observation Language is clear,
concise, avoids jargon and abbreviations Tone is supportive and
phrasing is positive Action-oriented and provides specific and
practical suggestions for improvement
Slide 45
Characteristics (contd) One, joint report per instructor -
Structure of report should be agreed upon by all reviewers Return
report in a timely manner Review with faculty member within 3 days
of classroom visit Consider faculty member response before
submitting for administrative review Confidential (Review report
template in appendices)
Slide 46
Peer Review for Online Courses Online Peer Review Form (4
categories) Student behaviour Faculty-student interaction (multiple
interactive communication strategies) Technology support Learning
environment
Slide 47
Challenges - Peer Review for Online Courses A connection
between specific materials and instruction and student action that
manifest learning objectives. Technology and accessibility Are
instructors responsible for these? Yes, no, maybe, more
nowadays?
Slide 48
What Is A Good Online Course? Content? Easy use of technology?
Interaction? Learner-centered outcomes-based instruction: Clear
objectives as stated in: Syllabus: must be clear. Super important.
Grading Rubrics: explains how they will be graded Contd.
Slide 49
What Is A Good Online Course? Content? Easy use of technology?
Interaction? Learner-centered outcomes-based instruction Clear
objectives as stated in: Models or Examples of quality work from
successfully submitted and graded student work Explanations of the
relevance of course materials to learning. WHY are we doing this?
(think of students perspective). Contd.
Slide 50
What Is A Good Online Course? Content? Easy use of technology?
Interaction? Learner-centered outcomes-based instruction Assessment
tools should be used: Throughout the course to establish: Learning
styles Skill/knowledge level Challenge preconceptions Clarify
current learning Contd.
Slide 51
What Is A Good Online Course? Content? Easy use of technology?
Interaction? Learner-centered outcomes-based instruction
Opportunities for students to construct and experience their own
knowledge: We all learn differently and its important to be clear
and understanding about this. Learning processes and outcomes are
not all the same. Contd.
Slide 52
What Is A Good Online Course? Content? Easy use of technology?
Interaction? Learner-centered outcomes-based instruction
Opportunities for students to become increasingly responsible for
their own learning: Instructor should provide consistent and timely
input. Encouragement, good examples, and instructor presence Try to
develop intrinsic motivation. Contd.
Slide 53
What Is A Good Online Course? Content? Easy use of technology?
Interaction? Learner-centered outcomes-based instruction Options
for demonstrating learning outcomes (if appropriate) Opportunities
for reflection Effective student support services Contd.
Slide 54
What Is A Good Online Course? Clear and captivating
instructional design the writing style, the images and plug-ins
selected, and the layout are appropriate to the intellectual
sophistication, technical savvy and needs of the audience a pathway
to guide student learning appealing presentation Contd.
Slide 55
What Is A Good Online Course? Varied forms of interactivity
with material, other students and the instructor Regular
Communication with peers and instructor Varied Interaction with
Content Source:
http://www.4faculty.org/includes/digdeeper/online/lc_outcome
s_based_instruc.htm
Slide 56
Online Course Material for Review 1.Instructional material and
resources 2.Assignments and student assessment (clear and detailed
instructions and guidelines, purposes, formal writing
standards/academic writing requirement) 3.Discussion forums
(ability for students to give feedback and ask questions during the
class is critical) 4.E-mail messages 5.Course syllabus (the
syllabus is the most important document of the entire course. It is
the road map or game plan for the course) 6.Chat space
Slide 57
Where to Go For Resources CAF website: Then click on Faculty
Evaluation Then click on Peer Review Faculty Evaluation
Coordinator