Embed Size (px)
DESCRIPTION
The presentation considers (a) general principles of peer-review, (b) remote peer-review of e-learning programmes. The presentation was created for an ITC/ILO training on the ecbcheck (http://ecbcheck.efquel.org) quality label, at its campus in Torino, 28.09.2012
Citation preview
www.efquel.org
Online Peer-Review... of e-learning programmes
Anthony F. Camilleri
ECB-Check TrainingITC/ILO Turin – 28.09.2012
www.efquel.org
What‘s in a Name?
Audit Expert-Review
Objective Subjective
Precise standards Broad guidelines
Verifies account of client Interprets account of client
Requires detailed knowledge of administrative procedures
Requires detailed knowledge of the subject area concerned
standards-basedreview
www.efquel.org
Who is a peer?
• an expert in the field• an expert in what field?• e-learning!• what is e-learning?
www.efquel.org
Features of an Expert-Peer
experience in course design and delivery
familiar with best-practice
comparative perspective
sense of missionnatural communicator
www.efquel.org
Inter-subjectivity
• different subjective perceptions, taken from different viewpoints, give an objective view of reality
• the quality basis of an external review• requires consensus
consensus is not a diplomatic nicetybut an essential pre-requisite of quality
www.efquel.org
Communicating with the institution
Communication starts
• This is not a one-way exercise!– Has the institution given enough
information to allow a successful review?–What don‘t you understand, and who can
answer your questions?• Be active in schedule-design
with self-assessment!
www.efquel.org
Conducting a Review
Familiarise yourself with the standards
1. Understand which standard covers which area (sometimes they can overlap).
2. Understand their relation to each other.3. Make sure you have a vision of successful and non-
successful completion of each standard – based on comparative experience
www.efquel.org
Inspect the Course
„Most Quality Managers are Magicians“
Look Here
don‘t look here!
www.efquel.org
Inspect the Course
• Log onto the LMS• Look through every part of the
course:– Lessons– Help-pages– Tests– Forums / support pages– Interactive material–What else?
www.efquel.org
Fill in your Forms
• Your forms are a letter to the agency
COMMUNICATE!
www.efquel.org
Communicating with the agency
„I don't know anything about art, but I know
what I like“
Gelett Burgess
www.efquel.org
Communicating with the agency
„I don't know anything about the standards, but I know quality when I see
it“Unnamed reviewer
www.efquel.org
Communicating with the agencyStandards of Proof
• Some credible evidence• Preponderance of evidence• Clear and convincing evidence• Beyond reasonable doubt
www.efquel.org
Communicating with the agencyIn your Report
• Be yourself
Bad: The institution showed....
Good: The review team saw / found / observed....
www.efquel.org
Communicating with the agencyIn your Report
• Be specific
Good: The institution showed....
Better: The review team found multiple and consistent examples of
www.efquel.org
Communicating with the agencyIn your Report• Say what you know
Bad: The institution lied....
Good: The review team found inconsistencies between evidence (x) and interview (y)
www.efquel.org
Communicating with the agencyIn your Report• Give your opinion (where relevant)
Bad: The institution is... / or NOTHING
Good: We suspect, It seems likely that, Given the evidence available, etc...
www.efquel.org
Communicating with the agencyIn your Report
• Give your reasoning
Bad: There is no quality.
Good: When we consider (X), (Y) and (Z), we find it impossible to say there is quality
www.efquel.org
Communicating with the agencyIn your Report• Be CLEAR
Bad: The structure of the LMS is in need of improvement.
Good: The LMS barely functions – it is littered with wrong links, the help-function is unusable and most sections still need to be populated
www.efquel.org
Communicating with the agencyIn your Report• Link Effect with Cause
Bad: Quality systems are in place, but there is no evidence of iterative improvement.
Good: Quality systems are not effective, due to lack of iterative improvement procedures.
www.efquel.org
One last thing
Remember that
Quality Labels should act as Quality-Enhancers
not only transparency tools
offer a path towards quality improvement
www.efquel.org
Hvala!Thank-you for your attention
Questions?
Anthony F. Camilleri ([email protected])Presentation available from: http://www.slideshare.net/anthonycamilleri/
Released under a Creative Commons Attribution ShareAlike 3.0 Slovenia License
You are free:• to Share — to copy, distribute and
transmit the work• to Remix — to adapt the work
Under the following conditions:Attribution — You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work).Share Alike — If you alter, transform, or build upon this work, you may distribute the resulting work only under the same or similar license to this one.
