Upload
karina-carter
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
Peer-Review of Student Centred Learningusing QA procedures as an instrument for changeAnthony F. Camilleri
SCL Training – 19th December 2014 Brussels.
pascl.eu
Which teacher changed your life?
Did you have a year of your life you hated because of bad teaching?
Elements of Quality Assurance
• comparison of activities with a standard
• monitoring of processes
• measurement of outcomes
• iterative improvement
• prevention of errors
pascl.eu
A look at the Quality Cycle
• Act • Plan
• Do• Check
アクション 計画
実行チェックCHECK should be read as STUDYin English Check implies ‚to hold back‘(Deming, 1980)
pascl.eu
How does QA help?
QA is a powerful toolto accelerate and systemise
change
pascl.eu
Application
• comparison of activities with a
standard
• monitoring of processes
• measurement of outcomes
• iterative improvement
• prevention of errors
use these
tools to
improve SCL
within an
Institution
pascl.eu
How we are Approaching It
Institution Applies for
QA Procedure
Completes Self
Assessment
PASCL nominate
review team
Review Team Assesses Institution
Recommendations for
Improvement
CHANGEpascl.eu
Process Requires Commitment
InstitutionalPrerequisites
pascl.eu
LEARNING FROM
PEER-REVIEW
pascl.eu
Who is a peer?
• an expert in the field• an expert in what field?• Student-Centred Learning• What is student-centred learning?
pascl.eu
Features of an Expert
experience in course delivery
familiar with best-practice
comparative perspective
sense of mission
natural communicator
pascl.eu
Features of a Reviewer
• Ethical• Open-minded• Diplomatic• Observant• Perceptive• Versatile• Tenacious• Decisive
• Self-reliant• Acting with
Fortitude• Open to
improvement• Culturally
Sensitive• Collaborative
pascl.eu
A review is about communication
University
AgencyReview Team
pascl.eu
„I don't know anything about art, but I know
what I like“
Gelett Burgess
pascl.eu
„I don't know anything about the standards, but I know quality when I see
it“
Unnamed reviewer
pascl.eu
Standards of Proof
• Some credible evidence• Preponderance of evidence• Clear and convincing evidence• Beyond reasonable doubt
pascl.eu
How to go about a Review
1. Re-familiarise yourself with the standard.2. ‘Take a Walk’ around the materials and
websites of the applicant3. Begin Review, slowly and progressively.4. Keep a Questions / Doubts pad
• Note missing evidence• Note inconsistencies
5. Check your Work
pascl.eu
William E.Deming
pascl.eu
NEVER
• leave questions unanswered• breach confidentiality• compare directly with your institution
pascl.eu
Standards of Compliance
• Substantial compliance• Partial compliance• Non-compliance
Report On:• Non-compliance• Opportunities for Improvement• Best Practice
pascl.eu
In your Report
• Be yourself
Bad: The institution showed....
Good: The review team saw / found / observed....
pascl.eu
In your Report
• Be specific
Bad: We observed a lot of....
Better: The review team found multiple and consistent examples of
pascl.eu
In your Report
• Say what you know
Bad: The institution gave us wrong information.
Better: The review team found inconsistencies between evidence (x) and interview (y)
pascl.eu
In your Report
• Give your opinion where relevant
Bad: The institution is….Bad: No Comment
Better: We suspect, It seems likely that, Given the evidence available, etc...
pascl.eu
In your Report
• Give your reasoning
Bad: There is no Quality
Better: When we consider (X), (Y) and (Z), we feel it impossible to say there is a well functioning quality cycle
pascl.eu
In your Report
• Link Effect with Cause
OK: Quality systems are in place, but there is no evidence of iterative improvement
Better: Quality systems are not effective, due to lack of procedures to mainstream improvement.
pascl.eu
In your Report
• Be Clear
OK: The physical conditions of classrooms are in need of improvement
Better: Classrooms are in a dismal state – no heating, broken desks and no boards lead to an environment which highly disincentivises learning
pascl.eu
Let’s Hear your Jokes
How to learn from self-assessment1. Read the quality standard2. Understand the quality standard3. Ask Questions4. DO NOT fill it in5. Discuss it – with all stakeholders in the institution
a. How do we meet the standard?b. What can we do better?c. Does it apply in all areas?
6. Discuss it some more – do some implementation7. Have you internalised standard?8. Now fill in the questionnaire
Be Honest
„Most Quality Managers are Magicians“
Look Here
don‘t look here!
What will self-assessment produce• A time to think, communicate and
strategize• A better understanding of institutional
processes• An overview of where we are and
hopefully where we need to go
• A report
Success Factors
• Single point of Contact• Unit or Individual Leading the Process
• Institutional Mandate / Commitment• Clear Support from Institutional Management• Inform your stakeholders
• Involvement of External Stakeholders• External viewpoint of your processes are
important
• Integration with existing Quality Tools• Map the criteria against your existing standards
Success Factors (2)
• Methods to Collect and Act on Evidence• Learning Analytics• QA Reports• Internal Development Plans
• Emphasis of Quality Improvement• how are you integrating PASCL Guidelines into
your mainstream activities?• What changes do you plan to institute?• Can you show us evidence of processes in flux?
TheShow-
Off
ThePolitici
an
TheGrump
y
TheBored
Watch out for these ppl…
How to approach a peer-review
• Understand the purpose of the review• Understand your role in the review
• Your impressions• Of your part in the instituion
• Be candid and informative• Don’t try to hide things (you’ll be caught)
Main result of peer-review
an external evaluation of your institutional quality
(procedures)
a report
a certificate
pascl.eu
Run the cycle again
• Act • Plan
• Do• Check
アクション 計画
実行チェック
pascl.eu
Remember
NO process is without errors
There is ALWAYS room for improvement
pascl.eu
What institutions should do with the report1. Read the report2. Understand the report3. Ask Questions4. Discuss it – with all stakeholders in the institution
a. What did we miss in our Self-Assessment?b. Have our assumptions been confirmed or refuted?c. How can we do it better
5. Discuss it some more – do some implementation6. Have you internalised the recommendations?7. Now start the cycle again.
pascl.eu
Principle
Only completing the circle
brings about real improvement
pascl.eu
Foundations of Auditing
• Integrity – the foundation of professionalism• Fair presentation – the obligation to report
truthfully and accurately• Due professional care – the application of
diligence and judgment in auditing• Independence - the basis for the impartiality of
the audit and objectivity of the audit conclusions
• Evidence-based approach - the rational method for reaching reliable and reproducible audit conclusions in a systematic audit process
ISO 19011:2001
pascl.eu
Under the following conditions:
Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
Share Alike — If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original..
Released under a Creative Commons Attribution ShareAlike 4.0 International License
You are free:to Share — copy and redistribute the material in any medium or formatto Remix — remix, transform, and build upon the materialfor any purpose, even commercially.
This project has been funded with support from the European Commission. This publication reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein.
pascl.eu
THANK-YOUfor your attention
Feedback is welcome!
Anthony F. [email protected]://www.slideshare.net/anthonycamilleri/