Embed Size (px)
Citation preview
Graduate Certificate in Learning & Teaching for the Health Professions
Master in Health (Medical Education)
CAM506Assessment in the Health Professions
Semester 2, 2010
Unit Outline
CRICOS Provider Code: 00586B
Contact details
Unit Coordinator Assoc Prof Craig Zimitat School of Medicine
Unit Lecturer: Assoc Prof Craig Zimitat Campus: Hobarte-mail: [email protected]: 03 6226 2660 fax: 03 6226 4854
Unit Lecturer: Assoc Prof Jan RadfordCampus: Launcestone-mail: [email protected]: 03 6234 XXXXfax: 03 6234 XXXX
Unit lecturer: Ms Rose MooreCampus: Rural Clinical Schoole-mail: [email protected]: 03 6430 4556fax:
Unit Administration:Campus: Hobarte-mail: @utas.edu.auphone: 03 6226 XXXXfax: 03 6226 XXXX
Medical EducationSchool of MedicinePrivate Bag 73University of TasmaniaHobart Tasmania 7000
www.medicine.utas.edu.au
CRICOS Provider Code: 00586B
© University of Tasmania, 2010
CRICOS Provider Code: 00586B
Contents
Contact details 2Contents 1Unit Summary 2Learning outcomes 3
Generic Graduate Attributes 3Prior knowledge and skills 4
Details of Teaching Arrangements 4Lectures/Intensive sessions 4Structured small group work (peer meetings) 4Specific attendance/performance requirements 4
Unit Schedule 1Learning expectations and strategies 4
Expectations 4Assessment details 1
Submission of Assignments 3Requests for extensions 4Penalties 4Resubmissions 4
Review of results and appeals 4Prescribed Text and Readings 4Academic referencing 7Plagiarism 7Occupational health and safety (OH&S) 9Further information and assistance 9
1
Unit Summary
CAM506: Assessment in the Health Professions
This unit focuses on aspects of assessment relevant to medical and health professional education. It has been designed to enable health professional academics and teachers to develop a critical appreciation of the appropriate use of assessment tools in a range of health settings and to use the results of assessment to contribute to continuous improvement in teaching practice. Furthermore, it will promote the enhancement of professional practice in university learning and teaching through critical analysis and reflection on the assumptions that underpin professional practice. This will be achieved primarily by developing knowledge of the theory and practice of evaluation-based teaching through critical self-reflection and peer observation. Group work and problem solving will be features of these processes.
Offered: Rural Clinical School
Launceston Clinical School
Hobart Clinical School
Staff: Assoc Prof Craig Zimitat, Assoc Prof Jan Radford,
Ms Rose Moore, Dr Bunmi Malau-Aduli
Unit weight: 12.5%
Teaching pattern: Block teaching (face-to-face). Structured small group work with negotiated activities or aligned with Assessment Committee participation.
Prerequisites: ELT501 or CAM502
Courses: [E5S] and [E5T]
2
Learning outcomes
On completion of this unit, you should be able to:
Critically appraise the range of assessment tools available in health professional education and choose the most appropriate for assessment of particular skills, knowledge and attitudes in different clinical contexts.
Explore the nature and function of a range of assessment instruments;
Explore, analyse and improve validity, reliability and objectivity of assessment tasks;
Apply understanding of the psychometric properties of assessment methods in the improvement of assessment items/tasks;
Demonstrate an understanding of strengths and weaknesses and application of standard setting techniques in knowledge and skill’s based assessments;
Design and implement assessment tasks relevant to the curriculum and subject area, based upon accepted concepts and theoretical principles of modern assessment practice.
Generic Graduate AttributesThe University has defined a set of generic graduate attributes (GGAs) that can be expected of all graduates (see http://www.utas.edu.au/tl/policies/index.htm). By undertaking this unit you should make progress in attaining the following attributes:
Knowledge will be developed by learning and applying new skills and understandings (related specifically to critically reflective practice and assessment design) to participants’ particular teaching/learning contexts. This will be assessed through: the application of knowledge in the development of practical teaching resources and strategies; the ability to reflect and evaluate on the application of these resources and strategies within learning environments; and the ability to situate this work within the literature.
Communication skills will be developed and assessed through: structured opportunities to contribute to class discussion; collaboration, negotiation and discussion with peers in regard to the process of peer observation; a written reflective response to this peer observation process; and the design and written critique of an assessment blueprint or rubric, informed by literature, experience and discussion with peers.
Problem-solving skills will be developed by: demonstrating, modelling and critiquing critical issues in teaching and learning; engaging with the tasks of organising, implementing, reflecting and acting upon outcomes from a peer observation process; and the tasks of designing, critiquing and refining an assessment blueprint or rubric based on interaction with literature, reflection on personal experience and discussion with peers.
3
Social responsibility will be developed through personal reflection on, and group-based discussion of: the ethical implications of engaging in, and reporting on, a peer observation process with a colleague; and the role and function of assessment blueprints and rubrics in making the process of assessment transparent to, and equitable for, all students.
Prior knowledge and skillsYou need to have completed ELT 501, CAM502 or equivalent. You must be teaching in a clinical health care setting, be committed to self-improvement in your teaching and other educational activities, and willing to engage colleagues in this process.
Details of Teaching Arrangements
Lectures/Intensive sessionsThere will be 10 x 2.5 hr teaching sessions throughout the semester. Attendance at 80% of sessions is expected, particularly for those related to assessment tasks.
Structured small group work (peer meetings) These meetings will occur throughout semester. They may be aligned with School assessment meetings. In addition, course participants are encouraged to form study groups.
Specific attendance/performance requirementsActive and ongoing engagement in the course is expected.
Learning expectations and strategies
ExpectationsThe University is committed to high standards of professional conduct in all activities, and holds its commitment and responsibilities to its students as being of paramount importance. Likewise, it holds expectations about the responsibilities students have as they pursue their studies within the special environment the University offers.
The University’s Code of Conduct for Teaching and Learning states:
Students are expected to participate actively and positively in the teaching/learning environment. They must attend classes when and as required, strive to maintain steady progress within the subject or unit framework, comply with workload expectations, and submit required work on time.
4
Unit Schedule
Topic References LCSThurs
HCSMon
RCSTues
1 Assessment BasicsFairness, Transparency; Formative, Summative, Norm-references, criterion referenced; Standards-based; Reliability, Validity;
Hager et al, 1994Knight, 2001Sadler, 1989
15 July 19 July 20 July
2 Blueprinting and SamplingSelecting most appropriate assessment strategies for stated learning outcomes/competencies
Crossley et al, 2004; Epstein, 2007; Hambdy, 2006; Schuwirth, 2004; Wilkinson, 2008
29 July 02 Aug 03 Aug
3 MCQ 1MCQs, EMQs. Writing MCQs.
Case & Swanson, 1998; Wood & Cole, 2004; Downing & Haladyna, 1997
12 Aug 16 Aug 17 Aug
4 OSCE 1History of development. Long case, OSLER, OSCE etc. Structure and details. Issues with OSCEs e.g. shared view of goal, training etc. Planning to write OSCEs
Bouriscot et al, 2007 26 Aug 30 Aug 31 Aug
5 Standard SettingConcept, Methods. Angoff & Hofstee for knowlegde-based assessment. Borderline method for skills-based assessment.
Bandaranayake, 2008Ben David, 2000Pell & Roberts, 2006
09 Sept 13 Sept 14 Sept
6 Workplace AssessmentAssessing intern and practitioner performance.
Norcini, 2003 23 Sept 27 Sept 28 Sept
7 Assessment of Groupwork and TeamworkSmall group tutorials and work-based teams.
Farmer et al, 2002Radford, 2008
07 Oct 11Oct 12 Oct
8 Assessment of Attitudes and Non-technical skills Epstein & Hundert, 2002 21 Oct 25 Oct 26 Oct
9 Portfolio Based AssessmentElectronic Portfolios
Driesen et al, 2006 ; Driesen et al, 2007 04 Nov 08 Nov 09 Nov
10 OSCE2 Reviewing and Improving OSCEsWriting an OSCE. Feedback on OSCE performance.
RCPSC, 2004; Bouriscot et al, 2007Thistlethwaite, 2002
18 Nov 22 Nov 23 Nov
11 MCQ 2 – Reviewing and Improving MCQsPscyhometric analysis of MCQs from past examinations. Reviewing MCQs.
Case & Swanson, 1998; Wood & Cole, 2004; Downing & Haladyna, 1997
02 Dec 06 Dec 07 Dec
1
Assessment details
Assessment task 1 – 30%
Task description Develop and critique a blueprint for assessment. The assessment may be the teaching that you are currently doing, a range of learning outcomes derived from the Medical Graduate Profile, a discipline section from one of the Year handbook, a section of the ACF Junior Doctors, or other professional competencies statement (e.g. ANCF, ACOPRA).
The task consists of two sections:1. Create an initial blueprint which will include the full
range of appropriate assessment instruments for learning outcomes/competencies;
2. Outline the strengths and weaknesses of the various assessment instruments relative to the outcomes they are to assess; and
3. Create a proposed blueprint which will include your selected assessment instruments for learning outcomes/competencies;
4. Reach a justified conclusion about the instruments that you consider most appropriate for your context.
The blueprint / rubric and the critique should be informed by literature, your own experience and discussion with peers.
Task length 2000-2500 words
Links to learning outcomes
1, 2,4
Links to GGAs Knowledge, Communication skills, Problem-solving skills, Social responsibility
Assessment criteria / guidelines
Articulate rationale that describes relationship between outcomes and assessment instruments.
Completion of assessment blueprint Correct relationship between instruments and
outcomes Justification on use of instruments and application to
context, with reference to assessment theory and principles.
Adherence to presentation conventions including legibility, spelling, punctuation, grammar and current referencing procedures for Medical Education.
Date due 31 August, 2010
Absolute final date due 10 December, 2010
1
Assessment task 2 – 30%
Task description Improvement of clinical attachment reports.Interview a clinician who completes competency-based assessment of health science students using clinical attachment reports/forms. Report on their use of the form and suggestions for improvement.
Task length 2000 words
Links to learning outcomes
1, 2,4
Links to GGAs Knowledge, Communication skills, Problem-solving skills, Social responsibility
Assessment criteria / guidelines
Presentation of assessor’s comments
Analysis of assessor’s use of clinical attachment forms
with reference to assessment theory and principles.
Recommendations for improvement
Adhere to presentation conventions including legibility,
spelling, punctuation, grammar and current referencing
procedures for Medical Education.
Date due 26 October, 2010
Absolute final date due 10 December, 2010
2
Assessment task 3 – 40%
Task description Develop assessment items through the full sequence of QA processes used within the School of Medicine. This will usually be achieved through engagement with the School’s local Assessment Committee.
1 x OSCE station with criteria 20 x MCQ Reflective essay Complex long case, etc. Other tasks by negotiation
The full QA process includes development of items, peer review for improvement, implementation in course, analysis and review of psychometric properties..
Task length 2500-3000 words
Links to learning outcomes
1, 2,4
Links to GGAs Knowledge, Communication skills, Problem-solving skills, Social responsibility
Assessment criteria / guidelines
Development of assessment item
Evidence of peer review and changes made
Final item for use
Evaluation/Analytical data
Revisions from Evaluation
Adhere to presentation conventions including legibility,
spelling, punctuation, grammar and current referencing
procedures for Medical Education.
If you are doing an alternative task, then it will be necessary to keep a journal that outlines your activities and reflections, data and products of your work, and some evaluation of the quality of the products.
Date due 10 December, 2010
Absolute final date due 10 December, 2010
Submission of AssignmentsCompleted assessment tasks should be submitted via email to [email protected] before 5pm on the due date. A hard copy of the assignment, including a signed cover sheet [attached], appendices and non electronic attachments, should be delivered or mailed to the following address before 5 p.m. on the due date:
Craig Zimitat
3
Medical Education UnitPrivate Bag 73School of Medicine
Hobart 7000
Due to occasional delays in mail between campuses and departments, the date and time the email was sent will be deemed to be the date and time the assignment is received.
Requests for extensionsAll requests for extensions should be submitted via email to the unit coordinator ([email protected]) before the due date of the assignment. Generally, foreseeable work commitments will not be grounds for an extension.
PenaltiesUnless an extension has been granted in writing, a penalty of 5% of the awarded mark will be deducted for each day the assignment is overdue. As copies of assignments are to be submitted electronically, submission on weekends is possible. Weekend days, therefore, will attract the same penalties as weekdays.
Task length and word limits have been set for every assessment task. Submissions that exceed the word limit by more than 10% will incur a penalty of 10% of the awarded mark for each 10% over the word limit.
ResubmissionsResubmission of assignments will only be allowed in exceptional circumstances and will be considered on a case-by-case basis by the unit coordinator.
Review of results and appeals
Appeals should go to the lecturer-in-charge in the first instance. If unresolved, appeals are referred to the Course Coordinator and, if necessary, to Faculty Executives (Faculty of Education and Faculty of Health Science).
Prescribed Text and Readings
Bandaranayake, R.C. (2008). Setting and maintaining standards in multiple choice examinations. Medical teacher 30: 836-845
Ben-David, M.F. (2000). Standard setting in student assessment. Medical Teacher 22: 120-130
Bouriscot, K.A., Roberts, T.E. & G. Pell. (2007). Using borderline methods to compare passing standards for OSCEs at graduation across three medical schools. Medical Education 41:1024-31
Bouriscot, K.A., Roberts, T.E. and W.P. Budrick. (2007). Structured assessment of clinical competence. Edinburgh: Association for the Study of Medical Education
4
Case, S. and N. Swanson. (1998). Constructing Written Test Questions for the Basic and Clinical Sciences. Philadelphia: National Board of Medical Examiners. Chapters 1-6.
Crossley, J., Humphris, G., & B.C. Jolly (2002). Assessing health professionals. Medical Education 36: 800-804
Downing, S. & T.M. Haladyna (1997). Test item development: validity evidence from quality assurance procedures. Applied Measurement in Education 10: 61-82
Driesen, E.W. et al (2006). Portfolios in medical education: why do they meet with mixed success? A systematic review. Medical Education 41: 1224–1233
Driesen, E.W. et al (2006). Validity of portfolio assessment: which qualities determine ratings? Medical Education 40: 862–866
Driesen, E.W. et al (2007). Web- or paper-based portfolios: is there a difference? Medical Education 41: 1067–1073
Epstein, R.M. & E.M. Hundert. (2002). Defining and assessing professional competence. JAMA 287:226-235
Epstein, R.M. (2007). Assessment in Medical Education. NEJM. 356: 387-396
Farmer, E., Beard, J.D., Dauphinee, W.D., LaDuca, T. and K.V. Mann. (2002). Assessing the performance of doctors in teams and systems. Medical Education 36:942–948
Gleeson, F. (1997) Assessment of clinical competence using the Objective Structured Long Examination Record. Medical Teacher 19: 7-14
Hambdy, H. (2006). Blueprinting for the assessment of health care professionals. The Clinical Teacher 3: 175-179
Hager, P., Gonczi, A. & J. Athanasou. (1994). General issues about assessment of competence. Assessment & Evaluation in Higher Education 19: 3-16
Knight, P. (2001). A briefing on key concepts – Formative and summative, criterion and norm-referenced assessment. Learning and Teaching Support Network Generic Centre, UK.
Norcini, J. (2003). ABC of Learning and Teaching in Medicine. Workplace assessment. BMJ 2003: 753-755
Pell, G. & T. Roberts (2006). Setting standards for student assessment. International Journal of Research & Methods in Education 29: 92-103
Radford, J. (2009). Assessment of medical practitioner competencies to be an effective team member, and assessment of the team.
5
Sadler, D. R. (1989) Formative assessment and the design of instructional systems Instructional. Science 18: 119-144
Shumway J. M. and R.M. Harden (2003) AMEE Medical Education Guide No 25: The assessment of learning outcomes for the competent and reflective physician. Medical Teacher 25: 569-584
Shuwirth, L. (2004). Assessing medical competence: finding the right answers. The Clinical Teacher 1: 14-18
Thistlethwaite, J. (2002). Developing an OSCE station to assess the ability of medical students to share information and decisions with patients: Issues relating to inter-rater reliability and the use of simulated patients. Education for Health 15: 170-179
Wilkinson, T & W. Wade. (2009). A blueprint to assess professionalism. Academic Medicine 84: 551-558
Other references
Biggs, J. & C. Tang. (2008). Teaching for quality learning at University. Oxford University Press: London
Case, S. and N. Swanson. (1998). Constructing Written Test Questions for the Basic and Clinical Sciences. Philadelphia: National Board of Medical Examiners.
CIPHER. (2007). Review of work-based assessment methods. Centre for Innovation in Professional Health Education & Research. The University of Sydney.
Downing, S.M. & R. Yudkowsky. (2009). Assessment in health professions education. Routledge: London
Jackson, N., Jamieson, A. & A. Khan. Assessment in medical education and training. A practical guide. Radcliffe Publishing: New York
Web links
Developing multiple choice questions http://www.ranzcog.edu.au/fellows/pdfs/diploma-mcqs/developing-mcqs-for-RCPSC.pdf
Assessment Quality Assurance processes at School of Medicine, Utashttp://www.medicine.utas.edu.au/meu
RCPSC. (2004). Guidelines for Developing OSCE stations. Ottawa: Royal College of Physicians and Surgeons of Canada. Available online: http://rcpsc.medical.org/residency/certification/examinerguide/OSCEguidelines_e.php
RCPSC. (2005) Guidelines for Developing a Royal College CanMEDS Examination Blueprint. Ottawa: Royal College of Physicians and Surgeons of Canada. Available online:
6
http://rcpsc.medical.org/residency/certification/examinerguide/CanMEDSblueprint_e.pdf
Wood, T. & G. Cole. (2004). Developing Multiple Choice Questions for the RCPSC Certification Examinations. Ottawa: Royal College of Physicians and Surgeons of Canada Available online: http://rcpsc.medical.org/residency/certification/examinerguide/index.php
Journals – General
Any relevant articles in the following journals, all available as e-journals through the UTAS library:
Medical Education Academic Medicine
Medical Teacher Medical Education Online
Learning & Teaching in Medicine British Medical Journal
Assessment & Evaluation in Higher Education
Journal of Higher Education
Higher Education Research and Development
Review of Higher Education
Academic referencing
In your written work you will need to support your ideas by referring to scholarly literature, works of art and/or inventions. It is important that you understand how to correctly refer to the work of others and maintain academic integrity.
Failure to appropriately acknowledge the ideas of others constitutes academic dishonesty (plagiarism), a matter considered by the University of Tasmania as a serious offence.
The required referencing style for this course is APA. The Library has information about how to reference in this style (http://www.utas.edu.au/library/info/subj/education.html).
Please read the following statement on plagiarism. Should you require clarification please see a unit coordinator or lecturer.
Plagiarism
Academic Integrity & PlagiarismAcademic integrity is about mastering the art of scholarship. Scholarship involvesresearching, understanding and building upon the work of others and requires that yougive credit where it is due and acknowledge the contributions of others to your ownintellectual efforts.
At its core, academic integrity requires honesty. This involves being responsible forethical scholarship and for knowing what academic dishonesty is and how to avoid it.
7
Plagiarism. Plagiarism is a form of cheating. It is taking and using someone else'sthoughts, writings or inventions and representing them as your own; e.g., using anauthor's words without putting them in quotation marks and citing the source; usingan author's ideas without proper acknowledgment and citation; copying anotherstudent's work.
If you have any doubts about how to refer to the work of others in your assignments,please consult your lecturer or tutor for relevant referencing guidelines, and theacademic integrity resources on the web at http://www.academicintegrity.utas.edu.au
Copying someone else’s work. The intentional copying of someone else’s work andpresenting it as one’s own is a serious offence punishable by penalties that may rangefrom a fine or deduction/cancellation of marks and, in the most serious of cases, toexclusion from a unit, a course or the University. Details of penalties that can beimposed are available in the Ordinance of Student Discipline – Part 3 AcademicMisconduct, see http://www.utas.edu.au/universitycouncil/legislation/
Self-copying/Re-submission of assessment. It is inappropriate to copy your own work,in part or in whole, and submit it for assessment in more than one Unit of study atthis, or another, university. This also applies to students repeating a Unit. Unlessotherwise approved, all assessment tasks undertaken in a unit must be done within theenrolment period.
Group work. It is important that all group members make appropriate contributions tothe required task. Copying from others, or contributing less, little or nothing to agroup assignment and then claiming an equal share of the marks are not appropriate.When working as a member of a group or team, it is important to keep records of yourown work. Even though you may have group discussions and work together – alwayswrite your own notes, and keep records what you have personally contributed to anygroup assessment product/s.
Collusion. Protect your academic work. The intentional sharing of your workpotentially allows others to copy your work and cheat and gain an academicadvantage. In these circumstances, both you and the person that copied your workmay be subject to allegations of academic misconduct.
The University and any persons authorised by the University may submit yourassessable works to a plagiarism checking service, to obtain a report on possibleinstances of plagiarism. Assessable works may also be included in a referencedatabase. It is a condition of this arrangement that the original author’s
8
permission is required before a work within the database can be viewed.
Academic Integrity & Experimental SciencesIn addition to plagiarism, responsible and ethical conduct of research requires that allresearchers have confidence in research undertaken and reported to peers. Thefalsification and fabrication of data are inconsistent with academic integrity.
Falsification of data refers to the selective modification of data collected in theconduct of experimental research, or the misrepresentation of processes or uncertaintyduring statistical analysis of the data. Falsification may also involve the selectiveomission, deletion, or suppression of data inconsistent with the research objectives.
Fabrication of data. This refers to the creation of records of research for which thereis no basis in fact, with the intent to mislead or deceive.
For further information on this statement and general referencing guidelines, see http://www.utas.edu.au/plagiarism/
Occupational health and safety (OH&S)
The University is committed to providing a safe and secure teaching and learning environment. In addition to specific requirements of this unit you should refer to the University’s policy at: http://www.admin.utas.edu.au/hr/ohs/pol_proc/ohs.pdf
Further information and assistance
If you are experiencing difficulties with your studies or assignments, have personal or life planning issues, disability or illness which may affect your course of study, you are advised to raise these with your lecturer in the first instance.
9
Assignment Return AddressOFFICE USE ONLY
Assignment received:
Student Name:
Student ID:
Unit Code: CAM506
Assignment Title: Assessment in Health Professions
I declare that all material in this assignment is my own work except where there is clear acknowledgement or reference to the work of others. I am aware that my assignment may be submitted to plagiarism detection software, and might be retained on its database. I have read the University statement on Academic Misconduct (Plagiarism) on the University website at www.utas.edu.au/plagiarism or in the Student Information Handbook.
Signed…………………………………………………………………………. Date ……………………………………………
Assessor’s comments: ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Assessment: ................................................Assessor’s Signature (optional) :................ Dated: ................................................
10
School of Medicine
Assignment Cover Sheet
11
