Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
A sustainable digital assessment tool for post-graduate health-science students in Optometry, Nursing and
Physiotherapy
Dr. Kwang Cham
Melbourne School of Health Sciences,
Department of Optometry and Vision Sciences,
University of Melbourne
@DrKwangCham
• Ms. Anthea Cochrane (Optometry)
• Dr. Rebecca Wong (Dentistry)
• Ms. Bronwyn Tarrant (Nursing)
• Mr. Thomas Mathew (Nursing)
• Dr. David Kelly (Physiotherapy)
• Dr. Elaina Kefalianos (Speech Pathology)
• Dr. Meg Keage (Speech Pathology)
• Mr. Iain Scott (Program Developer)
• Learning and Teaching Initiative Grant at The University of Melbourne
• Faculty of Medicine, Dentistry, and Health Sciences Seed Funding
• Melbourne School of Health Sciences Equipment Support
Project Team and Support
• OSCEs are integral for assessing clinical competency
• Feedback quality and timeliness
• Workload implications
• Adapted an app for oral presentations and trialed it for OSCE suitability
Issue
App Development
• No network connectivity required when operating.
• Scored items are underpinned by automated feedback that is randomly selected to provide a feedback report to students. Google Docs are used to populate different versions of feedback (Fig. 1).
Figure 1: An example of different versions of feedback (Nursing).
App Development
• Functionalities include an inbuilt timer and student number verification before commencing. There is also a Red Flag button that indicates automatic failure (e.g. unprofessional behaviour, performing a technique that is deemed unsafe, hygiene etc.) (Fig. 2).
Figure 2: Built-in timer.
App Development
• Grading options include both formative and summative feedback. Summative feedback provides the option of a checklist and a global score (Fig. 3, Optometry, Nursing and Physiotherapy versions).
Figure 3 : Examples of checklist and global scores (Optometry, Nursing and Physiotherapy versions).
App Development
• This is an example of the feedback report that the student will receive less than 24hrs after assessment.
• Feedback can also be edited (add on/delete) where needed to better match student performance to avoid comments being too generic. (Fig. 4).
• In Physiotherapy specific feedback is also given relative to the category of OSCE question asked e.g. length, strength, passive movement etc.
Figure 4 : An example of a feedback report (Nursing).
Preliminary Findings
SA / A
91%
96%
93%
71%
84%
76%
84%
71%
78%
80%
89%
Preliminary Findings
SA / A
69%
91%
84%
64%
73%
82%
91%
Preliminary Findings
SA / A
100%
100%
88%
100%
75%
63%
100%
100%
100%
100%
75%
Preliminary Findings
SA / A
63%
88%
63%
100%
100%
Discussion
• Traditional paper-based OSCEs are time-consuming to collate and subject to human error. The ability to deliver personalized student feedback is limited.1
• This iPad-based OSCE tool provides timely and specific written feedback in formative and summative assessment in a time- and resource-constrained educational environment. This technological intervention has been well-received by both users and receivers.
• More research is currently being undertaken to evaluate the value, impact, and engagement of the digital tool.
• A comparison of feedback retention and accuracy between verbal and written feedback is also being explored.2,3
References1.Snodgrass, S. J., Russell, T., Ashby, S. E., & Rivett, D. A. (2014). Implementation of an electronic Objective Structured Clinical Exam for assessing practical skills in pre-professional physiotherapy and occupational therapy programs: Examiner and course coordinator perspectives. Australasian Journal of Educational Technology, 30(2): 152-166.2.Humphrey-Murto, S., Mihok, M., Pugh, D., Touchie, C., Halman, S. & Wood, T. J. (2016) Feedback in the OSCE: What Do Residents Remember?, Teaching and Learning in Medicine, 28(1): 52-60.3.Denison, A., Bate, E., Thompson, J. (2016). Tablet versus paper marking in assessment: feedback matters. Perspectives on Medical Education, 5: 108-113.
What’s next
• Pool the data from Physio (110 students) and Nursing (56 students) and present it as Nursing and Allied-health outcomes
• Working with Dentistry and Speech Pathology for their version
• Further funding to work on more flexibility - ability to adjust different criteria and weightings for a particular OSCE