Development, Evaluation and Remediation · Clinical Reasoning Tool Development Definition of...

1

Clinical Reasoning Tool Development, Evaluation and Remediation

2

• Clinical Reasoning Tool development

• Testing the effectiveness of the Tool

• Using the Clinical Reasoning Tool

• Remediating clinical reasoning

3

Clinical Reasoning

Tool Development

Background

Canadian faculties of medicine use Chart Stimulated Recall to:

– identify critical thinking and reasoning skills

– stimulate reflective practice

– provide feedback

– improve documentation skills

– help demonstrate and evaluate roles and competencies

– help structure a teaching session

– identify gaps in knowledge

Dept. Family Medicine, University of Alberta (2009)

Background Health regulated colleges across Canada use different methods to evaluate their registrants’ clinical reasoning.

Background

Sample of registrant quotes following peer assessment:

“The chart review was fine, but I wanted more feedback and discussion about the charts.”

“I feel that my actual clinical work was not explored”

“I am looking forward to the addition of a more clinical part to the assessment process”

Goal:To develop a reliable and valid tool to evaluate speech language pathologists’ and audiologists’ clinical reasoning across all practice areas as part of the Quality Assurance Program.

Clinical Reasoning Tool Development

Definition of Clinical Reasoning

“Clinical reasoning describes the process by which registrants collect and evaluate information, come to an understanding of a patient problem or situation, plan and implement interventions, evaluate outcomes, and reflect on and learn from the process.”

Clinical Reasoning Tool Development

“What a hypothetical, typical registrant who exercises average care, skill, and judgment would do in similar circumstances and thereby serve as a comparative standard.”

Definition of Reasonable:

Testing the Effectiveness of the Tool

11

Clinical Reasoning Tool Development

Face and Content Validity

The Tool and the Guide were reviewed by 5 stakeholder focus groups:

• Managers and Administrators

• Randomly selected registrants

• University Clinical Educators

• Registrants recently peer assessed (SLPs and audiologists)

• Peer Assessors (SLPs and audiologists)

The groups recommended changes and additions which were incorporated.

They agreed that the content of the Clinical Reasoning Tool was valid,

that is, it measures clinical reasoning.

Testing the Effectiveness of the Tool

Participants: Peer Assessment 2017

• 45 clinicians completed the Clinical Reasoning Tool

• 3 French and 42 English administrations

• 9 Audiologists – providing services in clinic settings

• 36 Speech Language Pathologists– 9 provided adult services

– 8 provided preschool services

– 9 provided services to school boards

– 10 provided services through SHSS, LHIN and/or private

13

Testing the Effectiveness of the Tool

Methodology

• Two Clinical Reasoning Tools were completed at the site visit– The clinician selected one patient record (chart)

– The peer assessor selected another patient record

• Both administrations of the Tool were audio recorded

• The peer assessor scored the Tools

• The audio files and scores were uploaded to Box (cloud storage)

14

Testing the Effectiveness of the Tool

Measures:

1) Pre-site visit clinician survey

2) Post-site visit clinician survey

3) Post-site visit peer assessor confidence survey

4) Inter-rater reliability

5) Clinician and peer assessor comments

15

16

Testing the Effectiveness of the Tool

Clinician Survey Results

Surveys used a 5 point Likert Scale

2 Tailed t-test used to measure change between pre and post

One question showed a significant change from pre to post

• Do you think a measure of clinical reasoning skills can be valid(it actually measures clinical reasoning)?

2 Tailed t-test t = 2.2953, df = 44, p = 0.03153

17

Testing the Effectiveness of the Tool

Peer Assessment Confidence Survey (post-administration)

“How confident are you in your determination of clinical reasoning?”

5 point Likert scale to measure responses

Results:

N = 44

Mean 4.6 – between ‘confident’ and ‘very confident’

Range 3-5

18

Testing the Effectiveness of the Tool

Inter-Rater Reliability

• The audio recordings were sent to a second peer assessor working in the same practice area for scoring.

• The peer assessors were not allowed to discuss their finding with each other.

Results

Scores were allocated:2 = complete clinical reasoning

1 = questionable clinical reasoning

0.5 = Inadequate clinical reasoning

19

Testing the Effectiveness of the Tool

Inter-Rater Reliability

Results show good inter-rater reliability (alpha coefficient > .70)

20

Testing the Effectiveness of the Tool

Inter-Rater Reliability – Calibration Exercise

• All peer assessors listened to 8 audio recordings and voted anonymously on clinical reasoning

• Results of the discussions:

– Listen to the content of clinicians’ responses

– What is and what is not clinical reasoning

– Additional variables to be aware of: lack of preparation and atypical communication skills

21

Testing the Effectiveness of the Tool

Clinical Reasoning Tool Trial Results:

• 43 clinicians were found to have clinical reasoning

• 2 clinicians were found not to have clinical reasoning

22

Clinical Reasoning Tool

To date, 129 registrants have completed 258 CRTs in the peer assessment process and 1.5% were found to need assistance.

Or . . . . .

98.5% clinicians demonstrated clinical reasoning

23

24

Next Steps

• Council approved the Clinical Reasoning Tool for the peer assessment process

• The CRT and Guide are posted on the website for all registrants (link)

• Clinical reasoning indicator added to the 2020 SAT

• Inform the membership – today’s eforum!

• CRT Remediation Program approved by the Quality Assurance Committee (QAC)

• The CRT will be implemented in the Mentorship Program in 2020

The Clinical Reasoning Tool

26

Clinical Reasoning Tool

The Clinical Reasoning Tool is

available for all registrants to use

Website-> Registrants -> Quality Assurance -> Peer Assessment

• The Clinical Reasoning Tool

• The Clinical Reasoning Tool Guide

Scenarios and Polling Questions

28

29

• How did the background information you collected direct your assessment?

• Response: I always do an artic assessment with artic kids

• Sufficient Clinical Reasoning?

Clinical Reasoning Tool

CLINICAL REASONING PROCESSES

THE MEMBER. . .

• collected sufficient information.

• applied background/clinical information in the decision

making process.

• linked information from one phase of intervention to the

next.

• incorporated the patient’s context and situation into areas

of intervention and decision making

• considered options and provided a reasonable rationale to

eliminate other options

• was flexible in their approach regarding the patient, their

needs or other intervention options

• provided a reasonable rationale to explain why they did

what they did

31

• How did the background information you collected direct your assessment?

• Response: The boy was outgoing and happy to come with me, he had reasonable attention, he was able to tolerate a formal articulation assessment and I got the information I needed to develop a therapy plan

• Sufficient Clinical Reasoning?

Clinical Reasoning Tool

CLINICAL REASONING PROCESSES

THE MEMBER. . .

• collected sufficient information.

• applied background/clinical information in the decision

making process.

• linked information from one phase of intervention to the

next.

• incorporated the patient’s context and situation into areas

of intervention and decision making

• considered options and provided a reasonable rationale to

eliminate other options

• was flexible in their approach regarding the patient, their

needs or other intervention options

• provided a reasonable rationale to explain why they did

what they did

33

• What was unique about this patient?

• Response: Nothing really, he was a typical 80-year-old with a hearing loss who needed hearing aids

• Sufficient Clinical Reasoning?

Clinical Reasoning Tool

CLINICAL REASONING PROCESSES

THE MEMBER. . .

• collected sufficient information.

• applied background/clinical information in the decision

making process.

• linked information from one phase of intervention to the

next.

• incorporated the patient’s context and situation into areas

of intervention and decision making

• considered options and provided a reasonable rationale to

eliminate other options

• was flexible in their approach regarding the patient, their

needs or other intervention options

• provided a reasonable rationale to explain why they did

what they did

35

• What was unique about this patient?

• Response: The 80-year-old patient with a hearing loss came from a large family who got together often. His wife said that he is isolating himself at family gatherings and not participating, which is not typical behaviour for him. She was concerned that he was missing out on important conversations.

• Sufficient Clinical Reasoning?

Clinical Reasoning Tool

CLINICAL REASONING PROCESSES

THE MEMBER. . .

• collected sufficient information.

• applied background/clinical information in the decision

making process.

• linked information from one phase of intervention to the

next.

• incorporated the patient’s context and situation into areas

of intervention and decision making

• considered options and provided a reasonable rationale to

eliminate other options

• was flexible in their approach regarding the patient, their

needs or other intervention options

• provided a reasonable rationale to explain why they did

what they did

37

• How did you decide to continue with or move onto another section of the management plan?

• Response: This patient has advanced dementia and cannot tolerate a texture upgrade. There are real safety issues. However, the dietitian and I met with the family to discuss what foods she liked and disliked and we monitor her intake. The dietitian and I meet at weekly rounds to discuss the patient, and we do joint re-assessments when nursing inform us there is an issue

• Sufficient Clinical Reasoning?

Clinical Reasoning Tool

CLINICAL REASONING PROCESSES

THE MEMBER. . .

• collected sufficient information.

• applied background/clinical information in the decision

making process.

• linked information from one phase of intervention to the

next.

• incorporated the patient’s context and situation into areas

of intervention and decision making

• considered options and provided a reasonable rationale to

eliminate other options

• was flexible in their approach regarding the patient, their

needs or other intervention options

• provided a reasonable rationale to explain why they did

what they did

39

• How did you decide to continue with or move onto another section of the management plan?

• Response: She’s got severe swallowing problems and dementia so we haven’t moved on.

• Sufficient Clinical Reasoning?

Clinical Reasoning Tool

CLINICAL REASONING PROCESSES

THE MEMBER. . .

• collected sufficient information.

• applied background/clinical information in the decision

making process.

• linked information from one phase of intervention to the

next.

• incorporated the patient’s context and situation into areas

of intervention and decision making

• considered options and provided a reasonable rationale to

eliminate other options

• was flexible in their approach regarding the patient, their

needs or other intervention options

• provided a reasonable rationale to explain why they did

what they did

Remediating Clinical Reasoning

41

Remediating Clinical Reasoning

Remediation will differ for clinicians who show:

• Clinical reasoning in most but not all areas

Vs.

• Minimal clinical reasoning

• No clinical reasoning

42

Clinical Reasoning

Remediation Program

Much of the academic research focusses on teaching clinical reasoning to students, but the Quality Assurance Committee needed to develop a program for experienced professionals who need help in this area.

43

Clinical Reasoning Remediation Program

As part of the solution, adult learning principles (Kearsley 2010) were incorporated;

– making the program as self-directed as possible

– ensuring that it is relevant and practical

– incorporating learning by doing

– respecting the clinician and building on success

44

Clinical Reasoning Remediation Program

• The remediation program involves a trained peer coach working with the clinician at their place of work using the clinician’s patient records.

• The Quality Assurance Committee decides how long the remediation program will last.

45

Clinical Reasoning Remediation Program

• The program focusses on critical thinking (Kelly and Crawford

2007), that is thinking ahead, thinking in action and thinking back (reflection).

• With reflection the clinician verbalizes their thought processes. Through ‘thinking out loud’, tacit knowledge is made explicit and solutions are developed.

46

Clinical Reasoning Remediation Program

Following the constructs of self-direction and building on success, the clinician reviews and selects patient records where they have successfully used the clinical reasoning processes from the CRT. For example:

– evidence showing they collected sufficient information

– applied it and linked information from one phase of intervention to the next

– That they took the patient’s unique context into account and showed a flexible approach regarding the patient’s needs

– Considered other intervention options

– In summary, they could provide a reasonable rationale for why they did what they did.

47

Clinical Reasoning Remediation Program

• The peer coach then helps the clinician through a series of questions to problem solve (Nembhard 2014) those areas where clinical reasoning is lacking.

• Discussion and Practice Scenarios are included. Different clinical scenarios will trigger critical and reflective thinking and problem solving.

• Progress is measured through re-administering the CRT.

48

Scenarios – SLP, Pre-School

A very young child has been referred for delayed language. The child arrived to the assessment with her father. The father spoke Ukrainian and was adamant that his child speak Ukrainian at home. How is this new piece of clinical information going to impact on your clinical reasoning and decision making?

What further information would you collect?

How would you apply the information in assessment?

What is the unique patient context?

What options would you consider?

How would you apply the information in treatment?

How would you involve other professionals?

Scenarios – Audiologist

A 15 year old boy is culturally deaf and the mother wants to pursue cochlear implants. You are not sure from the 15 year old’s response if this is what he wants.

What further information would you collect?

How would you apply the information in assessment?

What is the unique patient context?

What options would you consider?

How would you apply the information in treatment?

How would you involve other professionals?

51

Practice Advice Team

Alexandra CarlingDirector of Professional Practice & QA

acarling@caslpo.com

Samidha JoglekarAudiology Advisor

sjoglekar@caslpo.com

Sarah Chapman-JayAdvisor, SLP and QA

slppracticeadvice@caslpo.com

David BeattieConseiller sur l’exercisede l’orthophonie

conseillerorthophonie@caslpo.com

Thank You!

Additional contact information and resources:acarling@caslpo.com | 416 975-5347, ext. 226

Follow CASLPO on social media:|

facebook.caslpo.com youtube.com/user/caslpo

www.caslpo.com

Publications:www.caslpo.com/express

www.caslpo.com/members/resources

Clinical Reasoning Tool

CLINICAL REASONING PROCESSES

THE MEMBER. . .

• collected sufficient information.

• applied background/clinical information in the decision

making process.

• linked information from one phase of intervention to the

next.

• incorporated the patient’s context and situation into areas

of intervention and decision making

• considered options and provided a reasonable rationale to

eliminate other options

• was flexible in their approach regarding the patient, their

needs or other intervention options

• provided a reasonable rationale to explain why they did

what they did

Clinical Reasoning Tool

55

Clinical Reasoning Tool

56

Clinical Reasoning Tool

57

Clinical Reasoning Tool

58

Clinical Reasoning Tool

59

Clinical Reasoning Tool

60

Clinical Reasoning Tool

61

Clinical Reasoning Tool

62

Clinical Reasoning Tool

63