1

Quality

Requirements: More on Competencies

Now that we have talked about the MOC requirements for PQI, let’s discuss the competencies in more depth.

2

ABR Maintenance of Certification

ABR Maintenance of Certification

Six Essential CompetenciesSix Essential Competencies

We will specifically address the PBLI and SBP competencies.

2

3

Competencies

These are somewhat nebulous terms, so let’s simplify them with some analogies.

4

Mirror = PBLI Village = SBP

We are going to use the mirror as an analogy for practice based learning and a village to represent systems-based practice and improvement. Let’s look at practice based learning and improvement first.[Source: Ziegelstein, RC and Fiebach, NH Academic Medicine January 2004 Vol 97, No 1 p83-88.]

4

5

Mirror = PBLI

Ziegelstein and Fiebach liken this to holding a mirror up to ourselves to document, assess, and improve our practice

• Specific tools reflect radiologists practice, for example:–CME/SAM, RADKAT, Path confirmed dx, etc.

–Morbidity and mortality conferences

–Peer review

Ziegelstein and Fiebach liken PBLI to holding a mirror up to ourselves to document, assess and improve our practice.

5

6

Mirror = PBLI

• From the ACGME/ABR, Practice Based Learning and Improvement requires that physicians:–Demonstrate the ability to investigate and evaluate care of patients 

–Appraise and assimilate scientific evidence

–Continuously improve patient care based on constant self‐evaluation and life‐long learning

For radiologists, practice-based learning and improvement means these three things:

6

7

PBLI

• So, PBLI is really about you…

and improving the quality of your own skills and practice

So, PBLI is really about you…and improving the quality of your own skills and practice.

8

Now let’s focus on systems-based practice.

8

9

SBP = the village

• In Ziegelstein and Fiebach’s analogy,  the village is equivalent to the healthcare system where the non‐physician members and health care delivery play an important role in a patient’s experience.

“It takes a village to raise a child”– African proverb

“It takes a village to raise a child” – African proverb, made famous by Hillary Clinton (Book “It takes a village”)

9

10

SBP = The Village

From the ACGME/ABR, systems‐based practicerequires that physicians:

- Demonstrate an awareness of and responsiveness to the larger context and system of health care

- Call effectively on other resources in the system to provide optimal health care

.

10

11

SBP = The Village

• So SBP for radiology is really about the whole process of creating an image and a report…. from the ordering of the exam to the delivery of the final report to the ordering provider

Note: The villagers are included in the village.  

You are a villager!

So SBP is really about the whole process of creating an image and a report…. from the ordering of the exam to the delivery of the final report to the ordering providerThe villagers are included in the village. You are a villager!

11

12

Radiology Quality Map

This larger circle represents the whole process - the village – and is composed of smaller circles representing each step. At each step, there is a opportunity for things to go well, or for error.This chart provides a diagrammatic way of outlining the steps where there are opportunities for implementing Continuous Quality Improvement.

12

13

•Errors•Timeliness•Succinctness•Emergent/important

•Errors•Timeliness•Succinctness•Emergent/important

•ACR Appropriateness Criteria•Ordering error

•ACR Appropriateness Criteria•Ordering error

•Access time•Scheduling error

•Access time•Scheduling error

•Patient wait time•Pt education (prep/expectations)

•Patient wait time•Pt education (prep/expectations)

•Optimal protocol

•Protocol error

•Optimal protocol

•Protocol error

•Time out procedure•Time out procedure

•Peer review•Accuracy•Structured report•Report answers clinical question

•Peer review•Accuracy•Structured report•Report answers clinical question

•Referring prov happy•Query answered?•Health improved?

•Referring prov happy•Query answered?•Health improved?

•Patient safety (falls, infection)•Comfort•Optimal technique

•Patient safety (falls, infection)•Comfort•Optimal technique

Radiology Quality Map

Radiology Quality Map events and metrics—and many areas for possible PQI projects.[Source: Swenson SJ and Johnson CD, Radiologic Quality and Safety: Mapping Value Into Radiology. JACR 2005;2:992-1000.]

13

14

• There are opportunities for errors and improvements at each step.

• Let’s go through an actual example using the Quality Map.

Remember what Kelvin said - “You cannot improve things that you cannot measure”

14

15

Radiology Quality Map

Req = requisitionHere’s the story with more details: Ortho writes order for MTT joint injection-MTT is an uncommon site of injection, orthopod wrote it on paper (no POE—physician order entry), hard to read, and said “MTT” rather than “metatarsal-tarsal”-MTP is common injection; scheduler assumed MTP rather than MTT, and didn’t clarify with ordering MD, or ask radiologist-Exam protocolled by fellow, who didn’t check requisition (they were short staffed that day, and it was crazy busy) –we’ve all been there!-Neither the attending or the fellow had ever encountered an error like this (MTT/MTP)-Technologist suspected/thought to herself “I think it was supposed to be the MTT joint, but maybe I’m just confused”, Oh well, I’m sure they know what they are doing – I won’t say anything.-Time out quickly performed without original requisition-Patient positioned so that she was unable to see her foot. The patient also had diabetic neuropathy and could not feel which joint was being injected-Procedure performed, patient sat up, looked at where the band-aid was and said “Hmm, that’s not where the band-was last time!”-At this point, the original requisition was reviewed and the error was discovered.-The patient was told, the ordering physician was told and the correct joint was then injected. This was documented in the report and a root cause analysis was performed

15

16

Unclear acronym, uncommon jointUnclear acronym, uncommon joint

Protocol based on scheduling error rather than req

Protocol based on scheduling error rather than req

Original req not used to confirmOriginal req not used to confirm

Tech doesn’t speak up 

Tech doesn’t speak up 

Joint not confirmed with patient, patient couldn’t see/feel foot 

(DM)

Joint not confirmed with patient, patient couldn’t see/feel foot 

(DM)

Scheduler assumed MTT = MTP 

(should have clarified)

Scheduler assumed MTT = MTP 

(should have clarified)

Patient notices error‐‐Correct joint then injected

Patient notices error‐‐Correct joint then injected

Physician notified promptly of errorPhysician notified promptly of error

Short staffed, fellow hurrying to catch up

Short staffed, fellow hurrying to catch up

Radiology Quality Map

What are some of the contributing factors that caused this error to happen?

Take some time now and examine each step of this case and identify what areas of the system were faulty.If you need to, ask yourself, “Why did this happen?” And answer this question repeatedly, until you think you understand the underlying reason a mistake was possible at that step. (The 5 Whystechnique for uncovering the root cause.)

Remember, when you have identified the root cause it is rarely a person’s fault but rather the processes made it possible to make mistakes.

16

17

“Swiss Cheese Model”of system failure

Slice of cheese = step (or small circle) 

eg “time out”, scheduling, report creation.

Each step is checkpoint to catch errorsHoles = potential errors

James Reason's Swiss cheese model of system failure is another of way of conceptualizing the Radiology Quality Map and to show that multiple errors often line up for a significant error to occur. Each step (or small circle in the prior diagram) represents an opportunity for improvement so that, in the future, these errors can be caught at an earlier step.

17

18

“Swiss Cheese Model”of System Failure

• Slice of cheese = step (or small circle) Each step is checkpoint to catch errors• Holes = potential errors

If all the holes align = Significant defect in the system, potentially causing harm to the patient ACCIDENT

In our MTT joint example, multiple errors were made at multiple layers or steps and the wrong joint was injected = the holes lined up so that a series of preventable, small errors lead to an accident.

18

19

The Village: What went wrong?

• The villagers: – Ordering Physician

– Scheduler

– MSK fellow protocol

– Technologist

– Doctor performing time out and procedure

• The village (system)– Use of unfamiliar acronyms

– Culture of safety• Scheduler did not clarify

• Tech afraid to speak up

– Exam protocol

– Time out

At the steps where errors occur there is potential for a PQI project. This involves YOU—if you are the ordering provider, the scheduler, the fellow, the tech, etc….

19

20

Spell clearly, use familiar terms

Spell clearly, use familiar terms

Protocol should be based on original req

Protocol should be based on original req

Refer to original req in time outRefer to original req in time out

Culture of safety: ANYONE can speak up!

Culture of safety: ANYONE can speak up!

Confirm procedure, location with patientConfirm procedure, location with patient

Clarify with radiologist or ordering providerif unclear/unsure

Clarify with radiologist or ordering providerif unclear/unsure

Ensure there is a way to document QA

Ensure there is a way to document QA

Critical event should be verbal and documented

Critical event should be verbal and documented

Ordering provider aware of confusing 

language

Ordering provider aware of confusing 

language

Radiology Quality Map

Examine this circle and understand how many opportunities there are for improvement in just this one example.

Do these suggested preventative measures address the issues discovered in your root cause analysis?

20

21

PDSA Cycle

The PDSA cycle is a PQI model for identifying problems and systematically implementing a solution. Then evaluating the effectiveness of that solution.  This cycle is repeated.

E.g. Ortho orders 

MTTjoint injection

Quality map small circle eg, provider orders an exam PDSA cycle

21

22

PDSA Cycle

Identify a problem and design method for evaluating the 

problem

Identify a problem and design method for evaluating the 

problem

Carry out the plan: collect the data

Carry out the plan: collect the dataEvaluate the dataEvaluate the data

Decide what actions should be taken to solve the 

problem

Decide what actions should be taken to solve the 

problem

Here are the steps in the PDSA process explained.You’re rotating at a smaller hospital and you notice that the neck CTA are almost always hard interpretWhat can you do?Options

Do nothing, I probably can’t fix this To try evaluate and come up with a solution.

23

Example:

You notice that you are interpreting more and more chest‐abdomen radiographs on infants when it seems that the indications are often more appropriately addressed with chest only radiographs. For example, increased oxygen requirements.  You have just learned about the PQI process and have been looking for a PQI project (year one of the PQI timeline for MOC).  You think this might be a worthwhile project as this is potentially exposing infants to excess radiation.  

Additionally, your department may not be getting reimbursed for these exams.

Let’s take a look at an example and work through the process, step by step.

What are the next three steps in the PQI process for the MOC?

24

The Next Three Steps in the PQI Process for the MOC are:

1. Select metrics to measure2. Collect baseline data3. Analyze the data

After deciding on a problem to investigate, it is necessary to objectively define the severity of the problem with data, but first you need to decide what you are going to measure, using metrics. Second, you will collect the baseline data and third, analyze the data. 

What are some examples of metrics to measure in this example?

Select metrics to measure (1)Collect baseline data (2)Analyze the data (3)

After deciding on a problem to investigate, it is necessary to objectively define the severity of the problem with data, but first you need to decide what you are going to measure, using metrics.Second, you will collect the baseline data and third, analyze the data.

What are some examples of metrics to measure in this example?

25

Answer: You could plan to collect indications for chest/ abdomen radiographs and investigate how often chest/abdomen radiographs are performed when the indication is for chest alone, and under what circumstances.  Other things to study include who is the ordering physician, which technologist is performing the exam, what time the exam is performed, and the age of the patient or other demographics, like patient location.

What are some examples of metrics to measure in this example?

Answers could include: indication for exam, exam type, ordering physician, technologist, time of exam

26

1. Select metrics to measure

2. Collect baseline data

3. Analyze the data

4. ________________?

After you have collected and analyzed the data, what do you do next?

After you have collected and analyzed the data, what do you do next?1. Select metrics to measure2. Collect baseline data3. Analyze the data4. ______________

( Fill in the blank )

27

1. Select metrics to measure

2. Collect baseline data

3. Analyze the data

4. Work on improvement plan

Answer: From the data you have collected, target the source of the problem and come up with a solution to implement. 

After you have collected and analyzed the data, what do you do next?

After you have collected and analyzed the data, what you do next is Work on improvement plan.After implementing the solution, reapply the same metrics to see if the solution was effective.

28

Quality, Closing Thoughts:

– What kinds of recurring issues or errors have you seen?

– How were they handled?

– What is your idea for a PQI project?

– What are your concerns about implementing it?

– Other comments: 

To Claim CME and/or SAM credit for this module, click “Next” and answer the online questions. 

Take some time to reflect upon what you have learned.

Consider your answers to the questions above and make some notes about what you will do next.

29

• Gunderman RB and Burdick EJ. Error and Opportunity AJR 2007; 188:901‐903.

• Steele, JR. and Schomer, D.F. Continuous Quality Improvement Programs Provide New Opportunities to Drive Value Innovation Initiatives in Hospital‐Based Radiology Practices. JACR 2009;6:491‐499.

• Strife MOC white paper: Strife et al, The American Board of Radiology Perspective on Maintenance of Certification: Part IV—Practice Quality Improvement in Diagnostic Radiology. AJR 2007; 188:1‐4.

• Swenson SJ and Johnson CD, Radiologic Quality and Safety: Mapping Value Into Radiology. JACR 2005;2:992‐1000.

Reading List

Additional reading for more information.

30

Thank you, to the Valerie P. Jackson Fellowship award that made this presentation possible and to Jennifer and Christine for their work on this project.

.