Instruments Used to Appraise the Performance of Hospital Pharmacists:

Discussing the Pharmacists’ Skills Development Grid

Prepared by: Jason Yung (APPE Pharmacy Student)Date: July 14, 2016

https://www.youtube.com/watch?v=OWa5rzEOumQ

Learning ObjectivesBy the end of this presentation, you will be able to:1. Describe the utility and development of the

Pharmacists’ Skills Development Grid (PSDG) at SickKids

2. Summarize development within the 8 domains of pharmacy practice, highlighted in the PSDG

3. Identify advantages, limitations, and future directions of the PSDG

4. Compare the PSDG to other tools used to appraise the performance of pharmacists

Summary of Abbreviations

• PSDG, Pharmacists’ Skills Development Grid• ASHP, American Society of Hospital Pharmacists• HCP, health care providers• DRP, drug-related problem• DI, drug information• TDM, therapeutic drug monitoring• CPD, continuous professional development• CSHP, Canadian Society of Hospital Pharmacists• OCP, Ontario College of Pharmacists• NAPRA, National Association of Pharmacy

Regulatory Authorities• PK, pharmacokinetics• P&P, Policy & Procedure• hospital M&V, hospital mission & vision• CQI, continuous quality improvement

• PTW, Pharmacy This Week• PSN, Pharmacy Specialty Networks• DTPs, drug therapy problems• f/u, follow up• CPG, clinical practice guideline• MedRec, medication reconciliation• TPN, total parenteral nutrition• QI, quality improvement• DUE, drug use evaluation• IPA, International Pharmaceutical Abstracts• ACCP, American College of Clinical Pharmacy• CoDEG, Competency Development & Evaluation Group• APhA, American Pharmacists Association• BPMH, Best Possible Medication History• GLF, General Level Framework• SHPA, Society of Hospital Pharmacists of Australia• SingHealth, Singapore Health Services

Outline of the Presentation

1. The Adoption of Pharmaceutical Care at SickKids

2. Overview and Utility of the Current PSDG3. The Search for Other Appraisal Tools for

Clinical Pharmacists4. Advantages and Disadvantages of the PSDG

compared to other appraisal tools5. Future Steps (Lessons Learned)

The Adoption of Pharmaceutical Care at SickKids

What is Pharmaceutical Care?

“the direct, responsible provision of medication-related care for the purpose of achieving definite outcomes that improve a patient’s quality of life”

~ ASHP

Pharmacists cooperate with patients and HCPs:• Identify potential and

actual DRPs;• Resolve actual DRPs; and• Prevent DRPs http://goodhomepharmacy.com/wp/

In what year did SickKids’ Department of Pharmacy implement the pharmaceutical

care model as a practice philosophy?

A. 1986B. 1992C. 1998D. 2004

Pharmaceutical Care was not always practiced at SickKids….

Evolution of Pharmacy Practice at SickKids

1992 After 1992

Consultant servicesPatient rounds

Clinical Responsibilities

(e.g., DI questions, P&Ps, TDM)

Changes in job expectations

Need for CPD document

Adopted “Pharmaceutical

Care” Model

Before 1992

Evolution of the PSDG: 1998 to 2012

1998

First version of PSDG developed

Standards of Practice:- SickKids

- CSHP, ASHP- OCP, NAPRA

Reviewed by pharmacists

Minor Revision

Minor Revision MAJOR Revision (Task Force) & PSDG Survey

Clinical Council Formed &Work Plan Prioritization

Survey

2000 2002/2003 2011 2012

2012 Feedback Survey Results (n=12)

Comments: not available

Overview and Utility of the Current PSDG

Overview of the Current PSDG

• 8 domains of pharmacy practice• Progress: Novice (Level 1) Mastery (Level 3/4)• Linked with Job Standards Timeline– Performance milestones --- Work experience

Level 1 Level 2 Level 3 Level 4

3 months 1 year 2 years ≥ 5 years

8 Domains of Pharmacy Practice

1. Administration 5. PK/TDM

2. Distribution 6. Production

3. Drug Information 7. Research & Projects

4. Pharmaceutical Care 8. Teaching

1. Administration

1. Administration (Levels 1-4)

Aware of legislation, practice standards, P&Ps, hospital M&V

Participate in CQI initiatives, committees, task forces

Expert resource for CQI & advancing pharmacy practice internally/externally

http://www.parents.com/baby/development/crawling/the-magic-of-crawling/ https://www.pinterest.com/pin/367465650828116560/

2. Distribution

2. Distribution (Levels 1-3)

- Distribution system and work flow, P&Ps- Identify and report medication incidents

- Drug supply issues- Report and handle all medication incidents & PTW- Reduce dispensing costs - Dispensing references

and guidelines- Improve order entry (program changes)- Role model and resource for other staff

3. Drug Information

3. Drug Information (Levels 1-4)- Urgency of question, refer complex Qs- Tertiary references, simple questions

- Info from multiple sources- Appraise primary literature, more complex questions- Tailored responses - Advanced literature

evaluation- Consolidate complex, controversial info into concise answers- Networking, PSNs

4. Pharmaceutical Care

4. Pharmaceutical Care (Levels 1-4)4A. Therapeutic Thought Process

• Retroactively identify DTPs• Develops care plan to

resolves DTPs• Basic monitoring and f/u for

all medications• Aware of CPGs

• Proactively prevent DTPs• Apply advanced knowledge (and

literature search) to address complex DTPs• Influence standards of practice• Develop P&Ps/CPGs/order sets

for subspecialty

4. Pharmaceutical Care (Levels 1-4)

• Collaborate at rounds• Basic interviewing and

counseling skills• Perform MedRec• Document interventions, key

patient info

• Discuss complicated/sensitive issues with patients/caregivers• Resource for others on

therapeutic issues• Resource and role model in

complex topics

4B. Communication

5. PK/TDM

5. PK/TDM (Levels 1-4)

• Design complex regimens for patients with organ impairment, on renal replacement therapy

• TDM consult on weekends, develop/revise TDM P&Ps

• Understand basic PK principles (impact of age)

• Recommend dose adjustments and sampling times

• TDM certification

6. Production

6. Production (Levels 1-3)Production = IV compounding, TPN, chemotherapy, hazardous drugs

• Follow P&Ps for IV preparations

• Complete worksheets for preparing IV and compounding special products

• Resource for addressing production issues

• Develop P&Ps for production areas

7. Research & Projects

7. Research & Projects (Levels 1-3)

- greater contribution in research (e.g. co-investigator in pharmacy residency project)

- conduct literature review, publish in peer-reviewed journal- function as PI or co-investigator

- participate in projects that meet departmental needs (QI, DUE), data collection, organization of project

8. Teaching

8. Teaching (Levels 1-3)

• Provide educational sessions in the hospital & at conferences/meetings

• Preceptor for EPE/APPE students

• Present at universities• Preceptor for residents &

mentor to newly hired pharmacists

Utility of the PSDG at SickKids

• Outlines standards of pharmacy practice

• Offers a quality assurance for pharmacy practice

• Standardizes performance assessments of pharmacists among clinical managers

http://entropyte.com/qhse/quality-assurance-policy.htm

Utility of the PSDG at SickKids

• Guides CPD (new pharmacists)

• Identifies developmental needs of pharmacists– Form objectives, action

plan & training programs• Clear expectations from

inter-disciplinary team

http://entropyte.com/qhse/quality-assurance-policy.htm

Is SickKids’ PSDG the only available appraisal tool for hospital pharmacists?

The Search for Other Appraisal Tools for Clinical Pharmacists

The Search for Other Appraisal Tools for Clinical Pharmacists

Literature Search Search Terms

• MEDLINE• EMBASE• PubMed• IPA• Google/Google Scholar

• Competence or Competency

• Performance• Clinical• Pharmacists• Assess or evaluate• Tool

Titles and/or abstracts were screened to identify pertinent literature

Literature Search Findings

There is a lack of instruments used to evaluate performance of hospital pharmacists!

Three unique tools were identified:1. ACCP (1993) – “Template for the Evaluation of the

Clinical Pharmacist”2. CoDEG (2007) – “General Level Framework (GLF)”3. APhA (2009) – “Pharmacist Clinical Services

Performance Evaluation”

CoDEG = Competency Development & Evaluation Group

1. ACCP (1993) – “Template for the Evaluation of the Clinical Pharmacist”

** Geared towards evaluation of hospital pharmacists

8 Clusters of Competencies:1. Perception of the need for a drug2. Selection of a specific drug3. Evaluation and review of drug regimen4. Monitoring effects of drug therapy5. Education6. Evaluation of drug usage and therapy7. Information retrieval 8. Committee Involvement9. Miscellaneous activities (e.g., participation in pharmacy

organizations)

1. ACCP (1993) – “Template for the Evaluation of the Clinical Pharmacist”

• Performance rating based on whether the criteria are met or not

• Provides an assessment method for each competency– E.g., review of selected monitoring forms or

medical charts

Critique of ACCP 1993Advantages Disadvantages• Provides method of

assessment• Hospital-specific e.g.,

includes research component and committee involvement

• Time consuming for clinical manager (e.g., joins rounds, reviews medical charts, documentation, etc.)

• Outdated: BPMH, MedRec• Lacks mention of teaching,

mentorship• No accompanying timeline

2. CoDEG (2007) – “General Level Framework (GLF)”Evaluates hospital, community, and primary care pharmacists on 15

Competencies within 3 Clusters

1. Delivery of Patient Care

2. Personal Attributes 3. Problem-Solving

• Patient consultation• Need for drug• Selection of drug• Drug-specific issues• Provision of drug

product• Medicines

information and patient education

• Monitoring drug therapy

• Organization• Effective

communication skills• Teamwork• Professionalism

• Gathering information

• Knowledge• Analyzing

information• Providing

information

2. CoDEG (2007) – “General Level Framework (GLF)”

Performance rating based on consistency of achieving competency:• Consistently (85-100%)• Usually (51-84%)• Sometimes (25-50%)• Rarely (0-24%)• Unable to comment

2. CoDEG (2007) – “General Level Framework (GLF)”

• Validated by McRobbie et al. (2001) and Antoniou et al. (2005) in appraising junior pharmacists accelerate CPD

• Adapted by organizations in…– Australia (SHPA, 2013; Coombes, 2010)– Singapore (SingHealth, 2011)

• SingHealth adapted tool provides timeline for when competencies should be performed at a given frequency

Critique of CoDEGAdvantages Disadvantages• Validated tool (accelerates

CPD)• Assesses consistency of

performance• Multiple assessments (4) in 1

year (self and facilitated)

• Time consuming for assessor (4 assessments/year)

• Lengthy (50 pgs + comments)• Consistency of performance

can be subjective• Not specific to hospital

pharmacy – lacks PK/TDM, research, production, teaching, MedRec

• No accompanying timeline

3. APhA (2009) – “Pharmacist Clinical Services Performance Evaluation”

Evaluate pharmacists who provide “medication therapy management and other clinical services”

• Section 1: Five Core Competencies– Patient Care– Practice Management– Evaluation of Patient Care Programs/Projects– Mentorship/Precepting– Professional Outreach

• Section 2: Professional Goals and Development– Develop up to 5 goals and objectives

3. APhA (2009) – “Pharmacist Clinical Services Performance Evaluation”

• Performance rating from 1-5– 1 = fails to achieve expectations– 3 = consistently meets expectations– 5 = consistently surpasses expectations

• Overall performance rating (average score)– >4.6: consistently surpasses expectations– 2.9-3.5: consistently meets expectations– <1.7: fails to achieve expectations

Critique of APhA

Advantages Disadvantages• Numerically assesses

consistency of performance

• Overall performance rating• Prompts development of

goals and objectives

• Not specific to hospital pharmacy (more community focused) – lacks PK/TDM, research, production, MedRec

• Subjective rating (cf. CoDEG)

• No accompanying timeline

Advantages and Disadvantages of the PSDG compared to other appraisal tools

Advantages of the PSDG at SickKids

• More specific to hospital practice (vs. APhA, CoDEG)– Includes PK/TDM, Research & Projects domains

• Provides a job standards timeline guides CPD and forming performance objectives

• Key: fosters CPD (timeline) vs. consistent performance

Limitations of the PSDG at SickKids

• Key: Lack of validation (vs. GLF)• Inconsistent uptake by clinical managers and

shortage of feedback from pharmacists• Lack of advanced practice standards for senior

pharmacists• Does not explicitly assess consistency of

performance

Future Steps (Lessons Learned)

Future Steps (Lessons Learned)• Tool validation (improving performance)– Prospective evaluation (baseline & repeat

measures)– Consider rating performance of each evaluated

competency on scale of 1-5 (see CoDEG, APhA)– Compare overall performance rating (see APhA)

over time

http://jaykumarlimbasiya.blogspot.ca/2012/09/steps-to-success_8.html

Future Steps (Lessons Learned)• Survey SickKids’ pharmacists – feedback– Tool’s utility and uptake– Improvement strategies (e.g., performance

objectives)• Engage a wider constituency of pharmacists

(i.e., from across Ontario/Canada) for input

http://jaykumarlimbasiya.blogspot.ca/2012/09/steps-to-success_8.html

Acknowledgements

• James Tjon RPh, BSPhm, PharmD• Lee Dupuis RPh, ACPR, PhD, FCSHP • Jaspreet Shokar, PharmD

References1. American Society of Hospital Pharmacists. ASHP Statement on Pharmaceutical Cate. Available

online: http://www.ashp.org/doclibrary/bestpractices/orgstpharmcare.aspx2. American College of Clinical Pharmacy. Template for the evaluation of a clinical pharmacist.

Pharmacotherapy 1993; 13: 661-667. 3. Competency Development and Education Group. General level framework – a framework for

pharmacist development in general practice. 2nd ed. 2007. Available from: www.codeg.org/fileadmin/codeg/pdf/glf/GLF_October_2007_ Edition.pdf.

4. McRobbie D, Webb DG, Bates I et al. "Assessment of clinical competence: designing a competence grid for junior pharmacists." Pharmacy Education 2001; 1: 67-76.

5. Antoniou S, Webb DG, McRobbie D et al. "A controlled study of the general level framework: Results of the South of England competency study." Pharmacy Education 5 (2005).

6. American Pharmacists Association. Pharmacist clinical services performance evaluation tool. Copyright 2009.

7. Society of Hospital Pharmacists of Australia (SHPA). Clinical competency assessment tool (shpaclinCAT version 2). In: SHPA Committee of Specialty Practice in Clinical Pharmacy. SHPA standards of practice for clinical pharmacy services. J Pharm Pract Res 2013; 43 (suppl): S50-S67.

8. Coombes I, Avent M, Cardiff L et al. Improvement in pharmacist’s performance facilitated by an adapted competency-based general level framework. Journal of Pharmacy Practice and Research 2010; 40: 111-118.

9. Singapore Health Services (SingHealth). A competency framework for SingHealth pharmacists to provide minimum standard of general pharmacy practice: The general level framework handbook. 1st ed. 2011. Available from: http://www.codeg.org/fileadmin/codeg/pdf/singapore/SingHealth_GLF_Handbook_1st_Edition_Feb2011.pdf

- Thank You –

Questions?