CLEAR 2008 Annual Conference

Anchorage, Alaska

“Approaches to Continuing Competency – Challenges

and Opportunities”

Danielle Fagnan, Quebec Order of PharmacistsAshifa Keshavji, College of Pharmacists of British Columbia

Sandra Winkelbauer, Ontario College of PharmacistsModerator: Della Croteau, Ontario College of Pharmacists

Why Continuing Competency?

• public expectations

• increased accountability

• competency throughout practice

• governments now legislating

Three Different Approaches

• Description

• Outcomes and Challenges

• Future Directions

How does this compare to your program?

College of Pharmacists of British Columbia

Ashifa Keshavji

Objective

• To provide an overview of the College of Pharmacists of BC’s quality assurance program – the Professional Development and Assessment Program (PDAP)

• To share the challenges faced

Professional and Legal Mandates

Code of

Ethics

PDAP

FPPLegisl

a-tion

Program Purpose

• To support the college’s mission to ensure pharmacists provide safe and effective pharmacy care to help people achieve better health

• To promote continuous learning and professional development

Program Philosophy & Principles

• Uses the Framework of Professional Practice as the foundation

• Focus should be on professional development• Program must build on what pharmacists already do

in their practices• Built-in opportunity for reflection and self-assessment• Pharmacists must have an opportunity to demonstrate

their public accountability (essential to self-regulation)• Fair and valid assessments support the legislated

mandate of the profession • Evaluate the program on a regular and on-going basis

The Framework of Professional Practice (FPP)

• A comprehensive description of the work pharmacists do

• Based on the outcome of an occupational analysis, beginning with defining the purpose of the profession

• Validated by registrants• Serves as BC’s standards of practice• Foundation for PDAP and all other college

programs and services – including site visits/inspections

Professional Development and Assessment Program (PDAP):

Structure and timelines

• Six-year cycle– One-half the membership participates every 3

years– The entire membership participates over a 6-year

period• First cycle - September 2003• Second cycle - September 2006• Each cycle consists of three phases

PDAP: Phase 1 (September 2003-February 2005)

• Professional development and preliminary assessment

• Complete a Self-Assessment• Select one of two options:

– Knowledge Assessment– Learning and Practice Portfolio

PDAP: Phase 2 (September 2005 – August 2006)

• Professional development and secondary assessment

• Complete a self-assessment• Select one of four options:

– Knowledge Assessment– Learning and Practice Portfolio– Practice Audit– Objective Structured Clinical Examination

(OSCE)

PDAP: Phase 3(Began September 2006)

• Individualized remediation and reassessment

• Peer Practice Committee and college staff review assessment results

• Work with participant to develop an individualized remediation plan and define re-assessment requirements

Self-Assessment

• Based on the Framework of Professional Practice

• Compare one’s own standard of practice to those reflected in the FPP

• Identify strengths and limitations and map out a professional development plan

The Knowledge Assessment…

• Linked to the Framework of Professional Practice• Standardized MCQ exam

– 2&1/2 hours, 65 questions, open book• Covers clinical decision making and ethical

issues in therapeutics, pharmacology and pharmaceutics

• Scoring is criterion referenced• Feedback received:

– Standards met/standards not met– Domain and disease states answered correctly or

incorrectly

Learning and Practice Portfolio…

• Linked to the Framework of Professional Practice• Individualized to reflect one’s needs, PD

preferences and practice priorities• A compilation of how one has planned,

implemented and evaluated their learning and practice achievements

• Feedback received:– Standards met/standards not met– Based on the published criteria– Individualized based on content and nature of the

learning and practice achievements

Practice Audit

• On-site process where a trained peer auditor observes performance based on FPP

• Includes follow-up interview, if required• Feedback includes:

– Standards met/standards not met– Individualized report and overall summary of

performance by FPP Role, on areas of strength and those needing improvement

OSCE

• Practical examination that assesses communication, problem-solving skills and overall performance

• Series of 7-minute simulated tasks commonly encountered in practice

• Feedback includes:– Standards met/standards not met– Individual average score in communication,

outcome and overall performance– Overall score in competency areas– Comparison to group average

Phase 1, Phase 2 Participants

104 (87%)82 (48%)1258 (91%)Total

15 (80%)-OSCE

-PA

2 (100%)198 (90%)LPP

60 (83%)56 (36%)1060 (91%)KA

VolunteerPhase 2Phase 1Tool

Program Evaluation

Total

1 (0%)-OSCE

27 (96%)10 (60%)-PA

2 (100%)15 (80%)LPP

KA

VolunteerPhase 2Phase 1Tool

Program Evaluation

Challenges

• Emphasize each tool follows the CPD cycle

• Encourage participants to engage in an on-going PD cycle

• Confirm the validity of the assessment tools in meeting the program purpose

• Explore the motivational factors by which assessment tools are selected

• Keep communicating with and engaging registrants

Ontario College of Pharmacists

Sandra Winkelbauer

OCP Program Description

Quality Assurance

Continuing Competency Department

Pharmacy Practice Department

1. Learning Portfolio 4. Pharmacy

Inspections

2. Self-Assessment

3. Peer Review and

Remediation

OCP Program Description

• A Quality Assurance program is mandated for all regulated health professions in Ontario

• Based on Standards of Practice

• Two-part Register– Part A – patient care– Part B – no patient care

Learning Portfolio

• All Ontario pharmacists, Part A and Part B, required to maintain a learning portfolio

• On-line tool available

• Tool helps to plan and document CE activities

• Non-traditional learning activities are recognized

• Submitted upon request

Self-Assessment Tool

• 20 % of Part A pharmacists each year (100 % of Part A pharmacists over 5 years)

• On-line tool

• Tool helps to identify learning needs and plan learning

• Includes clinical knowledge assessment

• Information is confidential

Peer Review

• ~ 2.3 % of Part A pharmacists each year

• Randomly selected

• Also for movement from Part B to Part A

• Consists of:– clinical knowledge assessment– standardized patient interviews– learning portfolio sharing session

Peer Review

• Components assessed:– Clinical knowledge– Gathering information– Patient management and follow-up– Communication skills

Peer Review - Validity

• Clinical knowledge assessment cases – 3 different committees of practicing

pharmacists write, review and set standards for each case

• Standardized patient scenarios – 3 different committees of practicing

pharmacists write, review and set standards for each scenario

Peer Review - Remediation

• Professional Development Advisor

• Peer Support Group

• Professional Development Workshop

Outcomes

• Quality Assurance program has been in place for 11 years

• 10 year review conducted last year

Outcomes – Place of Graduation

Peer Review Participants (n=1938)

Ontario, 53%Canada /

USA, 24%

International, 23%

Pharmacists in Part A (n=10,378)

Ontario, 48%

Canada / USA, 23%

International, 29%

Outcomes – Place of Practice

79.40%73.50%

16.60%16.50%

4%10%

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

Community Hospital Other

Place of Practice

Peer Review Participants Pharmacists in Part A

Outcomes - Results

Peer Review Results 1997-2007 (n=2013)

Peer-Guided, 12%

Self-Directed, 88%

Outcomes – Years since Graduation

1.10%

98.90%

4.40%

95.60%

8.10%

91.90%

25.10%

74.90%

0 - 5 6 15 16 - 25 > 25

Peer Review Results Based on Years Since Graduation (1997 - 2007)

Peer-Guided Self-Directed

Outcomes – Place of Graduation

10%

90%

5%

94%

23%

77%

Ontario Canada / USA International

Peer Review Results Based on Place of Graduation (1997 - 2007)

Peer-Guided Self-Directed

Challenges

• Resistance from members

• Not assessing actual practice

• Defining the number of pharmacists that should ideally undergo peer review

• Cost

• Difficult remediation cases

• Need to create QA program for pharmacy technicians

Future Directions

• Evaluation and enhancement of the on-line Self-Assessment and Learning Portfolio

• Evaluation of Peer Review process addressing:– Affect of changing scope of practice– Numbers chosen for quality assurance– Effectiveness versus other processes

• QA program for pharmacy technicians

Speaker Contact Information

Sandra Winkelbauer

Manager, Continuing Competency

Ontario College of Pharmacists

swinkelbauer@ocpinfo.com

416-847-8204

(insert OCP logo)

Quebec Order of Pharmacists

Danielle Fagnan

Mandatory statute of professional inspection in Quebec

• Supervise professional practice

• Inquire into the professional competence of any member of the Order

Pharmacy in Quebec

• 7262 members (62,9% F – 37,1% M)• 1669 community pharmacies (1690 owned, 3354

pharmacists)• 128 health care facilities (1316 pharmacists)• 158 pharmacists (replacement)• 744 pharmacists (others)• 4 inspectors for community pharmacies• 2 inspectors for health care communities

Professional inspection in Quebec

• General assessment of practice sites and of pharmacists if needed

• Evaluation according to specific indicators based on a minimum standard of quality practice

• Self-assessment/pre-inspection questionnaire filled by each pharmacist

Evaluation in community pharmacies

• Professional practice (technical aspect)– Staff– Workload – Work organization– Hygiene– Equipment

Evaluation in community pharmacies

• Respect of laws and regulations– Standards of practice– Labeling– Outdated products– Confidentiality– Ethics, etc.

Evaluation in community pharmacies

• Evaluation of Clinical aspect

• Assessment based on clinical indicators reflecting minimum standards of patient’s quality of care and pharmaceutical services (indicators related to antibiotic therapy, asthma, hormonal contraception, depression, diabetes, dyslipidemia, hypertension, hormonal replacement, insomnia, inflammatory diseases and palliative care)

• Data collection from 25 patients’ chart

Evaluation in community pharmacies

• Self-assessment• On site assessment (data collection, observation,

coaching and recommendations)• Evaluation report presented to our professional

inspection committee• Discussion on general recommendations and official

agreement on identified problems requiring more aggressive interventions such as follow up with written engagement from the owner, control inspection or information to Syndic

Evaluation in health care facilities

• Policies and procedures

• Medication distribution and control

• Sterile products and chemotherapy preparation

• Services and clinical programs

• Medication and clinical studies

• Human and material resources

• Quality control

Evaluation in health care facilities

• Clinical evaluation according to specific indicators of quality practice based on a level II of pharmaceutical care (selective monitoring of pharmacotherapy)– > 65 y. o– >7 drugs– Narrow therapeutic index– Major interactions– Duplication– Pertinence and clarity of communications with other

health care professionals

Assessment of individual competency

• Request from Committee of discipline, Executive Committee/Bureau, Professional inspection Committee

• Following a professional inspection• Thoroughly regulated process• Decisions with important consequences• Validity and reliability of tools• SOI (structured oral interview) used as a

diagnostic tool• Individualized pedagogic prescription and

remedial (course, training, limitation of practice)

Challenges

• 5-year cycle for inspection• More general inspection of sites and less

pharmacists’ individual assessment • Evaluation according to specific minimum

standards of quality practice• Regular updates of our standards of

practice, practice guides, publications of professional information and clinical cases for SOI

Challenges

Urgent need to rethink our processes and develop a

strategic plan

Future

• Use benchmarks/best practices/pilot projects to set up and update standards of practice (pro activity and improvement of practice)

• Be able to target practice sites and individual pharmacists with problems (efficacy and efficiency)

• Develop different mechanisms to be able to evaluate our membership in order to sustain a 5-year cycle

• Link with self-assessment, CPD and prevention activities

• Develop strategies to address overall performance and the relative influence of these determinants

Speaker Contact Information

Danielle Fagnan

Director of professional services

Quebec Order of pharmacists

dfagnan@opq.org

QUESTIONS?

Does anyone have any suggestions for improvement of these programs?

Do you think public safety is protected by such a program?

What are similar issues in your profession, and how have you approached quality assurance?