WELCOME TO THE 4th EDITION OF THE NAC PRA NEWSLET TER

This annual publication brings you news and views on the National Assessment Collaboration’s practice ready assessment (PRA) initiative and updates on the implementation of pan-Canadian PRA.

FEBRUARY

2017

click interactive icons for additional information [double tap if on a mobile device]

This annual publication brings you news and views on the National Assessment Collaboration’s practice ready assessment (PRA) initiative and updates on the implementation of pan-Canadian PRA.

Therapeutic decision-making exam

2016: Year in review

8 12

AR

TIC

LE

S14

2

Pan-Canadian PRA fee

CSAT program profile

16

PROFILE:Dr. Deborah Obu

NATIONAL ASSESSMENT COLLABORATION PRACTICE READY ASSESSMENT

NATIONAL ASSESSMENT COLLABORATION PRACTICE READY ASSESSMENT

NATIONAL ASSESSMENT COLLABORATION PRACTICE READY ASSESSMENT

NATIONAL ASSESSMENT COLLABORATION PRACTICE READY ASSESSMENT

NAC PRA Family Medicine committee meeting VANCOUVER BC

Nat

iona

l

A

sses

smen

t C

olla

bor

atio

n

ONLINE MODULES: Communication and cultural competence The Communication and Cultural Competence online program helps international medical graduates (IMGs) familiarize themselves with cultural issues in Canadian medical settings. It was launched in September 2015 as part of the pan-Canadian PRA initiative. Although it was developed for IMGs, it can also be used by Canadian medical graduates, as it is on an open website, physiciansapply.ca. The program launched with five modules, and a sixth involving care of the elderly was added in June 2016. A seventh module on

TR AINING PROGR AM FOR CLINIC AL ASSESSORS

A training program for clinical assessors is being developed and will be available for use by provincial PRA programs in mid- to late 2017. The content for Family Medicine assessors was developed in 2015 by CFPC, and as of December 2016 content for Psychiatry and Internal Medicine was in the final stages of development. While many PRA programs already have training programs for assessors, these tend to focus on medical content, whereas this training focuses on the non-medical expert roles that assessors play. Based on feedback from PRA programs, the following training needs were identified:

• Understanding the competencies to be assessed and observed

• Using the assessment tools• Documenting meaningful single observations that

can be combined to enable a summative practice- ready decision

• Providing feedback

The program involves a full day of face-to-face training. For Family Medicine assessors, this has been divided into eight modules dealing with specific skills. There is also some pre-training preparation and a post-training survey and feedback. The training is intended to be integrated into existing provincial programs.

32 3

2016: Building

foundations for the NAC PRA initiative

The last year saw the National Assessment Collaboration (NAC) set the foundations for full implementation of the pan-Canadian practice ready assessment (PRA) initiative, scheduled for 2018.

Family medicine group meets in December 2016.

VANCOUVER BC

Nat

iona

l

A

sses

smen

t C

olla

bor

atio

n

. . . c o n t i n u e d

Building foundations for the

NAC PRA initiative professionalism, ethics, legal and organizational issues is planned for 2017. Segments of this module will deal with caring for transgender patients and privacy breaches.

EXAMS TO SELEC T C ANDIDATES FOR PR A PROGR AMS

Many provincial PRA programs use point-in-time exams to select candidates for subsequent over-time assessments. Among these point-in-time exams are therapeutic decision-making exams,

see “Therapeutic decision-making: a tool to predict practice-readiness”

knowledge exams and objective structured clinical exams (OSCEs). Some PRA programs have developed their own exams, although similar exams sometimes exist or are being developed at the national level. Cindy Streefkerk, Chief of Strategic Development with the Medical Council of Canada (MCC) who is leading the NAC PRA project, explained, “We have done research as part of the NAC PRA project into how PRA programs could use the MCC Evaluating Examination, MCC Qualifying Examination Part I, and NAC Exam (OSCE) to replace their own individually developed and managed exams.”

In 2016, the MCC developed guidelines for use of these MCC exams as well as the NAC Examination, an OSCE-based examination developed for IMGs, to select candidates for over-time PRA assessments. “It is costly to develop new tests,” said Ms. Streefkerk. If PRA programs use existing assessment tools, economies of scale will make testing less expensive overall.

54 5

Small-group session at the Family Medicine meeting in Dec. 2016 VANCOUVER BC

Nat

iona

l

A

sses

smen

t C

olla

bor

atio

n

PROGR AM E VALUATION

A survey conducted in 2016 gathered perspectives on how PRA is functioning from assessors and candidates who have been involved in PRA. Ms. Streefkerk explained that the evaluation of PRA is a pilot. “It’s really about continuous improvement and transparency.” The plan is to conduct a program review periodically and use the feedback to make refinements to both PRA programs and the pan-Canadian model. Some results from this first survey are expected to be available in the first quarter of 2017. There will be a report for each PRA program as well as a report on what needs to be addressed at the pan-Canadian level.

RESEARCH PROJEC T

Are PRA programs assessing what they should be? What predicts which physicians will perform well? These are some of the questions being considered for a research study to look at IMGs’ success in PRA and in later performance and practice. This research would require long-term follow-up to track outcomes among PRA candidates. Planning for such a research project is currently at the exploratory stage.

76

. . . c o n t i n u e d

Building foundations for the

NAC PRA initiative

D R . D E B O R A H

O B U

9

P

ROFI

LE:

DR.

DEB

ORA

H O

BU

Nat

iona

l

A

sses

smen

t C

olla

bor

atio

n

“

“

98

There are many reasons Dr. Deborah Obu is thrilled to be practising in Ashcroft, BC. She mentions the “phenomenal” weather, the welcoming community, and the fact that her three children can run around outdoors in safety. But it’s the work satisfaction that she raves about.

“We have an opportunity to be a jack-of-all-trades here,” she says. She is currently one of only two physicians staffing the hospital in this town of 1,628*, which also serves other communities in the area. She loves the experience gained in rural medicine, saying, “The joy is there as well as the job satisfaction.” She now encourages residents she meets at conferences to try it.

Dr. Obu is one of the successful family physician candidates assessed through Practice Ready Assessment (PRA)-BC, a relatively new PRA program, which started assessing family practice international medical graduates (IMGs) in 2015. To date, 39 candidates have successfully completed the program and are now working in rural communities throughout BC, according to Program Coordinator Mary Chinni. A group of 17 candidates currently undergoing clinical field assessment will join that number soon.

PRA-BC’s candidates have come from 14 different countries, of which the most common points of origin are South Africa, Nigeria, Iraq and Egypt.

Dr. Obu was born in England, but her parents are from Nigeria, where she attended medical school. She later trained and worked in Dublin, Ireland, where her husband was also a physician. They decided to try life in Canada and learned about opportunities through Health Match BC, a provincially funded free recruitment service for health-care professionals.

* 2011 census

Upon coming to Canada, Dr. Obu applied for PRA-BC. The first step was multiple-choice, therapeutic decision-making (TDM) and objective structured clinical exams (OSCE). The program’s Clinical Director, Dr. Jack Burak, explains that this centralized assessment component has an 87 per cent pass rate to date. This is confirmed by Dr. Obu’s experience.

At first, PRA-BC used its own exams for these initial assessments, but in 2016 it brought in the Medical Council of Canada Evaluating Examination (MCCEE) and the new PRA TDM exam. It plans to introduce the NAC OSCE in 2018 and replace the MCCEE with the Medical Council of Canada Qualifying Examination (MCCQE) Part I in 2019.

Following the centralized assessment, Dr. Obu attended orientation sessions on working in BC, given by physicians and allied medical professionals “who are very experienced and have the inside view of practising in BC.” She found the orientation intense: “It was a good eye-opener.”

PRA puts you on a par with

your Canadian colleagues. You are expected to

know as much as a Canadian

-trained physician.

International medical graduate happy to be practising in Canada

“THE JOY IS THERE”

P

ROFI

LE:

DR.

DEB

ORA

H O

BU

Then it was on to clinical field assessment. Dr. Burak notes that this assessment has a 100 per cent pass rate to date, “but I don’t want to give the impression that we will have that rate forever.”

“Whatever concerns any of us had were addressed,” says Dr. Obu of the PRA-BC program. “Although intense, I really enjoyed it.” She believes that PRA programs are a good way to help IMGs bridge into Canadian practice. “It puts you on a par with your Canadian colleagues. At the end of the day, you are expected to know as much as a Canadian-trained physician.”

“You hear about international physicians in Canada working as security guards or driving taxis. It really saddens me.”

However, in BC, although that story is relatively common, the rate of IMGs in practice is very high, Dr. Burak said. IMGs account for 28 to 29 per cent of all BC registered physicians. They are urgently needed, as nearly half of the physician

workforce is over age 55, and retirements are increasing. As in other provinces and territories, staffing in rural and remote communities is also a gap that PRA is filling.

Dr. Obu chose Ashcroft in the BC interior about 100 kilometres west of Kamloops, partly because she preferred a warm, dry climate after spending many years in rainy Ireland. However, she also wanted the opportunity to work in the emergency department of the hospital, as she has worked in emergency medicine in Ireland.

“I have heard nothing but very positive comments from every community that has received one of our PRA-BC candidates who have successfully passed,” said Dr. Burak. “They are ecstatic to have new physicians in their community providing essential primary care to the community.”

Dr. Karolina MaslowskaINVERMERE, BC

1110 11

PRA-BC physicians featured prominently in their communities.

CLICK to access the websites

Drs. Fola and Abbie Olajide

FORT ST. JOHN, BC

Dr. Michael AbayomiENDERBY, BC

Dr. Shola SolomonTUMBLER RIDGE, BC

Therapeutic decision-making exam: a tool to predict practice -readiness

D R . J O N W I T T

“

“We’re finding the TDM exam

an excellent tool to predict

practice-readiness and success in the

clinical field assessment.

D R . S y d n e y S m e e

“

“These are all efforts that would

be extremely demanding on

one program.Why should each

program do it when a collaboration

can help share the workload?

A therapeutic decision-making (TDM) examination is being piloted during 2016 –17 as a useful tool to select candidates for practice ready assessment (PRA). The exam is the result of close collaboration between the Medical Council of Canada (MCC) and four provincial programs interested in using the TDM exam — Manitoba, Saskatchewan, Alberta and British Columbia, said Dr. Sydney Smee, Assessment Advisor at the MCC. “The MCC is providing central coordination and leadership, but the energy is coming from the programs,” she said.

The exam is being piloted in the Saskatchewan International Physician Practice Assessment (SIPPA) program, replacing a previous TDM exam that SIPPA had used since 2011, said SIPPA’s Medical Director, Dr. Jon Witt.

Like several other provincial PRA programs, SIPPA’s PRA pathway involves a centralized assessment before a clinical field assessment. In the centralized assessment, family medicine candidates undergo an orientation and must pass two exams: an objective structured clinical exam and the TDM exam.

Why screen candidates before the clinical field assessment? Dr. Witt said that clinical field assessment is “an intensive process.” International medical graduates who have another job must take off the six or 12 weeks required. “We want to make sure that the people coming into the clinical field assessment are truly ready.”

He has found that the TDM exam accurately predicts which candidates will succeed in the field assessment. A total of 62% of candidates passed the updated TDM exam, which has been used in three SIPPA centralized assessments in 2016, and those candidates have had a success rate of 96% in the clinical field assessment. A fourth centralized assessment is planned for January 2017.

Development of the TDM exam was led by a working group that first met in the fall of 2015, said Dr. Smee. The four provincial programs donated their existing item banks for TDM exams to the National Assessment Collaboration. The working group reviewed existing content — archiving some questions, updating others, eliminating duplicates and adding new content, said Dr. Smee. In addition to setting up this centralized item bank, MCC has provided support for the Test Committee and for standard setting.

The Test Committee members are clinicians and a pharmacist who use an iterative process to add new content, review existing content, and gather feedback from exam markers. In the fall of 2016, the Committee reviewed the test questions for 2017, so content will be ready in advance.

The standard setting process is separate and independent, said Dr. Smee. “It uses an agreed upon methodology leading to more defensible standards and pass marks for all of the exams.” Commented Dr. Witt, “It’s been just wonderful support from the MCC,” which has created “a high-quality, reliable and valid exam.”

Dr. Witt said the TDM exam content helps make candidates aware of the therapeutics environment in Canada. “This isn’t just about writing a prescription,” as the exam includes non-medicinal therapy or alternatives to pharmacologic agents such as physiotherapy and lifestyle change. As well, questions about drug therapy include contraindications, monitoring and follow-up. “The exam content is a good mix of acute, chronic and long-term care, appropriate for the Canadian context.”

The exam will move from the pilot phase to implementation in January 2018. Dr. Witt has become its biggest booster: “I would like to see it eventually become a national exam, similar to the other exams offered.” 1312

Warmly welcomed, IMGs come a long way

to practise in rugged and often remote

parts of the province.

Signal Hill ST. JOHN’S

NL

Nat

iona

l

A

sses

smen

t C

olla

bor

atio

n

15

We l co m i n g i n t e r n a t i o n a l p hys i c i a n s t o “ Th e R o c k ”

1514

If any province understands the value of international medical graduates (IMGs) in providing health care for its citizens, it’s Newfoundland and Labrador.

“About 60 per cent of the physician population outside of St. John’s have graduated from an international school. And once you venture off the Trans-Canada Highway, that percentage goes up,” said Dr. Carl Sparrow, Director of Clinical Skills Assessment and Training (CSAT), the province’s practice ready assessment (PRA) program.

Traditionally, many physicians in Newfoundland, affectionately known as “the Rock” for its rocky terrain, came from the United Kingdom and Ireland. The province lacked a medical school before the medical faculty at Memorial University of Newfoundland (MUN) was established in 1967. Dr. Sparrow says the source countries for IMGs changed beginning in the mid- to late 1960s to such countries as Egypt, Nigeria, Iran, Iraq, India, and Pakistan.

“The doctors are looking for a safe place where their families can grow and go to school. Protection of their family is often the motivator,” said Dr. Sparrow.

The IMGs bring a wide range of experience to the province.“We like taking physicians who have practised for a number of years in other countries. As a physician in Newfoundland and Labrador, every day something new comes into your office that you’ve never seen before. You have to be ready for things that are new and unique all the time.”

We l co m i n g i n t e r n a t i o n a l p hys i c i a n s t o “ Th e R o c k ”

1716 17

Dr. Sparrow knows whereof he speaks. His position as director is part-time, and about two-thirds of his time is spent in primary care practice, both clinic- and hospital-based, in Corner Brook, Newfoundland’s third-largest city, where the CSAT program is based.

But the program operates province-wide, working closely with the province’s four regional health authorities. This cooperation is key to meeting the needs of the province’s regions and to orienting physicians to the issues they will see in practice in rural areas, which have traditionally been underserviced.

In fact, CSAT came to be as a result of an underservicing crisis. Sparrow remembers working in one of the regional health boards in 1996. “They were faced with a real shortage of physicians in rural communities and high turnover of physicians.” To bring in IMGs to fill the gap, “they needed both to standardize the evaluation of credentials and training that would allow them to work in our environment.” In December 1997, CSAT was born under the auspices of the Faculty of Medicine at MUN, Health and Community Services, and the College of Physicians and Surgeons of Newfoundland and Labrador.

PathwayCSAT is a three-stage program, involving assessment, training and re-assessment. Results from assessments are used to

four weeks with a psychiatrist to learn about mental health issues, and evaluations by seven preceptors in addition to CSAT’s evaluation. A report on the candidate identifies goals that have been achieved and areas where improvement is needed; a follow-up scheduled exit exam may be provided as well. Success in the CSAT program makes the candidate eligible for provisional licensure, which may be upgraded to full licensure in time after passing the CFPC exam.

SuccessesCSAT’s PRA path has proven very popular. In 2016, within four days after applications opened, the program had received enough candidate applications to fill the program for two years. Physician supply and retention have improved “in every area of the province, especially in Labrador,” said Dr. Sparrow. “We have a good mix of Canadians and IMGs. The usual time physicians stay is four to five years, which is far better than before the CSAT program began.” Combined with efforts to use telecommunications, computer technology and other health-care providers, the program has helped improve service to rural and remote communities, keeping more patients at home and making the health-care system much more cost-effective.

D R . C A R L S PA R R O W

“

“When I ask candidates,

‘What do you want to get

out of CSAT?’ they say, ‘To be

a good doctor in Canada — that’s what I want to do’.

identify what the candidate’s needs are for an “educational prescription,” and training activities are prescribed to meet those individual needs. CSAT offers a two-day orientation, which focuses on communication with patients, other physicians and other health-care providers. Webinars at the beginning of the training phase orient candidates to common problems in practice in the province. The aging population and chronic disease are major concerns. As well, the program focuses on diabetes education, respiratory therapy, child and adolescent guidance, and addictions. Training involves 25 weeks of placements in various clinical disciplines, including

. . . c o n t i n u e d

The CSAT team.

191918

Candidates will be asked to pay a pan-Canadian fee for their practice ready assessment (PRA), beginning January 1, 2018. Knowing where the money goes and how pan-Canadian PRA will operate can help provincial PRA programs explain the fee to candidates, and help candidates understand why they are being asked to contribute to the program’s sustainability.

Why haven’t candidates had to pay for pan-Canadian PRA before 2018?As pan-Canadian PRA has come to the fore, much of the development work has been funded by Health Canada and the Medical Council of Canada. As well, many of the organizations involved have donated their time and effort to the project. In 2018, the pan-Canadian PRA initiative will move from the development phase to the operational phase, which will no longer receive funding. Yet there will still be costs for the pan-Canadian program to be sustained, and funding is needed for maintenance activities. Continuing costs will include:

• maintaining and updating standards for the three medical specialties covered by PRA as well as any new specialties added

• developing and keeping up a resource library and web-based orientation tools such as the cultural communication skills module

• updating the training material for assessors

• conducting research and evaluation of the PRA process(es)

• overseeing the pan-Canadian PRA program

Fe e t o s u p p o r t p a n - Ca n a d i a n P R A co m i n g i n 2 0 1 8

Nat

iona

l

A

sses

smen

t C

olla

bor

atio

n

In deciding that pan-Canadian PRA would be run on a “user-pay” basis, the National Assessment Central Coordinating Committee (NAC3) took several considerations into account. They looked at:

• other professions such as nursing and engineering, in which candidates pay fees for assessments

• exam and assessment fees paid by Canadian medical graduates, which are at a similar level

Levying fees was thus chosen as the best way to maintain a strong, comparable pan-Canadian PRA process that helps high-quality candidates move into practice.

20 21

What will the fee be?Based on several factors, the fee has been set at $1,750. The costs of pan-Canadian PRA and the estimated number of candidates were used to determine the fee per candidate that fully recovers costs. However, in some jurisdictions, the regional health authority or the provincial/territorial government may choose to defray the cost as part of local recruitment efforts. And the fee will be charged only when the candidate has completed selection-assessment exams, before he or she enters the “over-time” or clinical assessment. This means that candidates who will not be proceeding with the entire assessment do not have to pay.

These measures were adopted to help programs be flexible, to accommodate the different environments and circumstances among programs, and to make the fee as fair as possible to candidates.

Fees will support a strong and sustainable pan-Canadian PRA process that benefits the candidates paying them as well as their colleagues participating in a PRA in the future.

A change in the right direction is how a new video presentation

describes pan-Canadian PRA. The video has been prepared to help all stakeholders and candidates grasp key elements of the pan-Canadian

PRA model.

V I D E O P R E S E N TAT I O N E X P L A I N S

PA N - C A N A D I A N P R AThe three-minute video provides an introduction to pan-Canadian PRA and its development, emphasizing benefits to international medical graduates, regulatory authorities, and Canadian patients. It shows why patients can be confident that they are receiving care from a physician who is providing high-quality, safe health care in the Canadian context.

CLICK to view

This report can be provided in accessible formats upon request.

Please contact the MCC at 613-521-6012, or by email at communications@mcc.ca.

nac@mcc.ca

mcc.ca

NAC PRA NEWSLET TERMEDICAL COUNCIL OF CANADA

NAC PRA NEWSLET TERMEDICAL COUNCIL OF CANADA

nac@mcc.ca

mcc.ca