EVALUATION OF QRMSA - Health Workforce …web.healthworkforce.com.au/downloads/Publications/1355... · Web viewEVALUATION OF QRMSA LOCUM SERVICE MARCH 2004 School of Medicine James

RURAL HEALTH & WORKFORCE

RESEARCH UNIT

EVALUATION OF QRMSALOCUM SERVICE

MARCH 2004

School of Medicine

Evaluation of QRMSA Locum Service - 2004

James Cook University

EVALUATION OF QRMSA LOCUM SERVICEMARCH 2004

Professor Craig Veitch Ms Frith Hatfield

Rural health Research UnitSchool of Medicine

James Cook UniversityTownsville, Queensland

C Veitch & F Hatfield, Rural Health & Workforce Research Unit, JCU


EVALUATION OF QRMSA LOCUM SERVICEMARCH 2004

TABLE OF CONTENTS

EXECUTIVE SUMMARY 1INTRODUCTION 1FORMAT OF REPORT 1KEY FINDINGS AND RECOMMENDATIONS 2CONCLUSION 4

HISTORY OF LOCUM PROGRAM 5INTRODUCTION AND BACKGROUND 5LOCUM PROGRAM OFFICE & STAFF 6LOCUM RECRUITMENT / SELECTION 7LOCUM PLACEMENT PROCESS 10LOCUM EVALUATION PROCESS 13FINANCIAL ASPECTS OF THE LOCUM PROGRAM 14SUMMARY 15

METHOD 17INTRODUCTION 17

INTERVIEW DATA ANALYSIS 21INTRODUCTION 21GP/PRACTICE VIEW 21

Views and experiences of the QRMSA locum service 21GP/practice staff experience with QRMSA locums 29External issues 35

LOCUM VIEW 36Views and experiences of the QRMSA locum service 36Personal and professional support 40Experiences as a locum 42

SUMMARY 45

QUANTITATIVE DATA ANALYSIS 46INTRODUCTION 46GP/PRACTICE ASSESSMENT OF LOCUM 46

Professional aspects of locum performance 46Personal aspects of locum performance 48

LOCUM ASSESSMENT OF PLACEMENT 49Working conditions 49Living conditions 50

COMMENT 51

REFERENCES 53


1


EXECUTIVE SUMMARY

INTRODUCTION

This report outlines the findings of an evaluation of the QRMSA Locum Service undertaken during 2003/04. The Queensland Rural Medical Support Agency has provided a locum support service to rural GPs in Queensland since the Agency’s inception. The QRMSA service is largely a continuation of the successful scheme developed and managed by the Queensland Rural Divisions Coordinating Unit, which ceased to exist with the establishment of the state-based rural workforce unit. During 2001/02 QRMSA reviewed and restructured the Locum Service, including the level of financial support to rural GPs and relocation of the Locum Service office from Malanda to Brisbane.

The data collection for the evaluation was conducted prospectively over approximately 9 months (June 2003 – February 2004) and incorporated a series of interconnected steps. Some of these were undertaken concurrently, while others were on-going and some one-off. The study brief was to:

Review and analyse information contained in returned evaluation forms – both GP Principal and Locum forms. Forms collected during the June 2003 to February 2004 period were compared with data collected during the 2001-02 period (previous evaluation period). The two periods were compared in order to detect substantial changes in satisfaction and/or other issues between the two periods.

Interview the CEO and principal officer responsible for the Locum Service regarding changes that have occurred in the service (eg. policies, protocols, selection, training and financial support) in order to better compare the data from the two periods.

Interview key QRMSA personnel responsible for locum placements, evaluation data coding and input.

Conduct a monthly telephone survey of GPs and practice managers who had sought locum support from QRMSA. The survey used a similar interview proforma to that developed and used for the 2001-02 period. The questionnaire was modified (based on interviews with key QRMSA staff) as necessary to better reflect the current service. The aim was to conduct a total of 30 interviews across the data collection period.

Experience from the previous period suggested that there was no need to conduct additional telephone interviews with GPs and locums who reported problems.

Telephone interviews with Locums employed by QRMSA during the evaluation period. A similar interview proforma was used for the 2001-02 period, save some modifications as required to better reflect the current service.

FORMAT OF REPORT

The report consists of five sections. The remainder of this first section sets out key findings and recommendations arising from the evaluation. The second section outlines the history of the Locum Service from its beginning under the QRDCU and then under QRMSA, including the restructuring of 2001 and the subsequent stabilisation of the service. The third section


2


details the evaluation methods. The fourth section covers findings from the interviews with GPs, practice managers and locums, along with a comparison with similar data collected in 2001/02. The final section presents the analysis of the quantitative data collected by QRMSA during the evaluation period and also compared with similar data collected during the 2001/02 period.

KEY FINDINGS AND RECOMMENDATIONS

The qualitative data collection covered the period July 2003 to February 2004, with a total of 28 GPs and Practice Managers (16 and 12 respectively) being interviewed. QRMSA staff and the evaluators agreed that sufficient saturation of data had occurred not to warrant additional interviews. The response rate was 70% of those contacted. A total of 9 Locums were interviewed of 16 contacted (56% response rate). Key issues and recommendations arising from these interviews include:

The GP/practice manager interviews revealed general satisfaction with most aspects of the Locum Service, its staff and procedures, and with placements themselves. Difficulties arose in a minority of instances, some of which highlighted the need for review or tightening of QRMSA procedures. Other difficulties were beyond QRMSA’s ability to prevent or ameliorate, but have been reported because of their potential to impact negatively on QRMSA’s Service and its good standing in rural practice. There were no statistically significant differences between the 2001 and 2003/04 data.

The key findings include:Locum Service

A statistically non-significant increase in general satisfaction between 2001/02 and 2003/04.

Recognition amongst interviewees of QRMSA’s continued efforts to provide locum support for as many as possible and that the small number of available locums is the key limiting factor.

A need for QRMSA to regularly review all aspects of the locum process to ensure that efficiency is maximised and difficulties/oversights minimised.

It is important that staff who relieve the Locum Service staff are fully familiar with the procedures and requirements of the position and are able to immediately and efficiently take on outstanding tasks.

QRMSA staff may need to seek additional information from GPs, particularly in terms of individual GPs’ understanding of the Locum Service’s key objectives and GPs’ expectations of locums.

QRMSA management and Locum Service staff need to regularly review procedures relating to all aspects of communication and information provision in order to minimise information communication difficulties.

In terms of the costs of hiring a locum, the majority of respondents were positive. QRMSA’s locum subsidy is a valued aspect of the service. However, there does appear

to be some confusion amongst respondents regarding eligibility criteria, possible changes to these, and the booking fee. QRMSA may need to regularly inform rural GPs of these (whether they change or not) via various media.

There appears to be limited long-term dissatisfaction arising from the suspension of the locum subsidy in late 2001.


3


A service such as this needs to continually monitor its customers’ expectations and experiences with the Service and respond effectively to issues likely to breed dissatisfaction.

Experience with QRMSA locums There were no statistically significant differences in GP/practice manager satisfaction

with locums between 2001/02 and 2003/04. Issues of greatest importance were (i) locum’s refusal to undertake procedures for

which they possess the skills; (ii) refusal to undertake after hours work. QRMSA may need to consider means by which locums’ procedural capabilities can be more accurately assessed as part of the recruitment/orientation process. There may also be a need to impress upon rural GPs that locums rarely will be able to completely replace them.

One aspect of the orientation process that may need to be reviewed is locums’ knowledge of diseases and conditions common in rural Australia.

QRMSA might also investigate whether contracted, high-quality locums would be eligible for visa extension. If visa extensions are not possible, QRMSA might maintain a database of high quality locums who express a willingness to return to Australia at a later time.

It may be useful for QRMSA to include individual cases studies, supplied by Indigenous communities, in their cultural awareness training to better prepare locums for the types of issues that different communities face. QRMSA might also consider recruiting, or developing links with, an Indigenous Mentor.

QRMSA needs to have back-up plans in place to deal with instances where a locum placement needs to be cancelled (eg. due to late withdrawal of a locum). These might include: (i) dedicated, high-level negotiation with the requesting GP in an attempt to arrive at an acceptable date change; (ii) when compromise is not achieved, high-level negotiation regarding compensation for the cancelled booking; (iii) maintain a roster of potential locums who can respond to an urgent placement need (eg. academic or retired GPs with rural experience).

Locum view Locum respondents were overwhelmingly positive regarding the general administration

of the service and the quality of communication. There is a need for QRMSA to ensure that Global Medical Staffing provide adequate

information to all overseas locums prior to leaving their home country. Locums appreciate the training and orientation program. Consideration might be given

to tailoring the duration of the program to better meet the needs and confidence of individual locums.

Locums appreciate advance notice and information about their next placement. Locums were generally satisfied with most aspects of their last placement. Most

reported difficulties related to personal expectations, rather than issues amenable to QRMSA intervention.

QRMSA might instigate a process by which a QRMSA medical advisor visits each practice that requests locums at some regular interval.

The quantitative data collection covered a 30-month period (July 2001 to December 2003) in five six-month blocks and was based on data collected by the QRMSA at the completion of each locum placement. GPs, practice staff and locums are sent Placement Evaluation Forms


4


that seek respondents’ ratings of a variety of issues relating to the placement. The primary purpose of this data analysis was to provide a numeric picture against which to compare the information drawn from the dedicated interviews undertaken specifically for this evaluation. It is clear that there is a high level of consistency between the two data sets in terms of the issues raised and the likely importance of those issues. This is part reflects the fact that many of the people who completed the Placement Evaluation Forms were also interviewed. It also provides an indication of the reliability of each data set, because respondents (at least in aggregate) have provided similar responses on each occasion.

The key points arising from this analysis include: The majority of GPs, practice staff and locums were satisfied with most aspects of their

placements; There has generally been little change in satisfaction levels across the 2.5 year period

under study; The difficult period that the QRMSA, as a whole, and the Locum Service, in particular,

endured during late 2001 and early 2002 did not appear to adversely affect locum placements at that time, nor leave any long-standing disaffection amongst recent users of the system;

There appears to be a trend of decreasing satisfaction and corresponding increasing dissatisfaction amongst locums with telephone/communication facilities made available to them;

A similar trend is emerging in terms of accommodation standards and maintenance; There was a statistically significant difference in GPs’ rating of locums’ medical skills;

across time periods, that appear to be related to locums’ home country training and practice requirements;

In some instances, the cause of negative comments has changed with time.

CONCLUSION

There is no evidence for wholesale changes to be made to the Service’s procedures. Indeed, the Service continues to function efficiently and effectively within its limited financial and human resource base. Rather our suggestions have focused on reviewing existing procedures, increased and/or regular monitoring of key aspects of the Service and locum placements, and the possibility of increasing communication between the Service and locums and also between the Service and its GP clients. Some other suggestions have focused on extending QRMSA’s remit.


5


HISTORY OF LOCUM PROGRAM

INTRODUCTION AND BACKGROUND

This chapter outlines the establishment and evolution of the QRMSA Locum Program that is now in its tenth year of operation. A variety of information sources have been used, including reports and interviews with key staff. Under the umbrella of the QRMSA, the Locum Program provides short-term locum relief for rural and remote medical practitioners in the Rural Remote Metropolitan Areas (RRMA) 4-7 Classification System. The program also provides locum support to Aboriginal Community Controlled Health Services (ACCHS) in all locations, regardless of RRMA classification. This locum service is essential for rural and remote medical practitioners, and has been able to deliver high quality locums to practices where, prior to the locum service being instituted, rural and remote doctors certainly had difficulties accessing locum relief.

The Locum Program was originally established by the Queensland Rural Divisions Co-ordinating Unit (QRDCU) with a grant from the Department of Human Services & Health (DHSH) Rural Incentives Program (RIP) in late 1994, to develop and administer two important programs for rural and remote medical practice: Continuing Medical Education (CME) activities, and a commensurate locum relief service for rural practitioners in Queensland (Veitch, 1996). Divisions of General Practice are local networks of GPs operating within defined geographical areas, and were first established in 1992, to encourage GPs to work with other health professionals in order to improve the quality of health service delivery at the local level (QRMSA, 2003(a)). The QRDCU, as its title suggests, coordinated various activities and initiatives across Queensland’s five Rural Divisions of General Practice. Executive responsibility for the Locum Program rested with the Medical Co-ordinator of the QRDCU and the QRDCU Management Committee, with separate CME and Locum sub-committees being responsible for determining key issues relative to their particular focus activity. Project administration was undertaken through the QRDCU office in Malanda, Far North Queensland. The program’s objectives were designed to meet the overall goal of the RIP to improve access to GP services in rural and remote communities, and to encourage GP retention in rural and remote areas. In particular, the QRDCU sought to minimise professional isolation through the development and implementation of the CME activities, and the establishment of a pool of locums with skills appropriate to those required in rural practice (Veitch, 1996).

The rural Divisions of General Practice, working with the Rural Doctors’ Association of Australia (RDAA), lobbied the federal government to establish and fund Rural Workforce Agencies (RWAs) to replace the Rural Divisions Coordinating Units that ceased to exist as a result of the discontinuance of the RIP in 1997. The Queensland Rural Medical Support Agency (QRMSA) is one of seven State based RWAs, and was established in July 1998, and funded through the Commonwealth Department of Health & Ageing (QRMSA, 2003(a)). The QRMSA was established to address the critical shortage, and promote the recruitment and retention, of GPs in rural and remote areas; and to promote a high standard of health and greater health options for country Queenslanders (QRMSA, 2001(d)).

In the first three years of its operation (July 1998 – June 2001), the QRMSA Locum Service provided in excess of 8,500 days of locum support for rural and remote GPs for emergency, recreational and educational purposes, with an increase of approximately fifty per cent in


6


locum placements across the period (QRMSA, 2001 (d)). However, across the period July 2000 - June 2003, the number of days of locum support provided by the QRMSA Locum Service, declined steadily by 34 per cent (QRMSA, 2003 (c)). Locum availability is an issue of concern for the QRMSA, and in order to satisfy increased demand for locum relief, there is a need for an increase in the number of locums available.

LOCUM PROGRAM OFFICE & STAFF

Prior to evolving into the QRMSA, the QRDCU’s office was based in Toowoomba, with a small number of staff employed in this Office, and a number of other staff scattered throughout the State. One of these staff was the Locums Manager, who was based in Malanda. When the QRDCU moved their offices from Toowoomba to Brisbane and evolved to become the QRMSA, the Locums Program management remained in the Malanda Office, because the existing arrangement was efficient and organised and communication between head office and Malanda was easy. The Locum Program Manager was responsible for a number of tasks including arranging locum requests from rural and remote GPs; maintaining a database of information regarding the Locums Program, including GPs’ leave entitlements and balances on the type and amount of leave taken by GPs; arranging accommodation and travel for locums, and area of need and visa applications for overseas practitioners coming in as temporary residents; and keeping an up-to-date register of suitable locums. The Locums Manager also had a part-time filing assistant who worked two days a week and was responsible for minimal administrative tasks.

Upon the retirement of the Locums Manager in June 2002, the Locum Program administration was relocated to Brisbane, where a new Locums Co-ordinator was appointed. Prior to the retirement of the original Locum Manager, the new appointee spent some time in the Malanda Office gaining an understanding of the role and the current standing of the Program. The second Locums Co-ordinator resigned in December 2002 after six months in the position. This six-month period was a critical time for the QRMSA Locum Program, with a downturn in the performance and capability of the program; and a decline in service levels with significant reductions in the number of locum placements and days provided. The number of requests for locums also decreased, because rural and remote GPs were not familiar with the new staff member and system. During this period, there was a general decline in record-keeping standards declined and commensurate increase in the number of complaints from GP clients.

In September 2002, the Locum Program secured another staff member, with the appointment of a full-time Locum Administration Assistant, while another person was appointed to the position of Locum Co-ordinator in December 2002 and placed under the direct management of the Recruitment and Marketing Manager, to reflect the changes in the position’s responsibilities. With the resignation of the Recruitment and Marketing Manager in January 2003, another management structure was put in place and the role of the Locums Co-ordinator upgraded to take responsibility for locums and recruitment, while the marketing role was disbanded and replaced with a new strategy. The new position was renamed Recruitment /Locums Manager and currently, the Recruitment/Locums Manager arranges locum requests from GPs and allocates locums in areas of need within RRMAs 4 – 7. The Recruitment/Locums Manager’s responsibilities also include organising the Global Medical Staffing locum placements, and the ‘area of need’ and visa applications for overseas trained practitioners.


7


Previously, the Locums Co-ordinator was required to undertake administration and maintenance of the locums database, however, with the integration of this database into the main QRMSA database, its maintenance, along with a broad range of administrative support, including travel and accommodation arrangements is now the responsibility of the Locum Administration Assistant. The Locum Administration Assistant is also required to maintain an efficient filing system.

With the management of the Locum Program now located in Brisbane, and the incorporation of recruitment responsibilities into the Recruitment/ Locums Manager’s role, there is a greater co-ordination and flow of information between the locum and recruitment arms of the organisation, because all applications whether they be for locum, permanent or hospital positions are received and processed by the one staff member. There are also improved and more efficient management, administrative and operational strategies, involving a more hands-on approach by program staff. For example, with the Locum Program office located in Brisbane, the Recruitment/ Locums Manager has ease of access to the Health Insurance Commission (HIC) which is beneficial when locums have difficulties gaining provider numbers. The QRMSA has also implemented a Quality Management Model that has ensured that systems and processes were clearly documented and continually reviewed, so that the QRMSA has clear policies, procedures and practices in place for not only the Recruitment and Locum team, but also for GP and locum clientele.

Three other QRMSA staff play an integral role in the QRMSA Locums Program: the Data/Research Manager, Data Entry Officer and the Medical Advisor. The Data/Research Manager’s role within the Locums Program is to provide database research and management, data collection and evaluation, preparation of reports, and preparation of evaluation statistics and graphs for six and twelve monthly reports. The Data Entry Officer is responsible for the entry of data from the evaluation forms into the SPSS database. The role of the Medical Advisor, in relation to the Locums Program, is to peruse the resumes of potential locums, and then advise the Locums Co-ordinator where a potential locum might be most suitably placed given their skills. The Medical Advisor also has a role in discussions regarding indemnity issues with the QRMSA’s overseas trained locums, and provides clinical support and advice to locums during their placements.

LOCUM RECRUITMENT / SELECTION The QRMSA receives funding from the Department of Health & Ageing (DoHA) for rural doctors to be provided with locums for specified leave purposes. The QRMSA Locum Program recruits GPs that wish to work for a short time as locums in rural and remote areas, specifically within general practices in RRMAs 4-7, or in Aboriginal Community Controlled Health Services (ACCHS) within all RRMAs. While locums have generally been sourced from Australia as well as overseas, the QRMSA is increasingly finding there is a shortage of good quality locums who are willing to work in isolated situations, and who have the skills to cope with these types of environments and communities. This section outlines the sources used in the recruitment and selection of locums for the QRMSA Locum Program.

In the first years of its operation, the QRMSA Locum Program sourced most of its locums from two major avenues: the ‘City Docs Go Bush’ Scheme and Global Medical Staffing Limited. QRMSA had a pool of 32 practitioners (three of them female) on the locum register,


8


who were sourced from the ‘City Docs Go Bush’ Scheme. The objective of the Scheme was to establish a network of regular city GPs to provide locum relief for rural GPs, with the Scheme paying for a locum in the city practice, whilst the GP was working in the rural area, with the average placement lasting three to four weeks. Over three years, GPs from the ‘City Docs Go Bush’ Scheme provided approximately 40 per cent of QRMSA’s locum placements. During this period the QRMSA continued the long-standing arrangement with Global Medical Staffing Limited, a US-based agency, which provided five full-time equivalent (FTE) overseas trained doctors per year, to meet half of the QRMSA’s locum placements. These overseas trained practitioners provided locum placements for longer periods of time than the ‘City Docs Go Bush’ locums.

Currently, the QRMSA sources its locums from a number of avenues, with half of their locums being overseas graduates, including five per year from Global Medical Staffing Limited. These practitioners originate from countries such as the United States, South Africa, Canada and India. The QRMSA also employs sponsored Temporary Resident Doctors (TRDs) (eg. on working holidays), who are specifically employed to provide locum relief, and are able to negotiate their own wages. These locums are sourced from a variety of places, including the United Kingdom and Ireland. Recently, the QRMSA had a recruiting drive in the United Kingdom, and over the next year, the QRMSA plans to recruit four to five practitioners from the United Kingdom to undertake permanent or locum placements in rural and remote areas.

While the QRMSA has sourced a significant proportion of their locums from Global Medical Staffing, as well as utilising other temporary resident doctors over the past few years, they are now relying less on overseas trained practitioners, due to uncertainties related to medical indemnity which arose in 2002. The first major concern in this respect is the increased cost of locums. Rising medical indemnity costs have made it difficult for the QRMSA to hire procedural GPs through their Global Medical Staffing contract, because the premium to cover medical indemnity per day is considerably higher for procedural GPs. The QRMSA only places non-procedural GPs from Global Medical Staffing and, as a result, this has had an impact on the number of requests for locum relief that the QRMSA receives. Despite this, however, there has been no impact on the number of locum placements or days provided by the QRMSA. The same difficulties have arisen with respect to TRDs, as they are less likely to want to come to Australia to undertake locum work due to the expense involved in gaining indemnity cover.

Secondly, there is an issue regarding run-off cover for locums, which Global Medical Staffing Limited currently provides. The QRMSA assumes that this is going to be adequate for the GPs who are using those locums. The problem is if a GP employs a locum, there is some vicarious liability there, and this has not really been tested with an overseas locum thus far. The issue is not only with overseas locums from Global Medical Staffing, however. There are also concerns with Australian locums, and overseas locums who work under a private contractual arrangement. When a GP hires a locum, they should ensure that the locum has current medical indemnity cover. Added to this, they basically have to trust that the locum will maintain their indemnity cover in the future, but they have no control over this. If a complaint is made by a patient years down the track, and the locum is not covered, then all the liability will fall back on the GP. In an attempt to prevent this type of incident from occurring, some GPs are requesting that any locums they employ sign an agreement stating that they are not being employed by the GP. This confirms that, while the locum is working in the practice and


9


gaining a percentage of the takings, they are not an employee of the practice, which puts some distance between the GP and the locum.

In addition to these issues, many overseas trained practitioners are only in the country for a short-term, so there is a great deal of risk in employing them as locums. If there is a future problem with a patient, the fact that the practitioner is not based in Australia will make it harder to locate them and manage the situation. In the long-term, medical indemnity concerns are likely to increase costs not only for GPs and locums, but for the QRMSA Locum Service as well. Along with the medical indemnity issue, the QRMSA has also experienced supply problems with Global Medical Staffing Limited, and their contract with the QRMSA is due to expire in June 2004, further adding to the dilemma of using overseas trained practitioners.

As a result of the concerns raised in relation to the use of overseas trained locums, the QRMSA is attempting to source more locums from within Australia, either through the ‘City Docs Go Bush’ Program or independent Australian practitioners who wish to undertake short-term locum placements. Australian locums are generally more willing and able to perform procedural services, depending on their particular skills and attributes, because they already have medical indemnity coverage. At present, the QRMSA does not have a dedicated ‘City Docs Go Bush’ Scheme, and many of the GPs previously on the list are no longer active, either because they have not heard from the QRMSA, or have not recently worked for the ‘City Docs Go Bush’ Scheme. With the appointment of the current Recruitment / Locums Co-ordinator, an update of the ‘City Docs Go Bush’ locum register was undertaken, which identified a pool of four to five active locums, who were still interested in providing short-term locum relief. There are also currently one or two independent Australian locums that travel throughout the country doing locum work.

To satisfy demand for locum relief, the QRMSA is also in the process of developing Regional Self-Sustainable Locum Models, which place locums in regional centres so that they can provide locum relief to the outlying rural and remote centres. Initially, locums indicate their interest in working from a regional base. Then, under these models, the local Division of General Practice co-ordinates locum placements within the division, and identify which requests can be filled by their regional locum, and which require further supply from the QRMSA. So far, the QRMSA has discussed this model with every Division of General Practice in Queensland, and has supported one division in establishing their own Regional Locum Model in 2002. At present, the QRMSA is experiencing some hurdles with implementing these models, as the Commonwealth Government will not recognise some areas as districts of workforce shortage, which has led to problems obtaining provider numbers.

One of the most important issues for the QRMSA Locums Program Subcommittee is the ongoing quality of the professional competency of the locums that they enlist, and they always strive to provide the highest quality locums to rural and remote Queensland GPs. With the increase in overseas trained doctors being attracted to live and work in Australia, over the last few years, the QRMSA has continued to develop a recruitment process that is more complex and requires greater rigor in terms of aptitude and clinical assessment. In achieving this, the QRMSA continues to work with key State and national stakeholders and counterparts (eg. ACRRM, HIC, Medical Board of Queensland, RACGP, Queensland Health, Commonwealth Department of Health & Ageing, Department of Immigration and Multicultural Affairs), to ensure that their processes and procedures are consistent with attracting practitioners of good clinical standing, who are suited for life and work in rural and remote communities.


10



11


LOCUM PLACEMENT PROCESS

When a rural or remote GP needs locum relief for CME, recreation, sick or emergency leave, they initially forward a request to the QRMSA Locum Service, providing details of the type of practice they operate, and their specific requirements. Attached with the request forms that GPs/practices fill out is a list of services that may be provided. The practice identifies which procedures the particular GP that is being relieved usually undertakes. The majority of TRDs that the QRMSA employ are qualified to do most of the listed procedures, while Global Medical locums are usually non-procedural locums, and can only perform general practice procedures.

The QRMSA encourages practices to put requests for locum relief in up to six months in advance to ensure that they will be guaranteed a locum. The majority of requests, however, are received about four months in advance, with a significant proportion of practices contacting the QRMSA, one week in advance, to request a locum. The QRMSA is also trialing an emergency locum project, where a locum is based in Sarina, and the QRMSA can access her in the case of an emergency. Some parts of the year, such as the September school holidays and the Christmas period, are quite busy for the QRMSA, with three or four requests per week. Priority wise, the QRMSA generally has a policy of ‘first in best dressed’, but they also take into account RRMA, with the more remote practices given a higher priority over less remote practices. Similarly, in cases where a request is received from a solo GP, the QRMSA fills these requests first. If a GP cancels a request before the locum arrives, the QRMSA makes a note of this in the database, because the GP may have paid their application fee and, when cancelled, the application fee gets transferred to the next time that they request locum relief. In some instances, the Recruitment/Locums Manager has to relocate a locum to another location.

Each request is entered into the QRMSA database. Where possible, the Recruitment/Locums Manager allocates locums to fill these requests, by matching the type of practice with the locum’s skills and experience. When placing locums with practices, the QRMSA attempts to find a locum from their own pool. However, if that is not possible, QRMSA may use some of the private locum agencies to fill a locum request or locums from Queensland Health. The QRMSA does not provide private locum agencies with any of their locums, but they do assist these agencies information-wise. Although Queensland Health normally uses its own pool to provide relief to its rural and remote medical staff, Queensland Health occasionally requests locum support from QRMSA, but the QRMSA does not normally provide locums for these positions, except in the case of an emergency. In the past year, however, Queensland Health has been unable to satisfy their basic locum relief requirements, which is of some concern to the QRMSA given the latter’s own limited locum pool and commitment to rural GPs in private practice. The QRMSA has always had strong links with Queensland Health, and they have a co-operative relationship at service level.

Once a suitable locum has been identified, the Recruitment/Locums Manager contacts the practice to confirm the placement and provide an overview of the locum themselves, including their gender, age, place of origin, level of experience, and any specialist skills. Sometimes, the Recruitment/Locums Manager faxes a copy of the locums’ resume to the practice, if requested, and practices are able to obtain a copy of the Placement Guidelines prior to accepting a locum. Access to locums’ resumes and Placement Guidelines prior to the placement of a locum is a service that was previously not offered by the QRMSA when the management was based in Malanda. This service means that the option is always open for a practice to decide whether


12


they wish to have the locum, and some practices indicate their preference for either Australian or Global Medical locums depending on their previous experiences with locums.

Once the locum placement is confirmed, the Locum Administration Assistant sends out a placement letter to the Practice to confirm the locum’s details and the type of leave for which they are relieving. Along with the Placement Letter, the GP/Practice receives additional information including Guidelines for Placement, Guidelines for Orientation (in cases where the practice will be orientating the locum), Evaluation and Reimbursement Forms, and Practice Reference Guides to be filled out by the Practice and provided to the locum upon commencement of their placement.

The Guidelines for Placement and Guidelines for Orientation, inform GPs and practices of their duties and responsibilities in the provision of facilities and services for the locums. The Placement Guidelines provide GPs and practices with a general overview of what the QRMSA expects of them, in relation to the purpose, the management, the requesting process, how many leave entitlements per year for which they can claim a rebate, and other general information to make the transition a lot smoother. Locums may be at one placement from a week up to six months, depending on the practice.

The Orientation Guidelines are quite different to the Placement Guidelines, in that GPs and practices are not required to provide the locum with a vehicle nor accommodation, as this is the responsibility of the QRMSA. With orientation, the locum is only in the practice for a short period of time, usually two to three days, and the GP/practice is paid by the QRMSA for their services to the locum’s orientation. Since 2003, the Recruitment/Locums Manager and Locum Administration Assistant have updated the Guidelines and Terms and Conditions, closed some loopholes and generally clarified the requirements of GPs and practices. While it is normally the role of the Locums Subcommittee to identify any changes to policy, in this particular instance the Recruitment/Locums Manager and Locum Administration Assistant made the appropriate changes and submitted these to the Locum Subcommittee for approval, which was granted.

Locums are also provided with information to inform them of their placement details, and duties and responsibilities including copies of Orientation and Placement letters sent to the practices, as well as Orientation and Placement Guidelines and Evaluation Forms, a copy of their itinerary, and Electronic Transfer and After hours Billing forms, which are to be used by Global Medical locums when they do overtime or have been on-call. Within the information package, the Locum Administration Assistant also includes some general information and a website address for the town in which the locum will be undertaking their placement; a map of Brisbane City, so that they can find their way around for the Orientation training sessions; a map of Queensland so they have some idea where their placement is, and information on the Australian Driving Rules. The QRMSA is also putting together Placement Practice Profiles, which are to be filled out by the practice and sent to the locum prior to their placement, which gives them information on the practice, GPs, specialists and other relevant practitioners. The Placement Practice Profile is another recent innovation not previously provided during the placement process. Whilst undertaking their placements, locums usually have a professional support system available to them, such as other GPs in the practice, hospital practitioners, and/or specialists. Support is also provided by the QRMSA, with the Recruitment/Locums Manager being in regular contact with the locum during their placement and on the second and


13


last days of their placement. Both the Chairperson of the Locum Subcommittee and the Medical Advisor are also available to discuss clinical matters.

Locums who have a QRMSA contract, are met at Brisbane Airport by the Recruitment/Locums Manager upon their arrival. At that time, the locum receives a Welcome Letter; their travel and accommodation details (which can only be changed by contacting the QRMSA); and copies of application forms for Provider and Prescriber numbers and the Rural Other Medical Practitioner’s Program (ROMPP), which are then sent by the Locums Co-ordinator to the HIC. Also in the Welcome package are details of the locum’s four Orientation Training sessions, including dates, times and venues; and taxi vouchers to cover travel from one venue to another. The Orientation Training sessions include an interview with the Medical Board as part of the registration process; a Cultural Awareness Training session with Queensland Aboriginal & Islander Health; a Software Training session, which takes them through Medical Director or EMIMS and some other software that practices may have; and a HIC information session, where they go through provider numbers and scripts. The locum also receives details of their Orientation Training, which takes place with an experienced rural GP over a two to three day period in a rural practice or the practice of placement.

While the current placement processes are similar to the placement processes that were in place when the Locums Program was managed from Malanda, there have been some minor adjustments. QRMSA staff are providing GPs and practices with more information about the locum and their placement than previously, and locums are being greeted upon arrival in Australia at Brisbane airport, and taken to their accommodation, as well as having the Recruitment/Locums Manager go through the provider and prescriber number paperwork and processes with them (previously, locums were not met at the airport, but were informed where they had to go, and were required to find their own way there, as the program management was based in Malanda). Previous to December 2002, locums were also receiving a package containing information on prescribing in Australia, Medical Practice in Queensland, tips on radiology for short-term locums, an A-Z Encyclopaedia of Aboriginal and Torres Strait Islander information, and material on integration into rural and remote communities. However, this information is being updated, and is not currently sent out to locums. The Locum Program staff have also developed a visual aid, which indicates to them where each locum is placed and where their next placement location will be. The QRMSA website has also been considerably enhanced since 2001, and provides easy access to Request for Locum Relief forms and Locum Application forms.

QRMSA has been using a database for the Locums Program since it began in Malanda, and was maintained by the (then) Locums Manager. During the six-month transition period after the retirement of the Malanda-based Locums Manager (July-December 2002), most of the information and data gathered was recorded manually. The database was updated and redeveloped during this period, to integrate with the main QRMSA database in Brisbane. It was finalised and became operational again in mid 2003. The locums database records details on the GP/Practice requesting a locum placement; the locum allocated to fill each placement; the location of the placement; the type of leave being taken; the start and finish dates of the placement; and the travel arrangements. The QRMSA also keeps a record of when the confirmation letter and the HIC and ROMPP forms were sent, and the dates that evaluation forms were returned. The database allows the QRMSA to compare requests with placements, and the numbers and details of GPs that have made requests each month.


14


LOCUM EVALUATION PROCESS

Within the QRMSA Locum Program, an essential element of the evaluation process is that there are adequate avenues available for feedback for all involved in the placement of locums. The main feedback mechanisms are three self-completed ‘evaluation’ forms: two report on the locum’s performance – one completed by the GP, and the other completed by practice staff; the third is completed by locums on their placement experience. In the Practice Principal Evaluation Form, the GP provides a rating and additional comments on the standard of the locum, based on staff and patient feedback, medical records, and follow-up procedures. In the Practice Other Evaluation Form, another practitioner, the Practice Manager, a nurse or the receptionist provides a rating and any additional comments on the standard of the locum, based on their own experiences and patient feedback. Initially, evaluation forms are sent out to locums, GPs, and other practice staff. Once completed, these forms returned to the QRMSA Office for input on the database. The Recruitment/ Locums Assistant chases up locums, GPs and other practice staff who have not returned forms, to ensure that there is a high response rate. The response rate for evaluation forms is approximately 70 per cent. In the Locum Evaluation Form, the locum provides a rating on their experience during the locum placement, and provides any additional comments relating to their locum placement, including the quality of the medical and accommodation facilities and services provided, the types of medical issues they encountered, accessibility to other medical services and practitioners, and the level of support given by the GP and practice staff during their placement. After data entry, all forms are stored in the QRMSA filing system.

The QRMSA also encourages telephone feedback from GPs and practice staff their locum experiences, and how locum handled their placements. This gives the QRMSA an idea of whether they need to monitor a particular locum during their next placement, whether they can recommend that locum to another practice, or whether that practice would like to have that locum return again. Locums are encouraged to contact the QRMSA if they encounter any problems during their placement, have any specific questions that they may need to refer to the Recruitment/Locums Manager or Medical Advisor, or wish to expand on any comments that they provided within the evaluation form.

While the QRMSA has been sending out evaluation forms to GPs, practice staff and locums for many years, more efficient systems have been developed over the last year, which allow staff to follow-up on the progress of these forms. Dates are set aside when the Locum Administration Assistant follows up on the evaluation forms, specifically those that have not been received. The Recruitment/Locums Manager then checks through the forms, and the Locum Administration Assistant enters the date, that the evaluation forms were received, into the database. The forms are then given to the Data Entry Officer, who inputs the information into the database.

The QRMSA makes every effort to avoid problems between GPs, locums and practice staff by making all parties aware of the arrangements prior to the placement, and encouraging the locum to contact the practice by telephone before taking on the placement. Generally, any problems between GPs, locums and practice staff are made evident to the QRMSA early on. While there are no set processes or procedures for resolving problems between GPs, locums and practice staff, QRMSA Locums staff deal with each situation by following a set procedure, which may involve the Recruitment/Locums Manager speaking with the relevant locum, GP or practice staff members and gathering all the facts; mediating between all parties; identifying


15


solutions; and coming to an agreed and unbiased decision to the problem which may include making arrangements to return the locum to their place of origin, and finding another locum who is willing to undertake the remainder of the placement at short notice.

Clinical issues are handled by, either the Medical Advisor, or an external GP. While the QRMSA has employed a Medical Advisor since 2000, the status of this position has changed over the years with the establishment of a Locums Subcommittee. The current Medical Advisor, who came onboard in late 2002, was an inaugural member of the Locums Subcommittee and, because of her skills and experience, she has been able to provide additional support to the QRMSA Locum Program, through the provision of clinical advice to QRMSA clients, and by undertaking occasional locum work for the QRMSA as well. With the establishment and expansion of the Locums Subcommittee, it is expected that there will be further changes to the role and responsibilities of the Medical Advisor in relation to the Locum Program.

QRMSA Locum staff attend to problems with accommodation or travel. To assist GPs, practice staff and locums to understand their responsibilities and rights within the Locum Program, the QRMSA provides a set of Guidelines, which detail the standard of accommodation, motor vehicle and communication facilities expected. However, the QRMSA has found that the required accommodation standards are not always observed by GPs and Practice Principals and this has led to complaints and dissatisfaction from locums. The QRMSA has also noted that there are issues with motor vehicle provision in relation to licence types, with some US locums having an automatic only licence, and the GP providing the locum with a manual vehicle. In an effort to address these types of issues, it was found that the legal standing and contract between the QRMSA and the Practice Principal was unclear. To ensure, therefore, that each party’s responsibilities are clear, QRMSA has documented all policies; updated old documents and consolidated them into one; and sought legal advice, adding in statements about indemnification if a locum does not attend their placement, and liability being the responsibility of the GP, not the QRMSA.

FINANCIAL ASPECTS OF THE LOCUM PROGRAM

In the 2000/01 financial year, expenditure for the QRMSA Locum Program was approximately $900,000. During the 2001/02 financial year, the QRMSA received a reduced budget allocation of approximately $600,000 from the Department of Health & Ageing, which required a revision of QRMSA’s operating programs, and the curtailment of a number of services (Mitchell, pers comm. 2003). The Locum Program was, therefore, altered to match the available resources and to ensure the service was sustainable and available to all rural and remote GPs.

With the costs associated with hiring locums, and the need for rural and remote GPs to have access to adequate locum relief for educational, recreation, sick or emergency leave, the QRMSA offers a subsidy rebate to assist in covering costs. The amount of subsidy rebate that a GP receives is dependent on certain criteria. The original subsidy rebate scale was determined by type of leave (ie. CME, Sick, and Recreation) and type of practice (ie. Solo Remote, Solo, Group Practice 2-3, and Group Practice > 3). As a direct result of the reduced budget for 2001/02, the QRMSA Board accepted a recommendation, on 1 November 2001, to cease the subsidy rebate scheme, to ensure the continuation of the Locum Service. During this period, a


16


weekly rate of $2,000 was charged to practices for the provision of QRMSA Overseas Locums (TRDs). There was some concern amongst QRMSA management that the Locum Program may have lost some of their client market during the period when the rebate subsidy scheme was suspended.

On recommendations made to the QRMSA Board, the subsidy rebate scheme recommenced on 28 February 2002, with a reduced subsidy amount, which would ensure that the QRMSA was able to provide the same number of locum days, but at a lesser rate. There was also different payment tables for practices supplied with Global Medical locums, Aboriginal practices supplied with locums, and practices that had locums sourced from other agencies (ie. City Docs Go Bush or a private agency). Practices supplied with Global Medical locums or locums from other sources, received weekly rebate subsidies that were determined by type of leave (ie. CME, Recreation, and Sick / Emergency) and locality (ie. RRMAs 4-7), rather than type pf leave and type of practice, as previously occurred. Aboriginal Community Controlled Health Services that were supplied with locums received a standard $1000 per week for all locum placements, irrespective of leave type or locality.

The budget allocation for the QRMSA Locum Program was again reduced in the 2002/03 financial year. Further changes were made to the locum rebate subsidy scheme in November 2003. The QRMSA had different payment tables for Global Medical and other overseas locums, and for other locum placements (ie. City Docs Go Bush or Aboriginal and Islander Health Services), but the amount of subsidy rebate that practices received remained dependent on type of leave (ie. CME, Recreation, Sick / Emergency) and locality (ie. RRMAs 4-7). Practices that were supplied with Global Medical and overseas locums had an increase in the weekly rebate subsidy amount that was provided, while the weekly subsidy remained the same for practices that were supplied with locums from other sources. The purpose of these changes was to put the cost of Global Medical locums and Australian locums on par.

The cost of locums and the subsidy rebate amount that GPs receive for a locum placement can directly influence the amount of leave and the type of leave that GPs take. While GPs can take an indefinite amount of leave, they are only able to claim a certain amount of rebate from the QRMSA. Therefore, the cost of locums can be quite influential in GPs’ decisions on the length of leave, type of leave and type of locum that they request to undertake their locum placement. For the future, the QRMSA are looking at ways in which they can reduce spending with respect to the Locums Program, and this may involve reducing the amount of money spent on travel, by basing locums regionally through the Regional Self-Sustainable Locum Model, and by GPs co-ordinating their holidays with the QRMSA.

SUMMARY

Since the establishment of the QRMSA in 1998, the QRMSA Locums Program has undergone some major changes, including the transfer of the office from Malanda to Brisbane; an increase in the number of permanent staff from one to two, and changes to the roles and responsibilities of these staff members; improvements to the QRMSA locums database; increased costs of locums and liability risks associated with medical indemnity; an increase in the number of locum requests that the QRMSA receives; improvements to the locum placement and evaluation processes; and changes to the locum rebate subsidy scheme for GPs hiring locums.


17


These changes have impacted significantly on the service that has been provided to both GP and locum clients. The following chapters explore data collected as part of this evaluation, and highlight whether these structural and financial changes have, in any way, impacted on the views and experiences of QRMSA GP and locum clients regarding their locum placements.


18


METHOD

INTRODUCTION

Four interrelated data collections and analyses were undertaken as part of the evaluation: analysis of data derived from GP, Practice Staff and Locum evaluation forms returned to the QRMSA following the completion of locum placements; telephone interviews with GPs and Practice Managers who had sought locum support from the QRMSA between June 2003 and January 2004, and Locums who had undertaken locum placements for the QRMSA during the same period; and detailed interviews with key QRMSA Locum Service personnel. This section describes the data collection and analytical methods for each data set in detail.

SPSS analysis of GP, practice staff and locum evaluation forms

The QRMSA Locum Service conducts its own evaluation of locum placements by sending out an evaluation questionnaire to all GPs, Practice Staff and Locums on confirmation of the locum placement. These evaluation forms are completed at the conclusion of the locum placement, and returned to the QRMSA. The evaluation forms consist of a range of statements, which require the respondent to identify their opinion on likert-type scales (ranging from ‘strongly agree’ to ‘strongly disagree’, or ‘excellent’ to ‘poor’ depending on the question). The data from the evaluation forms is then entered into an SPSS database by QRMSA staff. For the purposes of this study, QRMSA sent the SPSS data to the evaluators for review and analysis. In order to most effectively analyse the data, the data sets for GPs, Practice Staff and Locums were sorted and divided into six monthly periods, commencing with the June – December 2000 period and ending with the June – December 2003 period. An analysis of the data was undertaken in SPSS using descriptive statistics, specifically frequencies. Statements were categorised under broad headings for GPs/Practice Staff such as: Consultation Habits; Communication Skills; Work Ethic and Habits; Medical Skills; Technological Skills; and Living Facility Issues, and for Locums, such as: Clinic Personnel; Clinic Setting, Medical Facilities and Equipment; Workload; Medical Skills; Work Conditions – Hospital; Living Conditions; and Local Community Conditions. The output was then compared across the time periods to identify whether there were any significant variations in the data in any of the above categories. Problems, difficulties and comments recorded during each six monthly period were examined to determine any recurring themes that could be compared with the quantitative results.

Telephone interviews with GPs/practice managers and locums

Instruments. The 2003/04 interview instruments replicated those used in 2001, with some minor refinements/additions to better reflect the Service as it currently exists (See Appendix 1 for GP/Practice staff questionnaire, and Appendix 3 for Locum questionnaire). The tables below compare the focus themes explored in each evaluation period, firstly, in interviews with GPs and practice staff and, secondly, with locums themselves.


19


Comparison of focus themes used in GP/Practice Staff interviews in 2003/04 and 2001

Focus Theme

2003/04 Evaluation Period 2001 Evaluation Period

1 Locum Service administration Locum Service administration

2 Comparison with other locum agencies Comparison with other locum agencies

3 Thoughts on Locum subsidy

4 Medical indemnity concerns when hiring locums and Use of Australian-trained locums

5 GP, staff & patient satisfaction with locums

GP, staff & patient satisfaction with locums

6 Differences between QRMSA locums and QRMSA-assisted locum placements

Differences between QRMSA locums and QRMSA-assisted locum placements

7 Suggestions for resolving problems experienced

Suggestions for refinements in the locum selection process

8 Other suggestions for improvement of the service

Other suggestions for improvements to the Service

9 Satisfaction with locums from other agencies

10 Intention to use the Service in the future Intention to use the Service in the future

Comparison of focus themes used in Locum interviews in 2003/04 and 2001

Focus Theme

2003/04 Evaluation Period 2001 Evaluation Period

1 Locum placement procedures Locum placement procedures

2 Pre-placement information Pre-placement information

3 Professional and personal support during placements

Professional and personal support during placements

4 Medical Indemnity issues

5 Last QRMSA placement experience Locum experience

6 Application of skills Application of skills

7 Suggestions for improvements to the service

Suggestions for improvements to the service

8 Intention to accept another rural placement

Intention to accept another rural placement


20


Recruitment. Interviews were sought with GPs and Practice Managers whose names had been selected from monthly GP contact and placement lists provided by the QRMSA office staff in Brisbane. Placement lists for the period covering July 2003 to February 2004 were used (N = 85). Candidate selection was initially planned to use a random numbers table (Pagano, 1990: 535), with replacement for refusals and non-contacts (3 attempts). However, due to the small size of the total population, which included candidates who had had more than one locum placement during the period and had already been canvassed, plus refusals, it became necessary to contact all candidates on the list in order to obtain a statistically useful sample size. In cases where the GP was unable to participate in an interview, attempts were made to interview the Practice Manager. Twenty-eight interviews were eventually conducted (16 GPs and 12 Practice Managers/Staff), which represents a 70 per cent response rate of the subset population of 44. The reduction in the pool from 85 to 44 was a result of disqualifying those GPs who had previously been canvassed (and who had accepted or declined an interview) from subsequent random selections, as well as difficulties in obtaining placement details and contacts from the QRMSA (due to workload and leave demands).

Locums were selected for interview from monthly locum contact and placement lists provided by the QRMSA office staff in Brisbane. These lists covered the same period as for GPs/Practice Managers - July 2003 to February 2004. There were a total of 16 locums used for placements during that period. Due to the small pool of locums and the large number of placements undertaken per locum, plus refusals, it was necessary to contact all candidates on the list. Eventually, 9 locums participated (5 Global locums, 1 Australian locum, and 3 Overseas Trained Doctors), which represents a 56% response rate. Originally, a total of 10 locum interviews (5 Global locums, 5 City Docs/Australian locums) was planned. However, that number and distribution was not achieved due to a combination of factors, including: the small total number of possible candidates, particularly the small number of City Docs Go Bush/Australian locums; difficulty obtaining contact numbers for some locums; refusals; and difficulties in obtaining placement details and contacts from the QRMSA (as for GPs/Practice Managers).

Data collection, collation and analysisThe GP/Practice Manager interviews were conducted by telephone between 28th October 2003 and 12th February 2004 usually during working hours, but occasionally at night or on week-ends. The average interview time was about fifteen minutes. In all cases, interviews were either, pre-arranged with the candidate and then recorded onto CD by Reach Conferencing, or interviews were arranged immediately upon canvassing the candidate, and recorded onto mini-cassette tapes by the interviewer.

The locum interviews were conducted by telephone between 30th October 2003 and 12th

February 2004 during working hours, and the average time for each call was about ten minutes. Most interviews were conducted immediately upon canvassing the candidate, due to locums being available when contacted, so interviews were recorded onto mini-cassette tapes by the researcher. In cases where an interview had been pre-arranged with the candidate, the interview was recorded onto CD by Reach Conferencing.

Interview notes were transcribed from the audio-records by the interviewer, who was the only person to view the original data. To further preserve confidentiality, names and identifying descriptions were removed, and unique identifiers used for files. Coding and analysis of text


21


was conducted with the assistance of the QSR N5 qualitative data analysis computer programme.

Data from 2001 telephone survey of GPs / practice managers and locums

In the 2001, the GP/Practice Manager study group population was drawn from monthly GP contact and placement lists provided by the Malanda QRMSA office staff. The placement lists covered the months of February to September 2001 (N =85). Candidate selection was made using a random numbers table. The interviews were conducted by telephone between 28th July and 19th October 2001. The average interview time was ten minutes. A total of 46 interviews was conducted, which represents a 54% per cent response rate.

For the locum interviews, the interviewer employed a purposive sampling process, whereby the Recruitment and Locums Manager provided the researcher with the names of locums throughout the State that had worked in a rural placement for the QRMSA for at least a month, and then assisted the researcher to identify their location and contact details. A total of seven locums were interviewed.

Interviews with QRMSA staff

In 2003, the key personnel for locum placements, evaluation data coding and input were contacted by the Evaluation Officer, to identify any changes that had occurred to the Locum Service since the 2001 evaluation. In-depth face-to-face interviews were conducted with the Recruitment/Locums Manager, the Locums Administration Assistant, the Medical Advisor, and the Data/Research Manager. An in-depth telephone interview was also conducted with the Chief Executive Officer. These interviews provided additional information on the areas that had previously been explored, as well as new information relating to the medical indemnity issues, and changes to the locum subsidy rebate. Regular repeat calls to the Recruitment/Locums Manager were also made to clarify matters, and to request monthly placement information and contacts. It was not always easy to obtain timely information from the relevant QRMSA staff, in order to conduct GP and locum interviews, for a variety of reasons that included Christmas/New Year holidays, staff leave and high workloads. The information collected in these interviews was used to guide the development of the interview instruments and to clarify issues that arose during data analysis.


22


INTERVIEW DATA ANALYSIS

INTRODUCTION

This chapter analyses various data collected on aspects of the QRMSA Locum service. Data sources include dedicated interviews with Locum Service staff, rural GPs and/or practice staff who requested QRMSA locums during 2003/04, QRMSA locums, and data collected by QRMSA made available to the evaluators, such as placement requests, ‘placement evaluations’ by both GPs and locums and miscellaneous data. The chapter is divided into three main sections: GP/practice views of the service; locum views, and a summary. Where similar data exist from 2001, these are compared with those of 2003/04. Where appropriate, statistical testing of differences between 2001 and 2003/04 was conducted using the Chi-squared test. It needs to be noted that the numbers involved in each of the two time periods are small and, therefore, the statistical results should be viewed as indicative only. Greater weight needs to be given to the qualitative data collected during interviews.

Generally, GPs, practice staff and locums reported positively on all aspects of the QRMSA Locum Service and locum placements. As a result, the focus is more on issues and instances that caused difficulties for respondents, with a view to identifying possible strategies for preventing or minimizing such difficulties in the future.

GP/PRACTICE VIEW

This section is divided into three sub-sections: views and experience of the QRMSA Locum Service itself (hereafter called ‘the Service’); GP/practice staff views and experience of QRMSA locums; and consideration of external factors that impact on rural GPs’ use of locums. Each sub-section examines respondents’ views and experiences of various issues. Although nominally separated into a series of distinct parameters, the data invariably impinge on each other and reflect respondents’ broad opinions of the Service. A summary of the key findings is provided at the end of each sub-section.

Views and experience of the QRMSA Locum Service

General satisfactionIn 2003/04, overall satisfaction with the QRMSA Locum Service was positive. Twenty-five of twenty-eight respondents (89%) replied positively, while the remainder (11%) expressed some dissatisfaction with the service and that there was room for improvement. One respondent pointed out that they were not at all satisfied with the locum service in late 2002, but noted that the QRMSA had taken steps to improve the quality of the service since that time. Concerns arose from difficulties relating to locum availability, a lack of communication by and assistance from the QRMSA during the placement process, cancellation of locums by the QRMSA or the locums themselves at the last minute, and issues relating to negotiation of fees for QRMSA-assisted locum placements. These issues are dealt with separately below.

In 2001, overall satisfaction with the QRMSA Locum Service was positive. Thirty-five respondents (76%) replied positively, while seven (15%) were dissatisfied, and another two


23


(4%) were neutral on the issue. Concerns arose from difficulties relating to locum availability and the personal qualities of certain QRMSA locums.

Comment. Although general satisfaction was higher in 2003/04, the difference between 2001 and 2003/04 was not statistically different.

Locum availabilityAvailability of locums when required was a major concern to respondents. In 2003/04, of 22 respondents that commented on availability, nine (41%) responded positively, and thirteen (59%) negatively. Of those respondents that gave a positive response, some mentioned that they had been able to secure locums not only at long notice (up to 12 months), but even at short notice (just weeks prior to the requested placement date). Typical positive comments included:

"I have found that a locum is available every time I request it."

"We've had no problems with availability, even recently at short notice."

Typical negative comments included:

"Locums are usually pretty hard to get and they are not always available when we need."

"Availability of locums at certain times of the year is a little bit difficult."

"It's very hard to get a locum when wanted. A request has to be put in a long time ahead. I don't know what the doctor would do if he wanted one at short notice. I don't think we'd get anybody."

"I am not happy with the locum availability part of the service. There's not always a locum available – very often there's not."

One respondent pointed out that they had not had even a quarter of their needs met, despite the fact that the QRMSA supplies them with between half and three-quarters of the locums that they apply for. The respondent noted that they do not initially apply, but rather phone the QRMSA to check on likely locum availability to determine whether or not it is worthwhile putting in an application. That said, two respondents who gave negative comments felt that, despite the limited availability of locums, the QRMSA had endeavoured to do all it could to ensure that they were supplied with locums, and that the reason locum availability is so poor is due to a lack of locums in general. As one respondent stated:

"having a locum available whenever we request it, doesn't always work out exactly, but I think that's the trouble everywhere isn't it?"

In 2001, of the GPs that contracted locums from the QRMSA (n = 39), twenty-four (62%) responded positively, and fifteen (38%) responded negatively. However, while the proportion of disappointed respondents was higher in 2003/04 than in 2001, the difference was not statistically significant. As in 2003/04, limited locum availability was the cause of most disappointment.


24


Comment. With just five contracted locums, supplemented by a pool of ‘Docs for the Bush’ locums, locum availability during periods of high demand is often insufficient to meet that demand. QRMSA’s long-established policy gives priority to remote and single GP practices; the intent being to provide support to those with potentially greatest need. The general intent and fairness of the policy is generally supported by rural GPs, but individuals who miss out will be disappointed. Generally, even those who had difficulty securing locum support, recognized the universal shortage of locums, and that QRMSA had endeavoured to secure them locums.

Late cancellationOne aspect of locum availability that causes GPs financial, as well as emotional, concern is late cancellation of a placement. Three respondents reported that cancellation of a locum placement by the QRMSA, or by locums themselves, had caused difficulties for them in the past, with holidays and other plans having to be cancelled at the last minute as a result.

Comment. Although some respondents indicated that there had been some improvements in this regard in recent times, it is important that GPs obtain confirmation from QRMSA that a locum has been secured before the GPs make travel and holiday bookings (this means both parties share the responsibility: GPs give QRMSA adequate time to secure a locum, QRMSA confirms availability as soon as possible so that a GP can confidently make holiday-related arrangements).

Service administrationGPs and their practice staff are the Service’s clients and therefore well placed to judge the administrative efficiency of the Service, at least in terms of its core business – dealing with requests for locums. Related to this, is the quality of communication between Service staff and their clients. This and the following sub-section deal with these two related issues.

The majority of GP respondents found the QRMSA locum service to be efficient. Of the twelve GPs that specifically commented on service efficiency, seven (58%) were completely satisfied. Respondents commented on the organisation of locums’ travel arrangements and provider numbers as positive aspects of the QRMSA Service.

Despite general satisfaction with service efficiency, five respondents (42%) believed that the service was inefficient in a range of areas including staff practices and availability, travel arrangements, the organisation of the locums' stay in Australia, provider number arrangements, orientation training, and general paperwork practices. Generally, only one or two respondents raised each concern. Typical negative comments included:

"I was not satisfied with the service and efficiency in terms of the administration."

"I think there is room for improvement [within the QRMSA]."

Comment. It is noteworthy that some aspects of the Service satisfied some GPs while disappointing others. This likely reflects individual experiences and outcomes in this regard. Nonetheless, QRMSA staff need to be cognisant of times when people are satisfied and of times when they are not. The variety of tasks and demands inherent in placing locums across Queensland is considerable. Added to this, is the need for multiple contacts with GPs or their practice staff, locums, various government and professional authorities, and often at short notice or within limited timeframes. However, as this is the very essence of the Locum Service


25


and the key to its success, it is imperative that QRMSA regularly reviews all aspects of the process to ensure that efficiency is maximised and difficulties/oversights minimised. It is particularly important, during Locum Service staff absences – planned or unplanned – that relief staff are fully familiar (and capable) with the procedures and requirements of the position and are able to immediately and efficiently take on outstanding tasks (via an efficient, ordered, consistent record system) as well as responding to new requests. Perhaps even more important, is the need for all information, particularly outstanding tasks, to be passed seamlessly to Locum Service staff upon their return to service.

Communication issuesThe issues raised within this broad topic included information on locums, information regarding payment and costs, and confirmation of placements. Seventeen of 21 respondents (81%) were satisfied with the amount of information that they obtained from the QRMSA prior to the locum placement. Typical positive comments included:

"We got onto using the email to communicate with the QRMSA, and they sent us updates on what the latest thing was, so I pretty much knew what was going on."

"Yes I am normally kept informed during the placement process. Depending on how far ahead you put in your request, a couple of weeks beforehand, they send you out the doctor's CV and let you have a look, and they have a list that they check, about what this doctor can do, and they always get us to fill in a form saying what do you require."

Two practice managers noted that they sometimes received faxes from the QRMSA when a locum was interested in working in a particular area and asking if they were in need of a locum. The respondents thought this was very helpful, because they never know if their GPs did want to do something at that particular time, so they were then given the option.

On the other hand, four GPs felt that they did not receive sufficient information to prepare them for the locum's placement. Two respondents reported that QRMSA could not always confirm availability of a locum, even when the GPs had travel and work commitments that were dependent on securing a locum. Typical negative comments included:

"I did receive contact from them, but my impression was the fact that I was ringing them, prompted them to do the same, whereas I'm not sure if they would have done that otherwise. I guess the communication aspect is the critical thing".

"I just needed some help as to what we needed to apply for the locum (eg. what things to get set up), and I just find that there's a lot of things that get left undone."

"We've had discussions with the QRMSA about their communication, so I believe they're aware of what we believe the communication problem was."

More specifically, one respondent reported that they were not informed, until the last minute, that they had to negotiate payment arrangements with the locum themselves. Another respondent indicated that they had been informed too late that the starting date for the locum placement had to be put back further. Yet another GP indicated that they were not informed that their locum would not be able to undertake procedures, and that they had booked a number of procedures to be undertaken during the locum placement, which had to be cancelled upon the locum's arrival.


26


One issue that arose during the interviews related to the focus of communication. Clearly, GPs decide when they need locum support and also what specific abilities they require of a locum, but it is the GP’s staff who will have most contact with the locum. In terms of the first issue, some GPs appreciated receiving copies of locums’ CVs from the QRMSA, prior to the locum placement, although most felt that they could not greatly rely on these, particularly as CVs do not indicate whether a locum relates well to people. With regard to the second issue, one Practice Manager noted that neither she nor other practice staff generally knew much about the arriving locum, and do not have access to information or the CV of the locum, despite being the ones responsible for meeting and greeting the locum at the airport and farewelling them at the completion of the placement. Although this is essentially an independent practice matter, it does raise the issue of whether there is a need to ensure that the appropriate staff, at each practice, are made aware of locum placement arrangements, other than just the GP requesting the locum, as quite often the GP will go on leave prior to the locum arriving in the town.

Comment. Some of the concerns raised above suggest a mismatch between the information provided by the QRMSA and the information needed by GPs. This is not to suggest that QRMSA is at fault in these instances, but rather that QRMSA staff may need to seek additional information from GPs, particularly in terms of individual GPs’ understanding of the Service’s key objectives and also in terms of GPs’ expectations of locums. The third instance above (procedural skills) suggests that the particular GP had high expectations of the locum for one of two reasons: (1) they had requested particular skills and were not informed that the available locum did not possess these; or (2) they did not inform the QRMSA of their requirements/expectations so that QRMSA had no opportunity to attempt to match these with an available locum.

It is realistic to expect the occasional communication difficulty, particularly when these are the result of a late change of some sort. Four cases, of themselves, are not many, but these do represent nearly 20% of the GPs interviewed. Thus, although a minority of respondents were critical of this aspect of the Service, their comments do point to the need for QRMSA management and staff to regularly review procedures relating to all aspects of communication and information provision to ensure that issues, such as those raised above, are very much the exception. Some form of pro-forma contact record, which contains all contacts and communication with each GP requesting a locum, as well as follow-up reminders, might prove useful in this regard. As one respondent said:

"It'd be nice for somebody to ring you back straight away, and explain to you what the situation is, and what you're going to be looking at, what you need to organise, and then just to keep you updated on what's happening."

Cost of the Locum ServiceThe general view about the cost of the locum service was positive, especially in comparison with private agency costs. Of the 16 GP respondents that commented directly on the cost of the service, ten (63%) were positive. Typical positive comments included:

"The cost is very reasonable, compared to private locums."

"I think the cost is in line with any other cost of service, other agencies charge similar fees."

Amongst the positive group, it was the rural subsidy that made the locum service affordable:


27


"I am satisfied with the cost of the locum service, especially the subsidy. Being a bulk-billing practice, it was really hard for me."

"I would consider the subsidisation that the QRMSA Locum Service provides has been the single biggest positive benefit, in terms of retention of rural practitioners, that has been specifically set up by the Federal Government."

Respondents believed that locum costs are a lot fairer since the introduction of differential rebates which has narrowed the gap between the subsidies provided for QRMSA locums and QRMSA-assisted locums. That said, one respondent believed that if they had a good quality locum, they would be prepared to pay a little bit more for them, because a good locum would earn the practice a bit more money as well.

Among the six respondents (37%) that had a negative view, comments included:

"I think the cost of the locum service is very expensive, but I don't know what you do about that."

"[I]t costs doctors a lot of money to go on holidays, that's why they don't go very often."

Some GPs reported that when they took leave and hired locums, they always lost money, even though they had access to the QRMSA Locum Subsidy, because:

"… by the time we pay for their accommodation and a car, and their fees, we actually are flat out earning that much back again."

"We always lose money with locums, whether we get a subsidy or not, because patients will wait till I come back. We always lose money, we don't cover our costs at all. So that is a big problem. So sometimes it's cheaper to close the practice than to pay the locum fees, and that's with the QRMSA subsidy as well."

Some practices also had major concerns about having to negotiate fee arrangements with private locums. In these particular cases, the practices only became aware of the private fee arrangements from the QRMSA at the last minute, and so had to go ahead with hiring the locum, because they had organised, and paid for their own travel and leave arrangements, and would have been left without a locum otherwise. Clearly, without knowledge of the fee arrangements, practices are at a disadvantage when negotiating directly with private locums.

Comment. It is important to differentiate between the costs of hiring a locum and other costs related to GPs’ leave. In terms of the former, the majority of respondents were positive, particularly the impact of the differential rebate. Other leave-related costs are outside QRMSA’s remit, but are included here as a reminder that GPs’ outlays do not end at hiring a locum. Also important is the commonly reported drop in practice income, during a locum’s tenure because patients await the return of their GP rather than visit a locum. There are also repercussions for a practice that is visited by a locum who has a negative impact on the practice, because of poor or limited English language skills, poor consultation habits, and limited medical skills. These have been reported to result in patient dissatisfaction and a loss of patients in some cases. Such occurrences cause longer term, indirect financial loss for rural practices. Although such outcomes are not directly within QRMSA’s remit, influencing rural GP retention is, therefore it is important for QRMSA’s staff to be aware that poor locum placements can have longer-term ramifications on practice viability and GP retention.


28



29


Locum subsidyAs illustrated in the previous section, the Locum Subsidy is an important aspect of the QRMSA Service. A number of issues were considered, including satisfaction with the subsidy, its value and its influence on the decision to take leave. The overall view of the subsidy was positive, with twenty-five respondents (89%) responding positively, including:

“Well it’s a good idea, because then [doctors] do actually take a break.”

“We couldn’t get away if the [QRMSA] didn’t subsidise it.”

“If a subsidy wasn’t available, it would be extremely difficult for me.”

The remaining three (11%) were not against the subsidy per se, but expressed a number of realities:

“I read in the information pack from [Recruitment and Locums Manager] about a reimbursement, but I never got that reimbursement because I wasn’t eligible.”

“The problem is the subsidy becomes irrelevant if they cannot provide you with a locum.”

“I know we have to get a lot of paperwork done on this, and it’s always a bother, because it’s up to the doctor [to do] part of it, and once they get back they’re really run off their feet.”

Some respondents raised concerns about the added cost of a non-refundable booking fee, at the time of application for locum relief. The principal concern being that if a locum was not available, then GPs likely lost this fee as well.

Of the 13 respondents that commented on the appropriateness of the subsidy amount, eight were generally satisfied with the amount provided:

“Compared to what we’d be paying otherwise, the subsidy amount provided is excellent.”

“I think the amount’s appropriate, because we took the QRMSA’s subsidy as a basis for our own divisional subsidy.”

Twelve of 19 respondents (63%) indicated that the subsidy was influential in hiring locums and taking leave. If the subsidy was not provided, many of these GPs believed that they would have been forced to take shorter leave or not take leave at all. This is because most GPs would lose income due to the cost of a locum and the costs involved in taking holidays. This was a major issue for solo practices. With the provision of the subsidy, GPs felt that they were able to take leave and hire a locum, and at least keep the practice open, and therefore not lose money. GPs felt that the subsidy not only made leave affordable, but also directly influenced the amount of leave they were able to take each year:

“[I]t’s a very influential and important factor in my decision to hire locums.”

“We couldn’t get away if [the QRMSA] didn’t subsidise it, because it’s just so expensive.”

“[The subsidy] is one of the reasons for someone like us to stay in the town, because they provide four weeks leave each year with a subsidy, and two weeks subsidy for study leave, you can really take six weeks off each year.”


30


Seven respondents (37%) indicated that the subsidy was not influential in their decision to hire locums and to take leave. In some cases, GPs indicated that they needed locum relief for work commitments and courses, so the presence of a rebate didn’t determine whether they took leave. The main issue in these cases was the non-availability of locums, with most respondents indicating that their first priority was to find a locum, and that cost was a secondary issue:

“The problem is the subsidy becomes irrelevant if they cannot provide you with a locum. The [QRMSA] could give me $2000 subsidy per week, but if they can’t provide a locum, then this is fairly immaterial.”

Some GPs also believed that the presence of a subsidy was the reason why they could afford to stay in rural practice, and therefore a major incentive in the retention of rural practitioners:

“I could probably state as clearly as this, that if it wasn’t for the QRMSA locums for the past 12 years, I probably wouldn’t be working in semi-rural practice.”

Some GPs believed that the subsidy was influential in their decision to hire QRMSA locums, because the cost of hiring locums is high, particularly compared to their earning potential. The subsidy was the main reason why these GPs chose the QRMSA over other locum agencies.

Comment. QRMSA’s Locum Subsidy is a valued aspect of the service. Some GPs believe that the subsidy is a vital part of their ability to hire a locum, and that they could not survive without it. The fact that it enables some GPs to take leave is important and is directly related to retention of rural GPs. Those who do not qualify for the Subsidy are not against it, but would like to benefit also. That said, most of those GPs recognize that QRMSA’s resources are finite and appreciate other aspects of the Service, such as the quality of QRMSA locums. There does appear to be some confusion amongst respondents regarding eligibility criteria, possible changes to these, and the booking fee. QRMSA may need to regularly inform rural GPs of these (whether they change or not) via various media.

Future use of QRMSA ServiceAll respondents (n = 28) indicated that they would use the QRMSA Locum Service in the future. Respondents indicated that, despite any problems that they had experienced with locum availability; the organization/confirmation of the locum placement; the high cost of hiring a locum; and the loss of income that they experience when they hire a locum, they were generally satisfied and prepared to use the locum service in the future. The point is well illustrated in the following GP statement:

“Yes, we would [use the QRMSA Locum Service again]. I do have the perspective that the situation that we’ve had the dissatisfaction with is fairly unique, because it was a long-term placement. Normally for just recreation leave, we’ve always had really good locums, and there’s been no problem at all.”

Some respondents pointed out that, even though they experienced problems with locum availability and a loss of income, they didn’t have much choice as this is the only locum service that they have had. Others were content to use QRMSA locums because they were often in need of locums and would take whichever locums they could get, as best illustrated in the following statement:


31


“Yes [I will use the QRMSA in future]. I’m not jack of it, the problems have not stopped me using it. I have a lot of [work travel] responsibilities, [s]o at the end of the day I need locums and I’ll use whoever.”

Comment. Some of the apparent negativity in the comments reflects the varying experiences of people during the past couple of years, during which the Service relocated to Brisbane and also, for a while, did not provide the Locum Subsidy. Those with more recent experience recognized improvements in the Service with which they were satisfied. That said, a Service such as this needs to continually monitor its customers’ expectations and experiences with the Service. In this way, the Service will be able to respond effectively to issues likely to breed dissatisfaction. Equally, rural GPs need to recognize that the QRMSA is effectively their service and therefore be cognizant of the constraints within which the service exists and operates.

GP/Practice staff experience with QRMSA locums

General satisfactionAspects of the locum placement for which respondents’ thoughts were sought included professional manner, consultation habits, communication skills, work ethic, knowledge of the Australian health system, and personal issues such as dress, care/respect for practice and private property. Satisfaction was generally high in each instance. The difficulties reported appeared to relate either to particular instances, and/or a very few individuals. Examples are provided below.

Overall, respondents’ satisfaction with the professional manner of their last QRMSA locum was positive, with twenty-three (82%) responding positively, three (11%) negatively, and two (7%) undecided. One respondent indicated that their locum was not ideally suited to working in a busy practice, and did not have the necessary level of skills to adequately undertake the job within a busy setting. Another respondent indicated that their locum had no communication skills - neither with staff nor patients. The third respondent reported that their locum was unable to speak English, and could not communicate with either the staff or patients. This particular locum had to be sent back overseas after a week, and the practice was refunded their money.

Overall satisfaction with the consultation habits of locums was positive, with nineteen (90%) satisfied and two (10%) dissatisfied. Dissatisfaction arose from poor consultation notes and writing out scripts. According to one respondent:

“[M]ost of the consultation time is made up - like half the time - I’ve got my receptionist in there writing his scripts out, so that he can sign them.”

This particular incident, however, appears to be isolated, and is more a reflection of the locum’s inability to deal with the heavy workload of a busy practice.

Sixteen respondents (75%) were satisfied with the communication skills of their locums. Four (25%) reported difficulties that arose from language barriers, and a lack of communication with staff and patients. As noted previously, one locum could not speak English and was sent back overseas after a week. In another case, the locum did not communicate with either staff or patients.


32


Overall satisfaction with locums’ work ethic and habits was positive, with seventeen (85%) respondents satisfied and 3 (15%) not satisfied. Negative responses mostly related to locums who did not do after hours work. The general sentiment amongst respondents is reflected in this comment:

“When the locum is here, they don’t take after hour calls. They can take after hours calls, but it always seems a great hassle. Not only do we pay or whatever, but it’s just so much hassle, so normally when the locum comes here, we ask them not to take after hours calls.”

Other issues included slow work rates, and a resistance by some locums to undertake work that practices requested of them. In terms of the latter issue, it is important to make the distinction between not being able to undertake a procedure, and refusing to undertake a procedure. It is the refusal or resistance, when the locum’s own CV indicates that they can do that procedure, that is the issue of concern:

“[The locum] wouldn’t do certain things that [this practice] would routinely do, and [the patients] had to wait until [the doctor] got back. What we ticked on the form, she said she could easily do, but then refused to do them when she got here.”

Sixteen of 21 respondents (76%) were generally satisfied with the medical capabilities of their last locum:

“Medical skills were 100 per cent satisfactory.”

“The locum’s medical skills were of a very high standard, in fact one of the best I’ve come across in twenty years I’ve been in the business. I was absolutely astounded at the quality.”

Five respondents (24%) were dissatisfied with the medical capabilities of their last locum, with most difficulties arising from locums not having adequate procedural skills, and not undertaking normal minor procedural practices, such as taking blood and removing skin lesions. Typical comments included:

“There were no procedures carried out, which was something that should have happened. It would actually be better if we had someone who was actually willing to, and actually trained to, do a lot of things.”

“I just wish that [the locum] could do more things. There were a lot of things she apparently couldn’t do.”

One respondent also expressed concern that the locum could not do ante-natal work, stating that:

“With something like ante-natal, she normally doesn’t do that, so she didn’t want to look at it, and that’s a normal part of a GP’s work.”

Respondents noted that many of the locums that were lacking in basic procedural skills were US-trained. Also, the ways in which US graduates undertake some procedures, and their use of drugs have required some short-term adjustment periods for both the practices and the locums.


33


One respondent noted that when locums lack medical skills or refuse to do certain basic procedures, word spreads quickly in rural settings, to the point that the patients specifically wait until the GP returns before seeking care. The same respondent also reported they had found, with long-term locums, that they lost some patients to other medical practices and even the hospital. A few respondents expressed a desire for access to dedicated procedural locums.

Twelve of 17 respondents (71%) were generally satisfied with the rural practice knowledge of their last locum. Nonetheless, five (29%) GPs had some concerns about the level of rural practice knowledge of their locums - in most cases overseas trained practitioners. While it was recognised that some may have had experience in rural settings overseas, respondents noted the distinct difference in cultural practices, social demographics, and health issues in Australian rural settings that should not be underestimated. Issues raised included a lack of knowledge about conditions particularly relevant to Australia (eg. Q-Fever), and systems and arrangements unique to Australian rural practice. Furthermore, there is considerable diversity of practice in rural Australia, which contributes to the uniqueness of health and cultural factors within each rural centre:

“I wouldn’t expect any doctor from anywhere to have a huge understanding [of rural practice knowledge], because each rural area is different too. What works out west where you’ve got a public hospital, and a surgery operating out of it, is very different to here where we’ve only got a private hospital.”

That said, it appears that many locums rapidly developed such knowledge and experience:

“I think [the locum] had been here for a fair amount of time before [doing this placement], so [he was] pretty well sussed out on the rural setting.”

“With regards to rural practice knowledge, [the locum] had worked at Longreach and a few other rural centres, so he had an idea before he got here.”

Twenty-six respondents (93%) were satisfied with the manner in which locums had treated the GPs’ property. One respondent reported that despite providing accommodation at the appropriate standard (3 ½ to 4 star), as addressed in the QRMSA Guidelines, and with which the locum was satisfied, the locum’s wife wanted higher class accommodation. The locum offered to pay the extra cost per night for them to be upgraded to five star accommodation. This one-off incident has no major implications for the QRMSA. However, if thought worthy of attention, the issue could be addressed, by indicating in the Guidelines that it is the locum’s responsibility to inform QRMSA if they or their partner or family have certain accommodation requests, in addition to noting that locums would be responsible for paying the extra cost to upgrade or change accommodation arrangements.

Comment. Negative comments about most aspects of the locums were in the minority and probably reflected individual variation rather than wholesale problems. Probably the issues of greatest importance are (i) locums’ refusal to undertake procedures for which they have indicated they have the skills and (ii) after hours work. This possibly reflects either limitations in the selection process, or limitations with the information provided to locums about the type of work and workload that they are likely to experience in Australian rural practice. QRMSA may need to consider means by which they can more accurately assess locums’ procedural capabilities as part of the selection and/or orientation process. Equally, although the locum pool is limited, QRMSA may again need to review the need for closer matching of locums


34


with practice needs. If reasonable matching is not possible, it might reduce GPs’ (and patients’) dissatisfaction if they are informed in advance of likely procedural skill deficiencies. It was clear from some comments, that some locums are clearly more adept procedurally and that some others rapidly increase their capabilities. QRMSA might consider conducting quarterly reviews of locums (review of recent GP evaluations and short telephone feedback from GPs) in order to assess how well each locum might meet certain practices’ need. Equally, QRMSA may need also to impress upon rural GPs that the service is rarely able to provide locums who can completely replace the usual GP.

One aspect of the orientation process that may need to be reviewed is knowledge of diseases/ conditions common in rural Australia. As noted by one respondent, each rural community and practice is different, so the most appropriate time and place for this training might be during the initial orientation period. Equally, QRMSA might remind GPs that if a particular condition is common in their area (eg. Q-Fever), they should inform the locum of common signs and symptoms and treatment regimen. The QRMSA might also put into their policy documents that rural practices are to provide locums with a list of emergency contacts (eg. Poisons Centre, RFDS, Medi-vac, local ED personnel, etc).

The QRMSA might also investigate whether contracted, high-quality locums would be eligible for visa extension. There is much to be gained from maintaining a steady base of procedurally skilled, experienced locums under contract for a longer period (eg. two years). One advantage is that the skills, knowledge and capability of the locum pool will be higher and more consistent over a longer period and better able to meet rural practitioners’ needs. Another advantage is that the costs and vagaries of recruiting/selecting likely locums overseas are reduced. A third advantage is that a second contracted year would be entirely dependent upon demonstrated high levels of skills and performance. If visa extensions are not possible, QRMSA might maintain a database of high quality locums who express a willingness to return to Australia at a later time.

There were no statistically significant differences in satisfaction levels between 2001 and 2003/04. This partly reflects the high levels of satisfaction reported at each time, combined with the small number effect.

Source of locumsRelated to the previous section, there was considerable support from GP respondents for the QRMSA’s proposal to increase the use of Australian doctors as locums, instead of using overseas doctors. Fifteen (54%) respondents agreed with the proposal:

“It’d be the best thing [the QRMSA] could do.”

“I think that’s better using Australian doctors.”

“I would love it if [the QRMSA] used more Australian doctors.”

Respondents concerns about overseas-trained locums, as outlined in the previous section, included a lack of knowledge about the Australian health system, poor quality, language and cultural barriers, short-term stays meaning that they don’t get to understand the country and its practices, and issues surrounding medical indemnity. Respondents also felt that Australia should support its own, and provide an Australian service, rather than relying on overseas graduates. The perceived benefits of Australian-trained locums are that they have knowledge


35


and experience of the Australian health care system, Medicare and the HIC, they are familiar with Australia’s rural communities, and there wouldn’t be so many language/cultural barriers and conflicting medical practices (eg. US graduates do not know how to take blood). Nonetheless, respondents also recognised some major barriers to using Australian locums, with the most common being the increased cost of using Australian doctors, and the lack of Australian doctors available to undertake locum work. Many respondents questioned whether Australian graduates would be willing or available to undertake locum work, as the following GP statements illustrate:

“[Australian locums] are not easy to get. I wouldn’t imagine they’re going to have a high success rate there.”

“Can you get the Australian [locums]? A lot of them don’t want to come out in the remote areas.”

“I think it’s going to be very hard for us to get locums, because Australian doctors don’t want to come and work here.”

Despite majority support for the use of Australian graduates, twelve (46%) respondents were ambivalent, indicating that they had had few problems with overseas-trained locums, and that they enjoyed having them. The following statement represents the broad view of these respondents:

“I’ve certainly had no complaints about the QRMSA doctors that they’ve had, whether they’ve been OTD or Australian, they seem to have a very good recruitment process, where they have only picked very experienced, competent doctors.”

Practice staff reportsOverall staff satisfaction with locums was positive, with 24 (86%) respondents indicating that their staff were satisfied with the last locum:

“There is only me [in the practice], and the hospital staff thought she was wonderful.”

“I just know that my two staff members worked with [the locum] and [were] very happy with him.”

While staff were generally satisfied with the locum, three respondents (11%) indicated that their staff were dissatisfied with the locum. One respondent did not comment. The negative comments related mainly to communication difficulties between staff and the locum; staff having to assist the locum to undertake duties that were the responsibility of the locum; lack of experience in isolated practices; lack of familiarity with patients; and locums having a lack of adequate skills in, or a refusal to undertake, what staff considered to be “normal everyday” GP duties within their practice. In relation to the latter, the difficulties related to a mismatch between locum capabilities and practice expectations, particularly in terms of procedural skills. There was no difference in staff satisfaction between 2001 and 2003/04.

Patient reportsTwenty-three (82%) respondents reported that their patients were satisfied with the last locum:

“Yes. Every single patient said ‘oh your locum was nice’. They thought that he was very good.”


36


“Yes. We were flat out the whole week. In a small country town, word gets around.”

“We had good feedback from patients. I think, in general, people up here respond well to a change, so they’re interested to meet new people, and they gave good reports to the staff about the doctors we’ve had.”

The latter two statements indicate that sometimes having a locum can be a welcome change for isolated rural communities. This is commonly reported when a female locum relieves a male GP.

Five respondents (18%) indicated that there was some patient dissatisfaction with the last locum:

“I had a few complaints.”

“I didn’t have as many complaints this time as previously.”

“No, I’ve had a lot of negative feedback about him.”

Negative feedback mostly related to the locum having poor communication skills and consultation habits; locums lacking basic procedural skills; Indigenous patients indicating that they experienced some cultural problems with the locum; and organizational difficulties with the locum placement.

In the case of the Indigenous cultural issue, the GP respondent believed that the complaints were few, and that the complaints were not typical of what actually occurred, as there were other Indigenous patients who sought the locum’s services. The respondent did not believe that there was a problem with the locum, but that the issue was more ‘political’ and focused on people deserving free medicine. The respondent believed that the locum had received adequate cultural awareness training and was well prepared. However, they also argued that each Indigenous community has its own distinct cultural issues, and it is therefore impossible to prepare locums for every individual town. As a result, some cultural issues will surface from time to time.

In the case of the organizational difficulties with the placement, the respondent indicated that their practice received a lot of negative patient feedback and lost a few patients because of the difficulties the practice had had with arranging a locum. The problem partly stemmed from the fact that the practice had a locum booked and confirmed, which was then postponed by the QRMSA. This respondent, along with others whose similar concerns were raised earlier, believed that locum placements once confirmed need to be honoured even if the booked locum is unavailable. Respondents argued that rural practices stood to lose too much, whether it be the cost of their leave bookings and arrangements, the stress on the GP and their family, or the impact on practice bookings and patient satisfaction.

Comment. Cultural awareness is an important requirement of all health professionals working with Indigenous people and communities. Clearly, the GP who reported some dissatisfaction among their Indigenous patients with the locum, did not feel that he had returned to a situation that had been caused by the locum. However, it may be useful for the QRMSA to include individual case studies, supplied by Indigenous communities, in their cultural awareness


37


training to expose locums to the sorts of issues that different communities face. QRMSA might also ask GPs, from areas with significant Indigenous populations, to provide locums with a basic outline of the important local Indigenous issues. These GPs might also recommend a local Indigenous person who could be a mentor/advisor to the locum. QRMSA might also consider recruiting, or developing links with, an Indigenous mentor, as the RACGP has done for registrars working in Indigenous areas.

Given the relatively small locum pool, instances will arise when a confirmed locum placement is at risk and, despite QRMSA’s best efforts, another locum cannot be found. Nonetheless, the QRMSA needs to have back-up plans in place to deal with such instances. QRMSA might consider the following suggestions: (i) dedicated, high-level (ie senior QRMSA staff) negotiation with the requesting GP in an attempt to arrive at an acceptable date change; (ii) as for (i), but when compromise is not achieved, regarding compensation for having to cancel the booking; and (iii) maintain a roster of potential locums who are able to respond to an urgent placement need (eg academic or retired GPs with rural experience may be able to indicate periods during which they could respond to such a call at short notice). The first two options will go a long way to relieving disappointment and reducing long-term dissatisfaction and/or disaffection with the Service. The latter is very much an ‘eleventh hour’ response to an untenable situation and has occurred in the past. It may be that the ‘emergency’ locum is only required for a few days until another locum is recruited and arrives. At other times, the ‘emergency’ locum may need to cover the entire booked placement.

External issues

Several issues were also raised that are beyond the QRMSA’s control, but nonetheless impact negatively on practices that use the QRMSA Locum Service. Two issues, in particular, warrant comment - delayed provider numbers for locums and medical indemnity. Other issues raised were remote area exemptions and Veteran's Affairs payment arrangements.

Two respondents indicated that late receipt of the locum’s provider number from the HIC had resulted in a late start for the locum, and the cancellation of appointments in the first few days (this equates to lost income potential, while having to pay for the locum’s accommodation and presence). There is a need for QRMSA staff to remember always that rural GPs are their customers and that a locum booking is effectively a contract to provide cover during a specified period and, therefore, it is their responsibility to ensure appropriate cover is provided during the entirety of that period. While the HIC is outside QRMSA’s direct influence (except for regular contact when a provider number is required), QRMSA can and should act to ensure that practices receive the locum cover promised. Therefore, as with late cancellations, QRMSA might consider placing an ‘emergency’ locum in the practice until the HIC sends the locum’s provider number.

The majority of GPs believed that the medical indemnity issue had not impacted on their decisions to hire locums as yet. However, some respondents were concerned about the future implications on the medical profession, and what it will mean for practices using locums. One respondent pointed out that as general practice is not a specialised area, and as locums just basically do emergency relief work, they have not been too concerned about the indemnity issue. Another respondent felt that the alternatives were not tenable - close their doors when on leave, or work continuously without leave. While aware of the potential dangers of the issue, one GP respondent indicated that they make sure, before employing a locum, that the locum is


38


covered with a company, and that the locum has a good rapport with patients, and is competent in their profession. The impact of the medical indemnity issue on rural practices using locums, is best summed up in the following comments from a GP respondent:

“With the medical indemnity crisis that we’ve got, we’ve just got to be careful that all locums are aware of the indemnity problems and to practice accordingly to reduce risk.”

In terms of hiring QRMSA locums, respondents were under the impression that the QRMSA provided medical indemnity insurance for their locums. Although respondents generally considered the standard of QRMSA locums was high, one was concerned that the QRMSA locum selection process was not entirely thorough, which could lead to some problems with the indemnity issue further down the track.

LOCUM VIEW

This section is based on interviews with nine QRMSA locums, both contracted locums and ‘Docs for the Bush’ locums. The section is divided into three sub-sections: locums’ views and experience of administrative aspects of the QRMSA Locum Service; professional and personal support; and experiences as a QRMSA locum. Each sub-section examines respondents’ views and experiences of various issues. Although nominally separated into a series of distinct parameters, the data invariably impinge on each other and reflect respondents’ broad opinions of the Service. A summary of the key findings is provided at the end of each sub-section.

Views and experience of the QRMSA Locum Service

Placement processEight locums’ were satisfied with the placement process. Positive comments included:

“I think [the QRMSA] is doing an excellent job.”

“Yeah, I’ve had a good experience with the QRMSA.”

“It’s a great service that [the QRMSA] is providing to rural doctors.”

The remaining respondent was not entirely satisfied because of their experiences during the previous year. However, the respondent indicated that their more recent experiences were more satisfactory.

AdministrationOf the six respondents that commented on administration, five considered the service extremely efficient, with typical comments including:

“The paperwork was processed very quickly and efficiently.”

“[The QRMSA] really had it down to a good routine. They know what they’re doing.”

“I asked [Recruitment and Locums Manager] for an extension of my visa, and I sent her the forms, and she promptly sent me the forms to complete, and I’m very glad that she’s taken that in hand, and she promised me that she would do her best, so I can’t wish for more.”


39


One respondent raised a complaint about the administrative process that related to the previous year when the Service was in a period of transition. The respondent reported that the QRMSA had no record of a planned placement despite QRMSA having organised the placement for the respondent. The incident is reported here to demonstrate the long-term ramifications of previous poor service. In this respondent’s recent contact with QRMSA:

“I think there were problems at the end of last year [2002] when I did my placement, but I think there were administrative problems in the agency then. But this year it’s been very smooth and everything’s been fine. I’ve had none of those [administrative] problems, so I’d say whatever hassles they were having has settled down. Obviously they’ve sorted it out.”

Respondents are satisfied with QRMSA staff, and have found them to be supportive and willing to assist, as is illustrated in the following statements:

“I’ve been really quite pleased with the way the [QRMSA has] treated me, and dealt with me.”

“The folks that were taking care of me, clearly knew what they’re doing.”

CommunicationSix respondents were satisfied with QRMSA communication procedures:

“The communication, I think, is good. I have regular contact with [Recruitment and Locums Manager] and she is very helpful.”

“[The QRMSA] kept me informed during the placement process.”

The majority of respondents indicated that the QRMSA contacts them with details of their placements, and if plans change, the QRMSA informs them as soon as possible, as the following statement shows:

“If any additional information turned out to be needed, [the QRMSA] got word to me very quickly, and they were able to get things done with amazing dispatch.”

Locum respondents were at ease contacting the QRMSA if they had any problems or wished to discuss any matters. For overseas locums undertaking placements with the QRMSA, having a contact person available when needed was really comforting. The following statements show the level of appreciation that locums had for the availability of QRMSA personnel:

“If there’s something urgent, I’ll just call. [The QRMSA] are very responsive.”

“I don’t think people really realise how much it means for a person in a foreign country, if there is [a problem], and there is someone [from the QRMSA] you can talk to [about the problem].”

That said, one locum was quite satisfied to have minimal contact with the QRMSA, and only contacted them when necessary.

Comment. Locum respondents were overwhelmingly positive regarding the general administration of the Service and the quality of communication. One negative experience was reported here in order to emphasise the need for regularly review and monitoring of the high standards QRMSA has in place in order to avoid complacency.


40


Global Medical StaffingWhile locums generally were satisfied with QRMSA-sourced information and communication, there would appear to be some concern about that provided by Global Medical Staffing to overseas-recruited locums. In one case, a locum was concerned about the lack of placement information that was provided. While the locum had been in contact with a Global Medical Staffing representative in the US, they did not have contact with the QRMSA until they arrived in Australia. They were provided with minimal details of their placement while still in the US, and were informed upon arriving in Australia, that the schedule that they had been given had changed. Another locum had brought communication equipment (eg. mobile phone and laptop) with them, so that they could keep in contact with family and friends, and were not provided with information on suitable mobile phone and internet companies, within Australia, prior to leaving the US.

Comment. There would appear to be a need for QRMSA to ensure that Global Medical Staffing provide adequate information to all overseas locums prior to leaving their home country.

Orientation and training Overall locum satisfaction with QRMSA orientation and training practices was positive. One locum mentioned that they received an Orientation Guide, while four locums indicated that they received Orientation Training in all or some of the following areas: Cultural Awareness, Medical Software, Medical Board and HIC provider applications, and undertook orientation experience in an approved rural practice (overseas-trained locums). Most comments relating to the provision of orientation information and training by the QRMSA were positive, as demonstrated in the following statement:

“[Y]ou get this Orientation Guide, which has been pretty helpful actually. You read up on medical care in Australia, and how to use Medical Director, and then when you get to Brisbane, they give you a three day orientation on the bureaucracy, PBS system and Medical Director. They also help you fill out your ROMPP and provider applications, and make sure you fill out the right forms.”

However, one locum felt that the orientation period was slightly rushed, and would have preferred an extra day or two. This locum indicated that in 2002 they had received a two-week orientation, compared to just one day prior to their 2003 placement.

Although, there were few comments on the content of QRMSA training and orientation practices, the training provided in computer software and cultural awareness was appreciated as indicated in the following statement:

“The training that I received when I first got here - this is my first overseas locum work - the training was very good. I was ready to deal with the computer system when I got here. I was ready to pay attention to the cultural differences, because I had a chance to go to the Aboriginal clinic, [and] I was well prepared to deal with the differences in cultures when I came here to the clinic.”

Locums did not have any suggestions for other orientation information that they would like to receive from the QRMSA, in addition to what they had received prior to their placement.


41


Comment. Clearly, locums appreciate the training and orientation program. Consideration might be given to tailoring the duration of the program to meet the needs and confidence of individual locums, as evidenced in the case of the locum who would have appreciated a slightly longer ‘refresher’ program. This is not to say that all locums should not receive training and orientation in the core areas, as currently occurs over a couple of weeks, but rather that a few may require slightly longer programs in order to raise their confidence levels before their first rural placement. Equally, if Australian locums have both the desire and the time to undertake slightly longer ‘refresher’ programs, these extended periods may positively influence locum confidence, performance and willingness to work for the Service again.

Locum placement informationIn 2003/04, five locums were generally satisfied with the placement information that was provided to them prior to their placement. Positive comments included:

“[The information that I receive before a locum placement has] been good.”

“[The information provided by the QRMSA] was helpful. I knew what to expect [at my placement], and when I got [there] it was as I expected, and actually better.”

Three other respondents were dissatisfied with the placement information that was provided to them. These three locums were all sourced from Global Medical Staffing and reported that they received brief information about the placement dates, the name of the town and medical practice, the contact details for the medical practice/GP, and a little bit of background information about the town. Their concern was that they were not adequately provided with: full placement details including the length and location of each placement; details about the practice and the procedures commonly undertaken in that practice, and the types of work that they were likely to encounter; full accommodation/travel details; and adequate orientation training within an approved practice. Some of this information was later provided to the locums when they arrived in Australia, just prior to the commencement of their placements, but locums felt that this was not adequate, as the following locum statement shows:

“I didn’t find out till I got [to Australia] that [the QRMSA] had changed my placement, and so it would have been nice to have known [the details of] when, where and how long [for my placement]. I don’t think I even knew how long I was going to be in each place, before I [arrived in Australia] either.”

In general, locums believed that they received the locum placement information enough in advance for them to undertake their placements. Five were satisfied with the timing of the information, as highlighted in the following statements:

“I had plenty of time to look over the information that was provided. I knew what I was going to be doing, and I knew what the working hours would be like. It was a good solid briefing before I got here.”

“I’d know [all the placement details] a couple of weeks in advance, but I knew all the places. The information is provided to me enough in advance prior to my placement.”

One other respondent, a first time Global Medical Staffing locum, was provided with the placement information on the day of their arrival in Australia, which was the day prior to the commencement of the placement.


42


Locum satisfaction with the provision of travel and accommodation details was mixed. While some locums were provided with full details, others received only sketchy details, as the following statements illustrate:

“With travel, [the QRMSA] make all my travel arrangements, so that’s sent to me before I go from one place to another. I don’t really find out where I’m staying till I get there, or ahead.”

“It wasn’t actually till I got [to Australia] that I found out how I was going to get to [town name], and so once I got here, they told me I would be flying to [town name], as opposed to bus or whatever.”

“You don’t know the specifics, like I don’t know where I’d be staying till I actually got here, but sometimes I’d know a couple of weeks in advance.”

Comment. The interviews indicated that locums clearly appreciate advance notice and information about their next placement. However, there are occasions when information is either limited, or provided very late. As already noted, QRMSA may need to ensure that Global Medical Staffing is providing locums with sufficient and appropriate information before they depart for Australia.

Personal and professional support

Eight locums were generally satisfied with the professional support they received during their placements, with six knowing of, or having used, the support of either the QRMSA, other doctors and staff within the practice, or other medical personnel within the town. One respondent was not satisfied, indicating that they received minimal support from the QRMSA or the practice itself. This locum was undertaking an emergency placement in a single-handed practice, but received little support from practice staff, as there is only a small staff presence and the only communication the locum had with the Practice Principal was by phone. The locum was disappointed that QRMSA did not contact them during that week, particularly in view of the nature of the placement, the heavy workload, and that this was the locum’s first placement.

Six respondents were generally satisfied with the professional support provided by QRMSA staff. These locums knew of the support available from the QRMSA and were satisfied with the support provided, with typical comments including:

“The QRMSA have provided me with good professional support.”

“[A]nytime time I need something from the QRMSA, it’s usually dealt with very quickly and well.”

Most QRMSA professional support was sought from the Recruitment/Locums Manager or the Locums Administration Assistant, and usually related to administrative matters. One locum did also seek professional guidance from the QRMSA Medical Advisor in relation to availability of vaccines and similar matters. Generally, locums appeared to seek the support of QRMSA when needed, and were generally aware of the support that the QRMSA could provide.

Three respondents reported that they received little professional support from the QRMSA, but pointed out that they were generally not looking for support, and believed that it would probably be available if they were in need of it. This is typified in the following comments:


43


“I can’t say, other than the orientation, that the QRMSA has given me any professional support, but I haven’t really needed any from them.”

“I find very little [support from the QRMSA], but I’m not looking for any really, generally, and I’m sure it would be available if I sought it.”

The one instance of dissatisfaction was outlined at the beginning of this section.

The five respondents that commented on the professional support provided by other medical personnel, were satisfied with support received from private practitioners, specialists and hospital practitioners and staff in the town:

“There are other doctors in the town that I can talk with, and I spoke to some doctors at the hospital for information one day, so there is that available.”

“I can speak to the hospital – the doctors and surgeons there. There’s a good referral system. I speak with a specialist at the hospital. I think that’s fine.”

Medical personnel provided support with not only ordinary clinical issues, but also emergency procedural information.

Six respondents were satisfied with the support provided by practice staff, including other doctors in the practice, nursing staff, and administrative personnel. Positive comments included:

“The support that I received professionally from the other folks in the clinic, both the two docs that are here and the nursing staff has just been superb.”

“[T]here are three other doctors in the practice, so I’ve got more than adequate support there from them, and the administrative staff here are very efficient as well. I haven’t felt the need to go elsewhere for any support, there’s a lot of support here in the practice.”

“The staff pretty much knows what goes on, and what sort of things they normally refer to, so anything I have problems with, they’ve been answering all of my questions.”

Three respondents reported limited support, generally because of a lack of available staff, or the locum not making contact with the relevant staff. The reports do not betray a lack of willingness on the part of practice staff to provide support, but rather firstly a lack of available staff for locums to contact, because of distance, workload, or leave; and secondly locums not seeking support, either because they did not want to impose upon staff, or they simply felt that they did not need assistance.

Eight respondents were satisfied with the personal support that they received during their placements, most of which was provided by other staff in the practices:

“It was very nice. I got taken out to dinner the first night that I was here, and got a chance to get acquainted with the other docs. Families invited me out for [tea], and [I had] a wonderful time.”

“All the staff here have been very nice, and make sure I’m being entertained and seeing things.”


44


“[W]e’ve had really good experiences with local staff, cause they’ll often point us in the direction of, or even take us on local tours, and local interest stuff. We’ve seen some marvellous stuff. At each place we go we get something.”

Some locums also received personal support from the QRMSA, or were aware that personal support was available if needed. That said, most admitted that they didn’t need a lot of personal support from the QRMSA, as this was mostly provided within the town and practice itself. The most important personal support that the QRMSA provides is friendly, helpful and supportive staff:

“Like … the two at QRMSA. You can establish a personal relationship with them, and that’s who your main focal point is…you talk to.”

“The QRMSA have been very good, [and] they’ve been very helpful to me. [T]he QRMSA staff were there providing support and everything, and we were very impressed.”

Comment. Although locums will generally not require QRMSA support, locums need to be aware that support is available. One means by which this can be demonstrated is in the form of a courtesy call during the first few days of each new placement to check that all aspects of the placement are satisfactory and to remind the locum to call if they require assistance. Although time-consuming for QRMSA staff, this process will maintain locum satisfaction and potentially reduce the likelihood of a locum running into difficulties. Such contacts should especially be conducted during placements early in a locum’s contract. Unfortunately, two respondents reported that they were not contacted by QRMSA during their first placements.

Medical indemnity Four locums were concerned about medical indemnity, four were not and one did not understand the issue. Two indicated that their concerns were the same as for GPs and they needed to ensure that they were still covered. A third locum indicated that indemnity cover was a considerable cost to them:

“Yes [the medical indemnity crisis is] costing me a fortune, and I wasn’t told about that before I came over here, and it’s going to cost me a fortune when I go back home, which I also hadn’t thought about. It would have been nice to have that information from somebody beforehand.”

Four US-trained locums were relieved to be working in Australia, where the problem is on a somewhat lesser scale than in the US. Some also indicated that the crisis doesn’t affect them as a locum, because they are only in the country for a short-term, and that they are not paying their own insurance.

Experiences as a locum

Eight locums were satisfied with their last locum placement, while one was not, stating that “[i]t was a bit of a nightmare.” That locum reported that the practice in which they undertook their placement was in “complete chaos, and didn’t have the [necessary] administration or clinical infrastructure for a [medical] practice to run smoothly.” While the locum found this to be a serious problem, they believed that the QRMSA did not realise the extent of the problems within the practice.


45


Locum satisfaction with workload was negative, with five respondents indicating that the workload at their last placement was either heavier or lighter than they expected. Four of these respondents indicated that the workload was heavy:

“Some of the places the workload is really heavy, like you work your tail off. [I] work pretty hard, and don’t worry about it too much. You know you see a ton of patients. Next week where I’m going, I already know my workload’s going to probably be the hardest one.” “It’s hard work at [town name], alternate nights on and alternate week-ends on. The workload was very very high.” “The workload was very very heavy, but I didn’t do any out of hours work.”

The other locum indicated that in their last placement, the workload was “a little light” and that they had to do after-hours work. Three locums were satisfied with the workload in their last placement.

Satisfaction with practice staff and patients was positive, with six respondents satisfied with the practice staff and patients at their last placement:

“My practice staff and patients have always been excellent. The practice staff are always more than helpful. Thank God, otherwise I’d be in a bind. And most of the patients are friendly.”

“Practice staff and patients were excellent, couldn’t ask for better.”

“The practice staff and patients have all been very good. I’ve been very surprised by the patient basis, very very interesting and very good. You get reasonable people, very easy to deal with.”

Three respondents were not entirely satisfied with the practice staff and patients at their last placement, for a variety of individual reasons.

Five respondents (of 6) were satisfied with the provision of clinical equipment at their last placement. One thought that some suturing materials were inadequate for patients’ needs.

“[T]here were only 1 – 5 O needles, but for big wounds you need big cutting needles, because gaping wounds, you can’t do anything with that. You need big needles and sturdy needle-holders to fix a wound, suture a wound, but I feel the selection of needles was not appropriate.”

Six (of 8) respondents were satisfied with the living facilities (ie. accommodation, transport and phone), at their last placement:

“I was very happily pleased with the housing that I had, and the support just for my basic needs and living was very very good. The living facilities, accommodation and vehicle that I was provided with, were first class.”

“The living facilities were fine, more than adequate.”

Locums stayed in a range of accommodation including flats, hotels, houses attached to practices, and the GP’s house.


46


Three respondents were not satisfied with the living facilities at their last placement, with negative responses relating to factors such as poor accommodation standards, a lack of cooking facilities, and a lack of phone connection, including bars on long distance calls. In relation to a lack of cooking facilities, one locum stayed in a hotel for several weeks, without access to cooking facilities. Despite finding themselves in accommodation that was lacking in certain areas, locums generally reported that practice staff and accommodation staff were willing to do all they could to assist the locum in this area.

Comment. In general, locums were satisfied with most aspects of their last placement. The instances of dissatisfaction tended to relate to personal expectations, rather than issues amenable to QRMSA intervention. The sole exception to this would appear to be the report that one practice was ‘in chaos’. QRMSA might instigate a process by which a QRMSA medical advisor visits each practice that requests locums at some regular interval (eg every one to two years). Accommodation standards could also be assessed at the same time. The standard of housing and rental accommodation varies from town to town, so it is reasonable to expect that some individuals will be required to live in less than adequate accommodation in their terms. Although such difficulties were reported, respondents also noted that local people attempted to assist to relieve the situation. While QRMSA can set accommodation standards, GPs and communities are ultimately limited by what is available in the local community.

Ability to apply skillsAll respondents (n = 9) reported being able to effectively apply their skills during placements:

“Yes. That was a really satisfying part of being here, as I felt like I really got a chance to do things, and exercise skills that I wouldn’t have got to exercise back in the US, because of our liability crisis.”

“Yes, and the QRMSA matches my skills with the practice I’ve noticed, so that’s fine. I use the skills I have.”

“Yeah. It pushed me to my limit.”

Global Medical Staffing locums from the United States all indicated that they were able to use their skills more in Australia, than they would be able to in the US. Locums indicated that they were able to do more procedures, and set fractures, which they would not be able to do in the US because of the threat of litigation.

Future locum placementsLocums’ opinions about future rural placements were positive, with seven indicating that they would undertake a rural placement in the future, and most already scheduled to do more. Two respondents indicated that they would not be undertaking a rural placement in the future, as both were under long-term placement arrangements, and would return to their own country at the completion of their placements. One of these locums (a QRMSA sponsored locum) indicated that they would not accept a rural placement in the future. This locum had experienced some difficulties during an emergency locum placement that the QRMSA requested them to do, and had serious concerns about the cost of the medical indemnity insurance.


47


SUMMARY

The GP/Practice Manager and Locum interviews revealed general satisfaction with most aspects of the QRMSA Locum Service, its staff and procedures, and with placements themselves. Difficulties arose in a minority of instances, some of which highlighted the need for review or tightening of QRMSA procedures. Other difficulties were beyond QRMSA’s ability to prevent or ameliorate, but have been reported here because of their potential to impact negatively on QRMSA’s Service and its good standing in rural practice. Where data and numbers permitted, statistical comparisons were made between 2001 and 2003/04 data. In no instance, was there a statistically significant difference. This is probably not surprising in view of the generally large majorities of respondents, in each time period, who indicated satisfaction with the various aspects of the Service.

There is no evidence for wholesale changes to be made to the Service’s procedures. Rather, our suggestions have focused on reviewing existing procedures, increased and/or regular monitoring of key aspects of the Service and locum placements, and the possibility of increasing communication between the Service and locums and also between the Service and its GP clients. Some other suggestions have focused on extending QRMSA’s remit.


48


QUANTITATIVE DATA ANALYSIS

INTRODUCTION

This chapter deals with data collected by the QRMSA at the completion of each locum placement. GPs, practice staff and locums are sent Placement Evaluation Forms that seek respondents’ ratings of a variety of issues relating to the placement. Respondents are also invited to make additional comments on the forms. These forms are returned to QRMSA where the information is transcribed onto a database for later analysis. The QRMSA provided data collected via these forms for the 2.5 year period beginning in July 2001 and ending in December 2003. The data were supplied in blocks of 6 months (July-Dec 2001; Jan-June 2002; July-Dec 2002; Jan-June 2003; and July-Dec 2003). The results of the descriptive statistical analyses are discussed here (the cross-tabulations are not provided within this report, but can be provided on request) and, where warranted, comparative analyses are undertaken using either the contingency Chi-Squared test, or One-way ANOVA test, as appropriate.

The chapter is divided into two sections: GP and practice staff ratings of placements; and locums’ ratings. GP/Practice Staff and Locum ratings of the same placement have not been linked for analysis. Instead, aggregated numbers are used. Each sub-section is further divided into ‘themes’ (eg ‘Consultation Habits’) which themselves incorporate several issues for which individual ratings were sought.

GP/PRACTICE ASSESSMENT OF LOCUM Professional aspects of locum performance

Consultation habits Overall GP and Practice Staff satisfaction with the consultation habits of locums was positive, with more than 90% of respondents satisfied. There was little variation in satisfaction between the time periods - no more than 15%, equating to 2-4 people. In general, consultations were considered to be well performed, the locum demonstrated respect towards patients, the locum was able to cope with the practice workload, medical records and reports were well maintained, the locum demonstrated a willingness to spend time with patients, office procedures were appropriately and well conducted, and patient feedback indicated that waiting times were acceptable. The only slight anomaly is in regards to the locum being able to cope with the practice workload. A drop of almost 24% in satisfaction occurred between the periods January – June 2002 and July – December 2002. However, the drop was statistically insignificant, with only one respondent (5.9 per cent) dissatisfied with the locum’s ability to cope with the practice workload, as opposed to 100 per cent satisfaction during January and June 2002. In addition, the number of respondents during the July – December 2002 time period was small (n = 18). There were no major problems reported about locums’ consultation habits. The only areas of slight concern were a locum’s use of homeopathic treatments on patients, workload issues, and the reluctance of some locums to admit patients to hospital, which cropped up across the different time periods.


49


Communication skillsMore than 90% of GPs and Practice Staff were satisfied with the communication skills of locums. There was little variation in satisfaction across the time periods, and most of the variation was likely due to the relatively small numbers involved at various times (eg. in the July-December 2002 period, each of the 18 respondents accounted for more than 5% of the total). In general, locums showed good interpersonal skills, related well to patients and staff, and demonstrated ability to deal comfortably with children. There were no major problems reported about the communication skills of locums. Slight problems with language barriers and poor communication skills were noted during early 2001 and late 2000. However, fewer complaints about communication skills were recorded since that period, with most relating to language barriers.

Work ethicMore than 90% of GPs and Practice Staff were satisfied with the work ethic and habits of locums. In general, colleagues and staff were comfortable working with the locums, patients were able to obtain home visits as needed, and visits were made where necessary, patients were able to see the locum outside normal working hours, and locums were reliable, had a good work capacity, a good attitude to their placement, and dressed appropriately. In most categories, there was little variation in satisfaction across the time periods, with much of the variation reflecting the small numbers involved. For example, although there was a relatively large drop in colleague and staff comfort with the locum (18%) during the January - June 2002 and July - December 2002 periods, this variation was not statistically significant (One-way ANOVA p>.05). While not statistically significant, it is interesting to note that, with regards to the locum being dressed appropriately and patients being able to obtain home visits as needed, during the last half yearly period (July – December 2003), there was a slight drop in the percentage of respondents that agreed with these statements (7% (n = 4) and 13% (n = 8) respectively), and a corresponding increase in the percentage of respondents that were unsure, but no change in the proportion of dissatisfied respondents.

Although there were no statistically significant differences in satisfaction across the 2.5 years study period, there was a qualitative change in the focus of dissatisfaction. Prior to mid 2002, most negative reports related to the locums’ attitudes to work, their late arrival at work, issues relating to locums’ workload, locums’ refusal to undertake after hours work, and locums working unnecessary overtime and making claims for this. Since then, negative comments related to the locums’ attitude to work have decreased, but there have been an increase in negative comments about the treatment of staff. There have also been a small number of negative comments relating to locums’ refusal to perform minor examinations/operations or procedures (eg. excisions, pap smears).

Use of practice equipmentAgain, more than 90% of GPs and Practice Staff were satisfied that locums had used practice equipment adequately, with only slight variations between the time periods, which can be attributed mainly to the small number effect. Although there were no major problems with technical skills, a couple of instances arose before July - December 2002 regarding some locums’ lack of computer skills or problems with the computer systems.

Medical skillsNearly 90% of GPs and Practice Staff were satisfied with the medical skills of locums. In general, locums were suited to the requirements of the practice, appropriate procedural


50


experience was demonstrated, the expected range of procedures was carried out appropriately, patient feedback indicated that the locums demonstrated knowledge relevant to patients’ presenting conditions, hospital procedures were appropriately carried out, locums were aware of their limitations, and locums had a high level of skill, experience and knowledge in medicine. Nonetheless, there was a statistically significant difference in GPs’ ratings of locums’ skills between time periods (Chi-squared = 22.16, df = 12, .05>p>.03. NB: ‘fair’ and ‘poor’ categories were combined because of small numbers in each of these.). Negative comments related to the lack of skills and experience that some locums (generally trained in the US) had in certain basic medical procedures, or refused to undertake; such as collecting blood, performing pap smears, applying plaster casts, performing excisions, giving local anaesthetics for sutures, and giving immunisation injections.

In relation to the locum being suited to the requirements of the practice, there was a 20% variation in positive responses, ranging from a high of 96% in the January - June 2002 period, to a low of 78% in the July - December 2002 period. However, this difference was not statistically significant.

Personal aspects of locum performanceLiving facilitiesGP and Practice Staff satisfaction with living facility issues was very positive, with at least 90% of respondents satisfied that locums cared for their accommodation and vehicles during their placements. There were only slight variations, of less than 10 per cent, between time periods. However, 7% (n = 4) of respondents expressed outright dissatisfaction during the period July – December 2003. This was the only period during the 2.5 years under study that GPs had expressed dissatisfaction. There was, however, a change in the causes for negative comment across time. Prior to July 2002, most of the negative comments from GPs and Practice Staff about living facility issues were related to the locums’ use of vehicles, with complaints including damage to the vehicle, not refilling the vehicle with fuel, not returning the key to the vehicle, and smoking inside the vehicle. There were also some complaints relating to the locums’ use of communication facilities, specifically overuse of the internet, and not paying private phone costs. Since January 2003, most negative comments have related to the locum’s use of accommodation facilities, with complaints including locum’s wife not being satisfied with the standard of accommodation, and the house being left in a mess (ie. carpets dirty, bed linen dirty).

One aspect of the living facility issue is worthy of note. On average, GP and Practice Staff satisfaction with locums reimbursing GPs for private travel/accommodation costs remained at approximately 60% across the five time periods. Of those who were not satisfied, around two-thirds were ‘unsure’ and one-third were clearly dissatisfied. This pattern did not change across the 5 six-month periods, with only minor variations of less than 10 per cent, across the time periods. In view of the relatively low satisfaction levels (compared with all aspects of GP/Practice Staff feedback and ratings), QRMSA might consider closer monitoring of this issue across time in order to guard against the development of deep-rooted dissatisfaction.

Personal attributesOverall GP and Practice Staff satisfaction with the personal attributes of locums was positive, with only a small number of negative comments made on the Locum Placement Evaluation Forms. These included poor personal presentation, poor hygiene, lack of physical fitness, and personality issues, none of which are cause for serious concern. Since July 2001, there have


51


not been any negative comments recorded, in the data, in relation to locums’ personal attributes.

LOCUM ASSESSMENT OF PLACEMENT

Working conditionsClinic personnelLocum satisfaction with clinic personnel was very positive, with an average of 97% of respondents reporting practice staff as helpful during their placement. There were only slight variations, of less than 6%, across the time periods that were not statistically significant. Although no respondents have disagreed, since December 2001, that practice staff were helpful, there were a few negative comments regarding practice staff lacking knowledge and training in some practice areas, practice staff having negative attitudes, and problems with staffing levels. The latter was more a matter of staff leaving or going away at the time that the locums commenced their placements, rather than inadequate staffing levels within the practices. That said, the incidence of such issues appears to be rare.

Clinic setting, facilities & equipmentLocum satisfaction with clinic setting, facilities and equipment was positive, with an average of 97% of respondents satisfied in relation to this. There were only slight variations in satisfaction, of less than 10%, across the time periods that were not statistically significant. There were no outstanding problems reported across time in relation to clinic settings, although there were a very few negative comments about clinic equipment, such as surgery equipment not functioning (n = 1), instruments were old and inadequate (n = 1), and that currently used methods for drawing blood (ie. not using vacutainers) in some practices were posing a health risk (n = 2).

There were also a few negative comments in relation to clinic technology, across the five time periods. Prior to December 2001, most of these comments related to locums experiencing problems with computer systems and networks within their placement practice, although these have declined since that period. There were few complaints after December 2001, and those related to practices lacking or under-using their computer equipment and systems, rather than locums having problems, or not being trained, in using them.

WorkloadLocum satisfaction with their workload during their placements was positive, with an average of 95% satisfied. There were only slight variations in satisfaction, of less than 15%, across time (not statistically significant). In general, locums felt that workloads and after hours calls were reasonable, and that practice procedures were compatible with expectations. Despite high satisfaction with workload, there were a number of negative comments (n = 9) from locums, prior to January 2002, that practices were extremely busy, and their workload was heavy. Since then, however, complaints about heavy workloads have decreased. There were also a very few comments about after hours work across time. A few locums indicated that they were not informed about after hours responsibilities, until they commenced their placement, but once again, these complaints were rare.


52


Hospital working conditionsLocum satisfaction with hospital working conditions was positive, with an average of 86% satisfied. In general, locums were satisfied that public hospital outpatient arrangements were agreeable, that public hospital inpatient facilities were adequate, and that public hospital equipment was adequate. In relation to the adequacy of public hospital equipment, there were minor (statistically insignificant) variations in satisfaction across time. There was, however, a dip in satisfaction in the July - December 2002 period, at which time overall satisfaction was only 65% - a 22% drop from the previous period. However, the satisfaction level rose by 35% in the following period. Of note is that the level of dissatisfaction did not increase significantly during the July – December 2002 period, but the proportion of respondents that were unsure increased considerably. Despite the fluctuations, the differences across time were not significant (Chi-squared test).

With regards to public hospital outpatient arrangements, there were only slight variations in satisfaction, of less than 15 percent, across time (not statistically significant). There were also only minor (not statistically significant) variations in satisfaction with the adequacy of public hospital inpatient facilities, except between July and December 2002 (satisfaction = 65%). During this period, satisfaction dropped by 28% from the previous six monthly period, and then increased 35% in the next six monthly period. While dissatisfaction did not increase during this period, the proportion of respondents that were unsure increased considerably. Despite these fluctuations the differences were not statistically significant (Chi-squared test).

As with other aspects of locums’ contact with the hospital system, there was considerable fluctuation in satisfaction with the adequacy of surgical facilities and equipment, but these were not statistically significant. In line with the lack of statistically significant differences across time, few negative comments were made in relation to hospital working conditions.

Medical skillsLocums’ satisfaction with their ability to use their medical skills was positive, with an average of 96% satisfied across time. Variations between 6-month time periods were less than 10% and not statistically significant. In general, locums were satisfied with use of their skills and experience during their placements, and felt that the practices, in which they undertook their placements, were well matched with their skills and experience. There were no complaints, from locums, in relation to medical skills use.

Professional supportOverall locum satisfaction with professional support during their placements was positive, with an average of 92% satisfied across the entire period. Variations of less than 15% were recorded between time periods (not statistically significant). In general, locums felt that ambulance/ evacuation services were satisfactory, and that professional support was available during their placements. There were a very few negative comments, but no consistent theme or focus to these.

Living conditions Living facilitiesLocums’ satisfaction with living facilities was mostly positive, with an average of 89% satisfied across the entire 2.5 years. In general, locums felt that the accommodation supplied was well maintained, and that the vehicle supplied was satisfactory. Despite this, however, there was an increase in the proportion of respondents that were dissatisfied with the


53


maintenance of the accommodation facilities with which they were provided – ranging from no dissatisfaction during July – December 2002, to 10% dissatisfaction during the January – June 2003 time period. The variation in satisfaction between time periods was statistically significant (Chi-squared = 26.67, df = 8, .01>p>.001). Reported problems with regards to accommodation facilities generally related to the standard of the accommodation, (eg. running toilet, dirty carpets, dirty kitchen, no locks on doors, no hangars, and spider webs in bedroom and living area). Other issues included that locums were required to stay in motels, caravan parks, shared accommodation, and accommodation next to the surgery; a lack of cooking and washing facilities, and a lack of air conditioning/heating within the accommodation. The frequency of these complaints matched the satisfaction levels for each period.

In comparison, there were fewer negative comments about satisfaction with vehicle supply, and complaints actually decreased across time. During the 2000/01 time period, most of the small number of complaints, related to the GP not supplying the locum with a vehicle. There were no complaints regarding vehicle supply during 2003.

Conversely, locums weren’t as satisfied with telephone arrangements during their placement. In 2001, 83% were satisfied with telephone arrangements compared with 73% in the January – June 2003 period. Added to this, dissatisfaction with telephone arrangements also increased, rising from 10% in 2001 to 18% in the July – December 2003 and January – June 2003. In the January – June 2003 period, 26% of respondents (n = 11) were dissatisfied with the telephone arrangements during their placement. The main problems in relation to the dissatisfaction were that locums were not provided with a phone in their accommodation, or that they were only able to make local calls, which was not suitable for overseas locums that wanted to phone family. A small number of respondents also complained that some phones were not suitable for using calling cards to phone home. Although there was no statistically significant difference between time periods (Chi-squared test, p > .10), there is a need to monitor the trend of decreasing satisfaction and corresponding rise in dissatisfaction.

Local community conditionsLocum satisfaction with local community conditions was positive, with an average of 94% of respondents indicating satisfaction in this regard. There were only slight variations between time periods, of less than 15% (not statistically significant). In general, locums were satisfied with the facilities in the town and with their integration and acceptance into the community, and felt that that they were given appropriate community information, and that the people in the community were friendly and helpful. There were only a few negative comments regarding the local community conditions across time - all related to the lack of social outlets and activities and limited services that were available in the towns.

COMMENT.

The primary purpose of this data analysis has been to provide a numeric picture against which to compare the information drawn from the dedicated interviews undertaken specifically for this evaluation. It is clear that there is a high level of consistency between the two data sets in terms of the issues raised and the likely importance of those issues. This is part reflects the fact that many of the people who completed the Placement Evaluation Forms were also interviewed. It also provides an indication of the reliability of each data set, because respondents (at least in aggregate) have provided similar responses on each occasion.


54


The principal points arising from this analysis include: The majority of GPs, practice staff and locums were satisfied with most aspects of their

placements; There has generally been little change in satisfaction levels across the 2.5 year period

under study; The difficult period that the QRMSA, as a whole, and the Locum Service, in particular,

endured during late 2001 and early 2002 did not appear to adversely affect locum placements at that time, nor leave any long-standing disaffection amongst recent users of the system;

There appears to be a trend of decreasing satisfaction and corresponding increasing dissatisfaction amongst locums with telephone/communication facilities made available to them;

A similar trend is emerging in terms of accommodation standards and maintenance; There was a statistically significant difference in GPs’ rating of locums’ medical skills;

across time periods, that appear to be related to locums’ home country training and practice requirements;

In some instances, the cause of negative comments has changed with time.


55


REFERENCES:

QRMSA, (2001 (a)), Guidelines for Requests to QRMSA Locum Service: Policy / Procedures.

QRMSA, (2001 (b)), Orientation Guidelines for Overseas Doctors.

QRMSA, (2001 (c)), Policy / Procedures for Funding Requests to QRMSA for Locums Provided by Other Agencies.

QRMSA, (2001 (d)), Queensland Rural Medical Support Agency Annual Report 2000-2001.

QRMSA, (2002), Queensland Rural Medical Support Agency Annual Report 2001-2002.

QRMSA, (2003 (a)), Analysis of the Queensland Rural & Remote Medical Workforce.

QRMSA, (2003 (b)), Guidelines for Requests to QRMSA Locum Service: Terms and Conditions.

QRMSA, (2003 (c)), Queensland Rural Medical Support Agency Annual Report 2002-2003.

QRMSA, (2003 (d)), QRMSA Orientation Guidelines for Practices Orientating Overseas Trained Doctors.

Veitch C. (1996), Evaluation of the Continuing Medical Education (CME) / Locum Program of the Queensland Rural Divisions Co-ordinating Unit, 1994-1995, North Queensland Clinical School, Townsville.

Pers comm. 2003. Interviews with Frances Parsons (Recruitment and Locums Manager) (2001); Chris Mitchell (CEO) (2003); Colin White (Data Manager) (2003); Madonna Richardson (Recruitment / Locums Co-ordinator) (2003); Meggan Nazarski (Locum Administration Assistant) (2003), Dr Sheilagh Cronin (Medical Advisor) (2003).


Documents

EVALUATION OF QRMSA - Health Workforce …web.healthworkforce.com.au/downloads/Publications/1355... · Web viewEVALUATION OF QRMSA LOCUM SERVICE MARCH 2004 School of Medicine James