Training Needs Analysis 4Aug

THE ROYAL AUSTRALIAN AND NEW ZEALAND COLLEGE OF RADIOLOGISTS®

TRAINING NEEDS ANALYSIS (TNA)

Training Needs Analysis (TNA)v2.indd 1 7/26/2011 12:02:31 PM

The

Roy

al A

ustra

lian

and

New

Zea

land

Col

lege

of R

adio

logi

sts

Stra

tegi

c P

riorit

ies

2010

- 20

14

2

Name of document and version: Training Needs Analysis (TNA), Version 1

Prepared by: Kelly Menzel - Educational Development ManagerSharon Messina - Director, Education and Research

Submitted to:Endorsed by College Council, 21 July 2011 ABN 37 000 029 863

Copyright for this publication rests with The Royal Australian and New Zealand College of Radiologists ® The Royal Australian and New Zealand College of RadiologistsLevel 9, 51 Druitt StreetSydney NSW 2000, Australia

Email: [email protected]: www.ranzcr.edu.auTelephone: + 61 2 9268 9777Facsimile: + 61 2 9268 9799


Education and Research Portfolio Training Needs Analysis 22 July 2011 Page 2

CONTENTS

1. INTRODUCTION……………………………………………………………….3

2. WHAT IS A TRAINING NEEDS ANALYSIS? ………………………...……4

3. BENEFITS OF UNDERTAKING A TNA……………………………………..6

4. INITIAL QUESTIONS ………………………………………………………..6

5. METHODOLOGY………………………………………………………………8

6. TRAINING NEEDS IDENTIFIED……………………………………………11

7. RECOMMENDATIONS………………………………………………………18

8. CONCLUSION………………………………………………………………...22

9. REFERENCES………………………………………………………………..24

10. BIBLOGRAPHY……………………………………………………………….26

11. APPENDIX 1…………………………………………………………………..27

12. APPENDIX 2…………………………………………………………………..28

13. APPENDIX 3…………………………………………………………………..36


1) INTRODUCTION

This training needs analysis (TNA) has been conducted to ascertain the training

needs and priorities of members of the Royal Australian and New Zealand

College of Radiologists (“the College’). Findings from the TNA will assist in

shaping the Learning and Development Framework, which, in turn, will influence

the future direction of training and education at the College. The Learning and

Development Framework will guide a broad range of activities designed to

improve the capabilities of the members within the College. This capability

development will include not only clinical and technical skills and knowledge, but

also attributes, attitudes and behaviours.

Currently the College has two training programs: The Radiodiagnosis discipline

is operating two programs concurrently (current training program and old training

program) and the Radiation Oncology discipline is operating two programs

concurrently (current training program and old training program). The most

marked differences between the current and old training programs is the move

from an apprenticeship-style of learning, largely based in one centre with loosely

structured rotations over 5 years, towards a structured, rotational training system

over a 5 year period. In addition, the current programs in both Radiodiagnosis

and Radiation Oncology have a more explicit curriculum reflecting the goals and

philosophy of the program, with a clearly defined syllabus and learning

objectives. The programs are more transparent and accountable with

appropriate assessment processes: both formative and summative; broad and

relevant assessment of all aspects of trainee competency, not just with a

medical expert focus but also within 6 non-medical expert areas of

management, professionalism, communication, patient support & advocacy,

teamwork and research. This reflects the CanMEDS Model of medical practice,

which has been implemented by the College. It has been acknowledged that the

role of medical expert is central to the work in both disciplines but the other six

roles are also important, and together the seven roles encapsulate all aspects of

a practitioner.

Further to this, Continuing Professional Development (CPD) was made

mandatory by Australian Health Practitioner Regulation Agency (AHPRA) from

July 2010. This means that all Fellows, post training, have to participate in


professional development and training in some way in order to maintain their

registration.

The outcomes and recommendations of this TNA aim to assist in determining

what way the College can best facilitate a more coordinated approach to both

training and CPD.

2) WHAT IS A TRAINING NEEDS ANALYSIS?

A TNA is a review of learning and development needs within an organisation. It

considers the skills, knowledge and behaviours that people need, and how to

develop them effectively (Lundberg, Elderman, Ferrell and Harper 2010, Chiu,

Thompson, Mak and Lo 1999). A TNA is considered to be the foundation of all

training activities. In order to deliver appropriate and effective training which

meets the needs of individuals and the organisation and represents value for

money, a TNA is essential (Reed and Vakola 2006, Reid and Barrington 1999,

Boydell and Leary 1996). There is general agreement in the literature that a

training needs analysis is a best practice first step in the systematic approach to

training (Reid and Barrington 1999, Wills 1998, Bartram and Gibson 1997,

Boydell and Leary 1996, Garavan et al., 1995, Reay 1994).

This systematic approach to training is the predominant model found in the

literature (Reid and Barrington 1999, Wills 1998, Boydell and Leary 1996,

Buckley and Caple 1995, Garavan et al. 1995). It is described slightly differently,

with varying stages and elements, but there are a number of core features to the

approach. The systematic approach is one which involves considering the

linkages between the parts of the training process (Altschuld and Lepicki 2009).

There is an assumption that training must be planned in a cyclical manner and

that this approach will lead to high quality, planned training (Iqbal and Khan

2011, Barrington and Reid 1999, Bartram and Gibson 1999, Wills, 1998,

Buckley and Caple 1995). Bartram and Gibson (1999) state that a “systematic

approach to identifying training needs ensures that people are offered

opportunities to learn which are efficient and effective” (p.107). All of the

systematic approaches in the literature outline a number of steps in the process

and cover similar basic elements. These are as follows:


• determining the training need

• designing (choosing) the appropriate methods to address the identified

need

• planning (developing) the training courses/programs

• implementing

• evaluating

The benefit of this approach is that nothing is omitted and there is a planned and

professional approach to training development and delivery. It also provides the

data necessary to justify or explain to senior management and key stakeholders

what training is required, what budgets are needed, the relative spending

breakdowns and how it will impact the organisation (Wills 1998).

Although much has been written about TNA in relation to medical and health

education (Gould, Kelly, White and Chidgey 2004), there is some disagreement

with respect to its impact on the training cycle and the potential to influence

service delivery (Lee, Altschuld and White 2007). Bearing this in mind, this TNA

has been undertaken to ascertain the training needs and priorities as perceived

by the members of the College (including trainees, Training Network Directors,

Directors of Training, examiners and committee members).

The process delineated below is one that the College will be following in

determining development of training and educational resources for the College.

The recommendations proposed as an outcome of this analysis will ideally

inform the Learning and Development Framework for the College (refer to

Recommendation 13 on p.20).

TNA

Design

Develop

Implement

Evaluate

Educational Resource

Development Process


3) BENEFITS OF UNDERTAKING A TNA The benefits to the College in undertaking a TNA are outlined below:

• Alignment with Strategic Plan: Consistent progression against a big picture

plan.

• Process improvements: reduced duplication of effort; online (faster) access

to information; a proactive rather than reactive approach (a planned

approach).

• Cost savings: lower trainee turnover; increased productivity; a planned

development which avoids repetition and/or duplication of effort (more

efficient).

• Performance improvement: in quality, quantity and speed of delivery;

knowledge/skill of clinical supervisors; knowledge/skills of trainees.

• Behavioural improvements: in attitude, motivation, leadership,

communication.

• Relationship building and improvement: A standard process for developing

training and educational initiatives allows openness, transparency and trust

to flourish.

• Good governance: A standard process provides a platform for good

governance and efficient operational management.

4) INITIAL QUESTIONS

Prior to commencing the TNA, a review of the following occurred:

a) AMC Assessments 2009;

b) College Response to the AMC 2010 and 2011;

c) College strategic and organisational plan 2010;


d) Education & Research Portfolio Strategic Plan October 2009, February

2011;

e) Education & Research Portfolio SWOR (Strengths, Weaknesses,

Opportunities, Risks) analysis October 2009;

f) Curricula competency framework (based on CanMEDS);

g) College mission and core values statements encompassing commitment

to the value of learning and development for our members;

h) College processes (formal and informal) for effective consultation across

the organisation;

i) Surveys and questionnaires previously undertaken by trainees and

Directors of Training (throughout 2009/2010) on the curriculum

development;

j) Faculty of Radiation Oncology 2010 workforce survey;

k) Radiology 2010 workforce survey;

l) Minutes of the meetings throughout 2009-2011 of:

• Continuing Professional Development (CPD) Committee in

Radiology;

• the Post Fellowship Education Committee (PFEC) in Radiation

Oncology meetings;

• the Radiology Research Committee (RRC);

• the Radiation Oncology Research Committee (FRC);

• the Faculty of Radiation Oncology Board (FRO);

• the Radiology Education Board (REB);

• the Radiation Oncology (ROEB);

• the Council.

What was learned from the review above is that:

• Consultation is crucial to get ‘buy in’ from all stakeholders.

• Communication is important and must be conducted through various means

to reach the largest audience.

• Cross discipline sharing between Radiology and Radiation Oncology is

imperative in order to leverage off all knowledge generated.

• Cross functional sharing within the College is essential so there is no

duplication of ideas and workload.


• Partnership between fellows and staff is imperative and the view that the

College staff are merely the ‘Secretariat’ must be shed.

• That research must become part of the organisational culture. This has been

particularly slow in developing in Radiology; however, with research

becoming an active part of the current training curriculum and the

rejuvenation of the Radiology Research Committee (RRC) this is expected to

change.

• A commitment to continuing professional development (CPD) is necessary in

order to keep abreast of changes in the evidence, best practice and

guidelines in both disciplines.

In addition to the above, it was important to consider the following questions prior to conducting the TNA:

• Why do people need the training?

• What skills need imparting?

• Who needs the training?

• When will they need the new skills?

• Where the training may be conducted?

• How may the new skills be imparted?

In addressing the above material and keeping the above questions in mind –it

ensured that in the recommendations proffered in Section 7 of this document,

expectations and suggested timelines for deliverables were realistic.

5) METHODOLOGY

a) Research Design and Scope

Various methods can be used to identify and review the training needs of an

organisation or professional body. These include:

i) Analysis of existing strategies and a plan to identify what skills are

needed for delivery;

ii) Questionnaires – paper based or online;

iii) One-on-one interviews;

iv) Focus groups - facilitated small group discussions with a representative

sample of people.


For the purposes of this TNA, (i) existing strategies for the development and

delivery of training were analysed in addition to conducting (iii) one-on-one

interviews and (iv) focus groups. Questionnaires (ii) either online or on paper

were initially considered as an additional method for data capture, however,

after receiving several comments from interviewees in the early part of the

TNA consultation process (January-February 2011) ‘survey fatigue’ was

identified and it was decided to abandon the questionnaire/survey as a tool.

Moreover, an action research approach was adopted for the College TNA.

Action research allows collaborative problem solving with those involved, as

well as the generation of new knowledge (Coghlan and Brannick 2001). It

allows the study of a process directly as it unfolds. Gummesson (2000)

considers that the level of pre-understanding and access to the reality of any

process balances against concerns around objectivity. In the context of this

project, the researcher was relatively new to the College (8 weeks) at the

time the process commenced. The advantages of access and the

development of some pre-understanding (and continuing development of

same), were balanced against not being completely socialised to the culture

over many years, and still having an element of an outsider view.

The semi/unstructured interviews were conducted on a one-on-one basis

face to face and in focus groups face to face throughout December 2010 to

April 2011. These were conducted with key stakeholders (as identified on the

following page) who either: influenced training; organised training delivery;

delivered training; and/or partook of training. In some instances, where a

face to face interview was difficult to arrange, a telephone interview was

conducted.


Of the 46 targeted for interview either on a one-on-one and/or focus group basis

(as indicated above), 45 participated, which is a participation rate of 97.8%. This

included consultation with members from Australia and New Zealand.

It should also be noted that the findings addressed such questions as:

• How can we capture the good practice and expertise that already

exists?

• How can we build on the strengths, skills and knowledge already in the College?

• What skills, knowledge or behaviours could help address the identified weaknesses?

• What skills, knowledge or behaviours could help the College make the

most of the available opportunities?

RANZCR Consultation College CEO Director of Communications & Membership Dean of Faculty Radiation Oncology Director of Quality and Standards of Practice Director of Standards of Practice and Credentialing Director of Training, Assessment & Accreditation Director of Education & Research Senior Education Officer Radiology

8 Council President Vice President

2 Radiology Chief Censor & Chair of Education Board Chair, RRC Chair, CPD Chair, CAC Trainees Fellows

17 Radiation Oncology Chair, Faculty Board Executive Officer, Faculty Board Chief Censor & Chair of Education Board Previous Chief Censor member, FRC Chair, PFEC member, TNDF Focus group of recent Trainees and current Fellows

19 Total interviewed 46


• What skills, knowledge or behaviours could help the College manage

and overcome any identified risks?

Once data had been gathered via the above methods, the findings and

outcomes of the TNA were analysed and a Learning & Development

Framework and subsequent plan was formulated (see Section 7).

b) Project Timeline

The TNA commenced in December 2010 shortly after the Educational

Development Manager was recruited (25 October 2010).

The following schedule was followed:

• Data Collection (literature review, interviews, attendance at meetings,

observations): December 2010 – April 2011

• Analysis: April 2011 - May 2011

• Write up: June 2011

6) TRAINING NEEDS IDENTIFIED

a) Primary Findings

General (across both disciplines)

Whilst the separate disciplines revealed discreet findings of their own, there

were also findings that were uniform across both disciplines. Interestingly very

few findings identified encompassed the delivery or development of material in

the area on the medical expert. The majority of findings were in areas of non-

medical expertise.

These were:

i. The need to development/facilitation of programs for:

• “Training the Trainer”

• Interacting with Trainees

• Dealing with a Trainee in difficulty/difficult Trainee


• Giving feedback

• Adult learning methods/models of adult learning

• Clinical Leadership

• Advocacy (on behalf of patients and the discipline)

• Communication

• How to turn the curriculum into practice

• Report writing (“how to”) – to go hand in hand with the

guidelines currently being developed

• Support for examiners and exam writers assessment in

standard setting: e.g. how to mark/how to set marks/how to

set criteria, etc

• Strategic Planning

• Change Management and managing through change.

ii. The need to develop online access to resources for Fellows so they can:

• Easily access CPD resources

• Submit information on activities/events completed/attended

and provide verification

• Easily print out their own CPD profile and points summary

through a self-service mechanism

• Access courses and programs

• Easily submit documents for scholarship applications and

research projects

• Easily provide/submit new educational resource materials

or links to such for consideration by a College Educational

Resources Review panel.

iii. The need to develop online access to resources for trainees so they can:

• Easily access training resources

• Submit information on assessments completed and provide

verification

• Maintain an e-portfolio

• Easily print out their own Training Profile (assessments

summary) through a self-service mechanism




research projects




iv. The need to develop online access to resources for Directors of Training

so they can:

• Easily access trainer resources

• Monitor their trainees’ progression against assessments

• Monitor their trainees’ progression against e-portfolio

• Easily print out their own Training profile (assessments

summary) through a self-service mechanism



research projects




v. Role clarification for:

• Directors of Training (DoTs)

• Training Network Directors (TNDs)

• Examiners

• Board of Examiners

• Accreditation Officers

• IMG Assessment Officers.

vi. Induction (orientation) programs for:

• Trainees

• New Directors of Training

• New Fellows

• Members who are accepting roles on a committee(s)

• Accredtitation Officers

• IMG Assessment Officers.


vii. Development of Policies and/or position statements for:

• Recency of Practice (after a period away from job, e.g.

parental leave)

• Recognising Prior learning

• Confidentiality

• Conflict of Interest

• Qualified Privilege.

viii. The college need to try and (re)engage with institutions and health

providers so that they acknowledge the importance of training

(acknowledgment of protected time etc).

Discipline specific findings

Although the majority of the findings were reflected across both disciplines,

there were discipline specific findings revolving around subject matter relevant

only to the discipline. These are listed below:

Radiology

Development of training in:

• How to best to incorporate “case-based learning”1 in workshop

delivery

• Evidence based education for trainees

• How to critically appraise articles for trainees and Directors of

Training

• How to become an effective leader

• Transitioning from a trainee to a consultant to a practice owner

• New CPD triennium program need to be developed for 2013

1 Using a case-based approach engages trainees in discussion of specific situations, typically real-world examples. This method is learner-centred, and involves intense interaction between the participants. Case-based learning focuses on the building of knowledge where a group works together to examine a case. The instructor's (DoT or supervisor or senior clinician) role is that of a facilitator and the trainees collaboratively address problems from a perspective that requires analysis. Much of case-based learning involves learners striving to resolve questions that have no single right answer.


• Radiology Integrated Training Initiative (R-ITI)

All of the training development needs discussed were discussed with a

blended learning approach (online as well as face to face component) in

mind.

Radiation Oncology

• Development of modules for trainees in:

o radiation therapy planning

o statistics and research

o anatomy

o paediatrics

o pathology.

• Review of existing modules for trainees in: statistics and Phase 1 and

Phase 2 exam preparation.

• Development of CPD modules for rural and remote radiation

oncologists.

• Training in rudimentary counseling (e.g. breaking bad news, resilience

and coping).

• Policy and procedures development with associated support material

in two areas:

o Protected time

o Trainees in Difficulty

• New CPD triennium program needs to be developed for 2013.

All of the training development needs discussed were discussed with a

blended learning approach (online as well as face to face component) in

mind.


b) Secondary Findings

A number of secondary but key points also emerged from the TNA:

• The need for a cultural shift from a dual discipline view to a ‘single’

college view which happens to have dual disciplines.

• An understanding of how cultural shifts occur and how best to

facilitate them.

• The need for an understanding of change management and how best

to implement change in the College. This might include:

o More coordinated support, buy in and endorsement by the senior

stakeholders

o Support and commitment from all levels

o Support and commitment from hospitals and service providers

o A strong communication strategy to bring people on board, and

sell the benefits to the fellowship at large

o The need to learn from previous experience and processes

o Project planning and budget monitoring

o Managing expectations.

• The importance of clear communication in a variety of ways/settings to

ensure that target audience is reached.

• An understanding of the importance of identifying process and

procedures to facilitate good governance and build trust within the

College.

• The imperative for a Learning and Development Framework, a

process for ongoing review, a model of learning (following adult

learning process) and a movement from a reactive to a proactive

approach.


• All of those interviewed indicated that there is no formalised approach

to educational development; the situation is currently ad hoc and

reactionary.

• Those interviewed indicated that training is not necessarily linked to

service or the College needs.

• The need for a strategic approach was identified.

• The need for ‘champions’ of change amongst Fellows, Trainees and

College staff was identified to ensure equity and cohesion.

• The importance of having members take part in content development

as subject matter experts in collaboration with educationalists on staff

was highlighted.

• Members taking part in user acceptance testing prior to changes and

system being launched was identified.

• It is important to acknowledge that with any new initiative or change

comes resistance (i.e. from those uncomfortable with change or from

those who might feel the change will upset the balance of power) and

it is necessary to allay these fears through:

o Good cross functional communication within the College

o Good cross discipline communication within the College

o Stakeholder engagement.

• Understanding that all College members should be given the

opportunity to contribute to the development of training and

educational resources through:

o Feedback mechanisms.


7) RECOMMENDATIONS

On considering the findings of the TNA, a number of recommendations are

proposed:

1. That a blended learning approach be adopted – which provides learning

access via a mixture of mediums that could include ( a & b ) in the first

instance and ( c ) in the second instance:

a. Face to face workshops;

b. E-learning (online including e-learning modules, pod casts, videos,

PowerPoint slides);

c. Web based learning including webinars (i.e. interactive tutorials).

2. Given that the College is small with limited resources, it is necessary to

better utilise key events, such as the Annual Scientific Meeting (ASM) and

the NZ Branch Meeting, as a prime avenue in delivering face to face

workshops in identified key areas to Directors of Training (DoTs) and

trainees with a follow up with online resources to support key learning.

3. Given that the College has limited resources, collaboration with other

Colleges in the development of learning content for use by members and

trainees in necessary. Such collaborations as the impending collaboration

with RACS on an indigenous health initiative and the impending

collaboration with AIR and APSCEM on the development of training across

a range of topics.

4. That the College develops an online platform, such as a “Learning Portal”

(Appendix 1) which will allow access to courses through a Learning

Management System (LMS). An LMS is a software application for the

administration, documentation (resources), tracking and reporting of training

programs. All of this will provide a ‘one stop shop’ area for members with

access to:

a. CPD Module (for all fellows)

b. Courses (through the Learning Management System or LMS)

i. Induction (through the LMS)

ii. Resources


iii. Research Publications Library

iv. Course Profile

c. A “What’s happening” area (e.g. upcoming workshops, etc)

d. An important Messages area (e.g. imminent deadlines for CPD,

exams, courses)

e. Trainee Information Management System (for trainees and Directors

of Training)

f. Examination Management System (for examiners)

g. Re-validation for those re-entering the workforce after a period away

(through the LMS)

h. Social Networking tool (similar to LinkedIn)

It should be noted that a phased roll out of the above items should occur;

that is, the Learning Portal should be built slowly, with each new

functionality added, tested and consolidated before the next functionality is

added.

5. That a review and update of any existing online resources (e.g. rural CPD

modules in Radiology) be conducted, with a view to providing them online in

a format reflecting ‘best practice’ in e-learning and in accordance with the

Colleges ‘new look’ (as reflected in the new website).

6. That the non-medical expert roles of leader/manager, communicator, health

advocate, collaborator, professional, researcher/scholar be a focus in the

development of learning modules (both face to face and online) over the

next 3 years (refer Appendix 2).

7. That the College develops a library of educational resources methodically

and consistently with a review cycle of 3 years. A suggested approach is to

develop 2-3 learning modules per year in key areas (refer Appendix 2).

8. That new programs should be piloted before roll out or at the very least

have significant access to a small coterie of members (users) for testing

prior to roll out.


9. That the College moves to an online CPD system (post fellowship training

and development) where fellows can self-administer their CPD – submit

data on activities/events undertaken, print their own CPD statements,

provide verification/evidence etc.

10. That the College actively seek funding from the Department of Health and

Ageing (DoHA) and others to fund the development of educational

resources, recognising that funding bodies will only agree to fund items that

are consistent with their priorities

11. That from 2013, e-learning modules be charged at a subsidized rate of

between $50-$100 per participant, per course, thus, providing a revenue

stream to the College for further development of courses. This will have to

be clearly stipulated from the outset of any online launch.

12. Identify an Educational Resources Review Panel for each discipline and a

standard process to follow, so that there is continual review of existing and

introduction of new materials (and links) for the website and the Learning

Portal.

13. That the Learning and Development Framework2 for the College be based

on the CanMEDS model which has served as the basis for the new

curricula and the basis for the CPD triennium handbooks for both

disciplines (2010-2012).

2 The Learning and Development Framework will guide a broad range of activities designed to improve the capabilities

of the members within the College. This capability development will include not only clinical and technical skills and

knowledge development, but also attributes, attitudes and behaviours.


14.That a best practice learning and development approach incorporating adult

learning principles be adopted including:

SEE And hear (through various means including but not

limited to workshops, interactive tutorials, e-learning,

WIKIs).

LEARN Gain knowledge.

DO Practice, experience, and develop skills.

REFLECT Think about the knowledge/skill gained and ask:

• Can it be done better?

• How can it be done better?

• Could I have done it better?

• How can I improve in future?

The diagram on the following page best encapsulates the above.

15. That standarised templates in relation to course materials be adopted as

per below:

• Course/Activity/Event Code

• Course/Activity/Event Name

• Course/Activity/Event Duration

o If face to face workshop – venue details required

• CPD point attributed

SEE• and hear• through various means

LEARN• Gain knowledge

DO

• Practice, experience• Develop skill

REFLECT

• Think about knowledge/skill learned• How can I improve on it


• Short Description (50 characters long)

• Target Audience

• Long Description (250 characters long)

• Learning Objectives

o At the conclusion of this course/activity/event you will

o Understand…

o Know…

o Be able to…

o Note that no more than 4 learning objectives should be listed.

• Pre-work required

o Could include pre-reading (e.g. Case studies)

8) CONCLUSION

A TNA is only the first critical stage in any training cycle. Thus, a TNA is quite

simply a way of identifying the existing gaps in the knowledge and the strengths

and weaknesses in the processes that enable or hinder effective training

programs being delivered. It is very clear from the process outlined above that

organisational culture and change issues have a very large role to play in the

development of training process. From the findings, and subsequent 15

recommendations, of this TNA, it is essential that a coordinated approach be

followed in the development and facilitation of training and education, that a

‘blended learning approach’ be utilised (where the key learnings are re-iterated

in more than one medium, e.g. face to face with online module or webinar that is

recorded and provided as an online resource) and that e-Learning and online

access to training and CPD is essential.

Further to this, taking an ‘action research approach’ helped to highlight the

significant impact of the dynamics of hierarchy and the legacy of a bureaucratic

system has had on the way a system operates and how people react to change.

Through the action research process these issues came to the fore and the

application of theory helped provide understanding of what was happening and

pointed to a possible way forward.

It is also important to note that this was a small scale case study. The study has

focussed only on the initial stages of the needs analysis process: identifying the


needs, identifying the gaps, drafting an educational resources development

timetable. It was not possible, within the scope of this research, to follow the

process further as it is implemented across the organisation. This should be

done as a matter of process as the changes are implemented within the College.


9) REFERENCES

Anderson, G. (1994) ‘A Proactive Model for Training Needs Analysis’ Journal of European and Industrial Training, 18 (3.) p. 23-28. Altschuld, J. W., and Lepicki, T. L. (2009) “Needs assessment and education”. in Mathison, S (ed.) Encyclopedia of Evaluation Sage Publications, Thousand Oaks, CA Boydell, T. and Leary, M. (1996) Identifying Training Needs Institute of Personnel and Development, London, UK. Buckley, R. & Caple, J. (1995) ‘A Systematic Approach to Training’, in The Theory and Practice of Training. Kogan Page, London Chiu, W., Thompson, D., Mak, W. and Lo, K.L. (1999) "Re-thinking training needs analysis: A proposed framework for literature review". Personnel Review 28(1/2) p.77 - 90 Coghlan, D., & Brannick, T. (2001) Doing Action Research in Your Own Organization Sage Publications, London Garavan, T. N., Costine, P., and Heraty, N. (1995) Training and Development in Ireland: Context, Policy and Practice’, Oak Tree Press, Dublin Gould, D., Kelly, D., White, I. and Chidgey, J. (2004) “Training needs analysis: A literature review and reappraisal”. International Journal of Nursing Studies 41(5) pp.471-486 Gummesson, E. (2000) Qualitative Methods in Management Research. Sage Publications Inc., Thousand Oaks. Holton, E. F., Bates, R. A. and Naquin, S. S. (2000) ‘Large-Scale Performance Driven Training Needs Assessment: A Case Study’ Public Personnel Management Vol. 29(2) pp. 249-267 Iqbal, M. Z. and Khan, R.A (2011) "The growing concept and uses of training needs assessment: A review with proposed model", Journal of European Industrial Training 35(5) pp.439 - 466 Lee, Y-F., Altschuld, J. W. and White, J. L. (2007). “Effects of the participation of multiple stakeholders in identifying and interpreting perceived needs”. Evaluation and Program Planning 30(1), pp.1-9. Leat, M. J., and Lovell, M. J. (1997) ‘Training Needs Analysis: Weaknesses in the Conventional Approach’, Journal of European Industrial Training 21(4) pp. 143-153 Lundberg, C., Elderman, J. L., Ferrell, P. and Harper, L. (2010) “Data gathering and analysis for needs assessment: A case study”. Performance Improvement 49 pp.27–34


McClelland, S. B. (1993) ‘Training Needs Assessment: An Open Systems Application’, Journal of European Industrial Training 17(1) p 12-17. Reay, D. G. (1994) Identifying Training Needs Kogan Page Ltd, London Reed, J. and Vakola, M. (2006) "What role can a training needs analysis play in organisational change?" Journal of Organizational Change Management Vol. 19(3) pp.393 - 407 Reid, M. A., and Barrington, H. (1999) Training Interventions: Promoting Learning Opportunities (6th Edition) Chartered Institute of Personnel and Development, London, UK. Schneier, C., Guthrie, J and Olian, J. (1988) ‘A Practical Approach to Conducting and Using the Training Needs Assessment’, Public Personnel Management 17(2) p191-205. Wills, M. (1998) Managing the Training Process: Putting the Principles into Practice Gower, UK.


10) BIBLIOGRAPHY Altschuld, J. W., & Kumar, D. D. (2004) “Needs assessment”. in Mathison, S (ed.) Encyclopedia of Evaluation Sage Publications, Thousand Oaks, CA Anderson, G. (1994). A proactive model for training needs analysis. Journal of European Industrial Training, 18 (3): pp. 23-28. Arthur, W. Jr., Bennett, W. Jr., Edens, P.S., and Bell, S.T. (2003), “Effectiveness of Training in Organizations: A Meta-analysis of Design and Evaluation Features”. Journal of Applied Psychology 88 pp.234–245. James, B. (1956). “Can "needs" define educational goals”. Adult Education Quarterly 7(19) pp.19-26. Kaufman, R., and Watkins, R. (2000). Getting serious about results and payoffs: We are what we say, do, and deliver. Performance Improvement Journal 39(4), pp. 23–32. Molenaar, K., & Saller, B. (2003). Educational Needs Assessment for Design/Build Project Delivery. Journal of Professional Issues in Engineering Education & Practice 129(2), p. 106.


Appendix 1: Learning Portal


Matched Against Course/Activity/Event L&D Target Evaluation Findings Area Audience Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Indicators

*Trainee Profiles

*Progression Alerts

* Regular email reminders.

*E-portfolio

*Trend Analysis & Stats

a)iii, p.12 online access *Simulate current clinical workplace environments

a)v p13 Role clarification *Exam format to be more structured

a)iii, p.12 online access *On line solution to be implemented

a)v p13 Role clarification * Exam content to be expanded to include images and diagrams

a)iii, p.12 online access a)v p13 Role clarification a)I, p12 Adult Learning

Radiology E-Exams 6 Part 2 Viva Project Lead: Alison Reilly

ME Radiology Trainees

e-E *Electronic format provides opportunity for more complex cases (number of images and modalities for review) – quality of exam improved *Limited image manipulation functions also introduced

a)i, ii, iii, iv, pp12-13 Education & Research Training, CPD and Research content & resources migrated to website & to LMS General - both disciplines as well as discipline specific Project Lead: Sharon Messina

All All members

Web LMS Straight migration – no evaluation requirements

e-E

a)i, p12 Adult Learning *Current high degree of variability to be minimized

Radiology E-Exams 5 Part 2 MCQ/Short Answer Project Lead: Alison Reilly


e-E

a)i, p12 Adult Learning *Improved efficiency of database management and delivery of exam metrics

Radiology E-Exams 4 Part 2 Film Reporting Project Lead: Alison Reilly


TIMS

Migration of content only – no evaluation indicators required

a) iii, p.12 Trainee Information Management System (TIMS) 3 Online Project Lead: Madeleine d'Avigdor

All Trainees DoTs

LMS

LMS

a iii, pp12-13 Amalgamate Radiation Oncology Trainees’ Website Educational Resources with RANZCR Learning Management System (LMS) Project Leads: Sharon Messina & Madeleine d'Avigdor

ME Radiation Oncology Trainee

*High usage by all members. *Trend analysis and statistics.

a) ii, iii, iv, p.12-13 Learning Management System (LMS) 2 Online

All All members

LP

*All website resources ported over into LMS *Trend Analyses & statistics.

Appendix 2: DRAFT Resource Development Timetable (Please note that this is a living document and will change as time progresses)

2011 2012 2013 2014

Online Projects*70% of fellow will enter CPD online in 1st yr & gradual increase to 100% *Progression Alerts *Regular email reminders. *CPD Profiles *Point Certificates self-service *rend Analysis *Triennium Statistics

CPDa) ii, p.12 Continuing Professional Development (CPD) 1 Online Project Lead: Sharon Messina

All Fellows

a)ii, iii, vi, v p.12 b) L&D Framewk,p16

Learning Portal Online Project Lead: Sharon Messina

All All members



2011 2012 2013 2014

a)iii, p.12 online access a)v p13 Role clarification a)i, p12 Adult Learning

Radiation Oncology E-Exam 7 Phase 2 Project Lead: Alison Reilly


e-E * Content, format and delivery options yet to be resourced *Design will be in line with curriculum requirements

a) iii, p.12 Radiology Exam Case Library 8 Part 2 exams Project Lead: Alison Reilly


*Digital images required for both Film Reading and Viva Exams *Submission process implemented in order to capture all required patient and imaging data for exam preparation *CPD credit allocated for e-Cases submitted

a)i, p.12 a)RON discipline specific – breaking bad news TID Giving Feedback

Communication in a Clinical Setting Workshop 1,5hrs x 2 General - both disciplines Project Lead: Kelly Menzel

C All members

NZ BM ASM NZ BM ASM *LO’s in line with CanMEDS model *Subject expert LO’s yet to be decided Module 1 to include: 1) Foundations of Communication; 2) Giving Bad News to Patients;. Module 2 to include: 3) Providing Feedback to direct reports; 4) Trainees in Difficulty; 5) How to Influence Upwards.

a)i, p.12 a)RON discipline specific – breaking bad news TID Giving Feedback

Communication in a Clinical Setting e-learning modules x 5 General - both disciplines Project Lead: Kelly Menzel

C All members

LMS *LO’s in line with CanMEDS model *Subject expert LO’s yet to be decided *Modules to include: 1) Foundations of Communication; 2) Giving Bad News to Patients; 3) Providing Feedback to direct reports; 4) Trainees in Difficulty; 5) How to Influence Upwards.

a)i,p1 - Advocacy Patient as Customer e-learning module General - both disciplines Project Lead: Kelly Menzel

HA All members

LMS *LO’s to be in line with CanMEDS model *Best practice and evidence based

a)i, p12 *Clinical L'ship *Interacting with trainees *Effective L'ship *Strategic Planning *Change Mgt

Leadership Essentials Workshop General - both disciplines Project Lead: Kelly Menzel

M Fellows ASM NZ BM ASM NZ BM ASM In line with CanMEDS model.

a)i, p12 *Clinical L'ship *Interacting with trainees *Effective L'ship *Strategic Planning *Change Mgt

Leadership Essentials e-learning modules x4 General - both disciplines Project Lead: Kelly Menzel

M Fellows LMS In line with CanMEDS model. Modules might include: 1) Transitioning from Manager to Leader; 2) Different Leadership Styles; 3) Creating a Vision; 4) Motivating your team.

a)RON discipline specific finding p15

Statistics Review Radiation Oncology 2 x Workshops, conferences, other Project Lead: Bianca Heggelund (Kelly Menzel)

R/S Radiaiton Oncology Trainee

LMS Requirements in development stage: Subject specialists developing.

a)RON discipline specific, p15 Identified prior to TNA

Research Requirements Review Radiation Oncology workshop, other Project Lead: Bianca Heggelund (Kelly Menzel)

R/S Radiation Oncology Trainee

LMS Requirements in development stage: Points system currently being considered.

Workshops and e-learning Modules Non-Medical Expert- Both Disciplines



2011 2012 2013 2014

a) iii p.12 Dean’s Fellowship Radiology Scholarship for PhD Project Lead: Kelly Menzel

R/S 5th year Radiology trainee

Web LMS

Sydney University to monitor candidate progress in line with universityprotocol

a)RAD discipline specific p14 CATs Identified & implementation commenced prior to TNA

CATs (Critically Appraised Topics) Radiology Project Lead: Alison Reilly

R/S Radiology Trainees in 1st instance

Web LMS *Develop the use of EBM in training and clinical practice in radiology *Radiology trainees to analyse, explain and present findings from a journal article. *Develop the skills to appraise the quality and applicability of publications

a)RAD discipline specific p14 CATs Identified & implementation commenced prior to TNA

CATs Radiology Workshop Project Leads: Kelly Menzel, Bianca Heggelund

R/S Radiology DoTs Clinical Supervisors

* Understand critical appraisal process *Understand the epidemiological principles behind various study types and measures of association. *Recognise various study types and potential pitfalls

a)RAD discipline specific, p14 - Identified & implemented Apr 2011

Evidence Based Medicine (EBM) Radiology e-learning module Project Leads: Kelly Menzel, Bianca Heggelund

R/S Radiology Trainees

*Bi-annual delivery of 12 week online EBM course for radiology trainees *Trainees to understand the basic concepts and skills required for the critical appraisal of clinical studies

a)RAD discipline specific,p14

Considering a Private Practice? Radiology e-learning modules Project Lead: Kelly Menzel

M Radiology Fellows

LMS Ongoing review of external providers: using appropriate adult learning techniques. Modules include: 1) Establishing a business case for new practice (market analysis); 2) Facilities/Equipment; 3) Marketing your Practice; 4) Financial Planning for a Practice; 5) Managing the Practice: Strategic Planning, Operational Planning, Risk management, Staff management).

a)i, ii, iii, iv, pp12-13 Australian Indigenous Health & Cultural Safety 9 Online portal General - both disciplines Project Lead: RACS with Kelly Menzel College rep

HA All members

*LO’s to be decided by steering Group from June 2011. *Focus on cultural competency models and transcultural competence models

a)viii, p14 Induction programs

IMG Assessors' Workshop IMG Radiology IMG Radiation Oncology Project Lead: Susan Nicols

ME IMG Assessors

work-shop

Full day workshop which will lead participants to: *Know who to contact at the College Office regarding IMG assessments and role of IMG Committee *Perform AON and specialist comparability assessments of IMGs using the guidelines and templates provided according to College policies *Identify the pathways to Fellowship for IMGs inAustralia & NZ *Know their role as an IMG assessor

Workshops and e-learning Modules Medical Expert Both Disciplines



2011 2012 2013 2014

a)RON discipline specific finding, p15 Radiotherapy planning

Radiotherapy Planning 10 Radiation Oncology webinar (inter-active workshop online) Pilot 6 webinars Project Lead: Kelly Menzel (TEDS Committee)

ME Trainee LMS *LO’s in line with CanMEDS model. *Steering Committee to decide on best practice/evidence based LO's


Radiotherapy Planning 11 Radiation Oncology webinar (inter-active workshop online) Additional 6 webinars Project Lead: Kelly Menzel (TEDS Committee)

ME Trainee LMS *LO’s in line with CanMEDS model. *Steering Committee to decide on best practice/evidence based LO’s


Radiotherapy Planning 12 Radiation Oncology webinar (inter-active workshop online) Additional 5 webinars Project Lead: Kelly Menzel (TEDS Committee)

ME Trainee LMS *LO’s in line with CanMEDS model. *Steering Committee to decide on best practice/evidence based LO’s

a)RON discipline specificfinding p15

Paediatrics Radiation Oncology workshop Project Lead: Bianca Heggelund

ME Trainee work-shop

work-shop

LMS Requirements in development stage: Subject specialists developing

a)i How to turn curriculum to practice

Radiation Oncology Workshop - Feedback on Curriculum & Practice Project Lead: Tindal Magnus

ME DoTs & Trainees

Visits to all major centres for feedback on Curriculum and current training practice

a)RON discipline specific finding, p15 brachytherapy

Brachytherapy Radiation Oncology workshop Project Lead: Bianca Heggelund (TEDS Committee)

ME Trainee *Have yet to discuss with ABG - the Australian Brachytherapy Group. *Details to come.

Workshops and e-learning Modules Medical Expert Radiation Oncology



2011 2012 2013 2014

a)i, p12 Support for Exams Understanding: *Assessment techniques *Assessment methods *Evaluation of results

a) iii, p12 Review of CPD Rural Programs x 6 13 Radiology Project Lead: Kelly Menzel

ME Fellows LMS In line with CanMEDS model. *Evidence based and best practice in relation to practice in rural settings.

a)RAD discipline specific finding p.14

R-ITI Workshops *Using R-ITI to stregthen training" Project Lead: Tindal Magnus

ME DoTs Trainees

*Ongoing evaluation.


Accreditation Officer Workshop Radiology Project co-Lead: Susan Nicols & Tindal Magnus

ME Training Site Assessors

work-shop

Full day workshop which will lead participants to: *Know who to contact at the College Office regarding Training Site accreditation and role of Accreditation Committee *Perform Training Site Accreditation visits using the guidelines and templates provided according to College policies *Know their role as an Accreditation Assessor


Accreditation Officer Workshop Radiation Oncology Project co-Lead: Susan Nicols & Tindal Magnus

ME Training Site Assessors

work-shop

Full day workshop which will lead participants to: *Know who to contact at the College Office regarding Training Site accreditation and role of Accreditation Committee *Perform Training Site Accreditation visits using the guidelines and templates provided according to College policies *Know their role as an Accreditation Assessor

a)i, p11 RON discipline specific TID

Trainee in Difficulty Policy Radiation Oncology Project Lead: Bianca Heggelund

M DoT Trainee

Web Policy so none required

a)viii, p14 Protected Time Policy/guideline Radiation Oncology Project Lead: Bianca Heggelund

M Fellows DoTs


a) vii, p14 Recency of Practice Guideline (for those re-entering the workforce after a period away) Radiation Oncology Project Lead: Kelly Menzel

P Fellows Web Policy so none required

a) vii, p14 Recency of Practice Guideline (for those re-entering the workforce after a period away) Radiology Project Lead: Kelly Menzel

P Fellows Web Policy so none required

Policies, Guidelines and/or Position Statements

Examination Question Standard Setting Workshop Both Disciplines Project Lead: Tindal Magnus

ME Radiology & RadiationOncology Examiners

MCQ w/shop

Workshops and e-learning Modules Medical Expert Radiology



2011 2012 2013 2014

a) vii, p14 Prior Learning Policy/guideline Radiology Project Lead: Bianca Heggelund



a) vii, p14 Prior Learning Policy/guideline Radiation Oncology Project Lead: Bianca Heggelund

ME Radiation Oncology Trainees


a)RAD discipline specific, p14 CPD 2013-15

CPD 2013-2015 Triennium Program Radiology Project Lead: Freya St. Julian

All Radiology Fellows

CPD In line with CanMEDS model. *LO: To achieve required points across the triennium.

a)RON discipline specific, p15 CPD 2013-15

CPD 2013-2015 Triennium Program Radiation Oncology Project Lead: Freya St. Julian

All Radiation Oncology Fellows

CPD In line with CanMEDS model. *LO: To achieve required points across the triennium.

CPD Guideline Radiology Project Lead: Freya St. Julian

ME Radiology Fellows & Ed Affiliates

Policy so none required

CPD Guideline Radiation Oncology Project Lead: Freya St. Julian

ME Radiology Fellows & Ed Affiliates

Policy so none required

Identified prior to TNA New Zealand Vocational Practice Assessment Guideline Project Leads: Kelly Menzel & Gail LeClaire

P NZ Branch Policy so none required

a)iii,p12 online trainee resources

Establishment of the Trainee Educational Development Steering (TEDS) Committee Both Disciplines Project Co-Leads: Kelly Menzel & Bianca Heggelund

All Radiology & Radiation Oncology Trainees

Identify priorities for the sourcing and roll out of e-learning materialsCurriculum Development



2011 2012 2013 2014

a)iii online resources

Educational Resources Panel Both disciplines Project Co-Lead: Bianca Heggelund

ME Members *Educational Resources online are relevant and up-to-date

a)i, p12 Examination Panels Radiology (P1 and P2-TWERP) Project Lead: Tindal Magnus

ME Radiology Examiners

*Part 2 e-exam change mgt *MCQ questions relevant for exams. *Exam questions not repeated over a 4 year cycle. *Good quality and quantity of exam material

a)i, p12 Examination Panels (P1 & P2) Radiation Oncology Project Lead: Tindal Magnus

ME Radiation Oncology Examiners

*Questions relevant for exams. *Exam questions not repeated over a 4 year cycle. *Good quality and quantity of exam material

b)pp15-17 Cross Discipline Meetings Radiology & Radiation Oncology Project Lead: Sharon Messina

M *ER Unit Mgrs*ER Director*TAA Director*Chief Censors*Workforce Planning Mgr*C&M Director

*Communication across disciplines *Leveraging ideas off each discipline

College Communication


Notes:

LegendDevelop

Pilot/Test

Launch (release)

Ongoing

MCQ Multiple Choice Questions

LO Learning Objective(s)

LP Learning Portal

TIMS Trainee Information Management System

CPD CPD Module

LMS Learning Management System

e-E e-Exams Radiology

WEB RANZCR Website

ASM RANZCR Annual Scientific Meeting

NZ BM New Zealand Branch Meeting

HA CanMEDS model – Health Advocate

C CanMEDS model - Communicator

M CanMEDS model - Manager

P CanMEDS model - Professional

R/S CanMEDS model – Researcher/Scholar

T/C CanMEDS model – Team Work/Collaborator

ME CanMEDS model – Medical Expert

10) These tutorials will be designed in collaboration with physicists, radiation therapists and radiation oncologists and will be facilitated by 3 people (1 physicist, 1 radiation therapist and 1 radiation oncologist). There will be 2 groups (one in Sydney based at the RPA and one in Melbourne based at Peter Mac) who will design 6 interactive tutorials on radiation planning as a pilot. If the pilot is successful 11 more tutorials will be designed. Much of the content is available, but the IT platform has not been explored.

11 & 12) As above. 13) Depending on outcome of the review, some or all of the 6 Radiology Rural CPD programs may be placed within the LMS.

14) It has come to our attention that there are some pharmaceutical and equipment companies who may consider sponsoring e-learning modules for CPD purposes. The College is currently exploring this possibility and will only accept these sponsorships for CPD program development on the proviso that any content developed must be impartial (will not promote a particular product).

* Learning Objectives in line with CanMEDS model. Steering Committee to decide on best practice/evidence based Learning Objectives.

4) Radiology e-exams (film reporting) timeline for launch is dependent on completion of requirements analysis (which has not yet commenced and is not expected to commence until mid to late August 2011). This date is tentative.

5) Radiology e-exams (MCQs/Short Answer) timeline for launch is dependent on completion of requirements analysis (which has not yet commenced and is not expected to commence until mid to late August 2011). This date is tentative.

6) Radiology Part 2 Viva timeline for launch is dependent on completion of requirements analysis (which has not yet commenced and is not expected to commence until late 2011). This date is tentative.

7) Radiation Oncology Part 2 e-exams timeline is dependent on resources; at the moment the e-Resources Manager is spending the first 2 years on Radiology e-exams after from which she will be able to leverage learnings to Radiation Oncology. She will attend Part 2 Viva exams in August 2011 to just observe, but no further development on digitizing for Radiation Oncology will occur until 12-18 months later.8) Radiology Case Library is for e-exam use only and the timeline for launch is dependent on the completion of the requirements analysis (which has not yet commenced). This date is tentative.

9) In collaboration with RACS – discussion yet to be had. Expected outcome is: Establish a portal to support the access of medical specialists to e-learning activities related to cultural competency and the health of Indigenous Australian and Torres Strait Islanders; Identify appropriate online links to professional development activities within the College. Not sure of timeline and steering committee has not met yet.

1) CPD Module timeline for launch is dependent on completion of requirements analysis (which has not yet been finalized). This date is tentative.2) LMS timeline for launch is dependent on completion of requirements analysis (which has not yet commenced). This date is tentative.

3) TIMS timeline for launch is dependent on completion of requirements analysis (which has not yet commenced and is not expected to commence until mid to late July 2011). This date is tentative.


Appendix 3: Acronyms

AMC AHPRA ASM CAC CDM CPD DOT EBM ER FRC FRO LMS PFEC REB ROEB ROTC RRC RTC TIMS TNDF

Australian Medical Council Australian Health Practitioner Regulation Agency Annual Scientific Meeting Curriculum Assessment Committee (RAD) Cross Discipline Meeting Continuing Professional Development Director of Training Evidence Based Medicine Education and Research Faculty Research Committee (R/O) Faculty of Radiation Oncology Learning Management System Post Fellowship Education Committee (R/O) Radiology Education Board Radiation Oncology Education Board Radiation Oncology Trainees’ Committee Radiology Research Committee Radiology Trainees’ Committee Trainee Information Management System Training Network Directors Forum (R/O


This page is intentionally left blank

THE ROYAL AUSTRALIAN AND NEW ZEALAND COLLEGE OF RADIOLOGISTS®


Documents

Training Needs Analysis 4Aug