DOCTRINE AND TRAINING SERIES ADFP 7.0.2 THE DEFENCE ... · Management responsibilities 1.17 THE ACCELERATED APPLICATION OF THE DEFENCE TRAINING MODEL Risk management 1.26 A RAPID

ADFP 7.0.2

DOCTRINE AND TRAINING SERIES

ADFP 7.0.2

THE DEFENCE TRAINING MODELAustralian Defence Force Publication 7.0.2 is issued for use by the Australian Defence Force and is effective forthwith.

A.G. HOUSTON, AO, AFCAir Chief MarshalChief of the Defence Force

Australian Defence HeadquartersCanberra ACT 2600

14 July 2006

© Commonwealth of Australia 2007

This work is copyright. Apart from any use as permitted under the Copyright Act 1968, no part may be reproduced by any process without prior written permission from the Australian Government Department of Defence.

Announcement statement—may be announced to the public.

Secondary release—may be released to the Australian Government Department of Defence and its equivalent in the United States of America, United Kingdom, Canada, and New Zealand.

All Defence information, whether classified or not, is protected from unauthorised disclosure under the Crimes Act 1914. Defence information may only be released in accordance with the Defence Security Manual and/or Defence Instruction (General) OPS 13–4—Release of Classified Defence Information to Other Countries, as appropriate.

The Commandant of the Australian Defence Force Warfare Centre is the approving authority for the release of Unclassified joint doctrine publications to countries not covered by the secondary release statement.

ADFP 7.0.2First edition 2006

Sponsor Chief of Joint OperationsHeadquarters Joint Operations Command

Developer CommandantAustralian Defence Force Warfare CentreRAAF BaseWILLIAMTOWN NSW 2314

Publisher Defence Publishing ServiceDepartment of DefenceCANBERRA ACT 2600

Defence Publishing ServiceDPS: March/2007

ADFP 7.0.2

FOREWORD1. Australian Defence Doctrine Publications (ADDP) and Australian Defence Force Publications (ADFP) are authorised joint doctrine for the guidance of Australian Defence Force (ADF) operations. ADDP are pitched at the philosophical and high-application level, and ADFP at the application and procedural level. Policy is prescriptive as represented by Defence Instructions and has legal standing. Doctrine is not policy and does not have legal standing, however, it provides authoritative and proven guidance which can be adapted to suit each unique situation.

2. We have an unquestionable mandate from the nation to maintain and enhance Defence capability to meet whatever challenges lay ahead. Quality training is the foundation from which we will develop the people we need today with the capability to meet tomorrow’s challenges and uncertainty. The Defence training model (DTM), for the first time, provides us with a common approach to quality vocational education and training (VET) across all the Services.

3. The DTM is based upon an update of the tried and proven systems approach to training with additional enhancements to better enable our trainers to deal with the increased tempo and uncertainty that are likely to remain features of our environment. These enhancements include a Rapid Development and Risk Management Process to better facilitate the timely availability of training resources when short notice training requirements arise.

4. The DTM is also aligned with the National Training Framework (NTF) so that Defence can take full advantage of national education and training systems and initiatives while reinforcing our already strong focus on quality assurance and continuous improvement. Alignment with the NTF also supports national recognition for ADF VET thereby aiding the recruitment, retention and eventually, transition back into civilian employment of our people.

5. This publication is the result of the efforts of many subject matter experts and trainers across Defence. The DTM is a significant milestone in our continual quest to improve the quality and efficiency of the training that is so vital to our work at home and abroad.

6. ADFP 7.0.2 embraces complex but complementary subject matter. The reader should therefore consult the following publications, with which ADFP 7.0.2 is congruous:

• ADDP–D—Foundations of Australian Military Doctrine, July 2005; and

• ADDP 7.0—Doctrine and Training, January 2006.

iii

ADFP 7.0.2

7. Every opportunity should be taken by the users of this publication to examine constructively its contents, applicability and currency. If deficiencies or errors are found, amendment action should be taken. Australian Defence Force Warfare Centre (ADFWC) welcomes any assistance or input from whatever source in order to improve this publication.

iv

ADFP 7.0.2

AMENDMENT CERTIFICATEProposals for amendment of ADFP 7.0.2 may be initiated in either of the following ways:

• By Minute to:

Director Doctrine and TrainingAustralian Defence Force Warfare CentreRAAF BaseWILLIAMTOWN NSW 2314

• By directly entering comment into the Joint Doctrine Development Environment (JDDE) found on the ADFWC Defence Restricted Network (DRN) web site (see http://intranet.defence.gov.au/VCDFweb/sites/adfwc/). Select JDDE on the ADFWC homepage and open either the ADDP or ADFP block as required. Open the relevant publication and utilise the ‘Add Comment’ function button of the summary page for each publication.

Note

The second option is an addition to encourage feedback from the wider ADF, as well as encouraging use of the JDDE in general.

Amendment Effected

Number Date Signature Date

This publication is current as at July 2006.

This publication will be periodically reviewed and amended. The latest version of this publication is available on the ADFWC DRN web site http://intranet.defence.gov.au/VCDFweb/sites/adfwc/.

v

http://intranet.defence.gov.au/VCDFweb/sites/adfwc/



ADFP 7.0.2

DOCTRINE PUBLICATION HIERARCHYThe hierarchy of ADDP and ADFP and the latest electronic version of all ADDP and ADFP are available on the JDDE found on the ADFWC DRN web site located at http://intranet.defence.gov.au/VCDFweb/sites/adfwc/.

vi


ADFP 7.0.2

CONTENTSPage

Foreword iiiAmendment Certificate vDoctrine Publication Hierarchy viContents viiList of Figures xvList of Tables xvii

Paragraph

CHAPTER 1 INTRODUCTION

OVERVIEW OF THE DEFENCE TRAINING MODEL

Introduction 1.1Phases of the Defence training model 1.5Management of the Defence training model 1.12GOVERNANCE AND MANAGEMENTThe role of the Defence training model in the

management of Defence training 1.13Application of the Defence training model 1.15Governance responsibilities 1.16Management responsibilities 1.17THE ACCELERATED APPLICATION OF THE

DEFENCE TRAINING MODELRisk management 1.26A RAPID DEVELOPMENT PROCESSCONTINUOUS IMPROVEMENT AND

QUALITY ASSURANCEPROCURING TRAINING AND RESOURCES

EXTERNAL TO DEFENCE

Annex:A. An example of a rapid development process

CHAPTER 2 THE ‘ANALYSE’ PHASE

Introduction 2.1Starting point 2.3Processes of the ‘Analyse’ phase 2.4Continuous improvement 2.12Key outputs and responsibilities 2.13Use of an external agency 2.15CONDUCT NEEDS ASSESSMENTInputs, process and products 2.24

vii

ADFP 7.0.2

Step 1: Establish the Terms of Reference 2.25Step 2: Determine information requirements 2.26Step 3: Plan and gather information 2.28Step 4: Collate and analyse information to make

recommendations 2.31Step 5: Report and record 2.32ANALYSE THE JOBInputs, process and products 2.38Step 1: Establish scope of job analysis 2.39Step 2: Determine information requirements 2.41Step 3: Plan and implement information collection 2.45Step 4: Collate and analyse information 2.50Step 5: Develop job specification 2.51Step 6: Gain authorisation of job specification and

disseminate 2.53ANALYSE TARGET POPULATIONInputs, process and products 2.59Step 1: Determine the target population 2.60Step 2: Determine information requirements 2.61Step 3: Plan and gather information 2.62Step 4: Collate and analyse information 2.66Step 5: Develop a Target Population Profile 2.67DETERMINE PERFORMANCE GAPInputs, process and products 2.72Step 1: Determine scope of performance gap 2.73Step 2: Determine scale of performance gap 2.74Step 3: Prepare a Performance Gap Statement 2.75CONDUCT FEASIBILITY ANALYSISInputs, process and products 2.82Step 1: Identify options for ensuring job holder

competence 2.83Step 2: Investigate options for ensuring job holder

competence 2.89Step 3: Compare options for ensuring job holder

competence 2.91Step 4: Identify options for assessing competence 2.94Step 5: Investigate and compare options for assessing

competence 2.96Step 6: Report the feasibility analysis and file 2.99SPECIFY TRAINING REQUIREMENTInputs, process and products 2.105Step 1: Identify and apply existing training

documentation 2.106Step 2: Customise existing training documentation 2.109

viii

ADFP 7.0.2

Step 3: Develop new competency standards 2.110Step 4: Setup competency details in Personnel

Management Key Solutions 2.112Step 5: Determine training context and accreditation

action 2.114Step 6: Develop a Training Requirement Specification 2.121Step 7: Gain formal approval of the Training

Requirement Specification, action and file 2.122

Annexes:A. Flow chart showing the ‘Analyse’ phase of

the Defence training modelB. Defence training model—purpose, inputs,

process and products of the ‘Analyse’ phaseC. Defence job specification—templateD. Target Population Profile—templateE. Target Population Capability Record—

templateF. Performance Gap Statement—templateG. Defence Feasibility Report—templateH. Training Requirement Specification—

template

CHAPTER 3 THE ‘DESIGN’ PHASE

Introduction 3.1Key product and responsibilities 3.4Starting point 3.6Processes of the ‘Design’ phase 3.8Continuous improvement 3.18Use of an external agency 3.19ANALYSE LEARNING AND ASSESSMENT

ENVIRONMENTSIntroduction 3.20Inputs, process and products 3.26Step 1: Determine information requirements 3.27Step 2: Plan, prepare and implement information

collection 3.30Step 3: Collate and analyse information 3.32Step 4: Develop and confirm environments profile 3.33ANALYSE PROSPECTIVE LEARNERSIntroduction 3.35Inputs, process and products 3.40Step 1: Determine information requirements 3.41Step 2: Plan, prepare and implement information

collection 3.43

ix

ADFP 7.0.2

Step 3: Collate and analyse information 3.45Step 4: Develop and confirm learner profile 3.46ANALYSE COMPETENCIESIntroduction 3.48Inputs, process and products 3.53Step 1: Analyse competencies to complete

Competency Analysis Sheets 3.54Step 2: Check completed Competency Analysis

Sheets 3.66GENERATE LEARNING OUTCOMES Introduction 3.67Inputs, process and products 3.72Step 1: Collect inputs 3.73Step 2: Determine required performances 3.75Step 3: Complete learning outcomes and assign

course learning outcome levels 3.83Step 4: Write outcome assessment methods 3.89Step 5: Check learning outcomes and file 3.92DESIGN ASSESSMENT STRATEGIES Introduction 3.94Inputs, process and products 3.100Step 1: Determine assessment needs and draft the

strategy 3.104Step 2: Select assessment methods 3.112Step 3: Refine the assessment strategy 3.116Product—input to a training and assessment strategy

document 3.122DESIGN TRAINING STRATEGY AND

PRODUCE THE TRAINING AND ASSESSMENT STRATEGY DOCUMENT

Introduction 3.123Inputs process and products 3.127Step 1: Select training delivery methods 3.129Step 2: Apply other principles of learning theory 3.137Step 3: Select associated media 3.138Step 4: Combine training and assessment strategies 3.143Step 5: Develop course map 3.146Step 6: Complete training and assessment strategy 3.148Step 7: Gain approval 3.150COMPILE CURRICULUMIntroduction 3.151Inputs, process and products 3.155Step 1: Compile curriculum 3.156Step 2: Confirm curriculum 3.157

x

ADFP 7.0.2

DESIGN TRAINING REVIEW AND EVALUATION STRATEGIES

Introduction 3.158Inputs, process and products 3.163Step 1: Determine need for training review and

evaluation 3.170Step 2: Determine information requirements 3.176Step 3: Select information sources and collection

methods 3.181Step 4: Record the training review and evaluation

strategies 3.184COMPILE DRAFT TRAINING MANAGEMENT

PACKAGEIntroduction 3.185Inputs, process and products 3.190Step 1: Compile Training Management Package 3.191Step 2: Gain formal endorsement and approval of

draft Training Management Package 3.192Step 3: Submit for accreditation 3.194

Annexes:A. Flow chart showing the ‘Design’ phase of the

Defence training modelB. Defence training model—purpose, inputs,

process and products of the ‘Design’ phaseC. National assessment principles and their

essential featuresD. Factors to consider when selecting

appropriate training modesE. Some criteria for selecting appropriate

training activitiesF. Guidance for estimating the costs and times

of developing new training materialsG. Model course map and explanationH. Environments profile—templateI. Learner profile—templateJ. Competency analysis sheet—templateK. Training and assessment strategy—

templateL. Training review and evaluation strategies—

templateM. Training management package—indicative

template

xi

ADFP 7.0.2

CHAPTER 4 THE ‘DEVELOP’ PHASE

Introduction 4.1Key products and responsibilities 4.3Starting point 4.5Processes of the ‘Develop’ phase 4.6Continuous improvement 4.12Use of an external agency 4.13CHECK EXISTING TRAINING MATERIALS

AND EQUIPMENTIntroduction 4.14Inputs, process and products 4.18Step 1: Identify requirements 4.19Step 2: Research existing Defence materials and

equipment 4.22Step 3: Research external sources 4.23Step 4: Decide and record whether to acquire existing

or develop new materials and equipment 4.24OBTAIN MATERIALS AND EQUIPMENT FOR

ASSESSMENTS AND TRAININGIntroduction 4.25Inputs, process and products 4.28Step 1: Obtain training materials and equipment 4.29Step 1.1: Obtain existing materials and equipment 4.31Step 1.2: Customise materials and equipment 4.32Step 1.3: Produce materials and equipment 4.35Step 1.4: Formatively evaluate materials and

equipment 4.38Step 2: Add materials to draft Training Management

Package 4.42PREPARE MAJOR TRAINING RESOURCESPILOT ASSESSMENTS AND TRAINING Introduction 4.47Inputs, process and products 4.52Step 1: Plan the pilot assessment and/or training 4.53Step 2: Plan and prepare the evaluation of the pilot

assessment and/or training 4.55Step 3: Implement the assessment and/or training 4.61Step 4: Review, report and follow-up the pilot

assessment and/or training 4.63Step 5: Approval of Training Management Package 4.66Step 6: Set up the course 4.67

Annexes:A. Flow chart showing the ‘Develop’ phase of

the Defence training model

xii

ADFP 7.0.2

B. Defence training model—purpose, inputs, process and products of the ‘Develop’ phase

CHAPTER 5 THE ‘CONDUCT’ PHASE

Introduction 5.1Key product and responsibilities 5.4Starting point 5.6Processes of the ‘Conduct’ phase 5.7Continuous improvement 5.16Use of an external agency 5.17ADMINISTER TRAINING PREPARATIONIntroduction 5.18Inputs, process and products 5.21Step 1: Promote the training 5.22Step 2: Training nominations and panelling 5.25Step 3: Prepare major resources 5.28Step 4: Prepare materials and equipment 5.30Step 5: Manage course sessions 5.32IMPLEMENT PRE–TRAINING

ASSESSMENTS/TESTSIntroduction 5.36Inputs, process and products 5.38Step 1: Assess/test trainees 5.41Step 2: Evaluate the evidence 5.44Step 3: Instigate appropriate action 5.45CONDUCT TRAININGIntroduction 5.46Inputs, process and products 5.50Step 1: Prepare training activity 5.53Step 2: Deliver training activity in accordance with

the plan 5.55Step 3: Monitor trainees’ progress and adjust training

as required 5.57CONDUCT ASSESSMENTIntroduction 5.58Inputs, process and products 5.62Step 1: Prepare summative assessments 5.63Step 2: Conduct summative assessments 5.66Collect the evidence 5.67Evaluate the evidence 5.69Confirm the results 5.71Step 3: Evaluate assessment conduct methodology

in accordance with evaluation plan/strategy 5.72Moderation 5.79

xiii

ADFP 7.0.2

POST–COURSE ADMINISTRATION Introduction 5.80Inputs, process and products 5.82Step 1: Record and report outcomes 5.83Step 2: Instigate training failure management 5.84Step 3: ‘Housekeeping’ 5.85Step 4: Issue course attendance certificates/records

of attainment 5.86CONDUCT TRAINING REVIEW (INTERNAL

EVALUATION)Introduction 5.88Inputs, process and products 5.91Step 1: Plan and prepare training reviews 5.95Step 2: Collect, collate and analyse training review

information 5.99Step 3: Report training review 5.104

Annexes:A. Flow chart showing the ‘Conduct’ phase of

the Defence training modelB. Defence training model—purpose, inputs,

process and products of the ‘Conduct’ phase

CHAPTER 6 THE ‘EVALUATE’ PHASE

Introduction 6.1Key products and responsibilities 6.4Steps and products of the ‘Evaluate’ phase 6.7Step 1: Plan and prepare the evaluation 6.9Step 2: Collect and collate evaluation information 6.21Step 3: Analyse and synthesise evaluation

information/data 6.24Step 4: Report evaluation 6.27

Annexes:A. Flow chart showing the ‘Evaluate’ phase of

the Defence training model B. Defence training model—purpose, inputs,

process and products of the ‘Evaluate’ phase

Glossary

Acronyms and Abbreviations

xiv

ADFP 7.0.2

LIST OF FIGURESFigure Title Page

1–1 Phases of the Defence training model 1–3

2–1 ‘Analyse’ phase of the Defence training model 2–12–2 Hierarchical structure of work 2–102–3 Algorithm illustrating the decision-making process 2–32

3–1 ‘Design’ phase of the Defence training model 3–13–2 Indicative relationships between jobs, competencies,

learning outcomes, and training 3–203–3 Training review, evaluation and the training process 3–423–4 Kirkpatrick’s model for evaluating training programs

‘The Four Levels’ 3–43

3G–1 Course map 3G–1

3K–1 Course map 3K–5

4–1 ‘Develop’ phase of the Defence training model 4–1

5–1 ‘Conduct’ phase of the Defence training model 5–15–2 Critical questions in the review of assessment

processes 5–18

6–1 ‘Evaluate’ phase of the Defence training model 6–1

xv

ADFP 7.0.2

LIST OF TABLESTable Title Page

2–1 Inputs, process and products of the conduct needs assessment process 2–6

2–2 Inputs, process and products of the job analysis process 2–11

2–3 Inputs, process and products of the analyse target population process 2–16

2–4 Inputs, process and products of the performance gap process 2–19

2–5 Inputs, process and products of the feasibility analysis process 2–21

2–6 Inputs, process and products for specifying the training requirement 2–27

3–1 Inputs, process and products of a competency analysis 3–7

3–2 Inputs, process and products of a learner analysis 3–123–3 Inputs, process and products of a competency analysis 3–163–4 Inputs, process and products of learning outcomes 3–193–5 Course learning outcome levels 3–233–6 Inputs, process and products for designing an

assessment strategy 3–263–7 Inputs, process and products for designing a training

strategy 3–333–8 Inputs, process and products for compiling curriculum 3–413–9 Inputs, process and products for designing training

review and evaluation strategies 3–433–10 Inputs, process and products for compiling a draft

Training Management Package 3–48

3L–1 Training review strategy 3L–23L–2 Evaluation strategy 3L–3

4–1 Inputs, process and products for checking existing training materials and equipment 4–4

4–2 Inputs, process and products for obtaining training materials and equipment 4–6

4–3 Inputs, process and products for pilot assessments and training 4–12

xvii

ADFP 7.0.2

5–1 Inputs, process and products of administering training preparation 5–6

5–2 Inputs, process and products for implementing pre-testing/assessments 5–9

5–3 Inputs, process and products of conducting training 5–125–4 Inputs, process and products for conducting

assessment 5–155–5 Inputs, process and products for post-course

administration 5–205–6 Inputs, process and products for conducting training

review 5–21

xviii

ADFP 7.0.2 Chapter 1

CHAPTER 1

INTRODUCTION 1

OVERVIEW OF THE DEFENCE TRAINING MODEL 1.1

Introduction 1.1

1.1 This chapter provides an overview of the Defence training model (DTM). In addition it introduces the phases of the model and significant features of their management.

1.2 Terminology. Much of the terminology used in this publication is national terminology, defined by the Department of Education, Science and Technology (DEST) for application within the National Training Framework (NTF).

1.3 Purpose. The purpose of the DTM is to achieve training that meets work force capability requirements effectively and efficiently. It does this by providing a common approach to work force performance problems, an approach that:

a. directs training towards workplace performance needs;

b. employs training strategies that reflect best practice, are feasible, and make the best use of available resources;

c. supports management of training implementation; and

d. bases decisions to improve training effectiveness and efficiency on evaluation information.

1.4 Underlying principles. The delivery of quality training in Defence and the content of the DTM are based on six principles:

a. Principle 1. All Defence personnel, particularly leaders and managers, are responsible for enhancing Defence capability by developing their people and themselves.

b. Principle 2. Defence training, education and other development experiences, develop Defence capability by preparing the work force for the roles, duties and tasks assigned and anticipated.

c. Principle 3. Defence training uses common terminology, processes and methods wherever appropriate.

1–1


d. Principle 4. Defence uses a systems approach that ensures training:

(1) is directly related to improving the skills, knowledge and attitudes required for Defence capability;

(2) is the most cost-effective intervention;

(3) uses learner-centred approaches to delivery; and

(4) is subject to continuous improvement1.

e. Principle 5. Defence training and assessment applies competency-based training and assessment (CBTA) in accordance with the NTF2 where practicable and cost-effective to do so.

f. Principle 6. Defence training and assessment is based on current, nationally endorsed training packages, where these are available and meet Defence requirements.

Phases of the Defence training model 1.5

1.5 The phases of the DTM are represented in figure 1–1. The figure shows a cyclic process structured to enable quality assurance and continuous improvement.

1 Continuous improvement processes should normally accord with the quality requirements of the Australian Quality Training Framework (AQTF) Standards for Registered Training Organisations (RTO).

2 The Defence application of CBTA in accordance with the NTF is described in Defence Instruction (General) (DI(G)) PERS 05–29—Implementation of the National Training Framework within Defence vocational education and training.

1–2


Figure 1–1: Phases of the Defence training model

1.6 The model represents the Defence approach to training. The realities of training are that:

a. the entry point for the cycle depends on the aspect of training under consideration;

b. work flows, particularly during the ‘Design’ and ‘Develop’ phases, can be two-way;

c. phases may overlap and may be addressed concurrently; and

d. the processes of continuous improvement overlap and link all phases.

1.7 ‘Analyse’ phase. The ‘Analyse’ phase of the DTM is when the need for training (new or revised) to improve human performance is analysed and specified. This phase incorporates six main processes. To begin, a needs assessment determines that training is likely to be required for modifying workplace performance. With this established, a job analysis identifies the characteristics of the job performance required, and a target population analysis identifies the characteristics of the group of people who are to undertake that performance. A comparison of the two sets of characteristics identifies the existing performance gap. A feasibility analysis is then undertaken to identify the best solutions for closing this gap. If training is a recommended solution, details of the required workplace performances are recorded in a Training Requirement Specification (this document becomes the starting point of the ‘Design’ phase of the DTM). If other solutions are recommended, they are reported to appropriate authorities.

1–3


1.8 ‘Design’ phase. The ‘Design’ phase of the DTM is when the assessment and learning strategies for training are selected, together with the strategies for evaluating the whole process of preparing for and conducting training. The object of these strategies is to ensure that the learning is achieved as efficiently and effectively as possible, and that it meets the original requirement. This phase begins with three analyses that may be undertaken concurrently. The competencies required for the job are analysed for their constituent skills, knowledge and attitudes, and the characteristics of both the likely learners and the circumstances in which assessment and learning could occur are identified. The products of these three analyses are used to generate the intended learning outcomes of the training. Once this end point is understood, strategies for both the assessing the achievement of these outcomes and the conduct of training are developed. These strategies are then combined to create a curriculum—the document that provides the specifications of the training to be conducted. With the curriculum established, strategies for the Training Review and Evaluation (TR&E) of the training can be developed. Finally, the strategies are combined to form the draft Training Management Package (TMP).

1.9 ‘Develop’ phase. In the ‘Develop’ phase of the DTM, the materials and equipment needed to conduct training and evaluate trainee performance are prepared in accordance with the agreed strategies, and a trial of the whole program/course is conducted. Important activities within this phase are the development of assessment and training materials, the compilation and development of learning materials and equipment (including technology-based aids), and the preparation of TR&E materials. The optimal sequence of learning activity is also mapped and learning facilitators/trainers and assessors are appropriately prepared. When ready, the developed training is tested and, where necessary, revised. The major product of the ‘Develop’ phase is a fully developed and tried program/course presented as a TMP.

1.10 ‘Conduct’ phase. The ‘Conduct’ phase of the DTM involves the ongoing implementation of training and associated assessment, and the training review (sometimes referred to as internal evaluation) processes. Fully developed training courses are programmed, administered, conducted, assessed and recorded using the materials and equipment prepared in the ‘Develop’ phase (notwithstanding this, within the parameters of the training documentation, learning staff may adapt the materials to suit their situation). Training may be conducted in training units, the workplace or other locations. The ‘Conduct’ phase can incorporate recognition processes to determine whether individuals already have the required competencies, proficiencies or other attributes.

1–4


1.11 ‘Evaluate’ phase. Evaluation is the process of checking how well training has prepared learners for the workplace and whether the original capability requirement has been met. Evaluation identifies under- and over-training, the strengths and shortfalls of training, and makes recommendations for improvement. These recommendations thereby become inputs to the ‘Analysis’ and other phases for the next cycle of the DTM. Evaluation incorporates four processes: planning the work, collecting information from the job environment, analysing and synthesising that information to identify ways in which training can be improved, and reporting to the appropriate authorities.

Management of the Defence training model 1.12

1.12 Training application and implementation responsibilities. The DTM applies to all Defence training (individual and collective training3), whether developed or being developed, and whether sourced internally or from an external agency. Responsibility for applying the model rests with the authorities designated by the Defence organisation with the controlling interest in the workplace performance and/or the training. These authorities are responsible for managing the respective DTM processes and for approving and maintaining their products.

GOVERNANCE AND MANAGEMENT 1.13

The role of the Defence training model in the management of Defence training 1.13

1.13 The role of the DTM is to provide a common philosophy and consistent approach to the development of training within Defence. Head Defence Personnel Executive, as the Policy Owner (PO) for Human Resources within Defence, has delegated the Director-General Defence Education, Training and Development to act on their behalf to develop and coordinate Defence-wide education and training policy. The model is sponsored and managed by Defence Education, Training and Development as part of that responsibility.

1.14 Individual training is the responsibility of each Defence Group and each PO. This is achieved by each Service Training Command or equivalent operating as an RTO within the NTF. The Defence Learning Services Network operates as an RTO and is managed by the Defence Personnel Executive (DPE).

3 Although this publication is focused on individual training, the general principles have application to collective training. Further details on the relationship between individual and collective training is contained in Australian Defence Doctrine Publication (ADDP) 7.0—Doctrine and Training.

1–5


Application of the Defence training model 1.15

1.15 Defence training policy4 requires that the DTM is to be applied to all Defence training, whether developed or being developed, and whether sourced internally or from an external agency. The policy acknowledges that the wide range of circumstances of Defence training is such that the DTM cannot always be fully applied. Where deviation from the DTM is necessary, it states that commanders and managers should ensure that the principles underpinning it should be followed as closely as possible.

Governance responsibilities 1.16

1.16 The governance of the DTM requires a whole-of-Defence approach and is managed by the DPE Training Committee System. At the highest level are the Defence Education and Training Committee (DETC). At the ‘working’ level, the subordinate committee which supports the DETC determinations is the Training Policy and Projects Group.

Management responsibilities 1.17

1.17 As the management of individual training is the responsibility of both Group and PO, each Group and PO should have a suitable organisational structure with clear lines of authority and accountability. At or near the top of this structure should be a position that designates the five authorities responsible for implementing the DTM.

1.18 The five authorities that oversee the phases of the DTM are the:

a. Analyse Authority,

b. Design Authority,

c. Develop Authority,

d. Conduct Authority, and

e. Evaluate Authority.

1.19 Each authority is responsible and accountable for managing the processes within their respective phase. This involves ensuring that suitable resources are available and that appropriate procedures and methods are followed. The authorities are also responsible for the accuracy, reliability, feasibility and affordability of the outputs from DTM work. They are to ensure that all relevant factors and all feasible options have been considered. Finally, they are also responsible for the quality assurance and continuous improvement of the processes and outputs for which they have responsibility.

4 DI(G) PERS 05–36—Implementation of the Defence Training Model in Defence Vocational Education and Training.

1–6


1.20 The close association of training design and development processes requires a close working relationship between the respective authorities. For this reason, the same person may undertake the roles of the Design and Development Authorities. To provide objectivity and rigour it is preferred that the Analyse and Evaluate Authorities are not the same as the Design, Develop and Conduct Authorities.

1.21 Policy Owner responsibilities. Under the governance arrangements for business skilling PO have responsibility for the training requirements for whole of Defence policy domains. Within these arrangements, PO have been specified by the Defence Committee as the Analyse and Evaluate Authorities for their domain, as well as having the responsibility for authorising the training strategy for personnel working in their domain. They are also responsible for ensuring that consultation and coordination is undertaken and maintained with other stakeholders, and sourcing the funding for the training programs required to support domain policy changes and new initiatives.

1.22 The PO may delegate a Business Process Owner (BPO) or another agency with the responsibilities of Design, Development and Conduct Authority for their domain. The BPO (or delegated authority) will negotiate with service providers (whether they are within Defence or an external organisation) for the provision of design, development and/or conduct services.

1.23 Rationalised Defence training. Where training is to be rationalised across Services and/or Groups and is not the responsibility of a PO, a Manager of Joint Training (MJT) is appointed. The MJT has the responsibility for forming a Training Advisory Group, comprising the major stakeholders. The MJT is responsible for ensuring that the DTM is applied to the rationalised Defence training.

1–7


THE ACCELERATED APPLICATION OF THE DEFENCE TRAINING MODEL 1.24

1.24 It is preferable for Defence training and assessment to be designed and developed in accordance with all the phases, processes and steps of the DTM. However, in response to an urgent or unforeseen requirement, it may be necessary to expedite the processes, which means that the DTM steps will need to be accelerated. This acceleration is to be based on a documented risk assessment. The decision to accelerate the DTM rests with the commander of the force5, the Analysis Authority or the BPO. No matter what level of command or management is applying the DTM, a commander or manager must ensure the responsible staff are clear on the degree of risk that they are prepared to accept.

1.25 In considering the time available to plan, the relevant commander or manager should consider:

a. the provision of specific direction limiting options and focusing training development staff on those planning aspects the training authority feels are most important;

b. limiting the number of training options to be developed in order to achieve a workable plan that meets the workplace requirements in the time available;

c. maximising parallel planning with the early issue of outputs, and the sharing of all information with other training authorities and stakeholders. While this is an extremely effective method of increasing tempo, it must be balanced against the risk that the efforts of the other authorities will be wasted.

Risk management 1.26

1.26 The DTM is a tool that allows for the management of risk within the training environment. Risk management is the systematic application of procedures and practises for analysing, assessing, controlling and monitoring risk and is applied throughout the DTM. Rather than limiting activities through restrictive safety concerns, risk management allows the full potential to be achieved through managing rather than avoiding risk. The identification of

5 During the mission analysis phase of the joint military appreciation process (JMAP), a review of own forces is completed. This review involves an examination of own forces to determine the capabilities the available assets can bring to the operation. This assessment examines issues such as readiness levels, training levels achieved and morale of personnel. The own forces review seeks to determine whether forces are at an appropriate level of readiness, with sufficient resources to conduct operations within a specified time frame. If they are not then the commander may require the DTM to be applied to bring those assets personnel to the required level of capability. Further details on the JMAP are in ADFP 5.0.1—Joint Military Appreciation Process (to be issued)

1–8


various methods of delivery allows the commander or manager to manage and at times ameliorate, the risk associated with the prosecution of a training activity. At all stages throughout the execution of the DTM the application of risk management allows a training authority to detail the cost of the planned options. This cost can be measured in different ways depending on the circumstances and direction from the higher levels. Risk management is concerned with balancing identified threats against the harm those threats may inflict to various outcomes.

1.27 The objectives of risk management6 as they apply to the DTM are to:

a. enhance operational effectiveness, and

b. ensure training is conducted in a manner that ensures force preservation.

1.28 The enhancement of operational effectiveness is done by:

a. facilitating employment of the most effective training techniques and procedures while managing the risks that are inherent in the conduct of training,

b. ensuring that risk treatment measures can be tailored to meet training circumstances, and

c. avoiding a risk averse approach to managing training.

1.29 Force preservation is achieved by:

a. achieving a rigorous assessment of benefit against cost as a basis for decision-making;

b. the acceptance of risk at an appropriate level of command or management;

c. reinforcement of awareness of risks and the need for appropriate treatment measures (including policy and procedures) in the planning and conduct of training; and

d. implementation of a process to identify risks and derive risk treatments that will mitigate risks as far as practicable.

1.30 The main point of risk management in training is that risks are best managed in the planning stages of an activity and in the development of policy regarding training management.

6 Staff undertaking risk management in the application of the DTM should consult the Defence policies on risk management found at the Defence Enterprise Risk Management web site see http://defweb.cbr.defence.gov.au/ig/ERM/ermintro2.htm and seek specialist advice when required.

1–9

http://defweb.cbr.defence.gov.au/ig/ERM/ermintro2.htm

http://defweb.cbr.defence.gov.au/ig/ERM/ermintro2.htm


A RAPID DEVELOPMENT PROCESS 1.31

1.31 In those circumstances where there is a great urgency in having a training product there is the possibility of using a rapid development process (RDP), supported by a risk management plan. Using the RDP involves accelerating the DTM. Making such judgments requires a higher level of expertise than normally required for full implementation of the DTM. However, the decision to proceed with an RDP rests with the commander of the force, Analysis Authority or the BPO.

1.32 Guidelines for the acceleration of the DTM through an RDP are:

a. the extensive use and customisation of existing learning, assessment and evaluation materials (if no such materials are available, the appropriateness of using the RDP process must be queried); and

b. combining some of the processes across phases of the DTM.

Annex A provides an example of how these guidelines are applied to the DTM.

1.33 The RDP described is not the only such approach available. However, the use of any RDP should facilitate the full application of the DTM at a subsequent time if circumstances justify it.

CONTINUOUS IMPROVEMENT AND QUALITY ASSURANCE 1.34

1.34 The DTM embodies the principles of continuous improvement and quality assurance. This accords with AQTF standards for RTO that state:

a. the RTO must collect and analyse stakeholder and client feedback and satisfaction data on the services it provides, and

b. the RTO must use the information mentioned to review its policies and procedures7.

1.35 The DTM enables continuous improvement of Defence training through:

a. a structure that provides for links across all phases, processes and steps;

b. integrated formal quality assurance checks;

7 DEST Standard 1.9 a and b (08 June 2001) .

1–10


c. establishing and maintaining auditable records from all processes and products; and

d. being a dynamic and transparent process.

1.36 The DTM is presented as a closed loop to represent its cyclical nature. The application of the DTM will result in training that is continually reviewed and improved. This continuous improvement is achieved through cooperative interaction between stakeholders which relies on communication that is timely, responsive and supportive.

1.37 Quality assurance checks. The principal quality checks incorporated in the DTM include the internal review processes.

1.38 Reporting. AQTF standards require RTO to document any actions taken to improve the quality and consistency of assessment and learning8. Formal reporting of findings from quality assurance checks should be provided to stakeholders. Methods for tracking the progress of recommended actions should be devised for each phase.

1.39 Record keeping. It is essential that accurate and comprehensive records are maintained in accordance with Commonwealth and AQTF audit requirements.

PROCURING TRAINING AND RESOURCES EXTERNAL TO DEFENCE 1.40

1.40 A Defence decision to acquire a product or service by contract is made when the product or service cannot be gained internally. All or any of the DTM processes may be sourced externally. While Defence can outsource all or any part of the DTM processes, under national and Defence guidelines, the designated authority has responsibility for all outputs and outcomes. Particular attention should be given to the ‘Analyse’ and ‘Evaluate’ phases if these are outsourced. Under Defence guidelines, the products developed must be subject to review by an agency independent of the developer prior to acceptance.

8 DEST Standards 9.2 ii and 9.3 i (08 June 2001).

1–11


1.41 Outsourcing and nationally recognised qualifications. Due care must be taken if outsourced training leads to the award of a nationally recognised qualification under the Australian Qualifications Framework. If the commercial training provider is awarding the qualification, the provider must be an RTO with the qualification listed under their scope of registration. If the qualification is to be awarded under the scope of a Defence RTO, then the contract governing the service must explicitly state the responsibilities for meeting required AQTF standards9.

1.42 Guidelines and instructions. The Commonwealth and Defence provide guidance and instructions to manage outsourcing. The Commonwealth Procurement Guidelines and Best Practice Guidance clarify what is required and expected in Commonwealth purchasing. Advice on the procedures for the procurement of goods and services in Defence can be found on the Defence Materiel Organisation web site see http://www.defence.gov.au/dmo/.

1.43 The capability and operational outcomes of Defence are dependent on the performance of its people. The DTM is a systematic approach to developing the competencies of Defence personnel. The DTM ensures a common and consistent approach to the analysis, design, development, conduct and evaluation of training. It incorporates quality assurance and continuous improvement. The DTM is sufficiently general for application to all Defence training requirements.

1.44 To support the successful implementation of the DTM there must be in place appropriate governance and management processes as well as adequate resources. This chapter has described how these two elements might be addressed. Finally, a methodology has been discussed for making training available in an expeditious manner. Adopting this approach involves special circumstances and the development of a thorough risk analysis.

Annex:A. An example of a rapid development process

9 DEST Standard 1.6 for RTO (08 June 2001) states that:

a. The RTO must have, and comply with, a written agreement with each organisation that provides training and/or assessment on behalf of the RTO.

b. The agreement must specify how each party to the agreement will discharge its responsibilities for compliance with the Standards for RTO.

c. The RTO must maintain a register of all agreements made under Standard 1.6.a.

1–12

http://www.defence.gov.au/dmo/

Annex A to ADFP 7.0.2 Chapter 1

AN EXAMPLE OF A RAPID DEVELOPMENT PROCESS A

Phase Process The application of the rapid development process

Analyse Analyse the job Identify available job specifications. Using either focus groups or interviews with key personnel, identify the performance gap statement, possible training and non-training solutions to the performance gap, and gather data for the Target Population Profile. If training forms part of the agreed solution, document the requirement in the form of a Training Requirement Specification (TRS). The job/Policy/Business Process Owner is to formally agree to the TRS. This agreement, together with the TRS is to be placed on the program/course file.

Analyse the target populationDetermine the performance gapConduct feasibility analysis

1A–1

ADFP 7.0.2

Design Analyse prospective learners

In the rapid development process, this is best done in conjunction with the ‘Analyse’ phase.

Analyse learning and assessment environments

Identify, seek out and assemble existing, relevant learning outcomes; assessment, training, Training Review and Evaluation (TR&E) strategies; assessment guidance; materials and equipment for: assessment, training and learning, TR&E.

• Do these resources address the requirement? If so, use them.

• If the materials are not immediately useable, can they be readily customised to address the requirement? If so, customise them.

• Are additional resources needed? If yes, determine what is needed and decide how best to acquire them. Action this determination.

Generate learning outcomesDesign assessment and training strategies

Determine evaluation requirement

Decide the minimum acceptable level of evaluation and draft an outline evaluation plan and program. Note: the outline evaluation plan and program and the development of required materials can then be completed during the ‘Conduct’ phase. The evaluation plan is to be placed on file as it is developed.

Plan and develop evaluation


1A–2

ADFP 7.0.2

Develop Produce/procure training material and equipment (commercial-off-the- shelf should be considered in the first instance)

Compile the training, learning and assessment materials into a logical learning order for the ‘Conduct’ phase. Write the learning outcomes based on the materials assembled and prepared. Confirm the validity of the learning outcomes against the TRS. Amend, redevelop as necessary and place a copy on the program/course file.

Prepare/brief administration staff, trainers and assessors

Keep stakeholders informed throughout the development process.

Administer assessment and training preparations

Use existing, proven staff, procedures and processes as much as possible.

Conduct The ‘Conduct’ phase should not be condensed or abbreviated. It is to be implemented in accordance with guidance provided in the Defence training model (DTM).

Evaluate The ‘Evaluation’ phase should not be condensed or abbreviated. It should be implemented in accordance with guidance provided in the DTM.


1A–3


CHAPTER 2

THE ‘ANALYSE’ PHASE 2

Introduction 2.1

2.1 The ‘Analyse’ phase is the first of five phases constituting the Defence training model (DTM), as shown in figure 2–1. It is during this phase that the need for training is analysed and specified.

Figure 2–1: ‘Analyse’ phase of the Defence training model

2.2 As with all phases of the DTM and their constituent processes and steps, the level of application should match the circumstance. The guidance provided in this, and the other process manuals, is applicable to situations of high significance (eg the introduction of training in support of a major new Defence capability). Abbreviated processes may be appropriate in some circumstances.

Starting point 2.3

2.3 The DTM is applied to produce new, or to modify existing training. There are three circumstances in which its first phase, the ‘Analyse’ phase, is used:

a. A new capability requirement. New training may be necessary if job holders are required to contribute to a new or modified capability requirement. The requirement might arise from the purchase of new, or modifications to existing

2–1


equipment, software, or technology; or a change in employment area, legislative requirements, product requirements, or employment specifications.

b. Continuous improvement and evaluation feedback.Feedback from continuous improvement processes and evaluation can indicate that the training presently conducted is failing to meet workplace needs and/or it can be improved.

c. Periodic review of competency standards. Changes to competency standards are likely to require corresponding amendments to training. Periodic reviews of competency standards may be initiated either from within or external to Defence. (Competency standards forming part of nationally endorsed training packages are reviewed, on average, every three to five years.)

Processes of the ‘Analyse’ phase 2.4

2.4 The ‘Analyse’ phase is made up of six processes:

a. needs assessment;

b. job analysis;

c. target population analysis;

d. determine performance gap;

e. feasibility analysis; and

f. Training Requirement Specification (TRS).

The first of these, needs assessment, might be considered external to the DTM however, some form of needs assessment precedes all applications of the DTM.

2.5 The flow chart in annex A illustrates the processes and outputs of the ‘Analyse’ phase of the DTM, and annex B presents the purpose, inputs and products of each of the processes in tabular form.

2.6 Needs assessment. The DTM is applied in response to an identified need, where a need is the difference between the present and a required performance in a workplace. Before the DTM as a whole, or any of its parts is applied, some form of needs assessment is completed and training is identified as a possible solution to the problem.

2–2


2.7 Job analysis. A job analysis identifies the performance requirement, both current and foreseen, for a given employment or job classification (such as task, job, or career/occupation). The requirement is recorded in terms of the duties, tasks, sub-tasks and elements that constitute the job. A job analysis may involve any one or combination of: a document search; direct observation; survey; and/or consultation with Policy Owners (PO), Business Process Owners (BPO), capability and trade sponsors, supervisors, managers and job holders. It also involves research of relevant existing competency standards and units and other proficiencies. Determining the training requirement requires understanding of what the job consists of, the standards of performance required, the context of performance, and how performance is judged. This information is recorded in a job specification.

2.8 Target population analysis. To identify what training is required, how much, and how it might best be conducted the characteristics and qualities of the target population must be understood. This is achieved through a target population analysis. The first step is to determine who comprises the target population (possibly the present job holders). The current competencies of its members are then identified, together with their typical characteristics and qualities (such as academic record and ability, employment experience, age, gender, physical characteristics, learning preferences, special aptitudes and motivation). This information is obtained from workforce records and plans, and from education and training records; other elements can be gained from surveys of likely learners and their supervisors/managers, or by direct assessment. Finally the information is recorded in a Target Population Profile.

2.9 Identification of the performance gap. A comparison of the competencies required to perform the job and those of the target population is used to identify the ‘performance gap’—the difference between current and required competence. The difference is recorded as a Statement of Performance Needs.

2.10 Feasibility analysis. There are several ways in which a deficiency in workplace competency can be met, of which training is just one. A feasibility analysis involves the determination of the best option for meeting the need. The analysis should include a cost/benefit analysis of training and non-training options, and should conform to current policies and instructions. Non-training options may include recruitment, automation, changing the job, re-allocating the job, and modifying the workplace. Three categories of training options should be considered: in-house, outsourced and a combination of these. For each option several factors must be reviewed, including the existence of similar training; the potential for using attendance, distance, and on the job methods; and the availability of training facilities and staff. The product of an analysis is a Feasibility Report that, when approved, is presented to the appropriate authorities.

2–3


2.11 Training Requirement Specification. Having identified the workplace performances for which training will be conducted, the final process of the ‘Analyse’ phase is to formally document the requirement in a TRS. This specification records the requirement in terms of the people (numbers required, rates of throughput, any time limitations, and the context and recognition of their training) and the competencies they need to undertake the job1. In preparing the document, relevant existing training is investigated and decisions made on whether it might be used unchanged, or whether it might be customised to meet Defence needs. If no existing competency standards apply, new ones are written. The TRS lists the competency standards requiring training together with other important factors defining the training requirement. It provides the baseline against which subsequent training activity is measured.

Continuous improvement 2.12

2.12 While working on any of the processes of the ‘Analyse’ phase, stakeholder input is sought for help in assuring the quality of the product. A consequence is that the product is likely to require repeated revisions before acceptance. When each process and step is completed, an accurate and comprehensive record of the decisions contributing to it should be maintained. This record is likely to be referred to during all subsequent development of the training, in the near and long-term.

Key outputs and responsibilities 2.13

2.13 While each process of the ‘Analyse’ phase has outputs, the two key products of the phase are the job specification and the TRS.

2.14 Training Analysis Authority. The PO/BPO is the Training Analysis Authority for approving the documents of the phase, particularly its key outputs, in accordance with their responsibility for overseeing the work standards of an occupation (category/employment category number/ mustering etc). The sponsor is also responsible for retaining key documentation for audit, quality assurance and continuous improvement purposes. Notwithstanding the responsibilities, the ‘Analyse’ phase is performed jointly by representatives of both the training authority and the job/occupational sponsor.

1 The use of competency standards, units and elements of competence should accord with the guidelines of the National Training Framework (NTF described in Defence Instruction (General) (DI(G)) PERS 05–29—Implementation of the National Training Framework within Defence vocational education and training.

2–4


Use of an external agency 2.15

2.15 In accordance with national guidelines, the quality assurance of all training by, or on behalf of Defence (by Defence staff, contractors, professional service providers, or other Registered Training Organisation (RTO)) remains the responsibility of Defence. As the ‘Analyse’ phase is one of the two key phases of the training cycle2, it should not be contracted out except where the scale of the project is beyond the in-house capacity and/or the necessary expertise is not available within Defence.

CONDUCT NEEDS ASSESSMENT 2.16

2.16 The DTM is a cyclical system. It is applied to a workplace situation when a difference is perceived between present and desired performance, and training is perceived as a way of resolving the difference. The workplace performance, or job, might be new—as when new equipment, systems or procedures are introduced—or the desired improvement might arise from policy changes, or simply when deficiencies in current performance are thought to exist.

2.17 Before the DTM is applied, the nature of the required performance should be identified and some assurance gained that training can significantly contribute to achieving it. This is accomplished through a needs assessment.

2.18 Training is not the only way of rectifying or improving workplace performance. In fact, training is often only one component of a total performance solutions package. In such situations all components are important, and training by itself may not solve the over-all performance deficiency. Other components to a solution package may include:

a. improving motivation by providing incentives;

b. employing staff with proven competence in the job, reorganising the job to make better use of existing staff;

c. provision of good quality job aids;

d. removal of obstacles preventing good performance;

e. automating all or parts of the job;

f. applying more or enhanced resources to it (including time); and

g. improving the quality of materiel used in the job, or the environment in which it occurs.

2 The other key phase is the ‘Evaluate’ phase.

2–5


2.19 Any one or combination of these options might be the most cost-effective means of improving workplace productivity/performance.

2.20 A needs assessment is used to justify the initial application of the DTM to a performance problem and, at a lower level, its application for updating and amending existing training. The resources applied to the process should be proportional to the situation and its associated risks.

2.21 Purpose. The purpose of needs assessment is to scope the nature of the performance problem or opportunity, and to determine whether the DTM should be applied.

2.22 Responsibility. Responsibility for managing the ‘Analyse’ phase of the DTM lies with the Analysis Authority of the organisation owning the job performance. The Authority, or their delegate, should be satisfied that an adequate needs assessment has been conducted before authorising work on subsequent processes of the phase.

2.23 Triggers. The triggers for conducting a needs assessment include:

a. new or modified job/employment requirements (possibly caused by new equipment and/or processes);

b. changes to a national training package; or

c. feedback from the ‘Evaluation’ phase.

Inputs, process and products 2.24

2.24 The inputs, process and products of the conduct needs assessment process are summarised in table 2–1.

Table 2–1: Inputs, process and products of the conduct needs assessment process

Inputs Directive, operational report, new or modified job or training package data, Evaluation Report, existing training.

Process Step 1: Establish the Terms of Reference (TOR).Step 2: Determine information requirements.Step 3: Plan and gather information.Step 4: Collate and analyse information to make recommendations for further action.Step 5: Report and record.

Products A Needs Assessment Report that provides a clear indication of the nature of the performance problem/opportunity, including its causes, and recommends further action, possibly including application of the DTM.

2–6


Step 1: Establish the Terms of Reference 2.25

2.25 To ensure that subsequent work focuses on an agreed requirement, a TOR for a needs assessment should be produced. The terms should clearly establish the scope of the job/workplace performance being considered and, if relevant, how it relates with associated work in the organisation. It should present the background that gave rise to the perception that workplace performance can be improved. This background might describe any relevant history, the current position, and any foreseen developments. The terms should present any pertinent time factors, indicate any urgency involved, identify the agency responsible for the conduct of the assessment, and any specific reporting requirements.

Step 2: Determine information requirements 2.26

2.26 The information needed to assess possible solutions to a perceived workplace performance problem will be context specific, as directed by the TOR. In most instances information will be required about:

a. the current and desired workplace output, the difference between them, and what caused this difference (including history, directives, any benchmarks);

b. the factors that contribute, or could contribute to the output (including the physical factors of environment, equipment, materiel; and the human factors of work force size, location, capability, management, and motivation; and any policy or funding factors); and

c. the stakeholders involved with the employment and training of the relevant work force.

2.27 Consideration of these types of information will give rise to suggestions for achieving the desired levels of performance. These suggestions will themselves need to be researched so that their validity and value can be assessed.

Step 3: Plan and gather information 2.28

2.28 The planning and gathering of information for the needs assessment should identify the method to be used, the resources required, the means of acquiring the resources and the time frames involved. Methods of collecting information can include:

a. workplace observation;

b. focus groups;

2–7


c. communication with individuals (using recorded interviews and emails);

d. group surveys; and

e. research of documents and records.

2.29 The method(s) used and the scope of the needs assessment will determine what information collection materials should be prepared and, possibly, tested.

2.30 The emphasis given to planning and collecting the information required should be proportional to its significance. The work should involve representatives of all stakeholders in the outcome (workplace performance). This includes the job or policy/BPO, a sample of any existing job holders, those responsible for resources and policies contributing to the achievement of the required performance, and the ‘customers’ of that performance. Existing training providers and reference groups should also be considered.

Step 4: Collate and analyse information to make recommendations 2.31

2.31 Once gathered, the information is collated and analysed. Particular attention is to be paid to identifying all options for achieving the required performance, not just the option of training people. The options need to be considered individually and in combination. They should be considered against such factors as:

a. the extent to which the performance is required (including the number/percentage of people involved);

b. the consequences of error if the performance is inadequate (threats to life and/or security);

c. the consequences of task delay (should the performance be automatic, instinctive or considered?);

d. the resource implications of implementing each solution;

e. the timeliness of the solution (including the scope for short-term and more strategic solutions);

f. integrating solutions; and

g. access issues (geographic locations, mobility etc).

From such consideration arguments for and against the implementation of each option are developed. These arguments will commonly cover the benefits, costs, and associated risks in relation to the expected performance

2–8


improvement. Logical consideration of these factors leads to recommendations being derived on which of the option(s) should be implemented.

Step 5: Report and record 2.32

2.32 The needs assessment should be recorded, reported and formally acknowledged as the basis of pursuing options to improve workplace performance. The Defence paper format detailed in Australian Defence Force Publication (ADFP) 04.1.2—Defence Writing Standards (currently ADFP 102) should be used for presenting the Needs Assessment Report.

2.33 The scale and scope of training needs assessments vary, and so therefore should the administrative effort applied to them. For a workplace output of low risk the effort might be minimal, but for high risk projects the effort required, and the detail of the analysis involved, would be greater.

ANALYSE THE JOB 2.34

2.34 Job analysis is the process of determining/confirming the workplace performance requirement, both current and foreseen, for a given domain or employment classification (task, job, or career/occupation). This performance requirement is recorded in a job specification. This document provides a benchmark for all subsequent related work.

2.35 Purpose. The purpose of a job analysis is to formally describe the career/job/task capability requirement in terms that can be commonly understood and used for training and other purposes.

2.36 Structure of jobs3. A job specification may be limited to one job within a career or occupation, or might cover all jobs within a specific career, occupation or employment group. Alternatively it can also be applied to describing the requirements of a particular domain (such as finance) which crosses employment classification boundaries. To understand workplace performance requirements, especially those that contribute to careers/occupations, it is useful to view jobs as part of hierarchical structure of work. The following diagram illustrates such a structure:

3 The DTM uses the term ‘job’ generically to describe the whole hierarchy of tasks, duties, jobs and careers.

2–9


Figure 2–2: Hierarchical structure of work

Note

Simple definitions of the terms used in the diagram are provided in the following table:

Term Definition Example

Career or occupation

Sequence of related jobs requiring developing qualifications and/or experience

NursePilotElectronics technician

Job A post of employment comprising a number of duties and tasks

Casualty nurseSeahawk pilotDiesel engine maintainer

Duty A major area of job responsibility comprising a grouping of closely related tasks

For diesel engine maintainer:maintain and repair engine cooling systemstune diesel engines

Task A discrete unit of work For diesel engine maintainer:conduct periodic servicesreport major malfunctions

2–10


2.37 Trigger. The trigger for this process is the approval of a needs assessment’s recommendation, as signified by the Analyse Authority (or their delegates) sign off on the Needs Assessment Report.


2.38 The inputs, process and products of the job analysis process are summarised in table 2–2.

Table 2–2: Inputs, process and products of the job analysis process

Sub-task A step in achieving a task For conduct periodic services:inspect/clean filterscheck/top-up coolant level

Task element A step in achieving a sub- task For check/top-up coolant level:visually check levelobtain coolanttop-up coolant

Inputs Existing or related job specifications, job duty statements and selection criteria, Australian Public Service work level standards, process manuals current competency standards, project requirements, exercise/operation reports, observations, research, surveys.Consultation with BPO, capability and trade sponsors, supervisors/managers and job holders andlegislative and licensing requirements and organisational policies.

Process Step 1: Establish scope of job analysis.Step 2: Determine information requirements.Step 3: Plan and implement information collection.Step 4: Collate and analyse information.Step 5: Develop job specification.Step 6: Gain authorisation for the job specification and disseminate.

Products An authorised job specification that provides a baseline description of the human performance requirements together with associated information.

Term Definition Example

2–11


Step 1: Establish scope of job analysis 2.39

2.39 Before work begins on a job analysis, the scope of the task should be defined, understood, agreed and recorded by key stakeholders4, possibly using a TOR that refers to any associated needs analysis. The importance of the task should be understood and any risk and resource (including time constraints) identified.

2.40 In establishing the scope it is important to determine whether a new job specification is required or whether an existing one will suffice. This requires locating existing documentation, comparing it with the problem/opportunity and deciding whether a new job specification is required.


2.41 Decisions affecting Defence jobs should be based on knowledge and understanding of those jobs and their relationships with others. The knowledge needed for a job analysis is common to all situations, and can be classified as information about the requirement for the job, and the characteristics of the job:

a. Job requirement:

(1) number of job positions (which may change over time);

(2) location of positions;

(3) anticipated staff turnover; and

(4) importance.

b. Job characteristics:

(1) classification (ie how they fit in organisational structures);

(2) components (ie contributing duties, tasks, and sub-tasks);

(3) required performance levels for the tasks and sub-tasks;

(4) inter-relationships with other jobs (including pathways between jobs and key impacts on the abilities of other jobs to be performed effectively);

4 Key stakeholders are the sponsor or BPO for the job, the staff undertaking the analysis, existing training providers/managers, and possibly representatives of both the job holders and their supervisors who are likely to be most affected.

2–12


(5) required equipment and materiel;

(6) physical conditions and social environment (including locations, levels of teamwork and operational circumstances);

(7) physical and mental skills and demands (that relate to the conditions);

(8) need for job holder attributes/qualifications; and

(9) links with employment and career factors.

2.42 The currency of the information should be considered when determining information requirements. Any anticipated changes should be researched and recorded to inform decision-making.

2.43 Where there are no previous records to work with or there are substantial changes to job requirements the information requirements of a job analysis can be large. Where records of preceding analysis exist, or there are records of associated analyses, the information in these may be current, and checking, updating and/or supplementing those records may only be required.

2.44 The information gathered by a job analysis has utility for other fields of human resource management apart from training. Consideration should therefore be given to advising managers of related responsibility areas (eg work study, organisational structures, pay and conditions) with a view to sharing the products of the analysis and the feasibility of tailoring the information requirement to meet their needs.

Step 3: Plan and implement information collection 2.45

2.45 When the nature and extent of the job information requirement is known, its collection can be planned and implemented. Planning should incorporate:

a. selecting sources of information (which may involve sampling);

b. selecting collection methods;

c. costing data collection;

d. gaining, where necessary, formal endorsement of the plan;

e. preparing for the collection, including the planning of visits, observations;

2–13


f. meetings and the preparation of any data collection tools; and

g. consideration of any confidentiality, validation and security implications affecting the process.

2.46 Likely information sources about a job include:

a. existing job specifications;

b. position duty statements and selection criteria;

c. job process and machinery manuals;

d. exercise/operation reports;

e. observation of the work (direct or recorded);

f. surveys of and consultation with job owners, PO or BPO, capability and trade sponsors, supervisors/managers, and job holders;

g. occupational analyses and surveys;

h. legislative, regulatory and licensing requirements;

i. organisational policies and procedures;

j. existing course evaluations and reviews;

k. training continuums;

l. accredited courses; and

m. training packages.

2.47 Collection methods should be selected on the basis of balancing effectiveness against resource costs (including time). Particular consideration should be given to the reliability of collection methods and tools, and where possible they should be trialed to check this. Methods may involve record searches, workplace visits with observations, interviews and group process methods such as expert panels and focus groups. The latter can be particularly valid, but requires time and administrative effort to conduct. Trade-offs will often be required between the quality of the information and the effort required in gaining it on the basis of associated risk.

2.48 If planned well, the collecting of job information should be straightforward. The time allocated for collection should allow for modifications to the plan in accordance with strengths and weaknesses that become apparent during implementation.

2–14


2.49 Whilst collecting information about the job, other information relating to training may be found. This can include feedback about past training, the existence of relevant competencies and training courses and materials. Such information should be collected, as it may be useful to the later training development work.

Step 4: Collate and analyse information 2.50

2.50 As information is gathered it will require both collation and review for its validity. It may be necessary to refer back to sources to add to, or check information. The need for this will vary with the criticality of the information and the reliability of the source. Once data has been determined to be valid, analysis of the data can be undertaken. Analysis, particularly of job component data, should be based on set criteria which may include the percentage of job holders who perform the task, the criticality and difficulty of the task, the requirement to identify sub-tasks and the extent or level to which tasks will be broken into sub-tasks. Ultimately the analyst should be able to substantiate and justify all the information included in the job specification.

Step 5: Develop job specification 2.51

2.51 A job specification records the product of a job analysis. Annex Cprovides a template that suggests a structure for the document, together with guidance on its content. The scope and depth of content of a job specification should be appropriate to its context. The key components of all job specifications are the information that describes the job: the duty task inventory (DTI) that details the work performed and the required performance standard, the conditions under which the work is conducted, and any information affecting the job-holder’s performance.

2.52 For formative evaluation purposes the draft job specification should be reviewed by authoritative stakeholders as it is being developed and revised as necessary.

Step 6: Gain authorisation of job specification and disseminate 2.53

2.53 Job specifications must be authorised by the Analysis Authority—the sponsor or BPO for the job, or their delegate. Approval is a significant decision, not just for the training implications, but because the contents of job specifications may be used for other purposes. Job information may be used, for instance, for personal reporting, and as a basis for performance management and development pathways.

2.54 When authorised, the job specification should be disseminated to all pertinent authorities. These could include career and personnel management staff and senior job supervisors. The specification should also be filed with the appropriate training development documents.

2–15


ANALYSE TARGET POPULATION 2.55

2.55 The group of people from which trainees are selected is called the target population. The development of quality training requires a detailed and clear understanding of this group. The role of a target population analysis is to help develop this understanding.

2.56 A full target population analysis is undertaken for major training projects, particularly when no relevant training records exist. For less significant problems and/or where relevant training documentation applies, a limited process might be applied.

2.57 Purpose. The information gained by analysing the target population serves two purposes. Knowledge of the target population’s current competencies, when compared with those required in the workplace, is used to determine the performance gap that training might fill. Other information about the target population supports decisions about the strategies for assessing and conducting the training.

2.58 Trigger. The trigger for a target population analysis is a decision, resulting from a needs assessment, to apply the DTM to a workplace performance problem.


2.59 The inputs process and products of analysing the target population are summarised in table 2–3 and explained in the following paragraphs.

Table 2–3: Inputs, process and products of the analyse target population process

Inputs Directive with parametersJob specificationWork force records and plansRecruiting dataEducation and training recordsSurveys of likely learners and their supervisors/managers

Process Step 1: Determine the target population.Step 2: Determine information requirements.Step 3: Plan and gather information.Step 4: Collate and analyse information.Step 5: Develop a Target Population Profile, report and file in accordance with policy.

Products Target Population Profile of likely learners. (The document may be an attachment to a formal report of the process.)

2–16


Step 1: Determine the target population 2.60

2.60 The composition of the target population will normally be determined by the career or personnel managers, although training specialists might also provide input. The target population may consist of people already in the organisation, including those who might already be undertaking complementary work, or new recruits who will be trained for the work. The group may therefore be large and heterogeneous, or small and homogenous. The individuals within it may be located close to or distant from the intended workplace and possible training venues. The criteria used to classify the target population (eg career/employment grouping, location) are recorded on the Target Population Profile.


2.61 Two types of information about the target population are required: that which defines the group as a whole, and that which describes individuals within the group. At the group level, its size, general levels of educational and/or professional competence, and its geographic distribution are factors that can contribute to decisions concerning training strategies. At the individual level, information contributes to decisions about the training required and the methods to be used for it. Knowledge of the typical individual’s competencies is critical, and is used for deriving the performance gap. Other useful information about the typical individuals can include their:

a. academic record and learning ability;

b. employment and training background;

c. age, gender, physical characteristics, learning preferences, and special aptitudes;

d. attitudes to the intended change to working practice;

e. workplace environment (in terms of suitability for assessment and training);

f. any factors limiting availability for training; and

g. special or unusual group characteristics.

2–17


Step 3: Plan and gather information 2.62

2.62 Good planning ensures efficiency in the gathering of information. Such planning should incorporate the sources to investigate and the methods to be employed. They might include:

a. research of the Defence Personnel Management Key Solution (PMKeyS) database, recruiting data, performance appraisal, and training documentation; and

b. survey, observation, interview and group processes involving a mix of sources including members of the target population, recruitment, career, and personnel staff.

2.63 The planning should also consider any potential for efficiency by using the information search for collecting other related information, such as that concerning the working and learning environments.

2.64 The collection strategy developed should balance the quality of the information against the effort required to gather it, the whole being proportional to the scale of the training development task. Sampling will usually be necessary, and the process should be demonstrably valid. The larger the sample the more valid will be the generalisations derived from, and decisions based on it. Once the strategy is determined, the planning needed to implement it should be undertaken. This can include arranging visits and interviews, and preparing aids such as questionnaires and data collection forms.

2.65 If planned well, gathering the required information should be straightforward. Meeting with likely future learners might also provide a useful opportunity to promote the workplace change to which training may contribute.


2.66 As the information on the target population is likely to come from a number of sources it will require collation. This process could identify inconsistencies that need resolution and gaps that require more information to be gathered. This work enables the development of an accurate picture of the likely learners.

Step 5: Develop a Target Population Profile 2.67

2.67 A Target Population Profile records the information of the analysis. Annex D provides a template. The scope of a Target Population Profile and the depth of its content should be appropriate to its context. The job classification data incorporated must match that of the job specification.

2–18


DETERMINE PERFORMANCE GAP 2.68

2.68 The ‘performance gap’ is the difference between the performance required for a job and the capabilities of the target population: the people likely to perform it. The gap has two dimensions: scope and scale. The scope of the ‘gap’ describes the work that the target population cannot presently perform to the required workplace standard. The scale describes the significance of this problem in terms of its extent, impact and importance to the organisation.

2.69 Documenting the performance gap further defines the performance problem addressed in the needs assessment. It is recorded on a Performance Gap Statement.

2.70 Purpose. The purpose of determining the performance gap is to specify the difference between the performance required for a job and the competencies of the target population.

2.71 Trigger. The Analysis Authority instigates a performance gap analysis following formal acceptance of the job specification and the Target Population Profile.


2.72 The inputs process and products of the performance gap process are summarised in table 2–4 and explained in the following paragraphs.

Table 2–4: Inputs, process and products of the performance gap process

Inputs Job specification.Target Population Profile.

Process Step 1: Determine scope of performance gap.Step 2: Determine the scale of the performance gap.Step 3: Develop a Statement of Performance Need.

Products Performance Gap Statement.

2–19


Step 1: Determine scope of performance gap 2.73

2.73 The first step in deriving the performance gap for a job is to determine which of the tasks and subtasks listed in the DTI (an element of the job specification) cannot presently be performed by members of the target population. A simple comparison of the proficiencies of the target population (as recorded in the Target Population Profile) is made against the tasks and subtasks listed in the job’s DTI. This determines whether members of the target population can satisfactorily achieve the task, or whether a gap in required performance exists. The Target Population Capability Record template in annex E5 is an aid for this work.

Step 2: Determine scale of performance gap 2.74

2.74 While comparing the scope of the performance gap, and by reference to the job specification, its scale should also be identified. This information describes the significance of the gap and indicates how difficult its closure is likely to be. Identifying the scale requires analysis of the extent to which the target population cannot perform the listed tasks, and whether they have the necessary attributes to do so. The affect of the gap on the organisation should also be considered. How important is the deficiency to Defence capability, both now and in the future? What risks are involved in failing to close the gap in any relevant time frames?

Step 3: Prepare a Performance Gap Statement 2.75

2.75 The third step in determining the performance gap is to develop a Performance Gap Statement. This document has two parts. The first collates the task performances defining the scope of the gap. The second presents information about the scale of the gap. It gives context to the need in terms of how many people lack the proficiencies, when they will need them, and their significance. Annex F presents a suitable template6.

2.76 Both the Target Population Capability Record and the Performance Gap Statement are working documents. Although they do not have to be formally approved by higher authority, informing stakeholders of their content can be worthwhile, and they should be retained for reference purposes.

5 The Target Population Capability Record form is a working document. It is not approved by a higher authority, but should be retained for reference.

6 The Performance Gap Statement is also a working document that should be retained for reference.

2–20


CONDUCT FEASIBILITY ANALYSIS 2.77

2.77 A feasibility analysis is the DTM process used to determine the best option, or mix of options, for ensuring the competence of job holders. It furthers the work of the needs assessment, which identified that job holders are needed to achieve the required workplace performance.

2.78 Ensuring the competence of job holders has two components: adopting measures to achieve job holder competence, and then assessing that the required competence level has been reached. A feasibility analysis addresses both components, with its results and recommendations being recommended in a Feasibility Report.

2.79 Depending on the situation, a feasibility analysis can take many months of intensive work by a large team, or it can be completed quickly and simply by a single person.

2.80 Purpose. The purpose of a feasibility analysis is to identify the most effective, efficient and, where required, flexible means of ensuring the required competency of job holders.

2.81 Trigger. The feasibility analysis process follows the determination of the performance gap as recorded in a Performance Gap Statement.


2.82 The inputs, process and products of feasibility analysis are summarised in table 2–5 and explained in the following paragraphs.

Table 2–5: Inputs, process and products of the feasibility analysis process

Inputs Performance Gap Statement; research data relating to options, including organisational policies and any regulatory or legislated requirements.

Process Step 1: Identify options for ensuring job holder competence.Step 2: Investigate options for ensuring job holder competence.Step 3: Compare options for ensuring job holder competence.Step 4: Identify options for assessing competence.Step 5: Investigate options for assessing competence.Step 6: Report the feasibility analysis and file.

Products A Feasibility Report that recommends training and non-training options for meeting the performance need and how competency should be assessed.

2–21


Step 1: Identify options for ensuring job holder competence 2.83

2.83 To identify options for ensuring job holder competence, both the performance gap and the circumstances that led to it must be understood. This information is contained in the Performance Gap Statement—the product of the preceding process. Important information includes the number of job holders or positions lacking the required competence, the turnover of people, any time frames involved, and the significance of the problem. This information should help determine whether the job tasks and the target population can be considered homogenously, or whether they should be grouped to meet special needs or characteristics.

2.84 There are many separate but complementary ways by which an organisation can achieve competent job holders working to required performance levels. For established workplace situations, management-based options such as supervision, motivation, and planning may suffice. Such options may also be useful where the workplace performances have changed or are new, but in these circumstances the following options are more commonly appropriate:

a. modifying the workplace, equipment, and tools;

b. process re-engineering;

c. recruiting competent staff;

d. reallocating work;

e. automating the performance;

f. informal on the job coaching; and/or

g. workforce training.

2.85 Each of these options may have two or more variations. This is particularly true for training, where options can include:

a. structured and unstructured on-the-job training;

b. structured off-the-job training that can be residential or distance, in-house or outsourced training; or

c. any mix of these (and each of these options might be applied in more than one location and situation).

2.86 Furthermore, the optimum choice may consist of a mix of any number of the options. For example, new partially competent recruits are employed in a new organisation, receive induction training in one location and are trained

2–22


at another location using a mix of off-the-job and on-the-job strategies. Their training involves the use of tools that are new to the organisation to achieve a new workplace performance.

2.87 Investigating all possible options would be unrealistic, and judgments are required to identify those that are sensible. In making such judgments, the options must be clearly appropriate to the performances listed on the Performance Gap Statement and should be checked for consistency with current policies and instructions.

2.88 It would be nugatory to identify and investigate all conceivable options for achieving competent job holders. Instead, those considered more effective and efficient should be selected. Such judgment is improved if key stakeholders are involved. This practice also assures acceptance of the outcomes of the feasibility analysis.

Step 2: Investigate options for ensuring job holder competence 2.89

2.89 To compare the options, information about the options and the situation must be gathered and considered. The required information is likely to include, but not necessarily be limited to, time frames for achieving the desired workplace performance, resource and policy limitations, recruitment data, availability of target population for training, minimum training throughput, and any applicable regulatory or legislated requirements. As the comparison will largely be based on cost, benefit (measured in money terms) and/or effectiveness (considered in monetary and other terms), such information is also required. Costing of options is often a lengthy and difficult process. Cost estimates should therefore be prepared only for otherwise strong options. Obtaining suitable information is rarely easy, not least because the effectiveness of an option can rarely be quantified. Consequently it is usually necessary to provide qualifying or explanatory notes, and to recognise and identify where judgments are subjective.

2.90 As the investigation of options progresses, some may prove inappropriate, in which case no more time should be given them. Conversely, new options may become apparent, in which case they should be investigated and reported. It is also likely that options will be better understood in later phases of the DTM (as when the cost of training are clarified when strategies for it are investigated in the ‘Design’ phase). In such instances, reviewing the feasibility analysis and revising its report may be worthwhile.

2–23


Step 3: Compare options for ensuring job holder competence 2.91

2.91 To determine the best option, or mix of options, for designating competent job holders the possibilities are subjected to an evaluation process. The type of process should suit the circumstance. In many situations it can be useful to determine and apply essential and desirable evaluation criteria such as:

a. policy;

b. resources (knowledge, human, financial, facilities and equipment);

c. time; and

d. risk.

2.92 It can also be useful to consider the required performances in terms of:

a. the number (percentage) of the target group performing the work;

b. the consequence of error if the skill is inadequately performed;

c. the learning difficulty—if a competency is difficult to learn it is more likely to be candidate for formal training; and

d. the task delay tolerance, for if a performance is required instinctively eg fire response, it is more likely suitable for formal training.

2.93 In many situations alternative or complementary methods involve the use of a cost/benefit analysis, a cost/effectiveness analysis, or both.7 In situations of significant risk, conducting risk analyses might also be worthwhile: for all the options, or just the recommended one(s).

7 The difference between the two types of analysis is that a cost benefit analysis uses monetary information to determine the return on investment, while a cost/effectiveness analysis uses a more relaxed approach towards the measurement of benefits—as when options are ranked in accordance with risk, or a training solution is ranked in order maximum possible throughput.

2–24


Step 4: Identify options for assessing competence 2.94

2.94 Assessment of job holders’ level of competence can be achieved in several ways. The basic methods are:

a. on-the-job measurement and evaluation of performance in either:

(1) real situations, or

(2) exercise situations;

b. off-the-job measurement and evaluation of performance (such as in a training or an assessment centre);

c. ‘desktop evaluation’ of qualifications (possibly against standardised measures); and

d. a combination of the methods listed.

The first three of these methods are listed in the order of their validity (how closely they might measure the required performance).

2.95 In considering which options should be considered for more detailed investigation, not only their validity, but their suitability (how feasible and at what cost?) and reliability (how consistently are their results likely to be?) should be broadly assessed. Another consideration that should be made is whether the assessment is likely to be recognised by the award of an Australian Qualification Framework (AQF) unit of competence (in which case Australian Quality Training Framework (AQTF) assessment guidelines are to be applied8), or whether recognition will be organisational specific, without links to the NTF.

Step 5: Investigate and compare options for assessing competence 2.96

2.96 Having identified possible ways of assessing job holders’ competence, an investigation of the strengths and weakness of each should be undertaken. This involves gathering detailed information about, and making judgments on the validity, suitability and reliability of each option in relation to the performances listed in the Performance Gap Statement. In this process the job tasks might be considered homogenously, or they might better be grouped to meet the needs of special characteristics.

8 AQTF Learning and Assessment Strategies—‘How to Kit’.

2–25


2.97 The hierarchy of validity being fixed, the comparison of options is made from the most valid to the least, with judgments being made about their suitability and reliability. For each field the judgments might consider the following fields:

a. Suitability: availability of workplace when required, disruption to normal work, costs involved, time constraints; and

b. Reliability: consistency of conditions (including human inputs).

2.98 The basic judgments to be made are whether an option is acceptable or not acceptable. If two or more at the same level of validity exist then comparisons of levels must be made.

Step 6: Report the feasibility analysis and file 2.99

2.99 The results of the two parts of the feasibility analysis, including the comparison of options, conclusions and recommendations, are reported to the Training Analysis Authority in a Feasibility Report. The form of this report should match the scope of the task, but for most occasions a Defence options paper format in two parts should be used (a template is provided in annex G).

2.100 In most cases training is likely to form part, if not the complete solution for ensuring job holder competence. Where a mix of options is selected for both this and the assessment of competence, the report should clearly link each option with the performance(s) to which it relates.

2.101 The Feasibility Report should be staffed through all significant stakeholders, particularly those outside of training that might be affected by its content. Appropriate levels of agreement should be assured before action arising from the recommendations is undertaken.

SPECIFY TRAINING REQUIREMENT 2.102

2.102 Introduction. The early processes of the ‘Analyse’ phase identified that training could contribute to achieving the required workplace performance. The final process of the ‘Analyse’ phase ensures the requirement is formally documented in a TRS. This requirement is expressed in terms of, or relates to, competencies: ie the abilities to perform tasks and duties to expected standards.

2.103 Purpose. The purpose of specifying the training requirement is to provide the baseline for subsequent phases of the implementing the DTM.

2–26


2.104 Trigger. The trigger for beginning work on a TRS is acceptance of a Feasibility Report recommending that formal training be used to achieve a required workplace performance.


2.105 The inputs process and products for specifying the training requirement are summarised in table 2–6 and explained in the following paragraphs.

Table 2–6: Inputs, process and products for specifying the training requirement

Step 1: Identify and apply existing training documentation 2.106

2.106 When developing new or revising established Defence training, consideration should be given to using any existing training documentation, particularly any that uses competency standards/units of competency9 from established training packages or accredited courses. If relevant qualifications, competency standards or other Defence competencies exist, they are to be used and/or customised in preference to developing new ones.

Inputs Endorsed Feasibility Report.Job specification.Occupational analysis data.Existing competency standards. Data from Industry Skills Councils (ISC).Products of compulsory and higher education.

Process Step 1: Identify and apply suitable existing training documentation.Step 2: Customise existing training documentation.Step 3: Develop new competency standards.Step 4: Set up competency details in PMKeyS.Step 5: Determine training context and accreditation action.Step 6: Develop a TRS.Step 7: Gain formal endorsement of the TRS, action, and file.

Products TRS.

9 Department of Education, Science and Technology’s (DEST) Training Package Development Handbook, together with other training documents, uses ‘competency standards’ and ‘units of competency’ synonymously. The only difference is that a competency standard consists of a number of individual units of competency that are closely associated and linked. DI(G) PERS 05–29, annex D refers.

2–27


2.107 A search for existing training documentation should begin by investigating local and then Defence records in training establishments and with training authorities. For competency standards, other important sources are the Defence accreditation cells (within Defence Education Training and Development and individual Services) and the Defence Sector Component of the Public Safety Training Package. A wider electronic search can be undertaken using the Competency Standard folder of the DEST’s National Training Information Service (see http://www.ntis.gov.au). This site has a key-word/phrase search function for identifying existing units of competency. Existing training packages and competencies can also be found through the Australian Training Products Ltd database (see http://www.atpl.net.au) and national and/or state or territory industry training advisory bodies (ITAB10). The ITAB incorporating Defence and serving Defence needs is the Public Safety ITAB.

2.108 Having identified existing training documentation, its suitability for meeting the required performances is evaluated. In making this evaluation, consideration should be given to ensuring that all Defence training documentation uses the levels and terminology of competency-based training, whether or not the training is nationally recognised within the AQTF.

Step 2: Customise existing training documentation 2.109

2.109 If existing training documentation closely matches the requirement it can be customised. This might involve writing performances in competency terms and can lead to the national recognition of those competencies within the NTF. If it already uses competency terminology, customisation is typically limited to the range of variables, the evidence guide and the addition of enterprise-specific (ie Defence) performance criteria. Customisation adds specific information to endorsed competency standards or units to reflect the Defence environment. It is not an open-ended process, for the integrity, or meaning of the standards must not be altered. Further details on the extent to which competency units and standards can be customised are available in DI(G) PERS 05–29, annex C, appendix 2 and in DEST’s Training Package Development Handbook. When competencies have been appropriately customised, they can be listed in the TRS.

10 Note: at the time of publication the ITAB are being reorganised into ISC.

2–28

http://www.ntis.gov.au



http://www.atpl.net.au




Step 3: Develop new competency standards 2.110

2.110 Where the performance requirement is not written in competency terms or existing competencies are unsuitable for use, even with customisation, new competencies must be developed. Whether these new competencies are to be nationally registered or not, they should be developed in accordance with the format used within the NTF. This work is detailed in DI(G) PERS 05–29 and, to a large extent, in part 2 of DEST’s Training Package Development Handbook. These references should be used for the work. When developed, new competencies have seven appropriately worded components:

a. Unit code—a unit’s identifying reference can either be registered as part of a recognised training package and awarded a unit code in accordance with the national coding system, or it might be an industry specific reference (those for Defence have DDD as a prefix), DI(G) PERS 05–29, annex C, appendix 1 refers.

b. Unit title—a succinct statement of the broad area of competency covered expressed in terms of the outcome.

c. Unit descriptor—an expansion of the title providing additional information relating to its purpose and relationship with other units of competency.

d. Elements of competency—the functions, described in demonstrable and assessable outcome terms that are the basic building blocks of a unit of competency.

e. Performance criteria—statements specifying what is to be assessed and the required level of performance to verify competent performance.

f. Range statement—a description of any contextual variables that will be used or encountered when applying the competency in a workplace situation.

g. Evidence guide—direction of how to assess the unit or learning outcome, with advice on the appropriate context and scope of the competency in its application.

2–29


2.111 Key competencies. Key competencies are the generic, underpinning, transferable skills/qualities essential for effective participation in work11. Seven key competencies have been defined: collecting, analysing and organising information; communicating ideas and information; planning and organising activities; working with other’s and in teams; using mathematical ideas and techniques; solving problems, and using technology. Units of competency in a training package can identify which key competencies are relevant and should specify one of three performance levels:

a. Level 1—undertakes tasks effectively using routine approaches;

b. Level 2—manages tasks selecting from routine approaches; and

c. Level 3—evaluates and reshapes tasks to establish new approaches.

Step 4: Setup competency details in Personnel Management Key Solutions 2.112

2.112 When the competency units required for workplace performance are agreed, they are entered in the Defence human resource database, PMKeyS. When there, they can be linked to the data on Defence personnel in the system and a competency profile developed. This profile is used for the award of proficiency’s (qualifications) and for the management of Defence people.

2.113 The setup of competency units in PMKeyS involves entering:

a. the criteria underpinning elements;

b. the elements, and then attaching the relevant criteria; and

c. the competency units, and then attaching the relevant elements.

Step 5: Determine training context and accreditation action 2.114

2.114 At this stage of the ‘Analyse’ phase, just before its primary output is documented, it is appropriate to consider whether the training will be recognised by award of an AQTF qualification, how it will be recognised in the Defence system, and how it relates to the learner’s career development.

11 Related terminology for key competencies are ‘generic skills’ and ‘employability skills’. DEST is presently undertaking several studies in how to use these concepts.

2–30


2.115 A learner’s achievement of competence is recognised by the award of a qualification or statement of attainment (SOA) recognised within the AQF12, or an organisation-based qualification. There are three types of AQF qualification:

a. a qualification that is listed as part of an endorsed training package and is endorsed by the National Training Quality Council;

b. an industry specific qualification that is accredited by a State or Territory Training Authority (Defence uses the Australian Capital Territory Accreditation Authority), and either:

(1) uses competencies from one or more training package(s) and/or groups of endorsed national competency standards; or

(2) is not linked to either a relevant training package or endorsed competency standards. (ie a training need has been identified and competency standards have not, or are not likely to be developed in the area covered).

Organisation-based qualifications may not be recognised by authorities outside Defence.

2.116 Where a combination of training and workplace experience equates to one or more competencies that can be mapped against the requirements of a nationally recognised qualification, then this should be done. The RTO (Navy, Army, Air Force, the Defence Learning Services Network or contracted RTO) is then approached to ensure that those units/qualifications are within their scope of registration and to determine any RTO requirements for their delivery and assessment.

2.117 Where training and workplace experience do not directly map to existing competencies or qualifications, the Analyse Authority, together with the RTO, must decide on whether to :

a. add new units or qualifications to an existing training package (such as the Defence Sector to the Public Safety Training Package);

b. apply to develop a new accredited course13; or

12 An AQF qualification recognises achievement (through education, training, experience etc) of a discrete package of competencies, and an SOA is a certificate issued by an RTO that recognises that part of a qualification has been attained.

2–31


c. issue an organisation specific qualification, which may be an SOA.

2.118 Full details of how to package competencies into AQF qualifications are contained in section 2.4 of DEST’s Training Package Development Handbook and in its Australian Quality Training Framework Guidelines for Course Developers: A guide to developing VET courses for accreditation under the Australian Quality Training Framework, July 2002. The guidance in these references should be followed when establishing or amending qualifications.

2.119 Incorporating new or amending existing qualifications in the AQF can be a lengthy process. It may be implemented concurrently with work on other phases and processes of the DTM.

2.120 If the new training is likely to affect the career development of learners as individuals or groups, particularly Australian Defence Force members, then their career manager should be consulted to ensure that any implications are properly managed.

Figure 2–3: Algorithm illustrating the decision-making process

13 An accreditation submission requires information available in draft form at the end of the ‘Design’ phase and confirmed following piloting at the end of the ‘Develop’ phase.

2–32


Step 6: Develop a Training Requirement Specification 2.121

2.121 A TRS specifies the training requirement in terms of the people needing training (the required throughput, and the characteristics of the target population), the required product (the competencies the learners should gain), specifications for the assessment and training (including anticipated major resource requirements), and how achievement of the learning will be recorded. It contains information collected, analysed and developed throughout the ‘Analyse’ phase. A template for a TRS is in annex H.

Step 7: Gain formal approval of the Training Requirement Specification, action and file 2.122

2.122 The job or BPO (for the PO) has the expertise and responsibility that, as the Analysis Authority, approves the TRS. The approved document is a key audit and internal ‘self assessment’ reference of the training system. Consequently it must be appropriately filed.

Annexes:A. Flow chart showing the ‘Analyse’ phase of the Defence training modelB. Defence training model—purpose, inputs, process and products of the

‘Analyse’ phaseC. Defence job specification—templateD. Target Population Profile—templateE. Target Population Capability Record—templateF. Performance Gap Statement—templateG. Defence Feasibility Report—templateH. Training Requirement Specification—template

2–33


FLOW CHART SHOWING THE ‘ANALYSE’ PHASE OF THE DEFENCE TRAINING MODEL A

2A–1

Annex B to ADFP 7.0.2 Chapter 2

DEFENCE TRAINING MODEL—PURPOSE, INPUTS, PROCESS AND PRODUCTS OF THE

‘ANALYSE’ PHASE B

‘Ana

lyse

’ PH

ASE

PUR

POSE

INPU

TSPR

OC

ESS

PRO

DU

CTS

To s

cope

the

natu

re o

f the

pe

rform

ance

pr

oble

m/o

ppor

tuni

ty a

nd

dete

rmin

e w

heth

er th

e D

efen

ce tr

aini

ng m

odel

sh

ould

be

appl

ied

Com

mun

icat

ion/

dire

ctio

n pr

opos

ing

train

ing

inte

rven

tion,

sub

ject

mat

ter

expe

rt st

akeh

olde

rs, r

elat

ed

docu

men

tatio

n, jo

b si

tuat

ion

Nee

ds a

sses

smen

tN

eeds

Ass

essm

ent R

epor

t

To d

escr

ibe

the

care

er/jo

b/ta

sk c

apab

ility

re

quire

men

t

Job

spec

ifica

tions

, dut

y st

atem

ents

, sel

ectio

n cr

iteria

, pro

cess

man

uals

an

d cu

rrent

com

pete

ncy

stan

dard

s an

d un

its. P

roje

ct

requ

irem

ents

, ex

erci

se/o

pera

tion

repo

rts.

Obs

erva

tions

, res

earc

h,

surv

eys

and

cons

ulta

tion

with

Bus

ines

s P

roce

ss

Ow

ners

, cap

abili

ty a

nd tr

ade

spon

sors

, su

perv

isor

s/m

anag

ers

and

job

hold

ers.

Leg

isla

tive

and

licen

sing

requ

irem

ents

and

or

gani

satio

nal p

olic

ies.

Job

anal

ysis

(of c

urre

nt o

r fo

rese

en jo

b, o

r wor

k gr

oup)

Job

spec

ifica

tion

2B–1

ADFP 7.0.2

PUR

POSE

INPU

TSPR

OC

ESS

PRO

DU

CTS

To d

escr

ibe

the

char

acte

ristic

s of

the

grou

ps/in

divi

dual

s w

ho a

re

to p

erfo

rm th

e jo

b

Wor

k fo

rce

reco

rds

and

plan

s. E

duca

tion

and

train

ing

reco

rds,

sur

veys

of

likel

y le

arne

rs a

nd

supe

rvis

ors/

man

ager

s,

test

ing

of li

kely

lear

ners

.

Targ

et p

opul

atio

n an

alys

isTa

rget

Pop

ulat

ion

Prof

ile o

f lik

ely

lear

ners

To s

peci

fy th

e hu

man

di

men

sion

s of

the

wor

kpla

ce p

erfo

rman

ce

prob

lem

Job

spec

ifica

tion

and

Targ

et

Popu

latio

n Pr

ofile

Det

erm

ine

perfo

rman

ce g

apPe

rform

ance

Gap

St

atem

ent

To d

eter

min

e an

d re

com

men

d th

e be

st w

ay

of a

chie

ving

the

care

er/jo

b/ta

sk c

apab

ility

re

quire

men

t

Sta

tem

ent o

f Per

form

ance

N

eeds

, dat

a on

exi

stin

g vo

catio

nal e

duca

tion

and

train

ing,

hig

her e

duca

tion,

pr

ofes

sion

al d

evel

opm

ent,

and

othe

r cur

rent

and

po

ssib

le s

olut

ions

Feas

ibili

ty a

naly

sis

Feas

ibili

ty R

epor

t, w

ith

train

ing

and

non-

train

ing

solu

tions

, tog

ethe

r with

es

timat

es o

f thr

ough

-life

re

sour

ce a

nd c

osts

es

timat

es

To fo

rmal

ly re

cord

the

train

ing

requ

irem

ent i

n co

mpe

tenc

y te

rms

Feas

ibili

ty R

epor

t, jo

b sp

ecifi

catio

n, o

ccup

atio

nal

anal

ysis

dat

a, e

xist

ing

com

pete

ncy

stan

dard

s,

indu

stry

trai

ning

adv

isor

y bo

dies

, com

puls

ory

and

high

er e

duca

tion

outp

uts

Trai

ning

Req

uire

men

t S

peci

ficat

ion

Trai

ning

Req

uire

men

t Sp

ecifi

catio

n

2B–2

Annex C to ADFP 7.0.2 Chapter 2

DEFENCE JOB SPECIFICATION—TEMPLATE1C

JOB TITLE 1

Introduction 1

1. Job classification. The job title is classified in Defence as follows:

a. Job code number. Provide the applicable job identification or reference descriptor.

b. Job designation. Provide a context for the job (eg Service core/branch/category, civilian group/occupational group)2.

c. Job description. Provide a short description3 of the role of the job4.

d. Function. Describe the function of the job5 (preferably in a single sentence)6.

e. Sponsor. Insert details of the job sponsor/Business Process Owner (BPO)7.

1 This template is provided as a guide. The scope and content of a job specification should be appropriate to its context.

2 For example—a crewman Australian Light Armoured Vehicle (ASLAV) is a Royal Australian Armoured Corps soldier, who may be either a permanent force or Reserve member.

3 For example—a crewman operates allocated equipment and maintains the ASLAV as a member of an armoured fighting vehicle crew. Participates in the conduct of reconnaissance and surveillance in support of operations in either a mounted or dismounted role.

4 If the job spans a number of ranks and grades that cannot be summarised in a single description statement, include sub-paragraphs for each level.

5 For example—to drive, operate and maintain an ASLAV.

6 Again, if the job spans a number of ranks and grades that cannot be summarised in a single function, include sub-paragraphs for each level.

7 Most Service job positions will have either a career or functionally-based sponsor. Generic work, as applicable to most civilian positions, is more likely to be managed by a BPO.

2C–1

ADFP 7.0.2

f. Rank/level. If a job embraces a range of ranks/levels, indicate the range, such as ‘PTE to WO1’. If specific ranks/levels/pay groups carry out the job, list them.

g. A possible layout is shown below:

8

2. Job requirement. The Defence requirement for the job title has the following specifications:

a. Numbers. State, as accurately as possible, the number of job positions in Defence. If changes are anticipated, this should be made clear.

b. Throughput. An estimate of the demand for trained people as expressed in numbers against time (either specific calendar dates, units of time or both).

c. Where employed. Describe the geographical locations and/or organisational units in which the job normally occurs (including Reserve units for Service positions).

3. Job analysis background. Provide some background to, and explanation of the analysis that led to this specification. Reference to preceding specifications might be made, as well as the reason for the analysis, when and how it was conducted, and any limitations that apply.

JOB CHARACTERISTICS 2.4

4. The following sections are indicative. They and/or others should be omitted or incorporated in accordance with their significance.

Duty task inventory 2.5

5. The duties, tasks and sub-tasks that comprise the job, together with details about the required performance levels for each unit, are listed in the duty task inventory (DTI) in appendix 1. Each listed element of the job should begin with an action verb and be described concisely.

Job code number8 Operator PQR Provisional PTE PG 1Job code number Operator PQR Grade 1 PTE–LCPL PG 2Job code number Operator PQR Grade 2 LCPL–CPL PG 3Job code number Supervisor PQR SGT–WO2 PG 3Job code number Manager PQR WO2–WO1 PG 4

8 If the entire career is in one category, the job code letters will be consistent (eg ECN 074 Clerk Administration and ECN 078 for Clerk Supervisor).

2C–2

ADFP 7.0.2

6. Existing units of competencies that apply to the job are listed in appendix 1. Current units of competency that apply to parts of the job should be listed as they are identified. This work complements and does not replace any completion of the DTI. It reduces the risk of unnecessary duplication later in the development process.

Responsibilities 2.7

7. Subordinates. State the supervisory requirement, indicating (where applicable) numbers of Service personnel and Defence civilians by ranks and levels, and/or employment groupings.

8. Delegations. Describe any delegations that relate to specific roles eg supervisors are likely to hold personnel delegations, and particular job holders could have financial delegations.

9. Classified matter. Describe any responsibility for classified matter, including the highest level associated with the job.

10. Stores, equipment and finance. Describe the level of supervision required for management of equipment and/or accounting procedures. Where materiel is of an attractive or valuable nature this is to be stated, as well as the level of supervision for demanding, receiving and issuing these items. Describe any responsibilities for financial matters such as bookkeeping or maintenance of accounts.

11. Assessment and training. Describe the responsibility for assessment and training, for on-the-job training (OJT) and formal training in the organisation, together with any requirement for specific qualifications.

12. Consequence of error. The consequence of error can involve the following:

a. Safety of others. Describe the responsibility for safety of others associated with this job, considering the type of work performed and the levels of risk involved.

b. Effect on the organisation’s mission. Describe the effect of error on the organisation’s mission. Be specific and avoid broad descriptions. Include a description of any consequential effects on other parts of Defence.

2C–3

ADFP 7.0.2

Equipment 2.13

13. List of equipment. List the major equipment used (specifically) in this job. Refer to generic equipment types rather than specific items and exclude minor items of equipment, complete equipment schedule and tools.

Environmental conditions 2.14

14. Working conditions. The working conditions for the job are as follows:

a. describe the conditions under which work is usually performed, with particular regard to exposure to climatic conditions, ventilation, working space and duration of work; and

b. differentiate between the conditions pertaining to any possible circumstances (such as exercise or combat, base and field).

15. Most demanding job conditions and hazards. Summarise the most demanding/highest duty conditions and any associated hazards. These must be stand-out duties. Be succinct:

a. Non-combat. Identify the non-combat or industrial conditions and hazards associated with the job. When the degree of risk requires qualification indicate this accordingly.

b. Combat. Indicate the degree to which any Service person is involved in combat, together with an assessment of the associated risks. If non-combat hazards are intensified in combat, indicate this also.

16. Team/social environment. The social environment is as follows:

a. Consider the need for and level of personal contact in undertaking the job and/or associated with it. Describe the type usually encountered and the most demanding.

b. Indicate the frequency of contacts with other Service personnel and civilians; and the requirements for handling these situations; the need for tact as part of a small team; and the ability to communicate, supervise, advise, answer questions and interview.

2C–4

ADFP 7.0.2

17. Physical demands. The physical demands of the job are as follows: describe both the normal and the highest physical demands placed on the job eg physical or environmental discomfort.

18. Physical skills. The physical skills required by the job are as follows: describe the physical skills required in the job, eg coordination and precision in operating equipment or manual skills.

19. Mental demands. The mental demands of the job are as follows:

a. Complexity and judgment. Illustrate the complexity of the work to be performed and the degree of judgment required by the job holder.

b. Scope of action. Explain the scope of action or the requirement for independent action, resourcefulness and ingenuity that is part of the job.

c. Communication skills. Describe the requirement for written and oral expression in the job.

20. Distastes. State any features of the job or duties which could cause certain job holders to be ineffective in their duties. Do not list factors that are accepted as being a normal requirement of military life.

21. Immediate supervisor. State the rank/level to whom the job holder is normally responsible.

Job holder specifications 2.22

22. Medical. State any relevant medical requirements from Service or other regulations (including medical categories). When issued electronically, this paragraph will contain a link to the parent document.

23. Psychological. State any specific psychological requirements for the job.

24. Security classification. State the minimum level of security classification required for the job9.

25. Other special requirements. State any other special requirements not covered above such as whether the job holder has to be a volunteer, any restrictions relating to physical size and the ability to swim.

9 This is not to include security clearances that are related to specific appointments or roles that may be undertaken by only some members.

2C–5

ADFP 7.0.2

2C–6

Employment and career factors 2.26

26. Prerequisite employment. List any career pathways, jobs or positions that are prerequisites for this job.

27. Prior training, qualifications, licenses and experience. State all formal and OJT, qualifications, civil licensing or other mandatory requirements, and/or experience known to be required before either starting in, or beginning training for the job.

28. Recognition of current competency or prior learning. Identify any recognition of current competency or recognition of prior learning issues.

29. Career profile and flow chart. The career profile to which the job relates, is in appendix 2. Relate progression from entry to exit in the career stream specific to that the job only. The career flow chart illustrating this career profile is in appendix 3. It must illustrate diagrammatically the details shown in the career profile. Any relevant training management system Personnel Management Key Solution career logic flow chart should be included as an annex to this document.

30. Follow on categories. List career pathways that the job holder can follow as direct progressions from this job.

31. Applicable units of competency. The existing units of competency applicable to this job are listed in annex H, appendix 2. This information is an incidental product of the job analysis, recorded at the time of data collection, but not necessarily researched. The competencies requiring training are identified in full when the training requirement is specified.

32. Accreditation. Provide details of any civil accredited qualifications that job holders might receive on attainment of appropriate units of competency (Defence Instructions refer, and details can be found on the accreditation web site see http://www.defence.gov.au/dpe/civilquals). The information may also cover any civilian licences awarded.

33. Authorisation. Provide position and contact details of the job sponsor/BPO authorising this document. This information provides context to the signature block.

http://www.defence.gov.au/dpe/civilquals

http://www.defence.gov.au/dpe/civilquals

ADFP 7.0.2

Miscellaneous 2.34

34. Miscellaneous information. List any information considered to be of relevance but not covered elsewhere in the job specification. Examples include the need to maintain skills and comply with ethical practices and any specific values/codes of conduct.

10

Appendixes: 1. Duty task inventory2. Career profile3. Career flow chart

Prepared by:10 Signature Authorised by: SignatureName NameRank RankPosition Position

Date Date

10 Finalised job specifications are to include the following signature blocks. They are to be forwarded to designated group authorities including those for job/position management and training:

a. the individual who prepared the specification, normally a career/trade manager, but possibly a training specialist; and

b. final sign off is by the sponsor/BPO of the job. This should not to occur until all major stakeholders have reviewed and provided feedback about the specification.

2C–7

Appendix 1 to Annex C to


DUTY TASK INVENTORY11

JOB/JOB TITLE 1

2

Note

(a) The document describes work performed, it does not list the competencies that have some relevance to that work. Existing units of competency applicable to this job are listed below.

1 A separate inventory will be required for each rank/grade.

DUTY:

Job composition Required performance

criteriaDTI Ref

Dif Imp Freq

TASK:SUB-TASKS:2

TASK:SUB-TASKS:

Key

Dif difficulty Imp importance Freq frequency

E easy L low D at least daily

S straightforward M medium W weekly

D difficult H high M monthly

VD very difficult C critical R rarely

2 To identify sub-tasks satisfactorily it may be necessary to identify the task elements that enable that performance.

2C1–1

ADFP 7.0.2
APPLICABLE UNITS OF COMPETENCY 2.2

Name of job 2

2. The units of competency3 known to be required for the (job title) job are listed below beside the associated task identifier from the duty task inventory (DTI) of the job specification.

3 Only the units of competency are to be listed. The elements of competency, performance criteria, range of variables and evidence guide will be described in the related competency standard contained in the training package.

DTI ref number Unit reference Unit title

2C1–2



CAREER PROFILE 2

Note

(a) The career profile and the components listed should be included if relevant.

Job code number

Rank/level

Career development/

promotion courses

Professional/trade courses

Number required

2C2–1



CAREER FLOW CHART 3

One or more simple linear flow charts with amplifying notes may be provided to support the career profile introduced in annex C, appendix 2.

2C3–1

Annex D to ADFP 7.0.2 Chapter 2

TARGET POPULATION PROFILE—TEMPLATE1D

JOB TITLE 1

General 1

1. Job classification. The job for which the target population is described is classified as follows: The information listed here should match that provided in the job specification.

a. Designation. Provide any job identification or reference descriptor.

b. Situation. Provide a context for the job (eg Service core/branch/category, civilian group)2.

c. Job description. Provide a short description3 of what the job involves4.

d. Function. Describe the function of the job5 (preferably in a single sentence)6.

e. Owner. Insert details of the job or Business Process Owner (BPO).

1 This template is provided as a guide. The extent of its use and the depth of the content it covers should be appropriate to the context.

2 For example—a crewman Australian Light Armoured Vehicle (ASLAV) is a Royal Australian Armoured Corps soldier, who may be either a Permanent Force or Reserve member.

3 For example—a crewman operates allocated equipment and maintains the ASLAV as a member of an armoured fighting vehicle crew. Participates in the conduct of reconnaissance and surveillance in support of operations in either a mounted or dismounted role.

4 If the job spans a number of ranks and grades that cannot be summarised in a single description statement, include sub-paragraphs for each level.

5 For example—to drive, operate and maintain an ASLAV.

6 If job spans a number of ranks and grades that can not be summarised in a single function, include sub-paragraphs for each level.

2D–1

ADFP 7.0.2

Target population group 2.2

2. Classification. The target population for the job title is defined by the following factors: The list might include:

a. career, trade, rank/level, experience; and

b. vocational education and training and educational qualifications.

The section may contain reference to the homogeneity of the group and highlight any significant disparities in its members.

3. Education and training prerequisites. Both mandatory and/or desirable education and training prerequisite qualifications might be listed.

4. Professional experience prerequisites. Both mandatory and/or desirable prerequisite employment experiences might be listed.

5. Group characteristics. Briefly state the characteristics of the work group people required to undertake the job, including: the numbers required in any pertinent time frames, any categories or classifications, any sub-groups evident, the locations of the work and any special factors affecting it.

6. Geographic distribution. Identify the present locations of the target population. Indicate the extent to which its members are collocated (as in a barracks or training establishment), or dispersed.

Target population characteristics 2.7

7. Proficiencies. The following proficiencies are characteristic of individuals in the target population that are relevant to the job. The qualifications and records (eg certificates and licences) indicating outcomes of education, training and employment and other experiences of relevance to the job situation are listed.

8. Other factors. Listed below are other factors that might influence decisions about training strategies and which are not covered elsewhere in the Target Population Profile. Possible fields include:

a. academic record and learning ability;

b. employment and training background;

c. age, gender, physical characteristics, learning preferences and special aptitudes;

2D–2

ADFP 7.0.2

d. attitudes to the intended change to working practice;

e. workplace environment (in terms of suitability for assessment and training);

f. any factors limiting availability for training; and

g. special or unusual group characteristics.

78

Prepared by:7 Signature Authorised by:8 SignatureName NameRank RankPosition Position

Date Date

7 The individual who prepared the specification, normally a career/trade manager, but possibly a training specialist.

8 Final sign off is by the sponsor/BPO of the job. This should not to occur until all major stakeholders have reviewed and provided feedback about the specification.

2D–3

Annex E to ADFP 7.0.2 Chapter 2

TARGET POPULATION CAPABILITY RECORD—TEMPLATE E

JOB/JOB TITLE 1

1. This form is a job aid rather than a formal record of the ‘Analyse’ phase.

2. The composition of the job title is described by the performances listed in the following table. Any unit of competency known to match specific tasks is identified. For each task sub/task performance, the target population either has the necessary skills, knowledge and attributes that enable satisfactorily performance, or is lacking competence. This determination is indicated by a

¸ in either the ‘currently able’ or ‘lacking competence’ column.12

Duty:

Job composition Unit of competency reference (if any)

Target population capability1

Ref No currently able

lacking competence

TASK:SUB-TASKS:2

TASK:SUB-TASKS:

1 Using information from the Target Population Profile, the classification should not be contentious. If necessary, footnotes explaining decisions can be incorporated.

2 To identify sub-tasks satisfactorily it may be necessary to identify the task elements that enable that performance.

2E–1

Annex F to ADFP 7.0.2 Chapter 2

PERFORMANCE GAP STATEMENT—TEMPLATE1

F

JOB TITLE 1

1. This document specifies the performance gap—the difference between the performances required for a job and the abilities of members of the target population. The Performance Gap Statement has two parts: the first lists the job tasks that the target population cannot presently perform, and the second presents information explaining the scale and significance of the performance problem.

2. The following table lists the tasks in which members of the target population lack workplace competence:

Related considerations 2.3

3. The following factors indicate the scale of the performance gap and, thereby, its significance:

a. List, possibly using a table, the number of current and/or anticipated job holders/vacancies that lack competence. Where significant the dates and details of any changes to this specification should be provided.

b. Provide an indication of the importance/criticality of the gap, and therefore the need to fill it.

c. Identify any jobs/performances that are dependent on the gap being closed, together with any urgency to the need.

d. As relevant identify the location of the jobs and/or members of the target population.


Duty:

Task number

Task description Unit code of any associated competency

2F–1

Annex G to ADFP 7.0.2 Chapter 2

DEFENCE FEASIBILITY REPORT—TEMPLATE1G

FOR DESIGNATING, AND ASSESSING THE COMPETENCY OF JOB HOLDERS FOR THE:

JOB TITLE

INTRODUCTION 2.1

Background 1

1. This report presents an analysis of options for:

a. ensuring job holder competence for the job title job; and

b. assessing a job holder’s competency to undertake this work.

2. It identifies the strengths and weaknesses of each option and recommends the best for implementation.

3. Provide a brief description of the situation that required the ‘Analyse’ phase of the Defence training model and led to the definition of the performance gap. The nature and scope of the gap must be explained, for which the relevant Performance Gap Statement might be included as an annex. Any Terms of Reference and/or limitations should also be stated.

4. The depth and extent of the investigation behind the paper should be described and any assumptions acknowledged.

5. The background should identify the authorities consulted in the preparation of the report.

Structure 2.6

6. Explain the structure of the report, and the way in which the evidence is presented and compared to derive the conclusions and related recommendations. The report may, for example, consider the tasks of the Statement of Performance homogeneously or they might be divided into two or more groups in accordance with particular characteristics.

Aim 2.7

7. State that the aim of the report is to recommend the best solutions for meeting the required job performance.


2G–1

ADFP 7.0.2

OPTIONS FOR ENSURING JOB HOLDER COMPETENCE 2.8

8. List all the appropriate existing or potential options that might be used for ensuring that Defence personnel are positioned to undertake the performances listed in the Performance Gap Statement. If current policies, time or resource constraints limit the options or their implementation, this should be explained.

9. For each option:

a. describe the option; and

b. state the advantages and disadvantages of implementing the option, including estimates of effectiveness (in monetary and other units of benefit), costs and deployment considerations (such as time and major resource implications). The aim of this section is to present data that can be compared (for training options this may require seeking estimates from potential contractors, consideration of instructional facilities and equipment, accommodation and catering facilities, and the opportunity costs of using current capacity. Other likely factors include instructor profiles: current numbers, rank/level categorisation, their competencies and capacity; and the capacity to develop courses). Where estimates and judgments are used, this should be identified.

Comparison of options 2.10

10. Present a structured, logical means of comparing the options that clearly identifies the stronger and weaker options or sets of options. If the performances constituting the gap are being treated homogeneously, only one set of comparisons will be required. If they have been divided into groups, a comparison will be needed for each separate group. The text should clearly illustrate progress towards a logical conclusion. A decision matrix or paired comparative analysis can help in this.

Conclusion 2.11

11. The conclusion presents the best option(s) for designating the Defence personnel who will become undertake the performances listed in the Performance Gap Statement.

2G–2

ADFP 7.0.2

OPTIONS FOR ASSESSING COMPETENCE 2.12

12. Present the options for assessing the competence of Defence personnel to undertake the required performances. Do so in the approximate order of their validity. If current policies, time or resource constraints limit the options or their implementation, this should be explained.

13. For each option:

a. provide explanatory detail; and

b. state the advantages and disadvantages of implementing the option, including estimates of effectiveness (in monetary and other units of benefit), costs and any deployment considerations (such as time and major resource implications). The aim of this section is to present data that can be compared (eg availability of environments, staff, facilities and equipment, accommodation and domestic facilities, and any significant opportunity costs). Where estimates and judgments are used, this should be identified.

Evaluation of options 2.14

14. Present a structured, logical evaluation of the options for assessing competency. The process is one of selecting the option or mix of options that has the greatest validity, suitability and reliability in relation to the costs involved. The text should clearly illustrate progress towards a logical conclusion.

Conclusion 2.15

15. The conclusion presents the best option for assessing the competence of the Defence personnel designated to undertake the performances listed in the Performance Gap Statement. It should also identify any implications arising from implementing this option.

2G–3

ADFP 7.0.2

Recommendations 2.16

16. The recommended course of action follows from the two conclusions is presented. The section should state who would be responsible for implementing the recommendations, together with any pertinent time frames and resource requirements.

Completed Feasibility Reports are to be forwarded for the attention of the stakeholders of the performance requirement.23


Date Date

2 The individual who prepared the Feasibility Report, normally a career/trade manager.

3 Final sign off is by the sponsor of the job. This should not to occur until all significant stakeholders have reviewed and provided feedback about the report.

2G–4

Annex H to ADFP 7.0.2 Chapter 2

TRAINING REQUIREMENT SPECIFICATION—TEMPLATE H

JOB TITLE 1

1. Job classification1.

a. Designation. Provide any job code or reference number.

b. Job description. Provide a short description of the role of the job.

c. Function. Describe the function of the job (preferably in a single sentence).

d. Sponsor. Insert details of the job sponsor.

Background 2.2

2. Briefly provide an account of the circumstances giving rise to the requirement.

3. Supporting documentation. This Training Requirement Specification (TRS) is based on analysis work that can be referred to in the documentation listed below:

Training requirement—people 2.4

4. Training throughput. Specify the numbers of trained people required, and any specified or preferred time frames, starting and/or completion dates.

5. Trainee profile. List the significant target population characteristics identified in the Target Population Profile. Include information how the training relates to individuals or groups’ career profiles.

1 The details of this section should match those of the job specification.

Document Location DatedReport of needs assessment Job specificationTarget Population ProfilePerformance Gap StatementFeasibility Analysis Report

2H–1

ADFP 7.0.2

Training requirement—competencies 2.6

6. Those competencies requiring training for the job title are summarised in the table in appendix 1. Those competencies that have prerequisite or co-requisite competencies are identified. The type(s) of training deemed appropriate for each competency2 is/are also identified.

7. Summary of competencies. A summary of the competencies requiring training is provided in appendix 1. The format conforms to the nationally consistent system used by Australian industries and training organisations.

8. Competency details. Details of each of the competencies requiring training are provided in appendix 2. They include the:

a. elements that combine to enable the competency,

b. key competencies that underpin the competency,

c. performance criteria by which successful achievement of competency elements are measured,

d. range statement that specifies the conditions under which the competency is performed, and

e. evidence guide that states how assessment of competency will be achieved.

Training and assessment specifications 2.9

9. This section should broadly specify how the training will be conducted (eg formal program(s), course(s), on the job, contractor provided) and how the required competency will be assessed (eg on-the-job, off job, through exercises, in training environment etc). It should also specify the environments in which these processes occur and any major resource implications, this information having been considered and reported in the Feasibility Report. The information is important, as it provides the major parameter for determining the assessment and training strategies in the ‘Design’ phase.

10. Any other known constraints or limitations that might influence the design and conduct of training (such as timing and resources) should also be identified in this section.

2 This information is derived from the feasibility analysis that considered the options of course training, distance-based course training and on-the-job training.

2H–2

ADFP 7.0.2

Learning accreditation 2.11

11. Describe how successful achievement of the training will be recognised (eg award of an Australian Quality Training Framework qualification, a statement of attainment, or some other proficiency award)3.

45

Appendixes: 1. Summary of competencies to be trained2. Details of competencies to be trained


Date Date

3 Finalised TRS are to include the following signature blocks.

4 The individual who prepared the specification, normally a career/trade manager, but possibly a training specialist.

5 Final sign off is by the sponsor/Business Process Owner of the job. This should not to occur until all major stakeholders have reviewed and provided feedback about the specification.

2H–3

Appendix 1 to Annex H to


SUMMARY OF COMPETENCIES1 TO BE TRAINED1

1C

ompe

tenc

ies

may

be

eith

er n

atio

nally

reco

gnis

ed a

nd w

ithin

the

Aus

tralia

n Q

ualif

icat

ions

Fra

mew

ork

or D

efen

ce-s

peci

fic.

Ref

eren

ce: c

ode

for u

nit/e

lem

ent o

f co

mpe

tenc

y, o

r ot

her

Dut

y/ta

sk

num

ber(

s)Ti

tle o

f uni

t or

elem

ent

Prer

equi

site

co

mpe

tenc

ies

Co-

requ

isite

co

mpe

tenc

ies

Type

of

trai

ning

re

quire

d

2H1–1

Appendix 2 to Annex H to


DETAILS OF COMPETENCIES TO BE TRAINED 2

1. Each competence for which training is required should have the following detail listed:

a. Classification

(1) Reference. Provide a reference code, either one provided when the competency is endorsed and registered in accordance with the national coding system, or one that is Defence specific in accordance with Defence Instruction (General) PERS 05–29—Implementation of the National Training Framework within Defence vocational education and training, annex E.

(2) Competency title. Provide the title that identifies the general area of competency encompassed by the unit. The area may relate to the duty level of job classification.

(3) Competency descriptor. Provide a short statement that provides added explanation to the competency title and which helps clarify the scope and intent of the competency.

b. Details

(1) Elements. List the elements that are the building blocks of the competency. They describe the specific outcomes or functions encompassed in the unit that can be individually demonstrated. They relate to the task level of job classification.

(2) Key competencies. Describe the generic, underpinning, transferable skills or competencies essential for effective participation in work. The mix of seven Key competencies applies to work generally, rather than being specific to work in a particular occupation or industry.

(3) Performance criteria. Define the specified performance for the individual elements of the competency. They focus on the evidence required to show that a person can undertake the activity to a level acceptable in the workplace.

2H2–1

ADFP 7.0.2
(4) Range of variables. Set out the context and conditions under which competent performance is expected in achieving the unit as a whole.

(5) Evidence guide. Outline the type and scope of evidence required to assist in determining whether competency have been achieved. The evidence guide identifies a number of relevant factors to be taken into account, including making a judgment about the achievement of competency. Any prerequisite or co-requisite units of competencies and key underpinning knowledge should be identified.

2H2–2


CHAPTER 3

THE ‘DESIGN’ PHASE 3

Introduction 3.1

3.1 The ‘Design’ phase is the second of five phases constituting the Defence training model (DTM), as shown in figure 3–1. In this phase, the assessment and learning strategies for training are selected, together with the strategies for evaluating the whole process of preparing for and then conducting training. The object of these strategies is to ensure that the learning is achieved as efficiently and effectively as possible, and that it meets the original requirement.

Figure 3–1: ‘Design’ phase of the Defence training model

3.2 The ‘Design’ phase involves making decisions about what is to be done, when and where, by whom, with what resources, and how the outcomes are to be measured and evaluated.

3.3 As with all phases, processes and steps of the DTM, the extent to which the DTM is applied should be appropriate to the circumstance. In situations of high significance (eg the introduction of training in support of a major new Defence capability) the DTM should be applied rigorously. Abbreviated processes (such as the rapid development process in chapter 1—‘Introduction’) may be appropriate in lesser circumstances based on the outcomes of a mandatory risk analysis.

3–1


Key product and responsibilities 3.4

3.4 The final and principal product of the ‘Design’ phase is a draft Training Management Package (TMP). A TMP contains a curriculum that describes what is to be learned, how it is to be delivered, and how the learning will be checked; and strategies for Training Review and Evaluation (TR&E). At the end of the ‘Design’ phase a TMP is a draft document because it is likely to be amended during work in the ‘Develop’ phase, and because it needs the learning materials of the training program/course to be added.

3.5 For the ‘Design’ phase, responsibility for shaping/designing the learning process shifts from the job sponsor/Business Process Owner (BPO) (the Analyse Authority), to a ‘trainer’ (the Design Authority). In the Australian Defence Force (ADF) the ‘trainer’ with that responsibility is generally at the Training Command level. In the Department, responsibility for design generally lies with the sponsor. In joint training the responsibility is held by someone agreed by the stakeholders.

Starting point 3.6

3.6 The ‘Design’ phase follows-on from the ‘Analyse’ phase and is based on the latter’s major product, a Training Requirement Specification (TRS).

3.7 Because the ‘Design’ phase of training is concerned with broad strategies, it is most commonly applied to develop new training. Once training is established, the phase may be employed to check and revise the reasoning behind the strategies, and the strategies themselves.

Processes of the ‘Design’ phase 3.8

3.8 The ‘Design’ phase has nine major processes:

a. analyse learners,

b. analyse learning environments,

c. analyse competencies,

d. generate learning outcomes,

e. design assessment strategy,

f. design training strategy and produce the training and assessment strategy document,

g. compile curriculum,

3–2


h. design TR&E strategies, and

i. compile a draft TMP.

Not all of these processes have to be implemented sequentially—as is indicated in the flow diagram in annex A. Paragraphs 3.9–3.17 explains each of the processes and annex B summarises the purpose, inputs and products of each one in table form.

3.9 Analyse learners. Good training is appropriate to the characteristics and circumstances of prospective learners. To help ensure this, the first process of training design is the analysis of those characteristics and qualities. A key area of this analysis is the prospective learners’ existing competencies and capabilities, as it helps minimise over-and under-training. The output of the process is a learner profile—a document that describes the existing competencies and capabilities of prospective learners, their preferred learning style, special aptitudes and motivation.

3.10 Analyse learning and assessment environments. The training and the assessment of job competency skills should be suited to the environments in which they are conducted. A learning and assessment environment analysis helps ensure this by studying the things or conditions in the environments that could affect learning and assessment. The information gained is recorded in an environments profile, and it is used to inform the design of training and assessment strategies.

3.11 Analyse competencies. Most competencies comprise complex performances that are best learned by identifying their components and then training each component in a preferred sequence. In analysing competencies, therefore, the skills (mental and physical), knowledge and attitudes/attributes that a person must acquire in order to perform each element of, and therefore the whole of a competency are identified. This work leads to the development of learning outcomes that serve as the objectives of training. As such, the product of the process, completed Competency Analysis Sheets (CAS), are working documents in the design of training.

3.12 Generate learning outcomes. Learning outcomes are the building blocks of training and learning. They clearly and concisely describe what a learner will be able to do on successful completion of a unit of training, the conditions under which they will do it and to what standard. They are therefore the objectives of the training system. Learning outcomes are generated from the skills, knowledge and attitudes listed on CAS, and using knowledge of the prospective learners and the environments in which training and assessments will be conducted. To be complete, each learning outcome should have an associated method for assessing its accomplishment.

3–3


3.13 Design assessment strategy. Assessment is the process of collecting evidence and making judgments about something. When applied to training, assessment can diagnose learners’ abilities (ie for placement, selection, and identification of strengths and weaknesses), motivate and guide their progress (formative assessment), and determine mastery of something learnt (summative assessment). It can be applied before, during, or on completion of training. For units of competence, it can also be applied following other developmental experience. In designing an assessment strategy, a broad plan is developed for using assessment to support learning and to determine whether intended learning outcomes have been achieved. A comprehensive strategy should state the purpose and approach to be used for assessment, present detail on the methods and sequence to be used, and indicate the likely resource requirements. This information is used to guide the development and conduct of assessment in a training program/course.

3.14 Design training strategy and produce the training and assessment strategy document. The term ‘training’ describes the development of skills, knowledge, attitudes, competencies etc through instruction or practice. A DTM training strategy is therefore a broad methodology for the conduct and management of training conduct. It provides guidance on the why, what, where, when and how of training. It states the purpose, methods, location, activities, media, recommended or prescribed tasks, formative assessment, sequence and likely resource requirements that training will involve. In designing a training strategy, alternative methods of achieving each of the learning outcomes are considered against an understanding of the learner and the learning environments. The purpose of this is to identify and select the option, or mix of options that most effectively and efficiently assess the learning outcomes. While focused on the required learning outcome, a training strategy must complement its associated assessment strategy. To help ensure this, the two strategies are combined for presentation in a training and assessment strategy document.

3.15 Compile curriculum. A curriculum presents the specifications for a training program, course or subject (module). It describes all the learning experiences a learner undergoes, the intended learning outcomes, the training and assessment strategies, the content, teaching methodology, and prescribed or recommended assessment tasks. This information is determined in the preceding processes of the ‘Design’ phase, which means that the process of compiling a curriculum is simply a process of collating the products of these processes. A curriculum is the major component of a TMP.

3–4


3.16 Design training review and evaluation strategies. For the purpose of maintaining the effectiveness and efficiency of training, strategies for two forms of continuous improvement process are produced in the ‘Design’ phase:

a. Training review. The evaluation and reporting of the conduct of training.

b. Evaluation. The evaluation and reporting of how well training has contributed to meeting the original performance requirement.

As with the other strategies in the ‘Design’ phase, the two strategies for these processes provide broad guidance on what, where, when and how. They cover the methods to employ, when to employ them and the resources required. Contained in a TMP, the two strategies are used in the ‘Develop’ phase for producing the materials for the TR&E document and in the ‘Conduct’ phase for ensuring the processes are properly administered.

3.17 Compile draft Training Management Package. The final process of the ‘Design’ phase is the compilation of a TMP. A complete TMP is the authoritative reference for the conduct of a training program or course. It contains the curriculum of a program/course, its TR&E strategies, and the materials developed for conducting the training. At the end of the ‘Design’ phase a TMP is in draft form and incomplete. This is partly because the strategies contained are likely to change consequent to work during the ‘Develop’ phase, and partly because the materials for conducting a program/course are added during that phase.


3.18 While working on any of the processes of the ‘Design’ phase, stakeholder input is sought to help assure the quality of the product. A consequence of this is that products are likely to be revised many times before acceptance. To keep track of the developments and to enable reference during later development work, an accurate and comprehensive record of development activity should be maintained.


3.19 In accordance with national and Defence guidelines, the quality assurance of all training by, or on behalf of Defence (by Defence staff, contractors, professional service providers or other Registered Training Organisations (RTO)) remains a Defence responsibility. External agencies may, however, be used to undertake design work. When this is being considered, the potential for packaging design and development work in a single contract should be looked at.

3–5


ANALYSE LEARNING AND ASSESSMENT ENVIRONMENTS 3.20

Introduction 3.20

3.20 For training and assessment of job competency to be effective and efficient they must be tailored for the environments in which they are to occur. It follows that the design of training and assessment strategies requires an understanding of these environments. Such understanding is gained by analysis.

3.21 Analysis of learning and assessment environments involves evaluation of the environmental influences that might affect the processes. The results are recorded in an environments profile and are used to inform the design of training and assessment strategies.

3.22 An analysis of learning and assessment environments complements the feasibility analysis work of the ‘Design’ phase. The scope and scale of an analysis will depend on the comprehensiveness of that work, as reflected in both the TRS and the Feasibility Report. Those documents should specify the intended training and assessment environments. Where they do not, a selection will have to be made. The environments profile provides a sound basis for this judgment.

3.23 Triggers. An analysis of learning and assessment environments is most commonly undertaken for the development of new training. In this circumstance the trigger for the work is approval of a TRS. Analysis can also be worthwhile when changes to environments offer potential improvements for current assessment and training, or when changes to either the training requirement or the learners arise. When existing processes might be redeveloped, the environments profile should be reviewed and, if necessary, updated—an outcome that might necessitate corresponding amendments to dependent training documentation.

3.24 A learning and assessment environments analysis is one of three processes that may be undertaken concurrently at the beginning of the DTM’s ‘Design’ phase: the others being analysis of the learner and the competencies that require training.

3.25 Purpose. The purpose of analysing learning and assessment environments is to help ensure that the associated strategies are appropriate.

3–6



3.26 The inputs, process and products of a competency analysis are summarised in table 3–1 and explained in the following paragraphs.

Table 3–1: Inputs, process and products of a competency analysis


3.27 The first step in analysing learning and assessment environments is to determine the information requirements. For this, the TRS is reviewed to identify the learning and assessment environments that are to be analysed. Ideally the TRS describes specific environments (such as on-the-job, Service/Defence facility, contractor facility, home and/or other). Where it does not, and it is necessary to select environments for use (as when the TRS specifies an ADF training school, but not which one), a comparative analysis may be necessary to identify the best option1.

3.28 Having determined which environments are to be analysed, reference should be made to the Feasibility Report from the ‘Analyse’ phase. That should contain, or refer to, information about those environments. The currency and comprehensiveness of this information is checked to determine the need for gathering new information.

Inputs TRS, Feasibility Report, current policies and instructions, workplaces, training organisations and schools, and the staff of these, resource specifications.

Process Step 1: Determine information requirements.Step 2: Plan, prepare and implement information collection.Step 3: Collate and review information.Step 4: Develop and confirm environments profile.

Products A detailed environments profile that identifies the strengths and weaknesses of the environments for assessment and learning.

1 Detailed work in analysing learning and assessment environments can reinforce or challenge the content of the Feasibility Report and dependent TRS. If a problem is apparent, the related analyse processes might need revision.

3–7


3.29 The analysis requires information on the features of each of the environments that can influence learning and assessment. Although information requirements are situation specific, the following categories are commonly relevant:

a. Environment description. For reference purposes, a broad description of the workplace/school/college etc. Its management arrangements might be relevant and the contact details of authorities should be sought. Any support facilities needed (eg accommodation, medical or social) should be identified.

b. Location. The location in relation to the workplace, learners’ accommodation, associated training facilities, external training sites and transport facilities might be considered. The relative isolation of the proposed training environment or its proximity to a major city may also be significant for the learners and staff.

c. Access and availability. The availability of the facility now and in the future should be checked. Ideally, design and development determine the length of a course and the assessment processes (work in the ‘Design’ phase should assume this). However, the duration of training may be determined by the availability of training environments.

d. Facility security. Identify the level of security that the facility offers and/or the capacity to upgrade security. This is particularly important where the proposed training may have a security classification that restricts general access or where current facility activity and the proposed training activity have different security requirements.

e. Spaces for activity. Various aspects of spaces must be considered, dependent on the type of space (classroom, workplace, practical area). These may include size and suitability of area, power and water resources, computer network access, storage, floor type, washing facilities, heating, lighting, and positioning of teaching and learning aids.

f. Training resources. Use of existing training resources can reduce costs. The availability of equipment, tools, computers, the types and numbers of desks/benches projectors, printed materials, screens, photocopiers and other electrical equipment could be pertinent.

3–8


g. Occupational health and safety. Defence, through its leaders and managers, owes a ‘duty of care’ to ADF members and civilian staff. This translates to providing a safe workplace with all the necessary protective equipment and supervision in place.

h. Impact on the environment. The impact of new or changed training requirements on the proposed environment and the local population can affect the planned learning and assessment activities. This is particularly important to consider where the new training is likely to involve large numbers of personnel, extensive training activity or a significant change in local activity eg the introduction of field exercises or adventurous training.

i. Staff. The availability of suitable training staff for both administration and the conduct of instruction and assessment is important. The numbers available and/or the ability to attract them there are significant factors. Similarly, the motivation of the staff and any need to prepare them for new responsibilities must be considered. Both the present and anticipated situations should be considered.

j. Finances. The availability of funds for establishing and conducting training and assessment in the environments might be relevant (they can influence consideration of modifying the environment, and procuring equipment and other resources that might be used for training and assessment).

k. Policy. Policies at many levels can influence the design of courses and assessment processes. At a local school/establishment level they will influence decisions on routines, organisation and the allocation and use of resources. At a branch or Service level, policies on such matters as career structures, staffing levels affecting the ability to undertake on-the-job training (OJT), and civil accreditation can affect course design. In the work environment they can influence the allocation of resources (particularly time) for training and assessment purposes.

3–9


Step 2: Plan, prepare and implement information collection 3.30

3.30 To gain the required information suitable sources must be considered and the most appropriate selected. Much information is likely to already be documented within Defence, but may require research. To collect the remainder, interview and first-hand direct observations may be appropriate methods. Planning of such collection is required, not least to avoid a disproportionate use of time, and it may be necessary to produce collection instruments such as questionnaires and check sheets. The arrangement of visits, observations and interviews can be necessary, requiring consideration of:

a. the precise definition of what information is required from the analysis;

b. the geographic distribution of likely respondents or interviewees;

c. the ease of access to relevant personnel, particularly where operational or exercise activity is likely;

d. the availability of competent personnel to design survey instruments, conduct interviews or observations and analyse data;

e. the resources required for the collection, collation and analysis of data;

f. time available for the analysis; and

g. contingency plans if survey or interview schedules cannot be met.

3.31 Meeting the learning and assessment staff employed in these environments provides an opportunity for promoting the changes that will accompany the conduct of the new training.


3.32 Information about learning and assessment environments is normally single-sourced, and the task of collating it should therefore be straightforward. Some information may, however, prove contentious, and checks of it can be worthwhile. If any inconsistencies or gaps are discovered, further collection and collation will be needed.

3–10


Step 4: Develop and confirm environments profile 3.33

3.33 Each environments profile is a working document specific to a unique situation. There is no set format, although a template is suggested in annex H. All profiles should clearly identify and evaluate the factors that can affect learning and assessment in the nominated environments.

3.34 When developed, the environments profile should be checked against the TRS. As the document is a supporting reference, it does not require high-level authorisation. Its value is in the support it provides to designing assessment and training strategies.

ANALYSE PROSPECTIVE LEARNERS 3.35

Introduction 3.35

3.35 The ‘Design’ phase of the DTM marks a shift in the preparation of training from the definition of its requirement to the processes of its assessment and conduct. For the processes to be effective, they must be tailored to the individuals who will experience them. Understanding the characteristics and qualities of the prospective group of learners is a prerequisite for such tailoring. Analysing the learners develops this understanding, which is recorded in a learner profile.

3.36 An analysis of prospective learners builds-on similar work undertaken in the target population analysis process of the ‘Analyse’ phase. The scope and scale of a learner’s analysis will depend on the comprehensiveness of that earlier work, as reflected in both the TRS and the Target Population Profile.

3.37 Purpose. The purpose of analysing the prospective group of learners is to ensure that the selection of assessment and training strategies is appropriate to their characteristics and circumstances.

3.38 Triggers. The major trigger for analysing prospective learners is the design of new training. In this circumstance the process follows on from approval of a TRS. However, changes in organisation and/or staffing can upset the validity of an analysis. In such cases, the Learner Profile should be reviewed and, if necessary, updated, possibly necessitating corresponding amendments to dependent training documentation.

3.39 A learner analysis is one of three processes that may be undertaken concurrently at the beginning of the DTM’s ‘Design’ phase, the others being analysis of the learning and assessment environments, and the competencies that require training.

3–11



3.40 The inputs, process and products of the prospective learner analysis are summarised in table 3–2 and explained in the following paragraphs.

Table 3–2: Inputs, process and products of a learner analysis


3.41 The first step in analysing learners is to determine the information requirements. These requirements depend on the comprehensiveness and currency of the trainee profile contained in the related TRS2. If comprehensive and current, the requirement for gathering new information is minimal. If it is not, then additional information will be required.

3.42 Typical characteristics of learners that, because of their potential to affect the training and learning processes, should be recorded include:

a. Employment classification. Understanding the representative rank/level of the typical learner, the employment specialisation/category and any particular stream or skill is a foundation for work in the phase.

b. Proficiencies. Any relevant proficiencies ie qualifications, certificates, licences and employment experience related to the job or course.

Inputs TRS, Target Population Profile (including learners’ current competencies and experience levels), Department and workplace data about the learners (including age, sex, classification, location, access and equity requirements, language, literacy and numeracy requirements, relevant experience and education), current policies and instructions, data from Defence Recruiting, job sponsors and career managers, survey results and national literacy standards.

Process Step 1: Determine information requirements.Step 2: Plan, prepare and implement information collection.Step 3: Collate and review information.Step 4: Develop and confirm learner profile.

Products A detailed learner profile that describes the preferred learning style, special aptitudes and motivation of likely learners. This information helps ensure the suitability of the assessment and learning strategies.

2 This profile was derived from the target population analysis, where it should be accessible in full.

3–12


c. Personal characteristics. List any characteristics of composition by gender, age, culture and location. Gender can be important where a mix requires appropriate domestic facilities. The age span of the learner group can be relevant to learning styles and the capability to learn new concepts, (research has shown that the older a person the more difficult it is to learn skills dealing with numerical functions. Conversely, a mature person is generally more responsible and can more readily accept the onus for personal development/learning).

d. Competencies. List any competencies that are likely to be held by learners, related to the required job performance, but are not covered by any identified proficiency.

e. Academic ability. Where relevant, indicate level of academic ability, especially in English and Maths.

f. Physical characteristics. Is height, eyesight, hearing, dexterity, ability to discriminate colours or any other physical characteristic important enough to be defined as an entry standard for the course? For courses with a special fitness dimension, the normal and likely minimum fitness levels of the new learner should be identified.

g. Learning styles. Any favoured or problematical learning styles should be identified, such as experience with self-paced, discovery and computer-based learning.

h. Special aptitudes. Are there any particular mental or physical characteristics of the job or course that require particular aptitudes and which cannot be identified with factors in subparagraphs c. e. f. and g. above?

i. Motivation. How motivated are the learners likely to be, as a result of perceptions about the job or course? Are the ‘rewards’ for passing the course likely to motivate learners or does passing mean longer hours, a dirtier or dangerous job etc, without corresponding ‘gains’. Such factors will influence decisions on design strategies and tactics.

Step 2: Plan, prepare and implement information collection 3.43

3.43 For required information not available from ‘Analyse’ data/documents/records, suitable collection sources need to be considered and the most appropriate selected for use. The selection of the collection methodology should be determined by what sources, time, expertise and other resources are available and by the nature of the data to be collected.

3–13


Much of the required information is likely to already be documented by Defence in the form of career/trade/recruitment data, but may require further research. To collect any remainder, interview, survey or direct observation and measurement can be used. This work requires planning, not least to ensure that a disproportionate time is not spent on it. Factors to be considered in the planning stage are:

a. a clear understanding of what information is required from the analysis;

b. the geographic distribution of and ease of access to likely respondents or interviewees (particularly where operational or exercise activity is likely);

c. the availability of competent personnel to design survey instruments, conduct interviews and analyse data;

d. the resources required for the collection, collation and analysis of data;

e. the time available for the learners analysis; and

f. contingency plans if survey or interview schedules cannot be met.

3.44 The task of collecting information can be aided by use of instruments such as questionnaires and check sheets. These, as appropriate, should be developed and tested. Other possible preparation tasks include the arrangement of visits, observations and interviews.


3.45 As the information on the target population is likely to come from a number of sources it will require collation. This process could identify inconsistencies that need resolution and gaps that require more information to be gathered. This work enables the development of an accurate picture of the likely learners.

Step 4: Develop and confirm learner profile 3.46

3.46 There is no set structure for a learner profile. It should serve the situation of its requirement. Most, however, should identify the course and the classifications of typical learners; together with the qualifications and proficiencies they should have; and their levels of relevant language, literacy and numeracy. A suggested template for a learner profile is in annex I.

3–14


3.47 When developed, the learner profile should be checked against the TRS and by reference to personnel managers and workplace authorities for suitability and accuracy. As the document is a supporting reference, it does not require high-level authorisation.

ANALYSE COMPETENCIES 3.48

Introduction 3.48

3.48 The analysis of competencies identifies the skills (mental and physical), knowledge and attitudes/attributes that a person must acquire in order to perform each element of a competency. Most competencies comprise complex performances that are best trained by identifying their components and focusing on each one in a preferred sequence. By this means, components that require greater or special attention can receive it, and those that require little or no training can be treated accordingly.

3.49 The analysis of competencies is a preliminary process in developing the learning outcomes that serve as the objectives of training. As such, the products of the process, completed CAS are working documents in the design of training.

3.50 Purpose. The purpose of analysing competencies is to identify the basic skills (physical and mental) knowledge and attitudes required to achieve competencies. This knowledge is an important basis for determining the intended outcomes of training.

3.51 Trigger. The analysis of competencies is only required when training is initially designed or, subsequently, when a change is made to a competency. The primary trigger for undertaking the process is therefore authorisation of a TRS and the start of the ‘Design’ phase of a training development project.

3.52 A competency analysis is one of three processes that may be undertaken concurrently at the beginning of the DTM’s ‘Design’ phase, the others being analyses of the learner, and the learning and assessment environments.

3–15



3.53 The inputs, process and products of a competency analysis are summarised in table 3–3 and explained in the paragraphs that follow.

Table 3–3: Inputs, process and products of a competency analysis

Step 1: Analyse competencies to complete Competency Analysis Sheets 3.54

3.54 There is only one substantive step in analysing competencies, and it can be considered the first step in generating learning outcomes.

3.55 Identifying the knowledge, skills and attitudes of elements of competency requires the combined expertise of job holders experienced in the job and an analyst competent in the process. Analysis can be based on observation of the work and questioning of job holders, or by questioning alone. Ideally, more than one job holder will be involved in the process, as this helps ensure that all aspects of the work are considered. Where major equipment acquisition is involved or new capabilities are being implemented, LSA task lists and manufacturers’ manuals and equipment specifications may provide a useful information source.

3.56 The first step in the process is to transcribe the competencies and their elements from the TRS to CAS. A template for a CAS containing sample data is in annex J (this particular example focuses on one element).

3.57 For each element of competency in turn the steps required for its performance are listed and numbered. As this is done, consideration is given to whether the learners already have the ability to perform the work. If they can, it is not necessary to analyse the performance any further. If not, then the CAS columns for skills, knowledge, attitudes, training notes and difficulty are completed (note the following explanatory paragraphs).

3.58 Physical skills. Physical skills are the learned capability of performing actions in an organised and fluid manner. Their outcomes are reflected in the speed, accuracy, force or smoothness of bodily movement. These skills are learned in connection with such common human activities as driving a car,

Inputs Competencies and related information from the TRS, job-holders, logistic support analysis (LSA) task lists, manufacturers’ manuals and equipment specifications.

Process Step 1: Analyse competencies.Step 2: Check completed CAS.

Products CAS.

3–16


using a keyboard and climbing a hill. Many complex physical skills can be broken-down into smaller self-contained sub-skills that can be learned separately before being combined for whole performance.

3.59 Mental skills. Mental skills are the learned capability of reacting with the environment by using symbols. They are characterised by knowing how, as contrasted with the knowing what of knowledge (eg converting fractions to decimals, analysing intelligence). They enable adequate responses to entire classes of natural phenomena. Mental skills may be divided into several sub-categories that can be ordered hierarchically, with complex skills requiring the prior learning of simpler skills.

3.60 Knowledge. Knowledge involves the recall of facts or sets of events—being able to communicate information. It is seen as a necessary prerequisite of all other learning and therefore a critical constituent of the learning process. Increasingly, with the application of information technology, the mental skill of accessing required knowledge is more important than owning it.

3.61 Attitudes. An attitude is an acquired mental state that influences the choice of personal actions and is reflected in those actions. They are evident in tendencies to respond positively or negatively to another person, group of people, an object, idea or situation. Attitudes may be rational or irrational. They are related to motives, beliefs, values, personality, emotions and the groups of people.

3.62 Difficulty. When the skills knowledge and attitudes columns have been completed, judgments can be made on the difficulty of learning the performance described in the element of competency. Guidance in this can be gained from current job holders and learners. This information will indicate areas for which training might usefully incorporate practice or incremental learning activities.

3.63 Training notes. Although the work of competency analysis focuses on job performance, thoughts and ideas related to training are likely to arise. A column of the CAS is provided for noting these points—which are only suggestions, as competency analysis does not formally consider the subsequent training implications.

3.64 The information gathered during the competency analysis process is vital to the development of training, for it provides the content of, and a guide to the structure of training. Information on the complexity or difficulty of the task will indicate the need for practice or the need for incremental learning activities. Some aspects of work may indicate the need for more than one individual to be involved and therefore the requirement for group learning activities.

3–17


3.65 Potential learning outcomes are derived by analysing all the data on the completed CAS (ie all the data in paragraphs 3.58 to 3.63). This process, (sometimes referred to as task analysis) although not an exact science, involves identifying the major concepts. These major concepts in turn form the basis of the potential learning outcomes. In addition, the process involves eliminating skills, knowledge and/or attitudes that have already been taken account of, so that the final list of learning outcomes represents a consolidated list. Finally, other data on the CAS that is related to a larger concept is identified for inclusion as a potential assessment criteria or content.

Step 2: Check completed Competency Analysis Sheets 3.66

3.66 On completion of the CAS it is useful to have them checked by an independent subject matter expert (SME) and someone with training development expertise. Although a ‘working document’, it is an important one. When checked, the completed CAS is used for generating learning outcomes.

GENERATE LEARNING OUTCOMES 3.67

Introduction 3.67

3.67 Generating learning outcomes is the process of establishing the intended results of training and therefore the objectives of the training system. It involves translating workplace performances into performances for the training situation and optimally structuring them to enable learning. Learning outcomes therefore clearly and concisely describe the intended results of training in terms of what a learner will be able to do having successfully completed a unit of training, the conditions under which they will do it and to what standard. Each learning outcome is also associated with a prescribed method for assessing its accomplishment.

3.68 In order to facilitate the development of curriculum documentation the DTM specifies two types of learning outcomes. The use of the two levels is merely to make one aware that in most instances there will be a hierarchy of Los. The first, the course learning outcome (CLO) is expressed in broad terms to specify the intended result of the course or program. The second, module learning outcomes (MLO) specify the intended outcome of ‘chunks’ or components of learning. MLO are supported by Assessment Criteria, which indicate what the student must do and how well they are to perform. In combination, MLO and assessment criteria state the performance and standards, while the specific conditions under which the outcomes are to be performed are provided in an accompanying statement.

3–18


3.69 Learning outcomes are the building blocks of training and learning. They are a major factor in the training strategy developed in the ‘Design’ phase, the production of training and learning materials in the ‘Develop’ phase, and the goals of all activity in the ‘Conduct’ phase of the DTM.

3.70 Purpose. The purpose of generating learning outcomes is to specify the objectives for the work of trainers and learners.

3.71 Trigger. In developing new training, learning outcomes are generated following the analysis of competencies. The trigger for the work is, therefore, completion of the CAS recording that analysis. When reviewing and updating existing training, the records from the original design work should be accessed, checked and amended.


3.72 The inputs, process and products of generating learning outcomes are summarised in table 3–4 and explained in the paragraphs that follow.

Table 3–4: Inputs, process and products of learning outcomes

Step 1: Collect inputs 3.73

3.73 The first step in generating learning outcomes is to obtain the required documentation (CAS, any regulatory or legislated requirements, and the profiles of learners and environments) together with any existing training documentation that might be used to save development time (Defence, other military, Government or industry).

3.74 While use of relevant existing training documentation can save development time, care must be taken to ensure that any material used either meets or is customised to meet the specified need.

Inputs CAS, existing industry and Defence training/learning materials, any regulatory or legislated requirements, and the learner and learning and assessment environment profiles.

Process Step 1: Collect inputs.Step 2: Determine required performances.Step 3: Complete learning outcomes and assign CLO Levels.Step 4: Write assessment criteria for each of the MLO.Step 5: Check learning outcomes and file.

Products Agreed learning outcomes with assessment method.

3–19


Step 2: Determine required performances 3.75

3.75 In generating learning outcomes the primary focus is on the performance elements (noting that the variables of performance frequently have a controlling influence and should also be considered). The competencies and their elements listed in TRS approximate to the two types of learning outcome performance (CLO and MLO), while the skills, knowledge and attitudes identified in competency analysis provide input to such documents as lesson plans (which do not form part of the formal curriculum documentation). The indicative relationships between the components of jobs, competencies, learning outcomes and units of training are illustrated in figure 3–2.

Figure 3–2: Indicative relationships between jobs, competencies, learning outcomes, and training

3.76 In writing the performances of learning outcomes consideration is given to how they might be structured to:

a. meet the required workplace performance;

b. efficiently use particular training methods and learning environments; and

c. form a course/program of training that is appropriate to the learners.

3.77 Some workplace performances can be replicated in the prescribed training environment, in which case they can be used as learning outcomes. Others cannot, for safety or cost reasons, in which case the performances, conditions, and standards of the learning outcomes are written to be as valid and appropriate as practicable.

3–20


3.78 Each performance should describe a single, observable behaviour that can also be measured. Its key term is an action verb. The description should be kept as short as possible. Qualifiers, explanations and expansions, when necessary, should be included in the conditions area of the outcome.

3.79 In this process thought is given to the sequencing of the outcomes. Having identified the required performances for all the competencies listed in the TRS, the list is rationalised to:

a. remove any that the learner profile identifies as being existing learner attributes;

b. avoid duplication and minimise overlap; and

c. group any that will be more easily learned by association (groups may be based on related performances or training methods).

3.80 Sequencing learning performances. The sequencing of training is important for effectively and efficiently achieving the desired outcomes. A number of factors should be considered:

a. dependent outcomes (identifying prerequisites and/or co-requisites so that dependent learning outcomes are sequenced accordingly);

b. learning principles that suggest sequencing from:

(1) simple to complex;

(2) whole-part-whole;

(3) concrete to abstract; and

(4) theory before practical;

c. modularisation (ensuring that all necessary assessment criteria are contained within a module);

d. training and qualifications frameworks (enabling entry and exit points from courses/programs in accordance with learner’s needs); and

e. personnel, resource and administrative limitations.

3–21


3.81 These factors are considered in relation to the following alternative bases of sequencing training:

a. job performance: the training sequence is the same as the job sequence;

b. critical sequence: the sequence is determined by the priority of the training;

c. spiral: similar outcomes are ‘revisited’ with an increasing degree of complexity, or to illustrate different perspectives;

d. chronological: the sequence follows the order in which events occur in time;

e. simple to complex: outcomes are presented in stages of increasing difficulty or complexity; or

f. mixture: a mix of the preceding bases is normally used in designing training.

3.82 Once the list has been rationalised and sequenced, a numbering system is applied to enable tracking of the learning outcomes.

Step 3: Complete learning outcomes and assign course learning outcome levels 3.83

3.83 Having developed the performance element of the learning outcomes, the related conditions and standards can be added, and CLO levels can be assigned.

3.84 Conditions. To ensure that training consistently meets the workplace requirement, the conditions and limitations under which the learner carries out the performance must be defined. There are six types of conditions that should usually be considered:

a. resources (including facilities, tools and equipment);

b. special job aids and manuals (eg fault finding algorithms, cue cards, procedural check lists);

c. environmental conditions (extremes of temperature, humidity, noise, motion etc);

d. simulated situations (by use of simulators, role-play, case studies and exercises);

3–22


e. physical location (where it influences performance, like when a marine engine is repaired in a workshop or in a ship’s engine room); and

f. associated personnel (eg supervisors, other operators and members of team/group).

3.85 When the performance is based on a nationally recognised or previously documented unit of competency these conditions are derived from the associated range of variables and checked using the job specification. To promote the transfer of learning to the workplace the conditions in the learning outcomes should match those of the job as closely as possible.

3.86 Standard. Assessment criteria list the standards which a participant will be judged against, to determine whether or not they have achieved a learning outcome. The standards of learning outcomes are the benchmark against which satisfactory performance is assessed. They should match the standards expected in the job situation. The standards are, therefore, derived from the performance criteria of the TRS. Standards can be set from the areas of quality, quantity, consistency and time; and any performance may have one or more of such criteria. In writing learning outcomes it is important to retain a focus on what the learner will be doing in the workplace and not be distracted by what might be easy to achieve in the training environment.

3.87 Assign course learning outcome levels. The purpose of assigning a level to CLOs is to provide a shorthand/easy method of indicating the meaning of CLOs to stakeholders, particularly the employers of recent training graduates. Four levels are used, as illustrated in table 3–5.

Table 3–5: Course learning outcome levels

Level Description of graduate in relation to prescribed

standard

Meaning

1 Trained, but not against a prescribed standard.

Background or familiarisation training.

2 Partially trained against prescribed standard.

Additional training and/or experience required before assessment. Employment under supervision only.

3 Fully trained and ready for assessment against prescribed standard.

Assessment required. Employment under supervision until assessed.

4 Trained and assessed competent against the prescribed standard.

Ready for employment.

3–23


3.88 The product of writing learning outcomes is a numbered list of outcomes of two types: CLOs and MLO with the first of these described in terms of levels. This list contributes to the development of a training strategy and is documented in the training curriculum. It is, therefore, a working document. A suggested template is in annex K.

Step 4: Write outcome assessment methods 3.89

3.89 A statement of how the performance is to be assessed accompanies each learning outcome. Various methods3 can be used and include assessment of:

a. demonstration of workplace performance (normal or on specified exercise tasks);

b. output sample—from test/exercise or workplace;

c. performance in training exercise (individual or team);

d. simulation (individual or team, and including role-plays and ‘hypotheticals’);

e. oral presentation;

f. interview;

g. written test/exercise (open or closed book); and

h. third-party evidence.

3.90 Selecting assessment methods involves evaluating the validity, suitability/practicality, and reliability of options. Validity is the extent to which the assessment measures what is intended. To be suitable/practicable, assessment should be administratively feasible and cost-effective. Reliability refers to the consistency with which it measures what it is supposed to measure.

3.91 If, in the course of writing assessment methods, it seems that a listed performance cannot be assessed, then it is probably necessary to amend the performance.

3 Many of the methods can be implemented using information technology, in which case they might be referred to as computer-based testing.

3–24


Step 5: Check learning outcomes and file 3.92

3.92 To prevent nugatory work it is important to check the completed learning outcomes against the relevant competency and the workplace to confirm that all performances and performance criteria have been adequately and properly addressed.

3.93 When the learning outcomes have been checked they are to be retained, together with documents showing how they were generated, on the departmental project/program/course file for audit and continuous improvement purposes, and to maintain a history of course development.

DESIGN ASSESSMENT STRATEGIES 3.94

Introduction 3.94

3.94 Assessment is the process of collecting evidence and making judgments about something. When applied to training, assessment can diagnose learners’ abilities (ie for placement, selection, and identification of strengths and weaknesses), motivate and guide their progress (formative assessment), and determine mastery of something learnt (summative assessment). It can be applied before, during, on completion of training and for units of competence, following other developmental experience.

3.95 In the DTM an assessment strategy is a broad plan for using assessment to support learning, and to determine whether intended learning outcomes have been achieved4. A comprehensive strategy should state the purpose and approach to be used for assessment, present detail on the methods and sequence to be used, and indicate the likely resource requirements. This information is used to guide the development and conduct of assessment in a training course/program.

3.96 The design and implementation of an appropriate assessment strategy is a standard required of RTO5.

3.97 To retain a focus on training outcomes, the design of an assessment strategy normally precedes the design of a training strategy. It is important, however, that the two strategies complement each other (the timing and methods of assessments can influence the design of the training strategy

4 The ultimate outcome for the DTM is workplace performance and learning experiences other than training that contribute to it. The assessment of workplace performance is more properly the responsibility of the job/BPO than trainers, but the latter are usually better qualified to design the processes and prepare the materials for the task.

5 Currently described by Standard 9 of the Australian Quality Training Framework (AQTF).

3–25


3–26

and, in some circumstances, training methods can influence the design of the assessment strategy). For this reason the two strategies are presented together in a training and assessment strategy document.

3.98 Purposes. The two purposes of designing an assessment strategy are to guide the use of assessment for supporting and measuring learning in a training course/program, and to determine the achievement of workplace competence.

3.99 Trigger. An assessment strategy is designed for any new training when it is known what is to be trained, who is going to receive it and in what broad circumstances. The primary source of this information is the TRS, but the necessary depth of information is contained in the list of Learning Outcomes, and the learner and environment profiles. Checks and revisions of a completed training and assessment strategy document are made whenever TR&E processes indicate the inputs have changed.


3.100 The inputs, process and products of the process for designing an assessment strategy are summarised in table 3–6.

Table 3–6: Inputs, process and products for designing an assessment strategy

3.101 Although the following steps are presented as discrete tasks, they usually interrelate and overlap not only with each other, but with preceding and subsequent steps. In consequence the process is highly iterative.

3.102 National assessment principles. A background understanding of the National Assessment Principles and their essential features as shown in annex C, should underpin the design of all Defence training assessment. The National principles, have been identified by the Department of Education, Science and Technology (DEST), in the context of training packages. Where some are criteria checked for registration as an RTO. However, they are applicable to any assessment of human performance against a required standard, and should therefore be considered and incorporated in Defence assessment strategies and tools.

Inputs Assessment specifications listed in the TRS, learner and environments profiles, learning outcomes, and any relevant existing assessment resources.

Process Step 1: Determine assessment needs and draft the strategy.Step 2: Select and add assessment methods.Step 3: Refine the assessment strategy.

Products Input to a training and assessment strategy document.


3.103 Occupational health and safety. A background understanding of occupational health and safety (OHS) issues should also underpin consideration strategies and methods for assessment. Defence is bound by the Occupational Health and Safety (Commonwealth Employment) Act 1991. Although there is a special provision for the ADF that recognises the potentially dangerous nature of the work which a national defence emergency may make acceptable or necessary, it does not allow for exposing members to dangerous situations whilst training. Where doubt exists, professional assistance should be sought through the OHS branch.

Step 1: Determine assessment needs and draft the strategy 3.104

3.104 To determine what assessment a training course/program should have, the types and functions of assessment are considered in relation to the course as represented by its learning outcomes. To retain a strategic focus, this work is limited to the higher two levels of outcome, the CLO and MLO6. The work requires judgment, and must take into consideration the characteristics of the learners, the environments available, and the resources that can be used.

3.105 Primary types and functions of assessment. Different forms of assessment serve a range of functions in support of training. They include the following:

a. Recognition of prior learning/recognition of current competencies/recognition is a type of assessment normally offered before the conduct of training. It is used to grant status or credit for related competency already held, thereby avoiding overtraining.

b. Diagnostic pre-tests are normally administered before the conduct of training for identifying specific learning needs.

c. The most common, and frequently the most important, function of training assessment is to determine progress towards, and achievement of intended learning outcomes. This formative assessment is used during a course to provide feedback on learner progress.

d. At the end of a course and/or other learning experience summative assessment is used to check overall achievement of competence. Defence training is directed at the summative assessment of workplace competency, which may be conducted on- or off-the-job.

6 Assessment at the level of assessment criteria learning outcomes is considered during the ‘Development’ phase of the DTM.

3–27


e. Holistic assessment. Holistic assessment is a particular form of assessment which determines in an integrated manner the way a range of competencies are combined to successfully complete practical workplace tasks. It is primarily used in workplace contexts when the specified learning outcome requires the demonstrated integration of a number of separate learning outcomes or competencies. It is characterised by:

(1) whole-of-work observation of workplace activity;

(2) synthesis of multiple sources of evidence in making a judgment;

(3) opportunity to gather evidence on more than one unit of competency at the same time; and

(4) close adherence to the National Assessment Principles.

3.106 The longer, more complex and difficult training is, the greater the need for both formative and summative assessments.

3.107 Other functions of assessment. Other functions of assessment that might be incorporated in an assessment strategy include:

a. the pre-course diagnosis of learners’ competencies for placement (as when selecting individuals with mathematical abilities for technical courses);

b. confirmation and/or reinforcement of course prerequisite learning;

c. identification of learners’ preferred learning styles7 (individual, group, self-paced, text, pictorial, theory or practical etc); and

d. the post-course job placement of course graduates.

3.108 The implementation of all assessment processes also serves to motivate or demotivate learners. These effects, particularly the negative effects, can be powerful. The design of an assessment strategy should therefore attempt to maximise motivational effects and minimise the negative effects.

3.109 Draft the assessment strategy. In drafting the assessment strategy the needs identified are applied to the list of learning outcomes to determine when assessments will be conducted and for what purpose. The summative

7 This can be of value in situations where choices of learning methods are incorporated in training conduct.

3–28


assessments of intended learning outcomes are determined first, thereby retaining an outcome focus. Work on formative assessment processes and assessment for other purposes then follows.

3.110 Where existing training documentation with proven assessment strategies and materials is of relevance to the training need, use of this material is encouraged, as it promotes efficiencies and facilitates standardisation of assessment practices. However, care should be exercised to ensure that adapted material is appropriate for the purpose and that any modification of the existing material does not adversely affect its validity.

3.111 The outcome of the step is a draft plan that combines the learning outcomes of a training course/program with a broad plan for the conduct of assessments.

Step 2: Select assessment methods 3.112

3.112 Once the basic assessment strategy is determined, the method(s) for conducting each of the assessments identified is selected. This decision usually also involves determining where the assessment should take place. Again, this work builds on, and can be informed by guidance provided in the TRS and thoughts recorded whilst generating learning outcomes.

3.113 Methods of assessment. The range of assessment methods that might be employed in a course or program of training includes:

a. demonstration of workplace performance (normal or on specified exercise tasks);

b. output sample—from test/exercise or workplace;

c. performance in training exercise (individual or team);

d. simulation (individual or team, and including role-plays and hypotheticals);

e. oral presentation;

f. interview;

g. written test/exercise (open or closed book); and

h. third-party evidence.

3–29


3.114 Selecting methods. The principal consideration in selecting the optimum assessment method, is the learning outcome. The verb chosen for the performance of the outcome indicates suitable methods of assessment8. Other considerations include the training methods employed; the location and suitability of the training and alternative venues; the costs involved; and the qualities of validity, reliability, flexibility and fairness.

3.115 Check. Before expending resources in refining the assessment strategy it can be worthwhile to check that:

a. all the CLO and MLO of the training course/program, (particularly any CLO that equate to units of competency), have an associated assessment process;

b. all the performance criteria and requirements for competencies listed in the TRS are appropriate to the workplace, training environments and learners; and

c. relevant requirements from the AQTF Standards for RTO have been addressed.

Step 3: Refine the assessment strategy 3.116

3.116 As the assessment strategy is refined, more detailed issues are addressed and determined. These are likely to include:

a. the location of the assessments (on- or off-the-job);

b. the resources the assessments will require (including qualified assessors);

c. their recording/administration (including log progress, Defence Personnel Key Solutions (PMKeyS) data entry, archiving etc);

d. the management of failure to meet standards; and

e. the processing of appeals against assessment decisions.

3.117 Administration of assessments and results. Administrative procedures for the conduct of assessment should be practicable, as should the management of the results obtained. This is likely to be straightforward where the normal practices of the organisation are followed, but can be problematical where new systems are introduced. In Defence, consideration

8 The term ‘describe’, for example, indicates the appropriateness of written or oral methods, whether off or on-the-job; and an objective of ‘tune a diesel engine’ suggests assessment by practical methods in a workshop setting.

3–30


should be given to the practical difficulties of workplace assessment for ADF units on operational service, and to the existence of PMKeyS for programming and panelling courses, and for managing course results.

3.118 Training progress management. In situations requiring it, assessment strategies are to include provisions for managing instances where standards are not met. Provisions might include control points restricting progress; and the resources, including time, for retesting. The requirement will derive from personnel management policies and its implementation will be governed by the training progress management policy of the Service or organisation conducting the training.

3.119 Programming of assessments. The programming of assessments is another task requiring professional judgment. The primary determinant of the sequence and timing of assessment activities is the training strategy, but the following factors should also be considered:

a. the value of assessment in reinforcing learning;

b. the motivating/demotivating influence of assessment;

c. the risks of pursuing training processes without confirmation of learning and achievement;

d. the implications of learners failing to achieve the required performance levels; and

e. the size/length and cost of assessment process in relation to the outcomes and training processes.

3.120 Estimate resource requirements. Each of the judgments and decisions of earlier steps in designing an assessment strategy have involved consideration of the resource implications. Where appropriate, the strategy should summarise these implications, particularly those involving significant: time for administration and conduct, facilities, equipment, personnel, funds, and organisation. If the estimates exceed the expectations of the organisation, appropriate action must be taken.

3.121 National principles of assessment. Final refinements of the assessment strategy may be achieved by ensuring that its content accords with DEST‘s National Principles of Assessment (annex C). Of these, principle 8 is of significance to all training and assessment situations.

Product—input to a training and assessment strategy document 3.122

3.122 The product of designing an assessment strategy is a broad plan that is merged with a training strategy and recorded in the training and assessment strategy document, an example template is in annex K. This plan

3–31


is likely to require amendment, first as it is combined with the training strategy and later as work on developing materials to implement the strategies suggests the need for change.

DESIGN TRAINING STRATEGY AND PRODUCE THE TRAINING AND ASSESSMENT STRATEGY DOCUMENT 3.123

Introduction 3.123

3.123 Training describes the development of skills, knowledge, attitudes, competencies etc through instruction or practice. In the DTM a training strategy is a broad methodology for the conduct and management of training. A training strategy should therefore provide guidance on the why, what, where, when and how of training. It should state the purpose, methods, location, activities, media, recommended or prescribed tasks, formative assessment requirements, sequence and likely resource requirements that training will involve9.

3.124 While focused on the required learning outcome, a training strategy must complement its associated assessment strategy. To help ensure this, the two strategies are presented together in a training and assessment strategy document.

3.125 Purposes. The purpose of designing a strategy for training is to ensure that training is conducted effectively and efficiently, consistent with relevant policies and learning theories. The purpose of producing a training and assessment strategy document is to formally record the guidelines for developing and conducting training.

3.126 Triggers. When preparing new training, the trigger for designing a strategy is knowledge of what is to be trained, who is going to receive it, and in what broad circumstances. The primary source of this information is the TRS, but the necessary detail is contained in the list of learning outcomes, and the learner and environment profiles. Checks and revisions of a completed training and assessment strategy are triggered by feedback from TR&E processes.

9 While this chapter refers to a single training strategy, some situations may require multiple layered or sequential strategies (eg linked strategies for introducing and then maintaining a new capability, or short, medium and/or long-term strategies that meet the requirements of a changing environment).

3–32


Inputs process and products 3.127

3.127 The inputs, process and products of designing a training strategy, and producing the training and assessment strategy document are summarised in table 3–7. The first three steps listed in the table are indistinguishably combined in practise. They are separated here for purposes of explanation.10

Table 3–7: Inputs, process and products for designing a training strategy

3.128 As is common in the DTM, the steps of designing the training strategy take the products of earlier phases and processes and develop them to a greater level of detail. The greater understanding this achieves often leads to revisions of the earlier work.

Step 1: Select training delivery methods 3.129

3.129 The first step of designing a training strategy involves the selection of a training method for each of the MLO previously generated11. A training delivery method comprises a mode, activity and setting. Selecting the training delivery method is based on consideration of the practical considerations, the estimated costs and times of development, and the potential effectiveness of alternatives in these fields.

3.130 Training modes. Training can be trainer-centred, learner-centred or a mix of the two. These training modes can be viewed as opposite ends of a continuum where the mix of methods, media and learning theory can be used

Inputs10 Training and assessment specifications listed in the TRS, learner and environments profiles, learning outcomes, assessment strategy, and relevant existing assessment and training resources.

Process Step 1: Select training delivery methods.Step 2: Apply other principles of learning theory.Step 3: Select associated media.Step 4: Combine training and assessment strategies.Step 5: Develop course map.Step 6: Complete training and assessment strategy.Step 7: Gain approval and file in accordance with policy.

Products Training and assessment strategy.

10 A supplementary resource for this DTM process is the DEST, 2003, Learning and Assessment Strategies Part 1:‘How to Kit’, and Learning and Assessment Strategies Part 2: Resource Guide that supports Standard 9 of the AQTF Standards for RTO.

11 Training methods for course learning outcomes were effectively set during the feasibility analysis of the ‘Analyse’ phase.

3–33


to achieve the most effective blend of trainer-centred and learner-centred modes. A predominantly trainer-centred mode has been the most commonly used by organisations, as it enables groups of trainees to be trained simultaneously. Catering for diverse trainees is, though, difficult, and delivery costs can be high. Learner-centred training has a growing potential to be the more efficient mode, as the rapid development of computer and network technology is enabling learning content, times and pace to be designed and developed for individual needs. Development of good materials is, however, difficult and costly, and some trainees respond poorly. A table showing factors to consider when selecting a training mode is in annex D.

3.131 Training activities. The selection of the most appropriate training activity will depend on the setting, the mode, what is to be learnt, its complexity, learner abilities and commitment, the position of the activity in relation to the course, the class size and the course/module/session duration.

3.132 A training strategy can employ any of a large variety of training activities, most of which can involve individual or group work, and many of which can be conducted either face-to-face or by distance learning mode. The more common activities include:

a. lecture;

b. demonstration;

c. lesson (theory and practical);

d. on-the-job work experience;

e. discussion;

f. independent study;

g. exercise;

h. tutorial;

i. simulation;

j. role-play;

k. case study;

l. mentoring;

m. games;

n. playlet;

3–34


o. sport;

p. adventure training;

q. seminar;

r. project;

s. debate; and

t. visit.

3.133 A table mapping relevant selection criteria against appropriate training activities is in annex E.

3.134 Training settings. Training may be delivered on-the-job or off-the-job, in residential or day schools or, for distance learning, in various settings. Any one or mix of these may be used for a course/program. Where the learning outcomes depend on a workplace context eg operating large, expensive equipment, then OJT will be the obvious choice. For other learning outcomes where the workplace environment is not so critical, then off the job training may be more cost-effective and less disruptive to the workplace. Learner choice of setting, as occurs in distance learning, is appropriate for learner-centred training where group interaction and time are relatively insignificant.

3.135 Practical considerations. In addition to the advantages and disadvantages of possible training modes, activities and settings, the selection of training methods must consider practical factors such as:

a. the appropriateness of the method to the learners; their numbers, location, interests, abilities and level of development as recorded in the learner profile;

b. the media requirements of the selected method;

c. the availability of environments, facilities, instructors and physical resources (equipment, tools, job and instructional aids); and

d. the time and cost of developing and implementing training methods12.

12 If significant costs or time are involved, then the need should be identified and managed early to ensure the required allocations.

3–35


3.136 Costs and times of development. The cost and time of acquiring and developing the resources necessary to implement a mode of training are difficult to accurately detail. Estimates are necessary though, for informing selection processes and for budgeting training. The feasibility analysis process in the ‘Analyse’ phase provided broad estimates. An additional level of detail, and therefore accuracy is provided in the ‘Design’ phase. That detail should consider:

a. initial acquisition or development costs and times of facilities, materials and equipment (annex F contains guidance for estimating the costs of and time required for developing new training materials);

b. delivery costs, including trainee salaries and support costs;

c. staffing costs, including acquiring, training and supporting staff;

d. ongoing operating and maintaining costs over the anticipated life of the training, including the costs of any service agreements;

e. depreciation/replacement costs;

f. administrative costs; and

g. any anticipated revenue, where this is applicable.

Step 2: Apply other principles of learning theory 3.137

3.137 Either following, or concurrent with selecting training methods related principles of learning theory should be applied. Some such principles were applied when designing the assessment strategy. Some are covered in step 5, generate learning outcomes, dealing with sequencing learning. Other principles that should be considered include:

a. opportunities for practice and feedback (learning is enhanced when performance is promptly followed by formative assessment, and personal and valid feedback);

b. variety in methods (variety in methods can be stimulating, promoting interest and learning efficiency; too much or too little variety can be counterproductive); and

c. activities promoting retention and transfer (relating to previous learning; repetition and practice, particularly in differing contexts, promotes learning retention; training methods that replicate workplace activity).

3–36


Step 3: Select associated media 3.138

3.138 The term ‘media’ refers to all training/learning aids that assist the learning process. Because some media are more appropriate to some methods than others, the two fields are often considered concurrently. However, for any delivery method, one or more media might be appropriately employed.

3.139 The choice of training/learning media includes:

a. instructor, supervisor, mentor;

b. SMEs (individuals and communities of practice);

c. computers (stand-alone or networked);

d. job aids (includes multipurpose devices);

e. real objects (including smart objects);

f. simulators and emulators;

g. boards (white, chalk, magnetic etc);

h. visits and field trips;

i. books (test, reference, programmed learning);

j. wall charts;

k. video and audio (compact disc or tape);

l. models; and

m. television (broadcast or closed circuit).

3.140 Each of these media has advantages and disadvantages, some of which are liable to change consequent to rapid technological development. In consequence selecting the media for a training method requires current knowledge. Media selection models may provide such knowledge and also assist with decision making. If used, their currency and basis for outputs should be understood.

3.141 Factors affecting media selection. Several factors need to be weighed when choosing which media should support a learning method. They include:

a. the learning outcomes (the media should enable the performance of the activity described in the learning outcome);

3–37


b. the characteristics of the learners (including experience, motivation, levels of English and computer literacy, numeracy, numbers, age and maturity);

c. the characteristics of the instructors, (can and will the instructor use the media well?);

d. constraints (eg unavailability due to cost; insufficient acquisition, development or course time; insufficient support staff; course schedule; class/group size; and rank/ability/gender of learners);

e. time required to develop/prepare;

f. availability of the selected media (eg simulators);

g. supportability of new media through life (eg maintenance);

h. life expectancy of the training and/or the media technology; and

i. possible spin-offs into other training areas.

3.142 Decisions on media are often a compromise between the best solution to a learning method and resource management consideration. Once made, those decisions are liable to be changed as new understanding is gained, not infrequently as a result of work in the ‘Develop’ phase of the DTM.

Step 4: Combine training and assessment strategies 3.143

3.143 It is important that the strategies for training and assessment be combined in a way that:

a. enhances learning,

b. promotes efficiency, and

c. enables continuous improvement.

3.144 If both strategies are concordant with the outcomes specified in the TRS, and the principles of learning theory described in preceding paragraphs are applied, the amalgamation should be straightforward and achieve these aims.

3.145 On completion of the merger it is timely to review and, if necessary revise the work to ensure it is valid (covering all that is required), suitable (covering it in the most effective and efficient ways), and administratively feasible within identified constraints. It can also be appropriate to review the

3–38


indicative time needed for the training to be completed. Course/program lengths cannot be ascertained until completion of the ‘Develop’ phase, but a good estimate at this stage of design can be useful for managing the pilot course process and for planning the subsequent conduct of training.

Step 5: Develop course map 3.146

3.146 When merged, the training and assessment strategies are likely to have a sequence arising from the generation of learning outcomes. In most situations alternative pathways for completing the course/program may be used. Alternatives that might be followed without detriment to training may be presented in a course map. A course map enables flexibility of scheduling and presents options for the day-to-day programming of training.

3.147 A course map may be produced either manually, or through use of a software tool (eg Syllabus + and Microsoft Project). The course map is derived by discovering the dependencies between learning outcomes/modules, taking into account prerequisites and co-requisites. Whatever means employed the resultant map should be easy to produce and use. A model course map with associated explanations is provided in annex G.

Step 6: Complete training and assessment strategy 3.148

3.148 To complete the training and assessment strategy document the material developed in the previous steps is prepared for incorporation in the draft curriculum and as a resource for designing strategies for TR&E. Although effectively a working document, the training and assessment strategy can be transferred directly into the draft curriculum. A template for a training and assessment strategy is in annex L.

3.149 Training and assessment strategy documents must meet appropriate standards. They are13:

a. audience focus:

(1) appropriate to industry and training context,

(2) consistent with the range of needs of potential users, and

(3) relevant materials ensured for different modes and pathways.

13 Training Package Development Handbook, DEST, pages 13–22.

3–39


b. sufficiently clear, relevant content for the intended audience:

(1) clear links to, and consistency with the competency outcomes of qualifications; and

(2) support the development of applied knowledge and skills, including workplace communication, language, literacy and numeracy.

c. written in an attractive style that is useable by the intended audience:

(1) written in plain English, avoiding training jargon; and

(2) having a user-friendly format and presentation.

d. encourage flexibility in training and assessment:

(1) provide for maximum flexibility in training and assessment delivery,

(2) facilitate gaining qualifications through a range of pathways, and

(3) promote integrated assessment.

e. are cost-effective to produce and maintain:

(1) incorporate generic and cross-industry material where appropriate to support portability with and across industry sectors; and

(2) avoid replication of existing resources where these meet training package requirements.

Step 7: Gain approval 3.150

3.150 A training and assessment strategy is a major input to the curriculum. Because there may be significant resource implications associated with implementing the strategies, their approval is critical, especially where only broad cost estimates have been identified in a feasibility report. Consequently, gaining approval of the document from the Design Authority is necessary before proceeding to development of the curriculum.

3–40


COMPILE CURRICULUM 3.151

Introduction 3.151

3.151 A curriculum presents the specifications for a training program, course or subject (module). The curriculum describes the learning experiences a learner undergoes during training, the intended learning outcomes, the strategies used for training and assessment, the content covered, the training methods and prescribed or recommended assessment tasks. A curriculum is the major component of a TMP—the complete set of documentation necessary for managing a training course or program.

3.152 The contents of a curriculum are the products of preceding processes of the DTM.

3.153 Purpose. The purpose of compiling a curriculum is to produce a training document, which includes the specifications, strategy, content and resources required to conduct a training course/program. This document is the major specification for the ‘Develop’ phase of the DTM.

3.154 Trigger. A curriculum for a training course/program is compiled following design of a training and assessment strategy.


3.155 The inputs, process and products for compiling curriculum are summarised in table 3–8.

Table 3–8: Inputs, process and products for compiling curriculum

Step 1: Compile curriculum 3.156

3.156 A curriculum builds-on the training and assessment strategy document by adding the content of the training program/course. The required training content is contained in the CAS that were the product of previous processes in the ‘Design’ phase. A training management package template for a curriculum is in annex M.

Inputs TRS, training and assessment strategy, CAS.Process Step 1: Compile curriculum.

Step 2: Confirm curriculum.Products Curriculum.

3–41


Step 2: Confirm curriculum 3.157

3.157 As a component of a TMP, the curriculum is approved by the Design Authority, at the end of the ‘Design’ phase. In preparation for that, training staff associated with the work (including those involved with the ‘Design’ and ‘Conduct’ phases of the training) should be invited to review its contents and confirm their suitability and practicability.

DESIGN TRAINING REVIEW AND EVALUATION STRATEGIES 3.158

Introduction 3.158

3.158 The effectiveness and efficiency of a training program/course are maintained by applying continuous improvement processes. Two forms of the processes are planned in the ‘Design’ phase:

a. Training review (internal evaluation). The evaluation and reporting of the ‘Conduct’ phase of the DTM; and

b. Evaluation. The evaluation and reporting of how well training has contributed to meeting the original requirement.

3.159 The two processes are largely independent of each other, but both relate to the structure and content of the course, as illustrated in figure 3–3.

Figure 3–3: Training review, evaluation and the training process

3.160 Strategies for the TR&E of a course/program provide broad guidance on the what, where, when and how of the processes. They cover the methods to employ, when to employ them and the resources required. This information, contained in a TMP, is used in the ‘Develop’ phase for producing the materials to be used and in the ‘Conduct’ phase for ensuring the processes are properly conducted.

3.161 Purpose. The purpose of designing TR&E strategies is to ensure that effective and efficient continuous improvement processes are applied to training.

3–42


3.162 Trigger. As the TR&E strategies are applied to the structure and content of a course, they are designed following completion of its training and assessment strategy.


3.163 The inputs, process and products for designing TR&E strategies are summarised in table 3–9.

Table 3–9: Inputs, process and products for designing training review and evaluation strategies

3.164 Defence uses the concepts of Donald Kirkpatrick’s model for evaluating training programs in its TR&E strategies14, it being a standard reference. The levels of assessment referred to in this chapter are those identified in the model. They are illustrated in figure 3–4 and explained in subsequent paragraphs.

Figure 3–4: Kirkpatrick’s model for evaluating training programs‘The Four Levels’

3.165 The four levels identified by Kirkpatrick represent a sequence of ways to evaluate training. Each level is important, but the processes of evaluation become more difficult and time consuming during the progress from level one to level four. The information gained from higher levels is, however, more valuable. The four levels identified by Kirkpatrick are explained below.

Inputs Training and assessment strategy.Process Step 1: Determine need for TR&E.

Step 2: Determine information requirements.Step 3: Select information sources and collection methods.Step 4: Record the TR&E strategies.

Products TR&E Strategies document.

14 Kirkpatrick D. L., 1994, Evaluating Training Programs—The Four Levels, Berret-Koehler Publishers, San Francisco.

3–43


3.166 Level 1: Reaction. As the word implies, evaluation on this level measures how the trainees ‘react’ to a training program/course. These reactions are usually recorded using end-of-course questionnaires, or by individual or group debriefs. The evaluation focuses on the ‘efficiency’ of training as judged by the trainees and training staff, as its effectiveness cannot be realistically measured until the trainee has spent some time in the workplace, applying the skills, knowledge and attitude they have learned.

3.167 Level 2: Learning. Kirkpatrick defines learning as ‘the extent to which participants/trainees change attitudes, improve knowledge, and/or increase skill as a result of attending the program’. These outcomes are measured by assessments conducted on completion of a training program/course/module.

3.168 Level 3: Behaviour. To be effective, learning resulting from training needs to be applied in the performance of workplace tasks. Kirkpatrick’s third level addresses the extent of this transfer. In the DTM the level is an important focus of the ‘Evaluation’ phase, with information gained being of importance to trainers.

3.169 Level 4: Results. Results are the effects of training on the organisation. Desired results can include enhanced capability, improved quality, decreased costs, increased sales, and reduced frequency and/or severity of accidents. There may also be undesired results. This level is also assessed during the ‘Evaluate’ phase, but at this level the information gained is of most relevance to the job or process owner.

Step 1: Determine need for training review and evaluation 3.170

3.170 All training should have its effectiveness and efficiency reviewed. The extent and rigour of such reviews should depend on the risks to those qualities. Assessing the risks requires judgment. If a course is new, its inputs untried and the course is high in significance the need for TR&E is likely to be high. Conversely, if a training course/program is well-established, of proven worth, and its inputs and processes do not change, the need will be less. Circumstances differ, but the following points provide guidance in making appropriate judgments.

3.171 Strategies for TR&E have two dimensions: their application to a single course/program and their application to a series of courses/programs over an extended period.

3.172 Most training courses/programs should incorporate level 1, reaction checks, at their conclusion and, if long or involved, at suitable points throughout. The frequency of such checks can vary. Some circumstances justify a higher frequency than daily and others being at greater than monthly intervals.

3–44


3.173 Most training situations will require level 2 training review. This level is the subject of checks incorporated in the assessment strategy. It is usually programmed for the end of training, but may also be used formatively at key points during its conduct.

3.174 Level 3 and level 4 evaluations (see chapter 6—‘The ‘Evaluate’ phase’) are usually conducted concurrently, as the information required is similar and the same methods and processes can serve both purposes. They can be planned as comprehensive reviews, routine checks, or reviews targeted at specific components or aspects of a course/program. All new training should be subjected to a comprehensive evaluation. Once the training is established, routine checks at a more superficial level may suffice. Targeted reviews are planned when a specific need is identified, possibly arising from another continuous improvement process.

3.175 Develop plan of Training Review and Evaluation activities. The product of determining the need for TR&E is an indicative plan of TR&E activities to be applied to a training course/program. The plan should list both the training review activities of a single course/program and the intended schedule of evaluation activities to be applied over the life of a course/program.


3.176 The information requirements of routine TR&E tend to be broad and enduring. They can therefore usefully be incorporated in the TR&E strategies document that informs the development of information collection instruments.

3.177 Level 1 training reviews commonly seek feedback on the efficiency and effectiveness of:

a. the purpose/concept of the training;

b. training administration;

c. locations/venues and facilities used;

d. pre-course requirements;

e. training content;

f. assessment strategies, instruments and conduct;

g. methods, media and material selected together with their use; and

h. the training program.

3–45


3.178 Level 2 training review seeks to determine what trainees have learned. This work is also a function of the assessment strategy, the design of which is focused on measuring and supporting trainees’ progress. In training review the focus is more on the improving the training process. Towards this end, information may be sought on all things learned by trainees, random sample areas or specifically targeted areas.

3.179 The information common to level 3 evaluation (see chapter 6) assesses how well training has prepared learners for the workplace and whether the original capability requirement has been met includes:

a. the required job performances for which training was not provided;

b. the required job performances at which training graduates are performing below the required standard;

c. the competencies trained but not used in the workplace;

d. the competencies trained to a standard higher than required in the workplace;

e. the competencies in which graduates experience difficulty in adapting from off-the-job training to the workplace;

f. reflections on the design and conduct of training received;

g. any workplace performance requirements that are assessed as inadequate;

h. any recent changes to the organisation of work, its procedures or equipment used; and

i. the attitudes of graduates to their work, training and their organisation.

3.180 Level 4 feedback (see chapter 6) is less commonly sought and more likely to be situation-specific. The information requirements are likely to relate to the effects of the training program/course on the organisation and the training staff. They might include attempts to cost the training.

Step 3: Select information sources and collection methods 3.181

3.181 The sources of information on the conduct of TR&E are limited. For training review they are the trainees, their instructors, training administration staff and the results of assessments. For evaluation the principal sources are the graduate trainees, their supervisors and, occasionally, ex-training staff employed in the workplace. Each source is likely to have differing, yet valid perspectives of training. Where possible, therefore, all should be used—albeit that sampling may be employed.

3–46


3.182 Many methods can be used to collect TR&E information. Only the major ones that require the allocation of resources and the development of collection instruments are considered during the ‘Design’ phase. Commonly used methods, which may be applied individually or in combination, include:

a. questionnaires;

b. individual or group structured interviews/discussions/workshops;

c. workplace observation and the use of check sheets;

d. examination of job product; and

e. competency testing.

3.183 Assessing what guidance on information sources and collection methods the TR&E strategies document should include requires judgments on the resource costs and likely benefits. As was noted in step 1, these factors will change over the life of a course/program. TR&E strategies should reflect this.

Step 4: Record the training review and evaluation strategies 3.184

3.184 The strategies developed for TR&E are recorded in a form enabling them to be incorporated in a TMP. The template in annex M provides a suggested format for TR&E strategies. The document will enable information collection instruments to be prepared during the ‘Develop’ phase, and the conduct of TR&E to be administered appropriately in the pilot course process and ‘Conduct’ phase.

COMPILE DRAFT TRAINING MANAGEMENT PACKAGE 3.185

Introduction 3.185

3.185 A TMP presents the complete set of documentation necessary for managing the conduct of a training program or course. It is the output of training preparation and the central resource for the conduct of training.

3.186 A complete TMP contains the curriculum of a program/course, its TR&E strategies, and the materials developed for conducting training. At the end of the ‘Design’ phase a TMP is in draft form, and incomplete. This is partly because the strategies contained are likely to change consequent to work during the ‘Develop’ phase and partly because the materials for conducting a program/course are added during that phase.

3–47


3.187 Despite being incomplete, a draft TMP has all the content needed for a program/course to be accredited by a State or Territory course accrediting body. If such recognition has been deemed desirable, a submission may be made at this stage of the DTM, although waiting until the completion of the pilot process of the ‘Develop’ phase is less risky.

3.188 Purpose. The purpose of compiling a draft TMP is to establish the basis of a single authoritative reference point for a training program/course.

3.189 Trigger. As the summative record of the ‘Design’ phase of the DTM, the draft TMP is compiled following design of the curriculum and the TR&E strategies.


3.190 The inputs, process and products of compiling a draft TMP are summarised in table 3–10. As with other processes in the DTM, the steps are not necessarily sequential and some are likely to be undertaken concurrently.

Table 3–10: Inputs, process and products for compiling a draft Training Management Package

Step 1: Compile Training Management Package 3.191

3.191 A TMP builds on a curriculum by adding course management information and the TR&E strategies. The requirements of a TMP will vary with the needs of the organisation and of the training course/program. An indicative template for a cover sheet, contents page and Training Management Information section is in annex M. This template incorporates the main features of a DEST national template designed to assist the development of courses for accreditation under the AQTF, that lead to an Australian Qualification Framework qualification or Statement of Attainment15.

Inputs Curriculum, training and assessment strategies.Process Step 1: Compile draft TMP.

Step 2: Gain formal endorsement and approval of the draft TMP.Step 3: Submit for accreditation (if required).

Products Draft TMP.

15 Further details about this template can be found in the DEST booklet, Guidelines for Course Developers, July 2002.

3–48


Step 2: Gain formal endorsement and approval of draft Training Management Package 3.192

3.192 Albeit in draft form and incomplete, the draft TMP is endorsed by the Design Authority when they are satisfied that within the context of the situation to which they apply:

a. the curriculum meets the requirement of the TRS;

b. the training and assessment strategies are valid, suitable, reliable and practicable; and

c. the TR&E strategies are appropriate and practical.

3.193 At this point the approval of the customer of training, the job/BPO, is formally sought and obtained. This action serves as a useful check that the requirement of the TRS remains valid. On receipt of this approval, the TMP can properly be used as the basis for work in the ‘Develop’ phase of the DTM.

Step 3: Submit for accreditation 3.194

3.194 If the Analyse Authority determined that the course/program should be recognised by the award of a National qualification, the application may be undertaken when the design of a course is complete. By this time all the information required by the related State or Territory accrediting body is drafted (note: if changes to the documentation are likely to be made consequent to the piloting in the ‘Develop’ phase, it might be preferable to delay the submission until the ‘Develop’ phase has been completed).

3.195 Broad guidelines and a template for submitting a program/course for accreditation are described in Guidelines for Course Developers, July 2002 produced by DEST and accessible on its web site (see http://www.dest.gov.au). Detailed advice on making a submission should, however, be obtained from the State or Territory accrediting body.

3.196 If a proposed program/course is accredited, the State or Territory course accrediting body will provide an accreditation document, such as a letter or certificate that includes the title and national code for the accredited course, and the start and end date of the accreditation. The body will also record the accredited course on the National Training Information Service.

Annexes:A. Flow chart showing the ‘Design’ phase of the Defence training modelB. Defence training model—purpose, inputs, process and products of the

‘Design’ phaseC. National assessment principles and their essential featuresD. Factors to consider when selecting appropriate training modes

3–49

http://www.dest.gov.au

http://www.dest.gov.au


E. Some criteria for selecting appropriate training activitiesF. Guidance for estimating the costs and times of developing new

training materialsG. Model course map and explanationH. Environments profile—templateI. Learner profile—templateJ. Competency analysis sheet—templateK. Training and assessment strategy—templateL. Training review and evaluation strategies—templateM. Training management package—indicative template

3–50


FLOW CHART SHOWING THE ‘DESIGN’ PHASE OF THE DEFENCE TRAINING MODEL A

3A–1



‘DESIGN’ PHASE B
‘D
esig

n’ P

HA

SE

PUR

POSE

INPU

TSPR

OC

ESS

PRO

DU

CTS

To e

nsur

e th

at th

e se

lect

ion

of a

sses

smen

t and

trai

ning

st

rate

gies

is a

ppro

pria

te to

le

arne

rs’ c

hara

cter

istic

s an

d ci

rcum

stan

ces

Trai

ning

Req

uire

men

t S

peci

ficat

ion

(TR

S),

Targ

et

Popu

latio

n Pr

ofile

, cur

rent

po

licie

s an

d in

stru

ctio

ns,

data

from

Def

ence

R

ecru

iting

, job

spo

nsor

s,

surv

ey re

sults

and

nat

iona

l lit

erac

y st

anda

rds

Ana

lyse

pro

spec

tive

lear

ners

Lear

ner p

rofil

e

To h

elp

ensu

re th

at th

e as

soci

ated

stra

tegi

es a

re

appr

opria

te

TRS

, Fea

sibi

lity

Rep

ort

curre

nt p

olic

ies

and

inst

ruct

ions

, inf

orm

atio

n fro

m w

orkp

lace

sup

ervi

sors

an

d m

anag

ers,

trai

ning

or

gani

satio

ns a

nd s

choo

ls,

reso

urce

spe

cific

atio

ns

Anal

yse

lear

ning

and

as

sess

men

t env

ironm

ents

Env

ironm

ents

pro

file

3B–1

ADFP 7.0.2

PUR

POSE

INPU

TSPR

OC

ESS

PRO

DU

CTS

To id

entif

y th

e ba

sic

skill

s (p

hysi

cal a

nd m

enta

l) kn

owle

dge

and

attit

udes

re

quire

d to

ach

ieve

co

mpe

tenc

ies

Com

pete

ncy

stan

dard

s fro

m

the

TRS

, exi

stin

g in

dust

ry

and

Def

ence

tra

inin

g/le

arni

ng m

ater

ials

, an

d th

e le

arne

r pro

file

Anal

yse

com

pete

ncie

sC

ompe

tenc

y An

alys

is

Shee

t (C

AS)

To s

peci

fy th

e ob

ject

ives

of

train

ing

CAS

, lea

rner

and

lear

ning

en

viro

nmen

t pro

files

Gen

erat

e le

arni

ng o

utco

mes

Out

com

es 2

type

s: c

ours

e le

arni

ng o

utco

mes

and

m

odul

e le

arni

ng o

utco

mes

To g

uide

the

use

of

asse

ssm

ent f

or s

uppo

rting

an

d m

easu

ring

lear

ning

in a

tra

inin

g co

urse

/pro

gram

and

fo

r det

erm

inin

g ac

hiev

emen

t of

wor

kpla

ce c

ompe

tenc

e

Com

pete

ncy

units

from

the

TRS,

lear

ner a

nd

envi

ronm

ents

pro

files

, le

arni

ng o

utco

mes

, rel

evan

t ex

istin

g as

sess

men

t re

sour

ces,

trai

ning

failu

re

man

agem

ent p

olic

ies

Des

ign

asse

ssm

ent s

trate

gyTr

aini

ng a

nd a

sses

smen

t st

rate

gy, i

nclu

ding

:

•lo

catio

n, m

etho

ds;

•co

ntin

uous

im

prov

emen

t pr

oces

ses;

•m

edia

, seq

uenc

e;

•re

sour

ce re

quire

men

ts.

To e

nsur

e th

at tr

aini

ng is

co

nduc

ted

effe

ctiv

ely

and

effic

ient

ly, c

onsi

sten

t with

re

leva

nt p

olic

ies

and

lear

ning

theo

ries

Lear

ning

out

com

es, t

he

com

pete

ncy

asse

ssm

ent

stra

tegy

, and

the

lear

ner a

nd

train

ing

envi

ronm

ent p

rofil

es,

exis

ting

train

ing

mat

eria

ls

Des

ign

train

ing

stra

tegy

and

pr

oduc

e th

e tra

inin

g an

d as

sess

men

t stra

tegy

do

cum

ent

3B–2

ADFP 7.0.2

PUR

POSE

INPU

TSPR

OC

ESS

PRO

DU

CTS

To e

nsur

e th

at e

ffect

ive

and

effic

ient

con

tinuo

us

impr

ovem

ent p

roce

sses

are

ap

plie

d to

trai

ning

The

TRS,

lear

ner p

rofil

e,

lear

ning

out

com

es, l

earn

ing

stra

tegy

and

the

com

pete

ncy

asse

ssm

ent s

trate

gy,

estim

ates

of a

vaila

ble

reso

urce

s

Des

ign

train

ing

revi

ew a

nd

eval

uatio

n st

rate

gies

Trai

ning

revi

ew a

nd

eval

uatio

n st

rate

gies

To p

rodu

ce a

trai

ning

do

cum

ent s

peci

fyin

g th

e sp

ecifi

catio

ns, s

trate

gy,

cont

ent a

nd re

sour

ces

requ

ired

to c

ondu

ct a

tra

inin

g co

urse

/pro

gram

Trai

ning

and

ass

essm

ent

stra

tegy

, CAS

C

ompi

le c

urric

ulum

Cur

ricul

um

To c

olla

te in

one

pac

kage

all

cour

se

docu

men

tatio

n/m

ater

ials

Cur

ricul

um, t

rain

ing

revi

ew

and

eval

uatio

n st

rate

gies

Com

pile

and

gai

n ap

prov

al

of d

raft

Trai

ning

M

anag

emen

t P

acka

ge (T

MP

)

Dra

ft TM

P

3B–3

Annex C to ADFP 7.0.2 Chapter 3

NATIONAL ASSESSMENT PRINCIPLES AND THEIR ESSENTIAL FEATURES C

PRINCIPLE ESSENTIAL FEATURESEndorsed industry/enterprise standards are the benchmarks for assessment, where they exist.

1.1 Endorsed industry/enterprise competency standards are the benchmarks for all assessment activity conducted for the purposes of national recognition, regardless of whether this is through a training and assessment pathway or assessment only pathway.1.2 Where national industry/enterprise standards do not exist, specified competencies agreed to by the appropriate industry, enterprise, professional or community group are the benchmarks for assessment.

Assessment guidelines within endorsed training packages provide the framework for assessment of the units of competence within that industry, industry sector or enterprise.

2.1 Where a relevant endorsed training package exists, assessment will be conducted in accordance with the assessment guidelines and competency standards of the endorsed training package.

Endorsed industry/enterprise standards form the basis of qualifications and Statements of Attainment in the vocational education and training sector, where they exist.

3.1 Qualifications within the vocational education and training sector of the Australian Qualifications Framework (AQF) are based on endorsed packages of national industry/enterprise competency standards.3.2 Where national industry/enterprise endorsed standards do not exist, qualifications or Statements of Attainment (SOA) must be based on specified competencies agreed to by the appropriate industry, enterprise, professional or community group.3.3 A qualification of an SOA under the AQF may be attained through a variety of pathways, including training and assessment pathways as well as assessment only pathways.

3C–1

ADFP 7.0.2

Assessment conducted for the purposes of national recognition should lead to a qualification or SOA under the AQF.

4.1 All assessment activity that is conducted for the purposes of national recognition, regardless of whether this is through a training and assessment pathway or an assessment only pathway, should lead to the issuing of a qualification or SOA under the AQF.4.2 A qualification of SOA, recognised under the AQF, should be issued when an individual is assessed as competent against endorsed competency standards.Assessment for national recognition purposes is underwritten by effective mutual recognition arrangements under the Australian Recognition Framework (ARF) which requires all Registered Training Organisations (RTO) throughout Australia to fully recognise and credit qualifications and SOA issued by all other RTO.

Assessment should be undertaken by, or auspiced through, an RTO.

5.1 Assessment for national recognition purposes should be undertaken by, or auspiced through, an RTO.

Assessment for national recognition purposes shall be conducted within a quality assurance framework.

6.1 Assessment for national recognition purposes, regardless of whether this is through a training and assessment pathway or an assessment only pathway, should be undertaken by qualified assessors, involve the use of industry-endorsed assessment guidelines and be underpinned by an appeals process.

PRINCIPLE ESSENTIAL FEATURES

3C–2

ADFP 7.0.2

Assessment against competencies in a training package will be carried out in accordance with the endorsed assessment guidelines within the training package. The endorsed assessment guidelines include the necessary qualifications for those conducting assessment and provide for those situations where more than one person may contribute to the assessment and where the required technical and assessment competencies may not all be held by any one person.6.2 All assessments require the application of the following endorsed competency standards:

• ‘The Assessment Competency Standard: Conduct Assessment in accordance with an Established Assessment Procedure’ and ‘Extension Unit: Plan and Review Assessment’ from the Competency Standards for Assessment.

• The relevant vocational competencies, at least to the level being assessed.

6.3 Assessment carried out by or under the auspices of an RTO will by subject to audit, consistent with ARF requirements.External audit procedures should focus on improving both the operation and outputs of assessment processes.


3C–3

ADFP 7.0.2

Responsibility for assessment resides with the body that issues the qualification or SOA under the AQF.

7.1 Responsibility for assessment should reside with the body that issues the qualification or SOA. 7.2 Partnership arrangements involving RTO can be developed to establish assessment arrangements that enable individuals to gain national recognition under the AQF.

Assessment processes shall be valid, reliable, flexible and fair.

8.1 Assessment processes must cover the broad range of skills and knowledge needed to demonstrate competency.8.2 Assessment of competence should be a process, which integrates knowledge and skills with their practical application.8.3 During assessment, judgments to determine an individual’s competence should, wherever practicable, be made on evidence gathered on a number of occasions and in a variety of contexts or situations.8.4 Assessment processes should be monitored and reviewed to ensure that there is consistency in the interpretation of evidence.8.5 Assessment should cover both on- and off-the-job components of training.8.6 Assessment processes should provide for the recognition of competencies no matter how, where or when they have been acquired.8.7 Assessment processes should be made accessible to individuals so that they can proceed readily from one competency standard unit to another.8.8 Assessment practices must be equitable for all groups of individuals.8.9 Assessment procedures and the criteria for judging performance must be made clear to all individuals seeking assessment.


3C–4

ADFP 7.0.2

8.10 There should be a participatory approach to assessment. The process of assessment should be jointly developed/agreed between the assessor and the candidate.8.11 Opportunities must be provided to allow individuals to challenge assessment decisions with provision for reassessment.

Assessment systems must incorporate mechanisms for recording, storing and accessing assessment outcomes.

9.1 Assessment systems must incorporate mechanisms for recording, storing and accessing assessment outcomes.9.2 The body that is responsible for issuing the qualification or SOA under the AQF should also be responsible for maintaining and operating a secure, permanent and reliable system for recording and storing assessment outcomes.

Assessment recording systems should indicate the units of competence that the individual has achieved.

10.1 Qualifications and SOA issued under the AQF should indicate the units of competency that the individual has attained.10.2 Assessment reporting systems should conform to any national nomenclature developed for reporting assessment outcomes.10.3 SOA issued under the AQF should have a nationally consistent format and include common categories of information.

Assessment systems should incorporate ongoing monitoring and review processes.

11.1 Assessment systems should incorporate ongoing monitoring and review processes.

Assessment processes shall provide for the recognition of current competencies regardless of where these have been acquired.

12.1 Recognition of prior learning (RPL) focuses on identifying the endorsed industry/enterprise competency units currently held by individuals as a result of formal and informal training, not how, when or where the learning occurred.


3C–5

ADFP 7.0.2

12.2 RPL underpins any system of competency based training. It is essential that RTO have a demonstrable commitment to recognising the prior learning of individuals.12.3 RPL shall be available to all potential applicants.12.4 RPL shall involve processes that are fair to all parties involved.12.5 RPL shall involve the provision of adequate support to potential applicants.


3C–6

Annex D to ADFP 7.0.2 Chapter 3

FACTORS TO CONSIDER WHEN SELECTING APPROPRIATE TRAINING MODES D

FACTORS TO CONSIDER WHEN SELECTING LEARNING MODEFactors to consider Trainer-centred Learner-centred

1. Subject matter is predominantly:dynamic Xstatic Xsimple Xcomplex Xconceptual Xprocedural X2. Lessons require:repetition Xconstant delivery Xdetailed record keeping and score tracking

X

instructor observation and evaluation Xself-evaluation by learner X3. Target group is:small X Xlarge X Xof varied backgrounds and capabilities

X X

geographically scattered Xnetwork connected X X4. Feedback required is:specific Xopen or varied X5. Development time is:limited Xflexible X6. Financial resources are:limited Xflexible X7. Simulation is required X X8. Learner’s education background is:extensive Xintermediate X Xpoor X9. Human resource limitations are:few instructors X Xplentiful instructors X X

3D–1

Annex E to ADFP 7.0.2 Chapter 3

SOME CRITERIA FOR SELECTING APPROPRIATE TRAINING ACTIVITIES E

Lecture

Lesson

Demonstration

Performance

Discussion

Independent study

Exercise

Tutorial

Simulation

Role-play

Case-study

Study text

Visit

Drill

Games

Inst

ruct

ion-

base

dx

xx

xx

xx

Expe

rienc

e-ba

sed

xx

xx

xx

xx

xS

kill-

base

dx

xx

xx

xx

xx

Know

ledg

e-ba

sed

xx

xx

xx

xx

xx

xx

Hig

h co

mpl

exity

xx

xx

xx

xx

xx

xx

xLo

w c

ompl

exity

xx

xx

xx

xx

xx

xx

xx

Hig

h re

sist

ance

xx

xx

xLo

w re

sist

ance

x

xx

xx

xx

xx

Hig

h in

tera

ctio

nx

xx

xx

xx

xx

Low

inte

ract

ion

xx

xx

xx

xx

xSt

art o

f cou

rse

xx

xx

xx

xx

xM

iddl

e of

cou

rse

xx

xx

xx

xx

xx

xx

End

of c

ours

ex

xx

xx

xx

xx

xx

xx

xLa

rge

clas

sx

xx

xx

xx

xx

xx

Smal

l cla

ssx

xx

xx

xx

xx

xx

Xx

xH

igh

abilit

yx

xx

xx

xx

xx

xx

Low

abi

lity

xx

xx

xx

xx

xx

Sho

rt tim

e fra

me

xx

xx

xx

xx

xx

xx

Long

tim

e fra

me

xx

xx

3E–1

Annex F to ADFP 7.0.2 Chapter 3

GUIDANCE FOR ESTIMATING THE COSTS AND TIMES OF DEVELOPING NEW TRAINING

MATERIALS F

1. The cost of, and time required to develop new training materials are closely linked, as the major cost involved is that of the personnel required for the work. The length of time required, the numbers of people involved, and the levels of expertise and experience required of those people will vary with the circumstance, each of which is unique.

2. Listed below are the indicative times required for developing the materials course development for an hour of course time. (For example, a ratio of 15:1 means that 15 hours of development are required for each course hour). For each method of training a range of values is presented. The range allows for the specifics of various training situations, including the expertise of the practitioners:

a. Seminar—15:1 to 30:1.

b. Audio tapes or audio compact disc—1:1 to 5:1.

c. Lecture—2:1 to 10:1.

d. Televised lecture—2:1 to 5:1.

e. Video—20:1 to 50:1.

f. Interactive video—50:1 to 100:1.

g. Face to face (F2F) instruction—2:1 to 30:1. Examples are as follows:

(1) adapting extant material—2:1 to 15:1; and

(2) new material developed (from scratch)—approximately 15:1 to 30:1.

h. Online learning/web-based1—7:1 to 1000:1 depending on complexity eg adapting a simple page format could take only 7:1. Other examples are:

(1) Converting F2F material—48:1. This assumes that:

(a) a library of graphics is available;

(b) subject matter experts (SME) are available;

1 Defence Flexible Learning Solution advice, 2003.

3F–1

ADFP 7.0.2

(c) content authors and instructional designers are available;

(d) four hours F2F coverts to approximately one hour of online courseware of approximately 200 screens; and

(e) testing, refinement and trialing are additional.

(2) New material (from ‘scratch’)—approximately 70:1. This assumes that:

(a) a library of graphics is available;

(b) SME are available;

(c) content authors and instructional designers are available; and

(d) testing, refinement and trialing are additional.

(3) Web-based developed within a template2—approximately 40:1 to 300:1 as follows:

(a) text only with limited interactivity—40:1 to 100:1;

(b) text and graphics—100:1 to 150:1;

(c) text, graphics and animation with moderate interactivity—150:1 to 200:1;

(d) learning object-based dynamic web page—60:1 to 300:1; and

(e) online help system 3:1 to 10:1.

(4) Web-based material developed (from ‘scratch’)—approximately 100:1 to 1000:1 as follows:

(a) text only and limited interactivity—100:1 to 150:1;

(b) text and graphics—150:1 to 200:1;

2 Web-based data adapted from Kapp, K.M. How Long Does it Take? Estimation Methods for Developing E-Learning in Learning Circuits—ASTD’s Online Magazine All About E-Learning, 14 July 2003.

3F–2

ADFP 7.0.2

(c) text, graphics and animation with moderate interactivity—250:1 to 400:1;

(d) televised lecture—2:1 to 5:1;

(e) fully interactive text, graphics and animation with full interactivity—400:1 to 600:1; and

(f) fully interactive virtual reality simulation and full interactivity.

i. Computer managed learning—40:1 to 60:1.

j. Distance broadcast (satellite)—15:1 to 30:1.

3F–3

Annex G to ADFP 7.0.2 Chapter 3

MODEL COURSE MAP AND EXPLANATION G

1. A course map is shown below. Each horizontal line represents a course ‘event’ (course introduction, module of instruction, major assessment, visit etc). Written above each line is an event identification letter, and under it is the time required to complete the event. In this example there are eight events, A–H, which take a total of 22 hours to conduct. The instructional sequence demands that A, B and C must occur before D, which, together with E must be completed before F. G can occur at any time after A—the course introduction.

Figure 3G–1: Course map

Key:

2. Each elongated vertical loop represents a point-in-time relevant to the starting time for the course. The two numbers in the loop are the latest event time—the maximum time required to reach that point-in-time, and the earliest event time—the minimum time required to reach that point-in-time.

3. No event to the right of a loop/point-in-time should be started until all the events to the left of that loop are completed. Module time includes instructional time as well as any linked assessment time. There is no significance in the differing sizes of the loops or in the differing lengths of the lines.

Code EventA Course introductionB Module 1—nameC Module 2—nameD Module 3—nameE Module 4—nameF Module 5—nameG Visit H Competency assessment

3G–1

Annex H to ADFP 7.0.2 Chapter 3

ENVIRONMENTS PROFILE—TEMPLATE H

COURSE TITLE 1

1. This document presents the features of the environments to be used for:

a. training for the competencies listed in the Training Requirement Specification, and

b. assessing achievement of the competencies required for the work of job title.

2. It identifies the features or conditions of those environments that can help or hinder the processes.

Learning environment(s)13.3

3. Facility description. Provide a broad description of the workplace/school/college. If relevant, detail its management arrangements. Include the contact details of authorities. If support facilities (eg accommodation, domestic) are needed for using the facility, the need and how it will be met should be identified. Planned modifications to the facility that can influence training should be noted.

4. Access and availability. Note any limitations concerning access to and the availability of the facility now and foreseen.

5. Location. If significant, describe the location in relation to the workplace, learners’ accommodation, associated training facilities and transport facilities.

Course: Provide any course title, code or reference number

Job:

Title. Provide the name or description of the job for which the course is designed.

Designation. Provide any job code or reference number.

1 Where more than one environment is to be used, each should be considered separately.

3H–1

ADFP 7.0.2

6. Activity spaces. Describe aspects of the spaces for activity that might influence training. Possible factors include size and suitability, power and water resources, computer network access, storage features, floor type, changing/washing facilities, heating, lighting, positioning of teaching and learning aids and screens.

7. Resources. Any available training resources should be listed, including equipment, tools, computers, the types and numbers of desks/benches projectors, screens, printed materials, photocopiers and other electrical equipment.

8. Staff. The availability of suitable training staff for both the administration and conduct of training must be covered. Their numbers for both training and administration and their readiness for such work should be described. Similarly, the motivation of the staff and any need to prepare them for new responsibilities should be noted.

9. Finances. Note any financial aspects of the environment that might be relevant (beware of the commercial sensitivity of such information).

10. Policy. Note any policies of possible significance at the local school/establishment, branch or Service levels.

Assessment environment(s) 3.11

11. Facility description. Provide a brief, broad description of the environment (workplace, exercise or training area). If relevant, detail its management arrangements. Include the contact details of authorities. Planned modifications to the environment that would influence assessment should be noted.

12. Access and availability. Note any limitations concerning access to and the availability of the environment now and foreseen. This is particularly important for assessment in working environments. Any security requirements for students and instructors should also be noted.

13. Location. If significant, describe the location in relation to the workplace, associated training facilities, accommodation and transport facilities.

14. Activity spaces. Describe any aspects of the spaces for activity that might influence the assessment. Possible factors include size and suitability, power and water resources, storage features, floor type, changing/washing facilities, heating, lighting.

3H–2

ADFP 7.0.2

15. Resources. Access to any resources that could be used for the assessment should be listed, including equipment, tools, computers, the types and numbers of desks/benches, electrical equipment etc.

16. Staff. The availability of suitable staff for conducting the assessment must be considered. Their numbers and readiness for the task must be described together with their motivation and any need to prepare them for the new responsibilities.

17. Policy. Note any policies at the environment that might have significance, particularly any which might constrain assessment in unusual circumstances.

3H–3

Annex I to ADFP 7.0.2 Chapter 3

LEARNER PROFILE—TEMPLATE I

COURSE TITLE 1

Profile of typical prospective learner 3.2

1. Employment classification. Describe the individual by rank/level, specialisation, stream skill.

2. Proficiencies. List any relevant proficiencies ie qualifications, certificates, licences, employment experience.

3. Aptitudes. List any common aptitude standards.

4. Personal characteristics. List any characteristics of composition by gender, age, culture, location. The age span of the learner group can be relevant.

5. Competencies. List any units of competency not covered by a proficiency and which relate to the course.

6. Academic ability. Where relevant, indicate level of academic ability, especially English and Maths.

7. Physical characteristics. For courses with a special fitness dimension, identify the normal and probable minimum fitness level of the typical learner.

8. Learning styles. Identify any favoured or problematical learning styles, such as experience with self-paced, discovery, on the job training and computer-based learning.

9. Motivation. Identify any significant factors of motivation—towards the course and/or the job.

Course: Provide any course title, code or reference number

Job:

Title. Provide the name or description of the job for which the course is designed.

Designation. Provide any job code or reference number.

3I–1

Annex J to ADFP 7.0.2 Chapter 3

COMPETENCY ANALYSIS SHEET—TEMPLATE J

Job:Title: Vehicle maintainerDesignation: ABMT

Competency:Unit title: Carry out maintenance operationsUnit code: AUR00208B

3J–1

ADFP 7.0.2

Elem

ent

num

ber

Com

pete

ncy

elem

ent

and

perf

orm

ance

cr

iteria

Phys

ical

and

m

enta

l ski

llsK

now

ledg

eA

ttitu

des

Trai

ning

not

es

208B

/02

Iden

tify

and

use

corr

ect

lubr

ican

ts a

nd fl

uids

In

form

atio

n re

quire

d fo

r id

entif

ying

and

co

nfirm

ing

the

lubr

ican

ts

and

fluid

s fo

r par

ticul

ar

appl

icat

ions

is a

cces

sed

and

corre

ctly

in

terp

rete

d.

Rea

ding

flui

d le

vel

indi

cato

rs.

Com

plet

e se

rvic

e lo

gs.

Use

a th

erm

omet

er

and

hydr

omet

er

(bat

terie

s).

Inte

rpre

t th

erm

omet

er a

nd

hygr

omet

er

read

ings

.

Safe

ty re

gula

tions

an

d ra

tiona

le.

Serv

icin

g pe

riods

an

d ro

utin

es.

Type

s, p

rope

rties

an

d us

es o

f lu

bric

ants

and

flui

ds.

Sou

rces

of f

luid

s.

Mix

ing

proc

edur

es

for b

atte

ry a

cids

. D

ange

rs w

ith a

cid.

D

ispo

sal o

f old

lu

bric

ants

/oils

.

Cho

ose

and

com

ply

with

saf

ety

regu

latio

ns, in

clud

ing

dres

s, n

aked

flam

es.

Cle

anlin

ess

pers

onal

an

d en

viro

nmen

tal.

Prac

tical

—us

e of

w

orks

hop

and

vehi

cles

.

Occ

upat

iona

l hea

lth

and

safe

ty

cons

ider

atio

ns—

cl

ose

supe

rvis

ion,

sm

all g

roup

s.

Supp

ly a

nd d

ispo

sal o

f m

ater

ials

.

Lubr

ican

ts a

nd fl

uids

to

be u

sed

for a

ppro

pria

te

syst

ems

are

iden

tifie

d an

d ap

plie

d in

ac

cord

ance

with

m

anuf

actu

rer’s

sp

ecifi

catio

ns.

Use

of

man

ufac

ture

r’s

spec

ifica

tions

.S

elec

ting

corre

ct

lubr

ican

ts/fl

uids

.

Loca

tions

of

man

ufac

ture

r’s

spec

ifica

tions

. R

isks

of

inap

prop

riate

se

rvic

ing.

Con

sist

ent w

ith th

ose

for t

he jo

b.C

heck

net

for

upda

tes/

mod

ifica

tions

. U

se o

f job

aid

.

3J–2

Annex K to ADFP 7.0.2 Chapter 3

TRAINING AND ASSESSMENT STRATEGY—TEMPLATE K

COURSE NAME

TABLE OF CONTENTS

Introduction 3.1

1. Incorporate an introductory statement identifying in broad terms the reasoning behind the approach taken to training and assessment with reference to the learner, environment and competency analyses.

Course name and reference 3.2

1

Course event Page Number

Course introduction 1

Module 1 title 2

Module 2 title 4

Module 3 title 7

Module 4 title 9

Module 5 title 11

Module 6 title 14

Mid course assessment 16

Module 7 title 18

Module 8 title 19

Module 9 title 22

Module 10 title 24

Module 11 title 26

End of course assessment 28

Approval date

COURSE INTRODUCTION

Duration1 Off-the-job training: On-the-job training:

Job

3K–1

ADFP 7.0.2

1 All times listed in this document will be estimates when first entered during the ‘Design’ phase. They will be confirmed following the pilot course process of the ‘Develop’ phase.

Content summary•

•

•

Learner pre-course preparation

RPL or pre-course testing

Summary of resource requirements

Human:

Physical:

$ Cost:

References Text:Web links:

MODULE 1

Module title

Module code

Duration

COURSE LEARNING OUTCOME: No: Level:

Performance

Conditions

Standard

Linked competency as listed in the Training Requirement Specification

Prerequisite and/orCo-requisite modules

3K–2

ADFP 7.0.2

Content summary This module covers the following topics:

•

Assessment method:

Training method:

Summary of major resource requirements

Human:

Physical:

$ Cost:

References Text:Web links:

3K–3

ADFP 7.0.2

Module learning outcomes 3.3

Module learning outcome 1

Conditions

Assessment criteria 1.1

1.2

1.3

Training method

Assessment method

Module learning outcome 2

Conditions

Assessment criteria 2.1

2.2

2.3

Training method

Assessment method

3K–4

ADFP 7.0.2

Figure 3K–1: Course map

3K–5

Annex L to ADFP 7.0.2 Chapter 3

TRAINING REVIEW AND EVALUATION STRATEGIES—TEMPLATE L

Course reference:COURSE NAME

Training review strategy 3.1

The program of training review activities listed in table 3L–1 should be applied to the standard name training course.

3L–1

ADFP 7.0.2

Tabl

e 3L

–1: T

rain

ing

revi

ew s

trat

egy

Act

ivity

Info

rmat

ion

requ

irem

ent

Col

lect

ion

met

hod

Col

lect

ion

inst

rum

ent

Dur

atio

n re

sour

ces

requ

ired

Pre-

cour

se/p

rogr

amD

escr

ibe

each

act

ivity

and

the

sour

ce to

w

hich

it re

fers

to e

g pr

e-co

urse

as

sess

men

ts, a

nd in

trodu

ce tr

aini

ng

revi

ew a

nd e

valu

atio

n pr

oces

ses

to:

1.tra

inee

s2.

inst

ruct

ors

and

staf

fD

urin

g co

urse

/pro

gram

Des

crib

e ea

ch a

ctiv

ity a

nd th

e so

urce

to

whi

ch it

refe

rs e

g:

1.da

ily fe

edba

ck fr

om tr

aine

es2.

end

of

mod

ule

feed

back

fro

m

inst

ruct

ors

End

of c

ours

e/pr

ogra

mD

escr

ibe

each

act

ivity

and

the

sour

ce to

w

hich

it re

fers

eg:

ev

alua

te a

sses

smen

ts o

f com

pete

ncy

Sum

mar

y of

maj

or re

sour

ce

requ

irem

ents

Hum

an:

Phys

ical

:

3L–2

ADFP 7.0.2

Eval

uatio

n st

rate

gy3.2

The

prog

ram

of e

valu

atio

n ac

tiviti

es li

sted

in ta

ble

3L–2

sho

uld

be a

pplie

d th

roug

hout

the

perio

d in

whi

ch th

e na

me

train

ing

cour

se

is c

ondu

cted

.

Tabl

e 3L

–2: E

valu

atio

n st

rate

gy

Act

ivity

Freq

uenc

yIn

form

atio

n re

quire

men

tC

olle

ctio

n m

etho

dC

olle

ctio

n in

stru

men

tD

urat

ion

reso

urce

s re

quire

dC

ompr

ehen

sive

revi

ewR

egul

ar re

view

Targ

eted

revi

ew

1.

2. Sum

mar

y of

maj

or re

sour

ce

requ

irem

ents

Hum

an:

Phys

ical

:

3L–3

Annex M to ADFP 7.0.2 Chapter 3

TRAINING MANAGEMENT PACKAGE—INDICATIVE TEMPLATE M

Copy No (Optional)

TRAINING MANAGEMENT PACKAGE

PROGRAM/COURSE NAME

Ref No

Date of issue: TMP approval date

Training Development Authority:

Location:

Conduct of Training Authority:

Location:

Curriculum approval reference:

3M–1

ADFP 7.0.2

Name and reference number of program/course

TABLE OF CONTENTSSECTION 1 TRAINING MANAGEMENT INFORMATIONCourse purpose

Course description

Course learning outcomes

Pathways

Course awards

Accreditation, recognition and licensing

Articulation and credit transfer

Course reporting

Copyright and licensing

Special instructions

SECTION 2 CURRICULUMCourse eventsCourse introduction

Module 1 title

Module 2 title

Module 3 title

Module 4 title

Module 5 title

Module 6 title

Mid course assessment

Module 7 title

Module 8 title

Module 9 title

Module 10 title

Module 11 title

End of course assessment

Course mapSECTION 3 TRAINING REVEW AND EVALUATIONTraining review strategy: 3–1

Evaluation strategy: 3–2

3M–2

ADFP 7.0.2

1

SECTION 4 MAJOR RESOURCE REQUIREMENTSHuman resources:

instructors:

assessors:

Physical resource requirements:

facilities and infrastructure:

accommodation:

utilities and electronic networks:

Major Defence units:

Financial requirements:

SECTION 5 MODULE PACKS (Contain the training materials used to

conduct training, developed in the ‘Develop’ phase of the DTM)

Course introduction

Module 1

etc

Training Management Plan approval date:Name and ref of course

Section 1

COURSE TITLETRAINING MANAGEMENT INFORMATION

Course purpose. A brief statement of the need for and purpose of the training.

Course descriptionCourse aim.

Course location.

Course design strategy.

Training prerequisite requirements.

Duration.

Phase Hours

Pre-joining preparation

Off-job components

Training modules

3M–3

ADFP 7.0.2

1 Course learning outcome levels and their meanings.

Non-instructional hours

On-job components

Total

Course learning outcomesOn completion of the course, the graduate is able to achieve the following:

Level1

1. 1, 2, 3 or 4 -

2. -

3. -

4. -

5. -

Pathways. Identify alternative pathways eg recognition of prior learning, other courses (eg overseas), workplace learning.

Course awards. List qualifications awarded for successful completion of the course.

Accreditation, recognition and licensingCourse accrediting body. Name of state or territory course accrediting body.

AVETMISS information. Provide classification codes for accredited courses.

Period of accreditation. Inserted after accreditation.

3M–4

ADFP 7.0.2

Level Description of graduate in relation to prescribed

standard

Meaning

1 Trained, but not against a prescribed standard.

Background or familiarisation training.

2

Partially trained against prescribed standard.

Additional training and/or experience required before assessment. Employment under supervision only.

3Fully Trained and ready for assessment against prescribed standard.

Assessment required. Employment under supervision until assessed.

4Trained and assessed competent against the prescribed standard.

Ready for employment.

Recognition. State any recognition given by professional or industry bodies.

Licensing. State, if applicable, the extent to which the course satisfies licensing/regulatory requirements.

Articulation and credit transfer. Provide details of articulation and/or credit transfer arrangements

Course reporting. Describe where and how the results of the course will be reported and recorded.

Copyright and licensing

Copyright. Detail any copyright restrictions on the documentation and identify the owner of the course.

Licensing. State any requirements relating to licensing and/or the franchise of the course for use by other providers. Provide useful contact details.

Special instructions. Provide any special information about the course.

3M–5


CHAPTER 4

THE ‘DEVELOP’ PHASE 4

Introduction 4.1

4.1 The ‘Develop’ phase is the third of five phases constituting the Defence training model (DTM), as shown in figure 4–1. During the ‘Develop’ phase the materials and equipment used to deliver training and assess outcomes are obtained, all the resources contributing to training are prepared, and a trial of the whole program/course is conducted.

Figure 4–1: ‘Develop’ phase of the Defence training model

4.2 As with all phases, processes and steps of the DTM, the extent to which the ‘Develop’ phase of the DTM is applied should match the circumstance. In situations of high significance (eg the introduction of training in support of a major new Defence capability) the guidance provided should be applied more closely and carefully.

Key products and responsibilities 4.3

4.3 The major product of the ‘Develop’ phase is a fully developed and practised program/course presented as a Training Management Package (TMP). A draft TMP was prepared in the ‘Design’ phase. In the ‘Develop’ phase it is completed by the addition of supporting materials and equipment.

4–1


4.4 Responsibility for the ‘Develop’ phase and for approving TMPs lies with an organisation’s Development Authority. The cooperation of the Conduct Authority and of Defence authorities responsible for major training resources is required.

Starting point 4.5

4.5 The processes of the ‘Develop’ phase are applied to develop new or, more commonly, to improve existing training. When applied to new training, development work follows-on from the approval of a draft TMP that signifies completion of the DTM’s ‘Design’ phase. The trigger for re-developing existing training materials and equipment is recognition that cost-effective enhancements can be made.

Processes of the ‘Develop’ phase 4.6

4.6 The ‘Develop’ phase of the DTM has four processes:

a. checking existing training materials and equipment,

b. obtaining training materials and equipment,

c. preparing major training resources, and

d. piloting the training.

4.7 The flow chart in annex A illustrates these processes, together with their products. Annex B presents the purpose, inputs and products of the processes in table form. The following paragraphs introduce each of the processes and subsequent chapters of this manual provide guidance on the steps for undertaking them.

4.8 Check existing training materials and equipment. The first step in the ‘Develop’ phase is to check whether suitable materials already exist. If they do, and may be acquired, then a decision is made as to whether they should be used or whether new resources should be developed.

4.9 Obtain training materials and equipment. Obtaining the materials and equipment required for making assessments, conducting training, and implementing Training Review and Evaluation (TR&E) covers a wide range of simple and complex activity, and may involve major expenditure. The resources can be obtained either by purchase or through production.

4–2


4.10 Prepare major training resources. The major resources needed to implement training include facilities (such as training venues, military platforms and accommodation); infrastructure (such as power, water and communication networks); and people (such as trainees, training and assessment staff, workplace supervisors and training administrators). Preparation of such resources ensures they are available and ready for implementing a training program/course.

4.11 Pilot the training. Training is piloted when, newly developed, it is tested under ‘operational’ conditions and subjected to an in-depth training review. The process is recognition that designing and developing good training is an iterative process, and the best way of identifying possible improvements is to test the program/course in circumstances that are as real as possible. If the training review identifies needed enhancements to the content, strategies, methods, media or conduct of training, they are reported for action.


4.12 While working on any of the processes of the ‘Develop’ phase, stakeholder input is sought to help assure the quality of the product. A consequence of such formative evaluation is that many revisions may be necessary before products are accepted. To keep track of the developments and to enable reference during later development work, an accurate and comprehensive record of development activity should be maintained.


4.13 In accordance with national and Defence guidelines, the quality assurance of all training by, or on behalf of Defence (by Defence staff, contractors, professional service providers or other Registered Training Organisations) remains the responsibility of Defence. Notwithstanding this, external agencies may be used to undertake work in the ‘Develop’ phase. Where this is being considered, synergies might be achieved by packaging design and development work in a single contract.

CHECK EXISTING TRAINING MATERIALS AND EQUIPMENT 4.14

Introduction 4.14

4.14 The ‘Develop’ phase of the DTM is when the materials and equipment needed to conduct, assess and evaluate training are produced and/or prepared. The first step in the ‘Develop’ phase is to check whether suitable materials and equipment already exist. If they do, and may be acquired, then a decision is made as to whether they should be used, or whether new resources should be developed.

4–3


4.15 The work of checking existing training materials and equipment is a continuation of:

a. the feasibility analysis (when existing training was first investigated);

b. the analysis of learning and assessment environments (in which available resources were listed); and

c. the design of strategies (which used knowledge of and/or assumptions about resources).

4.16 Purpose. The purpose of checking the existence of training and assessment materials and equipment is to minimise development costs and to standardise items used, where appropriate.

4.17 Triggers. The start of the ‘Develop’ phase triggers the checking of existing training materials and equipment. This follows on from acceptance of the draft TMP at the end of the ‘Design’ phase.


4.18 The inputs, process and products of checking existing training materials and equipment are summarised in table 4–1.

Table 4–1: Inputs, process and products for checking existing training materials and equipment

Step 1: Identify requirements 4.19

4.19 Before determining the suitability of existing materials and equipment, the requirement must be understood. This understanding is developed in the ‘Analyse’ and ‘Design’ phases and presented in section 4 (course resource requirements) of a TMP. It is refined in the ‘Develop’ phase.

Inputs Draft TMP, Feasibility Report, environments profile, training references.

Process Step 1: Identify requirements. Step 2: Research existing Defence materials and equipment.Step 3: Research external sources.Step 4: Decide whether to acquire existing and/or develop new materials and equipment.

Products Decision to either acquire existing and/or develop new training materials and equipment.

4–4


4.20 The requirement should be considered from several perspectives. At a macro level it consists of a whole course, one or more modules, and events and activities. At a micro level it consists of each item of material and equipment used during a program/course, together with the guidance that links them in the learning process. The requirement consists of one or more lists of the materials and equipment needed to conduct assessments, implement the training, support learning, review the training and conduct an evaluation. For each listed item, required specifications should be stated. Once the contents of the list have been drafted, the questions of how much of each item is needed, and when, should also be considered.

4.21 The primary references for the requirement are strategies described in a curriculum (section 2 of a TMP) and for TR&E (section 3 of the TMP). These strategies might make direct reference to the materials and equipment required. Alternatively, judgments will need to be made on the requirement. Such judgments need to consider the:

a. required learning outcome;

b. characteristics of the learner and learning environments;

c. contribution/value to learning required;

d. cost of resource through-life (acquisition and maintenance);

e. ease of update;

f. time to acquire; and

g. alternative uses.

The step is complete when a detailed list of all the materials and equipment required to implement the program/course in accordance with the draft TMP has been produced.

Step 2: Research existing Defence materials and equipment 4.22

4.22 The search for materials and equipment begins with the interrogation of Defence resource databases. Single-Service and Group policies and procedures will provide the detailed procedure for this step.

Step 3: Research external sources 4.23

4.23 Where resources are unable to be sourced from within Defence, support from external agencies may be considered. Single-Service and Group policies and procedures will provide the detailed procedure for this step.

4–5


Step 4: Decide and record whether to acquire existing or develop new materials and equipment 4.24

4.24 When existing training materials and equipment are found, their suitability and likely effectiveness are weighed against their affordability, reliability, availability, maintainability and supportability and need for and ease of customisation. This judgment is then compared with the options of developing a new resource, or progressing without the resource and a decision to pursue one of the three courses is made.

OBTAIN MATERIALS AND EQUIPMENT FOR ASSESSMENTS AND TRAINING 4.25

Introduction 4.25

4.25 The process of obtaining materials and equipment for making assessments, conducting training and TR&E, covers a wide range of activity and resource expenditure. Materials and equipment may either be purchased directly or produced. In either situation, processes may be complex or relatively simple dependent on the circumstances of the acquisition.

4.26 Purpose. The purpose of obtaining training materials and equipment is to gain cost-effective resources for implementing a program or course.

4.27 Trigger. The work of obtaining resources to implement training follows on from the decisions about the requirement for and means of, gaining the required materials and equipment.


4.28 The inputs, process and products for obtaining training materials and equipment are summarised in table 4–2.

Table 4–2: Inputs, process and products for obtaining training materials and equipment

Inputs Draft TMP, facilities, raw materials and equipment, finances, internal and external staff, training facilities and infrastructure.

Process Step 1: Acquire training materials and equipment for assessing and implementing training, and for TR&E.Step 1.1: Obtain existing materials and equipment.Step 1.2: Customise materials and equipment.Step 1.3: Produce new materials and equipment.Step 1.4: Formatively evaluate materials and equipment.Step 2: Add materials to draft TMP.

Products Draft TMP completed by addition of materials and equipment for assessing and conducting training, TR&E.

4–6


Step 1: Obtain training materials and equipment 4.29

4.29 The materials and equipment used can be obtained by either acquiring existing resources—in which case they may require customisation to meet Defence needs—or producing new resources. Whichever method is used, formative evaluation should be used to ensure the resources function as required. In most situations the number of items needing to be obtained is such that all four of the processes comprising the step will be managed concurrently.

4.30 There are four processes involved in this step. They are:

a. obtain existing materials and equipment,

b. customise materials and equipment,

c. produce new materials and equipment, and

d. formatively evaluate materials and equipment.

Step 1.1: Obtain existing materials and equipment 4.31

4.31 Existing materials and equipment can be obtained in three basic ways, either by an organisation alone or in partnership with another. They can be:

a. borrowed or reproduced at no cost from internal or friendly sources (eg other Defence organisations, Government departments, or military services overseas);

b. purchased, either outright, or under a license to use, from commercial or other sources; and

c. leased/rented/hired from commercial or other sources.

Step 1.2: Customise materials and equipment 4.32

4.32 Once acquired, materials and equipment are likely to need customisation to meet the requirements and standards of the organisation. Customisation can involve additions, deletions, updates and corrections; as well as amending the design and presentation of the resource to achieve the desired branding.

4.33 The work of customising existing resources to meet the requirement is little different to the work of producing new resources described in Process 3. As it progresses, care needs be taken when changing anything that might affect the external accreditation of the course. For this reason, customisation is often best carried out by the organisation that initially developed the materials and/or equipment. If these are being procured, customisation should be addressed in any contractual agreement.

4–7


4.34 Copyright and patent. Copyright and patent laws provide protection to the originators of materials and equipment from unauthorised use. If training resources that are subject to those laws are to be customised then written permission must be obtained from the owner, and a legal copy of the documentation should be secured. Similarly, if Defence is involved in the production of materials/equipment over which it wishes to exert copyright or patent, such action may be pursued. These actions will involve time and money, and require expertise. Guidance on copyright and patents is contained in the Defence Procurement Policy Manual and can be accessed from the Defence Legal Office.

Step 1.3: Produce materials and equipment 4.35

4.35 The costs of producing new equipment for training are such that it is normally purchased commercially rather than being developed or manufactured by Defence. This is not the case for materials, which are more easily and efficiently produced specifically for a known requirement.

4.36 The challenge in producing materials for use by learners, trainers, assessors, training administration staff and quality control staff is to make them:

a. achieve what is intended—so they should meet 100 per cent of the requirement;

b. understandable—so they should be presented clearly, at the appropriate level; and

c. attractive—so their use is encouraged.

These qualities are the products of effective planning that applies the resources available to the subject content, its structure and detailed learning strategies, its presentation, and related technical issues.

4.37 Subject content. In the iterative process of drafting and checking learning materials, attention must be paid to ensuring that the content matches the requirement specified in the curriculum, and that it is accurate and current. This could be the first time that the detailed contents, or the references where detailed content can be found, are committed to a program or course (the curriculum describes content rather than states it). Having subject matter expertise to provide or check content is therefore important.

4–8


Step 1.4: Formatively evaluate materials and equipment 4.38

4.38 Formative evaluation should be applied to materials and equipment as they are developed. This helps avoid unplanned delays and costs at the end of the process. Checks and tests of the materials and equipment during their development should evaluate their:

a. validity—the extent to which they meet the specified requirement;

b. suitability—how well they can be used by learners and training staff in the environments and circumstances they are designed for;

c. reliability—how consistently do they achieve their outputs; and

d. cost-effectiveness—with a view towards improvement.

4.39 A mix of stakeholders should contribute to this continuous improvement work. Subject matter experts representing the job/Business Process Owner (BPO) should check the accuracy and validity of what is being trained/learned. The training staff who will implement the training in the ‘Conduct’ phase should be used to check the suitability of the materials and equipment. Learning/training technologists with expertise in the processes of the DTM can make judgments on the reliability of resources, and all these stakeholders can contribute to judgments on the cost-effectiveness of the materials and equipment.

4.40 No matter how much work is invested in acquiring, customising, and producing the materials and equipment required to implement training, there is the potential to do more and better. Continuous improvement is highly applicable to this area of the DTM, but the resources used should be limited to the potential benefit to be gained.

4.41 The procedures and policies for the execution of these four processes are managed in accordance with single-Service and Groups direction.

Step 2: Add materials to draft Training Management Package 4.42

4.42 By adding the materials to the draft TMP produced during the ‘Design’ phase, the draft TMP is completed.

4–9


PREPARE MAJOR TRAINING RESOURCES 4.43

4.43 Preparing major resources to be used for a training program or course is either undertaken before the introduction of new training or when major changes to existing training are made. It is distinct from the more focused work of preparing to conduct a single training program or course. The process describes the work of ensuring the availability of, and making ready the major resources required to implement the program/course. Such resources normally include:

a. facilities and infrastructure—including schools, training rooms, workshops, accommodation and related buildings for traditional training; and electronic networks and software that enable e-learning;

b. people—including staff for training management, administration, and implementation (assessors, instructors, mentors, workplace supervisors); and

c. major Defence units and platforms—such as ships, aircraft, major simulators and units.

4.44 The major resources required for the training should have first been identified during the feasibility analysis process of the ‘Analyse’ phase and recorded in the Training Requirement Specification. As soon as the need for major training resources is known, their acquisition and preparation can be planned and implemented. This work can involve Defence organisations other than training, including: materiel, estate, personnel (including work force planning, and recruitment), budget and finance, and infrastructure. As such, it may be outside the direct scope of the DTM and the responsibilities of people working within it. If this is the case, the training staff involved with the DTM should retain an active involvement in that work, helping to ensure that the resources are prepared and ready when needed.

4.45 Even though not responsible for providing the major resources, the Development Authority will have responsibility for coordinating action that ensures they are ready: firstly for piloting the program/course and, subsequently, for conducting the training. This work might involve contributing to:

a. securing the facilities and infrastructure;

b. acquiring and fitting furnishings and equipment;

c. conducting acceptance testing of major simulators and equipment;

4–10


d. ensuring the long-term availability of any required domestic and social accommodation;

e. advertising for and recruiting, and/or assigning staff for training, administration, training conduct and evaluation;

f. coordinating the ongoing provision of resource inputs from other organisations (such as the Services); and

g. budgeting and allocating the funds required for the training.

4.46 When the major training resources have been prepared, the facilities, infrastructure, people, equipment and other significant management inputs are ready for the training to be piloted and implemented.

PILOT ASSESSMENTS AND TRAINING 4.47

Introduction 4.47

4.47 The purpose of the pilot is to identify any factors that prevent or restrict the implementation of the resources and materials that have been designed and developed. If the process identifies changes to the content, strategies, methods, media or conduct of either the training or assessment developed, they are reported for corrective action.

4.48 A report of the pilot process indicates whether or not the materials developed are ready for implementation. If the Development Authority is satisfied that it is, the TMP is approved. If not, then the necessary changes are made and, possibly, piloted again.

4.49 While management of the pilot process is the responsibility of the Development Authority, its success requires the coordinated efforts of staff from the Conduct, Design and Analyse Authorities.

4.50 Purpose. The purpose of piloting developed training and assessments materials is to evaluate their readiness for implementation.

4.51 Trigger. Training and assessment materials are piloted when a draft TMP, or a component of one, has been completed and all major resources have been prepared.

4–11



4.52 The inputs, process and products for pilot assessments and training are summarised in table 4–3.

Table 4–3: Inputs, process and products for pilot assessments and training

Step 1: Plan the pilot assessment and/or training 4.53

4.53 Planning the conduct of a pilot assessment and/or training (whether a program, course, module or lesser unit) is a responsibility of the Development Authority, but the work is undertaken in partnership with staff responsible for assessing competencies, and conducting training and training reviews. The work involves considering, answering and then responding appropriately to the following questions:

a. When and where will the assessment/training pilot be conducted?

b. Who will administer and conduct the activity?

c. Who will evaluate the activity?

d. What preparation of staff and other resources is required? (It may be necessary to provide training for the assessment and/or instructional staff. The staff may need instructional and/or assessment competencies in addition to the technical competencies relating to their occupation).

Inputs Draft TMP; training administration, instruction and assessment staff; trainees; learning and assessment facilities, equipment and prepared materials; and training review materials.

Process Step 1: Plan the pilot assessment and/or training.Step 2: Plan and prepare the evaluation of the assessment and/or training.Step 3: Implement the assessment and/or training.Step 4: Review, report and follow-up the assessment and/or training.Step 5: Approval of TMP.Step 6: Set up the course(s).

Products A report on the pilot process recommending revisions, amendments to the TMP and, finally, an approved TMP.

4–12


4–13

e. How much time should be allocated to conduct the activity? (A contingency must be allowed for faulty estimates in the time required, to support learners whose learning suffers through no personal fault, and to conduct a training review that will require more time than for subsequent courses).

f. How much revision of the resources can be achieved during the piloting process?

g. How long after the pilot activity will be required to complete this revision?

h. What special resources will be required to conduct the pilot process?

4.54 The timing of the pilot activity is determined partly by the schedule to complete development of the training and partly by the organisation’s need for qualified people. If the assessment/training is be piloted shortly after the required resources are obtained and prepared, preparatory planning should begin early in the ‘Develop’ phase. Another consideration is the allowance of sufficient time between the pilot process and the use of the resources to enable necessary revisions to be made.

Step 2: Plan and prepare the evaluation of the pilot assessment and/or training 4.55

4.55 While development staff ensure it occurs, the work of evaluating a pilot assessment or training should be undertaken by experienced training review staff that are independent of the inputs involved.

4.56 The breadth and depth of the evaluation should match the significance of the resources being piloted. In most instances training review staff should develop a plan containing an aim, objectives and the means of achieving those objectives. Gaining the Development Authority’s approval of the plan is recommended, as it confirms the evaluation addresses agreed requirements and helps authorise its implementation.

4.57 Reviews of assessment and training processes usually incorporate, or draw from, a common set of objectives:

a. to evaluate course documentation for:

(1) information accuracy;

(2) consistency and contiguity;

(3) conformance with set formats; and

(4) usefulness to learners, trainers, assessors and administrators;


b. to evaluate the effectiveness and efficiency of teaching strategies;

c. to evaluate the conduct of training and other programmed activities for consistency between intended and actual training events (eg time, sequence, content, facilities and equipment, administration);

d. to evaluate all assessments for validity, suitability and reliability;

e. to evaluate the suitability of the facilities and infrastructure used;

f. to evaluate the utility of materials and equipment used;

g. to establish the time required for training by timing each component;

h. to evaluate the efficiency of course administration; and

i. to evaluate assessment processes and results in relation to learning objectives.

4.58 In determining what data is needed to inform the evaluation work, consideration needs be given to how data will be collected, analysed and presented. This level of detail should be documented and agreed by the major stakeholders.

4.59 A wide range of methods and sources may be used to collect data and information for the evaluation. They include:

a. checks of the TMP, facilities and infrastructure;

b. pre-course assessment of learners;

c. observations, evidence from and results of assessment and training activities;

d. computer software and training devices that collect and record performance results;

e. learners’ written formative critiques and/or diaries;

f. end-of-course critique and review; and

g. interviews/discussions with learners, instructors, and other training staff (types in accordance with purpose).

4–14


4.60 Most of the methods listed use materials that need design and preparation. Some materials will be those previously designed and developed for ongoing training review (in these instances their design should also be a subject to evaluation). Other materials will need preparing specifically for use in the pilot process.

Step 3: Implement the assessment and/or training 4.61

4.61 Once planning is complete, the assessment and/or training being piloted and the collection of evaluation data can begin. To gain maximum benefit from the pilot process, it is important that everyone involved cooperates towards achieving both the desired outcomes of the activity and the objectives of its evaluation. Learners and individuals being assessed should both fulfil the requirements of the assessment/training and contribute to the evaluation process. Staff responsible should implement the assessment/training in accordance with the draft TMP, but also be mindful of achieving the required outcomes and of contributing to the evaluation. Training review staff should follow the plan for the review, aiming not to affect the activity. Meanwhile, the Development Authority coordinates the conduct and evaluation processes and, as required, provides guidance on the TMP.

4.62 The pilot assessment/training should be administered and conducted in accordance with both the draft TMP and the normal management procedures of the organisations responsible for them.

Step 4: Review, report and follow-up the pilot assessment and/or training 4.63

4.63 To review the pilot assessment/training:

a. data and information collected in accordance with the evaluation plan is collated and considered in relation to the plan’s objectives;

b. conclusions are drawn;

c. options for overcoming any identified problems are evaluated; and

d. preferred options are presented as recommended courses of action.

This process and its products are formally documented and reported so that the satisfactory elements of the TMP are confirmed, corrective action can be taken where needed and a record of the whole process is maintained.

4–15


4.64 The report of the assessment and/or training piloted should be addressed to the Development Authority, with other stakeholders being informed. The Development Authority should then request appropriate authorities to remedy any deficiencies described in the report, setting any applicable deadlines.

4.65 Although the ‘review, report and follow-up’ step is shown sequentially after ‘implement’, the two work processes are likely to be undertaken in parallel. This means that following completion of an assessment or training activity, evaluation and revision of it can occur whilst subsequent activities are being conducted. Adopting this approach means the period between completing the pilot process and finally approving the developed training resources is minimised.

Step 5: Approval of Training Management Package 4.66

4.66 When following revision and additional checks of any revised assessment and/or training resources, the Development Authority is satisfied with the TMP, the package of documents/materials is ready for approval. As the ‘customer’ of the training, the approval of the policy owner or job/BPO is formally sought. Gaining such approval signifies that training is ready for implementation in the ‘Conduct’ phase.

Step 6: Set up the course 4.67

4.67 A final administration task of preparing new Defence training may involve the set up of the new course(s) in the Defence human resource database, Personnel Management Key Solutions (PMKeyS). By entering the course(s) into PMKeyS the training is made visible to Defence personnel.

4.68 The setup process for new training requires course information to be entered on PMKeyS. The required information should all be recorded in the training specification section of a TMP.

Annexes:A. Flow chart showing the ‘Develop’ phase of the Defence training modelB. Defence training model—purpose, inputs, process and products of the

‘Develop’ phase

4–16

Annex A to ADFP Chapter 4

FLOW CHART SHOWING THE ‘DEVELOP’ PHASE OF THE DEFENCE TRAINING MODEL A

4A–1



‘DEVELOP’ PHASE B

‘Dev

elop

’ PH

ASE

PUR

POSE

INPU

TSPR

OC

ESS

PRO

DU

CTS

To m

inim

ise

deve

lopm

ent

cost

s an

d st

anda

rdis

e re

sour

ces

used

.

Dra

ft Tr

aini

ng M

anag

emen

t Pa

ckag

e (T

MP)

, tra

inin

g re

fere

nces

.

Che

ck a

vaila

bilit

y of

exi

stin

g m

ater

ials

and

equ

ipm

ent f

or:

asse

ssm

ent,

train

ing

and

lear

ning

, Tra

inin

g R

evie

w

and

Eval

uatio

n (T

R&E

).

Dec

isio

n to

eith

er a

cqui

re

exis

ting

and/

or d

evel

op n

ew

train

ing

reso

urce

s.

To g

ain

cost

-effe

ctiv

e m

ater

ials

and

equ

ipm

ent

for i

mpl

emen

ting

a tra

inin

g pr

ogra

m o

r cou

rse.

TMP

, exi

stin

g tra

inin

g re

sour

ces,

fina

ncia

l and

ot

her r

esou

rces

incl

udin

g hu

man

exp

ertis

e, D

efen

ce

inst

ruct

ions

gov

erni

ng

acqu

isiti

on p

olic

ies

and

proc

esse

s.

Obt

ain

exis

ting;

and

/or

prod

uce

new

trai

ning

m

ater

ials

and

equ

ipm

ent.

Dra

ft TM

P co

mpl

ete

with

m

ater

ials

, and

equ

ipm

ent f

orm

akin

g as

sess

men

ts a

nd

cond

uctin

g tra

inin

g,

and

TR&

E.

4B–1

ADFP 7.0.2

PUR

POSE

INPU

TSPR

OC

ESS

PRO

DU

CTS

To e

nabl

e th

e im

plem

enta

tion

of a

trai

ning

pr

ogra

m o

r cou

rse.

Des

igna

ted

adm

inis

tratio

n,

asse

ssm

ent a

nd

inst

ruct

iona

l sta

ff; le

arni

ng

and

asse

ssm

ent s

trate

gies

; cu

rricu

lum

; tra

inin

g in

frast

ruct

ure;

equ

ipm

ent

and

TR&

E re

sour

ces;

ad

min

istra

tion;

sys

tem

; fin

ance

.

Prep

are

maj

or tr

aini

ng

reso

urce

s.Fa

cilit

ies,

infra

stru

ctur

e,

peop

le, a

nd o

ther

maj

or

reso

urce

s ar

e re

ady

to

impl

emen

t tra

inin

g.

To p

rove

the

effic

ienc

y an

d ef

fect

iven

ess

of th

e pr

ogra

m/c

ours

e.

TMP,

trai

ning

and

as

sess

men

t sta

ff, ta

rget

le

arne

r pop

ulat

ion,

lear

ning

an

d as

sess

men

t fac

ilitie

s,

equi

pmen

t and

pre

pare

d m

ater

iel,

TR&

E m

ater

ials

.

Pilo

t the

trai

ning

.R

epor

t on

the

outc

omes

of

the

pilo

t pro

cess

lead

ing

to

an a

ppro

ved

TMP

and

train

ing

proc

edur

es.

4B–2


CHAPTER 5

THE ‘CONDUCT’ PHASE 5

Introduction 5.1

5.1 Following their design and development, training and assessment are implemented in the ‘Conduct’ phase of the Defence training model (DTM) as shown in figure 5–1. Each of the processes comprising the ‘Conduct’ phase will have been piloted and proven during the ‘Develop’ phase of the DTM. In the ‘Conduct’ phase they become normal workplace practice. Successful completion of the phase signifies that the needs specified in the Training Requirement Specification have been met.

Figure 5–1: ‘Conduct’ phase of the Defence training model

5.2 The ‘Conduct’ phase involves preparing for training and/or assessment, the conduct of those activities and, on completion, the conduct of a training review aimed at improving subsequent iterations.

5.3 As with all phases of the DTM, the extent to which the processes and steps of the ‘Conduct’ phase are applied should be appropriate to the circumstances. In situations of high significance (eg training and assessment closely associated with combat capability) the processes and steps should be applied rigorously. Abbreviated processes may be appropriate in lesser circumstances.

5–1


Key product and responsibilities 5.4

5.4 The key products of the ‘Conduct’ phase are trained people who, through assessment, are recognised as having the proficiencies required by the organisation. These are people who can be employed to achieve the required workplace performances identified in the ‘Analyse’ phase of the DTM. Not all training immediately leads to competence. Additionally, on-the-job training (OJT) and/or experience may be required before competency assessment is undertaken.

5.5 Responsibility for implementing training and assessment rests with the Conduct Authority. The Conduct Authority oversees off-the-job and OJT, training conducted internally and by external contractors. In the Australian Defence Force such authority is generally vested in, and delegated by the Training Command (for Army and Air Force) or Training Authority (for Navy), with responsibility for OJT and assessment being shared with the job owner. Joint training is the responsibility of a nominated manager of joint training. In the Department, responsibility for implementing training generally lies with the related policy, business process or job owner.

Starting point 5.6

5.6 A training course/program or an assessment activity is initially implemented following its design, development, and formal acceptance in a Training Management Package (TMP). Following this start, programs/courses and/or activities are undertaken as necessary and may be conducted concurrently, consecutively or intermittently.

Processes of the ‘Conduct’ phase 5.7

5.7 The ‘Conduct’ phase has five major processes which are illustrated in the flow diagram in annex A:

a. administer training preparation;

b. implement pre-training assessments/tests;

c. conduct training;

d. conduct assessment; and

e. conduct post-course administration and training review (internal evaluation).

5.8 The flow chart in annex A illustrates these processes, together with their products. Annex B presents the purpose, inputs and products of the processes in table form. The following paragraphs introduce each of the processes, and provide guidance on the steps for undertaking them.

5–2


5.9 Administer training preparation. Administration involves the coordination, provision and readiness of the required inputs, the trainees/assessees, staff, major resources, materials and equipment. This work must be started at an appropriate time before the activity is scheduled.

5.10 Implement pre-training assessments/tests. Not all Defence training programs and courses have pre-course tests and assessments. Where used, however, they can be used to:

a. determine recognition of prior learning (RPL);

b. check the prerequisite requirements of enrolled trainees; and/or

c. customise training to meet trainees’ particular needs.

5.11 The requirements for pre-course tests/assessments are contained in TMPs. The steps involved in implementing them are common to any assessment and can be summarised as preparation, conduct, evaluation of results and instigation of action appropriate to their original purpose.

5.12 Conduct training. The conduct of training is the process of implementing designed and developed activities for achieving specified learning outcomes that develop individuals’ competencies and related skills, knowledge and attitudes. The range of training activities is great, including instruction, distance learning, e-learning, coaching, individual and team/group work, and formative assessment. The activities may take place OJT, or off-the-job in training centres or other locations. The training and assessment activities to be used for programs/courses are specified in TMPs, and should be suitably prepared and implemented by qualified trainers/facilitators. To achieve quality training these staff members must apply appropriate learning principles and theories. The product of such work is people who have achieved the outcomes of the training. In due course, they can have their new competencies and/or attributes assessed and recognised.

5.13 Conduct assessment. Summative assessment, usually conducted on completion of training, determines the extent to which desired learning outcomes have been achieved1. Summative assessment may also be conducted in the workplace, sometime after completion of OJT or off-the-job training.

1 A form of summative assessment, RPL, may be conducted without reference to training, or before a training program/course, to avoid unnecessary training.

5–3


5.14 Summative evaluation is used with most Defence training to confirm that trainees have achieved required competencies, proficiencies or attributes. Summative assessments are conducted using steps common to all other assessments: preparation, implementation, the evaluation of evidence and administration of results.

5.15 Post-course administration and training review (internal evaluation). The two linked processes of post-course administration and the conduct of a training review are undertaken on completion of a program/course and the assessment of trainees. Post-course administration commonly involves the issue of qualifications (or, if required, the instigation of remedial measures); the recording of course outcomes; the ‘housekeeping’ tasks of restoring the training resources; and contributing to training reviews. It closes one course and enables the conduct of subsequent courses. Training review is a continuous improvement process aimed at making those subsequent courses as efficient and effective as possible. In the context of the DTM the term ‘training review’ is also referred to as internal evaluation. It involves the collection, collation and analysis of data and information about a training program/course in relation to its purpose and conduct. Feedback from stakeholders, including trainees, trainers/facilitators and training administrators is used. A training review confirms the inputs and processes of training and assessment that are satisfactory, and/or makes recommendations for improving those that require development.


5.16 The concept of continuous improvement should pervade the ‘Conduct’ phase of the DTM. Trainers/facilitators, other training staff and trainees involved in the ‘Conduct’ phase should evaluate their involvement with a view to improving training delivery. Improvements are possible in course design, the administration of training and assessment; the resources, materials and equipment used; and the application of competencies by trainees, training and administration staff. The formality of staffing the evaluation activity can vary. At a basic level the self-improvement of individuals involves little or no staffing. The conduct of training reviews on completion of courses is a more formal level, while one-off, targeted training reviews are a highly structured activity. To keep track of the developments in training and assessment, and to enable reference to measures of performance, accurate and comprehensive training, and training review records must be maintained.


5.17 In accordance with national and Defence guidelines, the quality assurance of all training by, or on behalf of Defence (by Defence staff, contractors, professional service providers, or other Registered Training Organisations (RTO)) remains a Defence responsibility. External agencies

5–4


may be contracted to provide training and assessment services, but where they are, the designated Conduct Authority retains responsibility for the products of the services provided.

ADMINISTER TRAINING PREPARATION 5.18

Introduction 5.18

5.18 The first process of conducting training and assessment is to coordinate the provision and readiness of all the required inputs; the trainees/assessees, staff, major resources, materials and equipment. This is primarily an administration function.

5.19 Purpose. The purpose of administering training preparation is to ensure that the implementation of training and assessment is achieved effectively and efficiently to meet the planned requirement.

5.20 Trigger. The administration of training preparations is triggered by a combination of the need for trained individuals and the readiness of the TMP. This does not mean that the process and phase begin on completion of the TMP. In many situations the preparations for the conduct of training must begin during the ‘Develop’ phase of the DTM. Occasionally they might be initiated a period after completion of the ‘Develop’ phase.


5.21 The circumstances of each training situation vary. Consequently the inputs, process and products of administering training preparation summarised in table 5–1 should be considered as indicative.

5–5


Table 5–1: Inputs, process and products of administering training preparation

Step 1: Promote the training 5.22

5.22 Training, particularly new training involves change, which must be promoted and managed. In promoting the positive aspects of training, the need for, and the benefits of the required competencies to both the organisation and the individuals should be explained, together with the implications of the developments (new work, employment opportunities, promotion, commitment of time resources etc).

5.23 The promotion of programs and courses should feature any opportunities for RPL incorporated. In situations where training is voluntary, their existence will encourage participation. It should also highlight any flexibility in the delivery options, for the same reason.

5.24 The promotion of training programs/courses should be started sufficiently in advance of the first programmed activity to ensure that stakeholders can plan their involvement and are motivated towards its success.

Step 2: Training nominations and panelling 5.25

5.25 Defence exercises close control of the career development and training of members, aiming for ‘just-in-time’ training for specific job or career requirements. Posting action managed by career managers is used to achieve attendance at long courses, and direction by supervisors for attendance at shorter courses. The requirement for trained people is known, and the supply of individuals to courses is managed accordingly. The enrolment of trainees to course sessions is managed by the panelling authority in consultation with schools/units conducting the training in consultation with posting staff or supervisors, and the individuals affected.

Inputs Policies and instructions, course venue, curriculum, learning and assessment resources, required materiel and financial resources, training staff and nominated trainees.

Process Step 1: Promote the training.Step 2: Training nominations and panelling.Step 3: Prepare major resources.Step 4: Prepare materials and equipment.Step 5: Manage course sessions.

Products Prepared inputs to training, including but not limited to training schedule, training staff, designated trainees, facilities, materials and consumable resources.

5–6


5.26 In accordance with their employment conditions (eg plan on a page); the system for Defence civilian personnel is more flexible. Courses are promulgated and both individuals and their supervisors can arrange nominations.

5.27 The Defence personnel management system, Personnel Management Key Solutions (PMKeyS), has a course nomination/enrolment and panelling function that links with the course and course session set up data. The person operating the system must have successfully completed training in the Professional Development and Training (PD&T) functionality of PMKeyS. Joining instructions are sent to the trainees involved by the conducting authority.

Step 3: Prepare major resources 5.28

5.28 Preparing the major resources to conduct a course ensures they are available and ready. Resources that typically need preparation include:

a. facilities and infrastructure—such as schools, training rooms, workshops, accommodation and related buildings for traditional training; electronic networks and software that enables e-learning;

b. people—including staff for training management, administration and implementation (assessors, instructors, guest speakers/subject matter experts, mentors, workplace supervisors); and

c. major Defence assets—such as ships, aircraft and army units.

5.29 If the conduct of training follows on closely from a pilot course for which resources were prepared, preparatory work may be minimal. It can, however, involve:

a. securing the facilities and infrastructure;

b. acquiring and fitting furnishings and equipment;

c. ensuring the long-term availability of any required domestic and social accommodation;

d. advertising for and recruiting, and/or assigning staff for training, administration, the conduct of training, assessments and evaluation;

e. ensuring staff have prerequisite qualifications for their work (including the registration of RTO assessors);

5–7


f. coordinating the ongoing provision of resource inputs from other organisations (such as Services); and

g. budgeting and allocating the funds required for the training.

Step 4: Prepare materials and equipment 5.30

5.30 The materials and equipment used for the conduct of training and assessments and for implementing Training Review and Evaluation (TR&E) are specified in, and constitute part of the TMP.

5.31 Before training begins, the specified resources must be obtained in sufficient numbers, pre-positioned, have their readiness assured and, possibly, tested. This work should be completed in sufficient time to enable the uninterrupted conduct of training.

Step 5: Manage course sessions 5.32

5.32 Training cannot be efficient unless stakeholders know when it is taking place. Off-the-job training programs are normally planned annually, although such plans may be updated during their life span. Programs give the start and finish dates of scheduled courses, together with information about the number of trainees. They are used within schools and training units to develop detailed timetables which include, on a session-by-session basis, what is to be covered, the location for instruction and the trainer who will lead the activity.

5.33 Course programs are centrally managed in Defence using the PMKeyS database. The software facilitates the programming of training courses, and the nomination and confirmation of trainees for them. It can also be used to provide feedback on the availability of course places, and the demand for them. The set up of course sessions in PMKeyS involves entering information about the times, duration, numbers and panelling of trainees, locations and resource requirements. This work requires the person entering the data to have successfully completed training in the PD&T functionality of PMKeyS.

5.34 The management of course sessions should be undertaken sufficiently in advance of implementation to allow adequate preparation time and to identify any planning problems.

5.35 In many situations it is appropriate to program not just the implementation of training and assessment, but also the TR&E processes and any post-course administration to be applied.

5–8


IMPLEMENT PRE–TRAINING ASSESSMENTS/TESTS 5.36

Introduction 5.36

5.36 The associated TMP specifies whether pre-course testing and/or assessment are necessary or not.

5.37 Trigger. Pre-testing is triggered by a session being scheduled.


5.38 The inputs, process and products of each of the three processes, central to the conduct of training, are summarised in table 5–2.

Table 5–2: Inputs, process and products for implementing pre-testing/assessments

5.39 As is true throughout the DTM, although the steps of conducting training are, to a large extent, sequential, the associated work normally overlaps. The work is managed accordingly.

5.40 Most courses do not require the assessment of individuals before training processes commence. Those that do may do so:

a. for RPL purposes—so that unnecessary training doesn’t take place;

b. to check that individuals enrolled on the course meet prerequisite requirements; or

c. to enable training to be customised to learners’ needs.

Their application can contribute to the avoidance of inefficiencies in the conduct of training.

Inputs Trainees, assessment and training staff, venue, TMP, specified major resources, equipment, materials and finances.

Process Step 1: Assess/test trainees.Step 2: Evaluate the evidence.Step 3: Instigate appropriate action.

Products Qualified people, those who are ready to undertake training or unsuccessful applicants.

5–9


Step 1: Assess/test trainees 5.41

5.41 Defence conducts a wide range of assessment and testing activities, ranging between the resource intensive selection of trainees for flying aircraft and the self-administering of multiple choice questions at the start of a short computer-based training program. Guidance on what pre-course assessment should be applied, when and how; together with the tools to be used, is contained in TMPs.

5.42 Arrangements for the conduct of pre-course assessment are made in preparatory administration. Further preparation at the level of the individual managing the activity may also be necessary (by an appointed assessor in most cases, but by the trainee for self-administered assessment). This can include detailed scheduling, liaison between people involved, the preparation of assessment tools and ensuring the availability of required resources.

5.43 The conduct of any assessment/test should follow the guidance contained in the TMP. This guidance normally requires an assessment to have:

a. an introduction explaining:

(1) the purpose of the activity,

(2) the requirements of people involved,

(3) the time allowed,

(4) how performance will be assessed, and

(5) what feedback will be provided;

b. the activity itself, during which:

(1) the set procedures are followed;

(2) guidance on those procedures is available;

(3) as required, the performance of the trainee is monitored; and

(4) evidence of that performance is collected;

c. a debrief, during which:

(1) appropriate feedback is given, and

(2) the trainee is invited to comment on the process.

5–10


Step 2: Evaluate the evidence 5.44

5.44 In most circumstances the evidence arising from an assessment/test should be evaluated shortly after collection, so that feedback can be provided and appropriate action instigated while the outcomes are most valid. The evaluation is based on criteria provided in the TMP. It results in three broad categories of outcome:

a. trainees progress to course training (CT) (which may be tailored to meet individual needs);

b. trainees have prior learning recognised (and their training program is individualised accordingly); or

c. unsuccessful applicants.

Step 3: Instigate appropriate action 5.45

5.45 Appropriate administrative action is taken when the evidence has been evaluated. The action taken should:

a. be appropriate to the purpose of the assessment/test;

b. involve feedback to the individual involved; and

c. involve the recording of the evidence, its evaluation and the outcome.

CONDUCT TRAINING 5.46

Introduction 5.46

5.46 The conduct of training describes trainers’ interactions with trainees for the purpose of facilitating learning. Within the DTM it describes the implementation of the activities designed and developed in previous phases of the DTM.

5.47 The conduct of training can have many forms. It includes classroom instruction, practical exercises, technology-based training, individual and group project work, discussions, seminars, lectures, tutorials, simulations, fieldwork and much more. It may take place in dedicated training environments, in workplaces or elsewhere.

5.48 Purpose. The purpose of conducting training is to develop the required skills, knowledge and attitudes in people to undertake their roles to meet the Defence mission.

5.49 Trigger. Training is triggered by a session being scheduled and the completion of any pre-training assessments or tests.

5–11



5.50 The inputs, process and products of conducting training are summarised in table 5–3.

Table 5–3: Inputs, process and products of conducting training

5.51 Defence uses most types of training activity, including instruction (on- and off-the-job), distance learning, e-learning, formative assessment, coaching, practical work, demonstration and group/team work. This publication neither lists them all, nor explains the details of how they are implemented. Guidance on what methods should be applied in a program/course, when, and how, together with the necessary equipment and material, is contained in the TMP. Training personnel using a TMP should be appropriately trained and qualified.

5.52 The following steps provide broad guidance applicable to most training/learning situations.

Step 1: Prepare training activity 5.53

5.53 While arrangements for the implementation of training are administered suitably before a program/course begins, more detailed preparation by the trainer(s)/facilitator(s) is necessary before each instructional activity. This preparation may include detailed scheduling, liaison between personnel involved, the preparation of training/lesson plans, ensuring the availability and readiness of the venue and other resources, and rehearsing activities. Contingency planning should also be considered, so that problems such as equipment failure, non-appearance of trainers, program changes, and any need for remedial training are not unduly disruptive.

5.54 In most instructional situations structure is provided through a planned sequence of activity based on appropriate learning theories. The broad structure and sequence of programs/courses is provided in TMPs. At this stage of the ‘Conduct’ phase, the structuring of particular training activities such as lessons, lectures and presentations is undertaken, with the results recorded in lesson plans or lecture/presentation notes.

Inputs Venue; TMP; required facilities, training staff, trainees, materiel and financial resources and the results of pre-training assessments or tests.

Process Step 1: Prepare training activity.Step 2: Deliver training activity in accordance with the plan.Step 3: Monitor trainees’ progress and adjust training as required.

Products Trained people.

5–12


Step 2: Deliver training activity in accordance with the plan 5.55

5.55 The conduct of training describes the task of implementing what has been designed, developed and prepared in accordance with the TMP and the trainer’s detailed plans. Plans guide efficient and effective training and learning. The ‘human factor’ of training and learning (the ability to adapt to differing characteristics, personalities, learning styles and needs) involves skills and knowledge in:

a. communicating,

b. group facilitation,

c. questioning,

d. giving and receiving feedback, and

e. managing relationships.

5.56 Whatever the method and medium of training and learning employed, successful outcome-orientated training and learning requires trainers to:

a. address learner needs and expectations;

b. possess the skills to implement the method/mode of delivery;

c. apply principles of adult learning;

d. take advantage of learning conditions;

e. use exercises so that learners can apply what they learn;

f. anticipate and help overcome problems with transferring learning to the job;

g. motivate learners;

h. plan, conduct and review assessments;

i. build and sustain interest;

j. stimulate many senses to aid retention;

k. repeat key points to aid retention;

l. provide for as much learner participation as possible; and

m. explain why the information (or experience) is or will be useful to learners.

5–13


Step 3: Monitor trainees’ progress and adjust training as required 5.57

5.57 Good instruction/training incorporates the monitoring of trainee progress and adjusting the training to match identified needs. This type of formative assessment should be undertaken continuously by a trainer through informal questioning and observation. It is also conducted more formally, using progress tests and assessments specified in the TMP. In both circumstances progress tests/assessments are useful not only for measuring what has been learned but also to provide feedback. They also reinforce the learning process and provide information that can be used for continuous improvement.

CONDUCT ASSESSMENT 5.58

Introduction 5.58

5.58 Assessment in Defence is the process of collecting evidence and making judgments about trainees. Further, assessment is an integral part of training and focuses on the total proficiency of the person in the workplace. Assessment may take place in a training establishment or it may occur in the person’s workplace.

5.59 Assessment can be used before instruction commences in order to determine the attributes of trainees on entry to training. Pre-testing can be used to modify entry lessons and to implement recognition processes. Assessment can be used as an ongoing process to provide feedback to trainees and trainers on the effectiveness of the training and the achievement of learning outcomes/competencies.

5.60 Purpose. Within the DTM, the major purpose of assessment is to determine job competency and/or achievement of learning outcomes. Other purposes of assessment may include:

a. formative assessment during training,

b. summative assessment of performance at the completion of training,

c. statutory certification requirements,

d. diagnostic assessment to determine training needs,

e. personal feedback on performance,

f. skills audit, and

g. identification of suitability to perform particular tasks or roles for promotion.

5–14


5.61 Trigger. Triggers are completion of some training or an application for recognition.


5.62 The inputs, process and products of conducting assessment are summarised table 5–4.

Table 5–4: Inputs, process and products for conducting assessment

Step 1: Prepare summative assessments 5.63

5.63 Arrangements for the conduct of summative assessments should accord with the guidance provided in the TMP and any provisions required by the organisation (school, unit, RTO2 etc). Most situations are likely to involve:

a. scheduling the assessment;

b. informing the individuals involved of arrangements and what will be expected of them;

c. ensuring the availability and readiness of major resources, equipment, materials and assessment tools, and personnel;

d. preparing/reviewing assessment tools to be applied;

e. identifying and addressing equity and diversity issues; and

f. planning for contingencies, such as equipment failure.

5.64 Assessors must be qualified to assess. This may involve training and professional development of assessors.

Inputs Trainees, assessors, assessment guidelines and other assessment materials from the TMP, recognition evidence, venues and resources.

Process Step 1: Prepare summative assessments.Step 2: Conduct the assessment in accordance with the guidelines.Step 3: Evaluate conduct of assessment in accordance with evaluation plan/strategy.

Products Individuals recognised as job competent, course qualified or neither.

2 The Department of Defence RTO maintains an assessment kit for the guidance of its registered assessors. The kit is issued on successful completion of assessor training.

5–15


5.65 This planning applies to both programmed tests on course and to workplace assessment of competence.

Step 2: Conduct summative assessments 5.66

5.66 The conduct of summative assessments should follow the designed and developed guidance contained in the TMP and the administrative procedures of the organisation (school, unit, RTO etc) involved. Most situations are likely to involve:

a. an assessor briefing of the assessment process;

b. assessees use of assessment tools and materials to produce/collect the required evidence of performance;

c. the assessors and/or designated authority following established procedures to act on any suspicions of fraud, plagiarism or other inappropriate actions;

d. the receipt of evidence by assessors;

e. the assessor makes a judgment comparing the evidence to the standard;

f. opportunities for assessees to provide feedback on the process or appeal the assessment decision; and

g. record results.

Collect the evidence 5.67

5.67 Of primary concern in assessment is adherence to the principles and processes involved in the collection of evidence to support a decision about the competence of the person being assessed. In a competency-based assessment system, evidence is something that supports the person’s claim of having achieved competency. Evidence of competency can take many forms, such as direct evidence, primary evidence, secondary evidence, supplementary evidence, multiple sources of evidence. The forms of evidence required will depend on the purpose of the assessment. In every case, one source of evidence is probably insufficient, however, the more critical the assessment then the greater the need for more evidence and/or sources of evidence.

5.68 To gain RPL/recognition of current competencies, a range of evidence is usually used, including:

a. examples of previous work;

5–16


b. documentary evidence of previous achievements;

c. personal references from people familiar with the learner’s achievements; or

d. demonstration of performance by means of practical test.

Evaluate the evidence 5.69

5.69 In aggregating the evidence to determine/make judgments of the assessees competence/achievement of learning outcomes, two strategies are proposed depending on the type of evidence collected:

a. Objective—aggregation of information by assuming that someone is not yet competent until there is evidence of competence that accords with the required assessment criteria/marking guide.

b. Subjective—balancing the evidence for and against competence, ie would a reasonable person in the occupation be prepared to overlook failure on a number of elements or performance criteria in the standards (eg successful performance on a particularly difficult task and one which tends to discriminate accurately those who are competent from those who are not, might outweigh evidence of failure on a minor task of lower difficulty level).

5.70 Generally speaking, grading of learner performance does occur within competency based assessment systems. It is, however, a complex area with a number of issues and no singularly agreed approach. The grading of assessment in Vocational Education and Training is the practice of assessing and reporting levels of performance with the general purpose of recognising merit or excellence.

Confirm the results 5.71

5.71 Inform the relevant personnel (including assessee, administrator) of the outcome of the assessment.

Step 3: Evaluate assessment conduct methodology in accordance with evaluation plan/strategy 5.72

5.72 Assessment processes should be regularly evaluated/reviewed. The evaluation/review of the assessment processes is often most easily and cost-effectively conducted when the training itself is evaluated.

5–17


5.73 Australian Quality Training Framework (AQTF) audits have identified the need for accurate and complete assessment documentation. The assessment package is a critical component of the TMP. It provides the plan for the assessment as well as the documentation. Figure 5–2 summarises those questions that are critical to the evaluation of the assessment processes.

Figure 5–2: Critical questions in the review of assessment processes

5.74 Why assess? The important first step in any review of the assessment processes is to establish why assessment is being conducted. The answer to this question will provide the benchmark criteria that will impact on many of the other checks that will be carried out in the review.

5.75 What criteria? The second step is to confirm that assessment is valid against the objective (learning outcome and assessment criteria or competency units).

5–18


5.76 How to assess? The third step is to examine how the evidence for the assessment is being gathered. This will include the use of procedures and assessment and the way the evidence is documented.

5.77 What is enough? The final step in the evaluation/review process will be to check for validity, reliability, fairness and flexibility.

5.78 Identified changes. The assessment methods and tools are evaluated and feedback from trainees and assessors involved in administering the assessments are used to establish appropriate amendments to the assessment methods/tools/guidelines. Improvements and changes to the assessment tools/methods/guidelines are recommended/made, where necessary, and fed back to appropriate phases of the DTM.

Moderation 5.79

5.79 Moderation is the process whereby instructors/assessors discuss and reach agreement about assessments in a particular assessment system in order to achieve consistent assessment judgments. In so doing, a shared understanding develops between instructors/assessors about what is being assessed and why, and how different evidence is being interpreted. AQTF standards require moderation processes.

POST–COURSE ADMINISTRATION 5.80

Introduction 5.80

5.80 The purpose of post-course administration is to record and report the outcomes of training and assessment.

5.81 Trigger. Post-course administration is triggered by the completion of a training program/course and assessment.

5–19



5.82 The inputs, process and products of post-course administration is summarised in table 5–5 and explained in the following paragraphs.

Table 5–5: Inputs, process and products for post-course administration

Step 1: Record and report outcomes 5.83

5.83 The outcomes of training and assessment activities must be recorded to meet AQTF standards for storage, access and recording requirements. The prime database for Defence is PMKeyS, which should be used to record training results.

Step 2: Instigate training failure management 5.84

5.84 When individuals are assessed as not yet meeting a required assessment standard they may be required to attend remedial training or to repeat a program/course/unit. Such training should be organised and administered by the Conduct Authority. Other actions could include removal from the course or discharge from the Service, as administered by the appropriate authority.

Step 3: ‘Housekeeping’ 5.85

5.85 On completion of training and assessment, administrative action should be taken to clean, maintain and restock the resources as required.

Step 4: Issue course attendance certificates/records of attainment 5.86

5.86 Graduates of courses receive certificates or ROAs to recognise attendance and achievement of course outcomes. These are issued by the organisation conducting training.

5.87 If formal Australian Qualifications Framework qualifications, including Statements of Attainment, are an outcome of training and assessment, the relevant Defence RTO issues these under their scope of registration.

Inputs Implemented program/course, TMP, assessment results.Process Step 1: Record and report outcomes.

Step 2: Instigate trainee failure management, if required.Step 3: ‘housekeeping’ (restoring, restocking etc).Step 4: Issue course attendance certificate/record of attainment (ROA).

Products Issued course attendance certificate/ROA, CT and assessment records.

5–20


CONDUCT TRAINING REVIEW (INTERNAL EVALUATION)5.88

Introduction 5.88

5.88 Training review is a continuous improvement process of measuring and evaluating the conduct of training and assessment. Implicit in this is the evaluation of all, or a targeted selection of their inputs, processes and products. The term training review, in the context of the DTM, is also referred to as internal evaluation.

5.89 Purpose. The purpose of conducting a training review is to determine that the training outcomes have been achieved and to bring about incremental change. Conducting a training review will also determine that the training has been effective and efficient.

5.90 Trigger. Conducting a training review is triggered by the completion of a training program/course. Typically training reviews are conducted on a routine basis, however, a training review may be conducted in response to a particular concern, such as excessive failures in one assessment.


5.91 The inputs, process, and products of a training review are summarised in table 5–6.

Table 5–6: Inputs, process and products for conducting training review

5.92 A training review involves collecting and analysing information about training and assessment with a view to their improvement. Two types of information are generally sought, equating to the first two levels of Kirkpatrick’s model (reaction and learning)3. The first of Kirkpatrick’s levels, reaction, considers participants’ perceptions to the training. The second level, learning, describes the difference between what the participant knew and could do before and after the training. Collecting and analysing information of these types is used to improve the conduct of training.

Inputs Implemented program/course, TMP with training review materials, assessment results, and feedback from trainees and staff.

Process Step 1: Plan and prepare training reviews.Step 2: Collect and collate training review information.Step 3: Report training review.

Products Training review reports with recommendations.

3 Kirkpatrick’s model was described in chapter 3—‘The ‘Design’ phase’.

5–21


5.93 Strategies for reviewing training, together with the materials and instruments for conducting a training review, are contained in TMPs. Based on the strategy that:

a. an individual review is planned and prepared;

b. information is collected, collated and analysed; and

c. decisions for continuous improvement are made and reported.

5.94 A training review is usually conducted by staff from the Conduct Authority. A training review is likely to be more objective if undertaken by a section or individuals who are not associated with the conduct of training.

Step 1: Plan and prepare training reviews 5.95

5.95 The planning and preparation of training reviews should be based on the strategy presented in the TMP, the procedures of the organisation conducting the training and past experience of the course/program being considered.

5.96 The planning of a training review should address the following questions:

a. What aspects of the training need consideration?

b. What information/data will provide indications about these aspects?

c. When will this information/data be available?

d. What is the best method for gathering the information/data?

e. How will the information/data be processed?

f. What time is available for gathering and processing the information/data?

g. Who will be involved in the gathering and processing of the information/data?

h. Who must be informed of the process and its products?

5.97 The answers to these questions form the basis of a plan for a training review. Plans may vary in their rigidity/flexibility according to circumstance, noting that it is commonplace for them to be adapted when, during their implementation, circumstances suggest value in doing so.

5–22


5.98 Preparing for a training review may involve compiling documentation, and preparing the participants and any information/data collection instruments to be used. The participants commonly include trainees, trainers, assessors, and/or administration staff. Their cooperation must be obtained early, and a constructive and purposeful relationship maintained. To this end, it is important that they be kept fully informed of the process and their part in it. Collection instruments (eg data collection software, interview questionnaires and briefing scripts) are useful for gathering and recording information/data. All contribute to the consistency and validity of the process. They are, however, less adaptable to unusual situations wherein it may be preferable to employ a less restrictive approach to collecting information.

Step 2: Collect, collate and analyse training review information 5.99

5.99 The collection of information for training review can involve:

a. observing training and assessment activities;

b. reviewing training documentation;

c. applying questionnaires and surveys (printed or electronic);

d. reviewing training results and records; and

e. interviewing/surveying individuals and groups.

5.100 During the collection process, particularly if it is unstructured, care needs be taken to:

a. collect only useful information/data;

b. avoid unproductive ‘herrings’; and

c. be receptive to potentially useful material not previously considered.

5.101 The collation of data/information involves its arrangement into a format that helps its analysis: questionnaire returns need sorting, interview notes need transcribing etc. All pertinent information should be classified into areas that are meaningful to the purpose for which it was gathered.

5.102 Analysis of training review information involves closely examining the collated material so that conclusions can be drawn, areas and degrees of agreement or contrast identified, and; from these, facts, probabilities and possibilities derived. These are then considered to determine whether:

a. no further action is justified;

5–23


b. conclusions should be communicated to stakeholders (eg credit for good work or results); or

c. areas of potential improvement should be investigated and reported.

5.103 Where areas of potential for improvement are identified they are considered, together with possible solutions, in relation to the purpose(s) of the review, and decisions are made about possible course of action (use of a problem solving strategy is recommended). The areas most likely to be identified will concern the ‘Conduct’ phase of the DTM, but their causes might require amendments to the products of previous phases of the Model.

Step 3: Report training review 5.104

5.104 The final step of a training review is the reporting of its results. A training review report should identify the successful elements of the training as well as areas of potential improvement with recommendations how this should be achieved. The report should be sent to all stakeholders and, if necessary, followed up to ensure proper administration of its contents.

Annexes:A. Flow chart showing the ‘Conduct’ phase of the Defence training modelB. Defence training model—purpose, inputs, process and products of the

‘Conduct’ phase

5–24


FLOW CHART SHOWING THE ‘CONDUCT’ PHASE OF THE DEFENCE TRAINING MODEL A

5A–1



‘CONDUCT’ PHASE B

‘CO

ND

UC

T’ P

HA

SE

PUR

POSE

INPU

TSPR

OC

ESS

PRO

DU

CTS

To e

nsur

e th

e im

plem

enta

tion

of tr

aini

ng

and

asse

ssm

ent i

s ac

hiev

ed e

ffect

ivel

y an

d ef

ficie

ntly

to m

eet t

he

plan

ned

requ

irem

ent.

Polic

ies

and

inst

ruct

ions

, co

urse

ven

ue, c

urric

ulum

, le

arni

ng a

nd a

sses

smen

t re

sour

ces,

requ

ired

mat

erie

l an

d fin

anci

al re

sour

ces,

tra

inin

g st

aff,

and

nom

inat

ed

train

ees.

Adm

inis

ter t

rain

ing

prep

arat

ions

.Pr

epar

ed in

puts

to tr

aini

ng,

incl

udin

g bu

t not

lim

ited

to

train

ing

sche

dule

, tra

inin

g st

aff,

desi

gnat

ed tr

aine

es,

faci

litie

s, m

ater

ials

and

co

nsum

able

reso

urce

s.

To c

heck

read

ines

s of

tra

inee

s.P

rosp

ectiv

e tra

inee

s,

asse

ssm

ent s

taff,

maj

or

reso

urce

s, a

sses

smen

t gu

idel

ines

, ass

ocia

ted

mat

eria

ls a

nd e

quip

men

t.

Impl

emen

t pre

-trai

ning

as

sess

men

ts/te

sts.

Eith

er q

ualif

ied

peop

le,

cand

idat

es w

ith

prer

equi

site

s or

un

succ

essf

ul a

pplic

ants

.

To tr

ain

peop

le.

Venu

e, T

rain

ing

Man

agem

ent

Pack

age

(TM

P), r

equi

red

faci

litie

s, tr

aini

ng s

taff,

tra

inee

s, m

ater

ial a

nd

finan

cial

reso

urce

s.

Con

duct

trai

ning

.Tr

aine

d pe

ople

.

5B–1

ADFP 7.0.2

PUR

POSE

INPU

TSPR

OC

ESS

PRO

DU

CTS

To d

eter

min

e jo

b co

mpe

tenc

y an

d/or

ac

hiev

emen

t of l

earn

ing

outc

omes

.

Ass

esse

es a

nd a

sses

smen

t st

aff,

asse

ssm

ent g

uide

lines

an

d ot

her m

ater

ials

, ven

ue,

finan

ces.

Con

duct

ass

essm

ents

.S

ucce

ssfu

l cou

rse

grad

uate

s, a

war

d of

co

mpe

tenc

ies.

The

reco

rdin

g of

as

sess

men

t and

trai

ning

ou

tcom

es a

nd th

eir

cont

inuo

us im

prov

emen

t.

Impl

emen

ted

cour

se/p

rogr

am,

TMP

with

trai

ning

revi

ew

mat

eria

ls, a

sses

smen

t res

ults

, fe

edba

ck fr

om tr

aine

es a

nd

staf

f.

Con

duct

trai

ning

revi

ew

(inte

rnal

eva

luat

ion)

.Is

sued

qua

lific

atio

ns, c

ours

e tra

inin

g an

d as

sess

men

t re

cord

s an

d tra

inin

g re

view

re

ports

con

tain

ing

reco

mm

enda

tions

for

deve

lopm

ents

.

5B–2


CHAPTER 6

THE ‘EVALUATE’ PHASE 6

Introduction 6.1

6.1 The ‘Evaluate’ phase of the Defence training model (DTM) provides a measure of effectiveness of training. Evaluation measures how well training has prepared trainees for the workplace and whether the original workplace performance need has been met. The activities described in this phase are derived from level three and four of the Kirkpatrick model. This phase usually follows completion of training and assessment in the ‘Conduct’ phase as shown in figure 6–1, however, an evaluation may be initiated at any time in the DTM cycle if necessary.

Figure 6–1: ‘Evaluate’ phase of the Defence training model

6.2 Evaluation provides feedback across all phases and steps of the DTM and identifies under and over-training, the strengths and shortfalls of training and assessment, and recommends any needed improvements. These recommendations then become the inputs to the ‘Analyse’ and other phases of the DTM, and represent the start of a new cycle of development.

6.3 Purpose. The purpose of evaluation is to measure the effectiveness of training in meeting workplace performance requirements.

6–1


Key products and responsibilities 6.4

6.4 The key products of evaluation are reports that recognise that the training system is either functioning satisfactory or in need of change. Changes may be recommended to:

a. the related units of competency and/or Training Requirement Specification (TRS);

b. the training design of the program/course;

c. the major resources, materials and equipment used for training and assessment;

d. the conduct of training and assessment; and/or

e. a non-training contributing factor to workplace performance.

6.5 The nominated Evaluate Authority is responsible for ensuring that training is evaluated and reported. They are also responsible for maintaining communication channels for the collection of ongoing evaluation feedback, and for the systems that record and use this information.

6.6 Use of an external agency. In accordance with national and Defence guidelines, the quality assurance of all training by, or on behalf of Defence (by Defence staff, contractors, professional service providers, or other Registered Training Organisations) remains a Defence responsibility. Because of this, and the need to develop an understanding of the state of training over time, ongoing evaluation type work is not contracted to an external agency. An external agency may, however, be used to undertake a one-off, in-depth evaluation study.

Steps and products of the ‘Evaluate’ phase 6.7

6.7 The ‘Evaluate’ phase has four processes, or steps. They are illustrated in the flow diagram in annex A, together with their purposes, inputs and products, in annex B. The process steps are explained in paragraphs 6.9–6.29. Whilst presented as applicable to programmed, project-type studies, the process-steps are applicable to ongoing evaluation work.

6.8 As with all phases of the DTM, the application of the steps of the ‘Evaluate’ phase should be appropriate to the circumstances. In situations of high significance (eg training and assessment closely associated with combat capability) the application should be rigorous. Abbreviated processes may be appropriate in lesser circumstances.

6–2


Step 1: Plan and prepare the evaluation 6.9

6.9 The planning and preparation of evaluation studies should be based on the strategies contained in the Training Management Package, any Terms of Reference provided, and the normal practices of the organisations conducting the training and the evaluation. If the workplace/jobs being addressed have been the subject of previous evaluation work, records of that work should also be considered.

6.10 The planning of an evaluation study should determine:

a. the aim of the study;

b. the data/information required;

c. the sources of this data/information;

d. how and when the data/information will be gathered and processed;

e. what will be done with the results; and

f. in what time frame will the work be completed.

6.11 Evaluation study aim. The aim of an evaluation study should be precise and clear. The standard aim is ‘to check whether the training provided by the XXX program/course meets the requirements of the YYY job’. This aim does not direct attention to the original workplace performance need (for which factors other than training will be pertinent). If a study is to consider factors other than training, its aim should reflect the greater scope.

6.12 Data/information required. While any feedback on the performance of graduate trainees on-the-job is likely to be useful for evaluation, most studies seek answers to the following questions:

a. How much has workplace performance been enhanced by training?

b. What tasks are performed on-the-job for which no training has been provided?

c. What tasks have been trained for which are not required on-the-job?

d. Did the job holders perform at a standard below that required of the job?

6–3


e. What tasks have performance standards, as defined in learning objectives, which are inadequate for early, effective employment on-the-job?

f. What tasks have performance standards, as defined in learning objectives, which are higher than required on-the-job?

g. With what frequency are competencies trained for used on-the-job?

h. What procedures or drills trained on course vary with job practices (and what are the consequences of this)?

i. In what tasks do job holders experience significant difficulty in adapting from the training to the job situation?

j. What attitudes have job holders displayed to their job, category/employment group, and the Service/department?

6.13 Sources of evaluation data/information. Information and data that can be used for evaluation may be gained from any one or combination of the following sources:

a. Course graduates and job supervisors. When employed on-the-job, course graduates can provide feedback about the requirements of their work in relation to the training received and assessment experienced. Their supervisors are the authorities on-the-job requirements and the standard of work being completed. Combined, these two workplace sources are the most focused and reliable source of feedback. When using them, the time between training and the evaluation must be considered. Too soon and all parts of the job may not have been experienced. Too late, and graduates will have forgotten the details of the training they experienced.

b. New training staff and visitors from workplace. New staff members joining training organisations from the job environment are sources of evaluation feedback. So too are visitors from the workplace, including past graduates. Such feedback tends to be piecemeal, but can indicate areas of strength or weakness and, if collated over time can provide clear and detailed understanding.

6–4


c. Reports and records. Information and data can be extracted from reports and studies prepared for purposes other than evaluation. Operational and exercise reports, defect reports and safety records are examples. Occasionally too, job owners and supervisors communicate unsolicited evaluation feedback. These sources are usually of indicative value, and in need of support through focused information/data gathering.

d. Defence documents and communications. Defence instructions, publications, manuals, memorandums and the communications that promulgate amendments to them should be monitored for changes to policies, procedures, structures and organisation that may affect jobs and the training conducted for them. Such changes are frequent, and require incorporating in training documentation and courses conducted.

6.14 Gathering and processing data/information. The planning of evaluation work should incorporate consideration of how and when the data/information will be gathered, and how it will be processed. Data/information gathering methods include:

a. Workplace measurement. Potentially the most valid feedback gathering method, workplace measurement is time consuming.

b. Questionnaires (electronic or paper-based). The use of questionnaires is a quick (if electronic) and inexpensive method of collecting feedback, particularly from large numbers of dispersed people. The value of the data/information tends to reflect the effort and skill in preparing and administering the questionnaire. Even if well implemented though, a poor response in quantity and quality can result in unreliable feedback. Because of this, questionnaires are usually best used in conjunction with a workplace visit, when their administration and completion can be controlled.

c. Interviews/discussions (structured and unstructured).Interviews and discussions can be undertaken in the workplace environment. They can be inexpensive and easy methods of gathering current evaluation feedback—provided that a small number will suffice. Structured interviews can focus on specific issues, but can result in potentially useful feedback being missed. Less structured discussions can capture such feedback, but risk being so individualised and subjective. Feedback gained from interviews and discussions may need to be crosschecked.

6–5


d. Document and network research. Research of Defence documents, the Internet and communications is useful when the data/information is known to exist and a focused search for it is undertaken. Unstructured searches and scanning of sources can be unproductively time consuming and wasteful in bandwidth.

e. Combination of methods. The gathering of data/information for most evaluation studies involves a combination of the preceding methods, thereby providing means of crosschecking results.

6.15 As large quantities of data/information can be gathered, planning for evaluation work should consider how it will be collated and processed. This is especially true for the comparatively unstructured and possibly seemingly insignificant feedback gathered from ongoing evaluation work.

6.16 Managing the results. Evaluation work should be planned to accord with a concept of how the results will be managed towards achieving worthwhile outputs and outcomes. The concept should be shared and accepted by the main stakeholders in these outputs and outcomes—the authorities responsible for training. Involving the authorities or their representatives in the planning and preparation of evaluation work is therefore invaluable.

6.17 Time frame. The time frame planned for evaluation work will depend on the breadth and depth of investigations, the reliability of the data/information and resulting conclusions required, and any external deadlines.

6.18 An evaluation plan incorporating a schedule for completing the work is developed to accord with the factors outlined in paragraphs 6.9 to 6.19. Plans can vary in their rigidity/flexibility, and it is normal for them to be adapted when, during implementation, value in doing so is seen.

6.19 Preparing for evaluation work involves:

a. gaining the needed training documentation;

b. selecting and making ready the people involved (including the training and briefing of any evaluation staff);

c. arranging any workplace or other visits; and

d. developing any information/data collection instruments to be used (eg data collection software, interview questionnaires and briefing scripts).

6–6


6.20 The cooperation of people involved must be obtained early, and a constructive and purposeful relationship maintained. Keeping everyone involved in the evaluation fully informed of the process and their part in it is very important.

Step 2: Collect and collate evaluation information 6.21

6.21 The collection of information for evaluation purposes can involve:

a. reviewing course and training development documentation;

b. observing and measuring workplace performance;

c. applying questionnaires and surveys (printed or electronic);

d. reviewing training results and records; and

e. interviewing/surveying individuals and groups.

6.22 During the collection process, particularly if it is unstructured, care needs be taken to:

a. collect only useful information/data;

b. avoid unproductive ‘red herrings’; and

c. be receptive to potentially useful material not previously considered.

6.23 The collation of data/information involves its arrangement into a format that helps its analysis: questionnaire returns need sorting, interview notes need transcribing etc. All pertinent information should be sorted into areas that are meaningful to the purpose for which it was gathered.

Step 3: Analyse and synthesise evaluation information/data 6.24

6.24 Analysis and synthesis of evaluation information involves closely examining and combining the material gathered and collated to identify areas and degrees of agreement or disagreement. From these areas facts, probabilities, possibilities and conclusions relating to the aim of the evaluation are derived. The conclusions are the outputs of the work and are of three types:

a. areas of training that are satisfactory,

b. areas of training that are unsatisfactory and need improvement, and

c. conclusions that other factors affecting performance require attention.

6–7


6.25 As evaluation work centres on the appropriateness of training, the most common problem identified is a mismatch between required job performance and the TRS. Other problem areas can involve derived training documentation and the conduct of training.

6.26 Where areas of potential improvement are identified, courses of action for achieving the improvement should be identified and evaluated (problem solving strategies might be used). This analysis work equates to the ‘Needs Analysis’ work that begins the ‘Analyse’ phase of the DTM (and accounts for the DTM’s circular form). Stakeholders should be involved in the process, not least as this supports any change management required.

Step 4: Report evaluation 6.27

6.27 The final step of the ‘Evaluation’ phase is the reporting of results. An Evaluation Report should follow the presentation conventions of the Australian Defence Force Publication (ADFP) 04.1.2—Defence Writing Standards (currently ADFP 102). It should recognise areas of training that are successful, affirm those that are satisfactory, identify areas where improvements are desirable and recommend how those improvements should be achieved. In some circumstances it might also be appropriate to recommend further investigation, particularly where non-training issues are involved.

6.28 As with most DTM work, stakeholders should be involved in the preparation of an Evaluation Report such that its contents are no surprise, but more a summary and formal statement of issues and actions that are understood and accepted.

6.29 The Evaluation Authority sends Evaluation Reports to all stakeholders, especially those responsible for implementing the recommendations. The report is also filed for use in any follow-up or new evaluation work.

Annexes:A. Flow chart showing the ‘Evaluate’ phase of the Defence training model

B. Defence training model—purpose, inputs, process and products of the ‘Evaluate’ phase

6–8


FLOW CHART SHOWING THE ‘EVALUATE’ PHASE OF THE DEFENCE TRAINING MODEL A

6A–1



‘EVALUATE’ PHASE B

‘EVA

LUA

TE’ P

HA

SE

PUR

POSE

INPU

TS

PRO

CES

S-ST

EPPR

OD

UC

TS

To e

nsur

e ef

fect

iven

ess

and

effic

ienc

y in

the

eval

uatio

n pr

oces

s an

d in

form

sta

keho

lder

s, g

ain

com

mitm

ent.

Eval

uatio

n st

rate

gy, t

erm

s of

re

fere

nce,

pre

viou

s Ev

alua

tion

Rep

orts

, w

orkp

lace

, rep

orts

of

wor

kpla

ce p

erfo

rman

ce

eval

uatio

n m

ater

ials

, Cou

rse

Rep

orts

, sta

keho

lder

trai

ning

an

d as

sess

men

t fee

dbac

k,

occu

patio

nal a

naly

sis

repo

rts,

job

spec

ifica

tion,

Tra

inin

g R

equi

rem

ent S

peci

ficat

ion,

cu

rricu

lum

, res

ourc

es

(incl

udin

g fin

anci

al).

1. P

lan

and

prep

are

eval

uatio

n.Ev

alua

tion

plan

and

pr

ogra

m, a

ppro

vals

for

wor

kpla

ce a

cces

s,

info

rmat

ion

colle

ctio

n m

ater

ials

, tra

ined

/pre

pare

d/pr

ofic

ient

ev

alua

tors

.

To g

athe

r use

able

dat

a an

d in

form

atio

n.

Det

aile

d pl

an fo

r eva

luat

ion,

in

form

atio

n co

llect

ion

mat

eria

ls, j

ob h

olde

rs, t

heir

supe

rvis

ors,

and

oth

er

stak

ehol

ders

.

2. C

olle

ct a

nd c

olla

te

eval

uatio

n in

form

atio

n/da

ta.

Eva

luat

ion

data

and

in

form

atio

n.

6B–1

ADFP 7.0.2

PUR

POSE

INPU

TS

PRO

CES

S-ST

EPPR

OD

UC

TS

To g

ener

ate

appr

opria

te

conc

lusi

ons

and

poss

ible

im

prov

emen

t stra

tegi

es.

Eval

uatio

n da

ta a

nd

info

rmat

ion,

sub

ject

mat

ter

expe

rts, i

nfor

mat

ion

on

non-

train

ing

fact

ors

affe

ctin

g pe

rform

ance

.

3. A

naly

se a

nd s

ynth

esis

e ev

alua

tion

info

rmat

ion/

data

.C

oncl

usio

ns a

bout

the

effe

ctiv

enes

s of

trai

ning

and

op

portu

nitie

s fo

r im

prov

emen

t.

To d

eter

min

e w

heth

er th

e pe

rform

ance

nee

d ha

s be

en m

et a

nd g

ain

endo

rsem

ent/c

omm

itmen

t to

impr

ovem

ent s

trate

gies

.

Eval

uatio

n co

nclu

sion

s,

oppo

rtuni

ties

for

impr

ovem

ent.

4. R

epor

t the

eva

luat

ion.

Eval

uatio

n R

epor

t with

re

com

men

datio

ns.

6B–2

ADFP 7.0.2

GLOSSARY

Unless stated otherwise, Australian Defence Force (ADF) terms and definitions are used within this publication. Externally sourced terms and definitions, approved for ADF use, are designated in brackets following the definition, using the following legend:

accredited courseA structured sequence of vocational education and training (VET) that leads to an Australian Qualifications Framework (AQF) qualification or Statement of Attainment. Accredited courses are listed on the National Training Information Service (NTIS) see http://www.ntis.gov.au. (DEST)

‘Analyse’ phase of the Defence training modelThe ‘Analyse’ phase of the Defence training model (DTM) is when the need for training (new or revised) to improve human performance is analysed and specified. The two main products of the ‘Analyse’ phase are job specifications that describe jobs and Training Requirement Specification (TRS) that specify Defence training requirements.

assessmentThe process of collecting evidence and making judgments about something.

assessment criteriaThe lowest of three levels of learning outcome from a training process. It contains a performance, the conditions under which that performance is undertaken and the standard required for successful achievement. An assessment criterion should be accompanied by the method to be used for evaluating achievement.

assessment guidelinesAn endorsed component of a training package that underpin assessment and which set out the industry approach to valid, reliable, flexible and fair assessment. Assessment guidelines include the assessment system overview, assessor requirements, designing assessment resources, conducting assessment and the sources of information on assessment. (DEST)

DEST Department of Education, Science and Training (previously known as the Australian National Training Authority (ANTA)

1



ADFP 7.0.2

assessment materialsAn optional component of training packages that complement endorsed industry assessment guidelines and could take the form of assessment exemplars or specific assessment tasks and instructions. (DEST)

assessment strategyA broad methodology for using assessment to support training and, finally, determining whether the required performance has been achieved. An assessment strategy for training should detail the program, its activities, their purpose and likely resource requirements.

Australian Qualifications Framework (AQF) A nationally consistent set of qualifications for all post-compulsory education and training in Australia. (DEST)

Australian Quality Training Framework (AQTF) The nationally agreed recognition arrangements for the VET sector. The AQTF is based on a quality assured approach to the registration of training organisations seeking to deliver training, assess competency outcomes and issue AQF qualifications and/or Statements of Attainment (SOA). AQTF ensures the recognition of training providers and the AQF qualifications and SOA they issue, across Australia. (DEST)

Authorities (Analysis, Design, Development, Conduct and Evaluation) In relation to the DTM are, in accordance with Defence Instruction (General) PERS 05–36—Implementation of the Defence Training Model in Defence Vocational Education and Training, responsible for managing key training processes (including their quality assurance and continuous improvement), and for approving and maintaining the products of their respective phase of the model.

competencyThe ability to perform tasks and duties to the standard expected. (also competence)

competency analysis The process of identifying the knowledge, skills (mental and physical) and attitudes/attributes that combine to enable mastery of a competency, together with any significant factor that can influence its training. The results are recorded on a Competency Analysis Sheet.

2

ADFP 7.0.2

competency assessmentDetermines whether competency has been achieved: whether an individual can perform something to the standard expected in the workplace, as expressed in the relevant endorsed industry/enterprise competency standards or the learning outcomes of an accredited course. (DEST)

competency assessment strategy Describes the formal assessment strategy for assessing workplace competency. It states what is to be assessed, where and when, and who is to gather and make judgments of the evidence. It is written during the ‘Design’ phase of the DTM.

competency based training and assessment (CBTA) An approach to learning which places primary emphasis on what the learner can do. It is focussed on outcomes rather than on learning processes or time spent engaged in those processes. It is concerned with the attainment and demonstration through application in the workplace of knowledge and skills to a specified level of competency. CBTA is concerned with the flexible use of endorsed competency standards in the workplace. It provides for national recognition of achievement and allows skills and knowledge to be transferable and portable under the National Training Framework (NTF). (DEST)

competency profile A document describing the competencies of an individual or group. The competencies listed should be either nationally endorsed or locally identified competency standards or units.

competency standardAn industry-determined specification of performance which sets out the skills, knowledge and attitudes required to operate effectively in employment. Competency standards are made up of units of competency, which are themselves made up of elements of competency, together with performance criteria, a range of variables, and an evidence guide. Competency standards are an endorsed component of a training package. (DEST)

‘Conduct’ phase of the Defence training modelThe phase of the DTM when training and associated processes are implemented.

3

ADFP 7.0.2

continuous improvementA planned process, which allows an organisation to systematically review and improve the quality of its products, services and associated processes. The continuous improvement processes of the DTM include training review, evaluation, and the formative evaluation of work in each of its phases. (DEST)

courseA structured and integrated program of education or training, usually consisting of a number of modules (subjects) or shorter programs, and leading to the award of a qualification. (DEST)

course learning outcome (CLO)The highest of three levels of learning outcome from a training process. It contains a performance, the conditions under which that performance is undertaken, and the standard required for successful achievement. A course learning outcome should be accompanied by the assessment method to be used for evaluating that success.

course learning outcome levels Provide a shorthand method of indicating the meaning of course learning outcomes to stakeholders, particularly the employers of recent training graduates. Four levels are used:a. trained, but not against a prescribed standard (background or

familiarisation training);b. partially trained against prescribed standard (additional training

and/or experience required before assessment, employment under supervision only);

c. fully trained and ready for assessment against prescribed standard (assessment required, employment under supervision until assessed); and

d. trained and assessed competent against the prescribed standard (ready for employment).

course map A diagram showing the major modules and sessions of a course or program of training, together with alternative paths for its undertaking. Its use enables flexibility in scheduling and programming without detriment to the achievement of learning objectives.

4

ADFP 7.0.2

curriculumThe specifications for a course or subject (module) which describe all the learning experiences a student undergoes, generally including objectives, content, intended learning outcomes, teaching methodology, recommended or prescribed assessment tasks, assessment exemplars etc. In the DTM it is based on training and assessment strategies designed for specific learning outcomes. (DEST)

Defence assessment kit A set of policies and procedures under which Defence competency assessments are conducted (the Defence Services Learning Network Recognition Kit, version 5).

Defence training model (DTM) The name of the model linking the processes of the systems approach to training used in Defence.

Department of Education Science and Training (DEST)A Commonwealth statutory authority providing a national focus for vocational education and training.

‘Design’ phase of the Defence training modelWhen the assessment and learning strategies for training are selected, together with the strategies for evaluating the whole process of preparing for and then conducting training.

‘Develop’ phase of the Defence training modelWhen the resources needed to conduct, assess and evaluate training are produced and/or prepared.

diagnostic assessmentDescribes assessment that helps learners and their supervisors determine the learner’s education, training and development needs. Recognition of prior learning (RPL) is a form of diagnostic assessment. If recognition of current competency leads to the award of a unit(s) of competency, then it is summative assessment.

education1. The process of imparting knowledge or developing skills, understanding, attitudes, character or behaviours. 2. The process of acquiring knowledge, skills, understanding, attitudes, etc.3. The knowledge, skills, understanding, attitudes, etc acquired.4. The field of study concerned with teaching and learning. (DEST)

5

ADFP 7.0.2

element of competencyA basic building block of a unit of competency which describe the key activities or elements of the work covered by the unit. (DEST)

‘Evaluate’ phase of the Defence training model When feedback on how well training has prepared learners for the workplace, and whether the original capability requirement has been met, is gathered and processed.

evaluation 1. The process or results of systematic collection and analysis of data or information to make an assessment or appraisal in relation to stated objectives, standards or criteria in education or training which may be applied to policies, organisations, organisational functions, programs, projects, courses or other activities. 2. In the context of the DTM:a. Evaluation is the process of checking how well training has

prepared learners for the workplace and whether the original capability requirement has been met.

b. Formative evaluation is the process that provides feedback on the progress of work contributing to achieving something.

c. Summative evaluation is the process conducted on completion of work to determine how well it achieved its purpose.

evaluation strategy A guide to how evaluation against the Training Requirement Statement is to be organised in workplaces. It should include information on the methods to employ; exemplar materials; and who is responsible for data collection, data analysis and completing the Evaluation Report.

evidence guide The part of a competency standard which provides a guide to the interpretation and assessment of the unit of competency, including the aspects which need to be emphasised in assessment, relationships to other units, and the required evidence of competency. (DEST)

feasibility analysis The process used to determine the best option, or mix of options, for ensuring the competence of jobholders. The results are presented in a Feasibility Report, which, if training is a solution, includes a Training Requirement Statement.

6

ADFP 7.0.2

formative assessmentDescribes assessment to assist and support a process (eg learning) during the undertaking of that process (eg by advising the learner about the quality of performance and progress towards the attainment of a competency).

job analysis The process used to determine/confirm the workplace performance requirement, current or foreseen, for a given employment classification (task, job, career or work group). The results are recorded on a job specification.

job specification The Defence document that records the product of a job analysis. It classifies a job and details its composition and characteristics.

key competenciesThe generic, underpinning, transferable skills or competencies essential for effective participation in work. They apply to work generally, rather than being specific to work in a particular occupation or industry. There are seven key competencies: collecting, analysing and organising information; communicating ideas and information; planning and organising activities; working with others and in teams; using mathematical ideas and techniques; solving problems and using technology. They have three performance levels:

Level 1—undertakes tasks effectively using routine approaches;Level 2—manages tasks selecting routine approaches; andLevel 3—evaluates and reshapes tasks to establish new approaches.

Note

Training packages developed from 2006 will include employability skills

learner analysis The process used to determine the characteristics and qualities of the prospective group of learners. The output is a learner profile, critical to which is a list of the typical learner’s competencies.

learner profile A document that is developed on the basis of a Target Population Profile. It describes the existing competencies and capabilities of prospective learners, their preferred learning style, special aptitudes and motivation. This information helps ensure the suitability of the assessment and learning strategies.

7

ADFP 7.0.2

learning analysis The process of identifying the desired outcomes of the training and learning process: the learning outcomes.

learning environment analysisA study of the things or conditions that could surround training and which should therefore influence the design and development of strategies for its assessment and conduct. Recorded in a learning environment profile, it is used to determine the assessment and learning strategies for competencies and training respectively.

learning outcomes Describe the intended results of training. They clearly and concisely describe what a learner will be able to do on satisfactorily completing a unit of training. The DTM uses two types of outcome: CLO and module learning outcome (MLO). Learning outcomes are the building blocks of training and learning courses. They are written with three parts: the performance (using an action verb), the conditions (such as use of tools, references or other support) and standards (defining the criteria of success, including any time limitations). Each learning outcome should be accompanied by the method to be used for evaluating achievement.

learning strategyA non-endorsed component of a training package which provides information on how training programs may be organised in workplaces and training institutions. This may include information on learning pathways, model training programs and training materials. (DEST)

module learning outcome (MLO)The middle of three levels of learning outcome from a training process. It contains a performance, the conditions under which that performance is undertaken and the standard required for successful achievement. An MLO should be accompanied by the assessment method to be used for evaluating that success.

National Training Framework (NTF)The system of VET that applies nationally. It is made up of the AQTF and nationally endorsed training packages. (DEST)

National Training Information Service (NTIS)Developed by DEST to provide access to current and emerging training market information and products in vocational education and training. The NTIS webpage is at http://www.ntis.gov.au/. It comprises a database of:a. VET accredited courses,

8

http://www.ntis.gov.au/

ADFP 7.0.2

b. competency standards,c. training packages, andd. training providers and text information on a range of

complementary issues.

needs assessment A precursor to the ‘Analyse’ phase of the DTM. It is the product of an analysis to establish whether applying the model is likely to achieve a worthwhile improvement in workplace performance.

performance criteriaThe part of a competency standard specifying the required level of performance in terms of a set of outcomes which need to be achieved in order to be deemed competent. (DEST)

performance gap The difference between the performance required for a job and the capabilities of the target population: the people likely to perform it. The gap has two dimensions: scope and scale. The scope of the ‘gap’ describes the work that the target population cannot presently perform. The scale describes the significance of this problem.

proficiencies A term defined by Defence Personnel Management Key Solutions (PMKeyS) as training, education or experiential outcome achieved by an individual which is essential to satisfy a specified workplace requirement associated with an established Defence position. There are five types of proficiencies in PMKeyS: certificate, licence, experience, stream and skill. Proficiencies are required in PMKeyS as they link training regimes to individual profiles and positions. Proficiencies are awarded to a trainee automatically upon the successful completion of a course session or attached to a member’s profile manually as required. A certificate of proficiency usually consists of a set of related competency units and/or elements.

range statement The part of a competency standard which specifies the range of contexts and conditions to which the performance criteria apply. (also known as range of variables) (DEST)

recognition of current competenceThe acknowledgment of competencies currently held by a person, acquired through training, work or life experience. More commonly known as RPL. (DEST)

9

ADFP 7.0.2

recognition of prior learning (RPL)The acknowledgment of a person’s skills and knowledge acquired through previous training, work or life experience, which may be used to grant status or credit in a subject or module. (DEST)

record of attainment (ROA)Defence certification issued to a student for partial completion of a qualification.

Registered Training Organisation (RTO)A title for an organisation registered by a State or Territory recognition authority to deliver training and/or conduct assessments and issue nationally recognised qualifications in accordance with the AQTF. (DEST)

skills, knowledge and attitude (SKA) analysis Describes the identification of the skills, knowledge and attitudes that together enable someone to consciously do something.

statement of attainment (SOA)Certification issued to a student for partial completion of a qualification, including, where relevant, the units of competency achieved under nationally endorsed standards. Achievements recognised by statements of attainment can accumulate towards a qualification within the AQF. (DEST)

Statement of Performance Need A document that records the performance gap: ie the difference between the competencies and other proficiencies required for workplace performance, and those the target population already has.

summative assessmentDescribes assessment on completion of a process. In training it can be used to determine both achievement of a training program/course, and whether a unit of competency has been achieved. RPL is a form of summative assessment.

Target Population ProfileA document describing the significant characteristics of the target population for training. It may incorporate the numbers in the group, age, gender, education and experience levels, probable turnover rate, and the current competencies they can be expected to have.

10

ADFP 7.0.2

to pilot training Refers to testing newly developed training and subjecting it to critical training review. The purpose of piloting training is to identify aspects of its design and development that can be improved.

trainingDescribes the development of skills, knowledge, attitudes, competencies etc, through instruction or practice. Within this instruction the term ‘training’ is used as a contraction of vocational education and training, the term more commonly used in the context of Australia’s NTF. Types of training within Defence include:a. collective, or team training: the training of one or more

groups of people, as when ‘working-up’ an ADF unit for operational readiness;

b. formal training: training conducted in an organised manner, usually towards defined outcomes such as qualifications;

c. individual training: the training of individuals;d. joint training: training in which elements of more than one

Service participate; ande. on-the-job training: formal training that occurs at a learner’s

workplace. (DEST)

training designThe process that produces the curriculum that specifies how training is to be conducted.

training environment profile A description of the situation(s) in which training is to be conducted. It is likely to include location, learning facilities and equipment, staffing, availability of external training policy and other constraints. First prepared for the ‘Analyse’ phase of the DTM, a more detailed version is used in the ‘Design’ phase.

Training Management Package (TMP)Presents the complete set of documentation necessary for managing the conduct of a training program or course. It contains the TRS, the curriculum, the training review and evaluation strategies, and the materials developed to conduct the course/program.

11

ADFP 7.0.2

training packageDescribes an integrated set of nationally endorsed standards, guidelines and qualifications for training, assessing and recognising people’s skills, developed by industry to meet the training needs of an industry or group of industries. Training packages consist of core endorsed components of competency standards, assessment guidelines and qualifications, and optional non-endorsed components of support materials such as learning strategies, assessment resources and professional development materials. (DEST)

Training Requirement Specification (TRS)The formal document that specifies the Defence training requirement in terms of the competencies to be trained, the need for people to be trained in them (how many and in what time frame), the broad type of training that is to be employed and how that training will be recorded.

training review The continuous improvement process of evaluating and reporting the ‘Conduct’ phase of the DTM.

training strategy A broad methodology for the conduct and management of training. A training strategy should provide guidance on the methods, location, activities, media, recommended or proscribed tasks, formative assessment, and likely resource requirements that training will involve.

unit of competencyA component of a competency standard. A unit of competency is a statement of a key function or role in a particular job or occupation. See also element of competency, performance criteria and range of variables. (DEST)

vocational education and training (VET)Encompasses post-compulsory education and training, excluding degree and higher-level programs delivered by higher education institutions, which provides people with occupational or work-related knowledge and skills. VET also includes programs that provide the basis for subsequent vocational programs. Alternative terms used internationally include technical and VET, vocational and technical education and training, technical and vocational education, vocational and technical education, and further education and training. (DEST)

12

ADFP 7.0.2

ACRONYMS AND ABBREVIATIONSADDP Australian Defence Doctrine PublicationADF Australian Defence ForceAQF Australian Qualifications FrameworkAQTF Australian Quality Training FrameworkARF Australian Recognition FrameworkASLAV Australian Light Armoured Vehicle

BPO Business Process Owner

CAS Competency Analysis SheetsCBTA competency based training and assessmentCLO course learning outcomeCT course training

DEST Department of Education, Science and Technology

DETC Defence Education and Training CommitteeDI(G) Defence Instruction (General)DPE Defence Personnel ExecutiveDTI duty task inventoryDTM Defence training model

F2F face to face

ISC Industry Skills CouncilITAB industry training advisory bodies

JMAP joint military appreciation process

LSA logistic support analysis

MJT manager of joint trainingMLO module learning outcome

NTF National Training FrameworkNTIS National Training Information Service

OHS occupational health and safetyOJT on-the-job training

PD&T professional development and trainingPMKeyS Personnel Management Key SolutionsPO Policy Owner

RDP rapid development processROA record of attainmentRPL recognition of prior learningRTO Registered Training Organisation

13

ADFP 7.0.2

SME subject matter expertSOA statement of attainment

TMP Training Management PackageTOR Terms of ReferenceTR&E Training Review and EvaluationTRS Training Requirement Specification

14

Documents

DOCTRINE AND TRAINING SERIES ADFP 7.0.2 THE DEFENCE ... · Management responsibilities 1.17 THE ACCELERATED APPLICATION OF THE DEFENCE TRAINING MODEL Risk management 1.26 A RAPID