Cfites Vol 5 e

DGPA Creative Services 2001CS-0103A-P9-050-000/PT-0052001-04

F O R E W O R D

A-P9-050-000/PT-005, Manual of Individual Training and Education, Volume 5, Development of Instructional Programmes is issued on authority of the Chief of Defence Staff.

This publication is effective on receipt.

Suggestions for changes shall be forwarded through normal channels to National Defence Headquarters, Attention: Director, Training and Education Policy (DTEP).

I

III

TPART 4 PRODUCE INSTRUCTIONAL MATERIAL

Overview . . . . . . . . . . . . . . . . . . . . . . . . .12Develop Print-based Material . . . . . . . . . .12Produce Visuals . . . . . . . . . . . . . . . . . . . .13Produce Conventional Audio-Visual Material . . . . . . . . . . . . . . . .14Produce Technology-based Material . . . . .15Develop Assessment Instruments . . . . . . .16Develop the Course Timetable . . . . . . . . .17Characteristics . . . . . . . . . . . . . . . . . . . . .17Consider Scheduling Factors . . . . . . . . . .18Scheduling . . . . . . . . . . . . . . . . . . . . . . . .20Instructional Week . . . . . . . . . . . . . . . . . .20Instructional Media . . . . . . . . . . . . . . . . . .20Evening Instruction . . . . . . . . . . . . . . . . . .20Operational Conditions . . . . . . . . . . . . . . .21Homework . . . . . . . . . . . . . . . . . . . . . . . .22Develop Lesson Plans . . . . . . . . . . . . . . . .22

PART 5 CONDUCT TRIALS AND REVISE MATERIALSPurpose . . . . . . . . . . . . . . . . . . . . . . . . . . .24Timing . . . . . . . . . . . . . . . . . . . . . . . . . . . .24Trial Material . . . . . . . . . . . . . . . . . . . . . .24Consider Partial and Full Trials . . . . . . . .25Pilot Course . . . . . . . . . . . . . . . . . . . . . . .25Revise Materials . . . . . . . . . . . . . . . . . . . .26Consider Retrials and Instruction Life Cycle Revision Rates . . . . . . . . . . . . .27

PART 6 PREPARE STAFFOverview . . . . . . . . . . . . . . . . . . . . . . . . .28Prepare Standards and Support Staff . . . .28Schedule Training for Instructional Staff . . . . . . . . . . . . . . . . . .28Review Training Documentation . . . . . . . .28Trial Equipment and Training Aids . . . . .29Rehearse Lessons . . . . . . . . . . . . . . . . . . .29

T A B L E O F C O N T E N T S

PART 1 INTRODUCTIONBackground . . . . . . . . . . . . . . . . . . . . . . . . .1Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . .1Overview of CFITES . . . . . . . . . . . . . . . . . .1Purpose of Development Phase . . . . . . . . . .2Overview of Development Phase . . . . . . . .2

PARTt 2 IDENTIFY REQUIREMENTS AND DETERMINE SOURCES OF MATERIAL Overview . . . . . . . . . . . . . . . . . . . . . . . . . .3Review Content, Instructional Strategy, and Population . . . . . . . . . . . . . . .3Review Requirements for On-the-Job Training . . . . . . . . . . . . . . . . . .5Unit Funded On-the-Job Training . . . . . . . .5Managing Authority Funded On-the-Job Training . . . . . . . . . . . . . . . . . .5Assess Development Options . . . . . . . . . . .5Determine Resource Requirements . . . . . . .7

PART 3 PROCURE INSTRUCTIONAL MATERIAL Overview . . . . . . . . . . . . . . . . . . . . . . . . . .8Estimate Development Time . . . . . . . . . . . .8Other Government Departments or Militaries . . . . . . . . . . . . . . . . . . . . . . . .8Contracting . . . . . . . . . . . . . . . . . . . . . . . . .8Contract for Services . . . . . . . . . . . . . . . . . .9Contract for Goods . . . . . . . . . . . . . . . . . . .9Ensure Materials Meet Copyright andIntellectual Property Right Laws . . . . . . . .10Translate Materials . . . . . . . . . . . . . . . . . .10Selecting Translation Sources . . . . . . . . . .10

TT A B L E O F C O N T E N T S ( C O N T ’ D )

PART 7 RECORD DEVELOPMENT COSTSPurpose . . . . . . . . . . . . . . . . . . . . . . . . . .30Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . .30Create Checklists . . . . . . . . . . . . . . . . . . . .30Record Cost Data . . . . . . . . . . . . . . . . . . . .30Determine Costs . . . . . . . . . . . . . . . . . . . .31Compare Forecast to Actual Costs . . . . . .31

ANNEX A: REFERENCES . . . . . . . . . . . . . . . . . . . . .A1

ANNEX B: GLOSSARY . . . . . . . . . . . . . . . . . . . . . . .B1

ANNEX C: DEVELOPMENT TIME . . . . . . . . . . . . . . .C1

ANNEX D: INSTRUCTOR TO LEARNER RATIOS . .D1

ANNEX E: LESSON PLANS . . . . . . . . . . . . . . . . . . . .E1

ANNEX F: LEARNING SUPPORT . . . . . . . . . . . . . . .F1

ANNEX G: DEVELOPING QUESTIONS . . . . . . . . . . .G1

ANNEX H: PROGRAMME DESIGN AND EVALUATION RATES FOR IT&E ESTABLISHMENT STANDARDS . . . . . . .H1

IV

M A N U A L O F I N D I V I D U A L T R A I N I N G A N D E D U C A T I O N — V O L U M E 5

P A R T 1

1

Background

Purpose

Overview of CFITES

II N T R O D U C T I O N

1. Canadian Forces policy on Individual Training and Education (IT&E) statesthat IT&E activities shall be conducted in accordance with the managementmodel known as Canadian Forces Individual Training and Education System(CFITES). The Manual of Individual Training and Education provides guid-ance on the application of the CFITES in a series of interrelated volumes,each focusing on a different aspect of the system.

2. This volume of the Manual of Individual Training and Educationprovides guidance to CF staff or external developers (the term “developers” is used throughout the following text) on the development of instructionalprogrammes. The extent to which these guidelines are followed will varywith respect to the size and nature of the particular instructional develop-ment context.

3. For an introduction to and description of the CFITES see Volume 1, CFITESIntroduction/Description. As shown in Figure 1 below, Development is thethird phase of the six-phase CFITES Quality Control model.

Figure 1: CFITES Quality Control Model


P A R T 1

2

4. The purpose of the Development phase is to provide effective instructionalmaterials that will preserve the design intent and prepare for the delivery of the instructional programme.

5. The input to the Development phase is the lesson guidance, often referred to as a training plan, that describes the learning programme and environmentthat enables learners to achieve performance objectives. The output of theDevelopment phase will be the instructional materials required to delivertraining, prepared staff and a record of the costs involved.

6. The fundamental processes of the Development phase, which are described in Parts 3 -7 of this volume, are as follows:

a. procure instructional material;

b. produce instructional material;

c. conduct trials and revise materials;

d. prepare staff; and

e. record development costs.

It should be noted that the above processes are not sequential and may be carried out concurrently where appropriate.

Purpose ofDevelopment Phase

Overview ofDevelopment Phase


P A R T 2

3

I I D E N T I F Y R E Q U I R E M E N T SA N D D E T E R M I N E S O U R C E SO F M A T E R I A L

7. Performance-oriented instruction requires carefully developed materials tosupport learning. Instructional material consists of all types of material thatsupport learning such as training aids, equipment, interactive courseware,lesson plans, testing materials and references.

8. In some cases, appropriate materials, or even complete instructional programmes, may be available for procurement on an “off-the-shelf” basis.More often, significant amounts of in-house effort will be required to preparefor the conduct of IT&E. In any case, developers must first determine whatmaterials are needed to develop the instructional programme.

9. Providing effective instructional material must begin with a thorough review of the results of the Design phase. The lesson guidance, which will bereferred to as a training plan, will specify the course content, the instruction-al strategy, estimated costs, the assessment plan, resource requirements andlesson requirements. Developers shall also review the proceedings of theQualifications Standards Board and associated references where available, to gain insight that may assist the development process. For a complete guideon the development of training plans and what to expect as output from theDesign phase, see Volume 4, Design of Instructional Programmes.

10. The training plan and the lesson specifications it contains serve as the basis for the development of an instructional programme. Factors that mustbe considered when reviewing the training plan and planning for productionand procurement of material are shown in Table 1.

Overview

Review Content,Instructional Strategyand Population


P A R T 2

4

Table 1: Review Factors in Developing Instructional Material

Quantity of equipment and supplies required for instruction.

Number of locations requiring equipment and supplies.

Operational equipment and supplies that are currentlyavailable (simulators, cutaways, stands).

Anticipated costs.

Sources and amount of funding available.

Incorporating costs into Business Plan.

Lead-time required for producing or acquiring items.

Time available to develop material.

Modifications or building a new facility and associated construction lead times.

Design factors to meet functions/tasks that the facilitymust support (power, drainage, ventilation, lightingetceteras). Utilization rate/cost justification.

Security considerations.

Availability of supplies and equipment to support learning, the achievement of performanceobjectives and the instructional equipment.

Costs

Timeline

Instructional facility requirements.

Operation and support considerations. Preventative and corrective maintenance to supportthe facility and equipment (how often, how long,how much).

Projected modifications/upgrades to equipment,hardware, or software (how often, how long, how much).

Instructional equipment and supplies are in line with operation equipment and supplies.

Overall instructional strategy for the programme. Occupation or specialty tasks selected for instruction,scalars, objectives and lesson specifications.

Appropriateness of instructional method, media, andenvironment for each objective, the population, theirdistribution and the available resources.

FACTOR SCOPE


P A R T 2

5

11. OJT may be selected as the instructional strategy for one or more performance objectives (POs) during the Design phase. The EnvironmentalCommand/Group Principal and the units concerned must agree to host theOJT during the Design phase. In this case, learners must achieve those objec-tives through actual job performance at a unit. During OJT, supervisors or skilled workers assist learners, monitor their progress and verify that the objectives have been met.

12. It is essential that OJT be identified as either a Unit or MA funded activity. OJTis Unit funded if it is conducted as a part of actual operations. In this case, thelearner performs the actual job under the supervision of a skilled worker whowill verify that the required standard of performance for the objective has been met. A learner may be required to perfect skills that have been learned on course with reference to job aids. Time, effort and resources to support theOJT, outside of or beyond what the job requires, should be minimal.

13. OJT is MA funded if it is conducted in the job environment but requires the unit to stage work or learning opportunities outside of or beyond actualoperations (such as providing formal lessons, or stripping and re-building an engine for training purposes only). In this case, there is a training cost intime, effort and resources that are diverted from unit operations. Therefore,the MA funds the resources required to conduct such OJT.

14. In either case, the MA and training establishment that delivers the instruc-tional programme are responsible to provide units with the instructions,checks, records and contacts associated with conducting OJT. Any instruc-tional materials required for OJT should be produced and trialed during the Development phase along with all other materials required for the instructional programme.

15. Once the instructional material required to support a particular IT&E activityis determined, developers can assess the development options available to them. The pros and cons of various options for the development of instructional material are provided in Table 2.

16. The lesson specifications should detail the instructional materials required ona “per-learner” basis. Developers must use this information to select a devel-opment option and provide sufficient quantities of instructional materials foreach course serial. The number of learners, the number of locations and thetimings of serials (concurrent, overlapping, one shortly after another) requir-ing these materials must be considered when determining the quantities needed.

MA Funded OJT

Unit Funded OJT

AssessDevelopmentOptions

Review Requirementsfor On-the-JobTraining (OJT)


P A R T 2

6

Table 2: Development Options

Use when environment exists or it is costeffective (per student) to establish.

Use when:

CF expertise is not available.

Development environment or infrastructure in the CF does not exist.

Development level of effort (hours)exceeds CF staff capacity.

Cost to establish and maintain.

Expertise and capability maybe short lived, depending onCF requirements.

Developer must have, or develop, expertise in projectdefinition and planning and in contracting processes suchas Request for Proposal (RFP)Request for Standing Offer(RFSO).

Less control of processesand/or outcome.

Long lead times may berequired, considering times forcontracting process, approvalturn-around and shipping.

Develop CFresources,expertise andcapability.

Well-definedcosts as long as requirement is establishedand stable.

Internal

CommercialContract

Use when each development componentcan be clearly defined and assigned toparticipants.

Use when the material most nearly meetsCF requirements with no, or minimal,modification.

Requires much planning, coordination and management.

High risks unless requirementsand responsibilities are clearlydefined.

Content may not be fully effective without modification.

May be legally or technicallydifficult to modify.

May not have vendor technicalsupport.

Makes efficientuse of resources.

In some caseslow start-upcosts.

Combination

CommercialPurchase orLease

OPTION CONS DECISION FACTORSPROS


P A R T 2

7

17. Instructional materials may include other resources, such as loaned unitequipment and stores. All resources fall into three general classes:

a. Consumables. Most consumable material required is identified in thetraining plan. However, some consumables may not be identified directly.For example, additional consumable material and supplies such as testsheets or booklets may be required to meet the needs of the assessmentplan. The quantities of each consumable item required should be trackedto ensure that each instructor and learner receives the appropriate allot-ment for a given course serial. Consumable materials need to be replacedfor each course serial. In the case of ammunition, POL and other hazardous materials, proper storage must be available;

b. Non-consumables. The lesson specifications should define any non-consumable material such as vehicles, tanks, or aircraft to name a few that must be available for the delivery of instruction. Non-consumablematerials may be reused from serial to serial. Developers must ensure thatsufficient serviceable, non-consumable items are available to cover courserequirements; and

c. Facilities. Facilities, such as classrooms, workshops, ranges et cetera,required for instruction should also be identified in the lesson specifi-cation under Resources Required. Facility requirements become more of an issue when learning environments, through distributed instruction,include learning centres, video teleconference sites, or workplace computerstations versus the traditional classroom. It should be noted that additionalpersonnel may be required to support facilities by assisting in the instruc-tion and/or operation of classrooms, theatres, system/equipment specificrooms and other equipment needed by learners. The use of an aircraftflight simulator for training is an example of this situation.

18. Developers must plan for the time, material and support personnel needed to ensure that facilities and non-consumables are available to meet the require-ments for the duration of each course serial. Additional resource requirementsmay be identified during lesson trials and rehearsals. A list of newly identifiedresources should be fed back to the Design staff to ensure that appropriate corrections can be made to the lesson specifications.

Determine ResourceRequirements


P A R T 3

8

PP R O C U R E I N S T R U C T I O N A LM A T E R I A L

19. Instructional material can be procured from a variety of sources. Pre-developedmaterials can be purchased “off-the-shelf” or the CF can procure instructionalmaterials as part of the purchase of new equipment. The CF can also contractfor the development of materials to meet a specific training requirement.

20. Due to the variety of IT&E activities required to meet CF requirements and thewide range of related instructional materials available to support these activi-ties, development time and effort can vary extensively. It is important to havean estimate of how long it will take to develop materials in order to ensurethey will be available for the first scheduled serial and to anticipate costs ifmaterials are to be procured externally. To calculate an estimate of timingsrefer to the general guidelines on the average level of effort (hours) to develop products detailed at Annex C.

21. Instructional material for the operation or maintenance of equipment, such as shipboard auxiliary machinery or airfield infrastructure, may be availablefrom other government departments either on loan or through purchase.Materials may require some modification in order to meet CF requirements.

22. Table 3 lists the two contracting authorities that DND and the CF uses to procure goods or services for contracts valued over $5000.00 as of the date of this publication. Director Contracting Policy (DC Pol) can authorize anyindividual, who has the funds, to enter into and sign a service contract val-ued over $5,000.00. The unit has authority to procure goods and services of $5000.00 or less. Personnel should refer to the departmental documentDelegation of Authorities for Financial Administration for the Department of National Defence and the Canadian Forces for further information on delegated authorities.

Table 3: Contracting Authorities

Overview

Estimate Development Time

Other GovernmentDepartments orMilitaries

Contracting

Public Works and Government ServicesDirector Contracting Policy

FOR SERVICES OVER $5000.00 FOR GOODS OVER $5000.00


P A R T 3

9

Contract For Goods

23. The Department of National Defence (DND) contracting policy must be fol-lowed when procuring services for the development of instructional material.This is essential to avoid problems during the procurement process and pos-sible future problems with respect to contract management and justificationfor letting the contract to the selected source. The DND standard procedure,as of the date of this publication, calls for the preparation of a Request forProposal (RFP) supported by a Statement of Work (SOW) through DC Pol for contracts valued over $25,000.00.

24. For complete details on preparing documents used in contracting refer to Volume 10, Managing Individual Training and Education in Projects.Particular attention must be paid to Annexes C, D and E. By tailoring theseAnnexes, the developer can make the SOW sufficiently detailed and preciseto ensure delivery of effective material for the course concerned.

25. If Standing Offers are sought a different set of rules apply. The use ofStanding Offers implies that there is an ongoing requirement to let a series of small contracts for the same requirement. A Request for Standing Offer(RFSO), with a Statement of Work (SOW) and other supporting documenta-tion, will normally be processed through a designated contracting authorityor DC Pol. The result may be a Standing Offer with one or more contractors,who can then be tasked on a rotational basis. A dollar limit is placed on anyone call-up. The dollar limit will vary depending on the Standing Offer.

Note: Standing Offers are not contracts. A Standing Offer is simply an offer from a potential supplier to provide specific goods or services under pre-defined terms and conditions. A contract does not exist until a call-up is made against the Standing Offer.

26. Usually, for RFPs and RFSOs for goods (instructional materials) PublicWorks, Government Services Canada (PWGSC) must act as an agent for the CF as DC Pol and DND contracting authorities can only process servicecontracts. In this case, DND can procure goods through PWGSC who will distribute the contracting documents and receive the responses from the bidders. PWGSC will also act as go between or conduit for queries frompotential bidders and the related responses from the CF.

Note: Personnel should refer to Defence Administrative Order and Directive(DAOD) 3004, Contracting, for further guidance on contracting.

Contract For Services


P A R T 3

10

27. The federal government has a CANCOPY license that authorizes the copyingof up to 10 percent of a published work for the purpose of conducting gov-ernment business and delivering government programmes and services. For example, one can copy an entire newspaper article, a short story or articlefrom a periodical issue or a chapter of a book. Copies, however, cannot bepermanently bound into anthologies. Personnel should refer to the Licence forthe Reprographic Reproduction of Published Works Subject to Copyright,Treasury Board Secretariat (May 1994, Amended 1998) for further information.

28. Developers must be constantly aware of their responsibilities with respect toincorporating copyrighted or proprietary intellectual property into CF materi-al. This is particularly true of material obtained from non-paper sources.Paper sources generally have copyright or proprietary claims bound into thedocument, whereas non-paper sources generally have this data recorded insuch a manner that it becomes separated from the main material, or is other-wise not readily evident. Developers must ensure that written authority fromthe copyright or proprietary rights claimant is received and filed at the origi-nating unit before any such material is converted or incorporated into CFmaterial. This requirement is different from “per seat” licensing agreementsthat may be needed if, for example, software is to be used as-is over a periodof time or on a repeated basis.

29. Translation of instructional material will be required in accordance with the Official Languages Act. The CF's goal is to conduct IT&E in the preferredofficial language (OL) of the learner. Personnel should refer to CanadianForces Administrative Order (CFAO) 9-53, Language of Instruction, for additional information.

30. Units are now responsible for managing OL translation within their respectiveorganizations using their own operating budgets. Therefore, units have theoption to purchase translation services internally or externally.

a. Internal. Services can be purchased from the PWGSC Translation Bureau. A Memorandum of Understanding (MOU) between the TranslationBureau and the CF/DND has been put in place for the provision of services.The Translation Bureau should be contacted in advance to determine howmuch time would be required to translate instructional material. If thetranslated material cannot be provided to meet deadlines, developers maydecide that a commercial contract should be explored as a more expedientmeans of obtaining translated work.

Translate Materials

SelectingTranslationSources

Ensure Materials Meet Copyright andIntellectual PropertyRight Laws


P A R T 3

11

b. External. Other than direct purchase of translated material from the produc-er of the instructional material, contracting from commercial sources is theonly other source for translated work. Developers must assure themselvesthat the companies considered for the work have the technical expertise andthe second-language skills to effect an accurate translation for the intendedaudience. When services are provided through contracting, the contractormust ensure the bilingual documentation's accuracy with an attestation thatthe Translation Accuracy Check (TAC) was done. A TAC is the means bywhich a content specialist ensures that the translation perfectly mirrors the original text in its content as well as its grammar.

31. All translations must be consistent with approved DND and CF terminology.Approved terminology sources include Defence Terminology ManagementSystem (DTMS) terminology bank, Concise Oxford Dictionary (for English),Petit Robert (for French) and TERMIUM (PWGSC Translation BureauLinguistic Data Bank). DAOD 5039-4, Translation of Texts and Acquisition of Bilingual Documentation, should be referred to for further information.

Note: Translation of materials should not be carried out until the product is relatively stable (i.e. after a pilot course, if possible). Once course materials are translated, the second language version of the course should then be piloted.


P A R T 4

12

Overview

Develop Print-basedMaterial

PP R O D U C E I N S T R U C T I O N A LM A T E R I A L

32. Instructional material consists of all types of material that support learning.Instructional materials can range in complexity from a simple handout tovery complex simulators. In-house development is appropriate when theequipment to produce materials is available. In-house production allows for the development of internal expertise and complete control of a project.Often a combination of in-house development with external support may be required.

33. These instructional materials can include reading materials, exercises, handouts, worksheets or books to name a few. Some points to consider when developing print-based materials include:

a. Font. Use select fonts that have large, well-spaced letters such as the Times New Roman style with a type size of 10 to 12 points. No more than two different fonts per page should be used;

b. Text. Only use all capitals in headings as it slows the reader down.Highlight text by using bold or italics;

c. Bullets. Add bullets to draw attention to ideas in text;

d. Icons. Icons are small pictures used to highlight important points or indicate activities to be performed;

e. Blank space. Leave ample space between sections of material and in the margins. This is easier to read than a page full of text; and

f. Content. In longer text, comparisons, examples and drawings should be included in order to reach all learners.

The text in Figure 2 illustrates the use of some of the points above.


P A R T 4

13

Note: Some interactivity can be incorporated into a workbook. For example, students can be directed to start certain activities, skip ahead if material isalready known, or choose what area to work in next.

34. Visuals support verbal presentations and enhance learning considerably asindividuals learn and retain over 50 percent of what they both see and hear.Some types of visuals and their associated development considerationsinclude:

a. Realistic Images and Objects. The real object is often the most effectivevisual aid because it replicates the job. To effectively use the real object itshould be safe, big enough for everyone to see or small enough and avail-able so that each person can have one. Realistic images refer to qualityphotographs of real objects. Realistic objects and images are most effectivewhen they are used after a new idea or concept has been introduced.

b. Graphics. Graphics include charts, graphs and maps. These types of graphics can effectively illustrate relationships between points or concepts.Diagrams, illustrations, drawings, cartoons and other pictorial material arealso graphics. They can convey a concept and catch a person's attention tostimulate thinking and learning. Some points to consider when developinga graphic include:

1. present one idea or concept;

2. avoid excessive details;

3. highlight important elements with brightest and lightest colours; and

4. ensure lettering and elements contrast with background.

Produce Visuals

STUDY ASSIGNMENT 1

Instructions:

1. Read the case study provided.

2. Write your response to the questions on page 8 in report form.

3. Once you have completed your report the instructor will provide you with a sample solution. Compare your solution to the sample. If you would like additional practice complete Study Assignment 1A.

∆ Remember to refer to the course-writing guide when writing your report.

Take a break…and proceed to the CBT Session 2.

Figure 2: Study Assignment


P A R T 4

14

ProduceConventionalAudio-VisualMaterial

c. Projected Visuals. Overhead transparencies or slides can be easily devel-oped by hand or by using computer software. Guidelines for developingprojected visuals include:

1. illustrate key points;

2. use key words, as headers, to facilitate retention of each point;

3. include a maximum of six words per line and six lines on each slide; and

4. lettering should be large enough to be clearly read from the back of the classroom.

Figure 3 provides an example (reduced in size) of an effective projected visual.

LEARNING SUPPORT

• Comparisons

• Reasons

• Examples

• Statistics

• Testimony

d. Mock-ups. These materials can provide three-dimensional representationof a concept or context such as an airfield or battleground when it isrequired for learning. In-house mock-ups can be constructed with the creative use of paint and construction materials. In addition, miniaturemodel tanks, planes, vehicles and other props can be purchased “off-the-shelf” as needed.

35. Films, audiotapes, slides and videos can be developed with the assistance oflocal Graphics Artists or in-house with the proper equipment, training andpractice. Referring to resource materials and attending courses that provideinstruction are recommended depending on the complexity of the project.Presentation software, such as PowerPoint, often includes tutorials and tips on developing effective presentations. The following points should be considered:

a. the audio portion of these materials must be clear and easily discerned by the listener; and

b. graphics for both audio-visual and presentation software, similar to projected visuals, should be kept simple, clearly visible and with a minimum amount of words per slide.

Figure 3: Sample Projected Visual


P A R T 4

15

36. Technology-based instructional materials such as computer-based training(CBT) and multimedia must be comprehensive in their design. It is importantfor developers to try to anticipate possible learner actions in order to makethe material as effective as possible. Generally, these types of interactivecourseware should provide a learning situation that cannot be provided inprint or as easily in the classroom. For example, when multimedia is used asa linear page-turner it may not provide the learner with anything more thancan be provided in a high quality text with graphics. On the other hand, multimedia that allows the learner to role-play in difficult or dangerous situations and as often as they wish is significantly enhancing the learningprocess. While role-play is effective with real people in the classroom theremay be limitations on how many roles, practice opportunities and the typesof situations that can be provided. Therefore, in this example, the interactivecourseware is effectively enhancing learning.

37. The developmental resources and expertise required for technology-basedinstruction will be substantially greater than those needed for basic classroominstruction. If in-house expertise is limited professional assistance may berequired. Generally, the more complex the learning requirement, the moreeffort and detail will be required for the Development phase of the systemsapproach to training. Effective technology-based instructional material hasbeen shown to have several key attributes such as:

a. materials focus on meeting the needs of the learner and provide the content that must be learned;

b. materials require that the learner interact in a meaningful way with course content;

c. materials are relevant to the real world;

d. materials permit learner control (choice of activities or sequence of completion); and

e. materials are easily accessible and readily available when needed.

Whether this material is partially developed by industry or not, a great deal of in-house work must be initially completed to accurately define what content and learning experience the material should provide to meet learners needs.

Note: The benefits and concerns associated with using various instructionalmedia are detailed in Annex E of Volume 4, Design of Instructional Programmes.

ProduceTechnology-basedMaterial


P A R T 4

16

38. Assessment is carried out to determine if learning is taking place. Assessment occurs when a measurement is compared with a standard, and a judgment is made which is based on that comparison. The Design phaseassessment plan specifies how achievement of the PO will be assessed andhow learner progress through enabling objectives (EOs) will be assessed. In some instances there will be provisions for cases where direct observationmay be limited or not possible such as for safety reasons.

39 Assessment usually involves two types of tests as follows:

a. Practical. In a practical test, the learner carries out a required action andperformance is measured against a standard using a checklist or ratingscale. To ensure validity these tests are job oriented and as realistic as possible. Refer to Volume 7, Part 3, Constructing Practical Tests for additional information; and

b. Theory. In a theory test, the learner describes, in writing or orally, theknowledge required to apply job skills. Theory tests are comprised of multiple choice; matching; true-or-false; fill-in-blanks and essay type questions. Theory tests require careful construction, scoring, recording and analysis of results. Refer to Volume 7, Part 4, Constructing TheoryTests for more information.

40. Testing occurs during or at the end of an instructional module. These two cases consist of:

a. Formative testing. These enabling checks help the instructional staff andlearners measure progress for confirmation of the learning process, or to recognize areas that require remedial measures. Formative testing is con-ducted throughout the programme and reinforces learning by strengtheningretention of the subject matter. The scores of formative testing are used onlyfor these purposes, and are not summed, averaged, nor used as a factor insummative testing; and

b. Summative testing. These performance checks determine learner achieve-ment of the POs and/or critical EOs that are prerequisites to further IT&E.Standards staff must determine the content to be assessed and design sum-mative tests. Whenever possible, standards staff will administer and scorethe summative tests. Instructional staff may assist if required. Summativetesting is applied at the end of an instructional phase/module.

Develop AssessmentInstruments


P A R T 4

17

41. Developers shall ensure that tests and testing material are developed to meet the requirements of the training and assessment plan. Tests are pilotedto ensure that they produce consistent results and are valid. Standards staff,subject matter experts and instructional staff will be involved in the develop-ment, piloting and evaluation of tests. The developer is also responsible forreporting the results of evaluation inconsistencies to the Design staff.

Note: Training and assessment should always be performance oriented.

42. The sequence of instruction is important to the success of any instructionalprogramme. Instructional activities should be scheduled to promote the bestpossible learning conditions.

43. Time estimates to conduct various parts of a lesson can be found in the lesson specification. In the example in Annex I of Volume 4, Design ofInstructional Programmes, time is allotted to each teaching point for instruc-tor presentation or demonstration and learner performance or practice of theapplicable action. The total time required indicates the sum of the instruc-tion, practice and testing time for the lesson. It is important to note the timeallotted for testing because it includes time for activities such as debriefing by an instructor or peers, on a task. For example, debriefing a learner on acommand appointment on a leadership course. Remedial training arisingfrom these activities could use up as much as 10% of the allotted test time. In addition, the aggregate of time estimates for individual phases/moduleswithin a course are valuable in establishing the course timetable.

44. A well-planned instructional programme has the following characteristics:

a. Progression. The programme must bring the course member to therequired standard through a logical sequence of instruction;

b. Variety. Wherever possible without being at cross-purposes, POs and EOsshould be presented in a variety of sequences to foster learner interest andalertness. Different methods of instruction can also be employed to offsetboredom and fatigue. A description of instructional methods is presentedin Annex C of Volume 4, Design of Instructional Programmes;

c. Tempo. The tempo of instruction should build through periods of intenseactivity and conclude with a final period of intense activity at the end ofthe course. Throughout the course, periods of intense activity should befollowed by periods of relative relaxation;

d. Efficiency. An efficient programme is one in which the time required forpurposes other than pure instruction has been correctly appreciated andallotted. For example, if deemed necessary, several non-training periodsper week can be allocated for Physical Training (PT), and an hour a weekcan be allocated for personal administration. Non-training periods do notcount as training days; and

Characteristics

Develop the Course Timetable


P A R T 4

18

e. Flexibility. The provision of spare periods for revision necessitated by unforeseen circumstances is a requirement. Without this reserve a programme is bound to run into difficulty. The addition of one spare training day for every thirty scheduled training days is reasonable.

45. Some factors to be considered by developers when sequencing instructionalactivities include:

a. Learner characteristics. The current skills, knowledge, attitudes and characteristics of the learners may affect their response to instructionalmodules or to how these are presented in relation to one another;

b. Structure and Choice. The learning order or structure of activities fallsinto three main categories, as follows:

1. Move from familiar to unfamiliar subject matter;

2. Move in logical order or natural progression through the subject matter; and

3. Move in job performance order, with task elements presented as theyoccur on the job.

c. Inhibition. Learning two or more similar subjects at the same time causesinhibition. For example, studying two languages at the same time maycause confusion due to their similarities. Inhibition occurs when the similarity between the two subjects causes the learning of one subject to inhibit or confound the other;

d. Reinforcement. Sometimes the presentation of two or more aspects of thesame subject causes reinforcement. For example, if learners studied theprinciples of leadership and military history together, the principles ofleadership would be confirmed or reinforced. The military history wouldprovide examples of effective or poor leadership and allow learners to putthe concepts learned about leadership into context. The goal is to reinforcethe learning process whenever possible;

e. Saturation. A point of saturation is reached when the rate of learning isfast, and the process, continuous. The point of saturation is attained morequickly when the material to be learned consists of series of similar facts.Facts taught too quickly and continuously for course members to absorbwill not be retained;

f. Fatigue and Effectiveness. In the daily life of an individual there is a nat-ural rhythm of rising soon after dawn and retiring sometime after sunset.This rhythm, called the circadian rhythm, is one of rising and falling energy levels and wakefulness depending on the time of day or night. The effect of this rhythm on learner fatigue is illustrated in Figure 4. This rhythm is extremely difficult to break. Even when a learner has been awake all night, there is a rush of energy just as dawn breaks. The pattern of the rhythm varies with individual temperaments, with habit and with climate, but it affects most learners so strongly that, in normal conditions, the following general rules should be applied:

Consider Scheduling Factors


P A R T 4

19

1. Schedule the hardest work between breakfast and lunch.

2. Schedule the most interesting work later on in the day.

3. Schedule short training exercises at night shortly after dusk.

4. Schedule night exercises designed for testing between midnight and dawn.

5. Do not schedule any training in the hours before dawn.

Figure 4: Daily Work Curve

g. Enabling and Performance Checks. Enabling checks must lead up to eachPerformance Check. Performance Checks should be spaced out over thecourse of a programme;

h. Environmental conditions. Consideration must also be given to excessiveexposure to climatic or environmental (natural and man-made) situationsthat are detrimental to learning. The exception to this is when trainingand/or assessment must be conducted in the conditions stated in the PO,such as at night, that reflect the job requirements;

i. Administration. Administrative activities must be scheduled to ensurethey complement the training activities. For example, supplies and storesmust be ordered and available at the time they are needed. This extends to ensuring staff are available to issue stores when required and to receivereturned stores at the end of the applicable training session or module;

j. Maintenance. System or training material maintenance should be sched-uled to occur between course serials. This may mean that scheduled andpreventive maintenance activities must be adjusted to accommodate thetraining requirement. Developers must carry out the administration toarrange for these changes when and as they are permitted; and

k. Schedule factors. Limiting factors, such as safety, and learning conditionswill generally be defined in the lesson specifications. These must beadhered to regardless of their impact on the sequence of instruction.However, new factors may arise during trials and rehearsals. Any adjust-ments that affect the training schedule but do not impact the learning orassessment processes should be deferred so that they occur between course serials. These new factors must be reported to the Design staff.

46. The developer must evaluate the requirements and time estimates in thetraining plan in conjunction with the factors addressed above. The aim is to structure the timetable to achieve the intent of the training plan, takefull advantage of the instructional staff and training material, and to meetthe needs of the learners. For on-site courses, a standard training day maybe built in 40 or 50-minute periods with 5 to 10 minute breaks betweenperiods. The actual daily instructional time may vary between 300 and 400minutes. Distance learning programmes should follow the same guidelinesfor periods and breaks, but due to the nature of the medium, restrain theactual daily instructional time to no more than 160 to 200 minutes.

47. The normal instructional week is five days, Monday to Friday. Resourceand facility usage may require working longer weeks, or for five days overa normal holiday. In extraordinary circumstances (crisis, mobilization, orsimilar urgent operational situations), it may be necessary to have a six-dayinstructional week. It is not recommended to exceed this rate, even for aten-day course. Learning, remembering and instruction are more effectiveif learners are not fatigued.

48. Excessive time in a day or a week with respect to particular training mediaor conditions that may have a mental or physical impact on the learningprocess should be monitored. Guidelines on the maximum amount of timea particular media/method should be used in a typical 6 to 8 hour trainingday and 5-day week are provided in Table 4.

Note: Assessment using these media should follow the same limits, unless otherwise specified in the conditions and standards of thePerformance Objective.

49. Generally, instruction whether classroom, self-study or distance learningshould be scheduled during normal working hours. The exception to this is when training and/or assessment must be conducted in the conditions stated in the PO, such as at night, that reflect the job requirements. Ifevening instruction is required for instructional activities, such as night firing, the schedule should allow time for dinner and administration andend in time to permit learners to get a minimum of six hours sleep after-wards. Evening instructional time should not exceed 160 to 200 minutes.Evening instructional time should be used sparingly to maintain thequality of learning.

20

Evening Instruction

Instructional Week

Instructional Media


P A R T 4

Scheduling


P A R T 4

21

50. When a PO requires a learner to perform for prolonged periods, it is necessaryto train and assess for that operational requirement. During the instructionalphase it is best to first teach and allow learners to practice while they arewell rested (allowing for a minimum of 6 hours undisturbed sleep) and thencontinue to practice under more demanding conditions. Assessment shouldmimic operational conditions as far as it is safe and practical to do so. Whilea course may operate on a 24 hour basis for days or weeks at a time, it is nec-essary for safety and instructional purposes that staff and learners obtain aminimum amount of sleep (varies from two to four hours) every 24 hours.

Operational Conditions

6 hours (using a variety of instructional strategies and activities)

2 hours

2 hours with scheduled breaks (Two 2 hour periods may be scheduled ifeach period is followed by another type of learning activity and media)

Instructor (classroom)

Instructor (shop, field, parade square)

Linear, computer based text,and videotape

4 hours (interspersed with other activities after one hour)

4 hours (interspersed with other activities for a minimum of 30 minutesafter each

Printed Text (paper-based)

4 hours: Given that materials are self-paced and interactive (include multiplebranching, graphics and sound relevant to task). Learners must be able totake unscheduled breaks as needed. The more interactive the materials, thelonger learners may be able to focus on course content.

Learner-centered technology:Computer Based Training(CBT), Multimedia and WebBased Training (WBT)

4 hours (depends upon simulated mission and lesson plan)Station Simulator (pilot, tank,games, surveillance operation)

Facilitated or group paced tech-nology: CBT, WBT, Multimedia,Audio, Video and ComputerConference

MEDIA/METHOD MAXIMUM PER TRAINING DAY

Table 4: Maximums by Instructional Media Per Day


P A R T 4

22

51. Self and group studies are not normally included in course schedules. After hours study and project work should not exceed two hours per evening or eight hours per weekend.

52. The instructors who will teach the programme should develop the lessonplans based on the teaching points and instructional strategy provided in the lesson specifications of the training plan.

53. The instructor must initially examine the lesson specification in order to groupteaching points and determine the number of lessons that will be needed toteach all points. The total time allocated for instruction in the specificationmust be considered here as well as the instructional strategy for each lesson.For example, the number of instructors available for a given skill lesson mayaffect whether the individuals practice a task consecutively or concurrently.Guidelines on instructor to learner ratios are further detailed at Annex D.

54. The format and style of an Instructor or Master Lesson Plan (MLP) will varydepending on the instructional strategy for the lesson. However, generally the steps outlined below will be followed to write a lesson plan:

a. review the lesson specification in order to determine lesson requirementssuch as method, time, references and the context of each lesson;

b. ensure that the teaching points are clearly defined and that the number ofpoints to be taught is appropriate for the allotted time. If not, amendmentsto the training plan may be required to adjust timings;

c. review references and relevant documents or master lesson plans if available;

d. write lesson objectives for each lesson;

e. draft end of lesson tests;

f. outline and sequence the teaching points that are to be conveyed for the lesson;

Develop Lesson Plans

Homework


P A R T 4

23

g. develop teaching points as per the instructional strategy specified in lesson specification. This will entail the review of references and existingmaterials as well as the development of learning support (comparisons,reasons, examples, statistics and testimony);

h. describe/identify learning aids such as visuals (pictures, graphics, diagrams, video sequences), models, audio recordings, simulations;

i. define learner interactions: describe what the learners must do (read, listen, discuss, answer questions, perform procedure etc.). Incorporatequestions, group work or individual activities to develop and assess learners abilities as needed; and

j. write the lesson in full to include an introduction, body, and conclusion.

Note: Refer to Annex E, F and G, respectively, for detailed information on the contents of a lesson plan, the development of learning support, and questions.

55. Throughout the lesson development process, developers should consult with subject matter experts (SME), instructors and instructional designers forassistance as required. The development of lesson plans is essential to sup-port the delivery of effective instruction. Developers may need to be creativein order to invent and/or adapt materials to foster the appropriate learningenvironment. Personnel should refer to Volume 6, Delivery of InstructionalProgrammes, for further details on the delivery of instruction.


P A R T 5

24

CC O N D U C T T R I A L S A N DR E V I S E M A T E R I A L S

56. The purpose of this process is to assess the effectiveness of lesson guidance and instructional material. This process is essential to avoid the costly procurement or production of instructional material that proves to be ineffective when used.

57. The review of lesson guidance and training materials by SMEs must startearly in the Development phase. This ensures that material can be reviewed,tested and proven effective before translation and the start of formal training.

58. The trial of instructional materials may consist of:

a. Initial review. Review materials to ensure they develop the learning activity;

b. Expert Review. Content SMEs review instructional materials and verify the accuracy and completeness of content. Instructional staff can assesshow well the materials are likely to promote learning. Technical staff can assess technical issues such as the fidelity of a recording;

c. One on One. This refers to a trial of materials with an individual learnerwho represents the target population. A one on one trial allows us toassess whether the instructional materials are aimed at the appropriatelevel, are of interest and able to get the message across; and

d. Group or field trials. Trials of instructional materials with learners whoare representative of the target population can be conducted with small orlarge groups depending on the context and the availability of personnel.As trials with groups simulate the actual learning environment, they allowus to assess the delivery of instructional material and their effectivenessassisting learners to meet the objectives.

59. If the instructional staff selected to conduct a trial is the same staff selected to teach the course serials, the trials may double as instructor rehearsals. The developer, subject matter experts and standards staff must attend the trialsto ensure that all aspects are evaluated and any discrepancies are noted. Thesestaff members should also assist in follow-up, ensuring any corrections theyare empowered to make are implemented. Adjustments and any uncorrecteddeficiencies must be reported back to the Design staff, thereby ensuring thatchanges to the lesson guidance are initiated. Any new limiting factors identi-fied during the trials should also be reported to the Design staff.

Purpose

Timing

Trial Material


P A R T 5

25

60. Consideration must be given as to whether the trials are to be conducted in one block, or segmented. The main consideration is to ensure that timings are met. The effect of the selected method for conducting trials must be mea-sured against its effect on staff, learners and the overall schedule.

61. A pilot course is a full trial of the instructional programme prior to imple-menting training at the IT&E establishment. A pilot course does not result in qualifications for participants unless there are no significant revisionsrequired to course content. The decision to award a qualification to partici-pants will be determined by the MA. Elements evaluated during a pilotcourse are outlined in Table 5.

Consider Partial and Full Trials

Pilot Course

Accuracy, consistency and completeness.

Usefulness, to instructors and administrators.

Validity and reliability.

Course documentation

Tests

Relevance to course content.

Enhancement of learning.

Appropriateness to learner background and needs.

Classroom, workshop or other learning environment is suitable.

Learning aids

Appropriate for the objective and content.

Effective for learners.

Efficient in terms of costs.

Instructional Strategy

Consistency between planned and actual events in terms of: sequence of instruction; content; and use of facilities and equipment.

Limiting factors and unforeseen events.

Implementation

Infrastructure

Appropriate course and lesson duration.Time Allocation

ELEMENTS: EVALUATION CRITERIA:

Table 5: Pilot Course Evaluation Factors.


P A R T 5

26

62. Problems and revisions required to the instructional programme are identifiedthroughout the course trial. Problems with the instructional programme mayarise from different sources and involve a variety of factors. Some of these areoutlined in the Table 6.

Table 6: Examples of Sources and Factors Requiring Revisions and Retrials.

Revise Materials

Incomplete design or other inaccuracies; Misinterpretations; incorrect media or strategies.

Inaccuracies in policies, functions, operation and support concept; problems in design; problems in operation and/or support functions.

Instructional programme

System being taught

Incomplete assessment of operational or support needs; inadequate lead times for construction, utilities.

Seasonal considerations; time of day in relation to other activities; thesequence causes learning problems; “all-weather” taught/assessed in goodweather; “daytime/night time” requirement taught/assessed in daytime;excessive hours per day or sequential days in training.

Facilities

Incomplete assessment of needs; inadequate effectiveness; development time/cost; availability.

Training material

Insufficient personnel with appropriate background; retention.

Staffing

Timing/Sequence of instruction

Insufficient skills relative to learning requirement; inadequate level of population representation.

Learners

SOURCES EXAMPLES OF FACTORS

Incomplete assessment of needs; availability.Support material


P A R T 5

27

63. The analysis of problems arising from the trial of the instructional programme and its impact on the IT&E activity can be conducted using the following three steps:

a. identify potential problems arising from the trial and ensure all areas are examined;

b. rank the potential problems in a range of concern. This includes an evaluation of the complexity of the problem and how critical it is to the success of the IT&E activity; and

c. define options/solutions to address the problems and how to implementthem. Then examine the cost-benefit of addressing the problem and thecost-benefit of each option.

64. When all the problems have been addressed, developers must considerwhether time and effort should be spent on a retrial of affected areas. The seriousness of the problems identified and the probability and consequence of failure on learners' performance will determine if a retrial is warranted.

65. Instruction life cycle revision rates for IT&E activities are dependent on the stability of the course and the instructional materials. Considerations in deciding revision rates include forecasts of changes to the instructionalprogramme and its support concept, planned or forecast changes to occupa-tion specifications, as well as the complexities, time and effort that will beinvolved in incorporating training to accommodate these changes. These, and system or equipment specific critical items, if any, can be used to devisea revision rate. Life cycle revision rates can also be technology based (hard-ware/software dependant). Annex G addresses programme design and evalua-tion rates for IT&E establishment Standards to assist in the development oflife cycle revision rates. Annex C, which details level of effort rates for IT&Eservices may also be of assistance.

Consider Retrials andInstruction Life CycleRevision Rates


P A R T 6

28

Overview

PP R E P A R E S T A F F

66. Developers are to ensure that an instructional staff is available and preparedto the required level in instructional techniques to carry out an IT&E activity.Instructors, standards and support staff must also be prepared with respect to the system and to handling pertinent administrative aspects of the IT&E activity.

67. As standards staff must attend the trial of instructional materials to evaluate its effectiveness it is essential that standards personnel are knowledgeable of the aim and content of the instructional programme. This will require allocating time to conduct a thorough review of the training documentation.Similarly, if support staff is required to participate in trials of the instructionalmaterial it is essential that they be fully briefed on the instructional programmeand materials employed. In the case of an instructional programme where newequipment is being introduced formal training on the operation of the equip-ment may also be required. Provision of any training or preparation requiredby standards or support staff must take place prior to programme implementation.

68. As a minimum, developers should measure potential instructors against the requirements for successful completion of Basic Instructional Techniquesdescribed in the CFTDC document Courses and Services, Volume 2. It is recommended that instructors also attend the Advanced InstructionalTechniques when time permits. The selected instructors must receive training on the new material to be taught to the point of being completelyfamiliar with the course content, lesson guidance and instructional material.

69. Instructors should review the course content, instructional materials andassessment plan used to instruct the programme. The assessment plan portionof the training plan, along with the lesson specifications, can provide theinstructional staff with guidance on the aim and extent of testing and specificdetails relating to performance or enabling objectives. This information is valuable in ensuring that critical material is emphasized during the differentphases of the lesson.

Prepare Standards and Support Staff

Schedule Training forInstructional Staff

Review TrainingDocumentation


P A R T 6

29

70. In general, training is accomplished by instructors who lead individuals orgroups in learning activities, or by alternate means, such as self-study, com-puter based training (CBT), inter-active multimedia or video conferencing.Depending on the technology involved, instructors may require additionalpreparation and training in facilitating distance learning and multimedia.Instructors should attend CFTDC courses such as Interactive CoursewareDesign and Distance Learning Technologies for Managers if they are usingthese technologies.

71. Instructors must also be given the time and resources to rehearse the trainingsessions as they are described in the lesson specifications and to pilot thetests that are developed to evaluate the learning. If the staffing, course materi-al, resources and scheduling permit, some staff training and rehearsals may be implemented as part of the Conduct Trials and Revisions process.

Note: The development of instructors should continue beyond their initial preparation for a new or revised course. Ongoing instructor development is further addressed in Volume 6, Delivery of Instructional Programmes

Rehearse Lessons

Trial Equipment and Training Aids


P A R T 7

30

RR E C O R D D E V E L O P M E N TC O S T S

72. Development costs are incurred throughout the creation of a new or revisedcourse. The purpose of recording costs is to control expenditures and manageresources. This process allows costs to be monitored or controlled and isessential in large-scale development efforts that entail substantial investment.

73. Recording development costs will benefit training establishments by allowing them to:

a. calculate estimates and budget for the development of future initiatives;

b. request additional funding if development costs are significantly higherthan estimated in the Design phase; and

c. assess the cost-effectiveness of the selected instructional strategy duringthe Evaluation phase.

74. The process of collecting cost data should begin as early as possible duringthe Development phase. Developers are encouraged to create checklists toensure that all costs are recorded. These forms must reflect all those costsidentified during the Design phase as well as any new costs that arise. The Development Cost Factors table at Annex F, Volume 4, Design ofInstructional Programmes, may be used as is, or adapted as required.

75. Cost data may arise in many forms and from a variety of sources. Sources include:

a. invoices from contracts for goods and services or for the purchase of commercial materials “off-the-shelf”;

b. charges to local budgets through Financial Managerial Accounting System (FMAS);

c. Canadian Forces Supply System (CFSS); and

d. Cost Factors Manual (CFM). This is an unofficial publication prepared bythe Directorate of Managerial Accounting and Comptrollership 2 (DMAC 2)that provides a common basis for the estimation of DND personnel, equip-ment and facility costs. Local data should be used where units have moreaccurate and complete information.

Note: Additional financial references are provided at Annex A.

Purpose

Benefits

Create Checklists

Record Cost Data


P A R T 7

31

Compare Forecast to Actual Costs

Determine Costs

76. In a large or complex training programme Design and Development phase cost data should be detailed in a Business Plan. CFSS operates a CustomerConsumption Awareness (CCA) module and can capture all costs associatedwith the development of a programme if it is requested at the onset. Any taxes,such as GST, are to be recorded as a separate entry against the related item.This ensures that only real training costs are analyzed in order to provide avalid picture of development costs.

77. For complex programmes involving several threads of instruction, such as a mounted weapons system (with vehicle mount, fire control system andweapon), separate partial cost reports should be produced for each subsystemand major element. An aggregate Total Cost would then be produced as theend product for the system. This enables more detailed analysis of the cost of developing an instructional programme.

78. When capturing development costs, it is essential that costs be recorded onlyfor funding, (personnel, materials, facilities etcetera) expended to develop anIT&E activity. Recurring costs to operate equipment and facilities are operat-ing costs and are incurred and recorded when the instructional programme is conducted. These recurring costs should be included in the unit businessplan. Provision must also be made to track and record development cost dataarising from changes due to iterative Design or Validation phase activities.

79. To determine actual costs each item developed and produced must be costed.The results are totaled and combined with overhead, general and administra-tive expenses, and any contract fees to arrive at a final cost for the develop-ment of each item. Facilities are costed for design, construction, and theinstallation of utilities. Equipment items are costed for development, production, trials and integration. Materials are costed for design, production, revisions and translation.

80. Comparing actual development costs with the forecast cost enables the evaluation of the cost-effectiveness of the training process. Comprehensivecost reports can point to discrepancies in the forecast or to areas in the IT&Eactivity that need to be further examined in order to reduce costs or make theactivity more efficient. The findings from these examinations can lead to bud-get adjustments which may in turn have an effect on course loading, frequen-cy of serials or adjusted levels of training facilities, materials or staff.


A N N E X A

A1

DepartmentalReferences

RR E F E R E N C E S

Canadian Forces Training Development Centre Courses, CFB Borden,

Basic Instructional Techniques (qualification code: AHCH)

Advanced Instructional Techniques (qualification code: 06679(PQ))

Interactive Courseware Design (qualification code: AHCR)

Distance Learning Technologies (qualification code: 07040)

Canadian Forces Training Development Centre, CFB Borden, Courses and Services, Volume 2

Note: The document Courses and Services, Volume 2 can be found on the Web at http://home.interhop.net/~cflstc.

Canadian Forces Administrative Orders 9-53, Language of Instruction

Directorate of Managerial Accounting and Comptrollership 2, Cost Factors Manual

Defence Administrative Order and Directive 3004, Contracting

Defence Administrative Order and Directive 5039-4, Translation of Texts and Acquisition of Bilingual Documentation

ADM Finance and Corporate Services (Fin CS) document, Delegation ofAuthorities for Financial Administration for the Department of National Defenceand the Canadian Forces

ADM (Fin CS) guide, Managers Guide to Financial Administration, December 1993

ADM (Fin CS) booklet, Financial Management Accountability Framework, March 1998

Hartnett, J. Is This Click Really Necessary? Online Learning, September 2000.

Heinich, R., Molenda, M., and Russel, J.D. Instructional Media and the NewTechnologies of Instruction, New York, and New York: Macmillan PublishingCompany, 1993.

Leshin, C.B., Pollack, J., and Reigeluth C.M., Instructional Design Strategies andTactics, Englewood Cliffs, New Jersey: Educational Technology Publications, Inc.,1992.

Ribler, Ronald I. Training Development Guide, Reston, Virginia: Reston PublishingCompany Inc, 1983

Sabiston, P., An Inquiry Into The Criteria That Identify Quality Adult Web-BasedLearning, Master of Arts Project, April 2000.

Zemke, Ron. How Long Does It Take? Training, May 1997.

External References


B1

A N N E X B

GG L O S S A R Y

Individual training and education includes all activities that provide knowledgeand skills to individual members and that normally result in a qualification beingconferred upon the member.

The combination of media, methods, and environment used in the delivery of IT&E.

Unit or MA funded training that enables personnel to learn tasks while on thejob. A supervisor or facilitator demonstrates the skills, observes the learner's performance and provides feedback as required. Supervisors must certify that the learner has performed to the minimum specified standard and passed thePerformance Objective.

All types of material that support learning such as visual, print, audio-visual and technology-based materials.

A full trial of the instructional programme prior to implementing training at the IT&E establishment.

A lesson plan is a guide, used by instructors, to ensure that instruction follows a specific, goal oriented plan. An instructor lesson plan is developed, based on a lesson specification, by the instructor who will teach the lesson.

A Master Lesson Plan (MLP) is a lesson plan that is developed for use by anyinstructor and held on file at the IT&E establishment. MLPs are useful when there is a high turnover of staff or frequent changes to Training Standards/Plansresulting in little time for instructors to amend their lesson plans.

Individual Training and Education (IT&E)

Instructional Strategy

On-the-Job Training (OJT)

Instructional Material

Pilot Course

Lesson Plan

Master Lesson Plan


A N N E X C

C1

DD E V E L O P M E N T T I M E

When determining how long it will take to have materials developed, the level ofeffort rates (in hours) below can be applied. Each rate is based on the premise thatthe developer is also a content Subject Matter Expert (SME) and includes 25%time for revisions. The estimated number of tasks, Performance Objectives (POs),Enabling Objectives (EOs), and teaching points (TPs) can be based on documenta-tion for similar systems courses. Task and skill (S), knowledge (K), and/or attitude(A) TPs are based on the CF definitions provided in Volume 3, Analysis ofInstructional Requirements and Volume 4, Design of Instructional Programmes.Similar Occupation Specification, systems and courses may be used for estimatingtasks and SKA. Development rates by activity are illustrated in the chart below.

For example, the development time for a SME to develop instructor lessons is one skill, knowledge or attitude per hour. Therefore, to develop lesson plans foran Occupation Specialty Specification that contained 10 skills, 15 knowledges,and 0 attitudes would be 25 hours.

Note: For level of effort rate for contractors the rates below apply, as the personhired to develop materials will be a SME and will be dedicated to this task. For in-house development the level of effort rates will be increased by a minimum of25 percent because, due to the interference of primary job tasks and secondaryduties, in-house personnel cannot normally dedicate 100 percent of their time todevelopment efforts. For in-house development, the level of effort rates can beincreased by another 25 percent, for consultation with SMEs, if the developer is not a SME.

Average level of effort (rates in hours)for IT&E services and products

4 tasks per hour

4 tasks per hour

10 tasks per hour

4 tasks per hour

1 task per hour

Analyze Tasks/Performance (OS/OSS)

Analyze Task Elements (SKA)

Analyze Training and Education Requirement

Develop POs

Develop Performance Checks (PCs)

1 SKA per hour

1 SKA per hour

10 SKAs per hour

4 SKAs per hour

1 SKA per hour

Develop Task Element Scalar (EOs)

Develop EO

Develop Lesson Specification

Develop EC

Develop Instructor Lesson

ACTIVITY EXPECTED PERFORMANCE BY SME


C2

A N N E X C

DD E V E L O P M E N T T I M E

The development of technology-based lessons is influenced by many factors such as those discussed for non-technology-based materials. The chart below provides an estimate of development time based on the level of technology (0-3).Due to the many variables involved, the instructional activity under developmentand learner population should be considered when applying this chart to a par-ticular context.

DevelopingTechnology-BasedLessons

2 hours

Level 0: Traditional delivery styles (low tech)

A skill, knowledge, or attitude

4 hours

12 hours

32 hours

Level 1: Basic Interactive Courseware presenting onelesson after another (linear)

One skill

One knowledge

One attitude

12 hours

24 hours

40 hours

Level 2: Medium simulation courseware allowing students more control over lesson presentation

One skill

One knowledge

One attitude

32 hours

64 hours

96 hours

Level 3: High simulation presentation providingextensive interactivity (branching, simulation ofevents, re-mediation opportunities)

One skill

One Knowledge

One attitude

2 hrs per hour of actual delivery

All Levels:

Prepare Instructor

Trial Lesson

LEVEL OF TECHNOLOGY BASED LESSON AND TASKS

DEVELOPMENT TIME


A N N E X D

D1

II N S T R U C T O R T O L E A R N E R R A T I O S

Generally, training is conducted in the same grouping that you will operate in on the job. In order to determine the ideal instructor to learner ratio (number oflearners for each instructor) the following factors should be considered:

a. Content (e.g. academic, practical, safety, resources);

b. Learner (e.g. experience, motivation, supervision);

c. Instructional Environment (e.g. classroom, lab, hangar, field);

d. Method of Instruction (e.g. lecture, demo, discussion, practice);

e. Instructional Time (e.g. course duration versus staff and resources); and

f. Schedule (e.g. learner availability, unit commitments, etcetera).

It is important to note that the basic ratios provided can be affected by the specific needs of content, learner, environment and method in different contexts.Therefore, the 6 factors listed above, which apply to a particular learning context,must be kept in mind to ensure that the instructor to learner ratio provided isappropriate in a specific situation.

Basic guidelines on instructor to learner ratios are presented in the followingchart. The type of learning activity is described and the basic instructor to learnerratio for this activity is provided. A lesson specification, for example, might listdiscussion as the instructional method for a set of knowledge teaching points. If you refer to the knowledge learning activities sections in the table the ratio pro-vided for discussion, as an instructional method is 1:10. This means there shouldbe one instructor to ten students to conduct an effective discussion. Where 1:n isthe listed ratio n refers to the number of participants appropriate for the specificlearning context.

Principle

Basic Ratios

Using the Chart


D2

A N N E X D

I N S T R U C T O R T O L E A R N E R R A T I O S

I

1:10 using discussion method

1:40 using the lecture method

1:25 for facilitated Distance Learning

1:n for Computer Mediated Courses (CMC)

Procedural, conceptual, experiential theoryor knowledge

1:40 for classroom lecture method

1: n for CMC

1:25 for facilitated Distance Learning

Equipment theory or knowledge

Employment theory or knowledge

1:10 for individual learning in a classroom

1: n for CMC

Equipment operation (individual learning)

Equipment maintenance (individual learning)

1: crew/team

Equipment operation (crew/team learning)

Equipment maintenance (crew/team learning)

1:10 or 1: crew (consisting of less than 10 persons)Employment practice (conceptual learning)

1: crewEmployment practice (crew/team learning)

TYPE OF LEARNING ACTIVITY BASIC RATIO (E.G. 1:5 – ONE INSTRUCTORFOR EVERY FIVE LEARNERS)

Equipment practice (individual learning)

Remedial practice (instruct or assist) 1:1

1:nAcademic Assessment (classroom)

1:1Practical Assessment (individual)

1: crewPractical Assessment (crew)


A N N E X E

E1

LL E S S O N P L A N S

The development and use of a lesson plan assists the instructor to provide an effective learning experience. The lesson plan consists of an introduction,body and conclusion.

The purpose of the introduction is to motivate learners and stimulate their interest. The introduction can take up to ten percent of the total lesson time.There will be differences in the contents of the introduction depending on themedia and the instructional method used. For example, in the introduction of a case study lesson plan the case is introduced, learner roles may be designated,and group tasks identified. In the introduction of guided discussion lesson planthe main discussion areas are outlined. The introduction for a typical knowledgeor skill lesson includes:

a. Review – If appropriate, a short review of previously learned material;

b. Objectives – A description of what the learners will be able to do at theend of the lesson;

c. Importance – An explanation of why learners need to achieve the lessonobjective(s);

d. Fit – A description of how and where this lesson fits in the course or onthe job; and

e. Approach – An overview of how the lesson will be conducted.

The body of a lesson presents the material to be learned. Teaching points will be introduced and developed through learner support and/or well written pre-planned questions. In typical knowledge and skill lesson plans, teaching pointsare grouped and presented in stages to facilitate learning. The main content foreach stage includes:

a. Introduction – An overview of the stage and lead in to teaching points;

b. Teaching points – Each teaching point must be accurate and clearly pre-sented using the most appropriate instructional method and training aids;

c. Learner participation – In a knowledge/theory lesson, learners participateby interacting with the class or instructor, considering questions and dis-cussing lesson content. In a skill lesson, learners will spend most of theirtime practicing and performing a skill; and

d. Confirmation – In a knowledge lesson, the instructor confirms learning byposing questions to the class. In a skill lesson, learning is confirmed byproviding practice and watching students perform a skill.

Introduction

Body


E2

A N N E X E

LL E S S O N P L A N S

During the conclusion of a lesson key points are summarized and linked to futurelessons and the job. The conclusion can take up to five percent of total lessontime and includes:

a. Summary – During the summary teaching points are reviewed and difficultpoints are re-emphasized;

b. Closing statement – The closing statement is used to link the content of thecompleted lesson to future lessons; and

c. Re-motivating statement – This statement re-emphasizes the importance of the lesson.

Note: Personnel should refer to Volume 6, Delivery of Instructional Programmes,for information on delivering lessons.

Conclusion


A N N E X F

F1

LL E A R N I N G S U P P O R T

Learning support is information that enhances the teaching point. This informa-tion adds credibility to teaching points and further explains or clarifies the point.Learning support should be developed for each teaching point in a lesson plan.Learning support is used to enhance the learning experience by:

1. creating interest and variety;

2. illustrating concepts;

3. clarifying points; and

4. emphasizing points.

Some types of learning support are comparisons, reasons, examples, statistics and testimony (CREST). Each type of support is described in the following chart.


F2

A N N E X F

LL E A R N I N G S U P P O R T

Comparisons are used to bridge the gapbetween concepts learners have already mastered to the new knowledge that is being presented.

The make up of the learner group must beconsidered when developing comparisons to ensure that they will be meaningful.

Comparisons

Reasons assist learners to understand whysomething is carried out in a certain way.

When dealing with controversial topics, reasons can help learners to accept teaching points.

Reasons

Examples can be used to clarify a concept or reinforce a point.

Examples Providing an example of how a table orchart should be used will make it easier forthe learner to understand the steps involved.

An example of how failure to observe safetyprecautions resulted in an accident can rein-force teaching points.

Statistics

TYPE DESCRIPTION EXAMPLE

Comparisons of the last rifle, vehicle, or air-craft used to the new model in terms of thesimilarities and differences can help learnersto retain the new information.

Word picture comparisons, such as light as a feather, reinforce and clarify the teachingpoint as well as add interest.

Safety concerns is a reason instructors may provide to explain why a task must be completed in a specific manner.

Statistics can lend support to teaching points and provide emphasis.

Statistics such as “50% of information technology projects succeed” can emphasizethe importance of using project managementtechniques.

Testimony Testimony or quotes relevant to the teachingpoint can be used to add credibility to teaching points.

Testimony should be from an appropriateexpert that the learners can relate to.

Testimony on the capabilities of new systemsor equipment introduced to the CF, fromsomeone who has been using it, can addcredibility to teaching points.


A N N E X G

G1

DD E V E L O P I N G Q U E S T I O N S

Questions that are carefully developed and incorporated into the lesson plan cansignificantly enhance learning. Lead off questions can be used to introduce a topicor discussion. Follow-up questions on the topic being taught can keep learnersfocused on a concept. Questions can be posed, to the class or to specific learners,throughout lessons to:

1. stimulate thinking and interest on salient points by provoking curiosityand challenging learners;

2. assess learner comprehension by posing questions on the material taught;

3. develop problem-solving skills by posing problem questions on relevantactivities for learners to solve; and

4. guide the learning process by posing questions that guide learners thinkingthrough the logical development of the lesson.

The instructional activity, learner characteristics and purpose of a question will affect how it is written. Criteria for developing well-written questions areprovided in the following chart.


G2

A N N E X G

D

Questions should be brief, yet complete to ensure learners understand their meaning. Comprehension

Questions should be written using common words and as clearly as possible.

For example, a first aid question such as “What is the first thing you do if someonehas a myocardial infarction” should be “What is the first thing you do if someone has a heart attack”.

Language level

Questions should:

1. challenge course members to apply their knowledge;

2. not be so easy that the answer is obvious; and

3. not be so difficult that only a few learners can answer.

For example with the question, “Is the naval rank equivalent of Master WarrantOfficer, Chief Petty Officer, second class”, the learner has a 50/50 chance of guessingthe correct answer. Asking instead “Name the naval rank equivalent of Master WarrantOfficer” requires the learner to think more.

Difficulty level

Relevant Questions should be developed to reinforce and support the main teaching points of the lesson.

For example, if teaching about survival in a nuclear war, learner interest can be pro-moted with a question such as “What would you do to protect yourself if you weretold there will be a nuclear explosion in twenty minutes”. This question is relevant to the topic. Then questions specific to various teaching points can be developed.

ELEMENTS CRITERIA

D E V E L O P I N G Q U E S T I O N S


A N N E X H

H1

PP R O G R A M M E D E S I G N A N D E V A L U A T I O N R A T E SF O R I T & E E S T A B L I S H M E N TS T A N D A R D S

When determining how long it will take conduct various evaluation activities, the level of effort rates below can be applied.

Note: For programme design rates refer to Annex C, Development Time.

1:1 period observed

1:1 period

Refer to level of effort rates at Annex C

Refer to Training plan allocations

1:1 test/period

1:1 period

1 hour for each training week



1 hour per session

5% of initial course design per session

10% of initial test design per session

10% of initial material design per session

Prepare to observe lesson

Observe the conduct of lessons or tests

Prepare PCs, ECs and test banks

Administer Tests

Score tests

Complete observation report

Prepare End Course Review

Conduct End Course Review

Prepare End Course Report

Present End Course Report to Comd/CO

Revise Design

Revise Tests

Revise Materials

ACTIVITY EXPECTED PERFORMANCE BY SME

Average Level of Effortfor IT&E EstablishmentStandards EvaluationActivities.

Documents

Cfites Vol 5 e