Copy 6 1 of 345 copies

AD-A259 478

IDA PAPER P-2743

DEFENSE WORKFORCE TRAINING PROGRAMS

J. D. FletcherRuth WienclawGary BoycanJames Bosco

Harold F. O'Neil, Jr. --

October 1992 E 0ELECTES JAN2. 7 1993

Prepared for S___ •Office of the Assistant Secretary of Defense

_=37 (Force Management and Personnel)and

Office of the Assistant Secretary of Defense for Public Affairso (American Forces Information Service)I ~Defense Audiovisual Policy Office

Approvd 1sr P0blic rlau; fdkhtr vlutsunlmild.

* A INSTITUTE FOR DEFENSE ANALYSES11801 N. Beauregard Street, Alexandria, Virginia 22311-1772

9T•' 1 26 072 IDA Log No. HO 92-42072

DEFINITIONSIDA publishes the following documents to report the results of its work.

RepoReports are the most authoritative and most carefully considered products IDA publishes.They normally embody results of major projects which (a) have a direct bearing ondecisions affecting major programs, (b) address issues of significant concern to theExecutive Branch, the Congress and/or the public, or (c) address issues that havesignificant economic implications. IDA Reports are reviewed by outside panels of expertsto ensure their high quality and relevance to the problems studied, and they are releasedby the President of IDA.

Group ReportsGroup Reports record the findings and results of IDA established working groups andpanels composed of senior individuals addressing major Issues which otherwise would bethe subject of an IDA Report. IDA Group Reports are reviewed by the senior individualsresponsible for the project and others as selected by IDA to ensure their high quality andrelevance to the problems studied, and are released by the President of IDA.

PapersPapers, also authoritative and carefully considered products of IDA, address studies that

are narrower In scope than those covered in Reports. IDA Papers are reviewed to ensurethat they meat the high standards expected of refereed papers in professional journals orformal Agency repoits.

DocumentsIDA Documents are used for the convenience of the sponsors or the analysts (a) to recordsubstantive work done In quick reaction studies, (b) to record the proceedings ofconferences and meetings, (c) to make available preliminary and tentative results ofanalyses, (d) to record data developed in the course of an Investigation, or (a) to forwardinformation that is essentially unanlyzed and onevaluated. The review of IDA Documentsis suited to their content and intended use.

The work reported In this document was conducted under contract MeDA 903 89 C 0003 forthe Department of Defense. The publication of this IDA document does not Indicateendorsement by the Department of Doefnse, nor should the contents be construed asreflecting the official position of that Agency.

I Form ApradREPORT DOCUMENTATION PAGE FM or, Pov,OMB No. 0704.188

ftM" bM i • f b N i d l6U, 61~ f •tugO I h*mnpwupmu. 4W i&*ag NW 6w s mw g u mmmigm -uimg Wr. , uam mimd u g mO d= nmded. mnd

1111 Wm0 l md = ow u8688dme . n .f m tW uin IM bAdM ummwdh r my o0w uet ofN *m e"kn of wm mdk ,g I -, I h uiadamr I - -" dn. l oOb hmim

Ime S mmd. h r w OOw 1m tId Rp 12116 IM lJ Do•V HI6 . SUM 1204. Abmom. VA C202- . ia d Ob a# bo id d Pgmm Ah Pn"q

1. AGENCY USE ONLY (Leave bbnk) 2. REPORT DATE 3. REPORT TYPE AND DATES COVERED

October 1992 Final-January - July 1992

c TITLE AND SUBTITLE S. FUNDING NUMBERS

Defense Workforce Training Programs C - MDA 903 89 C 0003

T- T-L2-883 andS. AUTHOR(S) T-AB2-629.13

J. D. Fletcher, Ruth Wienclaw, Gary Boycan, James Bosco,Harold F. O'Neil, Jr.

7 PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATIONREPORT NUMBER

Institute for Defense Analyses1801 N. Beauregard St. IDA Paper P-2743Alexandria, VA 22311-1772

1. SPONSORINGNMONORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSORING/MONORING

OASD(FM&P)(R&R) OASD(PA) AGENCY REPORT NUMBER

The Pentagon, Room 3B930 AFISWashington, DC 20301 601 N. Fairfax St.

Alexandria, VA 22314-200711. SUPPLEMENTARY NOTES

12. DISTRIBUTION/AVAILABLITY STATEMENT 12. DISTRIBUTION CODE

Approved for public release; distribution unlimited.

13. ABSTRACT (tNxhmam 10 War&)This report discusses the amount and kinds of DOD interactive courseware (ICW) programs that arecandidates for transfer to the private sector. The Defense Instructional Technology Information System(DITIS) was analyzed to identify ICW programs that are candidates for transfer from the DOD to theprivate sector. Out of 4644 ICW programs that have been reported to DMS since its inception in May1991, 2718 were identified as having possible utility in meeting private sector training requirements. Ofthese 2718 programs, 591 were computer-based instruction programs that run on the MS-DOS operatingsystem, 1997 were computer-based instruction programs that run on the NOS operating system and wereprepared using the TUTOR authoring system, and 130 were interactive videodisc programs that run onthe MS-DOS operating system. Final determination of the transferability of these programs will depend onmore authoritative suitability, legal, and security reviews that must be undertaken on a case-by-casebasis. The DOD has taken several initiatives to assist the private sector in transferring DOD ICWprograms to the civilian workforce for training. However, a number of issues, particularly those involvingthe availability of resources and the establishment of responsibilities, must be resolved before acomprehensive set of procedures for transfer can be successfully implemented.

14. SUBJECT ERMS NU. MUER OF PAGESTraining, Education, Institutional Technology, Computer-Based Instruction, 60Interactive Courseware, Technology Transfer IL PIMcE CODE

17. SECURITYNI . SECURITY CLASSFICATION 15. SECURTMY CLASSICAIMON M LMITATION OF ABSTRACTOP REPORT OF TNIS PAGE OF ABSTRACT

UNCLASSIFIED UNCLASSIFIED UNCLASSIFIED SARNSN 7540-011-20-6600 SWdWd F2M 2 & R. 2-69)

Pnmbimb l

IDA PAPER P-2743

DEFENSE WORKFORCE TRAINING PROGRAMS

DIY~C QiAT7-*j Vr

Accesion ForNTIS CRA&I

J. D. Fletcher DTIC TAB

Ruth Wienclaw Unannounced onGary Boycan JustificationJames Bosco

Harold F. O'Neil, Jr. By

Disti ibution I

Availability Codes

Dist Avail and /or

October 1992 Special

AWNW W PWt r pniiam; idldbutinm unmiltld.

IDA

INSTITUTE FOR DEFENSE ANALYSESContract MDA 903 89 C 0003

Task T-L2-883 andTask T-AB2-629.13

PREFACE

This work was jointly sponsored by the Office of the Assistant Secretary ofDefense for Force Management and Personnel and the Office of the Assistant Secretary of

Defense for Public Affairs, American Forces Information Service.

The paper was reviewed by James Ashcraft, Deputy Director, Defense AudiovisualPolicy Office, American Forces Information Service, by Raye Newmen, a private

consultant, and by Michael Fineberg of IDA. The authors express their sincere gratitude tothese reviewers. Their efforts and comments have substantially improved the paper. MarkStenger of the Defense Training and Performance Data Center spent many hours extracting

data from the Defense Instructional Technology Information System, and for his efforts we

are also most grateful.

iii

ABSTRACT

This report discusses the amount and kinds of DoD interactive courseware (ICW)programs that are candidates for transfer to the private sector. The Defense Instructional

Technology Information System (D1TIS) was analyzed to identify ICW programs that are

candidates for transfer from the DoD to the private sector. Out of 4644 ICW programs that

have been reported to DITIS since its inception in May 1991, 2718 were identified as

having possible utility in meeting private sector training requirements. Of these 2718programs, 591 were computer-based instruction programs that run on the MS-DOS

operating system, 1997 were computer-based instruction programs that run on the NOSoperating system and were prepared using the TUTOR authoring system, and 130 wereinteractive videodisc programs that run on the MS-DOS operating system. Final

determination of the transferability of these programs will depend on more authoritativesuitability, legal, and security reviews that must be undertaken on a case-by-case basis.

The DoD has taken several initiatives to assist the private sector in transferring DoD ICW

programs to the civilian workforce for training. However, a number of issues, particularly

those involving the availability of resources and the establishment of responsibilities, must

be resolved before a comprehensive set of procedures for transfer can be successfully

implemented.

v

CONTENTS

Preface ................................................................................................ iii

Abstract ................................................................................................ v

Tables . viii

Figgees .............................................................................................. viii

SUMMARY .S-1

I. BACKGROUND .I-1

H. PRIVATE SECTOR REQUIREMENTS .............................................. 1-1

III. DoD IN TIATIVES ..................................................................... I-1

A. ICW Portability ...................................................................... 111-2

B. Development and Management of ICW .......................................... n-6

C. Procedures for Transferring ICW ................................................ .011-6

IV. TRANSFERABLE DoD COURSEWARE .......................................... .V-1

A. MS-DOS Computer-Based Instruction .......................................... JV-9

B. NOS (Tutor) Computer-Based Instruction ............................... . .. V-9

C. MS-DOS Interactive Videodisc Instruction .................... IV-16

V. CONCLUSIONS ...................................................................... V-1

VI. REFERENCES . ................................................................. ...... VI-1

APPENDIX A - DITIS DATA ELEMENTS ...... .......................... A-1

vii

TABLES

1. Reports Documenting the Need for Job Skills ............................................ 11-2

2. Unmet Need for Retraining American Workers in 1989 ................................ 11-6

3. Sources of ICW Program Reported in DTIS ........................................... IV-2

4. Reported Characteristics of ICW Programs in DITIS .................................. IV-3

5. Authoring Systems Used to Prepare ICW Programs in DITIS ...................... V-5

6. Counts of Candidate ICW Programs ................................................... IV-10

FIGURES

Ia. Standard Architecture Without Portability ............................................... .In-5

lb. Standard Architecture With System-Level Portability .. ......... .................. MI-5

ic. Standard Architecture With Device-Level Portability ............................... n I-5

2. Materials Transfer ............................................................. 11M-7

viii

SUMMARY

I. BACKGROUND

This report is the second prepared in response to Congressional interest in

transferring DoD instructional materials to the private sector for workforce training. The

first report suggested DoD policy and procedures for instructional materials transfer. This

report discusses the amount and kinds of DoD interactive courseware programs that are

candidates for transfer to the private sector. In accordance with the Training Technology

Transfer Act of 1988, the emphasis in this report is on the transfer of interactive

courseware (ICW).

II. PRIVATE SECTOR REQUIREMENTS

The private sector is emphasizing greater worker responsibility and participation at

all levels of the workforce, thereby increasing requirements for improved worker skills,higher standards of performance, and more training. A number of studies have

documented the need for American workforce training in five categories of competenciesthat private sector managers seek in entry-level personnel. These categories are:(1) Personal Characteristics and Attitudes, (2) Interpersonal and Teamwork Skills,

(3) Higher Order Thinking Skills, (4) Basic Skills, and (5) Job-Specific Technical Skills.

If DoD ICW programs that address these competencies can be transferred to the private

sector, they may provide needed workforce training.

I11. DoD INITIATIVES

In accord with the Training Technology Transfer Act of 1988, the DoD has taken a

number of initiatives to increase the availability and transferability of its ICW programs.

These initiatives include (1) issuance of MIL-STD-1379D to ensure the portability of

ICW programs across many delivery platforms; (2) issuance of DoD Instruction 1322.20

and establishment of the Defense Instructional Technology Information System (DITIS)

to improve the development, management, and reporting of ICW programs; and

(3) development of procedures for transferring ICW programs to the private sector.

S-1

0

Procedures for transferring ICW programs that are audiovisual productions, such asinteractive videodisc programs, are already specified, and similar procedures are beingdeveloped for transferring ICW programs that are not audiovisual productions, such ascomputer-based instruction programs.

IV. TRANSFERABLE DoD COURSEWARE

The D1TIS database was analyzed to identify ICW programs that are candidates fortransfer from the DoD to the private sector, although final determination of transferabilitywill depend upon the results of more authoritative suitability, legal, and security reviewsthat must be undertaken on a case-by-case basis. Out of 4644 ICW programs that havebeen reported to DTS since its inception in May 1991, 2718 were identified as havingpossible utility in meeting private sector training requirements. Of these 2718 programs,591 were computer-based instruction programs that run on the MS-DOS operating system,1997 were computer-based instruction programs that run on the NOS operating system andwere prepared using the TUTOR authoring system, and 130 were interactive videodisc

programs that run on the MS-DOS operating system.

V. CONCLUSIONS

The following conclusions may be drawn from this review:

"A number of DoD ICW programs appear to be candidates for transfer to theprivate sector, but final judgements on their transferability require moreauthoritative and comprehensive reviews that must be performed on a case-by-case basis. Many of these ICW programs have proved to be of significantvalue in DoD training and may be useful in meeting private sector workforcetraining requirements if they can be transferred.

" Workable procedures are already in place to aid in the transfer of DoD ICWprograms that are audiovisual productions, such as interactive videodiscprograms. These procedures serve as a model for effecting transfer of otherICW programs that are not audiovisual productions, such as computer-basedinstruction programs.

" The problems of resourcing and responsibilities remain. Through its policiesand procedures, the DoD is prepared to assist the private sector in making DoDICW programs available for transfer, but resources both within the DoD andoutside the DoD remain to be identified and responsibilities for many necessaryfunctions, especially those to be performed outside the DoD, remain to beassigned. •

S-2

I. BACKGROUND

This report is the second prepared in response to Congressional interest intransferring DoD instructional materials to the private sector for workforcetraining. The first report suggested DoD policy and procedures forinstructional materials transfer. This report discusses the amount and kindsof DoD interactive courseware programs that are candidates for transfer tothe private sector. In accordance with the Training Technology Transfer Actof 1988, the emphasis in this report is on the transfer of interactivecourseware (ICW).

The National Defense Authorization Act for Fiscal Years 1992 and 1993 directedthe Department of Defense "to prepare an inventory on DoD training and educationprograms that relate to workforce training issues of concern to the private sector." This

report was prepared in response to that direction. An earlier report (Fletcher, Bosco,Wienclaw, Ashcraft, and Boycan, 1991) was also produced in response to Congressionalinterest in this topic. It discussed policies and procedures that might be used to effect the

transfer of DoD instructional materials to the private sector. In accordance with theCongressional direction, this second report discusses what and how many DoD interactivecourseware (ICW) programs might be candidates for transfer to the private sector.

Interest in the DoD as a source of instructional materials for the private sector has

been growing for several years. A number of Congressional actions have been taken tofacilitate and govern transfer of DoD materials and technologies to non-DoD applications.

Three of these initiatives are the following:

"The Stevenson-Wydler Technology Innovation Act of 1980. This act focuseson research and development. It authorizes the establishment of Centers forIndustrial Technology and is intended to stimulate use of federally fundedresearch and development products by state and kcal governments and theprivate sector. In the case of instruction, the act facilitates and encouragestransfer of research and development products from the DoD personnelresearch laboratories to non-DoD applications.

" The Federal Technology Transfer Act of 1986. This act also focuses onresearch and development. It amends the Stevenson-Wydler Act byauthorizing cooperative research and development agreements between Federaland non-Federal laboratories and by establishing a Federal Laboratory

I-1

Consortium for Technology Transfer within the National Institute forStandards and Technology.

The Training Technology Transfer Act of 1988. This act, which is a chapter ofthe Omnibus Trade and Competitiveness Act of 1988, focuses on the transferof federally-developed education and training software to non-Federalactivities. The act is intended to facilitate transfer of education and trainingsoftware from Federal agencies to applications that support the education,training, and retraining of industrial workers, especially workers in smallbusiness concerns. It specifically identifies the DoD as a potential source ofinstructional software. The act establishes an Office of Training TechnologyTransfer, which is to maintain a clearinghouse of information on federallydeveloped education and training software. •

Given the scope of this topic and the level of interest i-1 it, other Congressional

initiatives, some of which are now under discussion, are likely to affect requirements and

responsibilities for transferring DoD instructional materials to the private sector.

The DoD has responded to these initiatives with several actions of its own. Many

of these actions, including the development of the policies and procedures outlined earlier

(Fletcher et al., 1991), were undertaken jointly and cooperatively by the OASD(FM&P)

Readiness and Training Directorate and the OASD(PA) Defense Audiovisual Policy Office.These actions emphasize development, management, and transfer of the instructional

software that is the primary concern of the Training Technology Transfer Act

The DoD describes this software as "Interactive Courseware" (ICW), which is used

as a general term for all varieties of interactive, instructional software. ICW is defined ascomputer-controlled instructional material, or courseware, that relies on student input todetermine the pace, sequence, and content of instruction delivery. Courseware by itselfrefers to all training materials, including the curriculum database and all disks, tapes,books, charts, and computer programs, necessary to deliver an ICW prograxmn

The key distinction between an ICW program and other training programs is the

provision, through use of computer technology, of interactions that tailor the instruction to

the needs of individual students. With individually tailored instruction, e, ' sudentreceives the level of detail, pace, remediation, sequence of topics, and interactions needed 0

to learn the material efficiently within the limits imposed by time and access to otherinstructional resources. ICW programs can individualize instruction within currently

established, group-oriented instructional institutions. This advantage, combined with the

fact that ICW programs tend to be developed as transportable, or autonomous, modules of 0

1-2

0

instruction, makes them promising candidates for smooth transfer to training in the privatesector.

Transfer of DoD education and training materials and technologies is motivated by

the workforce requirements of the private sector. These requirements are briefly reviewedin Section II. Section III describes initiatives taken by the DoD to improve its ownmanagement of ICW programs. These initiatives also facilitate access and transfer of DoDICW programs to outside agencies and the private sector. Section IV provides anassessment of the amount and kinds of DoD ICW programs that might be made available tomeet workforce training requirements in the private sector. Section V presents a fewconclusions that might be drawn based on these findings.

0

1-3

II. PRIVATE SECTOR REQUIREMENTS

I The private sector is emphasizing greater worker responsibility andparticipation at all levels of the worikforce, thereby increasing requirementsfor improved worker skills, higher standards of performance, and moretraining. A number of studies have documented the need for Americanworkiforce training in five categories of competencies that private sectormanagers seek in entry-level personnel. These categories are: (1) PersonalCharacteristics and Attitudes, (2) Interpersonal and Teamwork Skills,(3) Higher Order Thinking Skills, (4) Basic Skills, and (5) Job-SpecificTechnical Skills. If DoD ICW programs that address these competenciescan be transferred to the private sector, they may provide needed workforcetraining.

What instructional programs are of interest to the private sector? This question isbest addressed by considering the workforce competencies that are of most concern tomanagers in the private sector. Several studies have been completed recently that address

this issue. Seven of them are listed and briefly described in Table 1.

These studies all note that the challenge of competing in the world marketnecessitates re-examination of our use of people in organizations. The early part of this

century witnessed a burgeoning concern about how to organize the efforts of individuals in

the large organizations that grew out of the industrial revolution. Increasingly, the humanfactor was recognized as critical to productivity. Frederick Taylor, who is considered the

father of American management, recognized that limitations in the skills of workers limited

the productivity of their organizations (Taylor, 1916). His answer to perceived deficienciesin worker skills was an extensive managerial structure to organize and supervise the

workers, leaving them with as little individual responsibility and discretion as possible.

This approach has been subjected to increasingly critical scrutiny in recent years.

Today, managers recognize that workers must assume more responsibility at pointsof production, sales, or service rendered, if their organizations are to compete successfullyin today's world markets (e.g., Johnston and Packer, 1987; Senge, 1990;, U.S. Congress,

Office of Technology Assessment, 1990;, and others). To introduce new products and

services quickly with high quality, managers are emphasizing greater worker participation,

flatter organizational structures, just-in-time management approaches, job rotation, and

11-1

0

Table 1. Reports Documenting the Need for Job Skills

Report Trends Needs

America's Choice: High Higher worker skill requirements Public/private sharing orSkills or Low Wages! responsibility for training and(National Center on U.S. embarking on a "thid educationEducation and the industrial revolution"Economy, 1990) Comprehensive employment/

training policy

High Schools and the American economy in a period of Need for education in bothChanging Workplace drastic change core competencies and(National Academy of specific skillsSciences, 1984) Workplace requirements will

continue to change

Michigan Employability Training and education needs are Need for training andSkills Employer Survey constantly changing education in both generic and(Mehrens, 1989) Formal education will never meet specific skills

all the needs of the marketplace Perceived and actual needsmay not be the same

What Work Requires of Globalization and technology Need for broader skills andSchools contain both threat and promise training for everyone(Secretary's Commissionon Achieving Necessary High performance workplaces are Requirement for bothSkills, 1991N) the model for a successful future foundation and specialty skillsSkills,_ 1991) training and education

Worker Training: Broader use of computer-based Need greater investment inCompeting in the New training transferable skills trainingInternational Economy(Office of Technology Leaming on demand" Need more Federal assistanceAssessment, Increased emphasis on training Encourage use of trainingU.S. Congress, 1990) technology consortia

Workforce 2000: Work More training and education for Training to meet changingand Workers for the skills required by today's workdorce occupational structure21st Century(Johnston and Packer, People are the primary asset of Need to address education1987) today's organizations and training needs of

workforce now

Workplace Basics: The Increasing nunmer of entry-level Need sound basic skillsEssential Skills Employers workers come from groups that are training to Increase U.S.Want educationally deficient competitive edge and(CaMevale, Gainer, and Employees must be able to quickly ind'iduars quality of lWfeMeltzer, 1990) understand and acquire new and Functional context approach

different skills to job skils training

H-2

team work. This decentralization requires more competency from all members of theworkforce, and managers pursuing these goals back them up with substantial, ongoing

training. The report issued by the Congressional Office of Technology Assessment

(Worker Training, 1990) concludes that "American companies that have adopted this model

have found that their workers can achieve levels of productivity and quality equal to the

best in the world" (p. 3).

The possibility that the U.S. workforce may not be prepared to meet current

challenges has been raised by many studies. Worker Training summarizes the discussion

in a way that is characteristic of many reports on this topic:

The quality of the U.S. workforce matters now more than ever. Well-trained, motivated workers who can produce high-quality goods andservices at low cost help enhance industrial productivity andcompetitiveness and keep American living standards high. In today'sinternational economy, workers must be prepared to change the way they dotheir jobs in order to capture the benefits from rapidly evolving technology.Training goes hand-in-hand with productivity, quality, flexibility, andautomation in the best performing firms.

Good training pays off for the individual worker whose skills are upgraded,for the company seeking a competitive edge, and for the Nation in overallproductivity and competitiveness. Conversely, inadequate training costsfirms and workers in down-time, defective parts or equipment, wastedmaterial, health and safety risks, late deliveries, and poor customer service.Poor training also can delay the implementation of new technology or workreorganization.

When measured by international standards, most American workers are notwell trained. Many in smaller firms receive no formal training. Largerfirms provide more formal training, but most of it is for professionals,technicians, managers, and executives. Our major foreign competitors placemuch greater emphasis on developing workforce skills at all levels.Experienced production workers at Japanese auto assembly plants, forexample, get three times as much training each year as their Americancounterparts. (Worker Training, p. 1)

In another analysis, Carnevale and Gainer (1989) discussed the declining

percentage of U.S. Gross National Product (GNP) that is accounted for by resources from

the earth such as minerals, energy, and food. In 1890, these resources accounted for

50 percent of the GNP. Today these resources account for 10 percent of the GNP. Their

place has been taken by human resources which now account for more than four-fifths of

the nation's economic output. Bishop (1989) analyzed the impact of training on the

productivity of human resources and found that workers who receive formal job training

11-3

have a 30 percent higher productivity level than workers who are not formally trained.Investments in training may therefore substantially improve economic productivity.

The requirement for workers with greater skills and capacities for responsibility hasspawned many commissions, task forces, and studies, including those listed in Table 1.

These studies have contributed to the growing evidence that high levels of workforce

performance are essential for successful competition in today's markets and that workforcetraining is needed to achieve them.

O'Neil, Allred, and Baker (1992), who reviewed most of the studies in Table 1,point out that these studies have begun in the same way. Educators, business leaders,scholars, and policy makers were assembled to identify skills necessary for the workforce.

These authorities generated a list of necessary skills based on their own knowledge andexperience and on independent investigations of the workforce and its requirements.Nearly all the studies included a validation phase in which employers and/or employeeswere asked to rate the relative necessity, desirability, or importance of each identified skill. 0Although different labels were used in the studies, there is consistent agreement across allthe studies on five major categories of competencies that are sought and needed by workersif their organizations are to compete successfully in the modern world. These categories ofcompet cies are:

" Personal Characteristics and Attiudes. Important themes in this category ofcompetencies are dependability, honesty, motivation, and responsibility.Many of the private-sector authorities rated workforce qualities in this categoryas the most important of all.

" Interpersonal and Teamwork Skills. Many of the studies noted that asresponsibility is pushed further down the management hierarchy, responsibilityfor communication and cooperation is increasingly shared by all members of anorganization. Increasingly, workers must communicate, negotiate, tolerate andperhaps capitalize on diversity, share leadership, and interact with others in a 0professionally and socially appropriate manner.

" Higher Order Thinking Skills. Each of the studies identified the need for skillsin this category of competencies. These skills were judged necessary becauseof the rapidity of change in the workforce and its high mobility. The higherorder thinking skills most commonly identified were skills in adapting tochanges. These skills include reasoning, problem-solving, creativity, decisionmaking, and learning how to learn. The latter skill was identified in one studyas the fundamental skill.

11-4

Basic Academic Skills. Each study identified the need for basic academicskills. These include speaking and listening skills as well as the three R's.Worker Training concluded that "many American workers--20 percent or morein some firms--are deficient in basic skills" (p. 10) and that very fewcompanies now make much effort to upgrade their employees' basic education;they are more likely to test job applicants for basic skills competencies and nothire applicants who fail.

Job-Specific Technical Skills. Many studies noted the need for job-specifictechnical skills in the workforce. These are the many, diverse skills needed tooperate and maintain equipment, control complex systems, and followtechnical procedures. These skills do not require academic preparation somuch as workplace experience and training tailored to the needs of specificjobs.

ICW programs have been shown to contribute significantly to developingcompetencies in the last four of these categories. ICW contributions to the last three ofthese catagories (Higher Order Thinking Skills, Basic Academic Skills, and Job-SpecificTechnical Skills) are generally acknowledged and documented, but it should be noted thatcomputer-based instruction and interactive multimedia technologies such as interactivevideodisc have also been found to contribute significantly to the development ofinterpersonal and teamwork competencies in ways that are both less labor intensive andmore effective than standard approaches using classroom instruction and role playing (seeBlaiwes and Weller, 1978; Schroeder, Dyer, Czerny, Youngling, and Gillotti, 1986, for

examples). Whether or not ICW programs can contribute to the development ofcompetencies in the first category (Personal Characteristics and Attitudes) has yet to be

assessed.

Worker Training (1990) also emphasized the plight of small firms, those with 100employees or less. These firms provide about 35 percent of total U.S. employment, but

they lack the expertise to provide in-house training, the resources to pay for outsidetraining, and sufficient numbers of people who need training at any one time to justify afocused training effort. Further, they are more likely to employ workers who have lesseducation and are older or younger than the average. They usually try to provide training

through unstructured, informal approaches that vary widely in quality. Transportable,

quality-controlled training, such as that available from many ICW programs, may helpsolve the training problems of small firms.

Larger firms are more likely to provide formal training than smaller ones, but the

training they provide is largely limited to manager and technicians. The lower the level of

1-5

skill a worker brings to a firm, the less likely it is that he/she will receive training from the

firm. Still, the amount that firms invest in training their workers is sizable. Estimates of

these investments range from $33-44 billion a year, with about $9 billion provided by

outside sources. However, this investment, which is 0.6-0.9 percent of GNP, may be far

short of what is needed. The American Society for Training and Development (undated)

estimated that in 1989 the total cost to retrain those parts of the American workforce

that required skill updates and enhancements would have been $64 billion of which

$45.4 billion would have been the employer share. The number (over 45 million) and

percent (about 42 percent) of workers who needed but did not receive retraining, and the

cost (over $45 billion) of that retraining are shown in Table 2. ICW programs may lessen

this shortfall through their accessibility and relative cost-effectiveness. A supply of already

prepared, available ICW programs could increase this possibility.

Table 2. Unmet Need for Retraining American Workers In 1989

Number (and Percent) of 0Workers Who Needed but Money Needed forDid Not Receive Training Retraining ($8)

Type of Retraining (CY 1989) (Employer Share)

New Technology Training

"* Technical Workers 9,255,270 (8%) $8.7

"* Skilled Workers 6,815,680 (6%) $6.5

Executive, Management, andSupervisory Training

5,490,374 (5%) $9.7

Customer Service Training 11,394,240 (9%) $5.4

Basic Sldls Training 16.559,243 (14%) $15.1

Total 49,514,807 (42%) $45.4

Over the past two decades the states have expanded their investment in training as a

way to attract new firms, to encourage resident employers to create new jobs, and to help

other resident employers to improve their business performance. Worker Training showed

that in 1989, 44 states supported one or more customized training programs. These

programs served a variety of goals, but the states reported an increasing demand for

11-6

training to upgrade already employed workers--for in-service training. Many of theseprograms relied on community colleges or vocational-technical institutes to deliver thetraining. There are numerous examples of community colleges providing customizedtraining for business, and it is likely that the states and their community colleges couldbenefit from the availability of DoD ICW oriented toward specific skills.

The U.S. economy will continue to support large numbers of low-skilled jobs inthe future, but, according to the Bureau of Labor Statistics (BLS), nearly all the new jobsthat will be created over the next 10 years will be in service industries. These new jobs willbe split fairly evenly between knowledge-based, high-skilled services (mainly involvinghealth, business services, and education) and traditional, low-skilled services (mainlyinvolving retail trade and personal services). Despite the rapid growth of knowledge-basedjobs, of the 10 occupations with the greatest number of new jobs that the BLS expects tosee created by the year 2000, only two require post-secondary school credentials for entry.However, all these jobs require training-especially upgrade training, which is infrequentlyprovided and is more likely to be given to workers with more rather than less previoustraining and education.

The problem of making substantial improvements in the skill levels of the Americanworkforce is complex and unlikely to be solved by any one action. However, it doesappear that transportable, quality-controlled training packages (such as those provided byICW programs) could help by making training both more efficient and more accessible.

11-7

III. DoD INITIATIVES

In accord with the Training Technology Transfer Act of 1988, the DoD hastaken a number of initiatives to increase the availability and transferability ofits ICW programs. These initiatives include: (1) issuance of MIL-STD-1379D to ensure the portability of ICW programs across many deliveryplatforms, (2) issuance of DoD Instruction 132220 and establishment ofthe Defense Instructional Technology Information System (DITIS) toimprove the development, management, and reporting of ICW programs,and (3) development of procedures for transferring ICW programs to theprivate sector. Procedures for transferring ICW programs that areaudiovis ual productions, such as interactive videodisc programs, are alreadyspecified, and similar procedures are being developed for transferring ICWprograms that are not audiovisual productions, such as computer-basedinstruction programs.

The courses presented by the DoD range in length from less than a day to more than

a year. The format of this instruction ranges from traditional platform lecture using

textbooks to correspondence courses to field exercises involving actual equipment and live

ammunition. The instructional technology used also spans a wide range and includes

lesson plans, study guides, workbooks, instructor guides, overhead transparencies,

textbooks, workbooks, programmed textbooks, audio cassettes, films, videotapes,

computer-assisted instruction, computer-managed instruction, interactive videodisc

instruction, CD-ROM instruction, part-task simulators, two-dimension simulators, high-

fidelity simulators, laser-simulated weapons fire, networks of simulators, and actual

equipment.

Increasingly, the DoD relies on interactive courseware (ICW) to support its

instrcional needs. There are several reasons for this:

(1) Stability. The development of movable type and books provided a capability tocapture the content of high-quality instruction and deliver it inexpensively tomany people. However, the effectiveness and efficiency of instruction vaniesif content is presented using different rules and procedures. In addition tocontent, computer technology captures the interactions of high-qualityinstruction and makes them widely, inexpensively, and reliably available tomany people. ICW programs cannot yet supply all the interactions of anindividual tutor working with an individual student, but they can capture asignificant portion of it. What interactions they do provide-along with the

I H-1

instructional content--can then be delivered in a stable, autonomous fashion tolearners, with a minimum of the variability that is characteristic of humaninstructors. 0

(2) Effectiveness. There have been enough reviews of the effectiveness ofcomputer-based instruction that a review of the reviews (Niemiec and Walberg,1987) has been published. It indicates an improvement of about 14 percentilepoints in student achievement through the substitution of computer-based 0instruction for more conventional approaches. A suggestion of theimprovements to be expected in the wider world of ICW using interactivemultimedia approaches were provided by Fletcher (1990), who foundimprovements of about 19 percentile points through the use of interactivevideodisc instruction. •

(3) Cost-Effectiveness. One significant measure of cost when students are beingpaid by the organization that trains them is time spent in training. A very stablefinding for lCW has been that it reduces training time by about 30 percent overmore conventional means (Orlansky and String, 1979; Fletcher, 1990). Asecond source of cost reductions through the use of ICW involves its ability tosubstitute simulation for actual equipment without adversely affecting studentachievement. Use of simulations that may cost a tenth or even a hundredth ofthe actual equipment used in military systems has provided a powerful costincentive for using lCW programs to train operators and maintainers of militarysystems.

Several actions to improve the management and use of ICW programs within theDoD have been initiated. These actions include the development of policies, technical

standards, and a new automated reporting system and data base that will improve the •sharing of materials both within the DoD and between the DoD and non-DoD activities.Many of these initiatives were undertaken jointly and cooperatively by the OASD(FM&P)

Readiness and Training Directorate and the OASD(PA) Defense Audiovisual Policy Office.They center on three areas: (a) ICW portability; (b) development and management of ICW; •

and (c) procedures for transferring 1CW.

A. ICW PORTABILITY

Interactive courseware (ICW) portability refers to the ability to operate the same 0software across many different hardware platforms. Portability (1) reduces the costs and

increases the usability of both computers and their software, (2) strengthens the market forboth, and (3) helps instit their use in many diverse application.

m-2

Without portability for ICW programs, routine and widespread use of this

promising technology is seriously handicapped. These handicaps include excessive cost,

difficulty of use, technological inertia, and unnecessary complexity in courseware design,

development, delivery, and implementation. These handicaps are removed by designing

interactive courseware to be portable.

Interactive courseware can be portable if developers use standard practices to create

it. Consequently, the topic of portability is also a topic of standards-military standards,

federal information processing standards, national standards, and international standards.

Implementation of these standards represents a significant opportunity and requirement forcooperation at all levels of technical development and across all communities concerned

with the use of ICW programs.

Portability means different things to different people. Transportability,

transferability, convertibility, and related terms are used to discuss the same concept. A

restricted, but straightforward definition has been articulated as "the ability to move an

application from one computer to another unchanged and get the same results" (Dahlstrand,

1984, p. 17). The DoD definition is not much different. DoD Instruction 1322.20 defines

portability as "the capability to run courseware and associated application programs without

modification on a delivery system other than the one for which they were originally

designed."

Courseware portability will increase the sharing of ICW within each Military

Service by providing ICW programs that can be used with little, if any, reprogramming

across a full range of within-Service instructional settings including initial skill training,

advanced skill training, garrison training, job-site training, officer education, and Reserve

Component training.

Courseware portability will also increase the sharing of ICW across the different

Military Services. All the Services teach basic courses such as electricity, electronics,

hydraulics, and motor vehicle repair. It is not unreasonable to suggest that materials for

these courses, or some portion of them, be usable by all the Services. This sharing is

neither an administrative nor political impossibility. Joint training is now offered in 62

inter-Service courses or skill areas (U.S. Dept. of Defense, Military Manpower Training

Report, 1991), and the need for it will increase as training budgets decrease.

Finally, portability makes the transfer of ICW from the DoD to the private sector

technically feasible and practical. It provides a foundation for the DoD to meet the

IM1-3

requirements of the Training Technology Transfer Act of 1988 with minimum impact on its

resources and maximum opportunities for transfer. 0

The objectives of the DoD portability effort are illustrated in the progression shown

in Figure la-c. Figure la illustrates the situation without portability. In this case,

application software must be uniquely developed for different operating systems and

hardware platforms. The interface between the authoring software and the operating 0

system is unique and usually proprietary for each combination of authoring software,

operating system, and hardware platform.

The current status of DoD portability is illustrated by Figure lb. It may be

described as system level portability. Its goal is to allow application software to run on 0

different operating systems and hardware platforms with minimal modifications to the

application. This goal is rarely reached to the point of requiring no modifications, but the

current scheme of portability minimizes these modifications and facilitates theirimplementation. The goal has largely been reached by the effort described here through 0

specification of a virtual device interface.

A more ambitious objective for portability is illustrated by Figure ic. This objective

may be called device level, or plug-and-play portability. Its goal is not only for application

software to run on different operating systems and hardware platforms with no

modifications to the application (system level portability), but also to permit substitution of

hardware devices, perhaps supplied by different manufacturers, in hardware platforms with

no modifications required in either the application or the operating system software. This

goal has been achieved for high fidelity audio, and it can similarly be accomplished in the 0

ICW world through specification of a virtual device interface.

Work on the systems level portability illustrated in Figure lb has been completed

for ICW programs using the MS-DOS operating system through a combination of DoD and •

industry efforts and has been incorporated in Appendix D of MIL-STD-1379D, "Military

Training Programs," issued 5 December 1990.

MI-4

ICW ANDAUTHORING

APPLICATIONSOFTWARE

OPERATINGSYSTEM AND

DEVICEDRIVERS

DEVICES

Figure Ia. Standard Architecture Without Portability

F~gre b. tanardArcItctur WihSsemLvlPDtblt

Fiue c SadadArchitecture WihIvc-ee otblt

APLIATO

B. DEVELOPMENT AND MANAGEMENT OF ICW

MIL-STD-1379D, which includes the ICW portability standard, is the principal 0DoD standard for training. It establishes: (1) procedures to follow when developingtraining programs, including guidelines for writing contracts and delivery orders for

training; (2) requirements for using DoD Computer-aided Acquisition and Logistic Support(CALS) for training documents; and (3) requirements for using the virtual device interface 0

described here.

Additionally, DoD Instruction 1322.20, "Development and Management ofInteractive Courseware (ICW) for Military Training" was developed and issued on14 March 1991. The objective of this instruction is increased cost-effective use of ICW for 0military training, and it applies to all interactive courseware developed by or for the DoD.

The instruction sets five policies for the development of new and substantiallymodified ICW programs and the management of both new and existing ICW programs inthe DoD. These policies are the following: 0

(1) ICW programs are to be designed to promote portability, following thestandard DoD programming protocols developed under the DoD portabilityinitiative and other technical requirements prescribed in MIL-STD-1379D.

(2) Payment of royalties, recurring license or run-time fees, use taxes, or similar 0

additional expenses for ICW and associated materials developed for or by theDoD are to be eliminated.

(3) The Defense Instructional Technology Information System (DITIS) isestablished to provide and maintain an inventory of DoD ICW programs for 0use by all DoD components. DITIS is also significant for non-DoD workforceissues because it provides essential information for determining the potential ofDoD ICW programs for transfer to the private sector.

(4) Reproduction master materials must be archived for the life cycle of each ICW 0

program.

(5) DoD Components must ensure the availability of all materials necessary tomodify ICW programs throughout their fe cycles.

C. PROCEDURES FOR TRANSFERRING ICW 0

The DoD (as well as other Federal agencies) has provided many training andeducation audiovisual materials to the private sector through an existing structure within theNational Audiovisual Center, which is an operating division with the National Archives and

Records Administration, as outlined in the Office of Management and Budget Circular

111-6

0

A-114, "Management of Federal Audiovisual Activities." Since A-114 is already in place,

the DoD intends to use it as much as possible in transferring ICW programs to the private

sector. However, ICW programs that are not audiovisual productions are not covered by

these provisions, and procedures are being established so that ICW management, including

that concerned with transferring programs to the private sector, is consistent across all ICW

programs.

General procedures for transferring all DoD ICW programs are shown in Figure 2.

First, the programs must be screened for potential relevance to the new target audience on a

well-defined set of criteria. Those ICW programs that successfully meet these criteria must

then be reviewed by legal counsel to ensure that no legal encumbrances preclude their sale

to the public and then be subjected to a security review to determine if they can be cleared

for public release. ICW programs that pass these reviews will then be available for transfer

to the private sector. These programs will be offered directly as they were produced for the

DoD. The DoD cannot take responsibility for any costs associated with secondary user

desired content changes, editing, duplication, updates, shipping, technical conversion,

licensing, patents, or royalties associated with these cleared ICW programs.

FORE LSECUr CATALOG OUTLET

Figure 2. Materials Transfer Process

This process then involves three "filters" or reviews through which ICW programs

must pass:

(1) Suitability Review. The first question that arises about a DoD ICW programsis: Would anyone else want it? A substantial amount of DoD instructionalmaterial is specifically focused on the main business of the DoD, which iswarfighting. Much DoD instructional material concerns combat, the operation,maintenance, and deployment of materiel required for combat, and proceduresfor managing large military organizations. ICW programs directly concernedwith combat or with systems and procedures that are only used for combat areexcluded from further consideration. Some ICW programs may pass thissuitability review and still be too martial in tone for the taste of many civilianusers, but at this point the issue is left to these users to resolve for themselves.

E11-7

(2) Legal Review. The most restrictive review for DoD ICW programs to passmay be the legal review. Here the question is: Does the DoD have the legalright to provide a particular ICW program for copying and distribution byothers? In short, does the DoD own it? Many DoD ICW programs haverestrictions on their distribution outside the DoD. These restrictions are theresult of several factors: the software used in ICW programs may becopyrighted; imagery, music, and sound effects used in audiovisualproductions may be copyrighted; and distribution rights may be restricted toDoD users only. Clearly, the DoD does not have the right to copy anddistribute these materials outside the DoD.

In other cases, the distribution rights for the materials themselves areunlimited, but the means to present them may be limited. This is especiallylikely to be true of ICW programs in which instructional content the curriculumdata base of items, scenarios, graphics, photographics, audiovisualproductions, and the like is only part of the instructional package. The rulesand procedures for presenting and individualizing the instructional content are aseparate matter. The content may be wholly owned by the DoD with free andunrestricted rights for its distribution within the DoD, but the rules andprocedures for presenting it may involve proprietary algorithms, licensing,royalties, or other limitations to the rights of the DoD to transfer it to non-DoDusers.

(3) Security Review. Classified material or material that reveals businessproprietary information or trade or technology secrets cannot be transferred tothe private sector. Instructional material that could be transferred by the DoDto the private sector would have to be reviewed to determine if it could be madeavailable for public release and unlimited distribution outside the DoD.

DoDI 1322.20 is intended to minimize restrictions on the transfer of new ICW

programs and of those undergoing major revisions. However, this instruction does not

cover already existing ICW programs, and it will not sweep away all barriers to ICW

transfer. Problems of resourcing transfer procedures, establishing responsibilities, and

adapting to inevitable and rapid technological changes in ICW remain to be solved.

The general process for transferring DoD ICW programs to non-DoD users, then,

centers on the three reviews discussed above. All three reviews will certainly result in

new, non-mission-related costs that the DoD will have to bear and that will require

additional resources.

Distribution rights to ICW programs remain to some extent uncharted territory. An

adequate body of decisions for establishing legal precedents for the distribution of ICWprograms to outside users has yet to be accumulated. Clear criteria for developing ICW

11-8

programs with well-understood distribution rights and clear criteria for determiningdistribution rights for existing ICW programs have yet to be developed.

It is also uncertain what legal liability issues may be associated with transferringcertain ICW materials to non-DoD and private users. Potentially transferable materials mayhave to be legally reviewed in this context as well.

Once these reviews are completed, the instructional materials that are determined to

be transferable will be released for publication in a catalog. There are additional tasks thatmust be performed by non-DoD organizations to complete the transfer of these materials tocivilian users. Among these are the following:

"Catalog Publication and Dissemination. The catalog must be published anddisseminated to potential users. This task includes maintaining mailing listsand developing a capability for marketing the DoD materials. The catalog mustinclude materials newly entered, previous materials that have been updated,and materials whose distribution status has changed. The catalog should bemaintained in a digitally accessible format as well as in a paper format.

" Material Location and Retrieval. Transferable materials must be located andretrieved. ICW programs are scattered widely across DoD commands.Locating and securing complete, fully workable sets of ICW programs mayconsume substantial resources.

Material Storage. Transferred materials must be physically stored with allnecessary environmental safeguards. These materials must be maintained ingood working order, and updated and modified promptly by the organizationresponsible for marketing them. Software, hardware, and an in-housetechnical capability must be available in order to provide this quality assurance.

Technical Assistance. Non-DoD users will require technical assistance ininstalling the DoD materials for their own applications, and this assistance willhave to be provided via telephone hot lines, technical documentation, technicalbulletins, user group seminars, and the like.

Other tasks may well develop with experience. What is needed is an adequately

resourced materials transfer organization that is outside the DoD but that has clearresponsibility to market, technically support, and transfer DoD ICW programs to non-DoDusers. If the DoD materials and other government-developed ICW programs are ofsufficient value, such an organization should become self-supporting through sales of ICW

programs after receiving start up support, but adequate seed funding will have to beprovided and an organizational "home" for the center will have to be established.

M-9

IV. TRANSFERABLE DoD COURSEWARE

The DITIS database was analyzed to identify ICW programs that arecandidates for transfer from the DoD to the private sector, although finaldetermination of transferability will depend upon the results of moreauthoritative suitability, legal, and security reviews that must be undertakenon a case-by-case basis. Out of 4644 ICW programs that have beenreported to DITIS since its inception in May 1991, 2718 were identified ashaving possible utility in meeting private sector training requirements. Ofthese 2718 programs, 591 were computer-based instruction programs thatrun on the MS-DOS operating system, 1997 were computer-basedinstruction programs that run on the NOS operating system and wereprepared using the TUTOR authoring system, and 130 were interactivevideodisc programs that run on the MS-DOS operating system.

The DIS database is the only DoD database that will satisfy the informationreporting requirements of the Training Technology Transfer Act of 1988. DoDI 1322.20requires DoD training developers to determine if an ICW program exists that meets their

needs before embarking on the development of any new training program, and DITIS wasestablished by the instruction as a resource for making this determination. ICW programsmust be entered into D1TIS when they are proposed, put under development, completed,modified, taken out of service, or terminated. Full-scale development of the database wasbegun in May 1991 when the Services were formally notified of the requirement to register

their ICW programs in DITIS.

At present, 4644 records have been entered into DITIS and are available foranalysis. These records are now being reviewed, validated, and checked for completeness.Also, more DiS records are being added all the time. These activities are continuing andshould substantially increase both the completeness of DIS coverage and the quality of itsrecords. However, the 4644 ICW programs now in D1TIS provided an initial, credible

basis for determining the amount and kinds of DoD ICW programs that may be transferableto private sector use.

The Service developers of the ICW programs reported in DITIS are listed inTable 3. As the table shows, most of the ICW programs reported came from the Army

(2134 programs), but there are significant numbers of programs reported by the Navy (331

programs), Air Force (655 programs), and DoD (555 programs). Most of the DoD

IV-l

programs came from joint commands, and most of the programs for which no Service was

reported as the developer were acquired jointly, by two or more Services. Most of the

programs were developed by active components (2908 programs). Only 14 programs were

developed by reserve forces, none by National Guard forces, but the developing

component of many (1722) programs was not reported.

Table 3. Sources of ICW Programs Reported In DITIS 0

No. ofService (Component) Records

Army - (Active) 1380

Army - (No Response) 754

Total 2134

Navy - (Active) 320

Navy - (Reserve) 11

Total 331

Air Force - (Active) 638

Air Force - (Reserve) 3

Air Force - (Other) 14

Total 655

Marine Corps - (Active) 5

Coast Guard - (Active) 10

DoD - (Active) 555

No Response 954

Total 4644

Only computer-based instruction and interactive videodisc were reported in DTS

as delivery media for these ICW programs. A total of 3499 programs were reported as

computer-based insruction, 965 were reported as interactive videodisc programs, and there

were 180 non-responses in which the delivery media data element was left blanL

IV-2

The information held in DITS is determined by its data elements, which are definedby DoDI 1322.20. These data elements and their definitions are listed in Appendix A.Some characteristics of DoD ICW programs that may be determined from these data

elements as they have been reported to DITIS are shown in Table 4. It should be

emphasized that many DMTIS records are in their early developmental phases and may not

be authoritative. Further review must be made on a case-by-case basis to determineconclusively what the actual status of an ICW program listed in DITIS is with regard to any

of its data elements.

Table 4. Reported Characteristics of ICW Programs In DITIS

Distribution Restrictions Security Classifiration DoD Rights Program Status

No. of No. of No. of No. ofResponse Programs Response Programs Response Proram Response Prs

None 3095 Unclassified 4625 Unlimid 239 Proposed

Legal 53 Confidential 4 Umied 3900 Development 480

Government Only 121 Secret 11 Government Use 321 Completed 3983

DoD Only 962 No Responm 4 Other 4 Revision 20

Other 99 No Response 180 Out of Service 1

No Response 314 Terminated 2

Other 2

_ No _____ NoI I

Data elements that are particularly important in determining transferability are the

following:

"Restrictions to Diribution. If there are any legal restrictions on publicdistribution of an ICW program, then clearly, and in accord with theprocedures outlined earlier, the DoD cannot transfer the program to the privatesector. As Table 4 suggests, 3095 of the 4644 ICW programs now listed inDITIS have no reported legal restrictions on their distribution. Again,authoritative determination of these restrictions must be made, and many ICWprograms that are reported to be unrestricted may turn out, after furtherscrutiny, to be restricted.

" Security Clearance Required. If a security clearance is required to view anICW program, then its transferability is clearly limited. In accord with theprocedures outlined earlier, only unclassified ICW programs will be releasedby the DoD for use in the private sector. Security classification does notappear to be a major issue in the transferability of DoD ICW. As Table 4

IV-3

shows, 4625 of the 4644 ICW programs now in DITIS are reported to beunclassified.

DoD Rights to ICW Programs. A specific stipulation in the FederalAcquisition Regulations determines the type of rights to a product held by thegovernment after acquiring it. DoD rights may turn out to be a major issuesince, as Table 4 shows, unlimited rights were reported to have beenpurchased for only 239 of the 4644 ICW programs in DITS as part of theiracquisition. In one sense, the 4405 programs for which the DoD does not own 0unlimited rights may not be transferable to the private sector, since the DoDdoes not strictly own them. On the other hand, there are no legal restrictions tothe distribution of 3095 of the DoD ICW programs in DITIS, and perhapsthese programs are transferable or the rights to these programs may bepurchased from the original supplier. This issue remains unresolved in general •and will have to be treated on a case-by-case basis in determining thetransferability of any ICW program. This problem should be alleviated in newICW programs that follow the guidelines established by DoDI 1322.20. Inthis report, ICW programs whose rights have not been purchased by thegovernment are nonetheless included in the inventory of programs that are •considered candidates for transfer to the private sector.

ICW Program Status. One factor that might be considered in the transfer of anICW program is whether or not it is operational. Proposed ICW programsmust be entered into D1TIS, but until they are actually completed, they cannotvery well be transferred. At present, 3983 of the 4644 ICW programs now 0listed in DrHS have been completed. An additional 480 programs are underactive development, which means they are likely to be completed, and 20programs are being revised, giving 4483 ICW programs that are eithergenuinely operational or are likely to be. The ICW programs that areinventoried here, however, are all reported in DITIS as being "completed."

Authoring System Product Name. An authoring system is a specializedsoftware package designed to help developers prepare instructional material forpresentation as ICW. ICW prepared with some authoring systems require aruntime version of the system to be used in presenting the instruction tostudents. In some of these cases, the DoD may have full rights to distribute the 0instructional materials, but the necessary runtime version of the authoringsystem must be obtained separately if transfer of the ICW is to be fullyeffected. The infonmation in DITIS does not identify these cases, and they areexpected to be minimized in the future as DoDI 1322.20 takes hold. Thenumber of ICW programs prepared for DoD use with each authoring system is 0shown in Table 5. As the table shows, TUTOR was used to prepare 2474 ofthe 4644 ICW programs in DITIS, and MicroTUTOR was used to prepare 584others. TUTOR programs require a proprietary computer operating system(NOS) and architecture for their presentation. MicroTUTOR programs can bepresented by most MS-DOS systems. It is likely that many of the TUTOR 0

IV-4

0

Table 5. Authoring Systems Used to Prepare ICW Programs In DITIS

Authorng Systemor Higher Order Number ofLanauage Used ICW Prorams

ADAPT 95

AIS-lL'PC 5

ASET 17

Authology 2

C 27

C++ 1

CAMS 1

CBESS 5

CDS 57

CDTS 17

DXTER 15

EIDS-Assist 518

GW Basic 2

ICON Author 1

IVDL 3

Lunarla 8

MERLIN 31

MicroTUTOR 586

OASYS 22

OASYS/PILOT 169

Pascal 2

PILOT 3

PMS 1QUEST 128

SABER 12

SAM 1

Teacher 1

TENCORE 33

TUTOR 2474

VISION 21

WISE 23

WYSE 53

No Reuonse 310

Total 4644

IV-5

programs could be successfully translated to MicroTUTOR, and many of theDoD TUTOR-based ICW programs may be of interest to the private sector, butthis translation is not a fully automated process and would have to beresourced.

"Operating System. An operating system is a computer program that schedulesaccess to a computer system's resources, increases the efficiency with whichapplication programs run, and makes using the computer system easier andmore convenient for programmers and other users, including students. The 0operating system on which an ICW program runs affects the ease with whichICW can be transferred. If the operating system is widely used, "open," andgenerally available, cost-effective transfer is more likely to be achieved than ifit is not. Two operating systems dominated the DITIS reports: MS-DOSsupporting 1792 ICW programs and NOS supporting 2474. NOS runs on 0mainframe computers manufactured by one supplier, MS-DOS runs on avariety of computer systems. NOS supports the TUTOR authoring system;MS-DOS supports a variety of authoring systems, including MicroTUTOR.

" Delivery Medium. Although DoDI 1322.20 lists eight possibilities for this dataelement, only two delivery media were reported in the current DITIS records. 0These are computer-based instruction, which was reported for 3499 ICWprograms, and interactive videodisc instruction, which was reported for 965ICW programs, 180 DTIS records did not report this data element. With time,the number of records reporting CD-ROM as a delivery medium shouldincrease, but none has been reported yet. 0

Potentially transferable programs are reported in three categories: (1) MS-DOS

programs using computer-based instruction for their instructional presentations, (2) NOS

programs using computer-based instruction for their instructional presentations (developed

using TUTOR), and (3) MS-DOS programs using interactive videodisc (IVD) for their 0inst ional presentations.

MS-DOS computer-based instruction is the most easily transferable of DoD ICW

programs because of the provisions of DoD 1322.20, the requirements of MIL-STD-

1379D, and the widespread availability of MS-DOS computer systems.

TUTOR computer-based ICW programs were examined as a group because of their

large number and general applicability to private sector concerns. However, they require

an operating system and a computer system architecture that is not widely available to either

military or private education and training communities. TUTOR programs can be and have 0

been prepared to operate on MS-DOS systems, usually by translating them toMicroTUTOR. Ibis translation can be 80-90 percent automated, and it is available from a

number of suppliers, making this investment attractive from a cost-benefit standpoint for

many TU'TOR progams 0

IV-6

Finally, interactive videodisc (VD) programs were separated from the others

because they can be treated as audiovisual materials, and there already exists a DoD

procedure for releasing audiovisual materials including interactive videodisc ICW programs

for sale to the public through the National Audiovisual Center. Also, IVD materials were

separated out because they require a different suite of presentation hardware (they require a

computer system connected to a videodisc player), and the status of many of theseprograms with regard to distribution rights and copyright is less certain than it is for mostcomputer-based instruction.

An assessment was made of the ICW programs in each of these three categories todetermine their potential for transfer. More specifically, the screenings suggested earlierwere applied to the 4644 ICW programs now reported in DITIS. ICW programs that dealtonly with military-specific equipment or procedures, course management tools withoutaccompanying training packages, and non-training computer programs and applications

software were omitted. Following this initial screening, the remaining ICW programs werescreened to select those that provide instruction in a competency, job, or vocational area

that may be needed in the private sector.

In order to characterize and better describe the final set of ICW programs, they wereclassified into four general categories based on their content:

"Basic Literacy and General Education covers topics in general education,including all basic verbal and mathematical literacy skills, preparation foreducation in specific subject areas, preparation for educational certificationtests, learning skills (especially learning to learn), and personal developmentskills necessary to function as an adult in modern society.

" Specific Technical Training covers ICW programs concerning instruction in theknowledge and skills needed to perform non-professional jobs, i.e., jobs thatdo not require education in the general liberal arts or sciences or advancedstudy in a specialized field. Most of these programs are intended to preparestudents to perform a specific, identifiable job.

Workplace Knowledge and Skills covers ICW programs providing instructionin knowledge and skills necessary to direct and administer groups of people ororganization procedures.

Professional and Para-Professional Knowledge and Skills covers ICWprograms providing instruction to maintain, update, and improve vocation-specific knowledge and skills related to specialized fields requiring advancededucation and training.

IV-7

The categories used to classify DoD ICW programs do not map exactly onto the

five categories of competencies sought by managers in the private sector. This is not

surprising since the DoD ICW programs were developed for specific DoD needs.

However, there are areas of overlap between the DoD categories and the workforce

competencies sought by the private sector. Specifically, the five private sector competency

areas can be mapped onto the DITIS inventory in the following ways:

Personal Characteristics and Attitudes. This competency is addressed by theDITIS Basic Skills Training and General Education category of "PersonalDevelopment," which covers the practical skills needed for daily living. Inaddition, the General Workplace Knowledge and Skills category contains thesubcategory "Personal Job Skills," which presents non-task-related knowledgeand skills that generally enhance workers' effectiveness on the job and withinan organization.

Interpersonal and Teamwork Skills. These competencies relate to two DITISsubcategories. The General Workplace Knowledge and Skills subcategory"Leadership" presents principles for supervising others in the workplace inorder to increase task efficiency and performance, and the subcategory"Teamwork" covers procedures for enhancing cooperative efforts to complete atask or job.

" Higher Order Thinking Skills. These skills are covered in the Basic SkillsTraining and General Education subcategory of "Academic Skills," which 0includes learning strategies (e.g., listening, memorizing, note taking, studying,test taking, reading strategies) and reference skills (e.g., finding information inthe library, finding information in books, skimming for relevant information,organizing information from multiple sources). Also the D1TIS inventoryincludes the subcategory "Higher Order Skills," which includes problemsolving (especially the application of knowledge and skills to novel situations)and spatial relations (mental manipulation of two- and three-dimensionobjects).

" Basic Academic Skills. This competency is directly addressed by the DITIScategory Basic Skills Training and General Education. It includes language 0skills (e.g., general language skills, applied language skills, generalvocabulary, job-related terminology, English as a second language) andmathematics skills (general mathematics skills, applied mathematics skills, andadvanced mathematics skills). This general category also covers topics relatedto physical sciences, social sciences, and foreign language and culture. •

" Job-Specific Technical Skills. These competencies are directly addressed byalmost every subcategory in the DITIS Specific Technical Training category aswell as by many subcategories in the Workplace Knowledge and Skills and inthe Professional and Para-Professional Knowledge and Skills categories.

IV-8

A. MS-DOS COMPUTER-BASED INSTRUCTION

Of the 4644 ICW programs now in DITIS, 1800 were reported as being prepared

for operation on the MS-DOS operating system. Of these, 854 were reported as computer-

based instruction and the remaining 946 were reported as interactive videodisc instruction.

The final set of MS-DOS computer-based instruction entries was selected based on the

following criteria:

"* As suggested, the program is reported to be MS-DOS computer-basedinstruction.

"* No restrictions are reported on its distribution to the public.

"* The development or acquisition of the program has been completed.

"* Its security classification is reported as "Unclassified."

"* There may be a need in the private sector for the competency, job, orvocational area for which it provides instruction.

There were 591 of the 854 MS-DOS computer-based instruction programs that met

these criteria. An inventory that shows into which subject categories these programs may

be categorized is provided in Table 6. It should be emphasized that the counts shown in

Table 6 are based solely on data currently contained in D1TIS and would probably change

upon closer review of the programs themselves. Still, this inventory gives a sense of the

variety and number of DoD MS-DOS computer-based instruction programs that may be

applicable outside the DoD. With the exception of a large cluster (263 programs) of MS-

DOS computer-based instruction programs for air traffic control and the general absence of

these programs from the General Education subcategory, they appear to be fairly evenly

distributed across all the subject categories.

B. NOS (TUTOR) COMPUTER-BASED INSTRUCTION

Of the 4644 ICW programs now in DITIS, 2472 were reported as being prepared

using the TUTOR authoring language for operation on the NOS operating system. The

final set of TUTOR computer-based instruction entries was selected based on the following

criteria:

"* As suggested, the program is reported to be computer-based instructionprepared using the TUTOR authoring language for the NOS operating system.

"* No restrictions are reported on its distribution to the public.

"* The development or acquisition of the program has been completed.

IV-9

Table 6. Counts of Candidate ICW Programs

BASIC SKILLS TRAINING AND GENERAL EDUCATION

Number of Programs

Subject Category Total MS-DOS IVD TUTOR

A. Basic Skills1. Academic Skills

a. Learning Strategies 36 4 32b. Reference Skills 36 36

2. General Job Skillsa. Charts, Diagrams, and Schematics 13 13b. Tool and Equipment Use 56 3 8 45c. Miscellaneous 15 15 0

3. Higher Order Skillsa. Problem Solving 34 34b. Spatial Relations 30 30

4. Language Skillsa. General Language Skills 15 15b. Applied Language Skills 30 2 28c. General Vocabulary 23 23d. Job-Related Terminology 79 18 61e. English as a Second Language 1 1

5. Mathematics Skillsa. General Mathematics Skills 86 1 3 82b. Applied Mathematics Skills 76 10 66c. Advanced Mathematics Skills 58 1 57

6. Personal Development 20 20

B. General Education1. Test Preparation 0

a. AFCT 6 6b. CLEP 6 6c. GEDd. GRE 5 5

2. Physical Sciencesa. Basic Science 3 3b. Biochemistryc. Biologyd. Chemistry 4 4e. Geology 4 2 2f. Physics 2 2

3. Social Sciencesa. Behavioral Sciencesb. Economicsc. Geographyd. Political Science

(continued) 0

IV-10

'v10

Table 6. Counts of Candidate ICW Programs (continued)

BASIC SKILLS TRAINING AND GENERAL EDUCATION (continued)

Number of Programs

Subject Category Total MS-DOS IVD TUTOR

B. General Education (continued)3. Social Sciences

a. Behavioral Sciencesb. Economicsc. Geographyd. Political Science

4. Foreign Language and Culturea. Arabic 11 11b. Chinese 4 4c. Czech 9 9d. Egyptian 1 Ie. French 4 4f. German 32 1 31g. Greek 1 1h. Hebrewi. Italian 6 6J. Japanese 21 21k. Korean 2 1 1I. Latin 4 4m. Norwegiann. Polish 5 5o. Portuguese 39 39p. Russianq. Serbo-Croatian 1 1r. Spanish 47 1 46s. Swedisht. SyrIanIraqi 1 1u. Tagalogv. Thaiw. Turkish 3 1 2x. Vietnamese 1 1y. Miscellaneous 5 5

Total Basic Skills Training andGeneral Education ICW Programs 835 41 16 778

(continued)

IV-I1

Table 6. Counts of Candidate ICW Programs (Continued)

1I. SPECIFIC TECHNICAL TRAININGNumber of Programs

Subject Category Total MS-DOS IVD TUTOR

A. Automotive Mechanics 48 1 47

B. Aviation1. Aerodynamics 5 52. Aeronautics and Astronautics3. Air Traffic Control 270 263 74. Aircraft Familiarization 6 5 15. Aircraft Maintenance 36 1 1 346. Emergency Procedures 5 57. Flight Crew Training8. Flight Inspection 37 379. Flight Planning10. Flying Fundamentals 11 1 1011. Ground Training 2 212. Helicopter Repair 3 313. Instrument Landing Systems (ILS) 46 4614. Pilot Training 32 4 1 27

C. Communications and CommunicationsEquipment 22 2 2 18

D. Computers1. Data Communications 2 22. Data Processing 9 93. Fundamentals 20 4 164. Hardware 12 3 95. Military Computer Systems6. Operating Systems 1 17. Operations8. Programming Languages

a. Ada 5 5b. BASIC 2 2c. C 19 19d. COBOL 44 12 32e. ECLf. FORTRAN 5 5g. GMAPh. JCLI. MASMJ. PASCAL 1 1k. PROLOGI. RPG 1 1m. SQLn. STORM

9. Security 6 5 110. Software and Applications Programs 6 1 511. Word Processing 2 212. Miscelaneous 1 1

(continued)

TV-12

Table 6. Counts of Candidate ICW Programs (Continued)

II. SPECIFIC TECHNICAL TRAINING (continued)

Number of Programs

Subject Category Total MS-DOS IVD TUTOR

E. Electronics1. Fundamentals 77 31 462. Generic Electronic Equipment

Operation and Maintenance3. Safety 3 1 24. Tool and Equipment Use 39 14 25

F. Hydraulics and Pneumatics 15 1 2 12

G. Journalism1. Broadcast 3 32. Photo 7 73. Print 5 5

H. Maintenance (Miscellaneous) 4 2 1 1

I. Map Reading 33 1 1 31

J. Materials Handling and Storage 5 1 1 3

K. Navigation1. Air 32 12 202. Celestial 1 13. Nautical 1 14. Rules of the Nautical Road 2 2

L. Postal Procedures 7 3 4

M. Radar and Sonar 106 43 3 60

N. Safety 30 1 29

0. Security, Law Enforcement,and Criminology 2 1 1

P. Soldering and Welding

0. Typing 23 12 11R. Miscellaneous 6 1 1 4

Total Specific Technical TrainingICW Programs 1060 391 69 600

(continued)

IV-13

Table 6. Counts of Candidate ICW Programs (Continued)

IN. WORKPLACE KNOWLEDGE AND SKILLS

Number of Programs

Subject Category Total MS-DOS IVD TUTOR

A. Finance and Accounting 34 2 32

B. Instruction and Training 39 3 36

C. Management1. Teamwork 18 1 172. Leadership 6 2 43. Performance Appraisal 9 1 a

D. Office Management1. Document Control 2 22. Administration 10 9 13. Secretarial Skills

a. Correspondence 16 5 11b. Filing 31 31c. Other Forms and Procedures 8 7 1

E. Personal Job Skills 10 10

F. Personnel Policies and EmployeeRelations1. General 18 3 152. Government-specific 35 7 283. Equal Employment Opportunity 29 29

and Affirmative Action

G. Miscellaneous 5 1 4

Total Workplace Knowledge and Skills

ICW Programs 270 42 1 227

(continued)

IV-14

Table 6. Counts of Candidate ICW Programs (Continued)

IV. PROFESSIONAL AND PARA-PROFESSIONAL KNOWLEDGE AND SKILLS

Number of Programs

Subject Category Total MS-DOS IVD TUTOR

A. Engineering1. Civil Engineering (General) 4 2 2

a. Bridge Design 32 11 21b. Concrete 6 3 3c. Construction Management 3 1 2d. Drainage Facilities 16 6 10e. Materials Quality 11 5 6f. Road and Airfield Construction 31 12 19

2. Combat Engineering3. Electrical Engineering 1 14. Mechanical Engineering 2 25. Production Control Management 14 14

B. Medicine1. Anatomy and Physiology (General) 55 16 8 312. Basic Medical Sdills 18 13 53. Combat Medicine 4 44. Dental Science 27 5 3 195. First Aid, Emergency Treatment,

and Medical Readiness 28 1 11 166. Health Care Administration 29 9 207. Infectious Diseases 8 88. Laboratory Technology 10 109. Medical Technology 36 2 3410. Nursing 12 1211. Nutrition and Wellness 21 3 1812. Pharmacology 19 9 1013. Public Health 4 414. Medical Specialties

a. Allergies 9 9b. Behavior/Psychiatry 4 1 3c. Cardiology 20 5 1 14d. Clinical Medicine 5 1 4e. Dermatology 2 2f. Ear. Nose and Throatg. Endocrinology 5 2 1 2h. Neurology 5 2 3I. Obstetrics/Gynecology 2 2I. Oncology 2 2k. Optometry/Ophthalmology 32 12 20I. Orthopedics 2 2m. Pulmonary Medicine 39 2 1 36n. Radiology 8 8o. Surgery 14 5 9p. Urokxw 3 1 2

(continued)IV-15

Table 6. Counts of Candidate ICW Programs (Continued)

IV. PROFESSIONAL AND PARA-PROFESSIONAL KNOWLEDGE AND SKILLS (continued)

Number of Programs

Subject Category Total MS-DOS IVD TUTOR

B. Medicine (continued)15. Veterinary Medicine 5 516. Miscellaneous 5 2 3Total Professional and Para-ProfessionalKnowledge and Skills ICW Programs 553 117 44 392

Overall Totals 2718 591 130 1997

* Its security classification is reported as "Unclassified."

* There may be a need in the private sector for the competency, job, orvocational area for which it provides instruction.

There were 1997 of the 2472 TUTOR computer-based instruction programs that

met these criteria. An inventory that shows into which subject categories these TUTOR

programs may be categorized is provided in Table 6. As above, the counts shown in Table

6 are based solely on DITIS entries and would probably change given closer examination 0of the programs. Still, this inventory gives a sense of the variety and number of TUTOR

computer-based programs that may be applicable outside the DoD. There is a TUTOR

program in almost every subject category listed. As discussed earlier, commercial

programs available from several suppliers will automate the translation of 80-90 percent ofthese TUTOR programs to MicroTUTOR and thereby to a form that can be presented on

most MS-DOS computer systems. Roughly, 1-3 person-weeks are required to translate a

TUTOR program to MicroTUTOR.

C. MS-DOS INTERACTIVE VIDEODISC INSTRUCTION

Of the 1800 ICW programs in DITIS reported to operate on the MS-DOS operating

system, 946 use interactive videodisc (IWD) instruction. The final set of MS-DOS IVD

instuction programs was selected based on the following criteria: S

* As suggested, the program was reported to be interactive videodisc instructionprepared for the MS-DOS operating system.

The development or acquisition of the program has been completed.

Its security classficaton is reported as "Unclassified."

IV-16

0

There may be a need in the private sector for the competency, job, orvocational area for which it provides instruction.

Lack of reported restrictions for distribution to the public was not among the criteria

for selecting ICW programs in the category of MS-DOS IVD instruction. Many of these

programs are new, and review of their distribution rights is currently underway. It seems

fair to identify these ICW programs as candidates for transfer as long as it is recognized

that their distribution rights are still being determined and may turn out to be restricted.

There were 130 IVD instruction programs that met the criteria for inclusion in the

Table 6 inventory of ICW programs that are candidates for transfer. The percentage

(14 percent) of IVD programs, then, that met these criteria is much lower than the

corresponding percentages for MS-DOS computer-based instruction (69 percent) and

TUTOR computer-based instruction (81 percent). This relatively low percentage appears to

be due to the large number of IVD programs the DoD has developed to train operators and

maintainers of specialized military systems and are therefore not candidates for transfer to

the private sector. Table 6 shows into which subject categories the IVD programs that are

candidates for transfer may be categorized. As before, the counts shown in Table 6 are

based solely on D1TIS entries and would probably change given closer examination of

these ICW programs. Still, this inventory gives a sense of the variety and number of DoD

IVD programs that might be applicable in private sector training.

IV-17

V. CONCLUSIONS

Based on the above considerations and the resulting inventories of DoD ICW

programs that appear to be transferable to the private sector, the following conclusions may

be drawn:

A number of DoD ICW programs appear to be candidates for transfer to theprivate sector, but final judgements on their transferability require moreauthoritative and comprehensive reviews that must be performed on a case-by-case basis. Many of these ICW programs have proven to be of significantvalue in DoD training and may be useful in meeting private sector workforcetraining requirements if they can be transferred.

" Workable procedures are already in place to aid in the transfer of DoD ICWprograms that are audiovisual productions, such as interactive videodiscprograms. These procedures serve as a model for effecting transfer of otherICW programs that are not audiovisual productions, such as computer-basedinstruction programs.

" The problems of resourcing and responsibilities remain. Through its policiesand procedures, the DoD is prepared to assist the private sector in making DoDICW programs available for transfer, but resources both within the DoD andoutside the DoD remain to be identified and responsibilities for many necessaryfunctions, especially those to be performed outside the DoD, remain to beassigned.

In brief, transfer of ICW programs from the DoD to the private sector appears to be

a potentially useful approach for meeting some non-DoD workforce training requirements.

Actual transferability of specific ICW programs must be determined authoritatively on a

case-by-case basis, and a number of issues, particularly those involving the availability of

resources and the establishment of responsibilities, must be resolved before a

comprehensive set of procedures for transfer can be developed and successfully

unpiementec.

V-I

VI. REFERENCES

Bishop, J. (1989). Achievement, Test Scores, and Relative Wages. Conference paper,Washington, DC: American Enterprise Institute for Public Policy Research.

Blaiwes, A.S., and Weller, D.R. (1978). A Computerized Evaluation and TrainingSystem (CETS) for Recruit Training Commanders: An Overview. (ReportNAVTRAEQUIPCEN IH-307). Orlando, FL: Naval Training Equipment Center.

Carnevale, A. P., and Gainer, L. J. (1989). The Learning Enterprise. Alexandria, VA:The American Society for Training and Development.

Carnevale, A.P., Gainer, L.J., and Meltzer, A.S. (1990). Workplace Basics: TheEssential Skills that Employers Want. San Francisco, CA: Jossey-Bass.

Dahlstrand, I. (1984). Software Portability and Standards. Chichester, United Kingdom:Ellis Horwood, Ltd.

Fletcher, J.D. (1990). The Effectiveness of Interactive Videodisc Instruction in DefenseTraining and Education (IDA Paper P-2372). Alexandria, VA: Institute for DefenseAnalyses.

Fletcher, J.D., Bosco, J., Wienclaw, R., Ashcraft, J.K., and Boycan, G. (1991).Assessment of Department of Defense Job Skill Enhancement Programs (IDA DocumentD-939). Alexandria, VA: Institute for Defense Analyses.

High Schools and the Changing Workplace. The Employers' View. (NTIS Report PB84-240191). (1984). Washington, DC: National Academy Press.

Johnston, W.B., and Packer, A.H. (1987). Worlforce 2000: Work and Workers for the21st Century. (Publication No. HI-3796-RR). Indianapolis, IN: Hudson Institute.

Mehrens, W. (1989). Michigan Employability Skills Employer Survey. (Technicalreport.) East Lansing, MI: Michigan State University.

National Center on Education and the Economy. (1990). America's Choice: High Skillsor Low Wages! (Report of the Commission on the Skills of the American Workforce).Rochester, NY: Author.

Niemiec, R., and Walberg, H.J. (1987). Comparative effects of computer-assistedinstruction: A synthesis of reviews. Journal of Educational Computing Research, 1,19-37.

O'Neil, H.F., Jr., Alred, K., and Baker, E.L. (1992). Measurement of WorkforceReadiness Competencies: Design of Prototype Measures. Sherman Oaks, CA: AdvanceDesign Information.

VI-1

Orlansky, J., and String, J. (1979). Cost effectiveness of computer-based instruction inmilitary training (IDA Paper P-1375). Arlington, VA: Institute for Defense Analyses.

Schroeder, L.E., Dyer, F.N., Czerny, P., Youngling, E.W., and Gillotti, D.P. (1986). 0Videodisc Interpersonal Skills Training and Assessment (VISTA): Overview andFindings, Volume I (ARI Technical Report 703). Fort Benning, GA: U.S. ArmyResearch Institute Field Unit.

Senge, P.M. (1990). The Fifth Discipline: The Art and Practice of the LearningOrganization. New York, NY: Doubleday. 0

Taylor, F.W. (1916). The principles of scientific management. Bulletin of the TaylorSociety, December, 13-23.

U.S. Congress, Office of Technology Assessment. Worker Training: Competing in theNew International Economy (OTA-ITE-457). (1990). Washington, DC: U.S.Government Printing Office.

U.S. Department of Defense, Office of the Assistant Secretary of Defense for ForceManagement and Personnel. Military Manpower Training Report for FY 1992. (1991).Washington, DC: Author.

U.S. Department of Labor, Secretary's Commission on Achieving Necessary Skills. WhatWork Requires of Schools: A SCANS Report for America 2000. (1991). Washington,DC: U.S. Government Printing Office.

VI-2

APPENDIX A

DITIS DATA ELEMENTS

A-I

APPENDIX A

DITIS DATA ELEMENTS

SECTION A: ADMINISTRATIVE INFORMATION

This section is to be completed for a proposed development, or any time a DITIS

database query is initiated. The section consists of eight data elements organized into the

following four blocks:

1. D= The date this record is forwarded to DITIS. Entry format is complete

year, month, and day (e.g., 22 August 1995 is entered as 19950822).

2. D IS Record Number. The alphanumeric tracking number, developed by the

originating activity, that is used to identify the ICW program record within the DIS

database. The number consists of five parts, separated by hyphens, as follows:

C = One character code to denote the service of the lead

Component for the ICW program.

A= ArmyF= Air Force

N= Navy

M= Marine CorpsP = Coast Guard

D= Department of Defense

0= Other

S = One character code to denote the unit status of the activity

responsible for the development of the ICW programR = Active

G= GuardV= ReserveO= Other

A-3

NAME - First four letters of the last name of the Service program

manager, point-of-contact, or similar individual in the

originating activity.

YYYY'AMMDD = The date on which data regarding this program are first

entered into the DMS. Entry format is complete year,

month, and day. (e.g., 19950822). 0

A unique six digit number

3. ICW Program Status. Code to denote the current phase of the program for

which data are being submitted to DITIS:

Q - Query only

P - Proposed development

U - Under development

C - Development or acquisition completed

R - Program revision

S - Out of service

T - Program terminated

0 - Other (specify)

4a. Originating Activity Organization Name. The full name of the Government

activity responsible for the development of the ICW program (e.g., Chief of Naval

Education and Training; USA Training and Doctrine Command; USAF Air Training

Command). If the current data entry is part of a query, the full name of the Government 0

activity requesting the data search.

4b. Originating Activity Office Symbol. The full office symbol of the

Government activity responsible for the development of the ICW program (e.g., CNET,

Program Management Support Activity, Code 0474; HQ TRADOC Atn: AITG-C1; HQ 0

ATCTITOR). If the current data entry is part of a query, the full office symbol of the

Govarnment activity requesting the data search.

4c. Originating Activity Point-of-Cntact (POO Name. The name and title of the

person at the originating activity who is most knowledgeable about the program content and 0

features.

(1) Commecinl. The commercial telephone number, including area code,

of the originating activity (e.g., (407) 555-1234). 0

A-4

(2) AUIQYOQ. The AUTOVON telephone number of the originating

activity.

SECTION B: ICW PROGRAM INFORMATION

This section is to be completed for a proposed development, or any time a DITIS

database query is initiated. The section consists of eight data elements organized into four

blocks, and is used as the basis of the DiTIS database search for comparable products.The data elements are the following:

5. ICW Program Key Words/Concepts. A list of up to 10 key words orconcepts separated by commas that describes the general training area supported by the

ICW program (e.g., aircraft, radar, maintenance, troubleshooting, fault isolation, repair).These terms are the key concepts used to search the database for comparable products.

6a. TargeL Audience, A general description of the intended trainee populationwho will use the ICW program (e.g., all aircraft maintenance personnel; advanced sonartechnicians; E-9 load masters).

6b. Specialty Code(s). The specific specialty codes (i.e., Navy = NEC,Army/Marine Corps = MOS, Air Force = AFSC) of the target audience for which the ICWprogram was designed (e.g., 305X4). If the ICW program is not targeted at a specific

occupational specialty or group, enter N/A for "not applicable." This item will be included

in the Key concepts used to search the database for comparable products.

6c. Specialty Code Title(s). The corresponding titles of the occupational specialty

code(s) listed in Item 6b (e.g., Electronic Computer & Switching Systems Specialist).These terms will be included in the key concepts used to search the database for comparable

products.

6d. SiY•,il Leels). For each of the specialty codes listed in Item 6b, the target

skill level(s) to which the ICW program was written. If the program is targeted at more

than one skill level, this element lists the appropriate ones, separated by commas (e.g.,

3,5,7). If the program is appropriate for all skill levels within a specialty code, the word

"all" is entered. This item will be included in the key concepts used to search the database

for comparable products.

7a. ,-rtoal FWumnt Design= The short title of the DoD designator of

the operational equipment supported by the ICW program (e.g., F-14A; AN/ANO-64)

and/or the primary associated support equipment (e.g., oscilloscope) covered by the

A-5

program. If the ICW program does not deal with a piece of equipment, N/A is entered.

These terms will be included in the key concepts used to search the database for comparable

products.

7b. Operational Equipment Nomenclature. The complete nomenclature of the

DoD designator of the operational equipment supported by the ICW program (e.g., F-14

Fighter Aircraft; AN/ANQ-64 Radar Fire-Fighter Close Support), and/or the primary 0

associated non-generic support equipment covered by the program. If the ICW program

does not deal with a piece of equipment, N/A is entered. These terms will be included in

the key concepts us-J to search the database for comparable products.

8a. Associated Course Title(s). The Service/agency-assigned title(s) of the 0

training course(s) that the ICW program supports (e.g., AT&T 3B15 Computer

Maintenance). If the ICW program does not support a specific course, N/A is entered.

8b. Associated Course Identification Number(s). The Service/agency-assigned

course identification number of the training course(s) that the ICW program supports (e.g.,

ES-370). If the ICW program is used to support on-the-job, OJT is entered.

SECTION C: ICW PROGRAM DESCRIPTION

This section is to be completed within 30 days following Component-approval to

develop or fund the ICW program procurement. The section consists of 12 data elements

organized into the following five blocks:

9a. ICW Program Title. The Service/agency-approved name of the ICW 0

program.

9b. ICW Program Identification Number. The Service/agency-approved

identification number of the ICW program.

9c. ICW Program Version. The version number and/or date of the latest update 0

of the ICW program.

9d. ICW Program Summary. A brief narrative description (maximum 100

words) of the instructional contents of the ICW program. •

10. Estimated Total Hours of Instruction. The projected average contact hours

necessary for a trainee to complete the ICW program. Entry format is hours:minutes (e.g.,

a course that averages four hours and 15 minutes to complete is entered as (C-: 15).

A-6

1 la. Number of Courseware Units. The total number of discrete units of

courseware (i.e., lessons or modules) contained in the ICW program that can be taught,

measured, and evaluated as a single unit.

I1 b. Courseware Unit Numbers. The corresponding identification number for

each courseware unit in the ICW program. If the ICW program contains only one unit,

N/A is entered.

1 Ic. Courseware Unit Titles. The full title for each courseware unit contained

within the ICW program. If the courseware units do not have descriptive titles, a short

description of the contents of the unit is entered (e.g., Rules of the Road; Oscilloscope

Use.) If the ICW program contains only one unit, N/A is entered.

12a. Development of Acquisition Method. Code to denote the method by which

the ICW program has been acquired or developed:

J - Joint development or acquisition with another Service/agency

E - Development or acquisition based on an existing program.

I - In-house design and development (with or without contractor support)

C - Commercially available off-the-shelf program

0- Other (specify)

12b. D1TIS Record Number of Shared Program. The DITIS record number of the

ICW program of the other Service/agency if the current program is shared with another

Service or agency, or if it is based on ICW program materials received from another

Service or agency (i.e., if the answer to Item 12a is "J" or "E"). If this item is not

applicable to the current ICW program, N/A is entered.

12c. Use of Shared Program. Code to denote the extent to which the materials of

the shared program are used in the current ICW program. (If this item is not applicable to

the current program, N/A is entered).

W - Shared program used without modification

M - Minor changes made to shared program (e.g., title change, additionalcredits)

C - Content changes made to shared program (e.g., video reshot, additions ordeletions made to instructional content)

F - Format changes only made to shared program (e.g., different presentationmedium used)

I - Shared program materials only used as inputs to the development of thecurrent program.

A-7

13a. ICW Prograr Materials. A list of all training materials used to support the

objectives of the ICW program. Include all plans; control documents; discs, tapes, books,

job performance aids, and other support materials; and instructor guides and other user

documentation (e.g., 1 compact disc, 2 floppy discs, instructor guide, trainee workbook).

13b. Deliver medium, Code to denote the primary type of technology used by the

OCW program to present information.

IVD - Interactive Videodisc

IDD - Interactive Digital Disc

DVI- Digital Video Interactive

CBI - Computer-Based Instruction (stand-alone computer)

CDI - Compact Disc - Interactive

IVT - Interactive Videotape

PIT - Part Task Trainer

OTH - Other (specify)

13c. DoD Rights to ICW Program. Code that denotes the type of rights (per the

Federal Acquisition Regulation) that are owned by the Department of Defense for the ICW

program:

U- UnlimitedL - Limited

0 - Other (specify)

SECTION D: SOFTWARE DESCRIPTION

This section is to be completed within 30 days following the completion of ICW program

development or acquisition. The section consists of seven data elements organized into the

following two blocks:

14a. Authoring System Product Name: The official product name or trademark of

the authoring system, assembly language, or higher order language (HOL) used to develop

the ICW program courseware (e.g., Instructional Support System (ISS)).

14b. Authoring System Version: The version number and/or date of the authoring

system assembly language, or higher order language (HOL) compiler used to develop the

ICW program courseware (e.g., Version 2.1, December 1989). If the courseware was

developed using a HOL, the compiler version number is given.

A-8

14c. Authoring System Vendor: The full name and acronym (if appropriate) of thecommercial, academic, or Government developer of the ICW authoring system, assemblylanguage, or higher order language used to develop the ICW program courseware.

14d. DoD Rights to Developmental Authoriny.Sstm: Code that denotes the type

of rights (per the Federal Acquisition Regulation) that are owned by the Department ofDefense to the developmental portion of the authoring system used to develop the ICW

program.

U - Unlimited

L - LimitedSN- None

0 - Other (specify)

15a. Operating System Product NameNersion: The official product name ortrademark, including vendor and version, of the computer operating system software used

by the ICW program (e.g., Microsoft DOS 4.0).

15b. Operating System Software Drivers Required: A list of any special softwaredrivers that are required to run the ICW program but are not included as part of the trainingmaterials (e.g., mouse driver, ANSI.SYS, CD-ROM extensions). If no additionalsoftware drivers are required, N/A is entered.

16. DoD Portability Standards Version: The version number of the portability

standards in MIL-H-1379D used in the development of the ICW courseware. If thecourseware was developed before the initial issuance of MIL-H-1379D, N/A is entered

SECTION E: HARDWARE DESCRIPTION

This section is to be completed within 30 days following completion of ICW

program development or acquisition. The section consists of eight data elements organized

into the following five blocks:

17. Hardware Required: A list of the official product names or trademarks that

uniquely identify each of the hardware components and peripherals required to run the ICW

program (e.g., XY Model 201 videodisc player, RPQ Model 6012 CD-ROM interface

card).

18a. Minimum Free System Memory (RAM) Reauired: The minimum amount of

unused memory (RAM) that must be available on the system to run the ICW program (e.g.,425K).

A-9

18b. CPU L=: Type of CPU for which the ICW program was designed (e.g.,

8086, 8286, 68000).

18c. Minimum Hard Disk Space Reuuired: If the ICW program requires the use of

a hard disk drive, the minimum storage space that must be available on the hard disk drive

to run the ICW program (e.g., 2.8 MB).

19a. Floppy Disk L=: If the ICW program requires the use of floppy disks, the

physical size and capacity of the floppy disks for which the ICW program was designed

(e.g., 3.5 inch, 720K; 5.25 inch, 360K). If the ICW program does not require the use of

floppy disks, N/A is entered.

19b. Number of Flopy Disk Drives Required: If the ICW program requires the

use of floppy disks, the number of floppy disk drives necessary to run the program (e.g.,

2). If the ICW program does not require the use of floppy disks, N/A is entered.

20. Graphics Adapter Type Required: The type of graphics adapter board that

must be installed in the system to run the ICW program (e.g., VGA).

21. Inout Device(s): A list of all input devices that the ICW program wasdesigned to use (e.g., keyboard, mouse).

SECTION F: MANAGEMENT INFORMATION

This section is to be completed within 30 days following completion of ICW

program development or acquisition. The section consists of ten data elements organizedinto the following four blocks:

22a. Life Cycle Management Organization Name. The full name of the activityresponsible for the life cycle management of the ICW program, as designated by the

Component (e.g., Chief of Naval Education and Training; USA Training and Doctrine

Command; USAF Air Training Command).

22b. Life Cye Managme Office Symbol, The full office symbol of the activity

responsible for the life cycle management of the ICW program, as designated by the

Component (e.g., CNET, Program Management Support Activity, Code 0474; HQTRADOC Atn: ATG-C1; HQ ATC/'TOMR).

22c. Life Cycl Msanaga~t Point-of-contact (POC) Name. 11e name and title of

the person designated by the Component to be the life cycle manager for the ICW program.

A-10

22d. Life Cygcl Managent Telehone.

(1) • The commercial telephone number, including area code,

of the life cycle manager for the ICW program (e.g., (407) 555-1234).

(2) AUT..QV_ . The AUTOVON telephone number of the life cycle

manager for the ICW program.

23a. Security Clearance Required. Code that denotes the clearance level required

for an individual to view or discuss the ICW program.

U - UnclassifiedC - ConfidentialS - Secret

0 - Other (specify)

23b. Restrictions to Distribution. Code that denotes the release status of the ICW

program, based on the content of the program materials.

LR - Legal restrictions to public distributionNR - No legal restriction to public distribution

CP - Cleared for public exhibition but not distribution

CG - Cleared for Government distribution onlya) - Cleared for DoD distribution only

RD - Restrictions to DoD distribution

RF - NOFORN (i.e., not available to foreign nationals)

Or - Other (specify)

23c. Adaptability to Civilian Training, Code to denote the ease with which the

ICW program can be adapted for use in training in the civilian sector.W - Usable without modification

M - Requires minor changes (e.g., title change, additional credits)

C - Requires content changes (e.g., video reshot, additions or deletions toinstructional content)

I - Requires major changes or can only be used as inputs to developmentof civilian training

N - Not applicable to civilian training

0- Other (specify)

23d. Wlit= Articles and Se Lising If the ICW program

is available for the Military Assistance Program, enter the Military Articles and Services

A-11

Listing (MASL) number assigned by the Component. If the ICW program is not available

for the Military Assistance Program, N/A is entered.

24a. Courseware Develo NamL The full name and acronym (if appropriate) of

the contractor or Government agency responsible for developing the ICW program

courseware (e.g., Chief of Naval Education and Training (CNET); USA Training and

Doctrine Command (TRADOC); USAF Air Training Command (ATC)). If the ICW

program is a joint in-house/contractor development effort, the name of the contractor is

entered.

24b. Year Developed or Last Updated. The full calendar year that the ICW

program was developed or, if the program has since been updated, the full calendar of the

latest revision. This data element applies only if this is the record of a currently fielded

program that has not previously been entered into the DITIS database; otherwise, N/A is

entered.

25. Defense Automated Visual Information System (DAVIS) ProductionIdentification Number(s) (PINso: The PIN(s) assigned to the ICW program by DAVIS, if

the ICW program contains video. If the ICW program does not contain video, N/A is

entered.

A-12