54
Best Practices Handbook Best Practices for Advanced Distributed Learning Joint ADL Co-Laboratory David J. Daly, Ph.D. and Amy Scott, Instructional Design Specialist www.jointadlcolab.org

Best Practices in e-learning Handbook

Embed Size (px)

Citation preview

Page 1: Best Practices in e-learning Handbook

Best PracticesHandbook

Best Practices for AdvancedDistributed Learning

Joint ADL Co-LaboratoryDavid J. Daly, Ph.D. and Amy Scott , Instruct ionalDesign Special is twww.jointadlcolab.org

Page 2: Best Practices in e-learning Handbook

ACKNOWLEDGEMENTS

The development of this handbook has been a collaborative effort. We thankour professional colleagues, subject matter experts, and the sample of userswho helped shape our thinking and work.

Special thanks to those who have reviewed and edited our summaries oftheir research. Since this handbook is a work in progress, we will continueto rely on the good will and help of many other researchers and developersin the distance learning community.

We also want to thank Susyn Stecchi, Visual Information Specialist with theJoint ADL Co-Lab, for her graphics and layout which significantly improvedthe look and feel of this handbook.

Page 3: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • Best Practices Overview PAGE 1

Overview Best Practices Handbook

This handbook addresses best practices associated with the analysis, design, development, implementation, and evaluation of distance learning.

The handbook will support DoD acquisition managers, instructional designers, and developers with information, checklists, and job aids on important distance learning topics. Requirements information and “lessons learned” from current literature is synthesized and summarized providing “scannable” web-based readings downloadable from the Joint ADL Co-Lab website. Key information to be remembered is presented either with examples, graphics, or summary boxes in the left column on the page, and reinforced with either a checklist or job aid at the end of each chapter.

Page 4: Best Practices in e-learning Handbook

<This page left blank intentionally>

Page 5: Best Practices in e-learning Handbook

Table of Contents

Chapter 1 DISTANCE LEARNING ISSUES AND CHALLENGES

Chapter 2

NEEDS ASSESSMENT OR “GAP ANALYSIS” Chapter 3

NEEDS ASSESSMENT & TRAINING NEEDS ANALYSIS CASE STUDY

Chapter 4

JOB AND TASK ANALYSIS Chapter 5

WRITING FOR THE WEB

Page 6: Best Practices in e-learning Handbook

<This page left blank intentionally>

Page 7: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • Distance Learning Issues and Challenges PAGE 1

Distance LearningIssues and Challenges

While the use of the Internet poses some significant advantagesfor education and training, experts note some concernsregarding the wholesale adoption of distance learning as the newmedium for learning. In the past, students have beendiscouraged and dissatisfied when bad content was simplydelivered by a new technology. There were obviously othermajor design, development, and implementation mistakes madedue to a lack in understanding student needs, each of therequired media for learning objectives, and each of the technicalsupport required by the new technology driven solution.

Advantages and Limitations of the TechnologyIn “Web-Based Training: Advantages and Limitations” WallaceHannum groups the advantages of web-based training, WBT,into three major categories: logistical, instructional, andeconomic. Logistical advantages refer to those advantages thathave to do with the ease of distribution and use. Instructionaladvantages refer to those advantages that directly impact thequality and potency of instruction delivered via WBT. Economicadvantages refer to cost advantages of WBT. (Hannum in Web-Based Training; Khan, B. H.; 2001)

Advantages of Web-Based Training

LOGISTICALFlexible delivery oftrainingLearn any time, anyplaceDelivery to learner’sdesktop or notebookcomputerCross platformcompatibilityNo schedulingproblemsEasy distribution

Security throughregistrationLinks to other sitesEase of contentupdates

INSTRUCTIONALDelivery of multimedia

Learner control

Immediate delivery ofupdated programming

Variety of instructionaleventsCollaboration

Consistency

Support EPSS

ECONOMICLess costly thantraditional trainingReduces duplication ofeffortDoesn’t requireexpensive facilities

Requires less technicalsupportCan bill per use

Inexpensive, widespreaddistribution of materials

Page 8: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • Distance Learning Issues and Challenges PAGE 2

The limitations were grouped in similar categories. There aresignificant challenges in the development and implementation ofquality courseware balanced by the needs of the student and theconstraints imposed by distance technology.

Characteristics of Quality Web-Based Teaching andLearningThe American Distance Education Consortium (ADEC) identifiesguidelines for evaluating web-based courses. ADEC’s basicassumptions are:

The principles that lend themselves toquality face-to-face learning environmentsare often similar to those found in web-based learning environments.

With all forms of media converging to adigital platform, advanced educationaltechnology may include a variety oflearning environments and informationappliances.

While rapidly emerging technologies offerunlimited potential for virtual learningenvironments for both face-to-face as wellas distance learners, practical applicationof existing technologies may often provehighly effective for various audiences andobjectives.

Limitations of Web-Based TrainingLOGISTICALLimited bandwidthMultimedia slow inloadingChanged or eliminatedlinksDial-up connections tooslow, especially formultimediaDifficult to authenticatelearners’ work andexaminations

INSTRUCTIONALNo face-to-face interactionRequires more learner initiative

Limited formatting in currentbrowsersDifficult to monitor learner’sprogress

Hard to pick up where learners leftoff

Learners must know or learn web-based toolsWBT authoring systems not assophisticated as CBT authoringsystemsNot for all subjectsMuch WBT is repurposed printedpagesInstructors must use non-lineardesignMisplaced emphasis on thetechnology itself

ECONOMICLonger development timeUpfront funding

Costly for small numbersof learnersCostly for single shottraining

Page 9: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • Distance Learning Issues and Challenges PAGE 3

ADEC Principles for quality distance learning include:

• The learning experience must have a clear purpose withtightly focused outcomes and objectives. Web-basedlearning designs must consider the nature of content,specific context, desired learning outcomes, and thecharacteristics of the learner. Learner-centric strategiesinclude modular, standalone units that are compatible withshort bursts of learning. Learning modules may also beopen, flexible and self-directing.

• The learner is actively engaged. Active, hands-on,concrete experiences are highly effective. Learning bydoing, analogy, and assimilation are increasingly importantpedagogical forms. Where possible, outcomes should relateto real-life experiences through simulation and application.

• The learning environment makes appropriate use of avariety of media. Various learning styles are best engagedby learning outcomes. Selection of media may also dependon nature of content, learning goals, access to technology,and the local learning environment.

• Learning environments must include problem-based aswell as knowledge-based learning. Problem-basedlearning involves higher order thinking skills, such asanalysis, synthesis and evaluation, while knowledge-basedlearning involves recall, comprehension and application.

• Learning environments should support interaction andthe development of communities of interest. Learning issocial and sensitive to context. Learning experiences basedon interaction and collaboration support learningcommunities while building support networks to enhancelearning outcomes. Multiple interactions, group collaborationand cooperative learning may provide increased levels ofinteraction and simulation.

• The practice of distance learning contributes to thelarger social mission of education and training in ademocratic society. Changing mental models andconstructing new knowledge empowers learners andencourages critical thinking. “Knowledge becomes afunction of how the individual creates meaning for his or herexperiences; it is not a function of what someone else says istrue.” (Jonassen, D., et al, 1995)

Page 10: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • Distance Learning Issues and Challenges PAGE 4

U.S. Army Case StudyIn a paper judged as “best paper” at the 2000Interservice/Industry Training, Simulation and EducationConference (I/ITSEC), Millie Abell, Futures Training Division,U.S. Army Headquarters Training and Doctrine Command(TRADOC), asserts that distance learning courseware mustaddress the diverse needs of adult Generation X* and Y**learners. As adults, all Army personnel have unique learningneeds. These include:

1. A need to know why learning is required.2. A need to direct their learning.3. A need to contribute their experiences to the learning

situation.4. A need to apply what they have learned to solve real

world problems.5. A need to feel competent and experience success

throughout the learning program.

Secondly, many soldiers have additional learning requirementscharacteristic of Generation X. These include: a desire forindependent learning experiences that incorporate fast-pacedand visually intensive instruction; a need for frequent interactionswith corresponding feedback; and a strong desire to experiencea sense of accomplishment. Lastly, soldiers’ needs as distancelearners include the following: increased student-instructorfeedback since face-to-face communication is reduced or absent;frequent student interactions to prevent distance learners frombecoming overly passive; highly structured learning activities toensure distance learners do not lose track of where they are; andfrequent and effective visuals, since imaging is critical tolearning.

Despite the advantages that distance learning provides, thisapproach challenges trainers because learner motivation and, inturn, attention are harder to sustain. Motivation energizes thelearner and directs his attention. Attention, which is prerequisiteto learning, is sustained when learners are relaxed and alert tothe stimuli that the trainer wants learners to perceive. Thefollowing four barriers reduce distance learner motivation and, inturn, attention:

1. Distance learners’ feedback to the trainer is reduced.2. Distance learners become passive.3. Distance learners lose track of where they were within

the instruction.4. Distance learners are told, not shown.

These barriers were further described as indicated in thenext section.

Motivation energizes thelearner and directs hisattention.

Attention, which isprerequisite to learning,is sustained whenlearners are relaxed andalert to the stimuli thatthe trainer wants learnersto perceive.

* Generation X is definedas having been bornbetween 1965 and 1980.www.jour.unr.edu/outpost/specials/genx.overvw1,html

** Generation Y isdefined as having beenborn in 1978 or later; alsoknown as the baby boomecho.www.wordspy.com/words/GenerationY.asp

Page 11: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • Distance Learning Issues and Challenges PAGE 5

Distance Learning Barriers and RecommendedSolutions

1. Distance learners’ feedback is reduced. In both one-wayvideo teletraining and in audio-graphic instruction, forexample, the instructor cannot see the learners’ looks ofboredom or confusion, both of which cue the trainer tochange his/her instructional strategy. Moreover, thecommon fear a learner experiences when asking questionsin a traditional setting is magnified in distance learning whenwhat he fears might be dumb questions are telecast acrossthe country or are preserved electronically for worldwidedistribution.

With this reduction in feedback, three skills become far morecritical to trainers – Questioning, Listening, and Feedback.Feedback should be:

• Based on performance standards rather than onpersonal opinion.

• Followed by new action.• Quantitative, since numerical feedback indicates small

improvements that produce changes in learnerperformance in the long run.

• Prompt, but not necessarily as soon as possible.Designers must consider whether delay is beneficial,remembering that excessive delay decreases motivation,weakens the feedback, and increases learner anxiety.

• Frequent, since an accumulation of errors can reducemotivation. If multiple errors are established, thebehavior the feedback seeks to encourage is foreign tolearners.

• Positive, emphasizing correctness rather than mistakes.• Differential, indicating improvement the learner has

made since he was last evaluated. (Wlodkowski, 1993)

2. Distance learners become passive. With a targetpopulation heavily conditioned by passively watchingtelevision programs in which no overt reaction is expected,trainers must vigorously solicit distance learner involvementso that they will not generalize passive viewing todistance learning.

Trainers should engage distance learners within the first twominutes of instruction and sustain participation throughoutthe lesson. In general, trainers should involve learners atleast 30 percent of the time for synchronous instruction andup to 40 to 50 percent of the time for asynchronousinstruction. (Ostendorf, 2000) Periods of learner involvementshould be planned throughout the lesson and should include

With this reduction infeedback, three skillsbecome far morecritical to trainers –Questioning,Listening, andFeedback.

Trainers shouldengage distancelearners within the firsttwo minutes ofinstruction and sustainparticipationthroughout the lesson.

Page 12: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • Distance Learning Issues and Challenges PAGE 6

student-to-student interactions ... Moreover, research showsthat an effective way for learners to handle content is tosummarize in their own words what they have learned.Students who give such explanations show 30 percentincreases in achievement after learning, compared to20 percent gains for those who do not summarize orparaphrase. The more the student explains the content, thegreater the learning gain. (Howard, 2000) Finally, thedistance learner should follow each interaction with a periodof reflection and make it a habit to ask two questions: Whatis the most important thing I learned, and What am I going todo about it?

3. Distance learners lose track of where they are withininstruction. Synchronous distance learning technologiesoften present transient rather than persistent stimuli, where adistance learner, for example, can receive real-time videoteletraining or audio graphic instruction one moment, but itdisappears the next. When stimuli are transient, learnersbecome frustrated if they lose sight of the lesson’sorganization and, as noted earlier, their confusion andfrustration may go unnoticed in distance learning if feedbackto their instructor is reduced.

For this reason, trainers must craft clear organization into at-a-distance courses. Experts recommend six activities forstructuring distance learning in order to help students stay ontrack.

a) Trainers should begin lessons with context; i.e., tellinglearners what came before, where they are now, how thetask fits into combat (or the job), and what will come.

b) Incorporate pre-tests, since research has shown thatlearners who take pre-tests do better on finals than thosewho do not.

c) Incorporate redundancy into distance learning byinserting review periods after an hour, a day, a week, amonth, and after six months. This sequence hasincreased recall by 400 percent, and studies show that70 percent of what is learned can be forgotten within24 hours unless the learned attempts to remember theinformation. (Rose and Nicholl, 1998)

d) Require learners to mind-map the content. This is agraphic technique for helping them structure information,and focus on and understand relationships between asubject and its main themes. (Buzan, 1993)

e) Use a whole-part-whole presentation sequence becauseit results in 50 percent faster learning over a part-wholemethod. (Rose and Nicholl, 1998)

An effective way forlearners to handlecontent is tosummarize in theirown words what theyhave learned.

Learners becomefrustrated if they losesight of the lesson’sorganization.

Design compellingbeginnings andendings to lessons,and find ways toreinforce the middle ofinstruction.

Incorporateredundancy intodistance learning byinserting reviewperiods.

Page 13: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • Distance Learning Issues and Challenges PAGE 7

f) Design compelling beginnings and endings to lessons,and find ways to reinforce the middle of instruction.Research shows that recall is highest (95 percent) at thebeginning of a sequence, but is reduced (20 percent) inthe middle. (Pike, 1994)

4. Distance learners are told, not shown. Althoughinstruction presented in multiple modalities is the mostrobust, visual communication is usually the most importantsense during instruction because learners take in so muchmore information visually that through other senses.(Stolovich, 1999) We gain 75 percent of what we knowthrough vision, 13 percent through hearing, and 12 percentthrough smell-touch-taste. (Pike, 1994)

The instructional developer’s most important skill is the abilityto think visually, using both word-pictures and iconicrepresentations. A picture is more effective than wordsalone, and words and pictures together are six times moreeffective than words alone.

Army research shows that distance learners performstatistically and significantly better when trainers pay closeattention to the quality of courseware design and delivery.More recent media research from Stanford University showshow gender, image size, motion, and scene change affectlearner attention. Other considerations that trainers mustaddress when using electronic distance learningtechnologies include the following: use of color in computerdisplays; ways to minimize The Wayfinding Problem (i.e., thelearner loses track of where he is in the exercise); ensuringsystem response times are acceptable to learners; enablingstudents to acquire those interpersonal skills that facilitatecollaboration and community building; and limiting group sizeto that which is appropriate for the content and nature ofassignments.

Visual communicationis usually the mostimportant senseduring instructionbecause learners takein so much moreinformation visuallythat through othersenses.

Page 14: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • Distance Learning Issues and Challenges PAGE 8

Job Aid for Distance Learning

1. Consider both the advantages and disadvantages of thetechnology when planning distance learning activities.

2. Ensure that:• The learning experiences have a clear purpose with tightly

focused outcomes and objectives.• The learners are actively engaged.• The learning environment makes appropriate use of a

variety of media.• Problem-based learning is included.• Communities of interest are developed.• The overall training goal or mission is evident.

3. Ensure that the learners:• Know why learning the knowledge or skill is necessary.• Direct their own learning.• Contribute their experiences to the learning situation.• Apply what they have learned in solving real world

problems.• Feel competent and experience success throughout the

training program.4. Design and develop distance learning activities with:

• Fast-paced and visually intensive instruction to vigorouslysolicit distance learner involvement.

• Frequent interactions with corresponding feedback tokeep the student on a productive learning path.

• Clear organization to help students stay on track.• Visually intensive instruction because learners take in so

much more information visually than through the othersenses.

Page 15: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • NEEDS ASSESSMENT OR “GAP ANALYSIS” – Determining the Impact of Performance Problems PAGE 1

Needs Assessment or“Gap Analysis” --

Determining the Impact of Performance Problems

What is a Needs Assessment?Suppose you are assigned to a task to develop a web-basedcourse on avionics, maintenance, or team training. Why not justgo right to doing a Job, Task, or Content Analysis?

Past best practices indicate that this type of request is oftenmore complex than initially indicated. The performance gap isusually not entirely due to a training problem, and the solution tothe problem can require substantial funding. A proper needsassessment can present the need in the context of an overallmission requirement in support of the Service and Commander-in-Chief (CINC). A needs assessment can document and justifyorganizational solutions, and can lead to the development ofplanning objectives.

Needs AssessmentA needs assessment is the process used to identify anddocument a gap between the desired and actual unit or individualhuman performance, as well as determine the cause for the gap.Needs assessment can be reactive in identifying deficienciesbetween what exists and what is required. It can also identifypotential deficiencies between current and future requirementsas a result of changes in threat level, doctrine, organizationalstructure, leadership development, and materiel. A needsassessment provides a means to identify the gaps betweencurrent results and desired results – a comparison of “what is”with “what should be.” Needs assessment is designed tomaximize the use of resources in identifying and resolvingperformance deficiencies. Training should not be developed orrevised unless needs assessment determines that training is themeans to resolve the deficiencies and that the needs analysisshows there is a requirement. (MIL-HDBK-29612, Part 2A)

Training Needs AssessmentTraining needs assessment is the systematic study of a problemor innovation, incorporating data and opinions from variedsources in order to make effective decisions or recommendationsinvolving training solutions. Performance discrepancies or needsare defined as the differences between the desired performanceor knowledge or the optimals, and the current performance orknowledge, or actuals. The goal of the trainer or instructional

A needs assessment isthe process used toidentify and document agap between thedesired and actual unitor individual humanperformance, as well asdetermine the cause forthe gap.

Training needsassessment is thesystematic study of aproblem or innovation,incorporating data andopinions from variedsources in order to makeeffective decisions orrecommendationsinvolving trainingsolutions.

Page 16: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • NEEDS ASSESSMENT OR “GAP ANALYSIS” – Determining the Impact of Performance Problems PAGE 2

developer is to reduce or eliminate performance deficiencies. (AirForce Handbook 36-2235, Volume 6)

Perspectives on Needs AssessmentWhat results are we now getting? How do those compare withthose we should deliver? Needs assessments provide a processfor defining the gaps between current and desired results, andprovide the justification for identifying and choosing the ways toclose those gaps. Before selecting any intervention, whether it istraining, human resources development, restructuring, or totalquality management, a needs assessment provides the basicdata for assuring that solutions, once selected, deliver desiredresults. (Kaufman, R., A Needs Assessment Audit,www.onap.fsu.edu)

Another school of thought says that a needs assessment is asystematic exploration of the way things are and they way theyshould be. These “things” are usually associated withorganizational and/or individual performance.

Why design and conduct a needs assessment? We need toconsider the benefits of any human resource developmentintervention before we just go and do it:

• What learning will be accomplished?• What changes in behavior and performance are expected?• Will we get them?• What are the expected economic costs and benefits of any

projected solutions?

(Robert H. Rouda and Mitchell E. Kusy, Jr.,www.alumni.caltech.edu/~rouda/T2_NA.html)

Often overlooked as the first step in performance improvementprocess is the training needs assessment. A need is not a wantor desire. It is a gap between “what is” and “what ought to be.”The needs assessment serves to identify the gaps, and considerif the problem can be solved by training. The assessment is partof a planning process focusing on identifying and solvingperformance problems. Why conduct a training needsassessment?

• To determine what training is relevant to your employees’jobs.

• To determine what training will improve performance.• To determine if training will make a difference.• To distinguish training needs from organizational problems.• To link improved job performance with organizational goals

and the bottom line.

(Jeannette Swist, www.amxi.com/amx_mi30.htm)

Needs assessments providea process for defining thegaps between current anddesired results, and providethe justification for identifyingand choosing the ways toclose those gaps.

A needs assessment is asystematic exploration of theway things are and they waythey should be.

The needs assessmentserves to identify the gaps,and consider if the problemcan be solved by training.The assessment is part of aplanning process focusingon identifying and solvingperformance problems.

Page 17: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • NEEDS ASSESSMENT OR “GAP ANALYSIS” – Determining the Impact of Performance Problems PAGE 3

How to Conduct a Needs AssessmentRobert H. Rouda and Mitchell E. Kusy, Jr. describe Four Steps toConducting a Needs Assessment:

Step 1: Perform a “Gap” Analysis

The first step is to check the actual performance of organizationsand people against existing standards or to set new standards.There are two parts to this:

Current situation: Determine the current state of skills,knowledge, and abilities of our current and/or future employees.This analysis should examine our organizational goals, climate,and internal and external constraints.

Desired or necessary situation: Identify the desired or necessaryconditions for organizational and personal success. Thisanalysis focuses on the necessary job tasks/standards, as wellas the skills, knowledge, and abilities needed to accomplishthese successfully.

Step 2: Identify Priorities and Importance

The second step is to determine if the identified needs are real, ifthey are worth addressing, and specify their importance andurgency in view of our organizational needs and requirements.Some factors to consider are:

Cost effectiveness/cost avoidance: How does the cost of theproblem compare to the cost of implementing a solution?

Legal mandates: Are there laws requiring a solution? (Forexample, safety or regulatory compliance)

Executive pressure: Does top management expect a solution?

Population: Are many people or key people involved?

Customers: What influence is generated by customerspecifications and expectations?

Step 3: Identify Causes of Performance Problems and/orOpportunities

Now that we have prioritized and focused on criticalorganizational and personal needs, we will next identify specificproblem areas and opportunities in our organization. We mustknow what our performance requirements are if appropriatesolutions are to be applied. We should ask two questions forevery identified need:

• Are our people doing their jobs effectively?• Do they know how to do their jobs?

• Perform a “gap” analysis.• Identify priorities and

importance.• Identify causes of

performance problemsand/or opportunities.

• Identify possiblesolutions and growthopportunities.

Page 18: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • NEEDS ASSESSMENT OR “GAP ANALYSIS” – Determining the Impact of Performance Problems PAGE 4

Step 4: Identify Possible Solutions and GrowthOpportunities

If people are doing their jobs effectively, perhaps we should letwell enough alone. But if our people are not doing their jobseffectively, training may be the solution, if there is a knowledgeproblem.

Organizational development activities may provide solutionswhen the problem is not based on a lack of knowledge and isprimarily associated with systematic change. Theseinterventions might include strategic planning, organizationalrestructuring, performance management and/or effective teambuilding.

(Robert H. Rouda and Mitchell E. Kusy, Jr.,www.alumni.caltech.edu/~rouda/T2_NA.html)

Best Practices for Needs Assessment1. Identify measurable performance needs or performance

gaps.

Step 1: Determine individual’s and/or group’s requiredperformance in terms of measurable accomplishments.

Step 2: Determine individual’s and/or group’s requiredcurrent performance status vis-à-vis the required standardsestablished in Step 1.

Step 3: List the measurable gap in performance or needs bycomparing the required performance to the actualperformance.

1. Identify measurableperformance needs orperformance gaps.

2. Define the impact of thejob in the context oforganizational levels.

3. Define the targetemployee population ortarget audience.

4. Conduct a cost/benefitanalysis or return oninvestment analysis.

Page 19: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • NEEDS ASSESSMENT OR “GAP ANALYSIS” – Determining the Impact of Performance Problems PAGE 5

2. Define the impact of the job in the context oforganizational levels.

Micro: This level of needs assessment and planningincludes a concern for the cumulative contributions of: (1)organizational resources (or inputs), plus (2) the proceduresand methods (or processes) to be employed inorganizational activities, plus (3) the immediate results (orproducts) accomplished). Examples of micro needsassessment include:

• Courses completed.• Competency test passed.• Skill acquired.• Learner accomplishments.• Instructor accomplishments.

Macro: This level combines the micro level contributions forform what an organization can or does deliver (outputs) to itsexternal (outside of the organization) clients. This level ismade up of what an organization uses, does, and delivers toitself as well as to its external clients. Examples of macroneeds assessment include:

• Graduates.• Drop-outs.• Job placements.• Certified licensees.

Mega: This level combines the results (products andoutputs) of the macro level with a consideration of how usefulthe organization’s contributions (outcomes) are to its clients

Example of Gap Analysis Table

RequiredPerformance

$25,000,000 incommercial loansprocessed properlyper annum.

$25,000,000 ofsecured individualreal estate loans(mortgages)processed properlyper annum.

No rejections ofloans processed byany Real Estate LoanOfficer.

CurrentPerformance

$12,000,000 incommercial loansprocessed properlyper annum.

$1,000,000 ofsecured individualreal estate loans(mortgages)processed properlyper annum.

12% rejections ofloans processed byany Real EstateLoan Officer.

Need(s) Gaps inPerformance

There is adiscrepancy of$13,000,000 incommercial loansprocessed properlyper annum.

There is a deficiencyof $1,500,000 ofsecured individualreal estate loans(mortgages)processed properlyper annum.

There is a deficiencyof 12% rejections ofloans processed byany Real Estate LoanOfficer.

(Kaufman, R.A., Rojas, A.M., and Mayer, M., Needs Assessment: A User’s Guide, Educational TechnologyPublications, Inc., Englewood Cliffs, NJ, 1993)

Define Job Impacts:• Micro• Macro• Mega

Page 20: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • NEEDS ASSESSMENT OR “GAP ANALYSIS” – Determining the Impact of Performance Problems PAGE 6

and to the world in which its contributions must function.

• Self-sufficient, self-reliant, productive individual whois socially competent and effective.

• Contributes to self and others.• No addiction to others or to substances.• Financially independent.• Continued funding of agency.

(Kaufman, R., Thiagarajan, S., and MacGillis, P., TheGuidebook for Performance Improvement: Working withIndividuals and Organizations, Educational TechnologyPublications, Inc., Englewood Cliffs, NJ, 1993, p 111-113)

The Air Force Training Needs Assessment Model below presentsthe type of analysis at each level (Micro, Macro, Mega). Theanalyst can decide on one or more levels of analysis dependingon the complexity of the need.

3. Define the target employee population or targetaudience.

This includes the demographics, education, experience,preferences, and past training of the population. Somesample questions might include:

• How long have you been with this organization?• What types of other jobs have you held both inside

and outside the organization?• How long have you been in your current job?• What education and/or training were required for

START HERE

HeadquartersU.S. Air Force

(Mega)

MajorCommands

(Macro)

Units(Macro)

Individual(Micro)

USAF-wideIssuesAchieve

strategic goals/objectives,

change policydirective, adoptfundamental,

adjustorganizationalenvironment

PerformanceIssues

Determineperformancerequirements

OperationalIssuesDevelop

performanceunits/ teams

Train forsuccess

JobPerformance

Determineperformance on

the job

New policieswith trainingobjectives,

new/ revisedtrng prgms,

management

New/revisedtraining

programs,managementintervention

New/revisedtraining

programs,managementintervention

Existing trainingcourses, new/

revised trainingcourses, job

aid,management

TypeAnalysis

Pur-pose

Out-comes

Page 21: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • NEEDS ASSESSMENT OR “GAP ANALYSIS” – Determining the Impact of Performance Problems PAGE 7

your job entry?• What training did you receive on the job?• Do you believe this training was adequate?• On a scale of one to five, how would you rate your

proficiency as a computer user (one being thelowest, five being the highest)?

• What application programs do you use on a regularbasis?

• Are you proficient in using the internet?• What are your general job duties?• What tasks do you like doing the best, and why?• What tasks do you like doing the least, and why?• What is the difference between a novice and an

expert in your job?• What motivates you to do your best work?• If there is one thing you could change, what would it

be?

4. Conduct a cost/benefit analysis or return on investment(ROI) analysis.

A ROI is developed after the implementation of the solution.However, sometimes a quantitative estimate oforganizational benefits can be made before theimplementation of the solution.

ROI compares the monetary benefits of the new solution,including training, new tools or resources, and/or otherorganizational solutions, with the actual cost of the program(ROI = Net Program Benefits/Program Costs X 100) .Sometimes it is also helpful to capture intangible data thatpurposely is not combined with the four levels of evaluationdeveloped by Kirkpatrick, a balance of different levels of dataare collected, representing a comprehensive measurementof the evaluation process. (Phillips, J.J., Phillips, P.P.,Lizette, Z.D., and Gaudet, C.; Evaluating the Return onInvestment of E-Learning: The ASTD E-Learning Handbook,McGraw Hill, New York, NY, 2002)

Case Study 1: The American Petroleum Companymeasured the effectiveness of a new technology program.The goal of the training program was to increase jobperformance in a sales environment. Two needs emergedwhich provided the impetus for change. One was the needto compete within the industry. The other was a lack of skillsand knowledge among the sales consultants in theorganization. Senior management had agreed to invest inthe e-learning program and the training department wantedto be able to provide information on whether there was areturn on investment. The ROI showed a 97 percent returnon investment because of implementing the new technologyprogram. (Phillips et al, 2002)

Case Study 2: Acme Incorporated used CD-ROMtechnology to train its field service engineers throughout thecountry with the objective of increasing customer satisfaction

An ROI compares themonetary benefits ofthe new solution withthe actual cost of theprogram.

Page 22: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • NEEDS ASSESSMENT OR “GAP ANALYSIS” – Determining the Impact of Performance Problems PAGE 8

and increasing the amount of sales. Each of the field serviceengineers already possessed a laptop computer. The costsfor the traditional program were weighed against the benefitsof the e-learning experience. Compensation for theinstructors was $65,000 per year plus benefits, bringing thecost to $78,000. Compensation for the learner was done onthe field service engineers, yielding $164 per day. Deliverycosts were assessed, including travel, overhead, equipmentand materials in an instructor-led workshop. After calculatingthe benefits, the ROI analysis showed a return of129 percent. (Phillips et all, 2002)

Case Study 3: The cost to design the program may beprorated over the expected life of the program. Onecompany developed a computer-based training program toteach maintenance administrators and repair service clerks aspecific system technology. An ROI analysis revealed areturn of 319 percent in Year 1 and 366 percent return inYears 2 and 3. There was no recurring cost of the initialtraining design and development, so the cost was spreadover a three-year period (the time cycle of the CBT program).Repeat calls were reduced by 22 percent. (Phillips et al,2002)

In Implementing ROI: Five Steps to Saving Time, Money andResources, Patti P. Phillips presents results from thedevelopment and implementation of a one-day sexualharassment course. The training was presented to first andsecond level managers, 655, and all employees in ahealthcare organization, 6,844. The fully-loaded programcosts were $277,987.

Page 23: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • NEEDS ASSESSMENT OR “GAP ANALYSIS” – Determining the Impact of Performance Problems PAGE 9

(www1.astd.org/news_letter/October/Linkd/Phillips.html)

Levels of Evaluation for a Sexual Harassment Course

Level 1:Reaction

Overallrating: 4.11out of 5

93%providedaction items

Level 2:Learning

65%increasepost-testvs. pre-test

Skillpracticedemon-stration

Level 3:Application

96%conductedmeetingsandcompletedmeetingrecord

4.1 out of5 onbehaviorchangestudy

68% reportall actionitemscomplete;92% reportsomeactionitemscomplete

Level 4:Impact

Turnoverreduction:$2,840,632

Complaintreduction:$360,276

Totalimprove-ment:$3,200,908

Level 5:ROI

1,051%

Page 24: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • NEEDS ASSESSMENT OR “GAP ANALYSIS” – Determining the Impact of Performance Problems PAGE 10

Needs Assessment Job Aid

This job aid follows the Needs Assessment/Needs Analysisprocess identified in the Department of Defense Handbook:Instructional Systems Development/Systems Approach toTraining and Education, MIL-HDBK-29612-2A. (See thehandbook for a detailed explanation of the recommendedprocess.) An example of this process is presented in theAppendix of this Handbook.

Step 1: Collect, group and analyze the triggering circumstances, symptoms orindicators in the problem identification process.

Step 1.1: Identify triggering circumstances.Step 1.2: Identify gaps in performance.Step 1.3: Identify “high driver” performance problems.

Step 2: Determine the reliability of triggering circumstances.

Step 3: Begin by raising questions. The answers to these questions can isolatethe essence of the performance discrepancy, and later analysis will suggestacceptable solutions.

Step 3.1: Identifying the impact of the job on the organization, client and society.

Step 3.2: Identifying the consequence of job errors.

Step 4: Document evidence supporting the possible problem.Step 4.1: Prioritizing performance problems.Step 4.2: Estimating the benefits of investments to improve performance.

Step 5: Determine whether doctrine, training, organization, leadership, materialor a combination of these is the cause of the performance problem.

Step 6: Identify possible solutions to the problem addressing each cause.

Step 7: If the conclusion of the problem solving points to a training deficiency,proceed with a Training Program Needs Analysis.

Step 8: Verify the training problem. The needs analysis confirms or rejects thehypothesis that a change in training can correct or relieve the deficiency.

Step 9: Determine solutions. The training developer will determine all possibletraining solutions or corrective actions to be taken to solve or alleviate theproblem.

Step 9.1: Develop alternative solutions.Step 9.2: Evaluation of alternatives.Step 9.3: Make recommendation.

Step 10: Develop needs analysis report and forward to appropriate agency foraction. The needs analysis report should contain a detailed description of theproblem, the selected solution, rationale for the solution, and the trainingdevelopment requirement.

Needs Assessment Job Aid

Page 25: Best Practices in e-learning Handbook

Needs Assessment and Training Needs Analysis Case Study

In August of 2000, the Central Office of the Veterans Benefits Administration (VBA) tasked the Technical Training and Evaluation Office (TT&E) in Orlando to develop a training program for Field Examiners. The Central Office estimated the training program costs to be from $200,000 to $400,000. A combined government and industry team completed the needs assessment/analysis report in March 2001. The needs assessment documented:

• A 40% error rate in the accuracy of Field Examiner investigations.

• A six month discrepancy in the time required to train new Field Examiners.

• The organization’s goal to improve Field Examiner accuracy by 36% within five years (i.e., over 7% improvement per year compared to the previous year’s 4.5% improvement).

• The benefit of improving accuracy by 36% would be increased confidence in the fiduciary management of $341,000,000 per year.

In addition to performance-based training, the training needs analysis indicated a requirement for an electronic performance support system, which was not anticipated. Management endorsed the recommendation. The approved budget for the program was $1,650,000. Decision makers acknowledged the benefits of improving performance in spite of the higher budget estimate because the analysis:

• Quantified the benefits of improved performance. • Linked improved job performance to organizational

goals.

The analysis was conducted using the following steps:

JOINT ADL CO-LABORATORY • NEEDS ASSESSMENT &TRAINING NEEDS ANALYSIS CASE STUDY PAGE 1

Page 26: Best Practices in e-learning Handbook

Step 1.1. Identify Demographics of Target Audience

Target Audience: • There are approximately 150-250 Field Examiners (Occupational Code 1801). • Mix of GS-9 and GS-10 level employees. • Primarily high school and junior college graduates. • Field Examiners are selected form other in-house jobs (they have organizational experience, and 5-10 years experience in related job categories).

• 91% of the Field Examiners have over 20 years experience. • 82% of the Field Examiners are over age 50. • Computer competency varies from non-users to experts. • Field Examiners work in Field. The training is now primarily on-the-job training.

Sources (both current and projected: • Personnel retirement/attrition rates. • Position descriptions and skill sets of Field Examiner personnel. • Aggregate personnel performance data. • Interviews with Field Examiners, Managers and Training Coordinators.

• Professional development pathways. • Training requirements and curricula. • Organizational hierarchies and reporting requirements. • Proposed regulatory changes impacting the organization and its operations.

• Facility systems and resources.

Step 1.2. Document the sources of job manpower, personnel, and training data.

Step 1.3. Identify Triggering Circumstances

Field Examiner Job Protecting veteran or beneficiary of veteran by: • Reviewing rulings of incompetence regarding client.

• Interviewing client in home, nursing home, prison, drug rehab center, or hospital.

• Determining if client is incompetent, dangerous to himself or society, taking medications, and being cared for by the guardian.

• Certifying guardians as fiduciaries. • Investigating misuse of funds and fraud. • Developing initial and follow -up field investigation reports.

Triggering Circumstances for performance problems: • The need to make judgments based on complex policies and statutes.

• The impact of working in dangerous work environments.

• The emotional fatigue and frustration from social work.

• The pressures due to the potential for mistakes to lead to congressional inquiries.

• The belief that this is a “dead end” job.

Step 1. Collect, group, and analyze the triggering circumstances, symptoms or indicators in the problem identification process.

JOINT ADL CO-LABORATORY • NEEDS ASSESSMENT &TRAINING NEEDS ANALYSIS CASE STUDY PAGE 2

Page 27: Best Practices in e-learning Handbook

Required Performance Experienced Field Examiners are required to complete five field exams with documented report per week with no significant errors. New Field Examiners should reach proficiency in six months.

Current Performance Experienced Field Examiners complete five field exams with documented report per week with an average accuracy rate of 60% . New Field Examiners reach proficiency in one year.

Need(s) Gaps in Performance There is a discrepancy of 40% in the error rate. For new Field Examiners, there is a deficiency of six months in reaching proficiency.

Step 1.4. Identify Gaps in Performance

Required Performance Current Performance Need(s) Gaps in Performance

Step 1.5. Identify “High Driver” Performance Problems

Most Critical Task Most Difficult Task Most Difficult Duties

Field Examiners • Appointing a fiduciary • Meeting beneficiary needs • Fund usage • Evaluation of capacity to manage funds

• Personal contact and interview

• Follow -up

FE Managers • Completing field exams • Identifying a fiduciary • Timely completion of tasks • Fund usage • Personal contact and interview

• Documentation

Step 2.1. Document the evidence supporting the performance discrepancies.

Evidence resides in the following databases: • Systematic Technical Accuracy Review (STAR) reports that identify job errors on a quarterly basis.

• Distribution of Operational Resources (DOOR) reports that pertain to the timeliness/elapsed time of completed field exams and the pending workload.

• Balanced Scorecard Goals that pertain to the product workload goals for each regional office.

Step 2.

Document the impact of the performance problems.

JOINT ADL CO-LABORATORY • NEEDS ASSESSMENT &TRAINING NEEDS ANALYSIS CASE STUDY PAGE 3

Page 28: Best Practices in e-learning Handbook

Step 2.2. Identify the impact of performance discrepancies on the Organization, Client, and Society.

Micro Level Products : The interim products, results, and performance. Macro Level Output: The aggregate products delivered to the client. Mega Level Outcomes: The effects on society.

Performance gaps impact the quality and timeliness of: • Initial Examination Reports with rationale supporting the decisions made by the Field Examiner.

• Follow Up Reports with rationale supporting the decisions made by the Field Examiner

Performance gaps impact: • The protection of the client – a stable and safe guardian and/or fiduciary agreement.

Performance gaps impact: • Improved quality of life for the “incompetent” veteran and/or beneficiary, the family, and community.

Step 2.3. Identify potential consequences of performance problems.

Task • Personal contact and interview. • Follow -up. • Documentation. • Appointing/certifying fiduciary. • Timely service. • Completing field exams. • Evaluation of fund usage. • Evaluation of capacity to manage funds.

• Benefits entitlement.

Consequence • Potential for overpayment. • Client complaints. • Rework. • Potential misuse of beneficiary funds. • Customer dissatisfaction. • Lack of creditability. • Potential waste of beneficiary funds. • Potential misuse of beneficiary funds. • Beneficiary may not get all benefits.

Step 2.4. Prioritize performance problems.

1. Timely service. 2. Documentation: completing field examinations accurately. 3. Fund usage 4. Appointing/certifying a fiduciary. 5. Personal contact and interview. 6. Benefits entitlement. 7. Evaluation of capacity to manage funds. 8. Follow -up.

Rationale: Balanced Scorecard Priorities and Percentage of Overall STAR Errors Identified Balanced scorecard goal. Balanced scorecard goal. 23%. 20%. 19%. 14%. 12%. 12%.

JOINT ADL CO-LABORATORY • NEEDS ASSESSMENT &TRAINING NEEDS ANALYSIS CASE STUDY PAGE 4

Page 29: Best Practices in e-learning Handbook

Step 2.5. Estimate the benefits of investing in performance improvement.

Benefits: • Field Examiners oversee the management of $948,000,000 per year. • The Five-Year Goal is to improve accuracy by 36% from 60% to 96%. • This 7+% improvement goal per year is extremely aggressive, since the previous two year’s performance improvement was 4.5% PER YEAR.

• This 36% goal accounts for the reduction of risk or uncertainty in the fiduciary management of $341,280,000 per year (36 percent by $948,000,000).

Step 3.1. Determine the root cause (i.e., doctrine, training, organizational, leadership, material, etc.) of the performance

problems. The root cause of the performance gaps included three primary domains:

Training • Limited initial training and performance evaluation.

• Non-standardized sustainment training.

• Management training on the role and responsibilities of the Field Examiner job.

• More frequent sustainment training.

Resources: Tools and

Capabilities • Computer. • Cell phone. • Official badge. • Access to FBS.

• Access to BDN.

• Internet access.

• E-mail.

Management Support • Effective information dissemination. • Additional in-house support. • Using partners in dangerous areas. • Ability to contact beneficiaries ahead of time.

• Resolving time pressures. • Field Examiner control of workflow/workload.

• Recognition of Field Examiner program value.

• Ability to work across state borders. • Ability to access multiple state benefits information.

• Minimizing travel problems.

Step 3.2. Develop planning objectives for each root cause.

Training • Develop interim Field Examiner training tools within one year.

• Develop and implement a standardized training program within two years.

• Develop training for other jobs impacting the field investigation (not time constraint is being placed on this objective).

Resources Ensure Field Examiners have all the necessary tools within one year.

Management Support • Develop an effective method for information dissemination within one year.

• Provide guidance on workload management within one year.

• Submit a proposal to reclassify the Field Examiner position to a GS-11 within one year.

Step 3.

Address the root cause of performance problems.

Since Training was identified as a “Root Cause” of the performance problem also, a Training Needs Analysis was completed.

JOINT ADL CO-LABORATORY • NEEDS ASSESSMENT &TRAINING NEEDS ANALYSIS CASE STUDY PAGE 5

Page 30: Best Practices in e-learning Handbook

Step 4.1. Define and analyze the training environment (training philosophy, staff, students, curriculum, training resources).

TRAINING PHILOSOPHY Current Situation: • On-the-Job training approach. • Training in two venues. • Field Examiner manager is instructor at RO. • Field Examiners mentor new hires in the field. Impact: • No standardization. • Errors • Takes one year to achieve proficiency. STAFF Current Situation: • No dedicated instructors • Manager is trainer. • Senior Field Examiners are trainers in the field. • Limited or no support staff. Impact: • Risk of non-standardized instruction. • Need to use instructors as facilitators where possible. • Instructor training required. • Need to have manager run all performance evaluation. • Instructor training required. • Need a good field guide for field training. • Need to get some support from other TPSS administrators. STUDENTS Current Situation: • Hired from within VBA (GS-9/10). • Variance in computer competency. • Work from home. • Have home computers. Impact: • Plus: Self-direction. • May need entry-level computer training. • Need help desk and home computer support. CURRICULUM Current Situation: • No standardized curriculum. • Mentor-based training in RO and Field. • No systematic feedback or performance evaluation. • Job evaluation via STAR feedback. Impact: • Need detailed task analysis for standardization. • Need both computer- and mentor-based performance evaluations. RESOURCES Current Situation: • Computer training rooms are shared resources. • Computers are outdated. • Limited storage space for training materials. • Lack of tools creates time/accuracy roadblocks. • Multimedia equipment is available. • Availability of portable video camera/playback equipment questionable. Impact: • Need assignment of upgraded computers with sound cards for RO and home web-based training. • Need designated long-term storage area. • Need all tools identified for the job. • Need dedicated equipment for Field Examiner training.

Step 4.

Verify and estimate the scope of the training requirement.

JOINT ADL CO-LABORATORY • NEEDS ASSESSMENT &TRAINING NEEDS ANALYSIS CASE STUDY PAGE 6

Page 31: Best Practices in e-learning Handbook

Step 4.2. Estimate the boundaries of the training requirement.

TOPIC Introduction Overview of Responsibilities Analysis of Week’s Work Team Responsibilities (shareable cross-training content required for LIE, VSR, RVSR, FE) Personal Contact and Interview (initial field exam, fiduciary interview, other interviews – shareable content) Legal Issues (shareable content) Physical and Mental Conditions (typical mental illnesses and medications) Capacity to Manage Funds Fund Usage Social Adjustment (shareable content) Evaluate Environment Dependency Needs Benefits Entitlement (shareable content from VSR benefits) Review of Payment Interview ing References Follow -Up Problem Resolution Documentation Respond to Inquiries Total Classroom Hours Rough CBT/WBT Estimate (Preliminary)

TIME RANGE

0.5 1.0

2.0

2.0

24.0

6.0

2.0

2.0

16.0 0.5

0.5

1.0

2.0 0.5

8.0

16.0 6.0

90.0

30.0

EST IN HOURS

1.0 2.0

4.0

4.0

40.0

8.0

4.0

6.0

24.0 1.0

1.0

2.0

6.0 1.0

24.0

24.0 8.0

160.0

53.0

MEDIA EST: FIELD

WBT, narration WBT, narration WBT, narration; flow charts; job aid WBT, narration, flow charts, job aid WBT, narration, flow charts, check lists, simulations, role play, video feedback WBT, narration, job aid WBT, narration, flow charts, check lists WBT, narration, flow charts, check lists, simulations, role play, video feedback

WBT, simulations WBT, simulations WBT Piggyback of VSR/RVSR training, check list WBT, narration, flow charts, check lists, simulations, role play, video feedback

WBT WBT, narration, flow charts, check lists WBT, narration, flow charts, check lists WBT, narration, flow charts, check lists NOTE: Research has shown that CBT and WBT presentations significantly reduce the time of traditional classroom hours.

MEDIA EST: CLASSROOM

OJT guide, workbook OJT guide, workbook OJT guide, workbook OJT guide, workbook OJT guide, workbook, evaluation guide OJT guide, workbook OJT guide, workbook, evaluation guide OJT guide, workbook, evaluation guide OJT guide, workbook, evaluation guide Workbook, evaluation guide Workbook, evaluation guide OJT guide, workbook, evaluation guide Workbook, evaluation guide OJY guide, workbook, evaluation guide OJT guide, workbook, evaluation guide OJT guide, workbook, evaluation guide

JOINT ADL CO-LABORATORY • NEEDS ASSESSMENT &TRAINING NEEDS ANALYSIS CASE STUDY PAGE 7

Page 32: Best Practices in e-learning Handbook

Step 5.1. Develop alternative solutions

MEDIA Web-based training Instructor-led training Instructor-led with workbook/ field guide Electronic performance support system Paper job aids

ALT 1: All Web-Based

Training 49 hrs

1 hrs

4 hrs

ALT 2: All Instructor-led

Training

45 hrs

75 hrs

1 hrs

4 hrs

ALT 3: Both Web-Based & Instructor-led

Training 25 hrs

10 hrs

20 hrs

1 hrs

4 hrs

Step 5.2. Evaluate alternative solutions.

Cost Training Hours Additional Shadowing Hours Technical Risk Benefits/ Limitations Supportability Schedule Risk Testing Manpower & Personnel

Alt 1: All WBT $1,8500,000 49 Hours 20-40 Hours Moderate • Trains knowledge and procedures. • Does not train performance under required conditions (i.e., personal contact and interview skills). • Training in only one venue. • Minimal train-the-trainer requirement. • Requires allocation of computers. • Requires some storage space. • Requires some staff support. • Minor addition to FE Manager’s workload at RO. • No addition to Sernior FE’s workload at RO. • Does not significantly impact student training time. Low • Evaluates knowledge and procedures. • Does not evaluate performance under required conditions (i.e., personal contact and interview skills). Not manpower intensive.

Alt 2: All ILT $2,140,000 125 Hours (included) Moderate • Trains knowledge. Does not train procedures effectively. • Trains performance under required conditions (i.e., personal contact and interview skills). • Trains in RO and field. • Intense train-the-trainer requirement. • N/A. • Requires vast storage space. • Requires significantstaff support. • Major addition to FE Manager’s workload at RO. • Major addition to Senior FE’s workload at RO. • Significantly impacts student training time. Moderate – a lot of material must be developed to support 125 hours of training.• Evaluates knowledge. • Does not evaluate procedures effectively. • Evaluates performance under required conditions (i.e., personal contact and interview skills). Manpower intensive.

Alt 1: Blended (WBT & ILT)

$1,650,000 60 Hours (included) Moderate • Trains knowledge and procedures. • Trains performance under required conditions (i.e., personal contact and interview skills). • Trains in RO and field. • Moderate train-the-trainer requirement. • Requires allocation of computers. • Requires some storage space. • Requires some staff support. • Moderate addition to FE Manager’s workload at RO. • Addition to Senior FE’s workload at RO. • Does not significantly impact student training time.Moderate – different types of media developed. • Evaluates knowledge and procedures. • Evaluates performance under required conditions (i.e., personal contact and interview skills). Moderate manpower.

Step 5. Develop and evaluate alternative solutions.

JOINT ADL CO-LABORATORY • NEEDS ASSESSMENT &TRAINING NEEDS ANALYSIS CASE STUDY PAGE 8

Page 33: Best Practices in e-learning Handbook

Step 5.3. Make recommendation

Alternative 3 was recommended and approved. It provides an EPSS to immediately support job performance. It also provides a blended training media approach in two venues. After the NA, organizational changes were made to eliminate risk associated with Alternative 3.

1st Year

2nd

Year3

rd Year

4th Year

5th Year

6th Year

7th Year

8th Year

9th Year

10th Year

Option 1

$400,000$1,250,000

$40,000$40,000$40,000$40,000$40,000$40,000$40,000$40,000

Option 2

$400,000$650,000$600,000$40,000$40,000$40,000$40,000$40,000$40,000$40,000

Option 3

$400,000$425,000$425,000$400,000$40,000$40,000$40,000$40,000$40,000$40,000

Step 6.

Develop and forward assessment report with funding options.

References. Daly, D.J., Pitts, T., Seiser, H., Locascio, F., and Melon, E. (2001). Field Examiner Needs Assessment/Training Situation Analysis Report. Office of Technical Training and Evaluation, Veterans Benefits Administration.

Note : The Needs Assessment and Training Needs Analysis Process with Examples (This example is taken from the Veterans Benefits Administration (VBA) Training and Performance Support System managed by the VBA Technical Training and Evaluation Office, Orlando, FL, Charles A. Beagles, Ph.D., Director)

JOINT ADL CO-LABORATORY • NEEDS ASSESSMENT &TRAINING NEEDS ANALYSIS CASE STUDY PAGE 9

Page 34: Best Practices in e-learning Handbook

<This page left blank intentionally>

Page 35: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • JOB AND TASK ANALYSIS PAGE 1

Job and Task Analysis

Job and task analysis are the first steps in building a structure forcapabilities-based learning. This process represents a job auditor inventory: knowledge, skills and attitudes are identified andisolated with a view to ultimately synthesizing them into ahierarchical organization relevant to the writing of a learningprescription. (Gilbert, T.F., 1962) In performing such an analysis,the analyst must consider not only the physical components ofthe subject, e.g., use of tools, references, job aids, etc., but alsothe mental components, procedures, decisions, abstractions, etc.(Davies, I., 1973)

Job AnalysisWhether developing a new training course or updating anexisting course, data is collected to conduct a job analysis. Jobanalysis is a method used to obtain a detailed listing of tasksnecessary to perform a specific job or duty. Job-related datashould include:

• The purpose.• The functional responsibility of personnel.• Required support equipment and materials.• Information on how the system works, is maintained, or

is used.

Collecting this date may involve:

• Observing personnel in the work environment.• Interviewing job incumbents and supervisors.• Using questionnaire surveys.• Interviewing jury of experts.• Studying applicable documentation, including:

• Occupational field descriptions.• Related training documents.• Engineering data and specifications.

(MIL-HDBK-29812-2A)

Task AnalysisTask analysis is the process of detailing task performance. Thetask performance details describe:

1. How the task is performed or the performance steps.2. Under what conditions it is performed.3. How well the individual must perform it or the

performance standards.

Page 36: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • JOB AND TASK ANALYSIS PAGE 2

These task performance details help the instructional developerestablish the individual training strategy and help design anddevelop the training programs and products.

A task is defined as, “A single unit of specific work behavior, withclear beginning and ending points that is directly observable orotherwise measurable.” A task may also be defined as a seriesof actions leading to a terminal outcome, such as “startingengines on an aircraft” or “performing an aborted take-off.” A taskis performed for its own sake, that is, it is not dependent uponother tasks, although it may fall in a sequence with other tasks ina mission, duty, or job.

Task analysis is in the process of breaking a task down toidentify the:

1. Component steps of a task.2. Sequence of those steps.3. Conditions under which the task will be performed (e.g.,

at night, in the field).4. Task cues.5. Standard of performance that must be achieved,

expressed in terms of accuracy, completeness,sequence, or speed. (MIL-HDBK-29612-2A)

See the example on the next page detailing the components of atask analysis.

A task is defined as, “Asingle unit of specificwork behavior, with clearbeginning and endingpoints that is directlyobservable or otherwisemeasurable.”

A task may also bedefined as a series ofactions leading to aterminal outcome, suchas “starting engines onan aircraft” or “performingan aborted take-off.”

Page 37: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • JOB AND TASK ANALYSIS PAGE 3

Task Analysis ModelsA fundamental assumption in performance technology, as well asin instructional systems development, is that the education andtraining a person receives should focus on enhancing his or herjob performance. Determining what a person must know andmust do to carry out a specific job is one very important aspect ofenhancing job performance. Task analysis serves this function.Task analysis is the process of identifying the various tasks aperson must perform while completing a job, determining whichof these tasks require training, and identifying the componentknowledge and skill that a person must possess in order toperform each task acceptably . (Hannum, Wallace; “Job-TaskAnalysis” in the Guidebook for Performance Improvement;Kaufman, R., Thiagarajan, S. and MacGillis, P.; Jossey-Bass

TASK TITLE: Assess the beneficiary’s physical and mental condition as itrelates to his or her capacity to manage funds.

TASK ACTION STATEMENT: Observe and assess the beneficiary’s physicaland mental condition, and note this information for inclusion in the fieldexamination report.

TASK CONDITION STATEMENT: Given case documentation, which mayinclude a copy of VA Form 21-592 (Request for Field Examination), a finalrating of incompetence, letters of guardianship, or other supporting legalevidence of incompetence, reference materials, including 38 CFR (1300series), M21-1 Part VIII, VBA Circulars, “Fast Letters,” Training Letters, PolicyLetters; access to BDN and BIRLS; and job aids.

TASK STANDARD:A. Before assessing the beneficiary’s physical and mental condition, the Field

Examiner will:1. Explain the provisions of the Privacy Act and document the fact that it

was explained on the check sheet.2. Note the beneficiary’s demographic information on the field

examination check sheet.B. The Field Examiner will note all of the following on the field examination

check sheet:1. The beneficiary’s appearance, as it relates to his or her capacity to

manage funds.2. The beneficiary’s physical condition, as it relates to his or her capacity

to manage funds.3. The beneficiary’s mental condition, as it relates to his or her capacity

to manage funds.4. The beneficiary’s diagnosis, as it relates to his or her capacity to

manage funds: (a) physical and (b) mental.5. The beneficiary’s prescribed medications and compliance with taking

medications, as it relates to his or her capacity to manage funds.6. The beneficiary’s orientation to:

a. Time, as it relates to his or her capacity to manage fundsb. Place, as it relates to his or her capacity to manage fundsc. Events, as it relates to his or her capacity to manage funds

7. The beneficiary’s ability to perform activities of daily living, as it relatesto his or her ability to manage funds.

Task Identification Sheet

Page 38: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • JOB AND TASK ANALYSIS PAGE 4

Pfeiffer, 1997)

There are a number of different task analysis models. Allpresent different approaches to analyzing different jobs. Threegeneral approaches to task analysis models include:

• Behavioral task analysis.• Procedural task analysis.• Cognitive task analysis.

Behavioral Job AnalysisThe most common method for analyzing job tasks requires adecomposition of job tasks into simpler sub-tasks, componenttasks, or steps. Often a task analysis will begin with a job, thenidentify separate duties that make up that job, and finally identifythe tasks that make up each separate duty. (Hannum, W. et al,1997) The chart below shows the behavioral breakdown of atask.

Process and Task AnalysisSome task analysis models seek to identify the procedures a jobincumbent must complete. These models extend beyond justbreaking an overall task down into its component subtasks.They seeks to identify the step-by-step procedures a jobincumbent follows when completing a task. (Merrill, 1980) Thesemodels result in flowcharts detailing what is done in a particularjob, such as the one depicted on the next page:

Decomposition of Job, Duty, Sub-Tasks

Job Perform avionics maintenance

Duty Maintain XYZ communication system

Task Perform diagnostic test

Sub-Tasks 1. Connect communication system to the tester2. Execute the test3. Analyze the test results

Page 39: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • JOB AND TASK ANALYSIS PAGE 5

Cognitive Task AnalysisAnalyzing tasks that are completed mentally requires a cognitiveapproach to task analysis, not a behavioral approach.Traditional task analysis methods rely on behavioralobservations or reports of behaviors. This is not adequate whenthe task is a thought process. Cognitive task analysis methodswould seek to determine what information an individual used incompleting a task and how he or she processed the information.One method used for cognitive task analysis is gatheringinformation about an incumbent’s thoughts while he or she iscompleting a task. Having the task performers describe whatthey are doing in a running dialog as they are completing sometasks does this. This is often called the “talk through” approach.

Best Practices: Common Task AnalysisFunctionsThere are many different task analysis models and procedures,but most share such common functions as:

• Task inventory.• Task description.• Task selection.• Task sequencing.• Analyzing performance and content levels.

Task InventoryA task inventory is a listing of all the specific tasks a personperforms when doing certain jobs. For example, a task inventoryfor an automobile mechanic would include such tasks as:

• Remove and replace spark plugs.

Process Flow

FE receives VA Form21-592 and other case

documentation from RO

1.1.1 Check forrating of

incompetence orevidence of legal

disability

Doesa rating of

incompetenceexist?

Note date ofincompetency

rating

Doesa court-

appointedfiduciaryexist?

Note fiduciary’s name,title, court of

jurisdiction, docketnumber andcomments

End Task 1.1

Do legal disabilitiesother than court

appointment exist?

Note date of disability.Describe legaldisability and

method of verification

1.2.2 Return case tooriginating RO End Task

Yes

Yes

Yes

No

No

No

Page 40: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • JOB AND TASK ANALYSIS PAGE 6

• Diagnose the cause for stalling.• Adjust the timing.

There is considerable agreement among task analysis modelsabout what is included in task inventories; the inventory lists thespecific tasks that make up a job. Some task listings includeonly tasks that are behaviorally stated that make up a job. Sometask listings include only tasks that are behaviorally stated anddirectly observable such as check voltage of circuits. Othersinclude non-behaviorally stated tasks such as troubleshooting ofelectrical problems.

Several different data collection methods can be used to gatherdata from job incumbents to construct task inventories. Sometask analysts may use a combination of observation, interviews,and surveys to create task inventories. This combination ofmethods can alleviate the weaknesses of any one method alone.

Task DescriptionTask description is in the process of elaborating upon the tasksor content identified during the task inventory to a more specificlevel of detail. Because job and content analyses are somewhatgeneral, they do not provide the amount of detail needed todesign instruction. Therefore, during the process, subtasks andsubordinate skills are identified. (Loughner and Moller, 1998)See the tables on the next page for more detailed information.

Page 41: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • JOB AND TASK ANALYSIS PAGE 7

Task SelectionWhen task analysts complete task inventories, they identifynumerous specific tasks that make up a job. Since trainingbudgets as well as the time available for training are limited, it isoften not economical or reasonable to train all tasks. Thusvarious task analysis models provide procedures for selectingtasks for training. For example, if a particular task is critical tooverall job performance, then training would be provided on the

Task Description

Task Number/Action 1.3.1 Send the appropriate letter to the veteran

Outcome Notification of veteran

Initiating Cue Agent receives signed Rating Decision andcompleted application 26-4555

Standard 100% for entitled veterans

Knowledge/Skills/Attitudes

Identifier Reference(s)

Pre-requisites

Meta-cognition

Knowledge

Knowledge

SkillsAttitudesNotes

P1

M1

K1

K2

S1A1

Read the Rating Decision forthe disability to determine thecorrect letter and enclosuresto send to veteran

Knowledge of disabilities

Understand the disability todetermine the correct letterand enclosures to send toveteran

Prepare the 26-39 or 26-39aform letter. (Letters may befound in the SAH program,letters screen)

PH Case:Send 26-39 letter to veteranwith SAH video

AH Case:Send 26-39a letter withappropriate information:

For visually impaired:Braille booklet or audio tapecassette; large print booklet26-69-1 Part IIFor loss of use of hands:26-69-1 Part II

M26-12 201b

Page 42: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • JOB AND TASK ANALYSIS PAGE 8

task instead of another task that was less critical to overall jobperformance.

Different task analysis models employ different criteria for taskselection. Tracey, Flynn, and Legere (1966) specified severalcriteria for use in task selection for a model developed for theU.S. Army. The following task selection criteria were identified:

• Universality.• Difficulty.• Criticality.• Frequency.• Practicality.• Frequency.• Practicability.• Achievability.• Quality.• Deficiency.• Retainability.• Follow-on training.

They suggest that each task in the task inventory indicate eitheryes or no on each of the criteria. Others have used a scalingapproach to rate tasks from high to low in regard to each of thecriteria. The most common set of the task selection criteriainclude only four of the original ten criteria: universality, criticality,difficulty, and frequency. (Jonassen, Hannum and Tessmer,1989) One common approach to task selection if termed the D-I-F model, because it uses difficulty, importance, and frequency asthe task selection criteria.

Analysis

Design

Deploy, Install &Configure

Manage

Monitor & Optimize

Troubleshoot, Repair &Restore

Job Duty

ImportanceDifficultyFrequency

Very Important/ Difficulty, Daily

ModeratelyImportant/Difficult,Weekly

Of Some ImportanceEasy, Monthly 2. 3. 4.

Page 43: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • JOB AND TASK ANALYSIS PAGE 9

Sequencing Tasks and Task ComponentsThe process of task sequencing requires further review of thetasks to determine the appropriate sequence of instruction. If thetask to be trained is procedural, the sequence in which the taskis performed may or may not be the best sequence forinstruction. For example, more complicated sub-tasks may betrained out of sequence as a prerequisite to ensure the learnerhas mastered the sub-task prior to performing the job task.Sequencing of instruction also varies according to the theory ormodel on which it is based. (Jonassen and Hannum, 1986/1991)

Task Sequence Diagram

Underwriterreceives loan

package

1.1Review the documentation

in the loan package andtransfer the information tothe Loan Analysis Form

1.1.1Determine

type ofloan

1.1.2Determineeligibility

1.1.3Review

appraisal &Notification

ofCertification

ofReasonable

Value

1.1.4Determine

debt &credithistory

1.1.5Determineoutcome

1.1.6Determine

assets

1.2Finalize LoanAnalysis Form

1.2.1Calculate

loanamount

1.2.2Calculatemonthlypayment

1.2.3Calculateresidualincome

1.2.4Calculateincome-todebt ratio

1.2.5Render finaldecision (Ifauthorized)

End Task

Page 44: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • JOB AND TASK ANALYSIS PAGE 10

Analyzing Performance and Content LevelsWhen analyzing performance and content levels, the behaviorthat the learner demonstrates as a result of the instruction iscategorized. Analysts often use Bloom’s Taxonomy ofEducational Objectives, first published in 1956, which classifieslearning outcomes into three domains:

• Cognitive.• Affective.• Psychomotor.

Each domain is then divided hierarchically. The cognitivedomains, for example, increase in complexity from knowledge tocomprehension, application, analysis, synthesis, and evaluation.

Another important taxonomy was developed by Robert Gagne.Gagne identified five categories of learning outcomes:

1. Intellectual skills.2. Cognitive strategies.3. Verbal information.4. Motor skills.5. Attitudes.

Each category is also divided hierarchically. The table on thenext page illustrates the categorization of tasks by learning levelafter they have been selected and incorporated as learningobjectives.

Page 45: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • JOB AND TASK ANALYSIS PAGE 11

Objective Hierarchy

Terminal Objective

1.0 Given the WBT and its resources (Glossary, Links, Help) and scenarios,the student will be able to recognize the foundational informationnecessary to perform the day-to-day tasks of an entry-level loanspecialist. The standard will be scoring 80 percent or higher on thecomprehensive course post-test.

Enabling Objectives Level

1.0.1 Given the training material (i.e., information,chart/tables, forms, examples) and scenariosegments, the student will be able to recognizethe steps to take to determine if the veteran iseligible for the VA Home Loan program by:

• Recalling definitions of the various types of veterans.• Recognizing the general rule of eligibility.• Recalling the exceptions to the 2-year requirement.• Defining active duty.• Identifying reservist and national guard eligibility

requirements.• Recognizing eligibility requirements for unmarried

surviving spouse.• Recognizing how to determine if veteran qualifies based

on proof of service documentation.• Identifying documentation needed for determination of

eligibility.• Recalling how to review documentation necessary for

the eligibility process.• Identifying the steps to verify if the veteran has

previously used his/her entitlement.• Recognizing the ways the veteran’s entitlement can be

restored.• Recognizing the procedures for issuing a certificate of

eligibility.

Bloom Level 2:Comprehension

Bloom Level 1:Knowledge

Bloom Level 2:Comprehension

Bloom Level 1:Knowledge

Bloom Level 2:Comprehension

Bloom Level 1:KnowledgeBloom Level 2:Comprehension

Bloom Level 1:Knowledge

Page 46: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • JOB AND TASK ANALYSIS PAGE 12

Job and Task Analysis Job AidIn 1986, the International Board of Standards for Training,Performance, and Instruction identified competencies forperforming job, task, and content analysis:

Page 47: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • WRITING FOR THE WEB PAGE 1

Writing for the WebWhy Writing Differently for the Web is Vital to Online Learning

So why write differently for the web?The advent of electronic media has provided the platform todisperse information to an even greater audience. Nowprograms like Dreamweaver, FrontPage and even MicrosoftWord allow even the most novice developers to post theircontent to the web. If you have even the slightest opinion youcan easily share it with the world through the Internet. But howto best convey your message to your audience? Many webauthors are under the mistaken impression that the way youwrite prose for a printed medium is the same way that you writeprose for an electronic medium. Readers approach electronicmedia very differently than print media. To provide an analogy toillustrate this concept:

• Reading a book is like taking a long, hot bath….• Reading a newspaper is like taking a hot shower…• Reading the web is like taking a cold, quick shower!

There are a few best practices for writing for the web that youshould keep in mind:

1. Know your audience.2. Be succinct and scannable.3. Copyedit.

1. Know Your AudienceThe most important thing that you can do when thinking aboutyour web content is to know your audience. Audiencecomplaints fall into mainly three categories:

1. Content.2. Navigation.3. Design.

Content. Audience complaints about content can include that itis too long, and takes too long to read or involves too muchscrolling. Another complaint is that it is not relevant or poorlyorganized.

Navigation. When planning and organizing your website andcontent, it is very important to think about how the reader willmove through the content. Some of the most common audiencecomplaints regarding content are:

• It requires too many clicks to get through the content.• There are too many links within the content.• There are dead ends where the content stops and you

• Reading is like taking along, hot bath …

• Reading a newspaperis like taking a hotshower …

• Reading the web is liketaking a cold, quickshower!

1. Know your audience.2. Be succinct and

scannable.3. Copyedit.

Audience complaints fallinto mainly threecategories:1. Content2. Navigation3. Design

Page 48: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • WRITING FOR THE WEB PAGE 2

cannot get back to where you once were.• The content is poorly organized in general.

Design. If your audience is distracted, you will never be able toconvey your message. That is, if you use too many fonts, colors,clutter, pop-up ads, or if your content is too slow to download,your audience will get impatient and just move on.

Before writing the first word of your content, it is useful to givesome thought to the following questions:

• What are the needs and concerns of your audience? (i.e.reference, marketing, research, etc.)

• Is your audience local, national, global?• How computer literate is your audience?• What is your audience’s primary learning style?• Does your audience have any special needs or

accessibility issues that will affect the text and formattingof your site, such as:

♦ visual♦ auditory♦ language♦ cultural

2. Be Succinct and ScannableIf Mr. Wilde said that during the start of the last century, whatwould he possibly think of life in the new millenium? There is somuch information available at our fingertips now, how do youdetermine what is relevant and what isn’t? The thing is, that thereader should not have to spend time trying to figure it out, it isup to you, the author, to tell the reader what is important andwhat you want them to know. The way in which you do that is bykeeping your content concise and easy to read.

The following is an example of the before and after of a websiteedited for succinctness and scanability:

Before…

United States Intelligence Community

Who We Are and What We DoThroughout history, the leaders of nations and armies havesought to be forewarned of dangers and forearmed withinformation that reduces uncertainty and provides a critical edgefor decisions. The effort to meet these fundamental needs ofdecisionmakers is what lies behind the practice of intelligence.That practice consists of collecting and interpreting information,overcoming in the process any barriers erected to keep secretthe activities, capabilities, and plans of foreign powers andorganizations.

“It is a very sad thingnowadays that there is solittle useless information.”

-- Oscar Wilde

Page 49: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • WRITING FOR THE WEB PAGE 3

Today, intelligence is a vital element in every substantialinternational activity of the US government. Every day, theagencies and offices that make up the US IntelligenceCommunity provide an important information advantage to thosewho manage the nation's strategic interests--political, economic,and military. Intelligence organizations support a broad range ofconsumers, from the national level of the President, the Cabinet,and the Congress, to the tactical level of military forces deployedin the field.

For intelligence officers, this means maintaining an ability to warnpolicymakers and military leaders of impending crises, especiallythose that threaten the immediate interests of the nation or thewell-being of US citizens. It also means giving government andmilitary officials advance knowledge of long-term dangers, suchas the threats posed by countries that covet weapons of massdestruction. It means helping to safeguard public security bycountering threats from terrorists and drug traffickers. It meanssupporting economic security by uncovering foreign efforts atbribery and other schemes to tilt the playing field of internationaltrade. And it means multiplying the effectiveness of US militaryforces deployed for operations.

A series of statutes and Executive Orders provides legalauthority for the conduct of intelligence activities. Key documentsinclude the National Security Act of 1947 (as amended), whichprovides the basic organization of the US's national securityeffort, and Executive Order 12333, which provides currentguidelines for the conduct of intelligence activities and thecomposition of the Intelligence Community. Together with otherlaws and orders, these two documents are meant to ensure thatintelligence activities are conducted effectively and conform tothe US Constitution and US laws. They also provide a statutorybasis of accountability to the Congress.

The national intelligence effort is led by the Director of CentralIntelligence (DCI), who oversees the Intelligence Communityorganizations described in more detail in the following pages.Resources for these organizations are tied together in theNational Foreign Intelligence Program--the budget for thesenational activities, which support political, economic, and militarydecisionmakers, is developed by the DCI and presented to theCongress annually. Intelligence activities that are more narrowlyfocused and intended to support tactical military forces arefunded separately in two programs within the Department ofDefense. These programs--the Joint Military IntelligenceProgram and the Tactical Intelligence and Related Activitiesaggregation--fall under the aegis of the Deputy Secretary ofDefense. In recent years, the line between national and tacticalactivities has become less distinct and, in fact, national andtactical capabilities have been brought to bear on intelligenceproblems in complementary ways. The goal of intelligence,however, has remained constant--to support decisionmakers withthe best possible information, no matter its source.

Page 50: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • WRITING FOR THE WEB PAGE 4

After…

United States Intelligence Community

Overview of Activities

The Intelligence Community serves a broad range of clients fromthe President, Cabinet, and Congress to military forces deployedin the field.

Our activities include:

§ Warning policymakers/military leaders of impedingcrises, especially those that threaten the immediateinterests of the nation or the well-being of US citizens.

§ Informing government/military officials about long-termdangers such as threats posed by countries that wish toacquire weapons of mass destruction.

§ Helping to safeguard public security by counteringthreats from terrorists and drug traffickers.

§ Supporting economic security by uncovering foreignefforts at bribery and other activities designed to disruptinternational trade.

§ Enhancing the effectiveness of US military forces inareas of action.

The legal authority for our activities is based on:

§ The National Security Act of 1947 (as amended) whichprovides the basic organization of the US’s nationalsecurity effort.

§ Executive Order 12333 which provides currentguidelines for the conduct of intelligence activities andthe composition of the Intelligence Community.

Our funding is based on:

§ The National Foreign Intelligence Program

§ The Joint Military Intelligence Program

§ Tactical Intelligence and Related Activities

What was done to the document:

§ Eliminated clichés such as “Tilt the playing field.”§ Created sub-heads to break up text and bolded them for

emphasis.§ Used bullets where possible for easier reading.§ Provided white space between bullets when the text was

longer than one line.

Page 51: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • WRITING FOR THE WEB PAGE 5

§ Created links for users who might be interested ingaining more information about particular items.

§ Rewrote emotive language such as “countries that covetweapons of mass destruction” to “countries that wish toacquire weapons of mass destruction.”

§ Rewrote passive language.

Here are some tips to follow when thinking about how to keepyour content succinct and scannable:

• Chunk your content.• Keep sentences short.• Create appropriate headings and sub-headings.• Use bullet points for readability.• Create links for users that are interested in pursuing sub-

topics.• Use active voice.• Remove emotive language and clichés.• Make your words your links — no “click here for…”

3. Copyedit your workNothing is worse than users discovering misspelled words,broken links, or badly flowing text. The following is a humorousway to remember some grammatical rules:

• Make each pronoun agree with their antecedents.• Join clauses good, like a conjunction should.• About them sentence fragments.• When dangling, watch your participles.• Verbs has got to agree with their subjects.• Don't write run-on sentences they are hard to read.• Don't use commas, which aren't necessary.• Try to not ever split infinitives.• Its important to use your apostrophe's correctly.• Correct speling is essential.• Eschew ostentatious erudition.• Avoid cliches like the plague.• Avoid run-on sentences they are hard to read.• Never use no double negatives.• Use the semicolon properly, always where it is appropriate;

and never where it is not.• Reserve the apostrophe for it's proper use and omit it where

it is not needed.• Proofread carefully to see if you any words out.• Avoid commas, that are not necessary.• When you reread your work, you will find on rereading that a

great deal of repetition can be avoided by rereading andediting.

• A writer must not shift your point of view.• Do not overuse exclamation marks!!! (In fact, avoid them

whenever possible!!!)• And do not start a sentence with a conjunction.• Place pronouns as closely as possible, especially in long

• Chunk your content.• Keep sentences short.• Create appropriate headings

and sub-headings.• Use bullet points for

readability.• Create links for users that

are interested in pursuingsub-topics.

• Use active voice.• Remove emotive language

and clichés.• Make your words your links

– no “click here for …”

Errors take away from themessage of the text.

Page 52: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • WRITING FOR THE WEB PAGE 6

sentences, as of ten or more words, to their antecedents.• Hyphenate only between syllables and avoid un-necessary

hyphens.• Write all adverbial forms correct.• Don't use contractions.• It is incumbent on us to avoid archaisms.• If any word is improper at the end of a sentence, a linking

verb is.• Steer clear of incorrect verb forms that have snuck into the

language.• Take the bull by the hand and avoid mixed metaphors.• Avoid modernisms that sound flaky.• Avoid barbarisms: they impact too forcefully.• Never, ever use repetitive redundancies.• Everyone should be careful to use singular pronouns with

singular nouns in their writing.• If we've told you once, we've told you a thousand times:

avoid hyperbole.• Also, avoid awkward or affected alliteration.• Do not string a large number of prepositional phrases

together unless you are walking through the valley of theshadow of death.

• Always pick on the the correct idiom.• "Avoid overuse of 'quotation "marks.""• Never use more words than are necessary to get your point

across: be concise.• Awayz check you're spelling. (Your spellchecker would only

pick up one of the two errors here.)• Always be avoided by the passive voice.• Every sentence a verb.• Last but not least, avoid cliches like the plague: seek viable

alternatives.

Source:http://www.chesapeake.edu/Writingcenter/writegood.htmlhttp://cisw.cla.umn.edu/faculty/responding/self_humorous.html

Copyediting tips:

• Check for inconsistencies in facts, names, and otherelements.

• Check spelling.• Check tables, charts, figures for accuracy.• Reorganize written content for clarity.• Improve flow of text.• Edit paragraphs and sentences for clarity and conciseness.• Check headings for consistency of level and presentation.• Check titles against table of contents.• Check for inconsistencies in font, color, size.• Check grammar and punctuation.• Check for errors, omissions, inconsistencies.• Check that text line breaks are appropriate.

• Check for inconsistenciesin facts, names, and otherelements.

• Check spelling.• Check tables, charts,

figures for accuracy.• Reorganize written content

for clarity.• Improve flow of text.• Edit paragraphs and

sentences for clarity andconciseness.

• Check headings forconsistency of level andpresentation.

• Check titles against tableof contents.

• Check for inconsistenciesin font, color, size.

• Check grammar andpunctuation.

• Check for errors,omissions,inconsistencies.

• Check that text line breaksare appropriate.

Page 53: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • WRITING FOR THE WEB PAGE 7

4. Remember the 4 C’s

• Be concise.• Be clear.• Be coherent.• Be credible.

5. Bibliography and Recommended Reading

WWeebb SSttyyllee GGuuiiddee.. PPaattrriicckk JJ.. LLyynncchh aanndd SSaarraahh HHoorrttoonn..YYaallee UUnniivveerrssiittyy PPrreessss:: NNeeww HHaavveenn,, 11999999..

DDeevveellooppiinngg OOnnlliinnee CCoonntteenntt:: TThhee PPrriinncciipplleess ooff WWrriittiinngg aanndd EEddiittiinnggffoorr tthhee WWeebb.. IIrreennee HHaammmmeerriicchh aanndd CCllaaiirree HHaarrrriissoonn.. WWiilleeyyPPuubblliisshhiinngg:: NNeeww YYoorrkk,, 22000022..

DDeessiiggnniinngg WWeebb UUssaabbiilliittyy.. JJaakkoobb NNiieellsseenn.. NNeeww RRiiddeerrss PPuubblliisshhiinngg::IInnddiiaannaappoolliiss,, 22000022..

LLaabbyyrriinntthhss:: tthhee AArrtt ooff IInntteerraaccttiivvee WWrriittiinngg aanndd DDeessiiggnn.. DDoommeenniiccSSttaannssbbeerrrryy.. WWaaddsswwoorrtthh PPuubblliisshhiinngg,, 11999988

The 4 C’s:• Be concise.• Be clear.• Be coherent.• Be credible.

Page 54: Best Practices in e-learning Handbook

JOINT ADL CO-LABORATORY • WRITING FOR THE WEB PAGE 8

Writing for the Web: Tips and TricksQuick Reference Sheet

Audienceq What are the needs and concerns of your audience?q Is your audience local, national, global?q How computer literate if your audience?q What is your audience’s primary learning style?q Does your audience have any special needs that will affect the text and formatting of your site? • Visual • Auditory • Cultural • LanguageScanabilityq Chunk your content.q Keep sentences short.q Create appropriate headings and sub-headings.q Use bullet points for readability.q Create links for users that are interested in pursuing sub-topics.q Use active voice.q Remove emotive language and clichés.q Make your words your links – no “click here for …”Copyeditingq Check for inconsistencies in facts, names, and other elements.q Check spelling.q Check tables, charts, figures for accuracy.q Reorganize written content for clarity.q Improve flow of text.q Edit paragraphs and sentences for clarity and conciseness.q Check headings for consistency of level and presentation.q Check titles against table of contents.q Check for inconsistencies in font, color, size.q Check grammar and punctuation.q Check for errors, omissions, inconsistencies.q Check that text line breaks are appropriate.