TOMMY HVIDSTEN, ed.

THERTF

INSTRUCTI N

THE ART OF INSTRUCTION

PART1

PART2

06 FOREWORD 08 COMPETENCE AND LEARNING 14 THE HOLISTIC APPROACH18 LEARNING OBJECTIVES32 ASSESSMENT

DEVELOPMENT OF TRAINING 48 FIRST PRINCIPLES OF INSTRUCTION 58

LEARNING STYLES 68 THE IMPORTANCE OF TASKS 80

SOFTWARE PRODUCTS TUTORIALS 82LESSONS PLAN TEMPLATE 84

4

5

6ACKNOWLEDGEMENTS

The Art of Instruction has many contributors. Chapter one through four is based on the work made by Finn Simensen, Arild Dyrdal and Tommy E. Hvidsten of Tinius Olsens Tech-nical College in 2009. Other contributors are credited in the text. The editorial work and un-accredited texts is done by Tommy E. Hvidsten in 2011.

Art direction, design & production by Jette Graaner.

Photography by Tommy Normann

Printed in Norway by Zoom Grafisk AS

Printed on Cyclus Offset 140 gsm (100% recy-cled paper)

Texts are copyright their respective contribu-tors as stated in source references. Unac-credited texts and compilation is copyright Kongsberg Maritime 2011.

FOREWORD

We all work for KONGSBERG, and our daily work has to do with learning processes. As KONGSBERGs slogan is World Class, we are determined to facilitate World Class Learning! That is an important ambition, which requires constant focus on how to provide our training services better than we do today!

Training is the process of facilitating learning. Quality training requires constant innovation and improvement. We do not develop a new course every day, but we strive to make every course better in all fields. That includes focus on improvements on the following; KONGS-BERG product knowledge, knowledge about our customers needs, learning processes, lan-guage and cultural differences plus more...!

This book deals with knowledge about learn-ing, and tools for creating and improving courses. I see it as the first step and a first com-mon framework to support our core activities. We provide training on many different prod-ucts, but the understanding of learning pro-cesses is common.

In a collaborative effort, we will work together creating World Class Training and improving our toolbox as we learn.

As a part of KONGSBERG we are reliable. Our customers can rely on our determination to provide the right training at the right time World Class!

Kongsberg, 29.08.2011

Anne Toril KasinBusiness Manager Simulation & Training

7PART1

COMPETENCE AND

LEARNING01Competence (or competency) is the ability of an individual to perform a job properly. A competency is a set of defined behaviors that provide a structured guide enabling the iden-tification, evaluation and development of the behaviors in individual employees. As de-fined, the term competence first appeared in an article authored by Craig C. Lundberg in 1970 entitled Planning the Executive Devel-opment Program. The term gained traction when in 1973, David McClelland, Ph.D. wrote a seminal paper entitled Testing for Com-petence Rather Than for Intelligence. It has since been popularized by one-time fellow McBer & Company (Currently the Hay Group) colleague Richard Boyatzis and many others. Its use varies widely, which leads to consider-able misunderstanding.

An attempt to have a holistic model of com-petence is to place the concept Action com-petence as the sum of four competences that address different sides of the human nature.

8

ACTION COMPETENCEThe Danish scientist Karsten Schnack de-scribes action competence: Action competence is not a skill, even though you may need skills to act in certain ways.Action competence is not an ideal of living as in an action movie, even though it sometimes may seem necessary to be active and feel like a hero to learn by experience.Action competence is a capability based on critical thinking and incomplete knowledge to get involved with other people in respon-sible actions and counter-actions for a more humane world.

A popular model of competence is to regard the concept as a combination of knowledge, skills and attitudes used to perform a task. As our concern is Vocational competence, we will use this model widely in this text, however, it is necessary to understand that this model has limitations and that other perceptions of the competence concept also exist.

VOCATIONAL COMPE-TENCE

Insight in a certain vocationl.e.:-Knowledge, skills and attitudes related to a vocation

COMPETENCE TO LEARN

The ability and inter-est in renewal of knowledge

l.e.:-The ability to change-The ability to pro-cess information-Knowledge of sources for relevant information

SOCIAL COMPETENCE

Competence of hu-man relations.l.e. the ability to:-Co-operate and solve conflicts-To show tolerence-To practice demo-cratic and humanis-tic values

COMPENTENCE OF METHODS

The ability to work methodically, systematically and creatively.

l.e.:-Ability to see new, improved solutions-To think interdisci-plinary-To apply familiar knowledge in new contexts

COMPETENCEIS OUR ABILITY TO ACT ACTIVELY AND DELIBERATELY IN OUR SURROUNDINGS

ACTION

9

A POPULAR CONCEPT IS THAT WE FIND COMPETENCE IN THE OVERLAP BETWEEN SKILLS, KNOWLEDGE AND ATTITUDES.

Competency is also used as a more general description of the requirements of human be-ings in organizations and communities.

Competency is sometimes thought of as be-ing shown in action in a situation and context that might be different the next time a person has to act. In emergencies, competent people may react to a situation following behaviors they have previously found to succeed. To be competent a person would need to be able to interpret the situation in the context and to have a repertoire of trained actions to choose from. Regardless of training, competency would grow through experience and the abil-ity of an individual to learn and adapt.

wikipedia.org

COMPETENCE

KNOWLEDGE

SKILLS ATTITUDES

10

LEARNING VERSUS TRAINING

The competence (skills, knowledge and at-titudes) required to perform a task in a way that satisfies demands and expectations can be obtained through training. The term train-ing comprises the actions needed to reach the objective. Training in the professional life as we discuss here, can be courses with differ-ent contents and methods. Such courses are often tied to the learning of skills, such as to maintain and operate a machine.

Learning, however, is to a great extent a psy-chological phenomenon. Learning entails change.

WE HAVE LEARNED WHEN WE HAVE CHANGED.

Such change may be in the mind, and thus invisible to others, or in the behavior where it is visible to other people.

The learning itself can be a result of experi-ences we make. Learning is work that is done in the interaction between tasks, trainer and trainee. Learning entails that the trainee ob-tains knowledge and develops skills and atti-tudes. The core is that the trainee is faced with a serial of new challenges and makes his own experiences. From this, new impressions and practices are formed in a meaningful way.

The trainees learning must be the basis for how the training is organized. To make the work regarding learning go as far as possi-ble in relation to the overall objectives for the training, you have to regard assessment in the context of several factors which is imperative to learning. Some of these factors are:

Objectives and partial objectives in the training plan, clarified as educational objectives.

Selection of means for learning/learn-ing venue

Facilitation of the training situation

Working methods and organization of the training

Educational frames for the training

Assessment criteria and methods

1

2

3

4

5

6

11

It is important to put the trainee in control of his personal learning.

The best requirement for good learning is that the trainee is actively involved. Thus the focus must not rest only on the result of the learn-ing. It is also imperative to assess the learning process and the participation of the trainees in the planning, implementation and assess-ment of the training. In other words, the result of the trainees learning cannot be separated from the way the training takes place.

Assessment should also be used as a strategy for development and learning, both on the in-dividual and organizational level. The experi-ence gained should affect how the training is organized and conducted. The company must continually assess the operation to en-sure a good environment for learning and work, with open and positive communication, and where people thrive and feel secure.

12

I FORGET WHAT I WAS TAUGHT, I ONLY REMEMBER WHAT IVE LEARNT. (PATRICK WHITE, 1912-90, AUSTRALIAN NOVELIST).

13

INTRODUCTION: THE HOLISTIC

APPROACH 02THE HOLISTIC MODEL OF DIDACTIC RELA-TIONS.

The illustration to the left (Hiim, Hippe 2009) shows all the elements that influence learn-ing, and the relations between them. It is a useful tool for the planning and analysis of learning.

All the elements of the holistic model have mutual influence on each other. Thus, to in-sure quality in learning, the quality of all the elements has to be ensured.

PREREQUISITES FOR LEARNINGOB

JECTIV

ESEDUCATIONAL FRAMES

THE LEARNING PROCESS

CONTENT

ASSE

SSME

NT

14

THE TRAINEES PREREQUISITES FOR LEARNINGTo adapt the training to the trainee, it is im-portant to map the following issues regarding the trainee:Has the trainee prior knowledge about the subject?What is new to the trainee?What are the trainees particular interests?Has the trainee any particular resources (i.e. prior knowledge) related to the training? Has the trainee any particular problems re-lated to the training? And so on

OBJECTIVESThe training objectives are the knowledge, skills and/or attitudes it is expected that the trainee has gained when he has completed the training. Some objectives are long term and spans several months, others are short term and apply to an hour or a day. The train-ee may participate in the discussion about objectives and thus share the responsibility for his or her own learning. The objectives can be regarded as an agreement or contract be-tween the instructor and the trainee.

THE LEARNING PROCESSWhat is the instructors opinion on the learn-ing process?What shall the trainee do? What shall the instructor do? Is the trainee supposed to sit quietly and listen most of the time? Why? Shall the trainee do independent tasks? Why? Shall the trainee cooperate? Why? Is the trainee responsible for a big part of what is going to happen? How? And why?

EDUCATIONAL FRAMESFrame factors are conditions that limit, or facilitate, learning. Equipment and means of learning (i.e. text books), available time, workplace and department are examples of frame factors. The frame factors may be limit-ing in the sense that necessary equipment or remedies are absent. Or, they may give new possibilities, for instance that the company borrows equipment from another company. Anyway, it is important to clarify available time, equipment and so on. The instructors resources, knowledge and values are also frame factors that enable or limit learning.

CONTENTThe objectives are closely connected to the content of the training. The content is what the training is about, and how the content is se-lected and adapted. The word adapt points to the next issue, the learning process.

ASSESSMENTAssessment may be done relative to the train-ing process, relative to the objectives for the training, and relative to the trainees learn-ing.Is the trainee supposed the give any feedback regarding the training?Was it too easy, too difficult, interesting or bor-ing?Did the trainee learn what was intended? Why? Why not?How did the trainee perform?How is the performance for this trainee in comparison to others?

Hiim, Hippe: Undervisningsplanleg- ging for yrkeslrere, Gyldendal (3.utg. 2009)

15

Shall the instructor instruct or coach?How may the instructor motivate and activate the trainee?

16

WE DONT SEE THINGS AS THEY ARE, WE SEE THINGS AS WE ARE.

(ANAIS NIN, FRENCH-BORN AMERICAN WRITER, 1903-1977.)

17

LEARNING OBJECTIVES03

The objectives describe the competence the trainee shall possess when the training is completed. Competence is described as something the trainee shall be able to do/achieve/master/contribute to/relate to, alone or together with others. I.e. the description places the trainee in the centre of the train-ings objective.

By knowing where you intend to go, you in-crease the chances of you and the learner getting thereGuides the teacher helps him/her focus and set priorities with regards to instruction plan-ning, evaluation of student achievement and during the actual instruction sessionsand evaluation of student achievement.Guides the learner; helps him/her focus and set prioritiesAllows for analysis in terms of the levels of teaching and learningShows customers, colleagues and students what we valueAre objects for the learners self-assessmentProvides a basis for analyzing the level of cog-nitive thinking expected from the learnerMakes teaching more focused and orga-nizedIs a starting point for making good exercises/tasks

18

LEARNING OBJECTIVES VS. COMPETENCE

The learning objectives are split into three categories:

Knowledge objectives (skills in the cognitive domain)

Skills objectives (skills in the psychomotor do-main)

Attitude objectives (skills in the affective do-main)

What kind of objectives is this?

The trainee shall be able to drill a suitable hole and enter a screw in a wooden cornice.

The trainee shall be able to recognize the most ordinary screw dimensions from a list.

The trainee shall be able to mention three places on the body were the pulse could be measured.

The trainee shall be aware of the importance of measuring pulse and blood pressure in a situation of crisis.

The trainee shall be able to explain the rela-tion between selecting a tap hole dimension/external diameter and the tensile strength in a bolt/nut binding.

The trainee shall be able to repair a rust dam-age on a car bonnet.

The trainee shall be conscious that the use of good raw materials are important for the final product

ExAMPLE OF LEARNING OBJECTIVES

Learning objectives from the bricklayer pro-fession:

To be able to clean the brickwork (knowledge and skills objective).

To know the current safety rules (knowledge and attitude objective)

To know the current regulations (knowledge and attitude objective)

Partial learning objectives broken down from learning objective no. 1:

The trainee shall be able to:

Sprinkle the brickwork

Mix the acid 10%

Spray the acid on the brickwork

Scrub the brickwork

Hose down the brickwork

CHARACTERISTICS OF GOOD LEARNING OBJECTIVES

Good learning objectives are:

RELEVANT: They are derived from or in accor-dance with superior objectives in the theme plans.ACHIEVABLE: The trainees are able to reach the objective within the time available, and with the resources available.MEANINGFUL: The objectives are adapted to the trainees prerequisites and needs, in a way so that the trainees accept the objectives and are willing to work to reach them. One way of securing this is to let the trainees participate in formulating process for the objectives.PRECISE: The learning objectives have a level of precision so that the trainees have no doubt about the knowledge, skills and attitudes that are expected.COMMUNICABLE: The learning objectives are formulated in such way that the trainees un-derstand them, to be able to know what the learning/work is about.ASSESSABLE: The learning objectives are for-mulated so that the trainee knows if he/she is close to reach them or have reached them.

1

1

2

2

3

3

1

1

2

3

4

5

6

7

2

3

4

5

19

ANALYSIS OF THE LEARNING OBJECTIVESGrading; less, some, good, very good

RELEVANT

YES - LESS YES - SOME YES - GOOD YES - SOME YES - VERY GOOD YES - SOME

ACHIEVABLE MEANINGFUL PRECISE COMMUNICABLE ASSESSABLE

REQUIREMENTS FOR LEARNING OBJECTIVES

The learning objectives must describe what the trainees shall be able to explain, conduct, reproduce, construct, do, and find adversary arguments to etc. after the training has taken place.It is the trainees learning, not the trainees training, which is the focal point when the learning objectives are formulated.Learning objectives must be relevant. That implies a connection between the overall ob-jectives and learning objectives. Learning objectives must be realistic. They must be obtainable for the trainee. To achieve this, different objectives have to be set for dif-ferent trainees.Learning objectives shall be meaningful for the trainee. This means that the trainee ac-cepts the learning objectives. If the trainee has participated in the setting of the objec-tives, it is likely that this demand is met.The learning objectives must be clear. Clear objectives facilitate the trainees understand-ing of what is expected.

20

MORE ABOUT LEARNING OBJECTIVES

Learning objectives are separated in areas such as knowledge, skills and attitudes. These are important in different contexts.

The knowledge objectives are important to es-tablish intellectual skills. This applies to theo-retical, social and physical skills. An example of a knowledge objective may be; the train-ees must know about the car engines differ-ent functions. Knowledge-objectives are also important for skills objectives and attitudes objectives.

The skills objectives are important to establish practical abilities. In this case the knowledge about relevant tools/devices is important. To obtain knowledge about the use of certain tools and to be able to utilize this knowledge is an example of a skills objective.

Skills also comprise linguistic skills (the knowl-edge about words, expressions and gram-mar, as well as skills in using the language in a social context). Social skills are important to show concern (empathy) and to follow moral rules (norms). Social skills demand the devel-opment of attitudes.

Attitude objectives are important to establish social feelings, or feelings and values which are responsible. We define responsibility as the ability to satisfy own needs in a way that does not affect other peoples possibility to sat-isfy their own needs. This means among other things that you dont think or act too short-sighted, dont exploit other people to your own benefit, or misuse the trust of other people. . In many professional contexts there are strict mandatory security rules and routines to be followed. If a trainee doesnt follow these rules, it may have severe consequences.

To be able to follow these rules requires knowl-edge about them, and the will, ability and skills to follow them.

Attitudes, knowledge and skills are connect-ed, even if they separate in some ways. You get nowhere without knowledge. Knowledge and skills are mandatory to make construc-tions and maintenance to facilitate our prac-tical living. Knowledge and skills affects and develop each other. To test and develop skills, certain knowledge is required. The experi-ence one make through the testing of skills leads to new knowledge, which leads to new

skills etc. The most important are the attitudes. It does not help the society that has given us much knowledge and many skills, if we do not use this in a way that makes it a better society.

Knowledge and skills are the tools we use to satisfy many of the needs we have. Knowledge and skills within a vocational category are used to satisfy certain basic needs, such as the need for food, housing and security.

21

In a training context it is important that the ob-jectives, whether it is attitudes, knowledge or skills, are expressed as precise and concrete as possible. This makes it simpler for the train-ees and trainers to assess if an objective is ob-tained. It is difficult to make well-formulated attitude objectives. At the same time the shap-ing of these objectives is very important. The trainer and the trainees must be able to as-sess which objectives that are obtained. This demands that we [in the objectives] must say something about:

What kind of demeanor the trainees shall showWhat situation(s) the trainees shall show the demeanorWhat demands there is to the demeanor

Welding may require knowledge-objectives about a series of details regarding machines and technology, multiple skills about the han-dling of the machines, and attitude-objectives regarding storage and handling for safety.

In many vocations the precision by which the objectives are described is important. The skills of a carpenter, mason or welder is dependant of precision, and it may often be a great gap between an apprentice and a skilled worker. At the same time it is not possible to describe some skills objectives by precise terms or ex-pressions.

In certain trades some tasks are so advanced, independent of skills but dependant of dexter-ity, that they can hardly be described. This is called tacit knowledge, because the knowl-edge behind the skills is difficult to explain, but has to be demonstrated and tested. Some forms of dexterity are so advanced that very few obtain them. This may apply within dif-ferent vocations. The classical example is the violin builder.

It is easy to form precise and detailed knowl-edge- and skills-objectives when they are re-lated to relatively simple knowledge and skills. The more complex they are, the harder it gets. It is especially difficult when knowledge and skills are tied to feelings through attitudes. Objectives within these areas often have to be less precise and have a longer time hori-zon. The problem with this is that it is easier to omit than to make an effort. The long-term and less precise objectives, whether they are tied to knowledge, skills, attitudes or all three, becomes an excuse for not following or for-mulating deliberate objectives at all.

The attitude objectives may be very difficult to obtain because they are tied to wishes and feelings which is developed, and is dif-ficult to change. It is not easy to change an eager youth who has not understood that he is not alone in the world, that certain forms of teamwork is necessary. The objectives may be complicated, and they may be tied to differ-ent levels. We call this taxonomy.

TAxONOMY OF EDUCATIONAL OBJECTIVES

The instructors expectations to the trainees may be placed on different levels. Taxono-mies are a way of grouping the objectives in different levels of difficulty, were the develop-ment from objective to objective is like a stair-case where the most difficult objectives are at the top. An example may be that the trainees first knows which parts that make an engine and the function of each part, then on the next step how the parts work together, then at the end the trainees assess how the engine may be disassembled and assembled based on the knowledge about functions and coop-eration.Benjamin Bloom developed a taxonomy of levels in the mid to late 50s. His taxonomies provide a stair-step approach to thinking about levels of learning. Most importantly, verbs associated with these levels help; 1) find an appropriate verb for the first step of writing an objective and 2) help us move to higher levels of more meaningful learning.

22

23

It is more difficult to measure changes of attitude than changes in knowledge and skills. In most cases it is possible to describe what the trainee shall do after the training is finished by means of the text of the learning-objective. If you feel that you miss a verb in term of demeanor, the reason may be that it is difficult to place on a certain level.

RECEIVING

RESPONDING

VALUING

ORGANIZING

CHARACTERIZING

LEVELS OF EDUCATIONAL OBJECTIVESAttitude-objectives (objectives in the affective domain)

24

AFFECTIVE LEVELS

RECEIVING

RESPONDING

VALUING

ORGANIZING

CHARACTERIZING

Examples of verbs in terms of demeanor that may be utilized in learn-ing objectives for attitudes

Pay attentionBe aware ofBe occupied withBe disposed forBe sensitive to

Obey, followEndureAccept, sustainTolerateVoluntary engage-mentDo

AcceptRecognizeAppreciateWish to developShow respect for

Identify withJudgeEstablishClarifyAssociate

GeneralizeExamineInvestigateRejectElectOppose

FightIntegrateGot through withChangeAdoptAssess

RealizeDevelopInfluenceBe marked by

FindShapeDevelopDecide

AssemblySe differences, con-nectionsAdjust

ImitatePreferLive byIdentify withAdapt

HeadTake initiative toTake care ofTake responsibility forBe bound by

ConductTake responsibility forStick your head intoSeekFollow-up

Be occupied byBe sensitive toBe happy forLikeEnjoyLook through

CaptureDetectFollowListenRegister

ConsiderObserveSee, speak, smell, sense, feel

25

SKILLS-OBJECTIVES (OBJECTIVES IN THE PSYCHOMOTOR DOMAIN)

PERCEPTION

READINESSTO ACT

IMITATION ANDPLANLESSTESTING

HABITUALACTION

COMPLEXSKILLS

ADAPT ANDDEVELOPSKILLS

26

PERCEPTIONPSYCHOMOTOR LEVELS:

READINESS TO ACT

IMITATION AND PLANLESS TESTING

HABITUAL ACTION

COMPLEx SKILLS

ADAPT AND DEVELOP SKILLS

Examples of verbs in terms of demeanor that may be utilized in learning objectives for skills. There are more possible verbs on each level than those mentioned here. Some of these may be tied to specific vocations or tasks.

DiscoverFocusRecognizeRegister

PrepareFocusParticipateFollow

ImitateCopyRepeatTry

ActAdaptPartAdjustUse

ShapeProduceCreateFitManufacture

ImproviseCombineCompleteGovern

Change, variationPrepareRepairRenew

MasterDevelopShow changeabilityControl

AccomplishRepair(Dis)assembleShapeTake apart

JoinApply powerShow speed, preci-sionPersistent

UtilizeGovernMeasureAssembleRepair

Act, doSelectMaintainEndureAccomplish

TestDoFollowRaise

PullLoosenTreatSelect

Show interestTouch/utilizeDecide/electAct/get

Expect, decide toPrefer, concentrate on

See, hear, listen, taste, smell, study, follow

Show attention, ob-serve, be aware of, receive impressions

27

KNOWLEDGE-OBJECTIVES (OBJECTIVES IN THE COGNITIVE DOMAIN

RE-COLLECTION

COMPRE-HENSION

ADAPTATION

ANALYSIS

SYNTHESIS

EVALUATION

A word that is used for perform-ing certain tasks, such as cut, saw etc, may be utilized in all the three/four highest levels. The rest of the text in the education-al objective decides what level of difficulty the objective is on.

28

RECOLLECTIONCOGNITIVE LEVELS:

COMPREHENSION

ADAPTATION

ANALYSIS

SYNTHESIS

EVALUATION

Examples of verbs in terms of demeanor that may be utilized in learn-ing objectives for knowledge.

RecognizeReproduceRepeatDenote

ReproduceExplainConfirmShow that

PredictSelectExplainUtilize

AnalyzeStudySeparateFind out

CombineReactStudyPropose

AssessEvaluateDiscussDefend

DecideStudyScrutinizeState reason for

Compare valueDistinguishApprove/rejectCriticize

ClarifyPlanSum upDocument

GeneralizeOrganizeFormulate rulesConclude

SelectConfirmPartExplore

ElaborateClassifyIdentifyCompare

ConstructFindCalculateUtilize

RegisterOrganizeDemonstrateTell (in own words)

InterpretShowPredictFormulate

Show (differences)TranslateDenoteRead

DefineDescribeReferName

ListDistinguishPresentUnderline

29

WRITING LEARNING OBJECTIVES: BEGINNING WITH THE END IN MINDA Learning Objective is a statement in specific and measurable terms that describes what the learner will know and/or be able to do and/or feel as a result of engaging this learn-ing activity.

Example: Students will list three characteris-tics that make the family medicine physician distinctive from other specialists in the health care system.

The ideal learning objective has 3 parts:

A measurable verb

The important condition (if any) under which the performance is to occur and

The criterion of acceptable performance (standard).

Frequently, as in the example above, you will not see the criterion or the condition specified if they are obvious. However, sometimes add-ing the condition(s) and/or the criterion adds clarity to a learning objective.

Effective Learning Objectives are:

Consistent with the goals of the curriculumClearly statedClearly measurableRealistic and doableAppropriate for the level of the learnerWorthy (Important stuff)The characteristics of effective objectives as stated above along with the verb, conditions and standard provide us with some criteria to evaluate objectives you construct.

QUICK OBJECTIVE WRITING TEMPLATE

Select a verb for performing the task and determine if the verb you have chosen best describes the type of behavior that the learn-ers need to display after training (see Blooms Taxonomy).Determine the conditions under which the task must be performed.Determine to what standards the task must be performed.

Bloom, B. S.et al (1956) - Taxonomy of educa-tional objectives: the classification of educa-tional goals; Handbook I: Cognitive Domain New York, Longmans, Green, 1956.

1

2

3

30

IF YOU DONT KNOW WHAT PORT YOU ARE SAILING TO, NO WIND IS FAVOURABLE.(SENECA THE YOUNGER, 4BC-AD65.)

31

ASSESSMENT 04ASSESSMENT IS A PART OF THE EDUCATIONAL PROCESS. THE ASSESSMENT MAY BE CONDUCT-ED BEFORE, DURING OR AFTER THE TRAINING.

ASSESSMENT BEFORE THE TRAINING COMMENCE

Before the training starts, it is common to as-sess the trainees prerequisites for learning. This is called diagnostic assessment. The pur-pose of this assessment is to get thorough infor-mation about the trainee or group of trainees. This information is the basis for further plan-ning of the training. Knowledge about the trainees prerequisites for learning is essential when preparing the learning objectives as it may affect both the selection of content and the methods.

ASSESSMENT DURING THE TRAINING

Assessment that takes place during the train-ing sequence is called formative assessment. The purpose of the evaluation during the training is in most cases to adjust the training according to the process. The instructor tries to catch signals from the trainees, and adjust the training accordingly. The trainees may lose concentration and be inattentive occa-sionally during the training. It is important that the instructor understands why this hap-pens. Is it caused by the arrangement? May-be the content/tasks are presented differently than usual? In that case the instructor must try to create a better connection between theory and practice.

One of the main purposes of the formative assessment is to map the trainees growth and development related to the objectives, so that the further educational process may be adapted in a best possible way. Training in a company will normally be carried out as practical guiding. In this case the instructor has a good opportunity to help the individual to a good learning result though conversa-tion, encouragement and constructive criti-cism. The guiding may in addition help the trainee to assess her own prerequisites in a realistic way, and to control the learning pro-cess herself.

One of the reasons that practical training of-ten works better than the teaching of theory, is probably that the formative assessment has a larger emphasis.

ASSESSMENT AFTER THE TRAINING IS COMPLETED

Assessment conducted after the training is completed is called summative assessment. The assessment can comprise questions, dem-onstrations, tests or observations that show to what degree the trainee has achieved the objectives (i.e. assessment of the result of the training). The summative assessment is not only about the trainees achievements. Even the learning process and the instructors train-ing can be assessed. The result of the assess-ment is the base for planning later training sessions.

32

WHY DO WE ASSESS?

As described in the previous sections, the as-sessment serves several purposes, however, . Primarily it is an educational tool. The trainee shall regularly get feedback of his work and development, enabling him to adjust during the learning process.

Using the assessment as a tool for his/her own development, the trainee can reflect on his/her own strengths and weaknesses and focus on his/her improvement areas.

The different elements of the training and learning process must be assessed on a regu-lar basis in relation to each other. All the as-pects of the The holistic model of didactic relations must be subject to regular assess-ment.

The instructor must find and apply means of assessment that serves the training and the trainees development best. Process oriented, guiding and informal assessment is more im-portant than control and formal assessment in this context.

FORMAL OR INFORMAL ASSESSMENT

Formal assessment means exercises, tests or examinations, which often has a form of approval. The emphasis is on the trainees achievement. Issues about the trainees per-sonality are not a part of the process.

Informal assessment means a daily assess-ment of the trainees work in the form of con-versations, praise and criticism. In addition to the subject at hand, the assessment also com-prises issues of a more personal nature, i.e. co-operative capabilities, effort and diligence.

The informal assessment normally takes place as a discussion between the instructor and the trainee. Many feel that objective-oriented, in-formal assessment is a good way to stimulate the trainees learning.

If the emphasis is on formal assessment, knowledge that can be measured by formal tests gets priority.

Knowledge, skills and attitudes that are not easily measured, may then be neglected. For-mal assessment in the form of grading may harm the trainees development. Grading may raise the competition among the train-ees, with subsequent loss of self-confidence for some, and exaggerated self-confidence for others as result.

33

THE MASTER SAID, YU, SHALL I TELL YOU WHAT KNOWLEDGE IS. WHEN YOU KNOW A THING, TO KNOW THAT YOU KNOW IT, AND WHEN YOU DO NOT KNOW A THING, TO RECOGNIZE THAT YOU DO NOT KNOW IT. THAT IS KNOWLEDGE.

KUNG FU TSE, 551 BC - 479 BC.:

34

35

ASSESSMENT OF TRAINING

The term assessment or evaluation has grad-ually received a complex meaning. Earlier most people were concerned about assess-ment by tests and grades. To review the train-ees knowledge was regarded as the most important. Today most people have become aware of more issues related to assessment.

For example; in a company the instructor wants to know how the arrangement, work-ing methods and learning process works in relation with the objectives for the training. In another situation the instructor may want to assess if the content/tasks suits a specific train-ees prerequisites. The instructor has to think through what shall be assessed, how the as-sessment shall take place and why the assess-ment must be done.

Both the instructors arrangement and the trainees learning may be assessed. The re-lation between tasks given and learning is often in focus. Has the trainee learned what he was meant to learn? What knowledge, at-titudes and skills related to the objectives has the trainee gained? If many of the trainees didnt reach their objectives, it is relevant to study the cause. Maybe the objectives were too difficult and should be placed on a lower level? Or maybe the trainees learning didnt proceed according to the instructors plan? Assessment of own working methods/adapta-tion of training is a help to the instructor to ad-just the training to the trainees prerequisites for learning.

REFERENCES FOR ASSESSMENT

A norm-referenced test is a type of test, assess-ment, or evaluation which yields an estimate of the position of the tested individual in a pre-defined population, with respect to the trait being measured. This estimate is derived from the analysis of test scores and possibly other relevant data from a sample drawn from the population. That is, this type of test identifies whether the test taker performed better or worse than other test takers, but not whether the test taker knows either more or less ma-terial than is necessary for a given purpose. The term normative assessment refers to the process of comparing one test-taker to his or her peers.

A criterion-referenced test is one that provides for translating test scores into a statement about the behavior to be expected of a per-son with that score or their relationship to a specified subject matter. Most tests and quiz-zes written by school teachers are criterion-referenced tests. The objective is simply to see whether the student has learned the mate-rial.

CRITERIONS FOR ASSESSMENT

A common misunderstanding regarding the term is the meaning of criterion. Many, if not most, criterion-referenced tests involve a cut score, where the examinee passes if their score exceeds the cut score and fails if it does not (often called a mastery test). The criterion is not the cut score; the criterion is the domain of subject matter that the test is designed to assess. For example, the criterion may be Stu-dents should be able to correctly add two sin-gle-digit numbers, and the cut score may be that students should correctly answer a mini-mum of 80% of the questions to pass.

The criterion-referenced interpretation of a test score identifies the relationship to the sub-ject matter. In the case of a mastery test, this does mean identifying whether the examinee has mastered a specified level of the subject matter by comparing their score to the cut score. However, not all criterion-referenced tests have a cut score, and the score can sim-ply refer to a persons standing on the subject domain.

36

Clarify the objective for the training:

Adapted so that the trainees had to solve problems:

Adapted so that the trainees had to be active themselves:

Give recognition for completed work:Create a sense of security so that the trainee dared to enter new and unknown learning tasks:

Adapted so the train-ees had to cooper-ate:

Give feedback on how the trainees mastered the tasks:

Comments:

TO WHAT ExTENT DID/WAS THIS TRAINING SESSION (CRITERIONS):

VERY MUCH MUCH SOME LITTLE VERY LITTLE

Example: form for assessment of a

subjects adaptation for learning

37

STRIKE AND/OR FILL IN AREAS SO IT FITS YOUR ACTIVITIES. VALUES NOTES

TRAINEE

UNDERSTANDING OF SUBJECT/VOCATION

CR

IT

ER

IO

NS

F

OR

A

SS

ES

SM

EN

T

PERFORMANCE OF SUBJECT/VOCATION

SOCIAL ADAPTATION/COOPERATIVE SKILLS

RELATION TO ENVIRONMENT AND SAFETY

ORDER AND PUNCTUALITY

WRITING OF LOG/SERVICE LOG

VOCATIONAL DEVELOPMENT

CONCLUSION

1 12 20

Example: Form for formative or summative

assessment with criterions, may serve as a checklist

for oral feedback

38

METHODS FOR ASSESSMENTTestsExaminationsObservationInterviewCertificationConversation

We must ask ourselves the following questions before we decide which assessment method to use:Is the method usable to measure what we want to know?

Is the method usable in this specific context?

(WHAT IS GOOD) (WHAT CAN BE IMPROVED)

WHY FORMAL ASSESSMENT?Regular feedback to the traineeVocationalstatusPersonaldevelopment

Documentation of the resultsObjectivesobtainedObjectivesnotobtained

Regular feedback from the traineeWhatworkswell?Whatshouldbechanged?

1

2

3

Plus delta, a tool for (semi-) informal formative

assessment

39

ASSESSMENT IS ABOUTDocumentationObjectivityFidelityPrecisionMethodsTechniques

Growth and developmentSelfesteemControl-powerParticipationReflectionInnovation

QUALIFIED JUDGMENT

The use of judgment takes place in all assess-ment. To assure the quality of the judgment it is important that qualified judgment is devel-oped. This can only happen in cooperation with other professionals, were you present your assessment and discuss the case.Judgment is based or a professional compe-tence where understanding is central. Further it will build on our assessment skills that can be summed up in these challenges:The ability to develop central criterionsThe ability to document according to these criterionsThe ability to document observationso The ability to interpret the observation and do an overall assessment of professional, voca-tional and affective relations.

Donald L Kirkpatricks training evaluation model - the four levels of learning evalua-tion. The four levels of Kirkpatricks evaluation model essentially measure:

reaction of student - what they thought and felt about the traininglearning - the resulting increase in knowledge or capabilitybehavior - extent of behavior and capability improvement and implementation/applica-tionresults - the effects on the business or envi-ronment resulting from the trainees perfor-mance

All these measures are recommended for full and meaningful evaluation of learning in organizations, although their application broadly increases in complexity, and usually cost, through the levels from level 1-4.

1

2

Kirkpatricks four levels of evaluation model

40

12

3

4

LEVELEVALUATION TYPE (WHAT IS MEASURED)

EVALUATION DESCRIPTION AND CHARACTERISTICS

ExAMPLES OF EVALUATION TOOLS AND METHODS

RELEVANCE AND PRACTI-CABILITY

Reaction

Learning

Behaviour

Results Results evaluation is the effect on the business or environ-ment by the trainee.

Measures are al-ready in place via normal manage-ment systems and reporting - the chal-lenge is to relate to the trainee.

Individually not dif-ficult; unlike whole organization. Process must attri-bute clear account-abilities.

Behaviour evalua-tion is the extent of applied learning back on the job - implementation.

Observation and interview over time are required to as-sess change, rel-evance of change, and sustainability of change.

Measurement of behavior change typically requires cooperation and skill of line-managers.

Learning evaluation is the measurement of the increase in knowledge - before and after.

Typically assess-ments or tests before and after the train-ing. Interview or obser-vation can also be used.

Relatively simple to set up; clear-cut for quantifiable skills. Less easy for com-plex learning.

Reaction evaluation is how the delegates felt about the train-ing or learning expe-rience.

Happy sheets, feed-back forms. Verbal reaction, post-training surveys or questionnaires.

Quick and very easy to obtain. Inexpensive to gath-er or to analyze.

Donald Kirkpatricks Learning Evaluation Model 1959; review and contextual material Alan Chapman 1995-2009. Retrieved from www.businessballs.com 2011. Full source material at http://www.businessballs.com/kirkpatrick-learningevaluationmodel.htm. Not to be sold or published. Alan Chapman/the author(s)/Businessballs accepts no li-ability for any issues arising.

41

WRITING MULTIPLE-CHOICE TEST ITEMS

BY JERARD KEHOE, VIRGINIA POLYTECHNIC IN-STITUTE AND STATE UNIVERSITY

A notable concern of many teachers is that they frequently have the task of constructing tests but have relatively little training or infor-mation to rely on in this task. The objective of this article is to set out some conventional wis-dom for the construction of multiple-choice tests, which are one of the most common forms of teacher-constructed tests. The com-ments which follow are applicable mainly to multiple-choice tests covering fairly broad topic areas.

Before proceeding, it will be useful to estab-lish our terms for discussing multiple-choice items. The stem is the introductory question or incomplete statement at the beginning of each item and this is followed by the options. The options consist of the answer -- the correct option -- and distracters--the incorrect but (we hope) tempting options.

GENERAL OBJECTIVES

As a rule, one is concerned with writing stems that are clear and parsimonious, answers that are unequivocal and chosen by the stu-dents who do best on the test, and distractors that are plausible competitors of the answer as evidenced by the frequency with which they are chosen. Lastly, and probably most important, we should adopt the attitude that items need to be developed over time in the light of evidence that can be obtained from the statistical output typically provided by a measurement services office (where tests are machine-scored) and from expert editorial review.

PLANNING

The primary objective in planning a test is to outline the actual course content that the test will cover. A convenient way of accomplishing this is to take 10 minutes following each class to list on an index card the important concepts covered in class and in assigned reading for that day. These cards can then be used later as a source of test items. An even more con-scientious approach, of course, would be to construct the test items themselves after each class. The advantage of either of these ap-proaches is that the resulting test is likely to be a better representation of course activity than if the test were constructed before the course or after the course, when we usually have only a fond memory or optimistic syllabus to draw from. When we are satisfied that we have an accurate description of the content areas, then all that remains is to construct items that represent specific content areas. In develop-ing good multiple-choice items, three tasks need to be considered: writing stems, writing options, and ongoing item development. The first two are discussed in this article.

42

WRITING STEMS

We will first describe some basic rules for the construction of multiple-choice stems, because they are typically, though not necessarily, writ-ten before the options.

Before writing the stem, identify the one point to be tested by that item. In general, the stem should not pose more than one problem, al-though the solution to that problem may re-quire more than one step.

Construct the stem to be either an incomplete statement or a direct question, avoiding ste-reotyped phraseology, as rote responses are usually based on verbal stereotypes. For ex-ample, the following stems (with answers in parentheses) illustrate undesirable phraseol-ogy:

What is the biological theory of recapitula-tion? (Ontogeny repeats phylogeny)Who was the chief spokesman for the Ameri-can System? (Henry Clay)

Correctly answering these questions likely de-pends less on understanding than on recog-nizing familiar phraseology.

Avoid including nonfunctional words that do not contribute to the basis for choosing among the options. Often an introductory statement is included to enhance the appropriateness or significance of an item but does not affect the meaning of the problem in the item. Gen-erally, such superfluous phrases should be ex-cluded. For example, consider:

The American flag has three colors. One of them is (1) red (2) green (3) blackversusOne of the colors of the American flag is (1) red (2) green (3) black

In particular, irrelevant material should not be used to make the answer less obvious. This tends to place too much importance on read-ing comprehension as a determiner of the correct option.

Include as much information in the stem and as little in the options as possible. For exam-ple, if the point of an item is to associate a term with its definition, the preferred format would be to present the definition in the stem and several terms as options rather than to present the term in the stem and several defi-nitions as options.

Restrict the use of negatives in the stem. Neg-atives in the stem usually require that the an-swer be a false statement. Because students are likely in the habit of searching for true statements, this may introduce an unwanted bias.

Avoid irrelevant clues to the correct option. Grammatical construction, for example, may lead students to reject options which are grammatically incorrect as the stem is stated. Perhaps more common and subtle, though, is the problem of common elements in the stem and in the answer. Consider the following item:

What led to the formation of the States Rights Party?a. The level of federal taxationb. The demand of states for the right to make their own lawsc. The industrialization of the Southd. The corruption of federal legislators on the issue of state taxation

One does not need to know U.S. history in or-der to be attracted to the answer, b.

Other rules that we might list are generally commonsensical, including recommenda-tions for independent and important items and prohibitions against complex, imprecise wording.

1 4

5

6

2

3

43

WRITING OPTIONS

Following the construction of the item stem, the likely more difficult task of generating op-tions presents itself. The rules we list below are not likely to simplify this task as much as they are intended to guide our creative efforts.

Be satisfied with three or four well constructed options. Generally, the minimal improvement to the item due to that hard-to-come-by fifth option is not worth the effort to construct it. In-deed, all else the same, a test of 10 items each with four options is likely a better test than a test with nine items of five options each.

Options which read none of the above, both a. and e. above, all of the above, _etc_., should be avoided when the students have been instructed to choose the best answer, which implies that the options vary in degree of correctness. On the other hand, none of the above is acceptable if the question is fac-tual and is probably desirable if computation yields the answer. All of the above is never desirable, as one recognized distractor elimi-nates it and two recognized answers identify it.

If possible, have a colleague with expertise in the content area of the exam review the items for possible ambiguities, redundancies or other structural difficulties. Having com-pleted the items we are typically so relieved that we may be tempted to regard the task as completed and each item in its final and per-manent form. Yet, another source of item and test improvement is available to us, namely, statistical analyses of student responses.

After the options are written, vary the loca-tion of the answer on as random a basis as possible. A convenient method is to flip two (or three) coins at a time where each possi-ble Head-Tail combination is associated with a particular location for the answer. Further-more, if the test writer is conscientious enough to randomize the answer locations, students should be informed that the locations are randomized. (Testwise students know that for some instructors the first option is rarely the answer.)

Construct distractors that are comparable in length, complexity and grammatical form to the answer, avoiding the use of such words as always, never, and all. Adherence to this rule avoids some of the more common sources of biased cueing. For example, we sometimes find ourselves increasing the length and spec-ificity of the answer (relative to distractors) in order to insure its truthfulness. This, however, becomes an easy-to-spot clue for the testwise student. Related to this issue is the question of whether or not test writers should take advan-tage of these types of cues to construct more tempting distractors. Surely not! The number of students choosing a distractor should depend only on deficits in the content area which the item targets and should not depend on cue biases or reading comprehension differences in favor of the distractor.

1

3

4

5

2

44

This article was adapted with from Testing Memo 4: Con-structing Multiple-Choice Tests -- Part I, Office of Measure-ment and Research Services, Virginia Polytechnic Institute and State University, Blacksburg, VA 24060

Further ReadingAirasian, P. (1994) Classroom Assessment, Second Edition, NY: McGraw-Hill.

Cangelosi, J. (1990) Designing Tests for Evaluating Student Achievement. NY: Addison Wellesley.

Grunlund, N (1993) How to make achievement tests and as-sessments, 5th edition, NY: Allen and Bacon.

Haladyna, T.M. & Downing, S.M. (1989) Validity of a Taxono-my of Multiple-Choice Item-Writing Rules. Applied Measure-ment in Education, 2 (1), 51-78.

Descriptors: *Culture Fair Tests; *Distractors (Tests); Educa-tional Assessment; Item Bias; Measurement Techniques; *Multiple Choice Tests; Scoring; *Statistical Analysis; Stereo-types; *Test Construction; Test Items; Test Theory

Writing multiple-choice test items. Practical Assessment, Research & Evaluation, 4(9). Kehoe, Jerard (1995). Re-trieved April 26, 2011 from http://PAREonline.net/getvn.asp?v=4&n=9 . This paper has been viewed 97,152 times since 11/13/1999.

Copyright 1995, PAREonline.net.Permission is granted to dis-tribute this article for nonprofit, educational purposes if it is copied in its entirety and the journal is credited. Please notify the editor if an article is to be used in a newsletter.

45

PART 2

46

47

DEVELOPMENTOF

TRAINING 05BY JOHN CLARK (HTTP://WWW.NWLINK.COM/~DONCLARK/INDEX.HTML)

INSTRUCTIONAL SYSTEM DEVELOPMENT (ISD)

When some people see or hear the word sys-tem, they often think of mega-methodologies that require several bookcases and intense training to use. ISD is not that difficult or com-plicated. The ISD model was designed to solve human performance problems related to learning or training (U.S. Department of De-fense, 1975). The figure shown below is a flow-chart of the ISD model. It was first established by Florida State University in conjunction with the Department of Defense, but can now be found in almost any type of organization (Wat-son, 1981). It grew out of the systems analysis concepts that became popular after World War II and is probably the most extensively used instructional design model in use today.

The flowchart model shown to the right lists the five phases along with their basic steps. While it does a good job of showing the pro-cesses and steps, it does not really show the dynamics of the ISD model. In addition, it only shows a selected number of steps under each phase that may or may not be needed for a particular learning process and omits steps that may be needed for a particular learning process. A better diagram is achieved using the following model:

48

Florida State University Five Phases of ISD (1975)

ANALYZE JOB

PHASE 1ANALYSES

PHASE 2DESIGN

PHASE 3DEVELOPMENT

PHASE 4IMPLEMENTATION

PHASE 5EVALUATION AND

CONTROLL

SELECT TASK FUNCTIONS

CONSTRUCT JOB PERFORMANCE

MEASURES

ANALYZE ExISTING COURSES

SELECT INSTRUCTIONAL

SETTINGVALIDATE

INSTRUCTION

DETERMINE SEQUENCE & STRUCTURE DEVELOP INSTRUCTION

DESCRIBE ENTRY BEHAVIOR

REVIEW/SELECT ExISTING MATERIALS REVISE SYSTEM

DEVELOP TESTS

SPECIFY INSTRUCTIONAL

MANAGEMENT PLAN & DELIVERY SYSTEM

CONDUCT INSTRUCTION

CONDUCT ExTERNAL EVALUATION

DEVELOP OBJECTIVESSPECIFY LEARNING EVENTS/ACTIVITIES

IMPLEMENT INSTRUCTIONAL

MANAGEMENT PLANCONDUCT INTERNAL

EVALUATION

49

ANALYSIS

DESIGN

DEVELOP

IMPLEMENT

EVALUATE

50

DYNAMICS OF ISD

This figure highlights the importance of eval-uation and feedback throughout the entire training program. It also stresses the impor-tance of gathering and distributing informa-tion in each of the five phases and shows the training process is NOT a waterfall model (static and linear), but rather an iterative flow of activities (dynamic and iterative, or spiral-ing).

Jeroen Merrinboer (1997) noted that while the ISD model may be listed in linear order (as shown in the first model), it is typically performed in an iterative and cyclic fashion. Hence the need to better diagram the model in the second figure.

The five phases are ongoing activities that continue throughout the life-cycle of a learn-ing program. After building the learning pro-gram, the other phases do not end once the learning process has began, but are continu-ally repeated as new challenges are encoun-tered.

51

ISD IS LIKE BUILDING A DELI SANDWICH

Using ISD to create a training program is simi-lar to shopping at a deli. In one process you create a sandwich, in another you create training. When some people see or hear the word ISD, they think of mega-methodologies. However, ISD is simply a common sense ap-proach to training. It does this by analyzing the problem, designing a specification, devel-oping the courseware, implementing the so-lution, and performing evaluations through-out the process.

ANALYSIS

Jumping straight into developing a training program is the same as heading for the deli to buy something nutritional and ending up at the candy shop buying junk-food. If you do not analyze the problem, you might end up with junk-training, which is just like junk-food - it provides no nutritional value and too much of it is harmful. Just as it requires some study to find out what is really healthy for you, it re-quires some study to find the real cause of a performance problem. The analysis phase is simply gaining an understanding of the prob-lem and determining the scope of the project before you attempt to solve it. In some cases it will require a few hours of thought and re-search, while complex cases might require several weeks of intense research. The end result should be a well thought out and researched goal that is result driven. This ensures that the training will have a positive impact upon the organization.

52

SUMMARY:

Build the objectives or goals. Build a test or measurement instrument. Build the activities that will lead to perfor-mance.Construct the Performance Steps - how do we go from a non-performer to a performer?

EVALUATION: Do the performance steps proceed logically from the objectives to task performance? Trial and test the evaluation instrument

DESIGN

The design and development phase can be thought of as the deli sandwich. First comes the objectives, which is top layer of bread, and then the tests, which is the bottom layer of bread. You then decide the type of meats and cheeses and how they should go in your sandwich (the performance steps). In the de-velopment phase, you will actually add the meat, cheese and various condiments. Just as you start building a sandwich with two slices of bread (the beginning and the end), you start the design and development by building the beginning (objectives) and the end (tests).

THE PERFORMANCE STEPS

The performance steps or learning steps is the basic outline that gets you from one layer of bread to the next. That is, you have your objectives (top layer of bread); then you have the various performance steps (meat and cheeses); and on the other end is the test (bot-tom layer of bread). Noticed how the training program is started by identifying the outcomes - the objectives. They tell you what must be done while the tests describe how the outcomes will be veri-fied, and finally the steps guide you in getting from the objectives to successful test comple-tion.

THE OBJECTIVES

The Learning Objectives tell you what should be taking place upon completion of the train-ing. In some cases you will be using dark rye bread by building Tasks, Conditions, and Standards; while in other cases you want a lighter wheat bread by building simple goals. Normally, complex training solutions require Learning Objectives, while training solutions that are more educational or developmental in nature might only require simple goals.

THE TESTS

Testing ensures that the objectives or goals are accomplished. Again, sometimes you want a heavy dark bread and will build hands-on performance tests or written tests, while at other times you want home-style bread and will simply check the learners when they re-turn to the workplace. But in any case, there is no use wasting your time building a solution if you are not going to measure it in some way. At the very least drop in on them at the work-place to see how the training has impacted their performance. . . anything less is uncivi-lized.

53

SUMMARY:

Ensure you understand the problem enough so that you can describe it. Work with the cli-ents to obtain a good grasp of the problem. Verify the problem. Observation is the best. Ensure it is a training problem rather than an-other performance problem.Use Subject Matter Experts (SME) to help you. Brainstorm a solution with the clients. Make them part of the solution. What is the exact task that must be performed and the performance gap that is causing the problem? What major media will best deliver the perfor-mance solution?

EVALUATION: Does your solution solve a business problem (positive impact upon the organization)? Is it a training problem or a different performance problem? Is there a similar solution available? Do NOT fall into the not invented here syndrome. That is, if it was not built by us, we are not go-ing to use it. Do not reinvent the wheel as it saves resources.

DEVELOP

Development goes hand in hand with Design. By performing the design phase first - the out-comes, you simplify the developmental pro-cess. It is much easier to build solutions when you know the desired outcome. Development is simply cutting and layering the meat and cheeses according to plans. This can be thought of as the instructional methods. You also add various fillings or condiments (things that add spice and crunch) that makes for not only a nutritional sandwich, but one which is also tasty to eat. Learning is the same way...you can develop instruction, but if it is not tasty to the learners, they will not consume it.

See the chapter Developing the courseware for ideas.

IMPLEMENT

It is now time to eat the sandwich (yes!) - the instruction is implemented.

Use learner feedback to enhance the pro-gram.

EVALUATION:

Does it solve the learners problems?

Does it solve the business units problem?

EVALUATION

Note that the evaluation, while traditionally discussed last, is actually performed through-out the process. When building a sandwich, you check each ingredient to ensure it is fresh and wholesome. You do not knowingly put bad meat or rotten lettuce in it and then hope it turns out OK. Evaluating ISD is the same. You should not spend all that time building a solu-tion, and then waiting until the end to see if it works.

You evaluate in the analysis phase by deter-mining if it really is a training problem. You also evaluate to see if there is a similar solu-tion (no need to reinvent the wheel). In the

54

design phase, the objectives are evaluated through the tests. That is, do the performance steps logical link the objectives and the tests together. The development phase has no built in evaluations but you could perform rapid-prototyping (implement-evaluate-implement-evaluate-etc) or have another training spe-cialist check your solutions.

The implementation is the final testing ground. Sometimes you might have to add a side-dish (another performance solution) or another sandwich (more training) or change the in-gredients (meet the learners expectations) to complete the nutritional void (performance gap).

SUMMARY:

Performed throughout entire process. During the:

analysis phase - ensure you identified the learning gaps and that your findings point you towards a proper solution.

design phase - create learning objectives that will solve the performance gap and the tests or performance instruments actually test what they are supposed to test.

development phase - use rapid prototyping techniques to ensure the instruction design works correctly.

implementation phase - get feedback on course in order to improve it.

evaluation phase - ensure the training does what it is supposed to do -- eliminate the per-formance gap.

55

56

IF YOU DONT KNOW WHERE YOU ARE GOING YOU WILL PROBABLY END UP SOMEWHERE ELSE.

(LAURENCE PETER, 1919-90, CANADIAN ACADEM-IC AND ExPERT ON ORGANIZED HIERARCHIES.)

57

FIRST PRINCIPLES OF

INSTRUCTION 06An adaptation of m. David merrills model for training by http://edutechwiki.Unige.Ch/en/

First principles of instruction is a attempt by M. David Merrill to identify fundamental invari-ant principles of good instructional design, re-gardless pedagogic strategy. It can be used both as an instructional design model and as evaluation grid to judge the quality of a ped-agogical design. First principles of instruction is the title of a frequently cited on-line paper in several versions (e.g. http://mdavidmerrill.com/Papers/firstprinciplesbymerrill.pdf).

THE FIVE PRINCIPLES OF INSTRUCTION Merrills first and central principle of instruc-tion is task-centered learning. Task centered learning is not problem-based learning, al-though it shares some features.

THE TASK / PROBLEMA task is a problem that represents a prob-lem that may be encountered in a real-world situation. Learning objectives or samples of the types of problems learners will be able to solve at the end of the learning sequence may also substitute for a problem. A progres-sion through problems of increasing difficulty are used to scaffold the learning process into manageable tiers of difficulty.

Does the courseware relate to real world problems?

... show learners the task or the problem they will be able to do/solve ?

...are students engaged at problem or task level not just operation or action levels?

... involve a progression of problems rather than a single problem?

This progressive teaching approach is also re-lated to Merrinboers 4C/ID model.

1

2

3

58

59

INTEGRATION

APPLICATION DEMONSTRATION

ACTIVATIONLearning is pro-moted when learn-ers collaborate and critique.

Learning is promot-ed when learners apply their newly aquired knowledge and skill.

Learning is pro-moted when learners

observe a demon-stration of the skills to

be learned.

Learning is promot-ed when learners

recall prior experi-ence.

PROBLEMLearning is promoted when learners ac-quire knowledge and skills in the context of real world problems or tasks.

60

(2) Parts-of practice requires the learners to locate, name, and/or describe each part. (3) Kinds-of practice requires learners to identify new examples of each kind. (4) How-to prac-tice requires learners to do the procedure. (5) What-happens practice requires learners to predict a consequence of a process given conditions, or to find faulted conditions given an unexpected consequence.

Does the courseware require learners to use new knowledge or skill to solve a varied se-quence of problems and do learners receive corrective feedback on their performance?

In most application or practice activities, are learners able to access context sensitive help or guidance when having difficulty with the instructional materials? Is this coaching grad-ually diminished as the instruction progress-es?

Integration in effective instruction occurs when learners are given the opportunity to demostrate, adapt, modify and transform new knowledge to suit the needs of new con-texts and situations. Reflection through dis-cussion and sharing is important to making new knowledge part of a learners personal store and giving the learner a sense of prog-ress. Collaborative work and a community of learners can provide a context for this stage.

Are learners encouraged to integrate (trans-fer) the new knowledge or skill into their ev-eryday life?

Is there an opportunity to publicly demon-strate their new knowledge or skill?

Is there an opportunity to reflect-on, discuss, and defend new knowledge or skill?

Is there an opportunity to create, invent, or explore new and personal ways to use new knowledge or skill?

is there an opportunity to demonstrate previ-ously acquired knowledge or skill ?

Demonstration through simulations, visualiza-tions, modelling, etc. that exemplify what is being taught are favoured. Demonstration in-cludes guiding learners through different rep-resentations of the same phenomena through extensive use of a media, pointing out varia-tions and providing key information.

Does the courseware demonstrate what is to be learned ?

Are examples consistent with the content be-ing taught? E.g. examples and non-examples for concepts, demonstrations for procedures, visualizations for processes, modeling for be-havior?

Are learner guidance techniques employed? (1) Learners are directed to relevant informa-tion?, (2) Multiple representations are used for the demonstrations?, (3) Multiple demonstra-tions are explicitly compared?

Is media relevant to the content and used to enhance learning?

Application requires that learners use their knew knowledge in a problem-solving task, using multiple yet distinctive types of prac-tice Merrill categorizes as information-about, parts-of, kinds-of, and how-to practice that should be used depending upon the kind of skill and knowledge identified. The applica-tion phase should be accompanied by feed-back and guidance that is gradually with-drawn as the learners capacities increase and performance improves.

Can learners practice and apply acquired knowledge or skill?

Are the application (practice) and the post test consistent with the stated or implied objec-tives? (1) Information-about practice requires learners to recall or recognize information.

PHASES / COMPONENTS OF MERRILLS FIRST PRINCIPLES OF INSTRUCTION.The task (or problem) is center stage. Here is a summary of the four remaining components:

Activation of relevant previous experience promotes learning by allowing them to build upon what they already know and giving the instructor information on how to best di-rect learners. Providing an experience when learners previous experience is inadequate or lacking to create mental models upon which the new learning can build. Activities that stimulate useful mental models that are analoguous in structure to the content being taught can also help learners build appropri-ate schemas to incorporate the new content. Does the courseware activate prior knowl-edge or experience?

do learners have to recall, relate, describe, or apply knowledge from past experience (as a foundation for new knowledge) ?

does the same apply to the present course-ware ?

THE FIVE PRINCIPLES OF INSTRUCTION (MERRILL, 2006)THE DEMONSTRATION PRINCIPLE: Learning is promoted when learners observe a demonstration

THE APPLICATION PRINCIPLE: Learning is promoted when learners apply the new knowledge

THE ACTIVATION PRINCIPLE: Learning is promoted when learners activate prior knowledge or experience

THE INTEGRATION PRINCIPLE: Learning is promoted when learners inte-grate their new knowledge into their every-day world

THE TASK-CENTERED PRINCIPLE: Learning is promoted when learners engage in a task-centered instructional strategy

1

2

3

4

61

62

HOW-TOTell about steps and sequencePortrayal: Illustrate steps for specific cases (work-through examples)

WHAT-HAPPENSTell about the process as a whole, conditions, consequencesPortrayal: Illustrate specific conditions and consequences for specific cases

In the third (application) phase students have to work on skills related to portrayals and then put things together in the forth (integration) phase.

Each increasingly difficult whole task (prob-lem) requires going back and forth from (1) demonstration of the whole task (2) to compo-nent teaching and (2) back to integration. Once the whole task is mastered, this proce-dure is repeated which the next whole task until the real world problem is mastered without much direct component teaching.

= PROBLEM / WHOLE TASK

= COMPONENTS

= GUIDANCE

IMPLICATIONS FOR EDUCATIONAL TECHNOLOGY

THE TASK-CENTERED PRINCIPLE

Learning is promoted when learners en-gage in a task-centered instructional strategy and when a progression through problems of increasing difficulty is used to scaffold the learning process into manageable tiers of dif-ficulty and whole-tasks are broken down to part-tasks (components)

THE INSTRUCTIONAL SEQUENCE IN MERRILLS FIRST PRINCIPLES OF INSTRUCTION.

To design the first four phases (activation - demonstration - application - integration), whole tasks have to be broken down into com-ponents and the components have to be ana-lyzed. Then one has to decide what should be taught in what way.Merrill suggests to teach individual com-ponents with a direct instruction approach (which is more efficient and often also more effective). Most tasks or problems include five different instructional components. Firstly. initial telling should always activate prior knowledge. Demonstration (phase 2) should focus on adequate portrayals of components (but linked to the whole), before the applica-tion phase is entered. Here are few hints on how to tell/demonstrate different sorts of com-ponents:

INFORMATION-ABOUTTell facts or associations and link them to pre-vious knowledge

PARTS-OFTell names and descriptionsPortrayal: Show location

KINDS-OFTell definitionPortrayal: Show examples and counter-exam-ples

63

PROBLEM

INTEGRATIONMOTIVATION

NAVIGATION

INTERACTIONCO

LLAB

ORAT

ION

APPLICATION DEMONSTRATION

ACTIVATION

64

A FEW PRINCIPLES FOR TEACHING MATERIALS AND LEARNING ACTIVITIES

Learning environ-ments should be in-teresting, relevant and achievable

Real tasks are more motivating than for-mal objectives, glitz and novelty

Known content is not motivating, students should be able to skip over

Performing whole tasks is more motivat-ing then decontex-tualized actions and operations

Navigation is not in-teraction (i.e. it is not cognitive interactiv-ity)

Interaction means solving real-world problems or tasks

Key elements are: a context, a challenge, a learner activity and feedback.

Favor small groups (2-3) to optimize in-teractions

Group assigments should be structured around problems (whole tasks), i.e. real products or processes

Learners should see how contents are or-ganized

They should be able go forth and back, correct themselves

65

BY THREE METHODS WE MAY LEARN WISDOM:

FIRST, BY REFLECTION, WHICH IS NOBLEST;

SECOND, BY IMITATION, WHICH IS EASIEST; AND THIRD BY ExPERIENCE,

WHICH IS THE BITTEREST. (KUNG FU TSE, 551 BC - 479 BC.)66

67

LEARNINGSTYLES07

The Multiple Intelligences concepts and VAK (or VARK or VACT) learning styles models of-fer relatively simple and accessible methods to understand and explain peoples preferred ways to learn and develop. Occasionally well-intentioned people will write that the use of such models and tests is wrong because it pigeon-holes people, and ignores the point that we are all a mixture of styles and prefer-ences, and not just one single type, which is true. Please remember that over-reliance on, or extreme interpretation of, any methodol-ogy or tool can be counter-productive.

In the case of the Multiple Intelligences model, and arguably to greater extent VAK (because VAK is such a simple model), remember that these concepts and tools are aids to under-standing overall personality, preferences and strengths - which will almost always be a mix-ture in each individual person.

Therefore, as with any methodology or tool, use Multiple Intelligences concepts, VAK and other learning styles ideas with care and in-terpretation according to the needs of the situation.

Multiple Intelligences concept: Howard Gardner 1983; review and other material: Alan Chapman 2003-2009. Retrieved from www.businessballs.com 2011. Full source ma-terial at http://www.businessballs.com/how-ardgardnermultipleintelligences.htm. Not to be sold or published. Alan Chapman/the author(s)/Businessballs accepts no liability for any issues arising.

68

INTELLIGENCE TYPE CAPABILITY AND PERCEPTION

LINGUISTIC WORDS AND LANGUAGE

LOGICAL-MATHEMATICAL LOGIC AND NUMBERS

MUSICAL MUSIC, SOUND, RHYTHM

BODILY-KINESTHETIC BODY MOVEMENT CONTROL

SPATIAL-VISUAL IMAGES AND SPACE

INTERPERSONAL OTHER PEOPLES FEELINGS

INTRAPERSONAL SELF-AWARENESS

HOWARD GARDNERS MULTIPLE INTELLIGENCES THEORYHoward Gardners Multiple Intelligence The-ory was first published in Howard Gardners book, Frames Of Mind (1983), and quickly became established as a classical model by which to understand and teach many aspects of human intelligence, learning style, person-ality and behavior - in education and indus-try. Howard Gardner initially developed his ideas and theory on multiple intelligences as a contribution to psychology, however Gard-ners theory was soon embraced by educa-tion, teaching and training communities, for whom the appeal was immediate and irresist-ible - a sure sign that Gardner had created a classic reference work and learning model.

This simple grid diagram illustrates Howard Gardners model of the seven Multiple Intel-ligences at a glance.

Gardner said that multiple intelligences were not limited to the original seven, and he has since considered the existence and definitions of other possible intelligences in his later work. Despite this, Gardner seems to have stopped short of adding to the seven (some might ar-gue, with the exception of Naturalist Intel-ligence) with any clearly and fully detailed additional intelligence definitions. This is not because there are no more intelligences - it is because of the difficulty of adequately and satisfactorily defining them, since the addi-tional intelligences are rather more complex than those already evidenced and defined.

Not surprisingly, commentators and theorists continually debate and interpret potential additions to the model, and this is why you might see more than seven intelligences listed in recent interpretations of Gardners model. As mentioned above, Naturalist Intelligence seems most popularly considered worthy of inclusion of the potential additional Gardner intelligences.

Roles and intrapersonal intelligence: Given that a role tends to imply external style/skills, engagement, etc., the intrapersonal ability is less liable to define or suggest a certain role or range of roles than any of the other charac-teristics. That said, there is a clear correlation between intrapersonal ability/potential and introverted non-judgmental roles/working styles. Intrapersonal capability might also be seen as the opposite of ego and self-projec-tion. Self-awareness is a prerequisite for self-discipline and self-improvement. Intraperson-al capacity enables an emotionally mature (grown-up) response to external and internal stimuli. The intrapersonal characteristic might therefore be found among (but most definite-ly not extending to all) counselors, helpers, translators, teachers, actors, poets, writers, musicians, artists, and also any other role to which people can bring emotional maturity, which commonly manifests as adaptability, flexibility, facilitation, reflection, and other grown-up behaviors. There are also asso-ciations between intrapersonal capacity and Eriksons generative perspective, and to an extent Maslows self-actualization, that is to say: both of these life-stages surely demand a reasonably strong level of self-awareness, without which adapting ones personal life, outlook and responses to ones environment is not easy at all.

69

INTELLIGENCE TYPE INTELLIGENCE DESCRIPTION TYPICAL ROLES, PREFERENCES, POTENTIAL

LINGUISTIC

LOGICAL-MATHEMATICAL

MUSICAL

BODILY-KINESTHETIC

SPATIAL-VISUAL

INTERPERSONAL

INTRAPERSONAL

WORDS AND LANGUAGE, written and spoken; retention, in-terpretation and explanation of ideas and information via language, understands relationship between communica-tion and meaning

Writers, lawyers, journalists, speakers, trainers, copywrit-ers, English teachers, poets, editors, linguists, translators, PR consultants, media consultants, TV and radio presenters, voice-over artistes

scientists, engineers, computer experts, accountants, statis-ticians, researchers, analysts, traders, bankers, bookmak-ers, insurance brokers, negotiators, deal-makers, trouble-shooters, directors

musicians, singers, composers, DJs, music producers, pia-no tuners, acoustic engineers, entertainers, partyplanners, environment and noise advisors, voice coaches

dancers, demonstrators, actors, athletes, divers, sportspeo-ple, soliders, fire-fighters, PTIs, performance artistes; er-gonomists, osteopaths, fisherman, drivers, craftspeople; gardeners, chefs, acupuncturists, healers, adventurers

artists, designers, cartoonists, story-boarders, architects, photographers, sculptors, town-planners, visionaries, in-ventors, engineers, cosmetics and beauty consultants

therapists, HR professionals, mediators, leaders, counsel-lors, politicians, educators, sales-people,dergy, psycholo-gists, teachers, doctors, healers, organisers, carers, adver-tising, professionals, coaches and mentors; (there is clear association between this type of intelligence and what is now termed Emotional Intelligence or EQ

arguably anyone who is self-aware and involved in the process of changing personal thoughts, beliefs and be-haviour in relation to their situation, other people, their purpose and aims - in this respect there is a similarity to Maslows Self-Actualisation level, and again there is clear association between this type of intelligence and what is now termed Emotional Intelligence or EQ

LOGICAL THINKING, detecting patterns, scientific reasoning and deduction; analyse problems, perform mathematical calculations, understands relationship between cause and effect towards a tangible outcome or result

MUSICAL ABILITY, awereness, appreciation and use of sound; recognition of tonal and rhythmic patterns, under-stands relationship between sound and feeling

BODY MOVEMENT CONTROLL, manual dexterity, physical agility and balance; eye and body coordination

VISUAL AND SPATIAL PERCEPTION; interpretation and cre-ation of visual images; pictorial imagination and expres-sion; understands relationship between images and mean-ings, and between space and effect

PERCEPTION OF OTHER PEOPLES FEELINGS; ability to relate to others; interpretation of behaviour and communica-tions; understands the relationships between people and their situations, including other people.

SELF-AWARENESS, personal cognisance, personal objectiv-ity, the capability to understand oneself, ones relationship to others and the world, and ones own need for, and reac-tion to change

70

RELATED TASKS, ACTIVITIES OR TESTS PREFERRED LEARNING STYLE

write a set of intructions; speak on a subject; edit a written piece or work; write a speech; commentate on an event; apply postive or negative spin to a story words and language

numbers and logic

music, sounds, rythm

physical experience and movement, touch and feel

pictures, shapes, images, 3D space

human contact, commu-nications, cooperation, teamwork

self-reflection, self-discov-ery

A Chapman and Chis-lett MSc 2005, based on Gardners Mulitiple Intelli-gence Model. From www.businessballs.com. Not to be sold or published. The authors accept no liability. See http://www.business-balls.com/howardgardner-multipleintelligences.htm for free training materials and multiple intelligences tests.

perform a mental arithmetic calculation; create a process to measure something difficult; analyse how a machine works; create a process; devise a strategy to achieve an aim; assess the value of a business or a proposition

perform a musical piece; sing a song, review a musical work; coach someone to play a musical instrument; specify mood music for telephone systems and receptions

juggle; demonstrate a sports technique; flip a beer-mat; create a mime to explain something; toss a pancake; fly a kite; coach workplace posture, assess work-station ergo-nomics

design a costume; interpret a painting; create a room lay-out; create a corporate logo; design a building; pack a suitcase or the booth of a car

interpret moods from fadal expressions; demonstrate feel-ings through body language; affect the feelings of others in a planned way; coach or counsel another person

consider and decide ones own aims and personal chang-es required to achieve them (not necessarily reveal this to others); consider ones own Johari Window, and decide options for development; consider and decide ones own position in relation to the Emotional Intelligence model

71

GARDNERS MULTIPLE INTELLIGENCES - PRINCIPLES AND INTERPRETATION

Howard Gardner asserts certain principles relating to his multiple intelligence theory, which are explained and interpreted here, along with implications and examples:

The multiple intelligences theory represented/represents a definition of human nature, from a cognitive perspective, ie., how we perceive; how we are aware of things.

This provides absolutely pivotal and inescap-able indication as to peoples preferred learn-ing styles, as well as their behavioural and working styles, and their natural strengths. The types of intelligence that a person pos-sesses (Gardner suggests most of us are strong in three types) indicates not only a persons ca-pabilities, but also the manner or method in which they prefer to learn and develop their strengths - and also to develop their weak-nesses.

So for example: A person who is strong musically and weak

numerically will be more likely to develop nu-merical and logical skills through music, and not by being bombarded by numbers alone.

A person who is weak spatially and strong nu-merically, will be more likely to develop spa-tial ability if it is explained and developed by using numbers and logic, and not by asking them to pack a suitcase in front of an audi-ence.

A person who is weak bodily and physically and strong numerically might best be encour-aged to increase their physical activity by encouraging them to learn about the math-ematical and scientific relationships between exercise, diet and health, rather than forcing them to box or play rugby.

The pressure of possible failure and being forced to act and think unnaturally, have a significant negative influence on learning ef-fectiveness. Happy relaxed people learn more readily than unhappy stressful people.

A persons strength is also a learning channel. A persons weakness is not a great learning channel. Simple huh?

When you add in what we know about per-sonal belief and confidence it all begins to make even more sense. Develop people through their strengths and we not only stimu-late their development - we also make them happy (because everyone enjoys working in their strength areas) - and we also grow their confidence and lift their bel