Embed Size (px)
Citation preview
Presented By James Henderson
Designing and Conducting Formative Evaluations
The concept underlying this chapter is formative evaluation, which is the process designers use to obtain data for revising their instruction to make it more efficient and effective. There are three basic phases of formative evaluation.
Background
OBJECTIVE To discuss and give clarity to the following subjects: Formative Evaluation Designs Role of Subject-Matter, Learning, and Learner Specialists in Formative
Evaluation One-to-One Evaluation with Learners Small-Group Evaluation Field Trial Formative Evaluation in the Performance Context Formative Evaluation of Selected Materials Formative Evaluation of Instructor-Led instruction Data Collection for Selected Materials and Instructor-Led instruction Concerns Influencing Formative Evaluation Problem Solving during Instructional Design
Formative EvaluationThe process is used by designers to make their instructions more
effective and efficient.A formative evaluation is the collection of data and information
during the development of instruction that can be used to improve the effectiveness of the instruction.
It was later found better to try out earlier components of the design process, thereby avoiding a lot of problems that would otherwise not be discovered until after the draft of the instruction was complete.
They were originally used as a process to improve instruction after the first draft of instruction was developed.
FORMATIVE EVALUATION DESIGNS These are questions that can be asked to evaluate the components of the instructional strategy: 1. Are the materials appropriate for the type of learning outcome2. Do the materials include adequate instruction on the subordinate skills, and are these skills sequenced and clustered logically3. Are the materials clear and readily understood by representative members of the target group4. What is the motivational value of the materials? Do the learners find the materials relevant to their needs and interests? Are they confident as they work through the materials? Are they satisfied with what they have learned 5. Can the materials be managed efficiently in the manner they are mediated
• It’s important to have the instruction reviewed by specialists. • (SMEs) may be able to comment on the accuracy and currency of
the instruction. • Learning specialist may be able to critique your instruction
related to what is known about enhancing that particular type of learning.• Learner specialist may be able to provide insights into the
appropriateness of the material for the eventual performance context.
ROLE OF SUBJECT-MATTER, LEARNING, AND LEARNER SPECIALISTS IN FORMATIVE EVALUATION
• The purpose of the first stage of the formative evaluation, is to identify and remove the most obvious errors in the instruction and to obtain initial performance indications and reactions to the content by learners There are three criteria: • 1. Clarity – Is the message, or what is being presented, clear to individual target learners • 2. Impact – What is the impact of the instruction on individual learner’s attitude and achievement of the objectives and goals• 3. Feasibility – How feasible is the instruction given the available resources (time/context)?
ONE-TO-ONE EVALUATION WITH LEARNERS
• Small group evaluation have two primary purposes.• The first one is to determine the effectiveness of changes made
following the one-to- one evaluation and identify any remaining learning problems that learners may have. • Secondly, small group evaluation is used to determine whether
learners can use the instruction without interacting with the instructor.
SMALL GROUP EVALUATION
• Final stage of formative evaluation• Instructor attempts to use a learning context that closely
resembles the intended context• Purpose is to determine whether that changes in the instruction
made after the small-group we effective
FIELD TRIAL
• The purpose of the performance context formative evaluation is to make three fundamental determinations • Do the learners find that it is appropriate to use their new skills in
the workplace and have they been doing so • If they have been used, what has been the impact on the
organization• What suggestions do the learners and other that they work with
have for improving the instruction
FORMATIVE EVALUATION IN THE PERFORMANCE CONTEXT
• Selected materials formative evaluation concludes that the instructor chooses existing materials be there is seldom time to or resources to conduct all faces of the evaluation. In this case, the instructor should proceed directly to the field trial. • The instructor who conducts a field trial is able to observe the
progress and attitudes of learners using a set of adopted or adapted materials.
FORMATIVE EVALUATION OF SELECTED MATERIALS
• Formative evaluation of instructor-led instruction is done to determine whether the instruction is effective and decide how to improve it. It is functions just like the field trial. • To prepare, the instructor must be concerned with the entry skills
and prior knowledge, the posttest knowledge, and that attitudes of learners. • This is an opportunity to evaluate instructional procedures.• Many instructions already use these types of formative
evaluation in their instructions
FORMATIVE EVALUATION OF INSTRUCTOR-LED INSTRUCTION
• Information dealing with collection of data in a field trial of original instructional materials applies equally well to date collection procedures used in the evaluation of selected materials and instructional procedures.• When an instructor evaluates self-instructional materials,
selected materials, or an instructor’s guide, existing rapport with learners can be a great advantage.• When the instructors selects materials to implement an
instructional strategy, a number of unique concern arise.
DATA COLLECTION FOR SELECTED MATERIALS AND INSTRUCTOR-LED INSTRUCTION
• Distinguishes the instructional design process from a philosophical or theoretical approach• There are several concerns about the formative evaluation
context and the learners who participate in the evaluation that the designer should keep in mind when planning and implementing data-collection procedures • 1. Context concerns • 2. Concerns about Learners • 3. Concerns about Formative Evaluation Outcomes • 4. Concerns with Implementing Formative Evaluation
CONCERNS INFLUENCING FORMATIVE EVALUATION
• The purpose is to make a decision about what to do with a particular unit of instruction. The designer could collect enough date in the formative evaluation about the illustrations to make at least a tentative decision about their continued use in instruction.• The same methodology can be used to answer a wide array of
questions that will inevitably arise during the design process.
PROBLEM SOLVING DURING INSTRUCTIONAL DESIGN
Evaluations are important on any level, in any profession, to make sure that the overall goals of an organization are accomplished. Evaluations also help determine areas of improvement.
Life Long Learner
