Embed Size (px)
Citation preview
REVISING INSTRUCTIONAL MATERIALChristopher walker
OBJECTIVES• Describe various methods for summarizing data obtained from formative
evaluation studies• Summarize data obtained from formative evaluation studies• Given summarized formative evaluation data, identify weakness in
instructional materials and instructor-led instruction• Given formative evaluation data that indicates problems in a set if
instructional methods, identify problems in the materials and suggest revisions
BACKGROUNDdevelop instructional strategy
Develop and select instructional materials
Design and conduct formative evaluation of instruction
Design and conduct summative evaluation
TWO BASIC TYPES OF REVISIONS• Changes that are made to material and content that enables them to be
more accurate and a more effective learning tool
change that causes the previous procedures to be employed in a different manner
DATA ANALYSIS FOR ONE-TO-ONE TRIALS
• The designer has five kinds of basic information available• 1. learner characteristics and entry skills• 2. direct responses to the instruction
• 3. learning time,• 4. posttest performance
• 5. responses to an attitude questionnaire
DATA ANALYSIS FOR ONE-TO-ONE TRIALS
CONTINUED• The first step is to describe the learners who participated in the one-to-
one evaluation and to indicate their performance on any entry-skill measures
• Second step is bringing all the comments and suggestions about the instructions that resulted from going through it with each learner which can be done by integrating everything on a master copy of the instruction
• Next step is posttest data is summarized by obtaining individual item performance and then combining item scores for each objective for a total score
ANALYZING DATA FROM ONE-TO-ONE TRIALS
• Revise the instruction
Now the next task is to determine the rubric and test items. Depending on if there are errors with the performance determines which changes will have to be made in order to make sure the instruction is up to date and satisfactory
ANALYZING DATA FROM ONE-TO-ONE TRIALS CONTINUED
• There are three different sources when analyzing data from one-to-one trials:
• 1. learner suggestions
• 2. learner performance
• 3. the reactions from self based on instruction
FIVE TYPES OF INFORMATION AVAILABLE
• 1. learner characteristics and entry skills• 2. direct response to the instruction• 3. learning time• 4. posttest performance• 5. responses to an attitude questionnaire
SMALL-GROUP AND FIELD TRIALS
Individual item information required for three reasons:
1. Item information can be useful in deciding if there are particular problems with the item or if it is measuring the performance described in its corresponding objective
2. Individual item information can be used to identify the nature if the difficulties learners are having with the instruction
3. Individual item data can be combined to indicate learner performance on an objective and eventually on the entire test.
GROUP’S ITEM BY-OBJECTIVE PERFORMANCE
• Although any computer based software can help when it comes to data analysis to be used to create students performance summaries, or spreadsheets, EXCEL is great too and is widely recommended.
Item difficulty index - this is the percentage of learners who answer an item correctly1. Data can be used as posttest
GROUP’S ITEM BY-OBJECTIVE PERFORMANCE
SEQUENCE FOR EXAMINING DATA
• Instructional analysis and entry skills – after removing data for any defective items, you should examine the remaining data with regard to the entry skills of learners
• Objectives, pretest, and posttest - review the pretest and posttest data as displayed in the instructional analysis, also examine the pretest scores to determine the extent to which individual learners and the group as a whole
• Learning components of instructional strategy and materials – examine the instructional strategy associated with the various objectives with which learners had difficulty
• Media, materials and instructional procedures – data that relates to the implementation of the instructional materials must also be examined
TEMPLATE FOR SUMMARIZING INFORMATION FROM A FORMATIVE EVALUATION
Instructional StrategyComponent Problem identified Proposed changes Evidence and SourceEntry skillsMotivational materialPretestInformation presentationLearner participationPosttestAttitude questionnaireTransfer to performance context
REVISION OF SELECTED MATERIALS AND INSTRUCTOR LED INSTRUCTION
• 1. omit portions of the instruction• 2. include other available material or trials• 3. simply develop supplementary instruction
ANALYSIS OF DATA ACROSS TESTS
Category 1 Category 2 Category 3 Category 40
1
2
3
4
5
6
Chart Title
Series 1 Series 2 Series 3
ANALYSIS OF DATA ACROSS TESTS CONTINUED
REVISING INSTRUCTOR-LED INSTRUCTION
• Instructors working from an instructors guide have the same flexibility as the developer for changing instruction
• The instructor notes from the guide should reflect questions raised by learners and responses to those questions
• Unlike using written instructional materials the instructor can revise the presentation during its implementation and note the reasons for the change
LIFE LONG LEARNER• Look at revising material as a life long learner, really puts things in
importance. As a life-long learner it is of shear importance to make sure all instructional material is up to date and current with newer practices. Because these practices change and they are different with each field no matter materials should still be current and in line with practices of today.
