8/3/2019 Formative Evaluation of the ID Project

1/2

Formative Assessment of the ID Project

Because the instructional design process is iterative in nature - there is a constant quality analysis

going on at all times and the designed material is constantly being revised - formative assessment of the

instructional material takes place throughout the design process. That being said, most of the data as a

result of the formative assessment takes place toward the end of the design process. It is customary tohave four different sets of feedback during this part of the design process: feedback from a Subject Matter

Expert (SME), feedback from a select few members of the target audience, feedback from small groups of

members of the target audience, and then finally feedback from trial runs of the material with the target

audience.

Expert Review

Ideally a Subject Matter Expert (SME) is selected early on in the design process and is consulted

and solicited for specific feedback throughout the process. In a school setting, an SME could be someone

like a department chair or divisional administrator, someone well versed in learning theory and the

expectations and goals of the school. Outside of a school setting, an SME can be any professional with a

deep working knowledge of the topic of instruction.

In the case of the lesson on mapping East Asia, Mr. William Corey, Upper School Principal at

Qingdao #1 International School of Shandong (QISS) has been selected to serve at the SME. Mr. Corey

has agreed to examine the instructional material within the next seven days and complete a survey of

questions related to the material.

Although the detailed survey given to the SME is located in the following section (see

Formative Evaluation Report {not yet completed}), the SME will be responding to questions

concerning: (1) the accuracy of the lessons content, (2) the degree to which the lesson fits in with the

overall goals of the school, (3) the degree to which the activities planned during the instruction are

realistic for the learners in the school, (4) the degree to which the pedagogical approach of the lesson is

consistent with contemporary instructional theory, and (5) the degree to which the planned feedbackdelivered to the students is ample and realistic.

One-to-One Evaluation

The one-to-one phase of the formative evaluation process is really the first trial run of the

material that has been created. It involves presenting the material to two or three 11-year-old or 12-year-

old students of average ability. The main purpose of presenting the material to these students in a one-to-

one setting is to expose any major errors or shortcomings that might occur with the materials that have

been created for student use. The two or three students sampled are given the various handouts created for

the learners in the lesson and then also shown the instructional video created for the learners to use as a

guide during the lesson. The students sampling the material are asked first about the level of vocabulary

used in the material; its appropriateness and level of difficulty. Students are also asked to comment on the

clarity of the written instructions delivered to students throughout the lesson. Because assessment rubrics

are used for evaluating student work but are likewise handed out to the learners during the lesson, the

students completing the one-to-one evaluation are asked to comment on the language and clarity of the

assessment rubrics, too. After the evaluation here is completed, revision of the instructional material

occurs as need dictates after the feedback from students is analyzed. Then, the parts of the material that

8/3/2019 Formative Evaluation of the ID Project

2/2

were revised and the feedback from the students that led to specific revisions being made are documented

and included in a section of the projects documentation.

Small Group Evaluation

Small group evaluation occurs after the one-to-one evaluations have been conducted and the

revisions based on those evaluations have been completed. Given that the class of students who are to begiven the actual instruction here is only composed of thirteen students, a small group of eight students

should be selected to conduct this portion of the formative evaluation. Students selected for the small

group evaluation examine the course material, either meeting as a small group or examining the material

as individuals. The point is to have a sufficient number of students go through all off the instructional

material. It is common at this point to have students participating in the small group analysis to respond to

a questionnaire examining the students feelings and attitudes toward the instruction to record the time it

took each student to finish the instructional activities. Because the actual target group is so small, the

small group analysis and the field trial analysis (see next sub-section) would be almost identical were it

not for the questionnaire on attitudes and data collected with respect to students finishing the instructional

activities. So, it is quite important here that students completing this part of the formative evaluation

complete the questionnaire and have their completion times recorded. As with one-to-one evaluations, the

data collected during this small group analysis is likewise collected, analyzed, and published as a part of

the documentation of the instructional material. Revisions are likewise made as a result of the student

feedback. Unlike the one-to-one evaluation, it is common to really focus on the results that students

achieve on their formative and summative evaluations given during the instruction. Their scores are

usually added to the data collected here.

Field Trials

In this case, field trials would occur when the revised instructional materials, in their final form,

are used with 6th grade social studies classes in the school. Of course, further revision of the material is

possible as a result of the feedback obtained during these field trials, however the materials presented tothe students here are considered final for the time being. Ideally multiple teachers and classes are

involved in this stage of the evaluation, but given the context of the school in this case a small

international school the field trials would include only the one group of students in the 6th grade social

studies class. Realistically speaking, the field trial portion of the evaluation then, would need to extend

over a period of at least two years, long enough to have at least two different classes of 6th graders

experience the instructional materials. The critical aspect of the field trials to provide the opportunity for

analysis of the summative assessments that the students complete; to see the extent to which the original

goals and objectives of the lesson are actually being met. As with the other forms of the course material

evaluation, data is collected during field trials and is analyzed. Further revisions to the material are made,

and the data, the analysis and the revisions are documented as a part of the documentation of the

instructional design project.