Conducting Material Evaluation

Puja Shrivastav

Purposes of Learning Material Motivation: materials to catch the learners’

attention Participation: materials to familiarize learners

with content through hands-on activity Instructional: materials to teach necessary

information Follow-up: materials to reinforce knowledge

and provide additional information; also to maintain literacy skills.

Objectives for Material Evaluation

Are the objectives set in the curricular unit achieved after using the material?

Are the contents set in the curricular unit relevant to the learners’ context and needs?

Are the words used in the material appropriate to the learners’ literacy level?

Should also focus on its Visuals, Formats and Teaching – learning process set as per curriculum.

What should be evaluated?

1. To evaluate quality of the material some indicators are

Achievement of objectives Relevance of contents Attractiveness of visuals Appropriateness of language level Appropriateness of teaching-learning process Others (format, durability, portability, size,

etc.)

Requirements for Material Evaluation

Agenda- Designing of pre-test to get maximum feedback.

Draft material- Planning a course and syllabus development

Curricular units- Questionnaire- Conducting discussion Exams and assignments Field experiences

What is missing?

Material Evaluation

Pre testingPost testing

Material Evaluation

Material Evaluation

Pre testing of material/ content

Post testing of material/ content

Peer pre testing

Learner pre testing

Peer post testing

Learner post

testing

Pre-Test Learners pre- test Invite learners or visit another learning centre

to have clientele Provide a copy of the curricular unit of the

material to the facilitator, and ask her/him to use the draft material according to the teaching learning process set in the curricular unit

Observe the performance of the facilitator and reactions of the learners.

Continue…. Distribute the questionnaire and ask the

learners to fill them out. After conducting pre-test, analyze the result of

the pre-test aggregating the remarks in the questionnaire.

Find out and list up the points to be improved based on the analysis and observation.

Pre-Test Peer pre- test Invite material developers/ facilitators and

school teachers as clientele. Ask one facilitator to demonstrate the draft

according to the teaching learning process set in the curricular unit

Observe the performance of the demonstrator and reactions of the clientele.

Continue… Distribute the questionnaire to all the clientele

and ask them to fill out. Discuss general impression of the draft among

clientele. Take note all the points raised. After conducing pre-test, analyze the result of

the pre-test aggregating the remarks in the questionnaire.

Find out and list up the points to be improved based the analysis and

Purpose of the Evaluation To produce the highest possible quality of

student learning To help students learn more effectively and

efficiently than they otherwise would on their own

To develop a good quality material for curriculum.

Revision and Finalization After the pre-test, we should revise and finalize the

material in the following way: Read through the questionnaires from learners and/or

peers one by one. Analyze both the summary data of the questionnaires

and comments. Interpret the data and list up the points for

improvement of the materials.

Continue.. Transfer all the improvement points onto a

master copy of the draft, along with comments and suggestions made by the learners and peers, if any.

If the material is developed in a group, discuss with group members (illustrators, writers, etc.) points to revise and how to revise.

Revise the curricular unit, especially the teaching-learning process following the changes to be made with the material

Post testing Evaluation is a measurement of achievements

in relation to objectives set in the curricular unit.

The major part of material evaluation is included in post testing of the finished product.

Getting feedback from the users, including facilitators and learners, through evaluation is very helpful for us to develop better material next time and to improve our material development skill.

Formative vs. summative evaluation Formative evaluation is an evaluation conducted

during the development stage of the material.

The result of the evaluation tells the developer where and how to make necessary changes.

In contrast, a summative evaluation is one that provides information necessary to make a decision at the end, whether to disseminate, re-develop, modify, adopt, or adapt the material.

Formative Evaluation It help individual teachers to obtain useful feedback

on what, how much, and how well their students are learning

Teachers use this information to refocus their teaching to help students learn more effectively

Formative evaluation is Learner-centered, teacher-directed, mutually beneficial,context-specific, ongoing, and firmly rooted in good practice.

Formative Improve quality of learning, not provide

evidence for evaluation or grading Rarely graded, almost always anonymous Aim is to provide faculty with information

what, how much, and how well students are learning

Learner-centered Focuses attention of students and teachers on

observing and improving learning, not teaching

In the end, if students become more independent, lifelong learners, students must take full responsibility of their learning

Teacher Directed It respects the autonomy, academic freedom,

and professional judgment of teacher Individual teacher decides when, what and

how to assess and how to respond to material gathered.

Mutually beneficial Since focused on learning, requires active

participation of the students By cooperating in assessment, students reinforce their

understanding of course content and strengthen their own skills at self-assessment

Students can participate more actively in their learning (taking ownership in process)

Increased motivation when realize faculty are invested in their success

Context-specific You need the right tool to do the right job Different content areas, Types of activities, and Even class dynamics warrant different

assessment/evaluation techniques

Ongoing Consider it like a classroom feedback loop Teachers get feedback from students on their

learning Faculty then complete the loop by providing

students with feedback on the results of assessment and suggestions

Loop continues

How do students directly benefit from Evaluation?

Meta-cognitive skills – describe students’ awareness and understanding of their own learning skills, performance and habits

Understand what they are learning and why they are learning it

Good learners engage in more metacognitive activities than poor learners do “The more time and effort students invest in the learning

process, the more intensely they engage in their own education, the greater their educational experiences, persistence in college, and continuation of learning”

Process and content based feedback

How do teachers directly benefit from classroom assessment?

Benefits to students = benefit to teachers Develop a culture of openness and consistent feedback Constantly monitor student views on process and

understanding of content Answer difficult questions such as:

“How can I find out whether my students are learning the essential skills and knowledge I am trying to teach”

“How can I help students learn better?” Reduce uncertainty of student learning between exams Improve teaching skills and gain new insights Increase overall learning