Summary of course curriculum evaluation

ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL!

1

MPF 1483: CURRICULUM

EVALUATION

Prepared by

DR. SHAFEEQ HUSSAIN

VAZHATHODI AL-HUDAWI (2013),

Senior Lecturer, Faculty of Education,

UTM Johor, MALAYSIA

I. DEFINITIONS

Various definitions:

1. Tyler (1950): the process of

determining to what extend

educational objectives are being

attained

2. Borg & Gall (1983): the process of

making judgment about the merit,

value or worth of educational

programmes, projects, materials

and techniques

3. Smith & Glass (1987): the process

of establishing value judgments

based on evidence abt a program /

product

4. Stufflebeam et al. (1971): the

process of delineating, obtaining &

providing useful info for judging

decision alternatives 5. Provus (1971): the comparison of

performance to some standards to

determine whether discrepancies

existed

6. The process of collecting data on a

programme to determine its value

or worth with the aim of deciding whether to adopt, reject, or revise

the programme.

7. Joint Committee on Standards

for Educational Evaluation

(2003) process of systematic and

focussed determination of a

curriculum’s value. “Evaluation is

the systematic assessment of the

worth or merit of an object”

(1994, p. 3)

II. A BRIEF SKETCH ON

EVALUATION

-Evaluation is closely associated with

assessing achievement of students

against behavioural objectives or

conducting norm-reference testing.

-From 1970, emphasis was given to

professional judgment, i.e., collection

and analysis of quality of information

for judging and deciding on a

curriculum.

In short, Evaluation is an integral part

of the curriculum development

process

III. TYPES OF EVALUATION

Acccording to Scriven (1967), there are

two types of evaluation in curriculum:

1) Formative evaluation

– an on-going program

- it is a program improvement

– provide data about educational

program

to assist developer in improving the

program

For example, results of formative

evaluation may help in 1. Selection of programme

components

2. Modification of programme

elements

2) Summative evaluation – it is done at the completion of a

program

- it concerned with overall

effectiveness of the program

- it provides data to determine the

merits and demerits of the programme

For example, results of formative

evaluation may help in 1. Determining what were the

effects of the curriculum, and

evaluate them whether or not,

they were intended

2. Noticing whether something

was overlooked by that

everyone else


2

IV. GOALS OF EVALUATION

IN CURRICULUM

DEVELOPMENT

• Enables curriculum makers

reviewing and modifying – to

cater the current and future needs

• Curriculum should be continuously

reviewed & reviewed

WHY??

• To MAINTAIN, MODIFY

- Quality

- Relevancy

- Adequacy

• OR ELIMINATE

QUESTIONS SHOULD BE ASKED:

1) Does the program (curriculum)

meet existing or expected

needs?

2) Does the program contain

extraneous and outdated

materials?

3) Are the students able to perform

adequately once they finish

their study?

Wentling (1980): EVALUATION

MUST DO MORE THAN :

• just analyzing the extent to

which a program had adhered to

an original plan

• OR attained its primary goals

and objectives

Therefore, curriculum evaluation needs

to go beyond the assessment of student

behavior (thus, Tyler’s definition of CE

is inadequate)

• It should include the overall

effect on students, teachers and

society.

BECAUSE

Curriculum is inclusive of

• INTENTS (Aims, Goals and

Objectives)

• CONTENTS (Various

subject-matter contents

included, their Scope and

sequence, balance;

Teaching-learning materials,

resources and experiences,

i.e, syllabus, content outline,

textbooks)

• PROCESS(Pedagogy,

Learning strategies)

• PRODUCT or OUTCOME

• Curriculum is the link between

society and school

• Through Curriculum we meet

Educational goal of the nation

• The task of evaluating the

curriculum involves a

COMPLEX PROCESS

• Hence, CE is about evaluating

EVERYTHING about

curriculum.

CE should have strategies to

determine whether decision about

all the elements of curriculum

mentioned above is optimal

V. THE WORKING

DEFINITION OF

‘CURRICULUM

EVALUATION’

Evaluation is the provision of

information at…

• A: STAGES of determining of

aims, planning, tryout, field

trail, implementation and

quality control of program

development, concerning…

• B: ENTITY of Teachers’

guide, -study material, -

instructional tools, -the whole

package, from the point of view

of…

• C: CRITERIA of -Fit to

standards, -eliciting processes,

-yielding outcomes

on the basis of

• D: DATA such as -judgment, -

observation, -examination of

product

summarized in

• E: MODE OF SUMMARY of:

Quantitative, Qualitative,

Mixed Methods


3

for sake of making decision

about

• F: ROLE: -modifying, -

revising the use of the

program.

VI. MODELS OF

‘CURRICULUM

EVALUATION’

CIPP model aims to evaluate how

CORE VALUES are put in

1. GOALS (Context Evaluation)

2. PLANS (Input Evaluation)

3. ACTIONS (Process Evaluation)

4. OUTCOME (Product

Evaluation)

See the FIGURE in the appendix

DETAILS ON

CIPP MODEL (Finch & Bjorquist,

1977)

1) Context evaluation

Whether or not to offer a

curriculum

If so, what its parameters will be –

including goals and objectives

It includes- environment, students’

b/ground, school climate, goals and

objective of the curriculum

It involves analysis of goals and

needs in a specific education setting

Goals should be reviewed and

revised periodically in relation to:

1) the reasons for offering the

program

2) the intended target group

3) the content

As curriculum developers:

Establish conducive learning

environment to promote a positive

learning climate

Monitor current trends and issues in

education

Responsive to latest innovation and

development in education

EXAMPLE:

Context: Examine the environment

in which technology is used in

teaching and learning

2) Input evaluation

• Deciding what resources and

strategies that will be used to

achieve curriculum goals and

objectives

• This includes – the quality of the

curriculum and syllabus, students,

teachers, staff, facilities and

infrastructure

EXAMPLE:

Input: Examine what resources are

put into technology integration

3) Process evaluation

• Focuses on decisions associated

with curriculum effects on students

• Whether the content is learned by

students

• Involves the data collection

• Data is collected over a period of

time to detect strengths and

weaknesses of the program

• Process evaluation includes:

teachers’ methods of teaching,

students’ performance /

achievement, the facilities used

• Conduct professional development

courses to upgrade (teaching) skills

of teachers

• Review students assessment

EXAMPLE:

Process: Did the technology

integration work smoothly?

4) Product evaluation

• To determine the extent to which

the goals of the programs have been

achieved

• It is also involves data collection

through feedback from former


4

students or graduates, employer and

society

• Data collected - to make decisions

about the program

• Whether to retain or modify the

existence program

• It concerns on the number and

quality of students produced

• Can get from teachers and students

• Info – valuable in determining

adequacy and appropriateness of

the curriculum

EXAMPLE

Product: Did the learners learn using

technology? How do you know?

MODEL TWO: CONGRUENCE-

CONTINGENCY MODEL

by Robert Stakes

Here evaluation is done through

Formal means of collecting for

extensive data by

• Observing dynamics among

people involved in curriculum

process

• Allowing them greater

participation in judging

programs

• Taking position regarding a

program’s worth

By considering

1. Antecedents: Conditions prior to

teaching and learning that may

influence outcomes

2. Transactions: The Process of

interactions among

Students and teachers

Students and Students

Students and curriculum

materials

which are affected by time factors,

space arrangements,

communication flow, and teaching

process

3. Outcomes: the Immediate and

long-range program results of

Student achievement,

teachers’ perception of their

competence and their

influence on administrative

policies

Here we look at the CONSIDERED

and ACTUAL Antecedents,

Transactions and Outcomes.

SEE the diagram in the Appendix

VII. APPROACHES TO

CURRICULUM

EVALUATION

• Approaches to curriculum

evaluation mean the beliefs and

assumptions (often termed

'philosophies' or 'perspectives') of

designers of the curriculum on

‘what should be evaluated’?

• The important approaches are:

• Traditional: What are the

important aspects of our cultural

heritage that should be

preserved,

• What are the important aspects

of our cultural heritage that

should be preserved i.e.,

the students can be manipulated

in order to bring about desired

results’. THEREFORE, ‘the

need for efficiency as seen in

the manufacturing sector’

(Hopkins, 1994)

• Experiential: What experience

will lead to the healthy growth

of the individual?,

• What experience will lead to the

healthy growth of the

individual? i.e.,

learning occurred in a cycle of

a) experience

b) reflection

c) action

• Structure of Discipline: What is

the structure of the disciplines of

knowledge?

• What is the structure of the

disciplines of knowledge?, i.e.,

in order for members of society

to function well in the wake of


5

the global economy, there is a

need to acquire high levels of

literacy and numeric in subject

areas

• Behavioural: At the completion of

the curriculum, what should the

learner be able to do?

• What is the structure of the

disciplines of knowledge?, i.e.,

learning occurs when behavior

indicates that a given stimulus

was effective in producing the

desired outcome

• Constructivist: How can people

learn to make sense of world and to

think more productively and

creatively?

• How can people learn to make

sense of world and to think

more productively and

creatively?, i.e.,

learning takes place in such a

manner that the recipients of

information build up on those

skills and knowledge that they

receive from the environment’.

THEREFORE, ‘most of the

time, there is a need to teach

ideas in holistic manner’

VIII. ECLECTIC APPROACH TO

CE

Utilizing various scientific, humanistic

or scientific-humanistic approaches

simultaneously in order to overcome

the focus of CE model on

Some particular feature of

evaluation

Some unique functions / roles

For they prescribe

Specific patterns for evaluation

activities

Hence, they are not ALTERNATIVES,

but COMPLEMENTARY to each

other, and therefore ECLECTIC

approach, so as to

Analyze various political, social

and moral reals as to their

significance in Curriculum

IX. IMPORTANT CONCEPTS,

TO DERIVING GENERAL

AIMS AND SPECIFIC

MAJOR OBJECTIVES

a. Critical changes in Society

a. Most Important changes,

their implications on

objectives

b. Nature of changes:

economic, social, health,

development

c. Employment Patterns of

the time

d. Requirement in Health,

Welfare, Political and

Social Affairs

e. Students’ concerns of

learning needs

f. Development in the

content of the Subject-

matter

b. Relevance and Balance

c. The Language of Instruction

d. New Forms of Education

e. Minimum Learning

Requirements

• A comprehensive evaluation

framework or model is

necessary to achieve a

systematic, effective and

efficient evaluation

X. SUMMARY ON SIX

STAGES OF CURRICULUM

EVALATION

See the APPENDIX

XI. STAGE ONE: Determination

of General Educational Aims

Curriculum evaluation is NOT’ just

determining whether educational

objectives are achieved, but rather it

is about determining the worth and

merits of objectives themselves


6

XII. EVALUTION CRITERIA &

PROCEDURE at the

PLANNING STAGE

At the PLANNING STAGE, we look

how well we have selected,

conceptualized and formulated

1A. Instructional Objectives

1B. Scope and Sequence of the content

1C. Teaching Learning Strategies

1D.Instructional Materials

1Aa. Instructional Objectives:

CRITERIA: Are they

• RELATED to the OBJECTIVE OF

THE PROGRAM

• CLEARLY STATED

• APPROPRIATE FOR, AND

ATTAINABLE by TARGET

LEARNERS (based on level and

mental development).

• IMPORTANT ENOUGH to

encourage further learning

1Ab. Instructional Objectives:

PROCEDURE (How):

• Seeking and Analyzing relevant

opinions and judgments by

panel of experts (Curriculum

specialists, Psychologists, etc).

1Ba: Evaluation of the Scope &

Sequence of the Content of

Instruction: CRITERIA

• Relevance to Instructional

Objectives

• Up-to-datedness of Content

• Relevance to the child and his

environment

• Content Balance

• Balance in the content in

terms of students’ activities

envisaged.

• Balance between various

subdivision of the subject

matter

• Organizational Structure of the

content

1Bb: Evaluation of the Scope &

Sequence of The Content of

Instruction: PROCEDURE (How)

• Seeking and Analyzing relevant


panel of

• Subject Specialist

• Educational

Psychologists

• Teacher-Educators

• Experienced Teachers

1Ca. Teaching-Learning Strategies:

CRITERIA:

• Effectiveness & Economy

• Feasibility in Classroom

• Teacher Competence

1Ca. Teaching-Learning Strategies:

PROCEDURE (How):

In-House Analysis

Experts’ Panel

Simulation at Center

Limited Tryout in Schools

1Da. Instructional Materials:

CRITERIA:

• INSTRUCTIONAL

CHARACTERISTICS: Contents

are

academically • related to instructional

objectives

• valid and reliable

• logically Organized

psychologically

• relevant

• appropriately Sequenced

• Hierarchically structured

and sequence

• Reinforcing

• Promoting Understanding

and Critical Thinking

Linguistically • Accuracy and suitability

• PRACTICAL

CHARACTERISTICS


7

• Cost Range

• Flexibility to adapt the

programs to TL condition

• Easiness to use

1Da. Instructional Materials

Strategies: PROCEDURE (How): • Seeking and Analyzing relevant


panel of

• Subject Specialist

• Educational

Psychologists

• Teacher-Educators

• Experienced Teachers

• Production Experts

These FOUR elements of curriculum,

i.e., 1A, 1B, 1C, 1D should be analyzed

based on the CRITERIA (1Aa,1Ba…),

following the PROCEDURE (1Ab,

1Bb,….).

XIII. 5 QESTIONS OF VALUES

IN CURRICULUM

EVALUATION

These ‘5 Q’s’ address

‘What ‘evaluation’ can do?’. The

5Q’s are The Questions of:

a. INTRINSIC VALUE

This related to:

i. Goodness, up-to-datedness &

Appropriateness of ‘planned’ and

‘enacted curriculum’ at philosophical

and psychological, sociological and

subject-areas levels, and

ii. Their Congruence with

NATIONAL EDUCATIONAL

PHILOSOPHY & NATIONAL

EDUCATIONAL CONCERNS

How to do this (Procedure):

i. Getting ‘specialists’ and

‘experts’ to analyze the

curriculum document

b. INSTRUMENTAL VALUE

decides the ROLE of Evaluation at the

sociological or pedagogical context, i.e.

What is the CURRICULUM Good

for?, i.e.,

Will what is planned in the curriculum

be attained?

• Objective of the program:

‘planned curriculum’ versus

‘enacted curriculum’

To what extent?

• With the suggested ‘contents,

materials, methods’ can the

‘philosophical and

psychological’ orientations be

achieved

By which students?

c. COMPARATIVE VALUE

Is the NEW program Better than the

OLD one in terms of

• Skill training

• Students advancement

• Delivery and other costs

d. IDEALIZATION VALUE

How to have the BEST possible

CURRICULUM? in order to

HIGHTEN Students achievement, for

optimal benefits.

This is a continual concern to fine-tune

curriculum entities: Alternatives: to

involve students fully into learning?

f. DECISION VALUE

Making QUALITY decision on

whether to RETAIN, MODIFY OR

DISCARD the program based on the

above FOUR considerations AND

evidences and

Here it is important to consider

‘WHAT IS THE VALUE OF THESE

DECISIONS?

XIV. NORM & CRITERION-

BASED EVALUATIONS

Norm-Referenced Evaluation

Evaluating students’

performance relative to the

performance of other students

• The performance of current

students or of previous students


8

can be compared, according to

standards of group, where

students are grouped for

particular courses

Criterion-Referenced Evaluation

Measuring students’ actual

performance and comparing it

with the objectives of

instruction identified in the

syllabus, or

on a specific criterion (e.g.,

Evaluation of mastery learning

schemes)

XV. EVLAUTION OF THE

EFFECTIVENESS OF

TEACHER TRAINING

At the LARGE SCALE

IMPLEMENTATION, it is necessary

to provide INTRODUCTORY AND

SUPPORTING COURSES

(TEACHER TRAINING)

a. Training related to activities

and background information

b. Training related to program

objectives

c. Training related to teachers’

role and classroom

management

d. Training related to criteria

for monitoring students’

achievement.

Furthermore, the EVALUATION OF

THE EFFECTIVENESS OF SUCH

TEACHER TRAINING COURSES are

very significant.

CRUCIAL influence of teacher in

effectiveness of implementation is

acknowledged, but NOT REFECTED

in evaluating the effectiveness of such

courses. Such course can be of various

types, namely:

Concentrated workshop type

Preparatory course spread over

weeks

Extended courses with

specialists

Action-research

Development-Cluster groups

All these types of course need to be

evaluated for their effectiveness in

terms of

Teachers’ knowledge of

o the subject matter

o activities

o aims and objectives

o acceptance of classroom

activities with are

congruent to aims

changes in Teacher’s attitude to

the new program course and

students behavior

HOW that can be done:

Interviews

Checklist

Questionnaires

Follow-up observations

Informal discussion

Involving Teachers in devising

new methods, than they receive

information

XVI. THE NEED FOR QUALITY

CONTROL OF THE

IMPLEMENTED

PROGRAM

At the large-scale implementation

stage, curriculum tends to

DETERIORATE, by being less

effective.

Loss of effectiveness of particular

curricula,

methods or procedures

to

entire student population,

some schools,

subgroup of students.

By decrease in

Achievement Level

student interest

in higher mental process

(application, analysis, synthesis)

Therefore, the EVALUATOR should

CONTROL the QUALITY in terms of

1. Need

2. Cause

3. Corrective measures

By seeing


9

1. Problem

2. Process

3. Data

STUDY the Appendix on Schemata of

Quality Control

XVII. CONTINUITY IN THE

PROCESS OF QUALITY

CONTROL OF THE

IMPLEMENTED

CURRICULUM

Evaluation itself is a ‘quality control’

of

Curriculum

Instructional methods

Procedures

That is, it is a continuous process of

Collecting information on

problem, process,

Taking CORRECTIVE

measures.

It should be CONTINUED, because

EDUCATION is SIGNIFICANT

public enterprise, involving

MILLIONS of children.

Quality control require effort,

time & money but vital to detect

problem, Hence we need a

curriculum center to undertake

quality control,

CONTINUALLY and

SYSTEMATICALY to make

curriculum

RELEVANT

RESPOSIVE to the needs of

community and nation


10

APPENDIX: Different stages of program development and utilisation: different methods, strategies, personS

Determination of

General Aims

Decision about: general aims, school

structure

Studies on:

Expected changes, Cultural Values, Social forces, Present

level of achievement, feasibility of programs

Planning

Writing Outline: Preparing

instructional material

Examining adequacy of objective, contents, strategies

Judgement of material

Tryout Monitoring teaching in tryout

classes, Modifying material

Collect evidence through observation, judgement,

discussion with teachers, students

Student products

Field-Trail

Slightly modify program ,

Determine optimal conditions of

program use

Collect evidence about the efficiency of program under

various conditions

Implementation

Links with supervisors, examination

system, teacher training

Examining final form

Evidence on efficiency of teacher training

Quality Control

Implement recommendation, plan

‘second generation’ programs

Examining quality of implementation

Studying reason for changes in efficiency

Suggesting remedies if needed


11

APPENDIX: Schemata on Quality Control

Need for Quality Control Cause of deterioration Corrective measures and

effectiveness

Problem Does implemented curriculum remain

effective

Why and how deterioration originated -Appropriate quality control measures

-Is particular measure effective

Process Compare student achievement data in

the current term with those in the

previous terms or years

Survey how curriculum is

implemented, under what conditions,

to what students groups

-Analyze and compare a series of

formative tests

-derive hypothesis which explain why

the effectiveness of the curriculum

decreasing

-Appropriate quality control measures

-Is particular measure effective under

small-scale experimental situations

-apply the verified quality control

measures to the target population

Data -Summative achievement programs or

summative achievement tests

administered every year

-end of course examinations

-standardized achievement tests

-expert and teacher judgment

-attitude and interest survey

-Questionnaires

-Other unobtrusive measures

-formative tests

-school survey data

- Interview and questionnaires

-expert and teacher judgment

-classroom observation

-formative and summative tests

-attitude and interest survey

-Other unobtrusive measures


12


13

In CE, the worth could be gauged only after assessment of the graduates needs of a program. And needs of the following types

Concept Definition Example

Need Something that is

necessary or useful for

fulfilling a defensible

purpose

Competent, effect

instruction in the basic

skill areas

Defensible purpose A desired end that has

been legitimated

consistent with a guiding

philosophy, set of

professional standards,

institutional mission,

mandated curriculum,

national constitution or

public policies, etc

Students’ development of

basic academic skills

Treatment need A certain service,

competent service

provider, or other helping

agent

Competent instructors in

twelfth-grade courses in

math, science and

language arts

Outcome need Achievement or outcome

required to meet a

defensible purpose

Students’ demonstration of

proficiency in specified

areas, such as twelfth-

grade math, science, and

language arts

Assessment A systematic assessment

of the extent to which

treatment or outcome

needs are being met

Examination of students’

scores on national tests

and evaluation of the

involved teachers


14

APPENDIX: Robert Stakes’s Congruence and contegency model

Curriculum Development Curriculum Evaluation

Considered antecedents Actual antecedents involved

Considered Transactions Evaluation of Transactions

Identified Outcomes Attained Outcomes

Empirical contingency

Empirical contingency

Logical contingency

Logical contingency

Congruence

Congruence

Congruence