ADVANCED METHODSin

EDUCATIONAL ASSESSMENTand

EVALUATION ASSESSMENT OF LEARNING

2

EDDIE T. ABUGBSE-TLE 4A

DR. REBECCA AMAGSILA Ph. D.

CHAPTER 1

REVIEW OF PRINCIPLES OF

HIGH QUALITY

ASSESSMENT

CHAPTER 1

REVIEW OF PRINCIPLES OF

HIGH QUALITY

ASSESSMENT

CHAPTER 1REVIEW OF PRINCIPLES

OF HIGH QUALITY ASSESSMENT

CLARITYOF

LEARNING TARGETS

APPROPRIATENESSOF

ASSESSMENT METHODS

PROPERTIESOF

ASSESSMENT

METHODS

COGNITIVE

TARGETSSKILL

S,

COMPE-

TENCIES

AND

ABILITIES

TARGETS

COGNITIVE TARGETS

PRODUCTS,

OUTPUTS

AND

PROJECTS

TARGETS

WRITTEN -RESPONSE

INSTRUMENT

PERFOR-MANCE TEST

PRODUCT RATING SCALES

ORALQUESTIO

-NING

OBSER-VATION

AND SELF

REPORTS

VALIDITY

RELIABILITY

FAIRNESS

PRACTICA-LITY AND

EFFICIENCY

ETHICSIN

ASSESSMENT

A. Clarity of Learning TargetsAssessment can be made precise, accurate and dependable only if what are to be achieved are clearly stated and feasible .

We consider learning targets involving knowledge, reasoning skills, products and

effects.

Learning targets need to be stated in behavioral terms

orTerms that denote something which

can be observed through

the behavior of the student.

1. Cognitive Targets2. Skills, Competencies and Abilities Targets3. Products, Outputs and Project Targets

As early as the 1950’s, Bloom (1954), proposed a hierarchy of educational objectives as the cognitive level. These are:

1. COGNITIVE TARGETS

Knowledge Refers to the acquisition of

Facts, Concepts and

Theories.

Knowledge of Historical Facts like the DATE of EDSA revolution

Knowledge about the Discovery “Philippines”

MagellanMarch 15

1521

Knowledge

Forms the foundation of all other cognitive objectives for w/o knowledge, it is not possible to move up to the next higher level of thinking skills in the hierarchy of educational objectives.

Comprehension Refers to the same

concept as“understanding”.

It is a step higher than mere acquisition of facts and involves a cognition of awareness of the interrelationships of facts and concepts

Ex: (knowledge of facts).The Spaniards ceded the Philippinesto the Americans in 1898.

In effect, the Philippines declared independence from the Spanish rule only to be ruled by yet another foreign power, the Americans (comprehension)

APPLICATIONRefers to the transfer of knowledge from one fieldof study to another or from one concept in the same discipline.

Ex: The classic experiment Pavlov on dogs showed that animals can be conditioned to respond in a certain way to certain stimuli.

The same principle can be applied in the context of teaching and learning on behavior modification for school children.

ANALYSISRefers to the breaking down of a concept or idea into its components and explaining the concept as a composition of these concepts.

Ex: Poverty in the Philippines, particularly at the barangay level, can be traced back to the low income levels of families in such barangays and the propensity for large households w/ an average of about 5 children per family. (Note:Poverty is analyzed in the context of income and number of children.

SYNTHESISRefers to the opposite of analysis and entails putting together the components in order to summarize the concept.

Ex: The field of geometry

Replete w/ examples of synthetic lessons. from the relationship of the parts of a triangle for instance, one can deduce that the sum of the angles of a triangle is 180˚.

Evaluate the actors professionals, amateurs, or students?

Criticize the actors capable of dealing with the script's requirements?

(Be fair to the actors in your assessment of their talents and the level of their "craftsmanship.")

EVALUATION AND REASONING

Refers to valuing and judgment or putting the “worth” of a concept or principle.

Students make judgments about the value of ideas, items, materials, and more.Students are expected bring in all they have learned to make informed and sound evaluations of material.

Key Words for the Evaluation Category:evaluate, appraise, conclude, criticize, critique

Ex:Watch an stage play and write a critique of the actor’s performance.

2. SKILLS, COMPETENCIES AND ABILITIES TARGETS

Skills refer to specific activities or tasks that a student can proficiently do e.g. skills in coloring, language skillsSkills can be clustered together to form specific competencies e.g. Birthday card making.Related competencies characterize student’s ability. (DACUM, 2000)

Abilities can be roughly categorized into: cognitive, psychomotor and affective abilities

Ability to work well w/ others & to be trusted by every classmate (affective ability)

is an indication that the

student can most likely succeed in work that requires

leadership abilities.

Other students are better at doing things

alone like programming & web designing

(cognitive ability) and, therefore, they would

be good at highly technical individualized

work.

3. PRODUCTS, OUTPUTS AND PROJECTS TARGETS

Tangible and concrete evidence of student’s ability

A clear target for products and projects need to clearly specify the level of worksmanship of such projects e.g. expert level, skilled level or novice level.

Once the learning targets

are clearly set, it is now

necessary to determine an appropriate assessment

procedure or method.

B. APPROPRIATENESS

OFASSESSMENT

METHODS 1. Written-Response Instruments 2. Product Rating Scales 3. Performance Test 4. Oral Questioning 5. Observation and Self Reports

1. WRITTEN-RESPONSE INSTRUMENTS

OBJECTIVE TESTSa.Multiple Choiceb.True-Falsec.Matching or Short Answer

TESTS, ESSAYS, EXAMINATIONS AND CHECKLIST

OBJECTIVE TESTSAppropriate for assessing the various levels

of hierarchy of educational objectives.Require a user to choose or provide a response to a question whose correct answer is predetermined. Such a question might require a student to :a. select a solution from a set of choices (multiple choice, true-false, matching)b. identify an object or position (graphical )c. supply brief numeric or text responses

What is higher-level thinking?What do we mean by higher-level thinking? Benjamin Bloom described six levels of cognitive behavior, listed here from the most basic – Knowledge – at the bottom to the most complex – Evaluation – at the top:

EvaluationSynthesisAnalysisApplicationComprehensionKnowledge

1. MULTIPLE CHOICE TESTIn particular can be constructed in such a way as to

test higher order thinking skills.

MULTIPLE CHOICE TEST

a. Tim needs extra practice reading and writing problematic letters and words at home at least 30 minutes per day.

b. Please discuss the importance of schoolwork to Tim so that he will increase his efforts in classwork.

c. These are possible symptoms of dyslexia so I would like to refer him to a specialist for diagnosis.

d. Please adjust Tim’s diet because he is most likely showing symptoms of ADHD due to food allergies.

Explanation: C is the best answer because the behaviors could be symptoms of dyslexia.

Students must evaluate multiple pieces of evidence, then apply that evidence to solve a problem, student must select

the best action to take with the evidence.

Tim’s second grade teacher is concerned because of the following observations about Tim’s behavior in class:

Withdraws from peers on the playground and during  groupworkOften confuses syllables in words (ex: says mazagine instead of magazine)Often confuses b and d, p and q, etc. when writing or recognizing letters

The teacher has arranged a meeting with Tim’s mother to discuss these concerns. Which of the following statements is

best for the teacher to say to Tim’s mother?

When properly planned, can test the student’s grasp of the higher level cognitive skills

particularly in the areas of application analysis, synthesis, and judgment.

Questions - “precise”, PARAMETERS - “properly defined”

Write an essay about the first EDSA revolution.

(give add’l. requirements to give focus)

Focus on the main characters and their respective roles in the revolution

2. ESSAYS

2. PRODUCT RATING SCALESA Teacher is often tasked to rate products.1. Book reports2. Maps3. Charts4. Diagrams 5. Notebooks6. Essays7. Creative endeavors

Example of a Product Rating ScaleClassic “Handwriting” Scale

Used in California Achievement Test Form W (1957)

PurposeThe CAT is often administered to determine a child's readiness for promotion to a more advanced grade level and may also be used by schools to satisfy state or local testing requirements.

The test report includes a scale score, which is the basic measurement of how a child performs on the assessment scale score: determined by the total number of test items correct or through item-pattern scoring

One of the most frequently used measurement instruments is the checklist.

A performance checklist consists of a list of behaviors that make up a certain type of performance (e.g. Using a microscope, typing a letter, solving a mathematics performance and so on).

It is used to determine whether or not an individual behaves in a certain (usually desired) way when asked to complete a particular task.

If a particular behavior is present when an individual is observed, the teacher places a check opposite it on the list.

3. PERFORMANCE TESTS

4. ORAL QUESTIONING

The traditional Greeks used oral questioning extensively as an assessment method, Socrates himself, considered the epitome (perfect example of a particular quality) of a teacher, was said to have handled his classes solely based on questioning and oral interactions,

Oral questioning is an appropriate assessment method when the objectives are: a.) to assess student’s stock knowledge and/or b.) to determine the student’s ability to communicate ideas in coherent (logical and consistent) verbal sentences.

Of particular significance are the student’s state of mind and feelings, anxiety and nervousness in making oral presentations w/c could mask the student’s true ability.

A Tally Sheet is a device often used by teachers to record the frequency of student behaviors, activities or remarks.

5. OBSERVATION AND SELF REPORTS

Useful supplementary (additional) assessment methods when used in conjunction (connects) w/

oral questioning and performance tests.

A Self-checklist is a list of several characteristics or activities presented to the subjects of a study.

C. PROPERTIES OF

ASSESSMENTMETHODS

1. Validity2. Reliability3. Fairness4. Practicality and efficiency5. Ethics in assessment

The quality of the assessment instrument and method used in education is very important since the evaluation and judgment that the teacher gives on a student are based on the information he obtains using these instruments.

1. validity

Defined as the instrument’s ability to measure what it purports (intention) to measure.

Defined as referring to the appropriateness, correctness, meaningfulness and usefulness of the specific conclusions that a teacher reaches regarding the teaching-learning situation.

Content Validity refers to the content and format of the instrument How appropriate is the content? How comprehensive?

How adequately does the sample items or questions represent the content to be assessed? Is the format appropriate?

Does the instrument logically get the intended variable or factor?

Content and Format-Consistent w/ the definition of variable or factor to be measured-1. Do students have adequate experience w/ the type of task posed by the item?

Content and Format

2. Did the teachers cover sufficient material for most students to be able to answer the item correctly?

Content and Format

3. Does the item reflect the degree of emphasis received during instruction?

Two (2) Forms of Content Validity

Table

CRITERIA

I T E M

1. Material covered

sufficiently.

2. Most students are able to answer item correctly.

3. Students have prior experience w/ the type of task.

4. Decision:Accept or Reject

FORM A: ITEM VALIDITY

1 2 3 4 5 6

FORM B: ENTIRE TESTKNOWLEDGE/

SKILLS AREAESTIMATED PERCENT OF INSTN.

PERCENT.OF ITEMS COVERED IN TEST

1. Knowledge

2. Comprehension

3. Application

4. Analysis

5. Synthesis

6. Evaluation

Two (2) Forms of Content Validity

Table

FORM B: ENTIRE TESTKNOWLEDGE/

SKILLS AREAESTIMATED PERCENT OF INSTRUCTION

PERCENT.OF ITEMS COVERED IN TEST

1. Knowledge

2. Comprehension

3. Application

4. Analysis

5. Synthesis

6. Evaluation

Based on Form B, adjustments in the number of items that relate to a topic can be made accordingly.

Two (2) Forms of Content Validity

Table

CRITERIA

I T E M

1. Material covered

sufficiently.

2. Most students are able to answer item correctly.

3. Students have prior experience w/ the type of task.

4. Decision:Accept or Reject

FORM A: ITEM VALIDITY

1 2 3 4 5 6

While Content Validity is important

Two(2) Types of Validity

1. Face Validity Outward appearance of the

test lowest form of test validity.

2. Criterion-Related Validitythe test item is judged against specific criterion, correlating the test w/ a known valid test.

1.Face Validity

  A test can be said to have face validity if it "looks like" it is going to measure what it is supposed to measure. For instance, if you prepare a test to measure whether students can perform multiplication, and the people you show it to all agree that it looks like a good test of multiplication ability, you have shown the face validity of your test.

2. Criterion-related Validity

(more important tupe)The test item is judge against a specific criterion Can also be measured by correlating the test with a known valid test (as a criterion)

A test needs to possess construct validity

A “construct” is another term for a factor, and we already know that a group of variables that correlate highly w/ each other form a factor.

Construct

let us say we are conducting a study on success in college. If we find out there is a high correlation between student grades in high-school math classes and their success in college (which can be measured by many possible variables),

Construct

We would say there is high criterion-related validity between the intermediate variable (grades in high-school math classes) and the ultimate variable (success in college). Essentially, the grades students received in high-school math can be used to predict their success in college.

2. RELIABILITY

The reliability of an assessment method refers

to its consistency. It is also a term that is

synonymous w/

dependability or stability.

Stability or internal consistency as reliability measures can be estimated in several ways.a. The Split-half Method

(using Spearman-Brown prophecy formula)

b. The Kuder-Richardson formula

a. The Split-half Method

Involves scoring two halves of a test separately for each person and then calculating a correlation coefficient for the two sets of scores.

The coefficient indicates the degreee to w/c the two halves of the test provide the same results

Hence, describes the internal consistency of the test.

Splitting a test to estimate reliability.

Example:10 item test split (2)subtests, A. 1st 1-5, 2nd 6-10 Responses: 1st half different- 2nd half Reason: increase in item difficulty and fatigue

B. Odd items vs. even items Guarantee: each half will contain an

equal number of items from the beginning, middle, and end of the original test.

The Reliability of the test is calculated using

The Spearman–Brown

prediction formula, also known as the Spearman–Brown prophecy formula

The method was published

independently by Spearmanand Brown 

(1910).

Reliability of test=2 x rhalf

1+ rhalf

Where, rhalf=reliability of half of the test

Charles Edward Spearman(Father of the True Score Theory of Reliability)

The equation for the correlation coefficient is:

Correlation Score between the two halves

Example:Five (5) Students

Test: 10 items Split-Half: odd vs. even

Result: 0.1336

Spearman–Brown prophecy formula

Reliability of test=2 x rhalf

1+ rhalf

R = 2 x 0.1336 1 + 0.1336 R = 0.2672 1.1336

R = 0.2357

Reliability

b. The Kruder-Richardson is the more frequently employed formula for determining internal consitency,

particularly KR20 (more difficult to calculate/requires a computer program)

and KR21

Dr. Frederic Kuder (1903-2000) one of the premier innovators of vocational assessments.

His 1938 Kuder Preference Record became one of the most-used career guidance instruments in schools and colleges, and was taken by more than a million people worldwide over the course of several decades.

Reliability

The Kruder-RichardsonFormula:KR20 = K { 1 – __∑ pq__} (K – 1) (Variance)

Where, K = number of items in the testp = proportion of students who answered the item correctlyq = proportion of students who answered the item wrongly = 1 – ppq = variance of a single item schored dichotomously (right/wrong)

KR21 = K {1 – n (K – M)_} (K – 1) K(Variance)Where, K = number of items on the test,M = mean of the test,Variance = variance of the test scores

The mean of a set of scores is simply the sum of the scores divided bu the number of scores; its variance is by:

Variance = Sum of differences of individual scores and mean / n – 1

Where n is the number of test takers

Reliability

c. The Test-retest Method of estimating reliability

Reliability of a test may also mean the consistency of test results when the same test is administered at two different time periods.

The estimate of test reliability is then given by the correlation of the two test results.

The test results only affected by the amount of time.

The closer the period the test given to the same set of examiners between the 1st and the 2nd , the higher the correlation. The longer the gap between the two test, the lower the correlation.

3. Fairness

An assessment procedure needs to be fair.

Students needs to know exactly what the learning targets are and what method of assessment will be used.If students do not know what they are supposed to be achieving, then they could get lost in the maze of concepts being discussed in the class. likewise, students have to be informed how their progress will be assessed in order to allow them to strategize and optimize their performance.

Assessment has to be viewedas an opportunity to learn rather than an oppurtunity to weed out poor and slow learners

Fairness also implies freedom from teacher-stereotyping.(Biases)Ex. Boys are better than Girls in Math or Girls are better than Boys in Language

04/13/2023 52

4. PRACTICALITY AND EFFICIENCY

Another Quality of a Good Assessment Procedure

Practical in the Sense that the Teacher should be familiar w/ it.Does not require Too much Time (Implementable)A Complex Assessmentt Procedure tends to be Difficult to Score and Interpret. Resulting in a lot of MisdiagnosisOr Too Long a Feedback Period w/c may render the Test Inefficient

5. ETHICS IN ASSESSMENT

The Term “Ethics” refers to questions of Right and Wrong

When Teachers think about Ethics, they need to ask themselves

If it is Right to Assess a Specific Knowledge or Investigate a Certain Question.

Are there some aspects of the Teaching-Learning situation that should Not to be Assessed?

ETHICS IN ASSESSMENT

Here are some situations in w/x assessment may not be called for:Requiring Students to

answer checklist of their sexual fantasies;Asking elementary pupils to answer sensitive questions w/o consent of their parents;Testing the mental abilities of pupils using an instrument whose validity and reliability are unknown;

ETHICS IN ASSESSMENT

When aTeacher Thinks about Ethics the Basic Question to ask in this regard is.

“Will any Physical or Psychological harm come to any one as a result of assessment or testing?”

Naturally, no Teacher would want this to

happen to any of his/her student.

ETHICS IN ASSESSMENT

Ethical (behavior) “conforming to the standards of conduct

Of a given profession or group” (Webster)The Fundamental Responsibility of a TeacherThe Most Important Ethical Consideration of allTo Do All in his/her power to Ensure that Participants in an Assessment Program

Are Protected from Physical/Psychological harm

“ “ “ Discomfort or Danger that may arise due to the testing procedure

“A Teacher who wishes to test-Physical Endurance may ask Students to climb a very steep mountain thus Endagering them physically.”

ETHICS IN ASSESSMENT

Test Results

and Assessment Results

areConfident

ial Results

Should be known only by the student concerned and the teacher

Deception(3rd Ethical issue in

assessment)

a. Determine whether the use of such

techniques is justified by the

educational value of the assessment

b. Determine whether

alternative procedures are available that do not make

use of concealment

and

c. Ensure that students are provided w/

sufficient explanationn as

soon as possible

There are instances in w/c it is necessary to conceal the objective of the assessment from the students in order to ensure fair and

impartial results.

Teacher’s Special Responsibility

Finally, the temptation to assist certain individuals in class during assessment or

testing is ever present.In this case, it is best if the teacher does not administer the test himself if he believes that

such a concern may, at a later time, be considered unethical.