BASIC CONCEPTS IN ASSESSMENT

1.0 Introduction

As teachers, we are continually faced with the challenge of assessing the progress of our students as well as our own effectiveness as a teacher. Assessing decisions could substantially improve student performance; guide the teachers in enhancing the teacher-learning process and assist policy makers in improving the educational system. At the same time, however, poor assessment procedures could adversely affect the students, teachers and administrators. Assessment of learning is a tricky business, indeed, for it requires measuring concepts, ideas and abstract constructs quite unlike the assessment of physical quantities which can be done with appropriate degree of accuracy. In assessment of learning, we deal with intangibles and attempt to characterize them in a manner that would be widely understood.Not too long ago, assessment of learning was confined to techniques and producers for determining whether or not cognitive knowledge (memorization of facts and theories) was successfully acquired. Thus, assessment was essentially confined to pencil-paper testing of the cognitive levels of learning (Bloom, 1954). In the past two decades, however, educators and educationists recognized that not only are we expected to know facts and figures in todays society, but we are also expected to function effectively in the modern world, inter act with other people, and adjust to situations. Such expectations have not been matched with appropriate assessment methods which could identify successful acquisition of skills other than cognitive skills until the early to late 1990s. Consequently, the traditional assessment method of pencil and paper testing identified potentially high performing students who, in later life, have not been successful in coping with the demands of modern society.This book cover the why, what how, and of assessing students learning. The underlying philosophy is that of assessment for learning rather than assessment of learning per se: students are assessed in order to enhance the teaching-learning process guided by the principle that education is a lifelong activity and does not cease after once graduation from school. Assessment of learning, however, is the building block by which we introduce the idea of assessment for learning in Philippine schools.The most common method assessing student learning is though tests (teacher-made od standardized). Despite some criticisms levelled against using test in determining if students are learning or if schools are successful, these tests will continue to be used in the foreseeable future (Shepard, 2000). Test results provide an easy and easily understood means of informing the student about his progress or the school about his performance. Standardized tests, in particular, provide clear targets to aim for when teachers and administrators want improvement (Jason, 2003). Test, coupled with other observational performance-based techniques, provide a powerful combination for objective and precise assessment procedure.

1.1 Educational Measurement

The first step towards elevating a field of study into a science is to take measurements of the qualities of interest in the field. In the Physical Sciences, such measurements are quite easily understood and well-accepted. For instance, if we want to measure the length of a piece of string, we compare the length of the string with a standard ruler or meter stick; to find the weight of an object, we compare the heaviness of the object with a standard kilogram or pound and so on. Sometimes, we can measure physical quantities by combining directly measurable quantities to form derived quantities. For example, to find the area of a rectangular piece of paper, we simply multiply the lengths of the sides of the paper. In the field of educational measurement, however the qualities of interest are more abstract, unseen and cannot be touched. They cannot be observed thus makes the measurement process in education much more difficult.For instance, knowledge of the subject matter is often measured through standardized test results. In this case, the measurement procedure is testing. The same concept can be measured in another way. We can ask a group of experts to rate a students (or a teachers) knowledge of the subject matter in a scale of 1 to 5 with 1 being the lowest and 5 the highest. In this latter procedure, knowledge of the subject mater is measured through perceptions.

1.1.1 Types of Measurement

Measurement s can therefore be objective (as in testing) or subjective (as in perceptions). In the example above, testing procedures objective measurements. Objective measurements are more stable than subjective measurements in the sense that repeated measurements of the same quantity or quality of interest will produce more or less the same outcome. For this reason many people prefer objective measurements over subjective measurements whenever they are available. However, there are certain facets of the quantity or quality of interest that cannot be successfully captured by objective procedures but which can be done by subjective methods e.g. aesthetic appeal of a product or project of a student, etc. It follows that it may be best to use both methods of assessment whenever the constraints of time and resources permit.Whenever one uses an objective or subjective assessment procedure, the underlying principle in educational measurement is summarized by the following formula:Measurement of Quantity or Quality of Interest = True values plus random error.Each measurement of the quantity of interest has two components: a true value of the quantity and random error component. The objective in educational measurement is to estimate or approximate, as closely as possible, the true value of the quantity of interest, e.g. true knowledge of the subject matter. This is a tall order and one which will occupy most of our time in this particular course.Objective measurements are measurements that do not depend on the person or individual taking the measurements. Regardless of who is taking the measurement, the same measurement values should be obtained when using an objective assessment procedure. In contrast, subjective measurements often differ from one assessor to the next even if the same quantity is being measured.

1.1.2 Indicators, Variables and Factors

An educational variable (denoted by an English alphabet, like X) is a measureable characteristic of a student. Variables may be directly measurable as in X = height of a student. However, many times, a variable cannot be directly measured like when we want to measure class participation of a student. For those variables where direct measurements are not feasible, we introduce the concept of indicators.An indicator, I, denotes the presence or absence of a measured characteristic. Thus:I = 1, if the characteristics is present = 0, if the characteristic is absentFor the variable X = class participation, we can let I1, I2, ....., In denote the participation of a student in n class recitations and let X = sum of the Is divided by n recitations. Thus, if there were n = 10 recitations and the student participated in 5 of these 10, then X = 5/10 or 50%.Indicators are the building blocks of educational measurement upon which all other forms of measurement are build. A group of indicators constitute a variable. A group of variables form a construct or a factor. The variable which form a factor correlate highly with each other but have low correlations with variables in another group.Example: The following variables were measured in a battery of tests:

X1 = computational skillsX2 = reading skillsX3 = vocabularyX4 = logic and reasoningX5 = sequences and seriesX6 = manual dexterity

These variables can be grouped as follows:

Group 1: (X1, X4, X5) = mathematical ability factorGroup 2: (X6) = psychomotor ability factor

The first group is called a mathematical ability factor, the second group is called a language ability factor while the third group (with only one variable) is called a psychomotor ability factor.In educational measurement, we shall be concerned with indicators, variables and factors of interest in the field of education.

1.2 Assessment

Once measurements are taken of an educational quantity or quality of interest, then the next step is to assess the status of that educational phenomenon. For example, suppose that the quantity of interest is the level of mathematics performance of Grade VI pupils in a school district. The proposed measurements are test scores obtained by administering a standardized achievement test in Mathematics for Grade VI pupils in the district. The District Office decided to target an achievement level 85% for the Grade VI pupils. Based on the achievement test results, the school officials can assess whether their Grade VI pupils are within a reasonable range of this target i.e. whether they are above or below the achievement level target.

1.2.3 Various Roles of Assessment

Assessment plays a number of roles in making instructional decisions:Summative Role. An assessment may be done for summative purposes as in the illustration given above for grade VI mathematics achievement. Summative assessment tries to determine the extent to which the learning objectives for a course (like Grade VI Mathematics) are met and why. Diagnostic Role. Assessment may also be done for diagnostic purposes. In the case, we are interested in determining the gaps in learning or processes, hopefully, to be able to bridge these gaps. Thus, on the topic of sentence construction, a diagnostic examination may reveal the difficulties encountered by the students in matching subject and verb or in identifying subject and predicate, in vocabulary etc. This function of assessment is akin to a medical doctor trying to perform laboratory tests to determine a patients illness or disease. As in a medical doctor, a teacher needs to be extra careful in making a diagnoses of learning difficulties. Accurate diagnosis leads to appropriate teaching measures and intervention programs but a single misdiagnosis can also have potentially disastrous consequences. For instance, a pupil who is unable to read and unable to score well in a test for grammar may not actually have difficulties in the language per se but could, in fact, be suffering from physical disabilities like dyslexia. Consequently, the teacher may provide for tutorial sessions believing that the student may just be a slow learner or having language difficulties thus wasting precious resources to no avail.Formative Assessment. Another purpose of assessment is formative. In this role, assessment guides the teacher on his/her day-to-day teaching activity. Should a topic be taught again? Should there be more drills and exercises? In the context of teaching-learning situation, the formative value of assessment is perhaps the most important. It allows the teacher to redirect and refocus the course of teaching a subject matter.Placement. The final role of assessment in curricular decisions concerns placement. Assessment plays a vital role in determining the appropriate placement of a student both in terms of achievement and aptitude. Aptitude refers to the area or discipline where a student would most most likely excel or do well. Thus, an aptitude test determines if a student would do well in scientific or humanities courses; in technical-vocational or academic courses, etc. Placement examinations also determine if a students ability is equivalent to, sa, a typical 3rd year or 4th year high school student. The Departmant of Education has this type of placement examination and it used such an examination to calibrate the placement of such personalities on TV or widescreen.The defunct NCEE (National College Entrance Examination) was an academic placement examination.1.3 Evaluation of Learning and ProgramsEvaluation models are important in the context of education. Evaluation implies that measurements and assessments of an educational characteristic had been done and that is now desired to pass on value judgement on the educational outcome. In evaluating an outcome, we consider the objectives of the educational process and analyze whether the outputs and outcomes satisfy the objectives and to what extent these outputs and outcomes satisfy these objectives. If they do not, then we need to find the possible reasons for our failure to meet such objectives. The possible reasons can, perhaps, be identified from the context, inputs, process, and outputs of the educational system. Figure 1 illustrates these ideas:

CONTEXT INPUTS PROCESS

OUTPUT OUTCOME

Figure 1A System Model for EvaluationEvaluation provides a toll for determining the extent to which an educational process or program is effective and at the same time indicates directions for remediating processes of the curriculum that do not contribute to successful student performance (Jason, 2003). To this end, evaluation enhances organizational efficiency by providing focus for teacher and administrator efforts as well as allows resources (both time and money) to be directed to areas of greatest need.Improving student performance is inextricably linked to improvement in the inputs and processes that shape the effectiveness of teaching and learning. Evaluation, therefore, is of greatest interest to both teachers and administrators who plan and orchestrate the entire learning activities.According to Brainard (1996), effective program evaluation is a systematic process that focuses on program improvement and renewal and discovering peaks of programs excellence. Program evaluation needs to be viewed as an important ongoing activity, on that goes beyond research or simple fact-finding to inform decisions about the future shape of the program under study (Jason, 2003). Program evaluation contributes to quality services by providing feedback from program activities and outcomes to those who can introduce changes in the program or who decide which services are to be offered. Without feedback, human service programs cannot be carried out effectively (Posavac & Carrey, 1997).The systems model for evaluation is not the only available program evaluation model. The Delphi evaluation model, for instance, was a popular evaluation technique in the past. Essentially, the Delphi method relies on the perceptions and judgement of experts, normally, external to the system being evaluated. The program implementer identifies the areas of concern in the program which he/she desires to be evaluated and the external experts render judgement on these areas based on their collective past experience and expertise.Program evaluation need to be limited to evaluation of educational processes and systems. Program evaluation is also important in many programs of government undertake programs with the assistance of foreign funding agencies to target specific social concerns e.g. poverty reduction or governance and empowerment at the local government level. Such programs normally involve millions and it is very important that proper evaluation be undertaken to ensure that resources invested in these programs are not wasted. In such cases, the method called PERT (Program Evaluation Review Technique) is an indispensable quantitative evaluation tool. SummaryEvaluationIs the process of gathering and interpreting evidence regarding the problems and progress of individuals in achieving desirable educational goals.Chief Purpose of EvaluationThe improvement of individual learner.Other Purposes of EvaluationTo maintain standardTo select studentsTo motivate learnersTo guide learningTo furnish instructionTo appraise educational instrumentalitiesFunction of EvaluationPredictionDiagnosisResearchAreas of Educational EvaluationAchievementAptitudeInterestPersonalityA well defined system of evaluation:Enable one to clarify goals.Check upon each phase of developmentDiagnose learning difficultiesPlan carefully for remediationEvaluation and the Teaching-Learning ProcessTeaching, Learning and Evaluation are three interdependent aspects of the educative process (Gronlund, 1981). This interdependence is clearly seen when the main purpose of instruction is conceived in terms of helping pupils achieve a set of learning outcomes which include changes in the intellectual, emotional or physical domains Instructional objectives or in other words, desired changes in the pupils, are brought about by planned learning activities, and pupils progree is evaluated by tests and other devices.This integration of evaluation into the teaching-learning process can be seen in the following stages of the process:Setting instructional objectivesDetermining pupil variables that can affect instructionProviding instructional activities that are relevant and necessary to achieve the desired learning outcomesDetermining the extent to which desired outcomes are achievedPrinciples of Educational EvaluationEvaluation must be based on previously accepted educational objectives.Evaluation must be continuous comprehensive and a cumulative process.Evaluation should recognize that the total individual personality is involved in learning.Evaluation should be democratic and cooperative.Evaluation should be positive and action-directed.Evaluation should give opportunity to the pupil to become increasingly independent in self appraisal and self-direction.Evaluation should inculcate all significant evidence from every possible source.Evaluation should take into consideration the limitations of the particular educational situations.MeasurementIs a part of the educational evaluation process whereby some tools or instruments are used to provide a quantitative description of the progress of students towards desirable educational goals.Test or TestingIs a systematic procedure to determine the presence or absence of certain characteristics or qualities in a learner.Types of EvaluationPlacementFormativeDiagnosticSummative(These types show that evaluation is integrated with the various phases of instruction)PlacementEvaluation accounts for the students entry behaviour or performance. It determines the knowledge and skills he possesses which are necessary at the beginning of instruction in a given subject area.FormativeEvaluation provides the students with feedback regarding his success or failure in attaining instructional objectives.It identifies the specific learning errors that need to be corrected and provides reinforcement for successful performance as well.For the teacher, formative evaluation provides information for making instruction and remedial work more effective.Pointers in Formative Evaluation1. There should be an achievement continuum at every level of instruction.2. Criteria and standards be weal-defined at the beginning of the learning process.3. Criterion-referenced tests should be administered.4. Students should always be informed of their progress.5. Enrichment/remedial/opportunities should be made available.DiagnosticEvaluation is used to detect students learning difficulties which are revealed by formative tests or checked by remedial instruction and other instructional adjustments.Since it discloses the underlying causes of learning difficulties, diagnostic tests are therefore more comprehensive and detailed.SummativeEvaluation is concerned with what students have learned. This implies that the instructional activity has for the most part been completed and that little correction of learning deficiencies is possible.Stages of Teaching-Learning in which Educational Evaluation is integrated1. Clarifying objectives2. Identifying variables that affect learning.3. Providing relevant instructional activities to achieve objectives.4. Determining the extent to which the objectives are achieved.CHAPTER EXERCISESA. Classify the following either as objective or subjective measurement.1. Age of a pupil in years.2. Test score in Grade 3 science.3. Attitude score based on observations.4. Class participation.5. Length of a piece of paper in inches.6. Classroom behaviour scale.7. Grade of a project submitted.8. Showmanship in a school play.9. Height of a building in meters.10. Perceptions on the effectiveness of a teacher.B. The following test scores in First Year Algebra were observed for a particular student:(75, 82, 83, 76, 78, 81):1. Assuming that these test scores reflect the students true ability plus random error, what would be your best guess of the students true ability? Why? (Hint: Get the average of the scores.)2. From your answer in No. 1, calculate the difference between the actual scores and your best guess of the true ability of the student:D = actual score true ability guessWhat is the sum of these differences? From your answer, justify the choice of the average as your best guess.C. Identify the assessment function illustrated by the following:1. Entrance examination2. Daily quiz3. Unit test4. Periodic test5. District wide test6. National scholastic aptitude test7. Test of English as a Foreign Language (TOEFL)8. Licensure Examination for Teachers (LET)9. Attitudinal Test10. IQ TestD. Discuss the systems model for evaluation. How does one explain the outcome of an educational process in terms of context, inputs and processes?E. Differentiate assessment from evaluation. How do they differ and how are they similar?F. What are the benefits derived from proper program evaluation? Discuss these benefits.G. Consider the following measured variables:X1 = logical reasoningX2 = sentence constructionX3 = basketball skillsX4 = folk dancing skillsX5 = computational skillsX6 = geometry scoreX7 = vocabulary1. How many factors can you identify? Name these factors and justify your groupings.2. Find some indicators of X3 and X4. Discuss these indicators.H. Discuss the function and importance of the following assessments in a typical teaching-learning situation:1. Assignments2. Daily quiz3. Pre-test4. Post test5. Unit test6. Periodic test7. Projects8. Self-evaluation reports9. Oral recitations10. Performance testI. CASE ANALYSIS: Ms. Maria Jane Castillo was assigned to teach in a remote barrio up in the mountains of Agusan del Sure. She was assigned to teach English for high school students. Her class consisted of 25 male and 30 female students coming from different elementary schools in the municipality of Bayugan. Within the first month of her teaching in the barrio, she noted that male students tended to be absent every Monday and Friday. This phenomenon caused the performance of male students in the graded English tests to be lower than the female students.1. Were the differences in English scores of male and female students attributable to differences in mental abilities?2. To which would you attribute these differences?3. If you were Ms. Castillo, what would you do?Chapter 2PRINCIPLES OF HIGH QUALITY ASSESSMENT2.1 Clarity of Learning TargetsAssessment can be made precise, accurate and dependable only if what are to be achieved are clearly stated and feasible. To this end, we consider learning targets involving knowledge, reasoning, skills, products and effects. Learning targets need to be stated in behavioural terms or terms which denote something which can be observed through the behaviour of the students. Thus, the objective to understand the concept of buoyancy is not stated in behavioural terms. It is not clear how one measures understanding. On the other hand, if we restate the target as to determine the volume of water displaced by a given object submerged, then we can easily measure the extent to which a student understands buoyancy.2.1.1 Cognitive TargetsAs early as the 1950s, Bloom (1954), proposed a hierarchy of educational objectives at the cognitive level. These are:Level 1. KNOWLEDGE refers to the acquisition of facts, concepts and theories. Knowledge of historical facts like the date of the EDSA revolution, discovery of the Philippines or of scientific concepts like the scientific name of milkfish, the chemical symbol of argon, etc. All fall under knowledge.Knowledge forms the foundation of all other cognitive objectives for without knowledge, it is not possible to move up to the next higher level of thinking skills in the hierarchy of educational objectives.