Session Guide on Assessing Learning and Developing Assessment

Assessing Learning and Developing Assessment Items in Mathematics

Duration: 5.5 hours (the session includes discussion about assessment, workshop on developing assessment items, and critiquing of developed assessment items)

Objectives: At the end of the session, the participants will be able to:

solve assessment items in elementary school mathematics classify assessment items based on the TIMSS framework develop constructed response and multiple choice type of items explain distractors for the multiple choice items develop rubrics for the constructed response items critique the developed assessment items revise the assessment items based on the comments and suggestions given during the

critiquing

Resources Needed: LCD Projector, blackboard/whiteboard, chalk/whiteboard pen, Manila paper, permanent marker, Worksheets 1 and 2

Procedure:

Introduction

This session consists of three parts: discussion on assessment, workshop on developing assessment items for Grade 4 Mathematics and critiquing of the developed items. In this session, the participants will be exposed to assessment items, both multiple choice and constructed response, covering the cognitive domains of the Trends in Mathematics and Science Study (TIMSS) 2011 mathematics framework. They will be made to answer the items, explore different solutions for constructed response items such as open-ended problems, explain reasons for possible distractors, and classify the items using the TIMSS framework. They will also be shown actual answers of the pupils to the assessment items. From these answers, they will infer the understanding, difficulties and misconceptions of the pupils. For the workshop, the participants will be grouped and each group will develop one multiple choice- type item and one constructed response item. The items will be presented to the class for critiquing.

Activity• Distribute Worksheet 1 (Refer to the attached Worksheet 1).• Let the participants solve the assessment items in Worksheet 1. Encourage the participants

to think of several solutions or answers, if possible.• Let the participants present the answers to each problem one at a time.

Analysis• Discuss the answers/solutions to the problems and address misconceptions that may

surface. • Show actual pupils' responses to each item. Elicit from the participants possible pupils'

understanding/misconception related to the item.• Show a rubric which may be used to score the pupils’ responses to one constructed response

item. Let the participants give their comments and suggestions about the rubric. • Let the participants explain the distractors to one multiple choice item.

National Training of Trainers of Grade 4 Teachers for the K to 12 Basic Education ProgramUP National Institute for Science and Mathematics Education DevelopmentBatch 1: April 5-10, 2015 Batch 2: April 19-24, 2015 1


Abstraction• Let the participants identify the thinking skills involved in answering each item and let them

explain their answers.• Discuss the TIMSS 2011 Mathematics Framework for Cognitive Domains and let the

participants classify into which cognitive domain each item belongs.

Application

• Divide the participants into groups with 5 to 6 members.• Let each group develop one constructed response item with rubric and possible solutions,

and one multiple choice-type item with explanation for the distractors (Refer to Worksheet 2). Distribute Worksheet 2.

• Let each group present one item at a time while the other groups critique the item.



Notes for discussions for the assessment items in Worksheet 1:

Assessment item 1:

• Correct answer: D. PhP 400.00• Using the TIMSS Framework, it can be classified under the domain Applying• Possible explanation for its distractors

A. PhP 210.00 C. PhP 300.00

B. PhP 250.00

What to focus on: Show answers of the pupils. From these answers infer possible difficulties. Explain the possible effect of changing the phrase “twice the number of” to “twice as

many as the number of” . Other possible distractors which the participants can think of. Discuss how the options are arranged.

Assessment item 2:

• Correct answer: D. Scale on the vertical axis• Using the TIMSS Framework, it can be classified under the domain Knowing• What to focus on:

The difficulties that the pupils may encounter in answering the item. Discuss the importance of having pupils know the parts of the graph.

Assessment Item 3:

• Correct answer: 12 square units• Using the TIMSS Framework, it can be classified under the domain Reasoning• Possible solutions to the problems:

Solution 1:

••••


20 x 10 = 2002 x 5 = 10200 + 10 = 210

20 x 10 = 20010 x 5 = 50200 + 50 = 250

20 x 10 = 20010 x 5 = 50; 50 x 2 = 100200 + 100 = 300

Count the number of sides the figure has. Multiply the length of a side by the number of sides.

1 x 12 = 12 unitsThe perimeter of the figure is 12 units.


Solution 2:

Solution 3:

••••••

Solution 4:

••••

Solution 5:

•


Find the perimeter of one square.1 x 4 = 4 units

Get the perimeter of the 4 squares found along the figure.4 x 4 = 16 units

Subtract the length of the 4 sides found inside the figure.16 - 4 = 12 units

Imagine moving the sides of the squares, as shown, to form one big square that measures 3 units by 3 units.The perimeter of this big square is,

= 4 x 3 = 12 units

Five squares of the same size make up the figure. However, only 4 are along its perimeter. Out of the 4 sides of each squares, only 3 sides bound the figure.

To find its perimeter,4 x 3 = 12 units

The perimeter of the figure is 12 units.

Consider the 5 squares that make up the figure. If we consider the perimeter of all these 5 squares, it will be,

Perimeter of 1 square = 4 x 1 = 4 units

Perimeter of 5 squares = 5 x 4 = 20 unitsHowever, the square in the middle is not along the figure. So, its perimeter has to be deducted from the total perimeter.

20 - 4 = 16 unitsThe sides of the 4 squares touching the sides of the square in the middle cannot also be counted because they are not along the perimeter of the figure. So,

16 - 4 = 12 unitsThe perimeter of the figure is 12 units.


• Incorrect responses of the pupils:

• What to focus on: Based on pupils’ answers, discuss their possible understanding of length, perimeter and

area and how they visualize the problem situation. The misconceptions that may be inferred from the pupils’ answers and how these

misconceptions may be addressed. Discuss how the answers may be scored.


Answer: 5 units

Answer: 16 units

Answer: 20 units


Suggested Scoring Guide:

Points2 Correct numerical value with correct unit

Complete and accurate explanation1.5 Correct numerical value without unit

Complete and accurate explanationCorrect numerical value with correct unitIncomplete explanation

1 Correct numerical value with correct unitNo explanationCorrect numerical value without unit or wrong unitIncomplete explanation

0.5 Correct numerical value without unit or wrong unitNo explanation or wrong explanation

0 Incorrect numerical value and unitNo explanation or wrong explanation

Assessment Item 4

• Using the TIMSS Framework, it can be classified under the domain Applying• Correct answer:

Possible solutions to the problems:

Incorrect responses of the pupils:

• What to focus on: Concepts related to fractions The difficulty that may be inferred from the answers and how these difficulties may

be addressed.



Assessment Item 5

• Correct answer:

All the shapes in the options could be formed from two triangles. However, it is only the shape in option A which is not formed using two congruent right triangles as shown.

C.

Incorrect responses of the pupils


orA. B.

C. D.


• Using the TIMSS Framework, it can be classified under the domain Reasoning• What to focus on:

The properties of the shape that could be formed from cutting the rectangle. The properties of the shapes that could be formed from cutting each shape in the

options. Based on the incorrect responses of the pupils, infer the difficulties that they have in

answering the item.



TIMSS 2011 Mathematics FrameworkCognitive Domains

Knowing

Facility in using mathematics, or reasoning about mathematical situations, depends on mathematical knowledge and familiarity with mathematical concepts. The more relevant knowledge a student is able to recall and the wider the range of concepts he or she has understood, the greater the potential for engaging in a wide range of problem-solving situations and for developing mathematical understanding.

Without access to a knowledge base that enables easy recall of the language and basic facts and conventions of number, symbolic representation, and spatial relations, students would find purposeful mathematical thinking impossible. Facts encompass the factual knowledge that provides the basic language of mathematics, and the essential mathematical facts and properties that form the foundation for mathematical thought.

Procedures form a bridge between more basic knowledge and the use of mathematics for solving routine problems, especially those encountered by many people in their daily lives. In essence a fluent use of procedures entails recall of sets of actions and how to carry them out. Students need to be efficient and accurate in using a variety of computational procedures and tools. They need to see that particular procedures can be used to solve entire classes of problems, not just individual problems.

Knowledge of concepts enables students to make connections between elements of knowledge that, at best, would otherwise be retained as isolated facts. It allows them to make extensions beyond their existing knowledge, judge the validity of mathematical statements and methods, and create mathematical representations.

RecallRecall definitions; terminology; number properties; geometric properties; and notation (e.g.,

a x b = ab, a + a + a = 3a).

RecognizeRecognize mathematical objects, e.g., shapes, numbers, expressions, and quantities.

Recognize mathematical entities that are mathematically equivalent (e.g., equivalent familiar fractions, decimals and percents; different orientations of simple geometric figures).

ComputeCarry out algorithmic procedures for +, –, x, ÷, or a combination of these with whole

numbers, fractions, decimals and integer. Approximate numbers of estimate computations. Carry out routine algebraic procedures.

RetrieveRetrieve information from graphs, tables, or other sources; read simple scale.

MeasureUse measuring instruments; choose appropriate units of measurement.



Classify/OrderClassify/group objects, shapes, numbers, and expressions according to common properties;

make correct decisions about class membership; and order numbers and objects by attributes.

Applying

The applying domain involves the application of mathematical tools in a range of contexts. The facts, concepts, and procedures will often be very familiar to the student, with the problems being routine ones. In some items aligned with this domain, students need to apply mathematical knowledge of facts, skills, and procedures or understanding of mathematical concepts to create representations. Representation of ideas forms the core of mathematical thinking and communication, and the ability to create equivalent representations is fundamental to success in the subject.

Problem solving is central to the applying domain, but the problem settings are more routine than those aligned with the reasoning domain, being rooted firmly in the implemented curriculum. The routine problems will typically have been standard in classroom exercises designed to provide practice in particular methods or techniques. Some of these problems will have been in words that set the problem situation in a quasi-real context. Though they range in difficulty, each of these types of “textbook” problems is expected to be sufficiently familiar to students that they will essentially involve selecting and applying learned facts, concepts, and procedures.

Problems may be set in real-life situations, or may be concerned with purely mathematical questions involving, for example, numeric or algebraic expressions, functions, equations, geometric figures, or statistical data sets. Therefore, problem solving is included not only in the applying domain, with emphasis on the more familiar and routine tasks, but also in the reasoning domain.

SelectSelect an efficient/appropriate operation, method, or strategy for solving problems where

there is a known procedure, algorithm, or method of solution.

Represent Display mathematical information and data in diagrams, tables, charts, or graphs, and

generate equivalent representations for a given mathematical entity or relationship.

ModelGenerate an appropriate model, such as an equation, geometric figure, or diagram for

solving a routine problem.

ImplementImplement a set of mathematical instructions (e.g., draw shapes and diagrams to given

specifications).

Solve Routine ProblemsSolve standard problems similar to those encountered in class. The problems can be in

familiar contexts or purely mathematical.



Reasoning

Reasoning mathematically involves the capacity for logical, systematic thinking. It includes intuitive and inductive reasoning based on patterns and regularities that can be used to arrive at solutions to non-routine problems. Non-routine problems are problems that are very likely to be unfamiliar to students. They make cognitive demands over and above those needed for solution of routine problems, even when the knowledge and skills required for their solution have been learned. Non-routine problems may be purely mathematical or may have real-life settings. Both types of items involve transfer of knowledge and skills to new situations, and interactions among reasoning skills are usually a feature.

Problems requiring reasoning may do so in different ways, because of the novelty of the context or the complexity of the situation, or because any solution to the problem must involve several steps, perhaps drawing on knowledge and understanding from different areas of mathematics.

Even though of the many behaviours listed within the reasoning domain are those that may be drawn on in thinking about and solving novel or complex problems, each by itself represents a valuable outcome of mathematics education, with the potential to influence learners’ thinking more generally. For example, reasoning involves the ability to observe and make conjectures. It also involves making logical deductions based on specific assumptions and rules, and justifying results.

AnalyzeDetermine, describe, or use relationships between variables or objects in mathematical

situations, and make valid inferences from given information.

Generalize/SpecializeExtend the domain to which the result of mathematical thinking and problem solving is

applicable by restating results in more general and more widely applicable terms.

Integrate/SynthesizeMake connections between different elements of knowledge and related representations,

and make linkages between related mathematical ideas. Combine mathematical facts, concepts, and procedures to establish results, and combine results to produce a further result.

JustifyProvide a justification by reference to known mathematical results or properties.

Solve non-routine ProblemsSolve problems set in mathematical or real life contexts where students are unlikely to have

encountered closely similar items, and apply mathematical facts, concepts, and procedures in unfamiliar or complex contexts.

Source:

http://timss.bc.edu/timss2011/downloads/TIMSS2011_Frameworks-Chapter1.pdf


http://timss.bc.edu/timss2011/downloads/TIMSS2011_Frameworks-Chapter1.pdf


Worksheet 1

Answer the following assessment items.

1. Jessica has 5-peso and 10-peso coins. The number of her 5-peso coins is twice the number of her 10-peso coins. If she has twenty 10-peso coins, how much money does she have?

A. PhP 210.00 C. PhP 300.00 B. PhP 250.00 D. PhP 400.00

2. Willie constructed a bar graph to show the number of girls and boys in Grade 4– Roxas.

Willie forgot to write something in the graph. What is it?A. The legendB. Total number of pupilsC. Scale on the horizontal axisD. Scale on the vertical axis

3. What is the perimeter of the figure below if 1 small square is 1 unit by 1 unit?


Number of Pupils

Pupils of Grade 4 - Roxas

Gender

Bo

ysGir

Answer: ____________

Explain how you got your answer.


4. Draw a figure that shows 34

of a whole. Then shade

23 of

34 .

5. The rectangle below is cut as shown.

Which of the following CANNOT be formed out of the resulting pieces?

A. C.

B. D.



Worksheet 2

Do the following collaboratively.

1. Develop one constructed response item with rubric and possible solutions/answers and one multiple choice type-item with explanation for distractors.

2. Present the items to the class for critiquing.3. Revise the items based on the suggestions and comments given during the critiquing.

Original Assessment Item Revised Assessment ItemConstructed Response:



Original Assessment Item Revised Assessment ItemMultiple Choice:


Documents

Session Guide on Assessing Learning and Developing Assessment