Katherine L. McEldoon & Bethany Rittle-Johnson
Project GoalsDevelop an assessment of elementary
students’ functional thinking abilities, an early algebra math skill
Develop a model of knowledge progression
2
Functional Thinking
• A type of mathematical thinking which focuses on the relationship between two (or more) varying quantities, specifically the kinds of thinking that lead from specific relationships to generalizations of that relationship across instances. (Smith, 2008)
• Encapsulates important core components of early algebraic reasoning, such as generalization and covariation. (Carraher, Martinez, & Schliemann, 2008)
77
Out = (In x 2) + 1Y = 2X + 1
The table shows how the “In” numbers are related to the “Out” numbers. When a 38 goes in, what number comes out?
A.41 B.51 C. 54 D. 77
3
Functional Thinking Performance – Grade 4
The table shows how the “In” numbers are related to the “Out” numbers. When a 38 goes in, what number comes out?
A.41B.51C.54D.77
National Assessment of Educational Progress (NAEP), National Performance results in Mathematics at Grade 4; 2007
4
Function TablesFocus: Functional Tables
Determining Values and Rules
Typical Tasks (Carraher & Earnest, 2001; Schliemann & Carraher, 2000)
Fill in the missing values in this table
What is the rule for this table?Asked to select a rule from several
choicesAsked to write the rule verbally or
symbolically
5
Column A
Column B
2 6
3 7
4 8
5 9
6 10
14 18
21 25
41 45
Column A
Column B
2 6
3 7
4 8
5 9
6
14
21
41
Function Table Competencies
Within function table problems, we isolated required competencies and used this as a basis for our assessment
Loosely hypothesized order of difficulty:
1. Apply a Given Rule (prerequisite) 2. Determine Next Y value in Sequence3. Determine Near Y value in Sequence4. Determine Far Y value in Sequence5. Recognize a Rule (verbal/symbolic)6. Generate Rule Verbally7. Generate Rule Symbolically
6
Column B = Column A + 4
B = A + 4
Wilson’s Construct Modeling Approach
Wilson’s Four Building Blocks1) Construct Map2) Item Design 3) Item Score4) Measurement Model
Assess the student performance data to evaluate your construct map and items
7
Item Desig
n
Item Score
Measurement Model
Construct Map
Item Design: Assessment We designed items that tapped each of these
competenciesModified from e.g. Blanton; Schliemann; Warren, Cooper &
Lamb
Items varied in operation used in underlying function
33 responses to 11 items16 of which had an additive underlying function
Y = X + 2
10 had a combination underlying function Y = 2X + 2
8
Item Design: Assessment We developed items that
tapped each of these competencies1. Apply a Given Rule
2. Determine Next Y value in Sequence
3. Determine Near Y value in Sequence
4. Determine Far Y value in Sequence
5. Recognize a Rule
6. Generate Rule Verbally
7. Generate Rule Symbolically
9
Item Design: Assessment We developed items that
tapped each of these competencies1. Apply a Given Rule
2. Determine Next Y value in Sequence
3. Determine Near Y value in Sequence
4. Determine Far Y value in Sequence
5. Recognize a Rule
6. Generate Rule Verbally
7. Generate Rule Symbolically
10
Column A
Column B
2 6
3 7
4 8
5 9
6
14
21
41
Item Design: Assessment We developed items that
tapped each of these competencies1. Apply a Given Rule
2. Determine Next Y value in Sequence
3. Determine Near Y value in Sequence
4. Determine Far Y value in Sequence
5. Recognize a Rule
6. Generate Rule Verbally
7. Generate Rule Symbolically
11
Column A
Column B
2 6
3 7
4 8
5 9
6
14
21
41
Item Design: Assessment We developed items that
tapped each of these competencies1. Apply a Given Rule
2. Determine Next Y value in Sequence
3. Determine Near Y value in Sequence
4. Determine Far Y value in Sequence
5. Recognize a Rule
6. Generate Rule Verbally
7. Generate Rule Symbolically
12
Column A
Column B
2 6
3 7
4 8
5 9
6
14
21
41
Item Design: Assessment We developed items that
tapped each of these competencies1. Apply a Given Rule
2. Determine Next Y value in Sequence
3. Determine Near Y value in Sequence
4. Determine Far Y value in Sequence
5. Recognize a Rule – Verbal & Symbolic
6. Generate Rule Verbally
7. Generate Rule Symbolically
13
What is a rule used in the table above to get the numbers in column B from the numbers in column A? A) Multiply the number incolumn A by 2.B) Divide the number in column A by 2.C) Subtract 2 from the number in column A.D) Add 2 to the number in column A.
Item Design: Assessment We developed items that
tapped each of these competencies1. Apply a Given Rule
2. Determine Next Y value in Sequence
3. Determine Near Y value in Sequence
4. Determine Far Y value in Sequence
5. Recognize a Rule
6. Generate Rule Verbally
7. Generate Rule Symbolically
14
Column A
Column B
2 6
3 7
4 8
5 9
6
14
21
41
“The rule is that you add 4 to the A number to get the B number”
What is a rule for figuring out what number belongs in column B?
Item Design: Assessment We developed items that
tapped each of these competencies1. Apply a Given Rule
2. Determine Next Y value in Sequence
3. Determine Near Y value in Sequence
4. Determine Far Y value in Sequence
5. Recognize a Rule
6. Generate Rule Verbally
7. Generate Rule Symbolically
15
Column A
Column B
2 6
3 7
4 8
5 9
6
14
21
41
“B = A + 4”
Write this rule as a number sentence, using “A” to stand for any number in column A and “B” to stand for any number in column B.
Item Scores: CodingCoding
Each response only tapped one competency
Each was coded as correct or incorrect
16
Item Desig
n
Item Score
Measurement Model
Construct Map
Data Collection: Procedure231 second through sixth grade students
Middle class suburban community
Predominantly Caucasian population
During one 40 minute class period
17
Measurement ModelBased on Item Response Theory
Item Response Theory encompasses a set of ways to mathematically model how both Student Ability Estimate and Item Difficulty are related to a student’s Item Responses
It is a useful methodology to use when evaluating an assessment instrument both in terms of its ability to accurately
estimate student ability but it also give metrics of the quality of each
item on the instrument.
18
Measurement ModelWright Map
An Wright map generated by a Rasch model (a type of item response model) and was used in this evaluation
Logit Scale (log-odds ratio)
Student Ability Estimates
Item Difficulties19
--------------------------------------------------------------- 5 XX| XXXX| XX| 4 XXXXXX| XXXXXXXX| XXXXXX| XXXXXXXXXXXXXXXX| 3 XXXXXXXXXXXX| XXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXX| XXXXXXXXXXX| 2 XXXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXX| 1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|L4 XXXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXX|L3 0 XXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|L3 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|L3 -1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|L3 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|L3 XXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|L2 L2 -2 XXXXXXXXXXXXXXXXXXXX|L2 L2 L2 XXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXX| -3 XXXXXXXXXXXXXX| XXXXXXX|L1 L1 L1 XXXX| XXX|L1 L1 -4 X| X| XX| | ==================================================================
Student Ability ScoresItem Difficulty Scores
Wright MapAn Wright map was generated by a Rasch model (a
type of item response model) and was used in this evaluation
Item difficulties based on the Wright maps were used in the development of our Construct map
20
XXXXXXXX| XXXXXXXXXXX|Item E 3 XXXXX-3-Item C Item D XXXXXXXXXX| XXXXXXXXXXXX| 2 XXXXXXXXX-2- XXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXX| 1 XXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|Item B XXXXXXXXXXXXXXXXXXXXXXXXXX| 0 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|Item A
Student Ability ScoresItem Difficulty Scores
This item has a difficulty level of .98, meaning that the average student has a ~0.47 probability of getting it correct
This item difficulty is 3.1logits, or the average student has a ~0.28 probability of getting it correct
Measurement ModelWright Map
XXXXXXXXXXXXXXXXXX| | XXXXXXXXXXX| | 2 XXXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXX| | 1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|7-16 | XXXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXX|6-6 | 0 XXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|6-14 | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|4-15 | -1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|4-13 | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|3-12 | XXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|3-4 3-5 | -2 XXXXXXXXXXXXXXXXXXXX|5a-9 2-10 2-11 | XXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXX| | -3 XXXXXXXXXXXXXX| | XXXXXXX|1-3 1-7 1-8 | XXXX| | XXX|1-1 1-2 | -4 X| | X| | XX| |=======================================================================================Each 'X' represents 0.4 cases=======================================================================================0
21
7) Generate Rule Symbolically
6) Generate Rule Verbally5) Recognize a Rule4) Determine Far Y value in
Sequence3) Determine Near Y value
in Sequence2) Determine Next Y value
in Sequence1) Apply a Given Rule
Wright Map Addition Functions
Wright Map Combination Functions
---------------------------------------------------------------------------------------| 4 XXXXXXXXX| | XXXXXXXX| | XXXXXXXXXXX|4-8 | 3 XXXXX|3-6 4-7 6-9 7-10 | XXXXXXXXXX| | XXXXXXXXXXXX| | 2 XXXXXXXXX| | XXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXX| | 1 XXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|2-5 | XXXXXXXXXXXXXXXXXXXXXXXXXX| | 0 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|5b-4 | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|5a-3 | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | -1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | -2 XXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXX|1-2 | XXXXXXXXXXXXXXX| | -3 XXXXXXXXX| | XXXXXXXXXXX|1-1 | XXXXXXX| | -4 XXXXXXXXX| |=======================================================================================Each 'X' represents 0.4 cases
22
7) Generate Rule Symbolically
6) Generate Rule Verbally5) Recognize a Rule4) Determine Far Y value in
Sequence3) Determine Near Y value
in Sequence2) Determine Next Y value
in Sequence1) Apply a Given Rule
Construct MapConstruct Map
A representation of the continuum of knowledge that people are thought to progress through for the target construct (Wilson, 2005)
Placed competencies into a hierarchy based on We used item difficulty scores from IRT measures Their clumping on the Wright maps From theory
23
Mapping of Competencies into Construct Map Levels
Level Description
Competencies
Level 4: Generate Symbolic Rule
- Generate an explicit symbolic rule
Level 3: Generate & Use Verbal Rule
- Generate an explicit verbal rule- Complete a function table with missing values
Level 2: Recognize Rule & Determine Next
- Select a correct rule out of several choices- Determine the next Y value in a function sequence
Level 1: Apply Rule
- Use a given rule to determine new Y values
24
7. Generate Rule Symbolically
6. Generate Rule Verbally
5. Recognize a Rule (verbal/symbolic)
4. Determine Far Y value in Sequence
3. Determine Near Y value in Sequence
2. Determine Next Y value in Sequence
1. Apply a Given Rule
Benefits of a Construct Modeling Approach
First, it elucidated the relative difficulty of functional thinking abilities, and at times this was not in line with our predictions.
Second, the resulting assessment is a criterion referenced measure which is particularly appropriate for assessing
Students’ ability estimate levels
Learning gains from an intervention
25
Summary Identified key competencies that are important for
elementary-level functional thinking, with a focus on function table problems
These competencies were then incorporated into an assessment
Student performance data was used to develop a construct map, or proposed knowledge progression, of elementary-level functional thinking abilities
The resulting construct map provided insight into the acquisition of functional thinking knowledge in elementary-school students
This can be used as a research tool, and to guide instructional sequences for students
26
Thank you
For more information:
http://peabody.vanderbilt.edu/earlyalgebra.xml
27
The first author is supported by a predoctoral training grant provided by the Institute of Education Sciences, U.S. Department
of Education, through Grant R305B040110 to Vanderbilt University. The opinions expressed are those of the authors and
do not represent views of the U.S. Department of Education.
28
Wright Map - Multiplication
--------------------------------------------------------------------------------------- 5 XXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXX| | 4 XXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | 3 XXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|7-7 | XXXXXXXXXXXXXXXXXX|3-4 | 2 XXXXXXXXXXXXXXXXXXXXXXXXXXXXX|4-3 3-5 | XXXXXXXXXXXXXXXXXX|6-6 | 1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|5-1 | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | 0 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|2-2 | XXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | -1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | -2 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXX| | -3 XXXXXXXXXXXXXXXXXXXXXXXXXX| |=======================================================================================Each 'X' represents 0.3 cases=======================================================================================
29
1. Apply a Given Rule 2. Determine Next Y value
in Sequence3. Determine
Near Y value in Sequence
4. Determine Far Y value in Sequence
5. Recognize a Rule (symbolic)
6. Generate Rule Verbally
7. Generate Rule Symbolically