The Effectiveness of Concept Maps in Assessment and Learning in Mathematics Education … · 2015-07-29 · Effectiveness of Concept Maps 2 Abstract The effectiveness of concept maps

Effectiveness of Concept Maps 1

Running head: EFFECTIVENESS OF CONCEPT MAPS

The Effectiveness of Concept Maps in Assessment and

Learning in Mathematics Education

Gary Greer

Memorial University of Newfoundland

Cape Breton University


Abstract

The effectiveness of concept maps in mathematics education was examined through

review of relevant empirical research studies. Concept maps are identified by Novak as useful

tools in learning and assessment, comprising concepts connected by linking phrases that form

propositions relevant to a particular knowledge domain, organized in a hierarchical structure.

To examine claims of the effectiveness of concept maps in terms of assessment and learning

facilitation in mathematics education, analysis of validity and reliability was undertaken. The

studies evidenced only moderate empirical evidence for the effectiveness of concept maps in

assessment applications, and questionable empirical evidence that concept maps facilitate

learning. Qualitative and holistic researcher perspective was more positive, as was student

perspective. More research is required to allow educators to use these tools strategically and

confidently in mathematics educational contexts.


Contents

Abstract 2

Motivation 4

Research Questions and Hypothesis 5

Background 6

A Definition for Concept Maps 6

Theoretical Support for the use of Concept Maps 7

Assessing Concept Maps 8

Analysis of Published Research 9

Elementary Level Studies 10

Secondary Level Studies 11

Post-Secondary Level Studies 13

Discussion 18

Are Concept Maps Effective Assessment Instruments? 18

Can Learning be Attributed to the Use of Concept Maps? 21

Inconsistencies, Gaps and Limitations in Research 22

Relationships Identified and Implications 23

References 24

Figures and Tables

Figure 1. Conceptual framework. 5

Figure 2. Basic features of a concept map. 7

Table 1. Summary of relevant characteristics of studies. 10

Table 2. Summary of confidence indicators from studies. 18


The Effectiveness of Concept Maps in Assessment and

Learning in Mathematics Education

This study comprises an analysis of literature in mathematics education related to the

use of concept maps in the context of current educational theories and practice. Concept maps

are widely accepted as learning and assessment tools in science education (Coffey et al.,

2003). Although meta-analysis of 18 studies in science education by Horton et al. (1993)

indicates that the use of concept maps is generally associated with improved student

achievement and positive student attitudes, there is no assurance that the instrument will

prove similarly effective in mathematics education. In consequence, it is important to

establish the effectiveness of this instrument in mathematics education.

Motivation

Motivation for this study is based on timeliness and author interests. First, there are

increasing concerns voiced over poor student achievement in mathematics in Ontario (Ontario

Ministry of Education, 2004). There are many sources of evidence for this, including the

Educational Quality and Assurance Organization (2003) which notes widespread incidence of

low scores at the Grade 9 Applied level. Second, there have been significant changes in

educational perspectives on knowledge creation (Novak, 2003), conceptual understanding

(Mwakapenda, 2005), and the importance of both relationships and connections in

mathematics education (National Council of Teachers of Mathematics, 2005). Third, while

concept maps have been used for many years, the recent availability of powerful computer

software for constructing concept maps offers a number of advantages (Coffey et al, 2003).

Finally, the author is a teacher of secondary school mathematics and computer science with a

keen interest in supporting learning through the use of information technologies.


Research Questions

Although there are many perspectives on what comprises education, it can be argued

that two critical interactions are facilitating learning and assessing student knowledge, either

in preparing for learning or evaluating the effect of learning experiences. It is not known

whether concept maps will prove effective in supporting these two processes in mathematics

education, suggesting the research question: Will a review of literature provide supporting

evidence that concept maps are effective tools in facilitating learning and assessing student

knowledge in mathematics education?

An assessment will be considered to be an activity that provides some measurement of

student knowledge. Learning will be considered to comprise a long-term change in student

knowledge as measured by traditional assessments. Validity will be considered the ability of

an instrument to measure what it purports to measure, while reliability will be considered the

ability of an instrument to yield consistent results when the characteristic being measured has

not changed (Leedy & Ormrod, 2001). A conceptual framework has been developed to

represent the relationships and is shown in Figure 1.

Figure 1. Conceptual framework for the use of concept maps in educational settings.


Two specific questions will be used to analyse evidence of the effectiveness of

concept maps in mathematics education:

1. Are concept maps valid and reliable instruments for assessing mathematical

knowledge?

2. Can improvements in student learning be attributed to the use of concept maps?

Background

A Definition for Concept Maps

This discussion will utilize the definition developed by Novak and Gowin (1984). Not

all papers cited were as rigorous in their adherence to this definition. For example, a number

of studies did not require a hierarchical structure. Graphs are mathematical representations

containing a series of objects variously called nodes, vertices or points, with connections

variously called lines, arcs or edges (Weisstein, 2005). A concept map is a type of graph

where the nodes represent a “concept” and the labeled lines describe the relationship between

two concepts (Novak & Gowin), a concept being defined as “a regularity in events or objects

designated by some label”. (Novak & Gowin, 1984, p. 4) Further, Novak and Gowin (1984,

Chapter 2) describe mapping as an activity where the learner must engage to clarify meanings

by identifying important concepts, relationships, and structure within a chosen domain of

knowledge. The basic features of a concept map are diagrammed in Figure 2.

Unfortunately, in educational literature, the term concept map is used in a more

general sense to describe a number of different types of graphical representations of

knowledge, as well as in a specific manner to refer to the instrument developed by Novak and

Gowin (1984). However, the instrument developed by Novak and Gowin contains a number

of defining features as noted by Coffey et al, (2003). First, Novakian concept maps are


Figure 2. Basic features of a concept map.

grounded in Ausubel’s (1968) assimilation theory, being designed to support the integration

of new knowledge into existing cognitive structures. Second, they comprise a hierarchical

structure with the most general concepts located at the top and the most specific concepts

located at the bottom of the representation. Third, links between two concepts are labeled,

forming propositions which are “statements about some object or event in the universe, either

naturally occurring or constructed,” (Novak, n.d., p. 1) and are considered units of meaning.

Theoretical Support for the use of Concept Maps

Concept maps were developed by Novak and Gowin (1984) through research into

learning based on Ausubel’s (1968) work into the acquisition of knowledge by children.

Novak and Gowin proposed that concept maps encourage meaningful learning through the

identification and understanding of relationships between concepts, rather than through rote

learning. Ausubel proposed that learning takes place by a process called assimilation,

whereby new concepts and propositions are incorporated into the learner’s existing

knowledge frameworks, a process heavily mediated by language, social interaction and

interaction with the environment and/or objects (Coffey et al., 2003). Skemp (1987) supports


this, arguing that mathematical knowledge is organized into hierarchical conceptual structures

called schemas, and that mathematical learning may be modeled in terms of the assimilation.

Freeman and Jessup (2004) suggest that the use of concept maps in learning is also supported

by associationist theory (Deese, 1965) which is similar to assimilation theory except that the

former has no requirement for a hierarchical cognitive structure.

Support is also provided by Ginsburg (1989) who suggests that understanding may be

viewed as a connection between two pieces of information. Further, Hiebert and Carpenter (as

cited in Baroody & Bartels, 2000, p. 604) suggest that the degree of understanding may be

assessed by the number, strength and accuracy of the connections. These create a perspective

whereby concept maps may be used to measure the individual’s degree of understanding, and

provide theoretical support for assessment of understanding using concept maps.

Novak (n.d.) views concept maps as supporting learners by enhancing their ability to

work with many concepts at one time, a characteristic also noted by McAleese (1998). Novak

and Gowin (1984) and McAleese also suggest that concept maps assist students in

metacognitive processes, helping them learn to learn. The effectiveness of concept mapping is

supported by constructivist learning models like those proposed by Piaget (1959) and

Vygotsky (1978) who theorized that knowledge is actively constructed by the learner, and is

further supported by the zone of proximal development learning model (Vygotsky, 1978).

Assessing Concept Maps

Roberts (1999) suggests that there are two basic quantitative schemes for concept map

assessment: those based on the Novak and Gowin (1984) assessment rubric and those based

on assessment against a criterion. The classic Novakian assessment scheme awards 1 point for

valid propositions, 5 points for valid hierarchies, 10 points for valid crosslinks showing


synthesis and 2 for less significant ones, and 1 point for each example. Studies most often use

variants on this scheme (Jonassen, Reeves, Hong, Harvey & Peters, 1997), which are too

numerous to discuss within the scope of this review. Characteristics commonly evaluated

included significance, directionality, accuracy, centrality, number of links, misconceptions,

and structure. The alternative assessment scheme evaluates the student map against some

criterion, commonly expert knowledge or an expert constructed map. Qualitative and holistic

analysis is also applied in evaluating concept maps.

Analysis of Published Research

The literature related to concept mapping in mathematics is not extensive, and a

significant proportion of relevant papers was explored in the research process. Empirical

research was sought and 11 available studies were analysed for evidence of the effectiveness

of concept maps in both assessment and facilitation of learning. As it was determined that

some studies speak to both assessment and learning, the analysis section of this paper was not

divided into separate texts based on these themes.

Some common features were noted in the studies. Content and criterion validity

appear to be supported through the evaluation of maps by experts, generally professors or

teachers of mathematics, or professors of education. External validity is supported through the

placement of the studies in classroom situations unless noted otherwise. Table 1 provides a

summary of relevant characteristics of the 11 studies.


Table 1. Summary of relevant characteristics of studies.

Study n Student

Level

Corroborating

Measures

Interaction

Duration

Overall

Assessment

Type

51 Elementary interview &

problem solving

twice over

5 months

qualitative Hasemann and

Mansfield (1995)

23 Secondary traditional exam pre & post

instruction

quantitative

Barolos (2002) 48 Secondary traditional exam one time quantitative

Flanagan (2002) 20 Secondary hypermedia,

interview &

exam

one time holistic

Afamasaga-Fuata’i

(2004)

16 Post-

Secondary

vee diagram &

collaboration

semester quantitative

Bolte (1999) 108 Post-

Secondary

essay &

traditional exam

one time holistic

Zwaneveld (2000) 5 Post-

Secondary

unsupervised

exam

one time qualitative

Williams (1998) 28 Post-

Secondary

one time holistic

Roberts (1999) 19 Post-

Secondary

twice holistic

Mwakapenda

(2005)

17 Post-

Secondary

interviews one time quantitative

McGowen and Tall

(1999)

26 Post-

Secondary

traditional exam

& interview

3 maps

over a

semester

qualitative

Elementary Level Studies

Hasemann and Mansfield (1995) studied 25 grade 4 and 26 grade 6 volunteer students.

The students constructed concept maps on two separate occasions 5 months apart, by

arranging predetermined concepts on cards into groupings. The grade 4 maps were more

context oriented, whereas the grade 6 maps were more domain oriented, suggesting a

cognitive growth issue. No significant difference in mapping schemes was noted between

testing episodes despite apparent learning in class, suggesting that rote learning was taking


place, but not meaningful learning. Concept map assessments correlated with other

assessments, including problem solving exercises.

In this study the instrument is used solely for assessment. Concept maps appear to lack

face validity as they do not appear to measure a change apparent in the classroom. In defense

of this, the authors suggest that rote learning was emphasized in the classroom. Content

validity is not supported due to the lack of corroboration with classroom learning. Criterion

validity is supported by individual student correlation between map and problem solving

assessments. Reliability is supported through control of inter-rater variation. External validity

is supported by the reasonable number of subjects, although the voluntary nature of

participation raises concerns about the representative nature of the sample. This study offers

weak support for concept maps as assessment tools.

Secondary Level Studies

Hasemann and Mansfield (1995) also carried out a long term study of 23 grade 8/9

students, who constructed maps before and after instruction in a particular domain of

knowledge. Maps were scored for the number and correctness of propositions. A significant

increase in correct propositions and reduction in incorrect propositions was observed between

pre and post-test, indicating that learning had taken place. A delayed post-test 2 years later

found minor regression of propositional knowledge, suggesting that some “learning” did not

comprise long-term cognitive enhancements.

In this study, concept maps are not applied as learning tools. Criterion validity is

weakly supported through the time related correlation between testing and learning episodes,

but lacks support by corroboration with another instrument. Test-retest reliability is supported.


External validity is reasonably supported by the numbers of subjects. The evidence supports

the use of concept maps as an instrument for assessing student understanding.

Barolos (2002) studied 48 Grade 11 students in two classes with the same teacher,

comprising students bound for university and polytechnical destinations. Concepts relevant to

a particular mathematical domain were identified by students and experts. Students created

concept maps based on these concepts, and were then assessed using a conventional closed

test. The experts evaluated the tests and the maps using a variant of the Novak assessment

scheme in relation to an expert-created criterion map. Differences in test scores between the

two groups were not statistically significant, though the university group did score slightly

higher. In the concept maps the university-bound group scored statistically higher, using more

concepts and constructing more valid propositions. Insignificant (for the polytechnical

students) or weak (for the university students) correlations were found between the two

assessments.

This study does not offer evidence of concept maps as a learning tool. Criterion

validity is supported by the comparison with another instrument. Inter-rater reliability is

supported through good rater consensus, although the lack of a test and retest methodology

limits confidence in the reliability of the instrument. External validity is supported by the

relatively large number of subjects. Concept maps are not supported as assessment tools, as

there is little or no relationship correlation between the two types of tests.

Flanagan (2002) engaged in action research with 20 secondary school students who

used concept maps and Macromedia Flash in representing knowledge, allowing students to

represent mathematical problems in support of improved problem solving. Concept maps,

student journals and interviews were used in instructor evaluation. Instructor perspective


indicated that concept maps had a positive effect on student problem solving. Students

engaged in pre-tests and post-tests using a standardized mathematical aptitude test. The tests

were separated by a few months. Improvements were found in abstract reasoning, mechanical

reasoning, and space relations. Students saw the process as valuable and meaningful in terms

of learning, attributing enhanced success to the overall process.

While there is evidence of learning, it is difficult to assign a causal relationship in this

study as there is no clear connection between use of the instrument and student learning.

However, as the study comprises action research, there is value in the author’s perspective of

the effectiveness of concept maps in student learning. As there is no formal evidence

presented on the assessment of the maps themselves, it is not possible to draw conclusions on

the value of the instrument as an assessment tool.

Post-Secondary Level Studies

Afamasaga-Fuata’i (2004) studied 6 undergraduate students in a collaborative setting

where students constructed computer generated concept maps as well as vee diagrams over

the course of a semester, in an iterative peer-reviewed development cycle. The goal of

mapping was to develop and improve the students’ understanding of a particular mathematical

domain. Maps were assessed quantitatively against the instructor’s expert knowledge using a

modified version of the Novak assessment scheme, examining structural complexity, content

and proposition validity. Analysis indicated that student maps generally evolved from

procedural to conceptual, becoming more complex and more correct in terms of propositions.

In interviews students indicated increased confidence in understanding, suggesting that

constructing maps required a deep, meaningful understanding of the topic.


While an apparent change in student knowledge is measured, it is not clear that the

construction of concept maps facilitated learning as any number of external factors could have

influenced the outcome; in consequence internal validity is questionable. Criterion validity is

reasonably satisfied through peer review and comparisons to vee diagrams, however, the use

of the instrument itself to measure change limits confidence in internal validity. The lack of a

representative sample and the small number of subjects diminishes external reliability. This

study provides some support for concept maps as useful evaluation tools.

Bolte (1999) studied 108 pre-service teachers who constructed concept maps using a

list of words provided by the instructor, followed by the creation of an accompanying essay.

Maps were scored holistically by one rater, evaluating correctness against the instructor’s

expert knowledge. Evaluation of essays either corroborated misconceptions noted in concept

maps or indicated correct understanding. The combined concept map/essay assessment was

significantly correlated with traditional course assessment measures, especially the final

exam. Students felt that the approach helped to enhance mathematical knowledge. As well,

the author notes effectiveness of concept maps in identifying student misconceptions.

This study does not offer evidence of concept maps as a learning tool. Criterion

validity is established by the comparison of the instrument with traditional course evaluations.

It is weakened by the inability to discriminate between concept map and essay scores, and by

the inconsistent correlation between essay and concept map evaluations. Reliability is

supported through the use of equivalent forms of evaluation. External validity is supported by

the relatively large number of subjects. Overall this study provides moderate support for

concept maps as assessment tools.


Zwaneveld (2000) engaged 5 post-secondary students (ages 30 to 50) in creating non-

hierarchical concept maps as an on-going activity. The study assessed the structure and

content of the maps as well as student attitudes. Student attitudes supported the cognitive

effectiveness and thought-provoking nature of concept maps. A traditional, but unsupervised,

test was given after map completion, and it was found that students with more highly

structured and mathematically correct maps exhibited better performance.

This study provides evidence for a correlation between learning and the construction

of concept maps, however, internal validity is not satisfied as other possible factors have not

been excluded. Criterion validity is supported through comparison with another instrument,

although the unsupervised nature of the assessment raises concerns. External validity is not

well supported, as this activity was not part of the regular curriculum. Also the number of

subjects was small, hence the representative nature of the subjects is questionable. This study

offers some support for the tool as an effective assessment instrument.

Williams (1998) examined concept maps exploring the concept of function created by

28 volunteer post-secondary calculus students. These were compared to maps created by 8

professors of mathematics. Through qualitative holistic analysis of differences between

student and expert maps, differential understanding was observed. Student maps were more

variable, containing many trivial propositions, a lower proportion of conceptual content and a

larger proportion of procedural content.

As there is no evidence of changes in student cognitions, the study cannot be analysed

in terms of learning. Criterion validity is supported in this study by the involvement of 8

experts in creating the comparison maps, but reduced by the absence of alternate assessments.

External validity is supported by the number of subjects, but they may not comprise a


representative sample due to their voluntary nature. This study offers some support for the

effectiveness of the instrument in assessment.

Roberts (1999) engaged in action research involving 19 post-secondary students in a

statistics course, who voluntarily developed concept maps as an adjunct to the course. Maps

were scored using a scheme iteratively developed by the author during the course of the study,

which included many ideas from the Novak scheme. Scoring was performed by the author and

one other rater with high inter-rater reliability noted. Students with higher concept map scores

performed better on a practical statistical problem. A second set of maps were either not

completed or found to score lower. This was attributed to lack of student time and motivation,

as there was no course requirement for completion. Qualitative analysis by the author

suggested that the maps provided an effective way to identify student understanding and

misconception.

While it is possible that students enhanced their understanding of the knowledge

domain, there is no convincing connection between mapping and learning. The author

supports content validity through the iterative nature of the scoring scheme development.

Criterion validity is supported through correlation with other formal assessments. Inter-rater

reliability is supported through the use of experts with high inter-rater correlation. External

validity is poorly supported by the high subject mortality and only moderate subject numbers.

This study weakly supports the effectiveness of concept maps as an assessment instrument.

Mwakapenda (2005) studied 17 first year university mathematics students from

various streams, who constructed concept maps based on a list of terms. Maps were scored on

inclusiveness, correctness and linkages, and then corroborated with data from student

interviews. Students from the higher level mathematics streams appeared to have a more


integrated knowledge base, though the low numbers make a generalization questionable.

Students tended to link concepts contextually rather than conceptually, indicating more

dependence on rote learning than on meaningful learning.

As the study does not record changes in student cognitions, conclusions with regard to

student learning are inappropriate. Criterion validity is supported by the use of interviews as a

comparative assessment instrument. Confidence in the results is undermined as it is unclear

who performed the assessments. External validity is modestly supported by the moderate

subject numbers. This study suggests that concept maps are effective assessment instruments.

McGowen and Tall (1999) worked with 26 college students enrolled in Intermediate

Algebra, who constructed 3 successive maps on the same topic over a 16 week course. Maps

were analysed for changes in structure, elaboration, and placement of new materials in

successive maps. The least and most successful groups of students were identified through

formal evaluation. Students in the more successful group were found to elaborate on earlier

maps, adding new concepts to existing structures. In interviews, these students tended to

exhibit more conceptually based knowledge. Students in the less successful group tended to

re-make the maps each time, rather than elaborating on previous versions. In interviews, these

students exhibited more procedurally based knowledge, and tended to rely on rote learning.

This study does not attempt to ascribe learning to the use of concept maps, instead

viewing maps as assessment tools. Criterion validity is supported through the use of

interviews and correlation with traditional classroom assessments. External validity is

supported by the reasonable subject numbers. The study offers good support for the

effectiveness of concept maps as assessment tools. Confidence in the conclusions is

strengthened by the long-term use of concept maps.


Table 2. Summary of confidence indicators from studies.

Study Learning Attributed

to Concept Mapping

Validity: Criterion (C),

Internal (I), External (E)

Reliability

Estimate

questionable moderate (C), low (I),

good (E)

modest Hasemann and

Mansfield (1995)

moderate (C), moderate (I),

moderate (E)

moderate

Barolos (2002) moderate (C), low (I),

modest (E)

moderate

Flanagan (2002) no clear link

Afamasaga-Fuata’i

(2004)

no clear link moderate (C), modest (I),

modest (E)

low

Bolte (1999) modest (C), good (I),

good (E)

moderate

Zwaneveld (2000) no clear link modest (C), moderate (I),

low (E)

low

Williams (1998) moderate (C), good (I),

moderate (E)

modest

Roberts (1999) questionable moderate (C), moderate (I),

low (E)

modest

Mwakapenda

(2005)

moderate (C), low (I),

modest (E)

modest

McGowen and Tall

(1999)

good (C), moderate (I),

moderate (E)

moderate

Discussion

Are Concept Maps Effective Assessment Instruments?

If concept maps are to be considered effective assessment instruments, their reliability

and validity must be understood. As McClure, Sonak and Suen (1999) note, educational

assessments measure and provide a score for some characteristic of an individual. Further the

differences in scores between individuals must be attributed to differences in the individuals

where the effects of other contributing factors have been excluded or reduced.

While the results from McClure et al. (1999) suggest that evaluation using concept

maps in science is associated with reasonable validity, there are theoretical concerns. First, it

is not known whether constructivist models like Ausubel’s (1968) accurately represent


learning. If this is not the case, the validity of concept maps as assessment instruments is

questionable. Second, there are many perspectives on what concept maps represent. While

many authors like Novak & Gowin (1984) suggest that concept maps represent student

knowledge, others like McAleese (1998), and McGowen and Tall (1999) posit that it is

impossible to know whether they represent an individual’s actual mathematical knowledge.

Roth and Roychoudhury (1993) suggest that a concept map externalizes only part of an

individual’s cognitions, while McAleese (1998) takes the view that concept maps are not

representative of the cognitions, but rather show the result of engaging in knowledge

construction. McGowen and Tall (1999) question whether students have different ways of

knowing which, based on a constructivist philosophy, reduces support for the tool as a valid

assessment instrument.

The validity of the concept map instrument could be affected by errors from two

sources: the mapping process (for example, an overly complex task) and the evaluation

process (for example, an inexperienced evaluator) according to McClure et al. (1999). The

mapping process was not detailed in the reviewed studies, so only the evaluation process is

examined in this paper. Analysis suggests that the studies vary in validity (as summarized in

Table 2). Confidence in all studies is supported by content validity; criterion validity varies

between modest and good, internal validity varies between low and good, and external

validity varies between low and good. The variations noted in validity are in the main due to

the dissimilarity in methodology and approaches taken by the researchers. As each study

assessed the concept maps using a different assessment scheme, it is very difficult to make

generalizations on the validity of the instrument as an assessment tool. It is also not clear

whether quantitative, qualitative and holistic methods are comparable in terms of validity.


However, some patterns emerge from the studies. Generally, concept map assessment

correlates with other indicators of knowledge, such as examinations, vee diagrams, essays,

comparison to expert maps and interviews, supporting confidence in terms of criterion

validity. The number of studies reviewed is insufficient to draw conclusions with regard to

correlation between concept maps and other assessment instruments. In cases where

correlations were not present, researchers theorized that the assessments evaluated different

types of knowledge. Researchers state that concept maps are excellent instruments for

identifying student misconception (Bolte, 1999) and that maps evolve over time to more

closely reflect instructor knowledge, supporting their use through content validity. Studies

noted that maps evolved from procedural to conceptual (Afamasaga-Fuata’I, 2004) or showed

improvements in correct vs. incorrect propositions (Hasemann & Mansfield, 1995). This

provides support for validity as these changes would be expected to occur as student

cognitions developed. Some studies reflected an action research approach, in various degrees,

which supports the use of author perspective in validity. Researchers in general, for example

Roberts (1999), indicated confidence in the validity of the instrument.

The reliability of concept mapping may be viewed in two ways: reliability of the

mapping process and reliability of the map as an assessment instrument. Trochim (1993)

suggests the process of creating concept maps is in general reliable after analysis of 38

mapping projects, though none in education. The reliability of an assessment instrument may

be analysed in a number of ways. McClure et al. (1999) examined various scoring approaches

and found that the scoring method had significant impact on reliability. Further they noted

that error in reliability might be caused by variations in the student’s concept mapping ability,

variation in the content knowledge of the evaluator and consistency in the evaluation process.


Student mapping ability was not examined in the studies, and there is an underlying

assumption that all subjects were equally skilled. This assumption is questionable, as

variability will be present in any skill, for example essay writing. Although all studies

provided an introduction to concept mapping for students, Smith and Dwyer (1995) note that

mastery of concept mapping is time consuming. It is reasonable to assume that a large enough

sample would reduce the impact of this error. Sample sizes varied in the studies between 5

and 108, with the majority representing common class sizes of 20 to 30 students. While a

statistical analysis of sample size is beyond the scope of the current paper, it is anticipated that

the effect of variability in ability could be significant in the common sizes noted above.

Variability in evaluator content knowledge was addressed in most studies where there were

multiple raters present and does not appear to have been a factor in reducing the reliability,

although there is no way of evaluating the reliability of single raters. Reliability of the

evaluation method was examined by McClure et al. (1999), and reliability coefficients for

holistic, structural and relational methods were calculated. Relational methods were found to

be the most reliable, and the authors theorized that this was due to reduced cognitive load on

the evaluators. Relational scoring, in conjunction with a master map, was found to have

improved reliability, while using master maps with the other two methods reduced reliability.

In the current review paper, studies used a variety of scoring schemes, so it is difficult to draw

conclusions based on the studies reviewed.

Can Learning be Attributed to the Use of Concept Maps?

Evidence of learning would comprise some long term change in student knowledge

that could be attributed to the use of concept maps. Only 5 of the studies reviewed in this

paper examined changes in student cognition, limiting confidence in any conclusions that


could be drawn. No study offered a clear link between learning and the application of concept

maps. In consequence, there is insufficient empirical support for the effectiveness of concept

maps in facilitating student learning. However, if researcher and student perspective are to be

considered important, there is support for concept maps as facilitators of learning, based on

researcher holistic analysis, researcher interaction with students and student perspective.

Inconsistencies, Gaps and Limitations in Research

Validity and reliability are based on underlying assumptions which have not been

established as accurate. It is not clear whether there is a similarity between the external

representation and the learner’s internal representation (McGowen & Tall, 1999) or whether

concept mapping models learning processes in an individual. There is also limited

understanding of the correlation between concept maps and traditional assessments. Validity

could be better established through (a) research into what mathematical knowledge concept

maps represent, (b) a better definition of mathematical concepts (Huerta, Galán & Granell,

2004), (c) correlation of concept maps with traditional measures of knowledge in

mathematics, (d) research into the validity of various assessment approaches, (e) the use of

control groups, and (f) the use of larger and more representative samples. Reliability could be

established through (a) use of test and re-test procedure, (b) use of equivalent forms and (c)

improved student mastery of concept mapping. A clear link between the use of concept maps

and learning could be established by limiting external factors and using control groups.

Only 11 studies were reviewed in this analysis, each using a different assessment

scheme, limiting confidence in the conclusions that can be drawn with respect to assessment

methodology. Finally, of the 11 studies, 7 explored post-secondary subjects while only one


explored elementary aged subjects. In consequence, more studies would need to be reviewed

to draw valid conclusions correlating age and effectiveness.

Relationships Identified and Implications

Are concept maps valid and reliable instruments for assessing mathematical

knowledge? In general the 11 studies evidenced only moderate validity and reliability, failing

to provide good confidence in the results of the empirical research. Can improvements in

student learning be attributed to the use of concept maps? Only 5 studies evidenced changes

in cognition, comprising weak or questionable empirical evidence supporting learning through

the use of concept maps. Viewing the studies from an action research perspective, there is

support for concept maps in both assessment and learning. Further, the combined results of

the studies are suggestive evidence for the effectiveness in both assessment and learning.

The overarching research question for the current paper stated: Will a review of

literature provide supporting evidence that concept maps are effective tools in facilitating

learning and assessing student knowledge in mathematics education? Only weak to moderate

evidence was found supporting the effectiveness of concept mapping in mathematics

education. This implies that further research is required before the tool can be confidently

used in these educational settings.


References

Afamasaga-Fuata’i, K. (2004). Vee diagrams & concept caps as tools for learning new

mathematics topics. Retrieved March 25, 2005 from

http://cmc.ihmc.us/papers/cmc2004-271.pdf

Ausubel, D. (1968). Educational psychology: A cognitive view. New York: Holt, Rinehart and

Winston.

Barolos, G. (2002). Concept mapping as evaluation tool in mathematics. Paper presented at

the 2nd International Conference on the Teaching of Mathematics, Crete, Greece.

Retrieved May 21, 2005 from

http://www.math.uoc.gr/~ictm2/Proceedings/pap451.pdf

Baroody, A. J., & Bartels, B. H. (2000). Using concept maps to link mathematical ideas.

Mathematics Teaching in the Middle School, 5(9), 604-610.

Bolte, L. A. (1999). Using concept maps and interpretive essays for assessment in

Mathematics [Electronic version]. School Science & Mathematics, 99(1), 19-31.

Coffey, J., Carnot, M., Feltovich, P., Feltovich, J., Hoffman, R., Cañas, A., et al. (2003). A

Summary of Literature Pertaining to the Use of Concept Mapping Techniques and

Technologies for Education and Performance Support. Retrieved June 12, 2005 from

http://www.ihmc.us/users/acanas/Publications/ConceptMapLitReview/IHMC%20Liter

ature%20Review%20on%20Concept%20Mapping.pdf

Deese, J. (1965). The structure of associations in language and thought. Baltimore: Johns

Hopkins Press.


Educational Quality Assurance Organization. (2003). Grade 9 assessment of Mathematics

2002-2003: Report of provincial results. Retrieved May 18, 2005 from

http://www.eqao.com/pdf_e/03/03P032e.pdf

Flanagan, F. (2002). An educational enquiry into the use of concept mapping and multimedia

to enhance the understanding of mathematics (working paper). Retrieved March 25,

2005 from http://webpages.dcu.ie/~farrenm/fionntech.PDF

Freeman, L. A. & Jessup, L. M. (2004). The power and benefits of concept mapping:

Measuring use, usefulness, ease of use and satisfaction [Electronic version].

International Journal of Science Education, 26(2), 151-169.

Ginsburg, H. (1989). Children's arithmetic: How they learn it and how you teach it (2nd ed.).

Austin, Tx: PRO-ED.

Hasemann, K. & Mansfield, H. (1995). Concept mapping in research on mathematical

knowledge development: Background, methods, findings and conclusions.

Educational Studies in Mathematics, 29, 45-72.

Hiebert, J. & Carpenter, T. P. (1992). Learning and teaching with understanding. In D. A.

Grouws (Ed.), Handbook of research on Mathematics teaching and learning (pp. 65-

97). New York: Macmillan.

Horton, P. B., McConney, A. A., Gallo, M., Woods, A. L., Senn, G. J., & Hamelin, D. (1993).

An investigation of the effectiveness of concept mapping as an instructional tool.

Science Education, 77, 95-111.

Huerta, M., Galán, E. & Granell, R. (2004). Concept maps in mathematics education: A

possible framework for students’ assessment. Department de Didactica de la


Matematica, Universitat de Valencia. Retrieved March 26, 2005 from

http://www.icme-organisers.dk/tsg27/papers/06_Huerta_et_al_fullpaper.pdf

Jonassen, D. H., Reeves, T. C., Hong, N., Harvey, D., & Peters K. (1997). Concept mapping

as cognitive learning and assessment tools. Journal of Interactive Learning Research,

8(3-4), 289-308.

Leedy, P. D. & Ormrod, J. E. (2001). Practical research: Planning and design. Columbus,

OH: Merrill Prentice Hall.

McAleese, R. (1998). The knowledge arena as an extension to the concept map: Reflection in

action [Electronic version]. Interactive Learning Environments, 6(3), 251-273.

McClure, J., Sonak, B., & Suen, H. (1999). Concept map assessment of classroom learning:

Reliability, validity, and logistical practicality [Electronic version]. Journal of

Research in Science Teaching, 36(4), 475-492.

McGowen, M. & Tall, D. (1999). Concept maps & schematic diagrams as devices for

documenting the growth of mathematical knowledge. Proceedings of PME23, Haifa,

Israel, 3, 281–288. Retrieved March 25, 2005 from

http://www.warwick.ac.uk/staff/David.Tall/pdfs/dot1999f-mcgowen-pme.pdf

Mwakapenda, W. (2005). Examining understanding in Mathematics: A perspective from

concept mapping. In R. Vithal, J. Adler, & C. Keitel (Eds.) Researching Mathematics

Education in South Africa Perspectives, Practices and Possibilities. Retrieved March

26, 2005

http://www.hsrcpress.ac.za/user_uploads/tblPDF/2034_10_Researching_Mathematics

_Education~04022005100130AM.pdf


National Council of Teachers of Mathematics (2005). Standards for school Mathematics.

Retrieved June 2, 2005 from http://www.nctm.org/standards/standards.htm

Novak, J. D. (n.d.). The theory underlying concept maps and how to construct them. Retrieved

March 12, 2005 from http://cmap.coginst.uwf.edu/info/printer.html

Novak, J. D. (2003). The promise of new ideas and new technology for improving teaching

and learning [Electronic version]. Cell Biology Education, 2(Summer), 122–132.

Novak, J. D., & Gowin, D. (1984). Learning how to learn. New York: Cambridge University

Press.

Ontario Ministry of Education. (2004). Leading math success. Toronto, ON: Queen’s Printer.

Piaget, J. (1959). The language and the thought of the child. London: Routledge and Kegan

Paul Limited.

Roberts, L. (1999). Using concept maps to measure statistical understanding [Electronic

version]. International Journal of Mathematical Education in Science & Technology,

30(5), 707-718.

Roth, W., & Roychoudhury, A. (1993). The concept map as a tool for the collaborative

construction of knowledge: Microanalysis of high school Physics students. Journal of

Research in Science Teaching, 30(5), 503-534.

Skemp, R. (1987). The Psychology of Learning Mathematics. Hillsdale, NJ: Lawrence

Erlbaum Associates.

Smith, K. M., & Dwyer, F. M. (1995). The effect of concept mapping strategies in facilitating

student achievement. International Journal of Instructional Media, 22(1), 25-31.


Trochim, W. (1993). The reliability of concept mapping. Paper presented at the Annual

Conference of the American Evaluation Association, Dallas, TX. Retrieved June 23,

2005 from http://www.socialresearchmethods.net/research/Reliable/reliable.htm

Vygotsky, L., & Cole, M. (1978). Mind in society: The development of higher psychological

processes. Cambridge: Harvard University Press.

Weisstein, E. (2005). Graph. MathWorld. Retrieved March 22, 2005 from

http://mathworld.wolfram.com/Graph.html

Williams, C. G. (1998). Using concept maps to assess conceptual knowledge of function

[Electronic version]. Journal for Research in Mathematics Education, 29(4), 419-421.

Zwaneveld, B. (2000). Structuring mathematical knowledge and skills by means of

knowledge graphs [Electronic version]. International Journal of Mathematical

Education in Science and Technology, 31(3), 393-414.

Documents

The Effectiveness of Concept Maps in Assessment and Learning in Mathematics Education … · 2015-07-29 · Effectiveness of Concept Maps 2 Abstract The effectiveness of concept maps