View
7
Download
0
Category
Preview:
Citation preview
Effectiveness of Concept Maps 1
Running head: EFFECTIVENESS OF CONCEPT MAPS
The Effectiveness of Concept Maps in Assessment and
Learning in Mathematics Education
Gary Greer
Memorial University of Newfoundland
Cape Breton University
Effectiveness of Concept Maps 2
Abstract
The effectiveness of concept maps in mathematics education was examined through
review of relevant empirical research studies. Concept maps are identified by Novak as useful
tools in learning and assessment, comprising concepts connected by linking phrases that form
propositions relevant to a particular knowledge domain, organized in a hierarchical structure.
To examine claims of the effectiveness of concept maps in terms of assessment and learning
facilitation in mathematics education, analysis of validity and reliability was undertaken. The
studies evidenced only moderate empirical evidence for the effectiveness of concept maps in
assessment applications, and questionable empirical evidence that concept maps facilitate
learning. Qualitative and holistic researcher perspective was more positive, as was student
perspective. More research is required to allow educators to use these tools strategically and
confidently in mathematics educational contexts.
Effectiveness of Concept Maps 3
Contents
Abstract 2
Motivation 4
Research Questions and Hypothesis 5
Background 6
A Definition for Concept Maps 6
Theoretical Support for the use of Concept Maps 7
Assessing Concept Maps 8
Analysis of Published Research 9
Elementary Level Studies 10
Secondary Level Studies 11
Post-Secondary Level Studies 13
Discussion 18
Are Concept Maps Effective Assessment Instruments? 18
Can Learning be Attributed to the Use of Concept Maps? 21
Inconsistencies, Gaps and Limitations in Research 22
Relationships Identified and Implications 23
References 24
Figures and Tables
Figure 1. Conceptual framework. 5
Figure 2. Basic features of a concept map. 7
Table 1. Summary of relevant characteristics of studies. 10
Table 2. Summary of confidence indicators from studies. 18
Effectiveness of Concept Maps 4
The Effectiveness of Concept Maps in Assessment and
Learning in Mathematics Education
This study comprises an analysis of literature in mathematics education related to the
use of concept maps in the context of current educational theories and practice. Concept maps
are widely accepted as learning and assessment tools in science education (Coffey et al.,
2003). Although meta-analysis of 18 studies in science education by Horton et al. (1993)
indicates that the use of concept maps is generally associated with improved student
achievement and positive student attitudes, there is no assurance that the instrument will
prove similarly effective in mathematics education. In consequence, it is important to
establish the effectiveness of this instrument in mathematics education.
Motivation
Motivation for this study is based on timeliness and author interests. First, there are
increasing concerns voiced over poor student achievement in mathematics in Ontario (Ontario
Ministry of Education, 2004). There are many sources of evidence for this, including the
Educational Quality and Assurance Organization (2003) which notes widespread incidence of
low scores at the Grade 9 Applied level. Second, there have been significant changes in
educational perspectives on knowledge creation (Novak, 2003), conceptual understanding
(Mwakapenda, 2005), and the importance of both relationships and connections in
mathematics education (National Council of Teachers of Mathematics, 2005). Third, while
concept maps have been used for many years, the recent availability of powerful computer
software for constructing concept maps offers a number of advantages (Coffey et al, 2003).
Finally, the author is a teacher of secondary school mathematics and computer science with a
keen interest in supporting learning through the use of information technologies.
Effectiveness of Concept Maps 5
Research Questions
Although there are many perspectives on what comprises education, it can be argued
that two critical interactions are facilitating learning and assessing student knowledge, either
in preparing for learning or evaluating the effect of learning experiences. It is not known
whether concept maps will prove effective in supporting these two processes in mathematics
education, suggesting the research question: Will a review of literature provide supporting
evidence that concept maps are effective tools in facilitating learning and assessing student
knowledge in mathematics education?
An assessment will be considered to be an activity that provides some measurement of
student knowledge. Learning will be considered to comprise a long-term change in student
knowledge as measured by traditional assessments. Validity will be considered the ability of
an instrument to measure what it purports to measure, while reliability will be considered the
ability of an instrument to yield consistent results when the characteristic being measured has
not changed (Leedy & Ormrod, 2001). A conceptual framework has been developed to
represent the relationships and is shown in Figure 1.
Figure 1. Conceptual framework for the use of concept maps in educational settings.
Effectiveness of Concept Maps 6
Two specific questions will be used to analyse evidence of the effectiveness of
concept maps in mathematics education:
1. Are concept maps valid and reliable instruments for assessing mathematical
knowledge?
2. Can improvements in student learning be attributed to the use of concept maps?
Background
A Definition for Concept Maps
This discussion will utilize the definition developed by Novak and Gowin (1984). Not
all papers cited were as rigorous in their adherence to this definition. For example, a number
of studies did not require a hierarchical structure. Graphs are mathematical representations
containing a series of objects variously called nodes, vertices or points, with connections
variously called lines, arcs or edges (Weisstein, 2005). A concept map is a type of graph
where the nodes represent a “concept” and the labeled lines describe the relationship between
two concepts (Novak & Gowin), a concept being defined as “a regularity in events or objects
designated by some label”. (Novak & Gowin, 1984, p. 4) Further, Novak and Gowin (1984,
Chapter 2) describe mapping as an activity where the learner must engage to clarify meanings
by identifying important concepts, relationships, and structure within a chosen domain of
knowledge. The basic features of a concept map are diagrammed in Figure 2.
Unfortunately, in educational literature, the term concept map is used in a more
general sense to describe a number of different types of graphical representations of
knowledge, as well as in a specific manner to refer to the instrument developed by Novak and
Gowin (1984). However, the instrument developed by Novak and Gowin contains a number
of defining features as noted by Coffey et al, (2003). First, Novakian concept maps are
Effectiveness of Concept Maps 7
Figure 2. Basic features of a concept map.
grounded in Ausubel’s (1968) assimilation theory, being designed to support the integration
of new knowledge into existing cognitive structures. Second, they comprise a hierarchical
structure with the most general concepts located at the top and the most specific concepts
located at the bottom of the representation. Third, links between two concepts are labeled,
forming propositions which are “statements about some object or event in the universe, either
naturally occurring or constructed,” (Novak, n.d., p. 1) and are considered units of meaning.
Theoretical Support for the use of Concept Maps
Concept maps were developed by Novak and Gowin (1984) through research into
learning based on Ausubel’s (1968) work into the acquisition of knowledge by children.
Novak and Gowin proposed that concept maps encourage meaningful learning through the
identification and understanding of relationships between concepts, rather than through rote
learning. Ausubel proposed that learning takes place by a process called assimilation,
whereby new concepts and propositions are incorporated into the learner’s existing
knowledge frameworks, a process heavily mediated by language, social interaction and
interaction with the environment and/or objects (Coffey et al., 2003). Skemp (1987) supports
Effectiveness of Concept Maps 8
this, arguing that mathematical knowledge is organized into hierarchical conceptual structures
called schemas, and that mathematical learning may be modeled in terms of the assimilation.
Freeman and Jessup (2004) suggest that the use of concept maps in learning is also supported
by associationist theory (Deese, 1965) which is similar to assimilation theory except that the
former has no requirement for a hierarchical cognitive structure.
Support is also provided by Ginsburg (1989) who suggests that understanding may be
viewed as a connection between two pieces of information. Further, Hiebert and Carpenter (as
cited in Baroody & Bartels, 2000, p. 604) suggest that the degree of understanding may be
assessed by the number, strength and accuracy of the connections. These create a perspective
whereby concept maps may be used to measure the individual’s degree of understanding, and
provide theoretical support for assessment of understanding using concept maps.
Novak (n.d.) views concept maps as supporting learners by enhancing their ability to
work with many concepts at one time, a characteristic also noted by McAleese (1998). Novak
and Gowin (1984) and McAleese also suggest that concept maps assist students in
metacognitive processes, helping them learn to learn. The effectiveness of concept mapping is
supported by constructivist learning models like those proposed by Piaget (1959) and
Vygotsky (1978) who theorized that knowledge is actively constructed by the learner, and is
further supported by the zone of proximal development learning model (Vygotsky, 1978).
Assessing Concept Maps
Roberts (1999) suggests that there are two basic quantitative schemes for concept map
assessment: those based on the Novak and Gowin (1984) assessment rubric and those based
on assessment against a criterion. The classic Novakian assessment scheme awards 1 point for
valid propositions, 5 points for valid hierarchies, 10 points for valid crosslinks showing
Effectiveness of Concept Maps 9
synthesis and 2 for less significant ones, and 1 point for each example. Studies most often use
variants on this scheme (Jonassen, Reeves, Hong, Harvey & Peters, 1997), which are too
numerous to discuss within the scope of this review. Characteristics commonly evaluated
included significance, directionality, accuracy, centrality, number of links, misconceptions,
and structure. The alternative assessment scheme evaluates the student map against some
criterion, commonly expert knowledge or an expert constructed map. Qualitative and holistic
analysis is also applied in evaluating concept maps.
Analysis of Published Research
The literature related to concept mapping in mathematics is not extensive, and a
significant proportion of relevant papers was explored in the research process. Empirical
research was sought and 11 available studies were analysed for evidence of the effectiveness
of concept maps in both assessment and facilitation of learning. As it was determined that
some studies speak to both assessment and learning, the analysis section of this paper was not
divided into separate texts based on these themes.
Some common features were noted in the studies. Content and criterion validity
appear to be supported through the evaluation of maps by experts, generally professors or
teachers of mathematics, or professors of education. External validity is supported through the
placement of the studies in classroom situations unless noted otherwise. Table 1 provides a
summary of relevant characteristics of the 11 studies.
Effectiveness of Concept Maps 10
Table 1. Summary of relevant characteristics of studies.
Study n Student
Level
Corroborating
Measures
Interaction
Duration
Overall
Assessment
Type
51 Elementary interview &
problem solving
twice over
5 months
qualitative Hasemann and
Mansfield (1995)
23 Secondary traditional exam pre & post
instruction
quantitative
Barolos (2002) 48 Secondary traditional exam one time quantitative
Flanagan (2002) 20 Secondary hypermedia,
interview &
exam
one time holistic
Afamasaga-Fuata’i
(2004)
16 Post-
Secondary
vee diagram &
collaboration
semester quantitative
Bolte (1999) 108 Post-
Secondary
essay &
traditional exam
one time holistic
Zwaneveld (2000) 5 Post-
Secondary
unsupervised
exam
one time qualitative
Williams (1998) 28 Post-
Secondary
one time holistic
Roberts (1999) 19 Post-
Secondary
twice holistic
Mwakapenda
(2005)
17 Post-
Secondary
interviews one time quantitative
McGowen and Tall
(1999)
26 Post-
Secondary
traditional exam
& interview
3 maps
over a
semester
qualitative
Elementary Level Studies
Hasemann and Mansfield (1995) studied 25 grade 4 and 26 grade 6 volunteer students.
The students constructed concept maps on two separate occasions 5 months apart, by
arranging predetermined concepts on cards into groupings. The grade 4 maps were more
context oriented, whereas the grade 6 maps were more domain oriented, suggesting a
cognitive growth issue. No significant difference in mapping schemes was noted between
testing episodes despite apparent learning in class, suggesting that rote learning was taking
Effectiveness of Concept Maps 11
place, but not meaningful learning. Concept map assessments correlated with other
assessments, including problem solving exercises.
In this study the instrument is used solely for assessment. Concept maps appear to lack
face validity as they do not appear to measure a change apparent in the classroom. In defense
of this, the authors suggest that rote learning was emphasized in the classroom. Content
validity is not supported due to the lack of corroboration with classroom learning. Criterion
validity is supported by individual student correlation between map and problem solving
assessments. Reliability is supported through control of inter-rater variation. External validity
is supported by the reasonable number of subjects, although the voluntary nature of
participation raises concerns about the representative nature of the sample. This study offers
weak support for concept maps as assessment tools.
Secondary Level Studies
Hasemann and Mansfield (1995) also carried out a long term study of 23 grade 8/9
students, who constructed maps before and after instruction in a particular domain of
knowledge. Maps were scored for the number and correctness of propositions. A significant
increase in correct propositions and reduction in incorrect propositions was observed between
pre and post-test, indicating that learning had taken place. A delayed post-test 2 years later
found minor regression of propositional knowledge, suggesting that some “learning” did not
comprise long-term cognitive enhancements.
In this study, concept maps are not applied as learning tools. Criterion validity is
weakly supported through the time related correlation between testing and learning episodes,
but lacks support by corroboration with another instrument. Test-retest reliability is supported.
Effectiveness of Concept Maps 12
External validity is reasonably supported by the numbers of subjects. The evidence supports
the use of concept maps as an instrument for assessing student understanding.
Barolos (2002) studied 48 Grade 11 students in two classes with the same teacher,
comprising students bound for university and polytechnical destinations. Concepts relevant to
a particular mathematical domain were identified by students and experts. Students created
concept maps based on these concepts, and were then assessed using a conventional closed
test. The experts evaluated the tests and the maps using a variant of the Novak assessment
scheme in relation to an expert-created criterion map. Differences in test scores between the
two groups were not statistically significant, though the university group did score slightly
higher. In the concept maps the university-bound group scored statistically higher, using more
concepts and constructing more valid propositions. Insignificant (for the polytechnical
students) or weak (for the university students) correlations were found between the two
assessments.
This study does not offer evidence of concept maps as a learning tool. Criterion
validity is supported by the comparison with another instrument. Inter-rater reliability is
supported through good rater consensus, although the lack of a test and retest methodology
limits confidence in the reliability of the instrument. External validity is supported by the
relatively large number of subjects. Concept maps are not supported as assessment tools, as
there is little or no relationship correlation between the two types of tests.
Flanagan (2002) engaged in action research with 20 secondary school students who
used concept maps and Macromedia Flash in representing knowledge, allowing students to
represent mathematical problems in support of improved problem solving. Concept maps,
student journals and interviews were used in instructor evaluation. Instructor perspective
Effectiveness of Concept Maps 13
indicated that concept maps had a positive effect on student problem solving. Students
engaged in pre-tests and post-tests using a standardized mathematical aptitude test. The tests
were separated by a few months. Improvements were found in abstract reasoning, mechanical
reasoning, and space relations. Students saw the process as valuable and meaningful in terms
of learning, attributing enhanced success to the overall process.
While there is evidence of learning, it is difficult to assign a causal relationship in this
study as there is no clear connection between use of the instrument and student learning.
However, as the study comprises action research, there is value in the author’s perspective of
the effectiveness of concept maps in student learning. As there is no formal evidence
presented on the assessment of the maps themselves, it is not possible to draw conclusions on
the value of the instrument as an assessment tool.
Post-Secondary Level Studies
Afamasaga-Fuata’i (2004) studied 6 undergraduate students in a collaborative setting
where students constructed computer generated concept maps as well as vee diagrams over
the course of a semester, in an iterative peer-reviewed development cycle. The goal of
mapping was to develop and improve the students’ understanding of a particular mathematical
domain. Maps were assessed quantitatively against the instructor’s expert knowledge using a
modified version of the Novak assessment scheme, examining structural complexity, content
and proposition validity. Analysis indicated that student maps generally evolved from
procedural to conceptual, becoming more complex and more correct in terms of propositions.
In interviews students indicated increased confidence in understanding, suggesting that
constructing maps required a deep, meaningful understanding of the topic.
Effectiveness of Concept Maps 14
While an apparent change in student knowledge is measured, it is not clear that the
construction of concept maps facilitated learning as any number of external factors could have
influenced the outcome; in consequence internal validity is questionable. Criterion validity is
reasonably satisfied through peer review and comparisons to vee diagrams, however, the use
of the instrument itself to measure change limits confidence in internal validity. The lack of a
representative sample and the small number of subjects diminishes external reliability. This
study provides some support for concept maps as useful evaluation tools.
Bolte (1999) studied 108 pre-service teachers who constructed concept maps using a
list of words provided by the instructor, followed by the creation of an accompanying essay.
Maps were scored holistically by one rater, evaluating correctness against the instructor’s
expert knowledge. Evaluation of essays either corroborated misconceptions noted in concept
maps or indicated correct understanding. The combined concept map/essay assessment was
significantly correlated with traditional course assessment measures, especially the final
exam. Students felt that the approach helped to enhance mathematical knowledge. As well,
the author notes effectiveness of concept maps in identifying student misconceptions.
This study does not offer evidence of concept maps as a learning tool. Criterion
validity is established by the comparison of the instrument with traditional course evaluations.
It is weakened by the inability to discriminate between concept map and essay scores, and by
the inconsistent correlation between essay and concept map evaluations. Reliability is
supported through the use of equivalent forms of evaluation. External validity is supported by
the relatively large number of subjects. Overall this study provides moderate support for
concept maps as assessment tools.
Effectiveness of Concept Maps 15
Zwaneveld (2000) engaged 5 post-secondary students (ages 30 to 50) in creating non-
hierarchical concept maps as an on-going activity. The study assessed the structure and
content of the maps as well as student attitudes. Student attitudes supported the cognitive
effectiveness and thought-provoking nature of concept maps. A traditional, but unsupervised,
test was given after map completion, and it was found that students with more highly
structured and mathematically correct maps exhibited better performance.
This study provides evidence for a correlation between learning and the construction
of concept maps, however, internal validity is not satisfied as other possible factors have not
been excluded. Criterion validity is supported through comparison with another instrument,
although the unsupervised nature of the assessment raises concerns. External validity is not
well supported, as this activity was not part of the regular curriculum. Also the number of
subjects was small, hence the representative nature of the subjects is questionable. This study
offers some support for the tool as an effective assessment instrument.
Williams (1998) examined concept maps exploring the concept of function created by
28 volunteer post-secondary calculus students. These were compared to maps created by 8
professors of mathematics. Through qualitative holistic analysis of differences between
student and expert maps, differential understanding was observed. Student maps were more
variable, containing many trivial propositions, a lower proportion of conceptual content and a
larger proportion of procedural content.
As there is no evidence of changes in student cognitions, the study cannot be analysed
in terms of learning. Criterion validity is supported in this study by the involvement of 8
experts in creating the comparison maps, but reduced by the absence of alternate assessments.
External validity is supported by the number of subjects, but they may not comprise a
Effectiveness of Concept Maps 16
representative sample due to their voluntary nature. This study offers some support for the
effectiveness of the instrument in assessment.
Roberts (1999) engaged in action research involving 19 post-secondary students in a
statistics course, who voluntarily developed concept maps as an adjunct to the course. Maps
were scored using a scheme iteratively developed by the author during the course of the study,
which included many ideas from the Novak scheme. Scoring was performed by the author and
one other rater with high inter-rater reliability noted. Students with higher concept map scores
performed better on a practical statistical problem. A second set of maps were either not
completed or found to score lower. This was attributed to lack of student time and motivation,
as there was no course requirement for completion. Qualitative analysis by the author
suggested that the maps provided an effective way to identify student understanding and
misconception.
While it is possible that students enhanced their understanding of the knowledge
domain, there is no convincing connection between mapping and learning. The author
supports content validity through the iterative nature of the scoring scheme development.
Criterion validity is supported through correlation with other formal assessments. Inter-rater
reliability is supported through the use of experts with high inter-rater correlation. External
validity is poorly supported by the high subject mortality and only moderate subject numbers.
This study weakly supports the effectiveness of concept maps as an assessment instrument.
Mwakapenda (2005) studied 17 first year university mathematics students from
various streams, who constructed concept maps based on a list of terms. Maps were scored on
inclusiveness, correctness and linkages, and then corroborated with data from student
interviews. Students from the higher level mathematics streams appeared to have a more
Effectiveness of Concept Maps 17
integrated knowledge base, though the low numbers make a generalization questionable.
Students tended to link concepts contextually rather than conceptually, indicating more
dependence on rote learning than on meaningful learning.
As the study does not record changes in student cognitions, conclusions with regard to
student learning are inappropriate. Criterion validity is supported by the use of interviews as a
comparative assessment instrument. Confidence in the results is undermined as it is unclear
who performed the assessments. External validity is modestly supported by the moderate
subject numbers. This study suggests that concept maps are effective assessment instruments.
McGowen and Tall (1999) worked with 26 college students enrolled in Intermediate
Algebra, who constructed 3 successive maps on the same topic over a 16 week course. Maps
were analysed for changes in structure, elaboration, and placement of new materials in
successive maps. The least and most successful groups of students were identified through
formal evaluation. Students in the more successful group were found to elaborate on earlier
maps, adding new concepts to existing structures. In interviews, these students tended to
exhibit more conceptually based knowledge. Students in the less successful group tended to
re-make the maps each time, rather than elaborating on previous versions. In interviews, these
students exhibited more procedurally based knowledge, and tended to rely on rote learning.
This study does not attempt to ascribe learning to the use of concept maps, instead
viewing maps as assessment tools. Criterion validity is supported through the use of
interviews and correlation with traditional classroom assessments. External validity is
supported by the reasonable subject numbers. The study offers good support for the
effectiveness of concept maps as assessment tools. Confidence in the conclusions is
strengthened by the long-term use of concept maps.
Effectiveness of Concept Maps 18
Table 2. Summary of confidence indicators from studies.
Study Learning Attributed
to Concept Mapping
Validity: Criterion (C),
Internal (I), External (E)
Reliability
Estimate
questionable moderate (C), low (I),
good (E)
modest Hasemann and
Mansfield (1995)
moderate (C), moderate (I),
moderate (E)
moderate
Barolos (2002) moderate (C), low (I),
modest (E)
moderate
Flanagan (2002) no clear link
Afamasaga-Fuata’i
(2004)
no clear link moderate (C), modest (I),
modest (E)
low
Bolte (1999) modest (C), good (I),
good (E)
moderate
Zwaneveld (2000) no clear link modest (C), moderate (I),
low (E)
low
Williams (1998) moderate (C), good (I),
moderate (E)
modest
Roberts (1999) questionable moderate (C), moderate (I),
low (E)
modest
Mwakapenda
(2005)
moderate (C), low (I),
modest (E)
modest
McGowen and Tall
(1999)
good (C), moderate (I),
moderate (E)
moderate
Discussion
Are Concept Maps Effective Assessment Instruments?
If concept maps are to be considered effective assessment instruments, their reliability
and validity must be understood. As McClure, Sonak and Suen (1999) note, educational
assessments measure and provide a score for some characteristic of an individual. Further the
differences in scores between individuals must be attributed to differences in the individuals
where the effects of other contributing factors have been excluded or reduced.
While the results from McClure et al. (1999) suggest that evaluation using concept
maps in science is associated with reasonable validity, there are theoretical concerns. First, it
is not known whether constructivist models like Ausubel’s (1968) accurately represent
Effectiveness of Concept Maps 19
learning. If this is not the case, the validity of concept maps as assessment instruments is
questionable. Second, there are many perspectives on what concept maps represent. While
many authors like Novak & Gowin (1984) suggest that concept maps represent student
knowledge, others like McAleese (1998), and McGowen and Tall (1999) posit that it is
impossible to know whether they represent an individual’s actual mathematical knowledge.
Roth and Roychoudhury (1993) suggest that a concept map externalizes only part of an
individual’s cognitions, while McAleese (1998) takes the view that concept maps are not
representative of the cognitions, but rather show the result of engaging in knowledge
construction. McGowen and Tall (1999) question whether students have different ways of
knowing which, based on a constructivist philosophy, reduces support for the tool as a valid
assessment instrument.
The validity of the concept map instrument could be affected by errors from two
sources: the mapping process (for example, an overly complex task) and the evaluation
process (for example, an inexperienced evaluator) according to McClure et al. (1999). The
mapping process was not detailed in the reviewed studies, so only the evaluation process is
examined in this paper. Analysis suggests that the studies vary in validity (as summarized in
Table 2). Confidence in all studies is supported by content validity; criterion validity varies
between modest and good, internal validity varies between low and good, and external
validity varies between low and good. The variations noted in validity are in the main due to
the dissimilarity in methodology and approaches taken by the researchers. As each study
assessed the concept maps using a different assessment scheme, it is very difficult to make
generalizations on the validity of the instrument as an assessment tool. It is also not clear
whether quantitative, qualitative and holistic methods are comparable in terms of validity.
Effectiveness of Concept Maps 20
However, some patterns emerge from the studies. Generally, concept map assessment
correlates with other indicators of knowledge, such as examinations, vee diagrams, essays,
comparison to expert maps and interviews, supporting confidence in terms of criterion
validity. The number of studies reviewed is insufficient to draw conclusions with regard to
correlation between concept maps and other assessment instruments. In cases where
correlations were not present, researchers theorized that the assessments evaluated different
types of knowledge. Researchers state that concept maps are excellent instruments for
identifying student misconception (Bolte, 1999) and that maps evolve over time to more
closely reflect instructor knowledge, supporting their use through content validity. Studies
noted that maps evolved from procedural to conceptual (Afamasaga-Fuata’I, 2004) or showed
improvements in correct vs. incorrect propositions (Hasemann & Mansfield, 1995). This
provides support for validity as these changes would be expected to occur as student
cognitions developed. Some studies reflected an action research approach, in various degrees,
which supports the use of author perspective in validity. Researchers in general, for example
Roberts (1999), indicated confidence in the validity of the instrument.
The reliability of concept mapping may be viewed in two ways: reliability of the
mapping process and reliability of the map as an assessment instrument. Trochim (1993)
suggests the process of creating concept maps is in general reliable after analysis of 38
mapping projects, though none in education. The reliability of an assessment instrument may
be analysed in a number of ways. McClure et al. (1999) examined various scoring approaches
and found that the scoring method had significant impact on reliability. Further they noted
that error in reliability might be caused by variations in the student’s concept mapping ability,
variation in the content knowledge of the evaluator and consistency in the evaluation process.
Effectiveness of Concept Maps 21
Student mapping ability was not examined in the studies, and there is an underlying
assumption that all subjects were equally skilled. This assumption is questionable, as
variability will be present in any skill, for example essay writing. Although all studies
provided an introduction to concept mapping for students, Smith and Dwyer (1995) note that
mastery of concept mapping is time consuming. It is reasonable to assume that a large enough
sample would reduce the impact of this error. Sample sizes varied in the studies between 5
and 108, with the majority representing common class sizes of 20 to 30 students. While a
statistical analysis of sample size is beyond the scope of the current paper, it is anticipated that
the effect of variability in ability could be significant in the common sizes noted above.
Variability in evaluator content knowledge was addressed in most studies where there were
multiple raters present and does not appear to have been a factor in reducing the reliability,
although there is no way of evaluating the reliability of single raters. Reliability of the
evaluation method was examined by McClure et al. (1999), and reliability coefficients for
holistic, structural and relational methods were calculated. Relational methods were found to
be the most reliable, and the authors theorized that this was due to reduced cognitive load on
the evaluators. Relational scoring, in conjunction with a master map, was found to have
improved reliability, while using master maps with the other two methods reduced reliability.
In the current review paper, studies used a variety of scoring schemes, so it is difficult to draw
conclusions based on the studies reviewed.
Can Learning be Attributed to the Use of Concept Maps?
Evidence of learning would comprise some long term change in student knowledge
that could be attributed to the use of concept maps. Only 5 of the studies reviewed in this
paper examined changes in student cognition, limiting confidence in any conclusions that
Effectiveness of Concept Maps 22
could be drawn. No study offered a clear link between learning and the application of concept
maps. In consequence, there is insufficient empirical support for the effectiveness of concept
maps in facilitating student learning. However, if researcher and student perspective are to be
considered important, there is support for concept maps as facilitators of learning, based on
researcher holistic analysis, researcher interaction with students and student perspective.
Inconsistencies, Gaps and Limitations in Research
Validity and reliability are based on underlying assumptions which have not been
established as accurate. It is not clear whether there is a similarity between the external
representation and the learner’s internal representation (McGowen & Tall, 1999) or whether
concept mapping models learning processes in an individual. There is also limited
understanding of the correlation between concept maps and traditional assessments. Validity
could be better established through (a) research into what mathematical knowledge concept
maps represent, (b) a better definition of mathematical concepts (Huerta, Galán & Granell,
2004), (c) correlation of concept maps with traditional measures of knowledge in
mathematics, (d) research into the validity of various assessment approaches, (e) the use of
control groups, and (f) the use of larger and more representative samples. Reliability could be
established through (a) use of test and re-test procedure, (b) use of equivalent forms and (c)
improved student mastery of concept mapping. A clear link between the use of concept maps
and learning could be established by limiting external factors and using control groups.
Only 11 studies were reviewed in this analysis, each using a different assessment
scheme, limiting confidence in the conclusions that can be drawn with respect to assessment
methodology. Finally, of the 11 studies, 7 explored post-secondary subjects while only one
Effectiveness of Concept Maps 23
explored elementary aged subjects. In consequence, more studies would need to be reviewed
to draw valid conclusions correlating age and effectiveness.
Relationships Identified and Implications
Are concept maps valid and reliable instruments for assessing mathematical
knowledge? In general the 11 studies evidenced only moderate validity and reliability, failing
to provide good confidence in the results of the empirical research. Can improvements in
student learning be attributed to the use of concept maps? Only 5 studies evidenced changes
in cognition, comprising weak or questionable empirical evidence supporting learning through
the use of concept maps. Viewing the studies from an action research perspective, there is
support for concept maps in both assessment and learning. Further, the combined results of
the studies are suggestive evidence for the effectiveness in both assessment and learning.
The overarching research question for the current paper stated: Will a review of
literature provide supporting evidence that concept maps are effective tools in facilitating
learning and assessing student knowledge in mathematics education? Only weak to moderate
evidence was found supporting the effectiveness of concept mapping in mathematics
education. This implies that further research is required before the tool can be confidently
used in these educational settings.
Effectiveness of Concept Maps 24
References
Afamasaga-Fuata’i, K. (2004). Vee diagrams & concept caps as tools for learning new
mathematics topics. Retrieved March 25, 2005 from
http://cmc.ihmc.us/papers/cmc2004-271.pdf
Ausubel, D. (1968). Educational psychology: A cognitive view. New York: Holt, Rinehart and
Winston.
Barolos, G. (2002). Concept mapping as evaluation tool in mathematics. Paper presented at
the 2nd International Conference on the Teaching of Mathematics, Crete, Greece.
Retrieved May 21, 2005 from
http://www.math.uoc.gr/~ictm2/Proceedings/pap451.pdf
Baroody, A. J., & Bartels, B. H. (2000). Using concept maps to link mathematical ideas.
Mathematics Teaching in the Middle School, 5(9), 604-610.
Bolte, L. A. (1999). Using concept maps and interpretive essays for assessment in
Mathematics [Electronic version]. School Science & Mathematics, 99(1), 19-31.
Coffey, J., Carnot, M., Feltovich, P., Feltovich, J., Hoffman, R., Cañas, A., et al. (2003). A
Summary of Literature Pertaining to the Use of Concept Mapping Techniques and
Technologies for Education and Performance Support. Retrieved June 12, 2005 from
http://www.ihmc.us/users/acanas/Publications/ConceptMapLitReview/IHMC%20Liter
ature%20Review%20on%20Concept%20Mapping.pdf
Deese, J. (1965). The structure of associations in language and thought. Baltimore: Johns
Hopkins Press.
Effectiveness of Concept Maps 25
Educational Quality Assurance Organization. (2003). Grade 9 assessment of Mathematics
2002-2003: Report of provincial results. Retrieved May 18, 2005 from
http://www.eqao.com/pdf_e/03/03P032e.pdf
Flanagan, F. (2002). An educational enquiry into the use of concept mapping and multimedia
to enhance the understanding of mathematics (working paper). Retrieved March 25,
2005 from http://webpages.dcu.ie/~farrenm/fionntech.PDF
Freeman, L. A. & Jessup, L. M. (2004). The power and benefits of concept mapping:
Measuring use, usefulness, ease of use and satisfaction [Electronic version].
International Journal of Science Education, 26(2), 151-169.
Ginsburg, H. (1989). Children's arithmetic: How they learn it and how you teach it (2nd ed.).
Austin, Tx: PRO-ED.
Hasemann, K. & Mansfield, H. (1995). Concept mapping in research on mathematical
knowledge development: Background, methods, findings and conclusions.
Educational Studies in Mathematics, 29, 45-72.
Hiebert, J. & Carpenter, T. P. (1992). Learning and teaching with understanding. In D. A.
Grouws (Ed.), Handbook of research on Mathematics teaching and learning (pp. 65-
97). New York: Macmillan.
Horton, P. B., McConney, A. A., Gallo, M., Woods, A. L., Senn, G. J., & Hamelin, D. (1993).
An investigation of the effectiveness of concept mapping as an instructional tool.
Science Education, 77, 95-111.
Huerta, M., Galán, E. & Granell, R. (2004). Concept maps in mathematics education: A
possible framework for students’ assessment. Department de Didactica de la
Effectiveness of Concept Maps 26
Matematica, Universitat de Valencia. Retrieved March 26, 2005 from
http://www.icme-organisers.dk/tsg27/papers/06_Huerta_et_al_fullpaper.pdf
Jonassen, D. H., Reeves, T. C., Hong, N., Harvey, D., & Peters K. (1997). Concept mapping
as cognitive learning and assessment tools. Journal of Interactive Learning Research,
8(3-4), 289-308.
Leedy, P. D. & Ormrod, J. E. (2001). Practical research: Planning and design. Columbus,
OH: Merrill Prentice Hall.
McAleese, R. (1998). The knowledge arena as an extension to the concept map: Reflection in
action [Electronic version]. Interactive Learning Environments, 6(3), 251-273.
McClure, J., Sonak, B., & Suen, H. (1999). Concept map assessment of classroom learning:
Reliability, validity, and logistical practicality [Electronic version]. Journal of
Research in Science Teaching, 36(4), 475-492.
McGowen, M. & Tall, D. (1999). Concept maps & schematic diagrams as devices for
documenting the growth of mathematical knowledge. Proceedings of PME23, Haifa,
Israel, 3, 281–288. Retrieved March 25, 2005 from
http://www.warwick.ac.uk/staff/David.Tall/pdfs/dot1999f-mcgowen-pme.pdf
Mwakapenda, W. (2005). Examining understanding in Mathematics: A perspective from
concept mapping. In R. Vithal, J. Adler, & C. Keitel (Eds.) Researching Mathematics
Education in South Africa Perspectives, Practices and Possibilities. Retrieved March
26, 2005
http://www.hsrcpress.ac.za/user_uploads/tblPDF/2034_10_Researching_Mathematics
_Education~04022005100130AM.pdf
Effectiveness of Concept Maps 27
National Council of Teachers of Mathematics (2005). Standards for school Mathematics.
Retrieved June 2, 2005 from http://www.nctm.org/standards/standards.htm
Novak, J. D. (n.d.). The theory underlying concept maps and how to construct them. Retrieved
March 12, 2005 from http://cmap.coginst.uwf.edu/info/printer.html
Novak, J. D. (2003). The promise of new ideas and new technology for improving teaching
and learning [Electronic version]. Cell Biology Education, 2(Summer), 122–132.
Novak, J. D., & Gowin, D. (1984). Learning how to learn. New York: Cambridge University
Press.
Ontario Ministry of Education. (2004). Leading math success. Toronto, ON: Queen’s Printer.
Piaget, J. (1959). The language and the thought of the child. London: Routledge and Kegan
Paul Limited.
Roberts, L. (1999). Using concept maps to measure statistical understanding [Electronic
version]. International Journal of Mathematical Education in Science & Technology,
30(5), 707-718.
Roth, W., & Roychoudhury, A. (1993). The concept map as a tool for the collaborative
construction of knowledge: Microanalysis of high school Physics students. Journal of
Research in Science Teaching, 30(5), 503-534.
Skemp, R. (1987). The Psychology of Learning Mathematics. Hillsdale, NJ: Lawrence
Erlbaum Associates.
Smith, K. M., & Dwyer, F. M. (1995). The effect of concept mapping strategies in facilitating
student achievement. International Journal of Instructional Media, 22(1), 25-31.
Effectiveness of Concept Maps 28
Trochim, W. (1993). The reliability of concept mapping. Paper presented at the Annual
Conference of the American Evaluation Association, Dallas, TX. Retrieved June 23,
2005 from http://www.socialresearchmethods.net/research/Reliable/reliable.htm
Vygotsky, L., & Cole, M. (1978). Mind in society: The development of higher psychological
processes. Cambridge: Harvard University Press.
Weisstein, E. (2005). Graph. MathWorld. Retrieved March 22, 2005 from
http://mathworld.wolfram.com/Graph.html
Williams, C. G. (1998). Using concept maps to assess conceptual knowledge of function
[Electronic version]. Journal for Research in Mathematics Education, 29(4), 419-421.
Zwaneveld, B. (2000). Structuring mathematical knowledge and skills by means of
knowledge graphs [Electronic version]. International Journal of Mathematical
Education in Science and Technology, 31(3), 393-414.
Recommended