Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
WINTER 2009 1
WINTER 2009
Advances in Engineering Education
Using Concept Maps to Assess Interdisciplinary Integration of Green Engineering Knowledge
Maura Borrego
Chad B. NewswaNder
Lisa d. McNair
seaN McgiNNis
Marie C. Paretti
AbsTRAcT
Engineering education, like many fields, has started to explore the benefits of concept maps as
an assessment technique for knowledge integration. Because they allow students to graphically link
topics and represent complex interconnections among diverse concepts, we argue that concept maps
are particularly appropriate for assessing interdisciplinary knowledge integration. The results from a
year-long study of a design course in green engineering demonstrate the viability of this approach.
However, this research also highlights important issues in faculty scoring of interdisciplinary concept
maps that may not be present when maps are used in traditional single-discipline settings. The inter-
disciplinary setting revealed differences in (1) evaluation criteria, (2) expertise, and (3) investment.
We conclude with suggestions for selecting and training scorers to address these issues.
Key Words: assessment, concept maps, green engineering, interdisciplinary
I. INTRoducTIoN
Calls for engineers who can operate in interdisciplinary environments have emerged from both
government and industry in recent years. in response to the need for skills in interdisciplinary col-
laboration, multidisciplinary and interdisciplinary design experiences have proliferated in engineering
curricula, as even a brief survey of recent scholarship suggests [1, 2]. in engineering education, this
is often enacted by bringing students from various disciplines together to work on a group project.
as richter’s content analysis of asee conference papers demonstrates, engineering educators are
creating innovative experiences at all academic levels and scales that bring students from differ-
ent disciplines together to address complex technology needs [1]. smaller efforts involve a limited
2 WINTER 2009
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
number of engineering departments or a specific project. Chen et al., for example, describe a project
that brings together students from mechanical, electrical, and industrial engineering to design and
build a solar-powered boat [3]. in other cases, universities have well-developed programs supporting
a wide range of projects, often with industry sponsors, and typically reaching out beyond engineering
[4–7]. stanford’s institute of design, for example, includes business, medicine, the humanities, and
education; howard university brings mechanical and electrical engineering students together with
students in marketing and art to collaborate on industry-sponsored projects [8].
what such programs often lack, however, are effective assessment instruments to measure indi-
vidual student learning gains associated with the interdisciplinary dimension of the projects. analysis
of papers presented at the 2007 asee conference reveals that of 86 papers addressing multi- or
interdisciplinary design, only two offered methods to assess the multi-/ interdisciplinary dimen-
sion [1, 2]. to bridge this gap, we focus on one of the hallmarks of interdisciplinary collaboration:
knowledge integration. a number of leading researchers have highlighted the ability to synthesize
or integrate knowledge and approaches from multiple disciplines as the key factor separating true
interdisciplinarity from a multidisciplinary division of labor in which experts work only in their own
domains. interdisciplinary collaborations require individuals from different backgrounds to come
together, overcome seemingly incommensurable values and language patterns, integrate knowledge,
and reach a high degree of consensus. good collaborators therefore possess the skills to integrate
knowledge and communicate across disciplinary boundaries.
one relatively new assessment technique that can be beneficial in these settings is the use of concept
maps. these maps are tools that allow students to articulate knowledge by drawing or outlining core
concepts and showing links between ideas (see Figures 1 and 2 as examples). we postulate that concept
maps are a particularly appropriate means of evaluating individual interdisciplinary learning and devel-
opment. in dealing with the complexity of interdisciplinary integration, the multidimensional nature of
concept maps can help represent the shifts in understanding that individuals undergo as they exchange
knowledge, methods, and values [9]. however, for this tool to be useful, faculty and researchers need
an effective approach to scoring concept maps. Besterfield-sacre and coworkers [10] have developed
a reliable holistic scoring method for concept maps used to analyze students’ integration of knowledge
within a single discipline. their study, however, relied on faculty scorers who shared a common discipline,
knowledge base, and departmental affiliation. in adapting this approach to an interdisciplinary content
area lacking these commonalities, we sought to explore new complexities that might emerge when faculty
evaluators from different backgrounds and disciplines come together to apply this holistic scoring tech-
nique. our literature review argues that concept maps are an appropriate means of assessing integration
of interdisciplinary content. Consequently, the research question guiding our inquiry is: what factors or
concerns arise in faculty scoring of student concept maps when the topic is interdisciplinary?
WINTER 2009 3
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
this study examined concept maps as an assessment tool in an interdisciplinary green engineer-
ing design course. the findings both illustrate a successful implementation of this technique and
describe patterns associated with the interdisciplinary scoring process. specifically, differences in
evaluation criteria, expertise, and investment all influenced the scoring process. we conclude with
a discussion of how to address these factors when selecting and training scorers and implementing
the assessment in an interdisciplinary setting.
II. LITERATuRE REvIEW
A. Knowledge Integration in Interdisciplinary settings
the need to train and cultivate individuals who are willing and able to operate in an interdisciplinary
environment is a direct result of “the character of problems currently under study, many of which
Figure 1. Sample High-Scoring Green Engineering Design Student Concept Map (post).
This map was one of three tied for highest score overall, and faculty spent considerable time
discussing its merits.
4 WINTER 2009
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
require the combined efforts of scholars trained in different disciplines” [11]. the complexity and global
interconnectedness of many of the engineering challenges of this century require interdisciplinary
collaboration to examine the links across fields and to answer crucial questions requiring multiple
perspectives [12]. such interdisciplinary work requires a range of skills addressing both content and
process [13, 2]. Most notably, the ability to link and transmit knowledge across unfamiliar contexts is
becoming critical. research suggests that it is essential to develop people “who can serve as both
‘hubs’ and ‘bridges’” between different experts [11]. thus although many of the descriptions of inter-
disciplinary design experiences in engineering currently omit measurable, concrete learning outcomes,
there is a consensus among scholars that the ability to integrate knowledge, methodologies, and
values from multiple disciplines is one of the hallmarks of a good interdisciplinary collaborator [14, 15,
9, 13, 16, 12, 17–19]. it should be noted that although most interdisciplinary collaboration and learning
takes place in team settings, we are focusing on an individual’s ability to integrate perspectives and
content from different disciplines. specifically, Boix Mansilla and duraising identify evidence of this
disciplinary integration as a “conceptual framework, graphic representation, model, leading metaphor,
complex explanation, or solution to a problem” [19]. this individual ability, as noted earlier, is critical
to successful collaboration, and thus provides one indicator of how well individuals will function on
interdisciplinary teams. as a graphic representation of the relationship between concepts, concept
Figure 2. Sample Low-Scoring Green Engineering Design Student Concept Map (pre).
This was one of several maps tied for lowest score, and faculty commented on its lack of
interconnection and heavy reliance on jargon and “buzzwords.”
WINTER 2009 5
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
maps of an interdisciplinary content area would serve as evidence of disciplinary integration (as would
design products and projects, which are beyond the scope of this paper.)
despite this consensus, a number of challenges remain in developing robust techniques to assess
this and related outcomes. accurate assessment of interdisciplinary work is difficult because there
is often a lack of “shared strategies and motives” among participants and those who evaluate them
[1]. students, professors, and professionals from different backgrounds have diverse assumptions,
values, and epistemic frameworks for how knowledge is accumulated. in addition, participants often
bring varying levels of expertise to the collaboration [12]. these differences complicate the assess-
ment process because they result in differences in evaluation criteria, expertise, and professional
investment among those responsible for reaching consensus in evaluation. the result is that the
scoring process itself becomes an instance of interdisciplinary collaboration.
Current research suggests two key elements that can improve success of interdisciplinary col-
laboration: increased interaction time and the presence of individuals capable of bridging per-
spectives. without a considerable degree of interaction and investment, it is difficult to arrive at
shared consensus of what is excellent, average, and poor work (for either collaborators or evalua-
tors). similarly, the Committee on Facilitating interdisciplinary research recommends “extra time
for building consensus and for learning of methods, languages, and cultures” in interdisciplinary
collaboration [12].
b. concept Maps in Engineering Education
to address the gap in assessment tools for interdisciplinary learning and collaboration, we argue
that concept maps are an appropriate method of assessing the integration of content knowledge
from various disciplines into a coherent picture. Concept maps “are graphical tools for organizing
and representing knowledge” [20]. they are pictorial essays that allow people to structure knowl-
edge by connecting concepts in a hierarchical and/or linear fashion by identifying concepts and
sub-concepts and using cross-links to illustrate the relationships between them. Novak, who has been
developing and refining concept maps as a structure-based tool to assess student learning [21, 22,
20], argues that a concept map can be used “not only as a learning tool but also as an evaluation
tool, thus encouraging students to use meaningful-mode learning patterns” [20]. Concept maps
have been used in programs ranging from middle-school education [23] to statistics [24] to student
teacher development [25] to teaching culturally diverse learners [26].
Concept maps have been used to help both engineering and non-engineering students better
synthesize knowledge across and between disciplines. Krupczak [27] argues that one way to teach
engineering students more effectively is to learn how non-engineering students come to grasp
technical concepts. technological literacy courses have relied on concept maps and a “how things
6 WINTER 2009
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
work” approach to more effectively teach non-engineering students complex, technical ideas. with
these methods, students “are receptive and even enthusiastic about learning about technology”
[27]. Moreover, the close alignment of interests and expertise levels between first-year engineering
students and non-engineers makes this concept map strategy applicable to beginning engineering
courses [27]. the approach can help beginning engineering students grasp concepts more easily,
introduce them to design issues, “establish appropriate prior knowledge for future courses,” and
create a “sense of practical empowerment in novice or even tentative engineers” [27].
Nair has also used concept maps across a broad spectrum of educational levels and disciplines,
including secondary and post-secondary students and technical, non-technical, and interdisciplinary
topics [28]. For example, as an exercise to understand life cycle analysis and green design, concept
maps helped reaffirm the importance of: working in interdisciplinary teams, considering context in
problem solving, understanding the design process, making decisions ethically, and solving open-
ended problems, and. in addition, Nair notes that a concept map exercise “develops technical
context knowledge, interdisciplinary knowledge, decision making skills, and group interaction and
communication skills” [28].
in addition to serving as tools to promote student learning, concept maps are also increasingly
important in assessing that learning, particularly in engineering education. For example, walker et al.
[29] found that a key difference between students and experts in the engineering design process is
the ability to address the broader sociocultural context, especially in terms of ethics and marketing.
students were unable to see beyond the problem statement at hand and address concerns such as
motivation for the design, regulatory requirements, or ethics. the researchers used concept maps to
evaluate student learning from the beginning to the end of a year-long course based on engineering
design. the concept-map approach demonstrated that the student-expert gap closed in two areas:
design process and understanding the motivation behind the design.
similarly, Norman et al. [30] have used concept maps to “directly address how students were
able to better synthesize skills developed from multiple courses to new problems” [30]. the course
focused on helping students in industrial engineering develop their ability to solve unstructured
problems. Concept maps were the primary method used to assess student ability to synthesize
knowledge from multiple sources and apply that knowledge to new problems. Moreover, the authors
note that the department routinely uses concepts maps for programmatic evaluation to assess
knowledge integration in graduating seniors from one cohort to the next.
c. scoring concept Maps in Assessment settings
as Besterfield-sacre et al. point out, concept maps remain somewhat controversial as assessment
tools because of the difficulty of developing a robust approach to map scoring [10]. approaches
WINTER 2009 7
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
range from counting concepts, links and levels of hierarchy to comparisons of novice and expert
maps. in seeking to develop a valid and reliable scoring method appropriate for engineering educa-
tion, Besterfield-sacre et al. [10] proposed and validated a holistic assessment method that considers
comprehensiveness, organization, and correctness as its evaluation criteria. the approach was tested
on a within-discipline analysis of students’ ability to integrate knowledge across a curriculum. Con-
cept maps of industrial engineering were collected from students at multiple points in the curriculum
(sophomore, junior, senior); one cohort was followed from sophomore to senior year. scorers were
drawn from departmental faculty, and the results were used to identify and correct programmatic
weaknesses. Later, the method was also applied in chemical engineering [31].
their results demonstrated both the usefulness of concept maps to measure knowledge integration
within a discipline and the robustness of the holistic scoring approach. in particular, they found the
“holistic rubric is sufficiently sensitive to detect growth in students’ knowledge integration” [10].
two faculty scorers from the department scored the maps and, where differences arose, discussed
the maps to reach consensus. the scoring approach was then translated into a rubric that could be
used by other faculty to score the maps. the results demonstrated the validity of the rubric, though
they also suggested some unexpected variance across scorers in the organization category that
led to an improved final rubric.
importantly, however, Besterfield-sacre et al.’s study focused on knowledge integration within
a single discipline. the holistic scoring method proved valid where the scorers shared expertise,
departmental affiliation, and disciplinary expectations. in contrast, as the literature review demon-
strated, in interdisciplinary contexts these shared experiences are notably absent, and in fact pose
a significant barrier to the process of reaching consensus.
III. METhods
to determine the usefulness of concept maps as assessment tools in interdisciplinary settings
and to test the holistic scoring process in this context, we applied the method to an interdisciplin-
ary engineering design course in green engineering. data collection and analysis procedures were
designed to directly address the research question: what factors or concerns arise in faculty scoring
of student concept maps when the topic is interdisciplinary?
A. Green Engineering as a site for Interdisciplinary collaboration and Knowledge Integration
in considering concept maps as a mechanism for assessing knowledge integration, we turned to
a green engineering design course as our study site. “green engineering” and related concepts such
8 WINTER 2009
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
as sustainability are critical nodes of interdisciplinary collaboration because of their prominence in
contemporary engineering practice, and the complexity of these fields requires a high degree of
knowledge integration.
green engineering is not a new discipline, but rather a different perspective of traditional engi-
neering practice. it uses the same rigorous application of science and math, but asks disciplines to
consider environmental effects in designing products, processes, and systems. the us environmental
Protection agency (ePa), an early promoter of green engineering, defines it as
the design, commercialization, and use of processes and products, which are feasible and
economical while minimizing 1) generation of pollution at the source and 2) risk to human
health and the environment. green engineering embraces the concept that decisions
to protect human health and the environment can have the greatest impact and cost
effectiveness when applied early to the design and development phase of a process or
product [32].
this and similar definitions emphasize applying environmental impacts, including the extraction
and processing of raw materials and ultimate disposal of the project, as initial design constraints.
examples of green engineering have been around for decades across disciplines, but the topic has
emerged as a distinct approach only in the past ten years. a set of green engineering principles
was developed and discussed in detail by anastas and Zimmerman [33]. a similar list of green en-
gineering principles was developed in May 2003 at the green engineering: defining the Principles
Conference held in sandestin, Florida, through negotiated consensus among stakeholders [34].
green engineering has also been a topic at the annual american Chemical society (aCs) green
Chemistry and engineering Conference for the past twelve years.
european countries are often considered to have successfully implemented more technologies
which fit under the principles of green engineering. in the united states, green engineering is
being pursued at an increasing rate in industry due to the potential for costs savings, risk reduc-
tion, and employee and customer awareness and appeal. the presence of “green” or “sustain-
able” engineering courses and programs has been growing in recent years. For example, Virginia
tech has offered a concentration in green engineering since 2001 [35], and stevens institute of
technology [36] and san Jose state university [37] now have minors in the field. Various other
schools offer specific courses and programs related to green engineering without a formal degree
in this area.
importantly for this study, not only is green engineering increasingly prominent in engineering
curricula, it is also inherently interdisciplinary. the consideration of environmental impacts across
WINTER 2009 9
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
the life cycle of products and processes requires technical consideration of biology, chemistry,
and physics in engineering applications. Moreover, there are often non-technical issues such as
economics, politics, aesthetics, psychology, and others, that factor in the success of a green engi-
neering approach to a problem. For example, computer scientists, materials engineers, and electrical
engineers can collaborate to develop more effective data storage methods that reduce hardware,
energy, and resource requirements for computers and data storage facilities. similarly, computer
simulations of experiments or product/process models can reduce or eliminate the need for pro-
totypes in a number of fields.
the ways in which the integration of environmental considerations requires collaboration among
diverse disciplines also makes green engineering a popular approach to interdisciplinary design
projects in engineering programs in the u.s. and internationally [38–42].
b. Research setting and Participants
this study involved eleven students enrolled in a two-semester green engineering course;
one student dropped the course after the first semester. the course included four students from
biological systems engineering, two from materials science and engineering, and one each from
management, industrial systems engineering, civil and environmental engineering, and engineering
science and mechanics (see table 1). the course was co-taught by three faculty members whose
primary academic backgrounds were chemical engineering/english, english/linguistics, and materials
science/chemical engineering.
the course was patterned after a typical capstone design course. to support student enrollment,
four departments agreed to allow this course to directly substitute for senior design with no addi-
tional requirements. other departments allowed credit but added limitations including the following:
Table 1. Team composition by student major.
10 WINTER 2009
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
(1) students still needed to go to a department-specific design lecture; and (2) students needed to
complete some specific assignments independent of this course. the class met as a group weekly
for 75 minutes in a recitation section that covered traditional design topics including the design
process, project management, communication, teamwork, and conflict resolution. in addition, the
recitation included discussions of green engineering, life cycle analysis, and sustainability. Cradle
to Cradle served as the central text for this component [43], and students were expected to both
discuss the text in class and apply its principles to their design project.
two teams were formed based on project interest, as summarized in table 1. team #1 (a fifth
member dropped after the first semester) had a corporate client interested in reducing the envi-
ronmental impact of their production waste. team #2 developed a design for a water filter for the
developing world; they had no industry sponsor.
c. Implementation of concept Maps for Assessment
to evaluate gains in interdisciplinary understanding of green engineering, students were asked
to draw two maps, one at the beginning of the course and one at the end of the course. to ensure
that the students understood concept maps, an outside expert (a faculty member unaffiliated
with the course) introduced the process using the French fry example originally developed by
Jennifer turns and described in Besterfield-sacre et al. [10]. students were free to ask questions
about how to construct a concept map, but were given full latitude in building their own maps on
green engineering design. they were asked to complete a second map at the end of the course,
again with the guidance of the outside expert. as reference, students were given a handout dur-
ing both sessions summarizing the steps to constructing a concept map (table 2). Consistent
with irB regulations, course instructors were not present for these activities, nor did they find
out which students consented to participate in the research until after final grades had been as-
signed. eleven students completed the initial exercise independently; one student did not enroll
in the second semester and thus did not complete the post-course exercise; that student’s map
was dropped from the assessment pool to allow for comparisons of pre and post maps. the maps
were stripped of names and information identifying them as pre or post course maps. (students
created the maps using sticky notes on larger pieces of paper; then a graduate assistant recreated
the maps in PowerPoint.)
the 20 concept maps were scored by faculty from three departments: materials science and
engineering, biological systems engineering, and green engineering. the green engineering
scorer was one of the course instructors; just prior to the course, he constructed his own con-
cept map of green engineering. these three faculty were selected as scorers because of several
key characteristics. First, all were committed to the place of design in undergraduate education:
WINTER 2009 11
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
one member was the capstone design instructor for his department, another had many years of
experience as a capstone project mentor, and the third was a new mentor of capstone projects.
all were also committed to providing students with opportunities for interdisciplinary design and
had been instrumental in helping the course receive approval as a capstone experience in mul-
tiple departments. Finally, all had been consistently supportive of and interested in the concept
of green engineering both in their own research and in terms of the institution’s management of
its green engineering courses.
the faculty scored the maps in a single two-hour session using the holistic approach developed
by Besterfield-sacre et al. [10] and the rubric shown in table 3. this rubric required evaluators to
score each category on a range of 1 to 3 points; the scores in each category were then summed to
determine an overall score for the map. the scoring session involved three components:
1. an engineering education expert “normed” the group by setting expectations for understand-
ing and interpreting concept maps and using examples to bring the group together on scoring
values. the expert explained how to use the rubric and answered questions.
Table 2. Handout: Steps to Constructing a Concept Map.
12 WINTER 2009
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
2. each scorer then rated all 20 maps individually, without consulting the other scorers. there
was an occasional question to clarify concepts and evaluation practices, but no dialogue about
scoring was allowed.
3. the scorers came together to discuss each map and reach a consensus on the scores.
the scoring took place in the summer, approximately 2 months after the course ended; none of
the faculty scorers were teaching summer school.
d. data sources and Methodology
to analyze the scoring process, this study relied on a mixed method approach [45] using a
variety of data sources:
1. the concept maps themselves
2. Quantitative individual and consensus scores for each concept map
3. student information linking pre and post map scores
4. student majors and faculty scorer disciplinary backgrounds
Table 3. Concept Map Scoring Rubric, reproduced from [10].
WINTER 2009 13
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
5. Qualitative observation field notes and transcription of the recording from the entire scoring
session
this approach applies a triangulation mixed methods design. the term “triangulation” in re-
search was first used by denzin [46] to describe bringing together complementary methods or
data sources to offset weaknesses in each (exactly three sources are not required). applied to
mixed methods, data are collected concurrently and combined during the analysis phase, and
interpretation involves comparing the results of each to best understand the research question,
specifically: what factors or concerns arise in faculty scoring of student concept maps when the
topic is interdisciplinary? the quantitative and qualitative components are also assigned equal
priority and weighting [47, 45], though the qualitative results require more publication space to
describe.
E. data Analysis
the qualitative data was analyzed and integrated with the quantitative findings by iteratively com-
paring data sets to each other and to the literature. the first three authors listened to the audiotape
of the scoring session and developed an initial list of factors associated with scoring interdisciplinary
concept maps, supported by the literature on interdisciplinary collaboration. these themes led to
hypothesis testing and a final thematic coding scheme for the qualitative data:
1. evaluation criteria
2. expertise
3. investment
Constant comparative method [48] forced us to consider all possible explanations, and to test
these against both qualitative data and quantitative relationships. the primary statistical analysis
techniques applied to the quantitative data were t-tests comparing pre and post scores of individual
students and of groups of students to each other, as well as Cronbach’s alpha for inter-rater reli-
ability among scorers and Pearson correlations to compare individual and consensus scores. after
the codes were created and applied to the qualitative data, a second researcher independently
checked the codes and applied them to the entire data set. Finally, we shared our findings with the
three course instructors and two additional faculty scorers as a final check of our interpretations.
Iv. REsuLTs ANd dIscussIoN
Figures 1 through 4 illustrate two pre-course and two post-course maps; Figures 3 and 4 are
from the same student. as these figures show, in general the post-course concept maps show an
14 WINTER 2009
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
increase in the number of topics included, the number of appropriate topics included, and the de-
gree of integration among topics. this increased visual complexity suggests an expected increase
in knowledge integration as a result of the interdisciplinary experience. the quantitative analysis of
the concept map scores demonstrates the student learning gains.
table 4 presents the pre and post concept map scores for all three individual scorers as well
as the consensus scores. it should be noted that for one student (J), the pre score was actually
higher than the post test (due to changes in both comprehensiveness and organization subscores).
as explained below, the lower post score reflects an inability to adequately address design, while
the higher pre-course map score demonstrates more connections and links between concepts. in
addition, another student (i) showed no gains between the pre and post-course maps and in fact
received the lowest possible score on both maps. despite these two cases, a paired samples t-test
confirms that the student gains were statistically significant (pre: mean 5 3.80, s.d. 5 1.32, post:
mean 5 5.10, s.d. 5 1.29, t 5 22.90, df 5 9, p 5 .018). the distribution of scores is not statistically
significantly different from those for junior and senior level students published by Besterfield-sacre
et al.’s [10] table 7 (t 5 .470, df 5 44, N.s.). the two concept maps for student a (with the largest
gain) are presented in Figures 3 and 4. these overall results provide a foundation for the following
three sections, which combine quantitative and qualitative results (where relevant) for each of the
three themes.
Figure 3. Pre-course Concept Map for Student A, with the Largest Gain.
WINTER 2009 15
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
A. Evaluation criteria of Faculty from different disciplines
differences in table 4 seem to reflect a lack of consensus about the structure and boundaries
of interdisciplinary content. similarly, discussion during the scoring session indicated an initial lack
of consensus, which required faculty to articulate their interpretation of the criteria. statistical
Figure 4. Post-course Concept Map for Student A, with the Largest Gain.
16 WINTER 2009
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
significance tests of the quantitative data describe the impact of these differences on the concept
map scores.
at separate times, each instructor observed that they were scoring the maps quite differently
from one another. the biological systems engineering scorer posed the question, “do we agree on
anything?” while the green engineering scorer noted, “we don’t have any agreement.” Finally, the
materials science and engineering scorer said “we are all over the place.” From the discussion be-
tween the three faculty during the consensus scoring, it was clear that they were relying on different,
possibly disciplinary, standards to judge the concept maps. there was limited shared expectation
of how to apply the criteria to interdisciplinary concept maps, and this was reflected in the faculty
scores; Cronbach’s alpha was 0.69. For this measure of inter-rater reliability, 0.70 is considered
acceptable, while 0.80 or higher is good.
analysis of the session transcript suggests that much of this difference may be disciplinary. the
biological systems engineering scorer, himself a capstone design instructor in a department that
emphasizes the design process, relied heavily on processes feeding back into design to score his
concept maps. in one instance, he noted he loved one map because it “feeds back into design” and
in general disliked most maps because “everybody has missed taking information over here and
having it feed back into the problem statement.” another criticism he had of the maps was excessive
linearity, which he identified as inconsistent with his biology training. in particular, he did not like
Table 4. Student Concept Map Scores. Scores from the three criteria (Table 3) are
combined. BSE 5 biological systems engineering faculty scorer, MSE 5 materials science and
engineering faculty scorer, and GrE 5 green engineering faculty scorer. Students D, F, G and
I are biological systems engineering majors, and students B and J are materials science and
engineering majors.
WINTER 2009 17
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
when students failed to show the relationship between concepts. he said, “the reason i didn’t like
it, maybe because i am in the biological sciences, is that it was all linear.” his standards of scoring
were based on the students’ ability to integrate green thinking with design.
while the biological systems engineering scorer relied heavily on integrating process with design,
the green engineering scorer noted omissions and inferred what the students knew about green
engineering but did not draw. he relied heavily on omissions and gaps in knowledge, particularly
when scoring comprehensiveness and correctness. he told the rest of the group to “think of gaps
as wrong.” in a similar instance, he said, “there [are] two ways to look at correctness: focusing on
the wrong stuff and focusing on how right it is.” to him, for a map to be correct, it must also be
comprehensive. interestingly, however, his scoring was also tempered by his assumption that stu-
dents possessed core knowledge about green engineering design, but failed to accurately draw
that knowledge on a piece of paper. he said, “My assumption was that i think they understood it,
but drew it poorly.” on another map he scored, he commented that “they understood it, but drew
[it] incorrectly.” this faculty scorer was also one of the course instructors, and may have allowed
his familiarity with the class to influence his scoring.
the issue of visualization may explain why the biological systems engineering scorer evaluated
the maps differently than the green engineering scorer. although studies have found that most en-
gineering students at a large public institution prefer a visual learning style [49], the scorers did not
agree that these students were able to produce effective visual representations of their knowledge.
For example, the faculty member who teaches capstone design in biological systems engineering
focused on how the students failed to show the connection between design and process, while the
green engineering expert who taught the course assumed that students knew more about green
engineering content than they conveyed. in response to a comment from the biological systems
engineering scorer about the failure to link process to design, the green engineering instructor noted
that “we talked a lot about that [in class].” also, the actual practice of designing according to green
engineering principles was integral to the students’ design projects. another primary difference be-
tween the two raters was that the biological systems engineer focused on organization (connecting
concepts), while the green engineering instructor zeroed in on correctness and comprehensiveness.
the Cronbach’s alpha for these two faculty scorers was a low 0.56.
the materials science and engineering professor said little about his scoring criteria, but was also
concerned about the variance between different students’ abilities to visualize concepts; this factor
is discussed further in section C below. rather than describing specific scoring criteria, this faculty
member expressed a lack of confidence in his green engineering design expertise. specific com-
ments are included in the following section (B), but it appears that during the consensus phase, this
faculty member deferred to the judgment of the two others. in the absence of relevant qualitative
18 WINTER 2009
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
data, we can use the quantitative data to test a hypothesis related to discipline-based evaluation
criteria. if this faculty member did not have a clear sense of this interdisciplinary content area, did
he perhaps substitute disciplinary criteria? if so, then he might have inadvertently scored students
from his own discipline higher than other students. an independent samples t-test reveals that this
is indeed the case: the materials science and engineering faculty member scored materials science
and engineering students higher than other students (t 5 5.088, df 5 18, p 5 .001). Neither of the
other two faculty scored these students higher than other students (biological systems scorer t 5
.916, df 5 2.606, N.s.; green engineering scorer t 5 1.111, df 5 18, N.s.). table 4 lists all the scores, and
readers will note that some of the highest scores were assigned by the materials science faculty
member to the materials science students, even though names and majors were stripped from the
maps. ultimately, however, the sample size is too small to draw definitive conclusions. this particular
finding suggests interesting directions for future investigations.
b. Expertise in the Interdisciplinary content Area
Both quantitative and qualitative data demonstrate that the green engineering scorer was viewed
as an expert by the other two faculty scorers. the green engineering instructor’s scores were
highly correlated to the final consensus scores (Pearson Correlation 5 .85, p 5 .002). the materials
science faculty member’s scores were also statistically significantly—but less strongly—correlated
to the consensus scores (Pearson Correlation 5 .67, p 5 .036). it should be noted that although the
green engineering instructor had several years of experience in green engineering, his disciplinary
training was also in materials science and engineering. the biological systems faculty member’s
concept map scores were not statistically significantly correlated to the consensus scores.
throughout the consensus process, the faculty scorers deferred heavily to the green engineer-
ing scorer both because he was perceived as the content expert and because he was one of the
course instructors. they asked the green engineering scorer multiple questions about the structure
of the class, how the concept map assignment was given, and technical details about the design
and process of green engineering. green engineering questions ranged from simple ones such as
spelling out abbreviations for green engineering Principles and Life Cycle analysis to more complex
questions about how a nutrient can be a material.
Furthermore, the green engineering scorer took an active role in trying to shape how the others
viewed and scored the concept maps. after the norming session, the green engineering scorer noted
that “we will re-norm ourselves when we talk about correctness.” he then proceeded to explain to
the other two scorers that assessing correctness can either focus on errors or “how right [the overall
map] is.” with this explanation, he referred back to gaps in knowledge as a key scoring criterion for
both correctness and comprehensiveness.
WINTER 2009 19
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
while the green engineering scorer was seen by both himself and the other two scorers as the
expert, the materials science and engineering scorer reflected doubt about his ability to objectively
score the maps. he admitted in response to the green engineer’s criteria for correctness and compre-
hensiveness that he lacked the knowledge to know whether something was missing or right. he said,
“i don’t know if i am familiar enough with the subject area to know if it is comprehensive.” instead
of citing specific criteria about what was comprehensive and correct, he used general criteria in his
individual scoring more than references to green engineering expertise. For example, he aligned
his judgments with the biological systems faculty member’s insistence on connecting process to
design—a more central concept of engineering in general. in fact, the materials science and engi-
neering scorer started to use the biological systems scorer’s logic in making arguments. he noted,
“it is hard to follow. it didn’t feed back into design.” Because of his lack of content knowledge about
green engineering, the materials science and engineering scorer appeared to rely on the expertise
of the two members in the group scoring session and used general principles in his scoring.
c. Investment in concept Maps and the Interdisciplinary Program
in addition to evaluation criteria and expertise, the theme of investment also emerged from
the qualitative data. the commitment, in particular, to the use of concept maps to assess student
performance was mixed at the beginning of the discussion. there were statements related to not
fully embracing the idea of concept maps as a learning tool or a good evaluation mechanism and
references to hurrying the evaluation process because of other commitments. however, at the end
of the scoring session there was an increased interest in how concept maps could be used to fur-
ther student learning in all of the various programs with which these faculty were affiliated. there
were even statements related to using concept maps in their own classes. thus over two hours the
scorers evolved from being skeptical about concept maps to wanting to learn more about them and
use them as a teaching and assessment tool.
at the start of the discussion, the three faculty members questioned whether concept maps were
a viable tool to evaluate student learning. the materials science and engineering scorer was the
most skeptical. he argued, “some people have a hard time reading maps; it doesn’t reflect on their
knowledge of geography. they have a hard time looking at two dimensions.” he continued with this
argument, noting, “there is just a huge variation in how people do homework. some people have
the ability to organize their thought in two dimensions. that is probably related to their understand-
ing of the subject, but probably not.” the green engineering scorer, in contrast, had mixed feelings
about the reliance on concept maps to evaluate student learning. he had his doubts, but also relied
on his own map to structure the flow of the class. he said, “i taught the course based on the map i
had drawn.” although he had drawn his own map, he still had doubts of whether students (or even
20 WINTER 2009
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
instructors) could accurately draw the complexity of the design process in green engineering. he
pointed out, “it is a very hard concept map to do. it is almost too linked to put on paper well.” in
scoring one map, he questioned whether the problem was with the map itself or with the students’
knowledge. he said, “it’s a clear miss. they’re not getting that. so the question is, ‘did they miss it
because of the drawing’” or because “it’s hard to draw[?]”
a second type of investment to consider is investment in the green engineering design course
itself. while all three faculty scorers were supportive of the course and asked or answered questions
during scoring about how it was run, they did not necessarily translate gaps in student learning to
discussion of future improvements for the course. in contrast, Besterfield-sacre and coworkers [10]
reported that departmental faculty scoring their own majors were inspired to restructure curriculum
after seeing how industrial engineering majors could not connect statistics to other concepts. in the
case of an interdisciplinary program, we would expect faculty to have limited investment due to a high
level of department-related disciplinary commitments. the departmental organizational structure of
u.s. universities provides a strong disincentive for faculty to work with interdisciplinary programs,
including scarce resources and diminished recognition for cross-disciplinary efforts [50]. one mode
of discussion that reflected a lack of investment in the project (scoring and/or the design course)
was the concern of being late for other commitments, particularly missing a bus to get home for
the evening. at one point, one of the scorers had to acquiesce to speed up the consensus process.
Noting his dilemma, he said, “i was getting desperate. Just for the record, i really wanted a 3.”
despite time pressures and initial concerns about the validity of concept maps as an assessment
tool, the instructors did show a high degree of interest in using them in their own classes. they repeat-
edly asked the green engineering instructor about the use of concept maps in class. they peppered
him with questions about the instructions, examples he gave in class, how much time the students
had, who was present during the activity, who was the outside expert who explained the process to
students, and whether he had constructed a map himself. By the end of the discussion, the biologi-
cal systems engineering scorer said, “i think this would be a nice thing to develop for a teaching
aid.” he also asked the green engineering instructor for his personal concept map. with enthusiasm,
he said, “i can’t wait to get yours.” at the end, the biological systems and materials science scorers
also asked for the instructions and wanted to know if they could keep the printout of the concept
maps they scored. even though there was lack of initial investment in the assessment technique, the
scorers spent considerable time and commitment in talking about and examining the maps. even
with time concerns, they rarely accelerated the process of assessing the maps. they attacked each
map with vigor and a stringent analysis of what was included and what the students failed to do. as
they read them and scored them, there was a general degree of excitement about the possible use
of them in the classroom.
WINTER 2009 21
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
v. coNcLusIoN
given the centrality of knowledge integration in interdisciplinary environments and the power
of concept maps to represent complex knowledge networks, we argue here that concepts maps
are a valuable tool for assessing students’ interdisciplinary development. as our literature review
demonstrates, prior work on concept maps illustrates their assessment value in settings such as K-12
science learning, teacher education, engineering design, and technological literacy. they have been
successfully used to evaluate student development in individual courses as well as across curricula
because of their ability to represent not simply content mastery, but connections across content
areas. this representational complexity makes them ideal vehicles to evaluate one of the hallmarks
of successful interdisciplinary collaboration: knowledge integration.
a comparison of pre and post concept maps of students in a green engineering design course
clearly illustrates the ways in which this interdisciplinary design experience increased both
the number of topics students were able to associate with green engineering (as measured by
comprehensiveness and correctness) and the number of connections they were able to make
among those topics (as measured by organization). importantly, the maps also captured varia-
tions in student learning, showing little or no gain for some students, as would be expected in
any course.
however, while this study illustrates the use of concept maps as an interdisciplinary assessment
tool, it also identifies several core implementation issues that faculty need to address when adopting
the tool. specifically, selecting and training scorers is complicated by three factors:
1. Because any interdisciplinary topic, by its very nature, draws together individuals from different
backgrounds, scorers may come with highly divergent evaluation criteria. such variations can
occur even in single discipline assessments, but the qualitative data presented here suggests
that interdisciplinarity may exacerbate the issue. Besterfield-sacre reports that disciplinary
faculty in industrial engineering and chemical engineering came to consensus quickly when
applying the same rubric to disciplinary student concept maps [51].
2. differences in background also produce differences in levels of expertise with the interdisci-
plinary topic, even when scorers share a commitment to or affiliation with that topic. Faculty
remain rooted in their own knowledge domains, and may be reluctant to trust their expertise
in a more complex interdisciplinary area. this reluctance can lead to a reliance on a single
evaluator to arbitrate rather than on a more robust process of negotiation seen in disciplinary
settings [52, 51].
3. scorers in interdisciplinary settings working with a new assessment instrument may have
varying levels of investment in both the process and the program. Concept maps, though
22 WINTER 2009
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
gaining in popularity, are still a relatively new assessment tool in engineering education.
in addition, interdisciplinary programs often reside outside individual departments and af-
filiate faculty may lack a sense of ownership or commitment—even when they support the
general mission of the program. these two factors can work together to again reduce the
robustness of the scoring process; the individual(s) with the most expertise or the most
investment can dominate the process and consensus can be reduced to deference to an
expert.
each of these factors can be mitigated when selecting and training scorers through a variety of
strategies:
1. while norming is always central to the training of scorers, it is even more crucial in interdisci-
plinary contexts. as with any interdisciplinary collaboration, upfront discussions of differences
can be very productive. Begin scoring sessions by asking the scorers to make their evaluation
criteria as explicit as possible, and allow substantial time for faculty to discuss the nature of the
criteria used for evaluation (more time than allotted for within-discipline norming sessions). we
would suggest, in fact, have scorers produce their own concept maps of the topic as a starting
point for the discussion.
2. when selecting scorers, evaluate levels of expertise to insure that the faculty participants are
relatively evenly matched in their understanding of the topic. if only one individual is recognized
as an expert in the field, he or she may likely dominate the process. Balancing expertise levels
insures a more robust consensus process.
3. Build in strategies for insuring that scorers have a stake in the outcomes of the assessment.
For example, consider selecting individuals who will be involved in revisions of the course or
program, who may teach or advise in the program, or who are responsible for larger assess-
ment efforts that include the interdisciplinary setting. Provide the scorers with feedback on
the results of the assessment and the subsequent changes to the course or program that stem
from the findings. invite scorers back in subsequent years to build continuity.
Concept maps, as we have shown here, are robust tools for evaluating knowledge integration in
interdisciplinary settings, particularly, as described above, when the process of selecting and train-
ing scorers takes disciplinary differences into account.
AcKNoWLEdGEMENTs
the authors wish to thank the National science Foundation for supporting the green engineer-
ing design course and graduate assistant Chad Newswander through grants due-0633537 and
WINTER 2009 23
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
eeC-0648439. the views expressed in this paper are those of the authors and do not necessarily
represent those of the National science Foundation. we are grateful to Mary Besterfield-sacre for
helpful discussions about the scoring process, Catherine amelink and euan Lindsay for helpful dis-
cussions about statistical analysis, whitney schaefer for transcription, and the three faculty scorers
and ten students who agreed to participate in this study.
REFERENcEs
[1] richter, david M. 2008. a Case study of Pedagogy in an interdisciplinary green engineering Course, Mechanical
engineering, Virginia tech, Blacksburg.
[2] richter, david M., and Marie C. Paretti. submitted. identifying Barriers to and outcomes for interdisciplinarity in
the engineering Classroom. European Journal of Engineering Education.
[3] Chen, J., e.a. hughes, s.L. wang, and w.J. Collis. 1997. implementing a Multidisciplinary system design
and engineering Course using solar splash ’97. Paper read at Proceedings—Frontiers in education Conference.
http://fie-conference.org/fie97/papers/1104.pdf
[4] Hasso Plattner Institute of Design at Stanford. 2006. 2005 [cited 20 april 2006]. available from http://www.
stanford.edu/group/dschool/index.html.
[5] Jenkins, s. rod, James B. Pocock, Patrick d. Zuraski, ronald B. Meade, Zane w. Mitchell, and Jodi J. Farrington.
2002. Capstone Course in an integrated engineering Curriculum. Journal of Professional Issues in Engineering Education
& Practice 128 (2):75–82.
[6] hundhausen, Joan r., and richard Yeatts. 1995. an experiment in integration: Calculus and Physics for Freshmen.
Journal of Engineering Education 84 (4):369–374. http://www.asee.org/publications/jee/PaPers/display.cfm?pdf=113.
pdf&special_issue=113
[7] stiebitz, Paul h. 2004. Multidisciplinary engineering senior design at rit. Paper read at american society for
engineering education annual Conference and exhibition, June 2004, at salt Lake City, utah. http://soa.asee.org/paper/
conference/paper-view.cfm?id=20471
[8] thigpen, L., e. glakpe, g. gomes, and t. McCloud. 2004. a Model for teaching Multidisciplinary Capstone
design in Mechanical engineering. Proceedings—Frontiers in Education Conference 3 (Conf 34):s2g-1–s2g-6. http://
fie-conference.org/fie2004/papers/1439.pdf
[9] Borrego, Maura, and Lynita K Newswander. 2008. Characteristics of successful Cross-disciplinary engineering
education Collaborations. Journal of Engineering Education 97 (2):123–134. http://www.asee.org/publications/jee/
PaPers/display.cfm?pdf=975.pdf&special_issue=975
[10] Besterfield-sacre, Mary, Jessica gerchak, Maryrose Lyons, Larry J. shuman, and harvey wolfe. 2004. scoring
Concept Maps: an integrated rubric for assessing engineering education. Journal of Engineering Education 93 (2):105–115.
http://www.asee.org/publications/jee/PaPers/display.cfm?pdf=811.pdf&special_issue=811
[11] rhoten, diana. 2003. a Multi-Method analysis of the social and technical Conditions for interdisciplinary
Collaboration. san Francisco: the hybrid Vigor institute.
[12] Committee on Facilitating interdisciplinary research. 2004. Facilitating Interdisciplinary Research. washington:
National academies Press.
24 WINTER 2009
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
[13] Borrego, Maura, Lynita K. Newswander, and Lisa dupree McNair. 2007. applying theories of interdisciplinary Col-
laboration in research and teaching Practice. Paper read at asee/ieee Frontiers in education Conference, at Milwaukee,
wi. http://fie-conference.org/fie2007/papers/1399.pdf
[14] Boix Mansilla, Veronica. 2005. assessing student work at disciplinary Crossroads. Change 37 (1):14–21.
[15] Borrego, Maura. 2007. Conceptual difficulties experienced by engineering Faculty Becoming engineering educa-
tion researchers. Journal of Engineering Education 96 (2):91–102. http://www.asee.org/publications/jee/PaPers/display.
cfm?pdf=931.pdf&special_issue=931
[16] Bradbeer, John. 1999. Barriers to interdisciplinarity: disciplinary discourses and student Learning. Journal of
Geography in Higher Education 23 (3):381–396.
[17] deZure, d. 1999. interdisciplinary teaching and learning. Clemson University: Class Action 2 (3).
[18] Fruchter, renate, and sarah Lewis. 2001. Mentoring Models in an a/e/C global teamwork e-Learning environ-
ment. Paper read at american society for engineering education annual Conference and exposition. http://soa.asee.
org/paper/conference/paper-view.cfm?id=16159
[19] Boix Mansilla, Veronica, and elisabeth dawes duraising. 2007. targeted assessment of students’ interdisciplinary
work: an empirically grounded Framework Proposed The Journal of Higher Education 78 (2):215–237.
[20] Novak, Joseph d., and alberto J. Cañas. 2006. the theory underlying Concept Maps and how to Construct them.
in Technical Report IHMC CmapTools 2006-01: Florida institute for human and Machine Cognition.
[21] Novak, Joseph d., and dismas Musonda. 1991. a twelve-Year Longitudinal study of science Concept Learning.
American Educational Research Journal 28 (1):117–153.
[22] Novak, Joseph d. 1998. Learning, creating, and using knowledge: concept maps as facilitative tools in schools
and corporations. Mahwah, NJ: Lawrence erlbaum.
[23] snead, donald, and Barbara Young. 2003. using Concept Mapping to aid african american students’ under-
standing in Middle grade science. Journal of Negro Education 72 (3):333–343.
[24] schau, Candace, and Nancy Mattern. 1997. use of Map techniques in teaching applied statistics Courses. The
American Statistician 51 (2):171–175.
[25] Zanting, anneke, Nico Verloop, and Jan d. Vermunt. 2003. using interviews and Concept Maps to access Mentor
teachers’ Practical Knowledge. Higher Education 46 (2):195–214.
[26] artiles, alfredo J., and Karen McClaffterty. 1998. Learning to teach Culturally diverse Learners: Carting Change
in Preserve teachers’ thinking about effective teaching. The Elementary School Journal 98 (3):189–220.
[27] Krupczak, John. 2007. using insights From Non-engineers to improve introduction to engineering via Functional
analysis. in American Society for Engineering Education Annual Conference. http://soa.asee.org/paper/conference/
paper-view.cfm?id=4524
[28] Nair, indira. 1998. Life Cycle analysis and green design: a Context for teaching design, environment, and ethics.
Journal of Engineering Education 87(4): 489–494. http://www.asee.org/publications/jee/PaPers/display.cfm?pdf=563.
pdf&special_issue=563
[29] walker, Joan M., d. s. Cordray, Paul h. King, and r. C. Fries. 2005. expert and student Conceptions of the design
Process: developmental differences with implications for educators. International Journal of Engineering Education 21
(3):467–479.
[30] Norman, Bryan a., Mary Besterfield-sacre, Bopaya Bidanda, Kim Lascola Needy, and Jayant rajgopal.
2005. integration and synthesis of industrial engineering Curriculum via an unstructured Problem solving Course.
in American Society for Engineering Education Annual Conference. http://soa.asee.org/paper/conference/paper-
view.cfm?id=21902
WINTER 2009 25
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
[31] McCarthy, Joseph J., adetola a. abatan, robert s. Parker, and Mary Besterfield-sacre. 2005. work in Progress:
Pillars of Chemical engineering. Paper read at asee/ieee Frontiers in education Conference, at indianapolis, iN. http://
fie-conference.org/fie2005/papers/1784.pdf
[32] What is Green Engineering. 2008. 2007 [cited august 1 2008]. available from http://www.epa.gov/oppt/greenen-
gineering/pubs/whats_ge.html.
[33] anastas, P., and J. Zimmerman. 2003. design through the 12 Principles of green engineering. Environ. Sci.
Technol., American Chemical Society 37 (5):94a–101a.
[34] abraham, M., and N. Nguyen. 2003. green engineering: defining the principles-results from the san destin
Conference. Environmental Progress 22 (4):233–236.
[35] http://www.eng.vt.edu/green/. 2008 [cited 8/26/2008.
[36] http://www.stevens-tech.edu/press/pr/pr1058.htm. 2007 [cited 8/26/2008].
[37] http://www.engr.sjsu.edu/departments/general/greenengr. N.d. [cited 8/26/2008].
[38] Mulder, K.F. 2006. engineering Curricula in sustainable development. an evaluation of Changes at delft university
of technology. European Journal of Engineering Education 31 (2):133–144.
[39] Pritchard, J., and C. Baillie. 2006. how can engineering education contribute to a sustainable future? European
Journal of Engineering Education 31 (5):555–565.
[40] Cooper, J.s. 2007. evolution of an interdisciplinary Course in sustainability and design for environment Interna-
tional Journal of Engineering Education 23 (2):294–300.
[41] schäfer, a.i., and B.s. richards. 2007. From concept to commercialisation: student learning in a sustainable
engineering innovation project. European Journal of Engineering Education 32 (2):143–165.
[42] grimberg, stefan J., tom a. Langen, Larry d. Compeau, and susan e. Powers. 2008. a theme-Based seminar
on environmental sustainability improves Participant satisfaction in an undergraduate summer research Program.
Journal of Engineering Education 97 (1):95–102. http://www.asee.org/publications/jee/PaPers/display.cfm?pdf=970.
pdf&special_issue=970
[43] Mcdonough, w., and M. Braungart. 2002. Cradle to Cradle: Remaking the Way We Make Things. New York: North
Point Press.
[44] white, r., and r. gunstone. 1992. Probing understanding. New York: Falmer Press.
[45] Creswell, J. w., and V.L. Plano Clark. 2007. Designing and Conducting Mixed Methods Research. thousand oaks,
Ca: sage Publications.
[46] denzin, Norman K. 1978. the Logic of Naturalistic inquiry. in Sociological Methods: A Sourcebook, edited by
N. K. denzin. New York: Mcgraw-hill.
[47] Morse, J.M. 1991. approaches to Qualitative-Quantitative Methodological triangulation. Nursing Research
40:120–123.
[48] strauss, a., and J. Corbin. 1998. Basics of Qualitative Research: Techniques and Procedures for Developing
Grounded Theory. 2nd ed. thousand oaks: sage.
[49] robson, V.e., V.K. Lohani, and J.a. Muffo. 2008. assessment in engineering education. in Assessment in Engineer-
ing Programs: Evolving Best Practices, edited by w.e. Kelly. tallahassee, FL: association for institutional research.
[50] Boden, daniel, and Maura Borrego. in review. disciplinary department structure and related institutional
Barriers to interdisciplinary research.
[51] Besterfield-sacre, Mary. 2008. personal communication, 7/29/2008, 7/29/2008.
[52] thaiss, Chris J., and therese M. Zawacki. 2006. engaged writers and dynamic disciplines: research on the
academic writing Life. Portsmouth, Nh: Boynton/Cook Publishers, inc.
26 WINTER 2009
advancEs in EnginEEring EducaTion
using concept Maps to assess interdisciplinary integration of green
Engineering Knowledge
AuThoRs
Maura borrego is an assistant professor of engineering education at Virginia tech. dr. Borrego holds an
M.s. and Ph.d. in Materials science and engineering from stanford university. her current research interests
center around interdisciplinary collaboration in engineering and engineering education. she was awarded a
Career grant from NsF to study interdisciplinarity in engineering graduate programs nationwide. address:
McBryde hall (0218), Blacksburg, Virginia 24061; telephone: (540) 231-9536; email: [email protected]
chad b. Newswander is a Ph.d. Candidate at the Center for Public administration and Policy at
Virginia tech. his research focuses on organizational theory, governance issues, and public-private
partnerships. address, 104 draper road, Blacksburg, Va 24060; email: [email protected]
Lisa d. McNair is an assistant professor of engineering education at Virginia tech and co-director
of the Vt engineering Communication Center (VteCC). she is co-Pi on several NsF-funded projects
that explore issues of learning, practicing and teaching interdisciplinarity in university and industry
settings. her primary research interests are communication and collaboration in interdisciplinary
and distributed settings, institutional structures that encourage transformational learning, and the
possibilities of liberal education in engineering programs. address: McBryde hall (0218), Blacksburg,
Virginia 24061; telephone: (540) 231-1144; email: [email protected]
sean McGinnis is the director of the green engineering Program at Virginia tech. he has
undergraduate degrees in both Chemical engineering and Materials science from the university
of Minnesota and a Ph.d. from stanford university in Materials science and engineering. he had
10 years experience in corporate research and development before transitioning his career back to
higher education in the area of sustainability. his current educational and research interests focus
on the environmental impacts of engineering practice, including life cycle analysis, as well as in-
terdisciplinary education using sustainability as a theme. address: holden hall (0237), Blacksburg,
Virginia 24061; telephone: (540)231-1446; email: [email protected]
Marie c. Paretti is an assistant professor of engineering education at Virginia tech, where she co-directs
the Virginia tech engineering Communications Center. her research focuses on communication in engineer-
ing design, interdisciplinary communication and collaboration, and design education. she was awarded a
Career grant from NsF to study expert teaching practices in capstone design courses nationwide, and
is co-Pi on several NsF grants to explore interdisciplinary collaboration in engineering design. address:
McBryde hall (0218), Blacksburg, Virginia 24061; telephone: (540) 231-1812; email: [email protected]