Embed Size (px)
Citation preview
1
Reports from The Panel Sessions 2011 NSF Engineering Education Awardees Conference March 13-‐15, 2011
Table of Contents Overview of The Panel Sessions................................................................................... 2 Panel Facilitator Bios ................................................................................................... 4 The Panel Session Summaries ...................................................................................... 7
Engaging Undergraduates in Research: Best Practices for Bridging Research and Practice .................................................................................................7
Pre-‐College Outreach and Curriculum Partnerships: Best Practices for Bridging Research and Practice .................................................................................11
Engineering Education Research Directions: Where Are We Going? ........................16
NSF’s New Data Management Policy: A Conversation for Engineering Education Research ....................................................................................................22
Increasing Diversity: Best Practices for Bridging Research and Practice ...................28
Graduate Students and Programs: Creating an Emerging Community of Practice for the Next Generation ...........................................................................33
Educating Engineers to Be Innovators........................................................................36
Interdisciplinary Collaboration: Helping Students and Faculty Work Across Boundaries ......................................................................................................41
2
Overview of The Panel Sessions The 2011 NSF Engineering Education Awardees meeting agenda was based on two primary goals: 1) to provide a networking venue in which principal investigators could share their research, and 2) to facilitate discussions focused on issues in engineering education that could be impacted by research and by the practices of the National Science Foundation. In order to achieve the second goal, Panel Sessions were organized with the following charge: In response to increasing calls for accountability across all levels of government, the National Science Foundation Engineering Education and Centers (EEC) division seeks to use the annual Awardees Meeting to gain valuable feedback from its PIs to insuring that research funding is effectively targeting critical issues and achieving transformative impact. Toward this end, these panels have been created as a forum for awardees to collaboratively generate insights and provide feedback to the EEC division about these critical issues in ways that can inform investment priorities, evaluation methods, and reporting mechanisms. They represent an opportunity for the NSF program officers and division leaders to learn from the PI community about what the current system (including solicitations as well as reporting methods) does and doesn’t capture regarding the impact and future direction of NSF investments. This report presents summaries of these discussions—the “Panel Sessions”—in which PIs and NSF personnel critically engaged in the following issues:
1. Best Practices for Engaging Undergraduates in Research: Bridging Research and Practice
2. Best Practices for Pre-‐College Outreach and Curriculum Partnerships: Bridging Research and Practice
3. Engineering Education Research Directions: Where are we going?
4. NSF’s New Data Management Policy: A Conversation for Engineering Education Research
5. Best Practices for Increasing Diversity: Bridging Research and Practice
6. Graduate Students and Programs: Creating an Emerging Community of Practice for the
Next Generation
7. Educating Engineers to Be Innovators
8. Interdisciplinary Collaboration: Helping Students and Faculty Work Across the Boundaries
Each panel was facilitated by two awardees and one NSF member, chosen by the conference directors and NSF personnel for their experience and expertise in the topics. Panels were
3
composed of diverse participants covering a range of research, practice and administration in order to elicit a broad and inclusive discussion. Facilitators were encouraged to take individualized approaches to generating discussions based on a few core questions based on barriers, challenges, what it will take to change, and where NSF should focus its efforts:
Critical Issues 1. Promising work, in both practice and research 2. Areas for improvement in the field 3. Priority areas for NSF to invest in (based on #1 & 2)
Methods of Dissemination 1. Ways to evaluate NSF grant outcomes 2. Ways to report NSF grant outcomes 3. Ways to disseminate findings and spur widespread adoption of best practices
Furthermore, facilitators were asked to structure the panels as discussions, not lectures. The panels were held twice, with a different group in the morning and afternoon. Panel sessions ran for 90 minutes and observers were present to help record the discussions. With assistance from the observers who took notes during each panel, the facilitators reported highlights on the second day of the conference via a revolving slide presentation of major points and quotes (http://www.vtecc.eng.vt.edu/2011_NSF_Awardees/ 2011_Panel_Outcomes.pdf). Then, over the next few weeks, they compiled notes and wrote the summaries that are reproduced in this document. Although NSF personnel participated in these sessions as reporters, observers, and co-‐facilitators, the views herein do not necessarily reflect the views of the National Science Foundation. We would like to thank the facilitators, the National Science Foundation, and the participants in the 2011 panels. Additionally, without the observers, we would not have been able to reproduce the outcomes of these panels; thanks to: Cheryl Carrico, Erin Crede, Stephanie Cutler, Kahyun Kim, Jongmin Lee, Jenny Lo, Rachel Louis, Julie Martin, Taylor Martin, Tamara Moore, and Lauren Thomas. This material is based upon work supported by the National Science Foundation under Grant No. 1048815. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
4
The Panel Session Facilitators
Martha Absher is the Assistant Dean for Education and Outreach Programs and serves as the Disability Services Liaison for the Pratt School of Engineering at Duke University. Her work focuses on engineering and science education and outreach programs to undergraduate, graduate, and K-‐12 students. Her special focus areas include: diversity, women's programs, and programs for persons with disabilities. She is PI of the NSF grant “REU Site for Increasing Diversity in Engineering at the Pratt School of Engineering of Duke University.” Fritz Claydon joined the University of Houston faculty in August 1999 after spending 12 years at the University of Memphis. Over the past five years Dr. Claydon's educational interests have revolved around programs to stimulate first-‐year engineering student learning. For the past 15 years, Dr. Claydon's research interests have centered around cardiac mapping and mechanisms of defibrillation. He is PI of NSF grant “REU Site: Innovations in Nanotechnology at the University of Houston.”
Claire Duggan is Associate Director of the NSF-‐supported Center for the Enhancement of Science and Mathematics Education (CESAME) Northeastern University. She is also Director for Programs and Partnerships at the Center for STEM Education at NEU, K-‐12 Outreach coordinator for The Center for Subsurface Sensing and Imaging (CenSSIS), Program Director for the Young Scholars Program, and Project Coordinator for GK12 – PLUS. She is PI on the NSF Grant “RET-‐PLUS (Partners Linking Urban Schools).”
LeAnn Faidley is an Assistant Professor in mechanical engineering at Iowa State University. Her research focuses on magnetically activated low modulus, smart materials that change their stiffness, viscosity, and shape when put in a magnetic field. She is co-‐PI on the NSF grant “REU Pathways to Engineering: A digital REU mentoring manual.”
Vikram Kapila is an Associate Professor in the Department of Mechanical, Aerospace, and Manufacturing Engineering at the Polytechnic University of Brooklyn. His current research interests include absolute stability theory, robust control, periodic and multirate systems, fixed-‐architecture absolute stabilization, stable stabilization, control of systems with saturating actuators, and control of time delay systems. He is PI of NSF grant “RET Site: Science and Mechatronics Aided Research for Teachers (SMART).”
Russell Long is the Director of Project Assessment in the Department of Engineering Education at Purdue and Associate Director of the Multi-‐Institution Database for Investigating Engineering Longitudinal Development (MIDFIELD). The MIDFIELD database is a powerful tool in learning more about the behaviors of students who matriculate to engineering programs. He has twenty years experience in institutional research, assessment, strategic planning, and higher education policy.
5
Sinead MacNamara is an Assistant Professor in the School of Architecture at Syracuse University. She teaches structural engineering courses as well as electives for the College of Engineering. Her research and dissertation subject concerns thin shell concrete in nuclear containment structures. She is PI on the NSF grant “Inspiring Innovation: Merging Pedagogical Paradigms from Engineering and Architecture.”
Robert Martello is an Associate Professor of the History of Science and Technology at Olin College. His background includes a PhD from MIT's Program in the History and Social Study of Science and Technology and a Master of Science degree from MIT's Department of Civil and Environmental Engineering. He has served as the Digital History Annotations and Features Producer for the Sloan Foundation's electronic textbook Inventing America. He is currently involved in the NSF grant “Collaborative Research: Role of Faculty in Supporting Lifelong Learning: An Investigation of Self-‐directed Environments in Engineering Undergraduate Classrooms.”
Ann McKenna is an Associate Professor in the Department of Engineering in the College of Technology and Innovation at Arizona State University (ASU). Prior to joining ASU she served as a program officer for the National Science Foundation in the Division of Undergraduate Education, and was on the faculty in the Department of Mechanical Engineering and Segal Design Institute at Northwestern University. Dr. McKenna received her B.S. and M.S. degrees in Mechanical Engineering from Drexel University and Ph.D. from the University of California at Berkeley. She is PI of the NSF grant “Collaborative Research: The Role of Intentional Self Regulation in Achievement in Engineering.”
Jill Nelson is an Assistant Professor in the Electrical and Computer Engineering Department at George Mason University. Her research lies in statistical signal processing and signal processing for communications. Specifically, her interests include equalization and coding for dispersive channels, iterative detection and decoding, blind equalization, and cooperative detection in multi-‐user communications. She is PI of the NSF grant “Encouraging Innovative Pedagogy through Long-‐Term Faculty Development Teams.”
Michael O’Rourke is a professor of philosophy in Neuroscience and Environmental Science and a Fellow in the Microelectronic Research and Communications Institute at the University of Idaho. His research focuses on critical thinking, philosophical semantics, and interdisciplinary studies. He is PI of the NSF grant “Improving Communication in Cross-‐Disciplinary Collaboration.“ Alice L. Pawley is an Assistant Professor in the School of Engineering Education at Purdue University. She holds an affiliate appointment in the Women’s Studies Program. Her research group, Research in Feminist Engineering (RIFE), is made up of diverse researchers and focuses on exploring feminist research questions to create a more democratic engineering profession by helping engineers and engineering educators to use new analytical tools and frameworks. She is PI of the NSF grant “CAREER: Learning from Small Numbers: Using personal narratives by
6
underrepresented undergraduate students to promote institutional change in engineering education.” James (Jay) Pembridge is a graduate student in the Department of Engineering Education at Virginia Tech. His research focuses on exploring the pedagogy of design, specifically examining how mentoring is used as a pedagogy in engineering capstone courses. Jay is also the president of the Virginia Tech’s student chapter of ASEE, a member of the graduate consortium steering committee, and a member of the leadership board for Virginia Tech’s Graduate Student Assembly. Donna Riley is an Associate Professor of engineering at Smith College. Her work combines methods in engineering and the social sciences to characterize and communicate chemical risk. In 2005 Riley received a CAREER award from the National Science Foundation for implementing pedagogies of liberation, based on the work of Paulo Freire, bell hooks, and others, into engineering education. She is also PI of the NSF grant “E-‐Book Dissemination of Curricular and Pedagogical Innovations in Thermodynamics.” Jennifer Turns is an Associate Professor in the Department of Technical Communication at the University of Washington. Her engineering education work has focused on engineering design learning, knowledge integration, and disciplinary understanding, and has involved the use of a wide variety of research methods including verbal protocol analysis, concept mapping, and ethnography. She received an NSF CAREER grant, "Using Portfolios to Promote Knowledge Integration in Engineering Education." She is also PI of the NSF grant “Promoting Lifelong Learning, Integrated Knowledge, and Professional Identity in Undergraduate Engineering Students Through a Portfolio Development Process.”
Linda Vanasupa is a Professor in the Materials Engineering Department at California Polytechnic State University. Her research focuses on hydrogen fuel cells; design of learning environments that foster engineering solutions that are sustainable; and design of learning experiences for greater retention of underrepresented groups within engineering. She is PI of the NSF grant “Establishing a Distributed Community of Educators to study a Transformational Education Experiment.”
7
The Panel Session Summaries Engaging Undergraduates in Research: Best Practices for Bridging Research and Practice
Facilitators: Esther Bolding (NSF) Fritz Claydon (University of Houston) LeAnn Faidley (Iowa State University) Our findings include four major points of discussion:
Finding # 1: Undergraduate research mentors must have the commitment to continuous interaction with students under their direction and should be carefully selected by the program director to ensure buy-‐in to overall program objectives.
Unfortunately, in many instances those involved as REU Site coordinators learn by trial and error which colleagues are suitable to effectively mentor undergraduate research projects. In an attempt to ensure positive undergraduate research outcomes and minimize uncertainty with colleagues we have the following suggestions.
As early as six to eight weeks prior to the beginning of any research program involving undergraduates, lead faculty (i.e., REU PI’s etc) charged with program coordination should meet with prospective faculty “mentors” to discuss the following: 1) appropriate development of summer projects, 2) selection of a qualified graduate student mentor from each participating laboratory, 3) program goals and details. Additionally, program coordinators should engage in graduate student mentor training. Suggested materials include adapting concepts from the Council on Undergraduate Research publication entitled “Mentoring Undergraduates” [1] and other relevant sources [2-‐4]. Students who have served as mentors in previous years will be invited to share their insights and experiences with new mentors. A list of suggested topics for the Mentoring Training is shown below:
Topic Format Why do undergraduate students matter? Small and large group discussions Expectations: what do you want from your undergraduate, and what do they want from you?
Individual exercise, large group discussion
Working as a team Group activity Time management Mini-‐lecture Communication Group activity Avoiding pitfalls Mini-‐lecture, think-‐pair-‐share How to deal with conflict Think-‐pair-‐share How to deal with varying levels of knowledge and abilities
Mini-‐lecture
8
Additionally, faculty and graduate student mentors must provide individual safety and laboratory training at the program’s onset and will participate in selected undergraduate professional development activities during the research program.
In addition to the selection of mentors, the student participants in the program should also be carefully selected for the specific projects and mentors involved. It is suggested that mentors help to identify the desired skills and student qualities that will contribute to student success on their project. When selecting a cohort of students it is important to build a group of students who can engage in peer mentoring and community building. Students with a variety of experience levels will likely be most successful at building this community. The REU sites are also encouraged to help build the cohort’s identity through social activities, grouped living, and/or online community links.
Finding # 2: An undergraduate research experience should be focused and must 1) have realistic deliverables for the time period allotted and 2) be appropriate for the background of the student.
An undergraduate research experience should be designed to 1) ignite and sustain excitement about the field of engineering, 2) allow students to expand their knowledge about scientific research and engineering career opportunities, and 3) provide resources and methods for subsequently making an informed decision about whether a “research path” is indeed an appropriate one. When undergraduate participants have completed their research experience, they should:
• Understand the basics of their faculty mentor’s ongoing research efforts, how their work contributed to its success, and its practical applications
• Be familiar with the relevant scientific literature for their research project
• Have a basic understanding about how research is carried out and funded
• Have acquired some relevant methodological, technological, and instrumentation skills
• Have developed skills in communicating their research results through a tangible deliverable.
• Feel they are a part of the research network (faculty, graduate students and other undergraduates)
• Have basic knowledge about opportunities and requirements for majoring in engineering
To ensure achievement of these goals, we recommend the following for consideration:
At least four weeks prior to the start of the research experience, undergraduates should be provided with additional information and reading materials by their faculty mentor in order to become familiar with their research project. Once the formal research program begins each student should be closely guided by her or his faculty and graduate student mentor in
9
formulating a detailed problem statement. The mentors and participants will then work together in planning the course of the subsequent investigation. The students should also work with a professional librarian to learn modern methods of conducting a literature search. As the participant gains command of the problem, the advisor will guide him or her toward more substantive inquiries about the particular problem, leading to improved design and revision of the initial experiment. With the mentor’s help, the participant will be expected to make decisions as to the necessary experimental work and data collection to substantiate existing theories or to permit formulation of new theories. The faculty mentor, in these latter stages, will continue to provide ideas and suggestions as needed. The student is expected to independently formulate conclusions from the experimental and theoretical results and to prepare the results for presentation.
Additionally, we believe that in order to be prepared for successful graduate work, undergraduate research participants should not only learn to conduct independent research, but should also be able to effectively communicate their results. Therefore, participants should be required to make oral presentations and prepare a final written report of their research work. Brief written progress reports may also be requested by faculty mentors during the program if such reports are judged to be important as a means of measuring and enhancing the particular student’s progress. Participants should be encouraged as part of their research experience to work toward formal presentations at professional society meetings and to submit their work to student paper competitions and technical journals, as appropriate.
Professional development of the REU students will help them to learn additional skills essential to graduate and research careers. It is recommended that a formalized series of professional development seminars be included in REU site activities. Topics could include technical communication skills (oral, posters, papers, etc.), laboratory documentation, conducting literature searches, engineering professionalism, resume and CV construction, graduate school and fellowship application processes, and research ethics. Including technical talks by peers, graduate student or faculty mentors or others and laboratory and industrial visits are also suggested to help students develop an understanding of the broader technical field and the engineering research profession.
To ensure productive and positive REU experiences formative feedback should be collected from student and mentor participants during the course of the REU. Students should be encouraged to discuss their experiences regularly with peers as well as program directors or coordinating staff that are not directly involved in their research. Mentors should also be asked to assess the program early in its progression so that any issues with mentor/student mis-‐matched expectations can be addressed early in the program.
Finding # 3: Undergraduate research programs (i.e., REU's) should be encouraged to disseminate their best practices and lessons learned through archival journal and conference papers. In parallel, REU’s should also make use of real-‐time dissemination tools such as blogs, social networking sites, etc. to maximize program impact.
10
Finding # 4: Through careful assessment and tracking, the value-‐added and associated deliverables for program participants should be captured and disseminated. Annual NSF highlights should be tied to program outcomes, and it is critical that program directors report their highlights in a timely manner.
The following represent mechanisms for consideration in which undergraduate research experiences improve impact and spur widespread adoption of best practices:
1. Longitudinal study of program efficacy – have a database of past participants.
2. Dissemination of NSF nuggets amongst REU program sites as appropriate.
3. Develop and encourage synergy between program websites, appropriate topical blogs, and social networking though Facebook and YouTube.
4. Encourage formal interaction between aspiring, new, and experienced undergraduate research PI’s.
5. Research site dissemination (separate from research outcomes) on topics of interest regarding the operation of REU sites.
The NSF can have enormous influence through conferences and grant application solicitations to stimulate implementation the items associated with Findings 3 and 4 above.
References
1. Merkel, Carolyn A., and Baker, Shenda M. (2002). How to Mentor Undergraduates. Council on Undergraduate Research, Washington, D.C.
2. Boyd, Mary K. and Wesemann, Jodi L. (Eds.) (2009). Broadening Participation in Undergraduate Research: Fostering Excellence and Enhancing the Impact. Council on Undergraduate Research, Washington, D.C.
3. Handelsman, J., Pfund, C., Lauffer, S. M., and Pribbenow, C. M. (2005). Entering Mentoring: A Seminar to Train a New Generation of Scientists. The Wisconsin Program for Scientific Teaching.
4. Vye, N. J., Schwartz, D. L., Bransford, J. D., Barron, B. J., Zech, L. and Cognition and Technology Group at Vanderbilt. (1998). “SMART environments that support monitoring, reflection, and revision,” In D. Hacker, J. Dunlosky, & A. Graesser (Eds.), Metacognition in Educational Theory and Practice. Erlbaum, Mahwah, NJ.
11
Best Practices for Precollege Outreach and Curriculum Development
Facilitators: Claire Duggan (Northeastern) Vikram Kapila (NYU—Poly) Mary Poats (NSF)
Two facilitators provided brief overviews of their RET Sites (NYU-‐Poly and Northeastern) and suggested strategies for: recognizing research mentors, bringing engineering into the classroom, developing teachers as leaders, showcasing success through video testimonials, and broadening and sustaining RET activities beyond NSF funding. A third facilitator highlighted the TeachEngineering Digital Library for disseminating engineering-‐based lessons produced by RET programs. Following the opening remarks of the facilitators, the session attendees divided themselves into five groups to discuss and suggest best practices from their RET Sites. The results of these discussions are summarized below.
Strategies to engage university faculty in RET experience: RET programs use a variety of approaches to engender faculty participation in the training and mentoring of teachers. A common approach is clear and convincing articulation of benefits to faculty and their research students. Faculty members who have previously mentored teachers understand knowledge, skills, and experiences that teachers bring to their laboratory group; and how their own research students benefit by collaborating and interacting with professional educators. Moreover, RET mentors often leverage their collaborative research experience with teachers to develop authentic and meaningful societal impact statements in their research proposals. For tenure-‐track faculty, the experience of mentoring RET teachers allows them to pilot-‐test educational innovations in preparation for their CAREER proposals. Participation in RET mentoring programs can also lead to additional local support from foundations and industry for workforce development projects. At many institutions RET projects are highly visible and are vigorously sought after by their media and development departments; providing numerous opportunities to RET mentors to showcase their research to diverse stakeholders of the university, e.g., administrators, trustees, alumni, and donors. Some RET Sites have formal mechanisms to recognize research mentors for their participation in the program, for example, announcement in department newsletter or alumni publication. Increasingly, RET PIs are leading the conversation at their universities to have faculty mentors’ contribution to the RET program counted in their annual merit review. For example, at one PI’s institution, an aerospace faculty mentor highlighted the number of Internet hits to her TeachEngineering lessons in her annual merit review and was favorably reviewed. Some programs organize formal mentor recruiting events which showcase prior-‐year faculty mentors’ collaborative research with their RET teachers. Other programs conduct orientation programs for faculty to learn about their role in the program and benefits of participation. Research mentors with prior RET and REU experiences typically champion the program and encourage participation of other faculty.
Strategies to prepare teachers to bring engineering into the classroom: In some RET projects, faculty mentors and engineering students visit their partner teachers’ classrooms to develop an
12
appreciation for their teachers’ classroom context and devise appropriate strategies for infusing engineering into the classroom. Alternatively, other RET projects assess their participating teachers’ initial attitudes, perception, and familiarity with engineering and follow a diverse array of approaches to prepare teachers to bring engineering into the classroom.
First, some RET Sites have structured workshops, with specific learning content, hands-‐on activities, and field trips to introduce teachers to foundations of one or more engineering disciplines.
Second, some RET Sites have developed modules that can be used by teachers to bring pre-‐engineering courses into the classrooms. In both the workshop and module approaches, RET Sites rely on engineering faculty and student assistants to deliver the content to teachers.
Third, some RET Sites use commercially available videos to introduce teachers to various engineering disciplines. For teachers, and even RET personnel, who are just starting out, it may be beneficial to adapt/adopt existing curricula models from successful RET Sites.
Fourth, at some RET Sites, teachers research various engineering disciplines and teach it to one another, thereby preparing the entire cohort of teachers to bring engineering into the classroom.
Fifth, teachers of some RET Sites work on developing classroom lessons and activities that are aligned with curriculum standards and illustrate some component of their summer research experience. This allows for making engineering connections in science and math courses and illustrates the real-‐world relevance of classroom learning.
Sixth, teachers of some RET programs create short videos of their engineering research and its broader impact. These videos are used to introduce their students to engineering.
Seventh, some RET Sites encourage their teachers to bring engineering into classrooms through lessons on the process of engineering design and engineering research.
Eighth, many RET teachers showcase their summer research experiences and their successes in bringing engineering into the classroom to other teachers, thereby encouraging best practices of RET projects in the classrooms of non-‐RET teachers. While some RET Sites hold formal dissemination workshops to do so, in many cases teachers lead professional development meetings at their schools or districts and in some cases they present at local, regional, and national science-‐, math-‐, and technology-‐focused teacher conferences.
Finally, in one illustration of successful classroom implementation, a PI described, among others, a school-‐wide engineering curriculum at an inner-‐city high-‐school that currently involves over 260 students in three engineering classes in grades 10-‐12.
RET teachers as change agents: Some RET teachers continue their research collaboration with their university-‐based research team well after the summer program ends. While initially these teachers may have participated in RET Site projects, their continued focus and interest in engineering research has led faculty research mentors to bring them back under their own RET
13
supplements. These sustained interactions often benefit the development, mentoring, and training of undergraduate and graduate engineering students, who gain a deeper appreciation of pedagogical concepts, societal relevance of their research, and connections of their engineering studies to K-‐12 STEM concepts. In addition, RET participants are assisting in the re-‐design of professional development courses at partner universities and serving as co-‐instructors for engineering and other STEM course offerings.
Having had a first-‐hand experience with modern tools and technologies used by engineers, many RET teachers become a resource to other teachers in their laboratory and facility development efforts. Many RET teachers have written mini-‐grants to philanthropic foundations, local corporations, online grant-‐writing sites, and school district programs such as VATEA, to raise funds for integrating their RET experiences in their own classrooms. Some RET teachers have also written lessons, units, and entire pre-‐engineering curriculum for local schools. Such dedicated efforts allow RET teachers to become ambassadors of engineering education in their schools, where they recommend participation in RET programs to other teachers, inspire their students to pursue careers in engineering, and prepare them to study engineering in college.
RET teachers have frequently authored/co-‐authored articles in technical publications, teacher literature, and TeachEngineering. For the last few years, the National Science Teachers Association (NSTA) Annual Conference has hosted special sessions where RET teachers showcase their engineering research and lesson plans to a national audience of science teachers and administrators. These experiences allow RET teachers to become instructional leaders and disseminate their knowledge of engineering disciplines, research, and tools to other teachers in their schools and school districts. Some RET teachers have transitioned from teaching to administrative roles as department chair, assistant principal, and even school district administration, in which they continue to champion a significant role for engineering in K-‐12 STEM education.
Interactions with RET teachers have sensitized university faculty to the K-‐12 environment and culture, curricula standards, and teacher professionalism. Thus, when RET research mentors write engineering research and education proposals, they frequently call upon their mentee teachers to serve as informed and authentic partners. In fact, RET teachers have also been catalysts in proposing concepts and designs of some K-‐12 engineering education proposals by university-‐based faculty. Moreover, in many instances, RET teachers are serving as a bridge to connect faculty from engineering and education schools at universities to spur new partnerships.
Evidence of transformative impact and success for RET programs: RET experiences have been transformative for both the teacher participants and faculty research mentors. Specifically, many RET teachers report that participation in the RET program has allowed them to better understand the engineering profession. The experience has allowed teachers to view themselves as learners again. They are now able to exploit the real-‐world appeal of engineering concepts, problems, and illustrations in teaching K-‐12 science and math concepts. Working in an intense research environment, where all participants are treated as professionals and are expected to make original intellectual contributions, tasting the joy of success—sometimes
14
having failed on first attempt, has led teachers to a confidence in their own abilities and has reinvigorated their teaching. Similarly, the experience of mentoring teachers has transformed the way engineering faculty view and value education of their own undergraduate and graduate students. The engineering faculty now have a better appreciation of challenges and opportunities in K-‐12 STEM education systems and are engaging with their surrounding communities in collaborative educational outreach efforts.
RET programs have been successful on a variety of fronts. For example, RET teachers’ students have decreased failure rate and higher grades. Moreover, RET teachers themselves and their students experience greater excitement and engagement in teaching and learning. RET teachers have been able to engender greater interest in engineering among their students some of whom have gone on to receive scholarships to study engineering at the college level. RET teachers have been able to affect institutional change as well. Specifically, schools have been appreciative of teachers’ efforts and have allocated increased resources and budgets to promote engineering activities in the classroom. For faculty research mentors, experience in RET projects has led to their participation as consultants on a variety of education and evaluation projects. Faculty mentors are also supporting their departments and institutions in assessing student learning. Engagement in RET experiences have led some higher education institutions to allocate budgets to create K-‐12 STEM Education Centers/positions. This has often been the result of university faculty’s advocacy for greater involvement of university faculty and researchers in K-‐12 STEM education.
Many RET programs use pre-‐ and post-‐program surveys to assess the effect of their education, training, and research model. Some programs adapt survey instruments from RETNetwork.org while others retain external evaluators to design and conduct program assessment. In one approach, teachers are surveyed and given a technical quiz (pre-‐ and post-‐program) to determine their gain in familiarity with skills, concepts, and devices used in the summer program. Often project evaluators conduct formal surveys and observations of teachers and their research mentors. Portfolios, consisting of research project reports, lesson plans, formal presentations, research posters, and websites are also often used to assess teacher activities. Teachers provide mid-‐ and end-‐of-‐program evaluation of the efficacy of project activities for mid-‐course correction and future enhancements. Teachers also provide feedback on student evaluation of their RET-‐based lessons and activities. In some programs engineering students and faculty mentors visit their partner teachers’ classrooms to observe their implementation of engineering-‐based lessons and activities. Annual workshop activities, participating in the NSTA meetings, sharing of lesson plans, dissemination through TeachEngineering, etc., provide additional means to assess the transformational impact and success of RET programs.
Sustaining RET programs post NSF funding: Post NSF support, RET programs may be sustained through following alternative funding streams: (1) partnership with non-‐profits, local businesses, and corporations; (2) grants from city, state, and federal departments of education; (3) “support an RET teacher” campaign through university’s alumni and development offices; (4) become a city/state certified entity to offer professional development programs to teachers or conduct a fee-‐based program; and (5) mini-‐grant proposals to provide curriculum kits to teachers for conducting engineering activities. RET Sites may also consider forming multi-‐
15
university partnerships to target larger sources of funding at the regional and national level, e.g., scale-‐up grants to the U.S. Department of Education’s Investing in Innovation (I3) program, evidence-‐based replicability grants to NSF’s ITEST program, and education research studies on successful models of RET Sites for national scale-‐up and dissemination through NSF’s DR K-‐12 program, among others. However, for RET programs to develop successful sustainability efforts, it is vitally important that during their NSF funding these programs document their successes along multiple dimensions (teacher engagement, faculty satisfaction and recognition, classroom implementation, and student impact) and raise the visibility of their projects in their institutions, school districts, and local communities through media campaign and compelling video documentaries.
16
Engineer Education Research: Where are we going?
Facilitators: Alan Cheville (NSF) Robert Martello (Olin) Linda Vanasupa (Cal Poly)
Crystallized Outcomes
1. Research that simultaneously informs and transforms engineering education practice;
2. Research that enables innovating for socially responsible aims;
3. Research that leverages existing knowledge, technologies and educational communities to individualize learning.
Executive Summary and Facilitators Reflections
The attendees were mostly faculty grantees of NSF funding for engineering education research. As a result, important voices were missing from the conversation such as administrators and students, groups with deep interests in engineering education research. A very interesting result was that we collectively recognized that the current states of engineering education and engineering education research look a great deal like what we would design if our intent was to utterly destroy each of them within ten years.
In general, the crystallized outcomes reflect a recognition that for both engineering education and engineering education research to thrive, they must serve the whole of society—that the artificial boundaries that we create between disciplines, peoples, nation states, impede our ability to fully “place service before profit, honor and standing of the profession before personal advantage and the public welfare [i.e., health, happiness and good fortunes] above all other considerations.” (Engineers’ Creed, National Society of Professional Engineers, 1954). The crystallized outcomes also reveal our belief that for high impact, engineering education research must be done in a way that operationalizes change within the engineering education community toward our aspired state. There exists a great deal of knowledge from the social sciences on learning and change in human systems; for high impact, engineering education researchers must collaborate across traditional boundaries to take advantage of what is already known. These types of transdisciplinary collaborations will also enable us to better serve the diverse student learning needs across all of society through individualized learning.
Overview of process and framework used to report the results
We ran two sessions, in the morning and afternoon of March 14, 2011, lasting 90 minutes each. The first session featured three "rounds" of group design activities, and the second session used two slightly longer rounds. We formed four-‐ to eight-‐member groups for these activities, and they shared the results of their work either by taking notes and reporting out to the whole group, or by writing ideas on notecards and grouping them into coherent categories on a stickywall™.
17
We began each session by setting the context for the conversation. We chose to reference the National Academies Grand Challenges for Engineering in the 21st century as the context for 2 engineering, engineering education and engineering education research. We spent about two minutes describing the four broad areas of human concern in which the Grand Challenges are organized: Health, Sustainability, Vulnerability, and The Joy of Living. We stated that the NAE acknowledges that the challenges are those that affect all of humanity in our shared and interconnected fate.
We then asked the participants to engage in a 2-‐3 minute silent meditation where they considered what the future would look like in 10 years. We asked the morning session participants to think about the future in broad terms, and refined the question for the afternoon attendees who instead imagined "the future of engineering education." After this meditation, we asked the audience members to share in pairs what they saw as the future. This exercise lasted a total of about 10 minutes. We then took them through some rounds of design in which they responded to design questions that elicited the aspirational states they have for engineering education research. We are using Aristotle’s four types of causality as a framework for describing the results of the workshop. They are:
Final cause: The "end," the goal or intent of the system or thing;
Formal cause: The form or design of the system or thing;
Efficient cause: The process used to create the system or thing;
Material cause: The physical substances comprising the system or thing.
It is important to note that the Final cause is the high-‐leverage point in any system. This is
the zone of paradigm shifts; all other decisions about design, processes, and natural capital follow from the intent. The bottom half of the above chart focuses on “objects” while the top half emphasizes relationships.
Session#1: Three rounds of design activities focused on engineering education 9:15-‐10:45 AM, 60 attendees
Round #1 Prompt: If you were to design an engineering education system that would be a complete disaster in 10 years, what would it look like? How would it function?
Generalized Outcomes (sorted into four causality categories): • Intent: Create an engineering education system that serves and represents the interests
of a subset of society, rather than the whole (e.g., one single socioeconomic demographic or one “customer,” such as “industry.”)
18
• Form: Abstract engineering education from the real world (e.g., ignore contemporary developments, remove human elements); Segregate and fragment engineering education at all levels in all ways (e.g., race, gender, socioeconomic status, disciplinary focus); Incentivize faculty performance in ways that fixate their attention on other things (e.g., 100% of salary tied to level of external grant funding); Remove accountability to professional bodies.
• Process: Create completely prescriptive curricula without freedom of choice (one size fits all); Advertise engineering to potential students as a really difficult subject with lots of math and science and highly competitive; Use a grading process of only two high-‐ stakes assessments (e.g., mid-‐term, final); Classroom experience is purely a transfer of information that occurs through “death by PowerPoint.”
• Content: No examples; More pencil/paper problems that students must complete; 100% non-‐human interaction in teaching; no projects; no office hours.
Reflections on Round#1: We all noticed, to our great amusement, that the output of this exercise bore a striking similarity to the current engineering education system. In general, participants seemed to determine the most destructive course of action in one of two ways: they either identified and magnified familiar educational elements, or listed an idealized course of action (e.g., "increase diversity in student populations”) and then reversed it (e.g., “prohibit diversity…”). We believe that this first round successfully helped the workshop audience to stretch their creative muscles, and it nicely set up the next question. Most groups formed their answers to our question with a great deal of consensus: it was not difficult for them to agree upon the pathway to destruction.
Round #2 Prompt: If you were to design an engineering education system so that it was wildly successful, what would it look like?
Generalized Outcomes (sorted into four causality categories): • Intent: Focused on innovating for socially-‐responsible aims (e.g., positive learning
cultures that welcome and celebrate diversity in all its forms; creativity taught and fostered; integration of sustainability, global issues, systems thinking, community outreach and community-‐based service learning);
• Form: Engineering learned in an integrated, holistically engaging environment (e.g., the learning includes students, faculty, staff, community, many disciplines, authentic projects); Individualize learning (e.g., autonomy support, self-‐directed, student-‐ centered).
• Process: System rules that that were congruent with the espoused value of teaching (e.g., reward system for teaching)
• Content: Public image campaign to increase the appeal of engineering (e.g., CSI for engineering, engineering content in high schools)
19
Reflections on Round #2: As with round one, we did not observe a great deal of disagreement between team members during this exercise. The members of most groups found ways to collaborate productively and one of their biggest challenges was paring down their ideas to a small number that could then be captured on notecards and posted on a stickywall™.
These sample outcomes can be considered with three frames:
Personal-‐How am I participating in ways that contribute to the current state?
Systemic-‐What are the dynamics at play that contribute to the current state?
Perceptual-‐What are the misperceptions that contribute to the current state?
As we discussed ways to change engineering education, the personal frame of consideration was often missing. That is, if you were asking “How can we fix the system?” the answer included ways to benevolently manipulate the system for more positive outcomes, such as changing the pedagogy or launching a marketing campaign. While there was some thought of individual students, there was little reflection on the role played by faculty agents within the system. Given that the audience was primarily made up of faculty, faculty apparently don’t see themselves as part of the system… or don’t believe they are a part of the system that needs to be changed.
One of the principles of change processes is that if there is focus on the perceptual levels and systemic levels without significant attention to personal levels, the change initiatives will be superficial and impermanent. Attempts to address the perceptual will occur to others as “candy coating” through public relations; attempts to address the systemic will occur as manipulation to those who are the object of the fix (e.g., K-‐12 teachers who are told to put greater emphasis on math and science). The source of both systemic and perceptual dynamic system behavior reside at the personal level, so changing systemic and perceptual leaves the source of these dynamics in place to recreate the problematized phenomena once the force applied to the systemic and perceptual levels is removed. So, the absence of the personal level of reflection in the dialogue predicts a conserving of the past behavior of the system. It also illustrates our tendency to believe the problematized phenomena of engineering education, such as chronically low ethnic and gender diversity, are outside of us, rather than something in which we are actively and causally participating.
Round #3: Prompt: Now that we have this picture, what do we need to know in order to bring about our aspired state for engineering education?
Generalized Outcomes:
• Questions around creativity: Can it be taught? Can it be measured? How do you ignite it?
• Questions around change: What is the process of creating change in engineering education? What are the perceived and real obstacles to change? How do we change to a culture of integrating engineering education research into practice?
20
• Questions around tools: How do we use technology to support our educational goals? How do we define or measure student success?
Reflections on Round #3: Everyone was quite tired at this point and we were also short on time. We expected the morning session results to be the platform for the afternoon session results, so we collected as much information as possible from the attendees and did our best to interpret it before the session ended.
Session #2: 2 rounds of design activities focused on engineering education research 2:45-‐4:15 PM, 40 attendees
Round #1 Prompt: If you were to design an engineering education research system that would be a complete disaster in 10 years, what would it look like? How would it function?
Generalized Outcomes (sorted into four causality categories):
• Intent: Focus on differentiating and legitimizing engineering education research as a unique discipline (e.g., fund only studies of abstract concepts; privilege theory over practice; privilege technical, quantitative over non-‐technical, qualitative; have only one journal for scholarly work that accepts nothing or accepts only the work of a few elite “experts”; insist that engineering education is something fundamentally different than education, requiring it to be studied by only engineers).
• Form: Have the form of engineering education research replicate the expert model of other disciplines. (e.g., Establish an implicit hierarchy, where an elite set of researchers and programs perspectives are more valued than emergent perspectives and important voices (e.g., student voices) are missing from the dialogue; Adopt a competition model for the research community rather than a collaborative model; Create a competitive system of funding research in which the parameters of that system are defined and judged by authorized “experts”—principle: to those who have, more shall be given).
• Process: Forgo all standard research processes (e.g., Ignore IRB issues; Misuse statistics; Base decisions only on gut feeling; Disseminate results only by tweeting; Forgo data collection—go with prefabricated results; Publish only results that reinforce the researcher’s prior beliefs; Do not publish “failures” or “negative” results; Don’t examine others’ work; Don’t collaborate).
• Material: Only use surveys or anecdotal data; Eliminate all funding.
Reflections on Round #1: As before, the audience immediately realized (and laughed about) the similarity between this activity and the current engineering education system. However, this time the deliberation was not one of complete consensus: some team members disagreed about whether certain aspects of engineering education research were positive or negative, such as the inclusion of IRBs. If anything we felt that the afternoon's first round was even more productive than the morning one, as it started some helpful discussions about the goals and methods of engineering education research while also inspiring attendees to indulge their creativity.
21
Round #2 Prompt: If you were to design an engineering education research system that would be wildly successful in 10 years, what would it look like? How would it function?
Generalized Outcomes (sorted into four causality categories): • Intent: Focused on addressing emergent challenges with meaningful, global societal
connections
• Form: Collaborative across disciplinary boundaries with transparent and open access to data and tools. (e.g., fundamentally transdisciplinary, open access assessment tools; open access to research data).
• Process: Research would be integrated with practice in a way that transforms and improves engineering education praxis and is thereby systemically valued (e.g., conducted in ways appropriate to change personal, systemic and perceptual levels of system behavior; institutional reward systems for EER and teaching; faculty would have a better understanding of student learning; learning environments that enabled deeper learning by students of all learning styles; teachers would also be researchers of their own praxis).
• Material: Theoretical foundations (e.g., grounded in theory, appropriate social science research and change methodologies & effective models).
Reflections on Round #2: Again, the round two deliberations invoked some disagreements as team members approached larger questions from different perspectives. Also, as in the morning session, participants tended to emphasize systemic considerations more than other types, with personal questions seldom, if ever, appearing. The increased emphasis upon educational research in round two produced a corresponding focus upon the appropriate methods of educational studies (for example, the need to consider interdisciplinary approaches and provide open access to research data) along with some talk about the goals of these studies.
22
NSF’s New Data Management Policy: A Conversation for Engineering Education Research
Facilitators: Russell Long (Purdue University) Russ Pimmel (NSF) Donna Riley (Smith College)
Overview of NSF’s new Data Management Policy
NB: The NSF-‐wide policy can be found at http://www.nsf.gov/bfa/dias/policy/dmp.jsp, with guidance from the Engineering Directorate at http://nsf.gov/eng/general/ENG_DMP_Policy.pdf and general Frequently Asked Questions at http://www.nsf.gov/bfa/dias/policy/dmpfaqs.jsp.
Producing a Data Management and Sharing Plan creates a place to articulate and formalize decisions we already make about data on our research projects. Thinking through this aspect of the work at the time of the proposal may improve the quality of the research plan and streamline data management later on in the project.
One of the most important points is clarification of the term “primary data” used in the NSF document. The Engineering Directorate’s guidelines spell out that “primary data” does not refer to preliminary or raw data, but rather to analyzed data. These might be in the form of graphs or tables – however the data are best presented.
The overview prompted several questions among the group:
1. What does the primary data “necessary to validate research findings” mean in qualitative work?
This question did not have an easy answer. Qualitative researchers approach data analysis as interpretive, and while sharing quotes that illustrate or form the basis for research findings, it is difficult to think of them as “validating” those findings. This phrasing requires some re-‐interpretation for the qualitative research community.
2. When you sign a copyright form for journal publication, you are prevented from putting data on a publicly available website. How can we address this conflict?
The group offered several different answers to this question. One person felt that in practice, this doesn’t matter. Another person suggested negotiating copyright with publishers in accordance with the Data Management Plan. Another person suggested that data sharing need not necessarily take place via a publicly available website, and that other methods might allow one to be in compliance with both the Data Management Plan and a publisher’s copyright agreement. Donna Riley noted that the NIH Public Access requirements (http://publicaccess.nih.gov/) make all publications based on NIH research publicly available, superseding copyright agreements and fundamentally changing the way NIH-‐funded scholars communicate. However, Russ Pimmel offered that the NSF is not as large an agency and was not likely to have similar leverage.
23
3. Is the period of data retention three years? What if your grant expires?
Data Management continues after the grant expires. Russell Long shared his practice of not destroying data until 10 years after the last publication (which can also occur after the grant expires).
4. What if your Data Management Plan changes?
Small changes can be made without consulting the program officer, but for significant changes, one should approach the program officer to discuss.
5. Will proposals with lower levels of data sharing be reviewed more negatively than those that can be completely open?
This question created a great deal of discussion because of the potential “chilling effect” on work that cannot openly share data, for whatever reason. Russ Pimmel asked the group to trust the wisdom of our peers in this matter, and that this is an evolutionary process where the community will learn alongside those submitting proposals.
Implications for Human Subjects and other Legal Compliance Issues
Both conversations spent significant time discussing the implications for human subjects research. The main take-‐away was that human subjects protections should come first, and the Data Management Plan tailored to fit what Human Subjects Protections would require, rather than the other way around. The same rule applies to other legal agreements, whether they are intellectual property agreements, protecting student privacy through FERPA, memoranda of understanding, or other contracts. Ultimately NSF defers to institutions on how they govern IRB, chemical safety, animal protocols, and the like. NSF will not police data management, but is looking for investigators to create a thoughtful plan for their specific research and institutional setting.
Some qualitative researchers were particularly concerned about sharing data. First, in some cases, particularly when students discuss identity or when the focus is on underrepresented groups, it is difficult to determine what content may identify the individual. When identity is the focus of study, primary data may focus on what it means to be a woman, a student of color, a person of a particular ethnicity, an lgbtq student, or a student with a disability in engineering. In those cases, sharing data may identify the student. Even if identifying information can be removed, this extra step could have significant cost implications for researchers who need to pay staff to accomplish this with interview data. There would have to be tradeoffs in reduction of sample sizes or other cuts in a project budget.
One attendee asked whether it is necessary to alter an IRB consent form that informs participants that results might be published. While the simple answer was that yes, forms need to change, it raised the further question of what would constitute meaningful informed consent when we cannot necessarily predict how data will be used once it is shared. Another concern raised by attendees related to others’ interpreting data shared out of context and not fully
24
understanding the full story in a given study. Some kinds of data should not be used by others in secondary analyses.
Data Sharing and Data Archiving
At the same time, people who work with other kinds of data saw great value in sharing data. The possibility for sharing interview protocols, and standardizing data collection across studies could provide powerful resources for future analysis. Sharing data can provide real data sets that can be used for training and practice with graduate students, or for junior faculty development. Other people coming to a data set with new eyes, perhaps in a new context, may make new discoveries or have new insights that bring the state of knowledge in the community forward.
Russell Long spoke about the value and importance of data archiving. He detailed the loss for the engineering education community when data are not archived and lost through moves, file corruption, and other processes. He also provided information on services that can be hired to archive data. One of these is the ICPSR (Inter-‐University Consortium for Political and Social Research) based at the University of Michigan ( http://www.icpsr.umich.edu/). This network of about 700 institutions supports best practices in data access, archiving, and analysis, and maintains a large archive for storing and (where appropriate) sharing data. Online security is an issue with data storage, and relying on professional services may be more secure than creating something from scratch.
Russell Long provided a sample data management plan from his work. [see Appendix 1]. In this sample plan, raw data are not shared due to legal requirements around protecting student privacy. The plan includes sections on data collection, data protection, data availability, and data destruction.
Summary of Emergent Discussion
The group provided the following summary of our conversations:
1. How do we ensure that data management plans conform to legal and ethical
requirements (e.g., IRB, FERPA, proprietary agreements, patents, MOUs, etc.)?
• Protecting individuals is paramount.
• There are legitimate reasons for not sharing data.
2. We need ongoing discussion among researchers, reviewers, and NSF to set appropriate norms that balance dialogue among researchers, protection of human subjects, and the integrity of research methods.
3. NSF needs to make clear the distinction between “primary data,” “preliminary data,” and “analyzed data.”
4. Will NSF discriminate against research proposals that deal with more sensitive data?
25
• Will proposals that say “cannot present data because of confidentiality” be considered weaker than those that say “all will be made available”?
5. What does “the recorded factual material … necessary to validate research findings” mean?
• The line is fuzzier for qualitative work.
Appendix 1: Example Data Management Plan
Understanding Diverse Pathways: Disciplinary Trajectories of Engineering Students Data Management Plan
Data Collection No new quantitative or qualitative data will be collected for this project, outside of the regular updating of the Multiple-‐Institution Database for Investigating Engineering Longitudinal Development (MIDFIELD). This project will analyze existing MIDFIELD data. The MIDFIELD team maintains the database on a secure computer in secured facilities. The computer is not networked or connected to the Internet. Member institutions transmit data to the MIDFIELD data steward via password-‐protected, encrypted DVDs sent via registered, next-‐day FedEx. These DVDs are stored in a locked filing cabinet in a secure office. Only the MIDFIELD data steward has access to these discs. Student identifiers are created especially for MIDFIELD – they are not Social Security Numbers or student IDs. MIDFIELD data has been cleaned and verified. MIDFIELD is backed up weekly.
MIDFIELD has binding agreements with each partner institution forged through Memorandums of Understanding (MOU). These MOU protect the confidentiality of both students and institutions. Reports aggregated by student and institution will be made available. If institutional identification is needed (e.g., to study policy differences), they will be conducted by the MIDFIELD team.
The MIDFIELD team obtains confidentiality agreements from all those who have access to the data, including those who only see aggregated data. No individually identifiable data are released. Data from partner universities are placed in a common format, so MIDFIELD can be used for cross-‐institutional studies. The common format further protects the identity of students.
Memorandums of Understanding and Researcher Confidentiality Agreements are stored in a locked filing cabinet in a secure office.
The student focus group electronic data is stored on a password protected computer. Focus group notes are both in electronic and paper formats. Paper formats are locked in a filing cabinet in a secure room.
New Data Generated To help the Project Team understand the trajectories taken by women and underrepresented
26
minorities through the curriculum of specific engineering majors the MIDFIELD data steward will produce password protected Microsoft Excel spreadsheets, aggregated by the variables to be studied. These spreadsheets will not contain individually identifiable student records. The spreadsheets will be transmitted via secure FTP (FileLocker) and stored on password protected computers.
Quantitative Data Analysis and Reporting MIDFIELD will be analyzed using SAS.® SAS output will be converted to password protected Excel spreadsheets. All SAS programs will be archived and available for examination to ensure that proper assumptions were made when querying MIDFIELD. Findings will not be linked to specific institutions, populations, and policies. Conclusions will show student trends and explore institutional variability, but stop short of exploring the causes of institutional variability, except in limited cases when we are able to speculate without compromising these three important principles:
1. Institutional data are provided to the MIDFIELD project on the condition that researchers using the data protect the identity of the partner institutions and each institution’s students.
2. Increasingly specific institutional descriptions discourage readers from considering this work to be generalizable, in spite of other significant evidence that there is much that is common among engineering programs and their interaction with students.
3. While this study includes data for very large numbers of students, only eleven institutions are represented, so institutional variation is treated using a case study approach. Conscientious institution-‐level analysis would require a large number of diverse institutions. MIDFIELD does not meet this standard yet.
Qualitative Data Analysis and Reporting Focus groups of women engineering students were conducted in the spring of 2009 as part of another project. This project will analyze the responses of Chemical Engineering students using standard analysis and coding, looking for themes. Only students' first names are used during analysis and any other data that includes their last names (e.g. receipts for their honoraria) have already been destroyed. Institutional variation will treated using a case study approach. No names will be used in reporting.
Destruction of Electronic Data All Excel files will be deleted ten years after the project’s final publication. All SAS programs will be archived and available for examination for up to five years after the project’s final publication. All electronic focus group notes will be destroyed ten years after the project’s final publication.
27
Destruction of Paper Data All paper and electronic focus group notes will be shredded and safely disposed of ten years after the project’s final publication.
Making MIDFIELD Raw Data Available to Other Researchers Because MIDFIELD data is bound by MOU that contracts specific usage, MIDFIELD raw data cannot be deposited into a public database. Putting MIDFIELD together was, and continues to be, an exercise in trust. MIDFIELD is able to collect and analyze student transcript record data through an exception in the Family Educational Rights and Privacy Act (20 U.S.C. §1232g, (B)(1)(f)) that allows institutions to provide student data to “organizations conducting studies for, or on behalf of, educational agencies or institutions for the purpose of … improving instruction, if such studies are conducted in such a manner as will not permit the personal identification of students and their parents by persons other than representatives of such organizations and such information will be destroyed when no longer needed for the purpose for which it is conducted.”
In an ideal world MIDFIELD would be public and made available to all researchers. Currently, MIDFIELD needs to remain available only to a core set of researchers as long as member institutions and FERPA require it to be.
28
Increasing Diversity: Best Practices for Bridging Research and Practice
Facilitators: Martha Absher (Duke University) Theresa Maldonado (NSF)
Alice L. Pawley (Purdue University) Promising Work
With NSF’s funding support, new insights on issues of diversity are arising from both programming and research. Among these insights are those centering on first generation college students, who are increasingly going into STEM areas. Summer bridge programs center on this group in some universities to ease the transition into college and bolster success. Other important underrepresented groups are low-‐income students, students with disabilities, non-‐traditional students, and transfer students, especially from community colleges, and students from dual-‐degree programs (such as 2/3 programs for example, from Morehouse or Spelman and Georgia Tech). With these distinct groups, the idea of community building is seen as important, via programs like one university’s program for graduate women in engineering, or another university’s multicultural computing lunch to include students from multiple colleges and universities to share ideas. This community building strengthens the integration of the group and its integration into the larger engineering community. Community living for women, minorities, engineering students, or other groups is also another approach that some have found to be very successful. Another important issue to many was reaching down into the high school and K12 communities to increase interest in engineering early, particularly in lower-‐income schools.
Suggested strategies to increase diversity included: community building, outreach with the K12 population, summer research opportunities, and reaching teachers. The value of the RET program was stressed, as teachers are needed to serve as catalysts in some communities—this was felt to be a good investment for NSF.
Others discussed the need for different kinds of research to be supported, including research grounded in social science theoretical frameworks, and embracing intersectional (looking at gender and race and other characteristics together rather separately through main effects) and qualitative methods to understand the experiences of small numbers of underrepresented students.
Areas for improvement, and what NSF priority areas should be
The group raised the question of practical justification for the value of diversity by asking: “Why do we do this – besides social justice? How will diversity make us do better engineering practice outside of the social network concept and economics? Some participants felt that we need to collect technical examples that were invented without diverse user groups in mind (such as 3-‐point seatbelts that are not safe for pregnant women to use, or airbags that were not designed for smaller people, including many women and children) to demonstrate the importance of diversity in engineering design. It was felt that we need a collection of examples to show how engineering products will be better with a diverse group of people designing them. Industry
29
knows they need diverse input and are ahead of some universities in this area. Examples will help guide faculty members towards using such examples in their classes, and may also help strengthen faculty buy-‐in to support diversity.
A priority area is NSF’s investment in research and value added of a multicultural and diverse work force. But what is a driver for universities? Industry exerts pressure on faculty for research outcomes.
Some participants talked about our collective need to invest in how to attract students to engineering that we historically have not been able to attract (like people from tribal colleges and people with disabilities, particularly learning disabilities). First generation students need a different kind of support than students whose parents have attended college. Faculty members need training and development to work with diverse populations.
Retention is another issue that needs to be addressed, with a focus needed on pivotal times of schooling—one participant claimed that 4th grade is pivotal in losing or gaining students to STEM areas.
The question of financial aid for non-‐US citizens was raised, as this is a major issue for some colleges, community colleges, and universities. How will NSF programs address this, as many of these immigrants may choose to stay in the US, thereby contributing to US STEM fields?
NSF Priority Areas should include the following areas:
1. K12 teaching and learning-‐ with efforts to achieve systemic and sustainable change in K12 classroom. NSF needs to support sustainable teacher development in and beyond the classroom, not just innovative new programs.
2. Assistance in developing K12 curricula by people who know science and engineering, as many K12 teachers are not specialists in these areas. These new curricula should be obviously aligned with new state standards.
3. Retention: developing new ways for NSF to share retention best practices.
4. Develop longer-‐term support mechanisms from NSF to help develop meaningful, comprehensive, and complex outcomes.
5. Consider mechanisms to support students at community colleges.
6. Support work that teaches students how to overcome stereotype threat and the imposter syndrome.
7. Develop AP tracks (particularly of English and Math) that can help students for whom English is a second language.
8. Explicitly invite proposals that investigate the educational experiences of other minority groups including parents, part-‐time students, LGBT students, immigrant students, and older students.
30
A key take-‐away from this discussion: NSF needs to decide which narrative it wants to communicate and then “walk the talk.” If NSF rewards competitive behavior, it will get competitive behavior such as a “not invented here” behavior. If NSF wants to encourage best practices (hence cooperation), it needs to reward and incentivize cooperation and use of best practices.
Will NSF communicate the narrative of “zero sum game” and only “innovation” will be rewarded? Or will NSF reward competition and collaboration?
Impact and Potential
Meaningfully and fairly evaluate NSF grant outcomes
Meaningful evaluation needs to be long term—it is hard to evaluate real impact in the short term. In addition, evaluation and impact are hindered by issues such as the educational restrictions in each state on research on the K12 population, which are cumbersome. IRB issues are another problem area, and some national NSF support and guidance on how to deal with them would be of great benefit.
Another problem is the problem of self-‐declaration-‐ if a student for instance, does not choose to declare their minority or disability status, what can be done in reporting and evaluating data on such students? How can they be counted in statistical reporting? How do we ensure that we don’t violate FERPA restrictions in trying to fill the holes?
NSF must expect PIs to align their reporting with what was proposed as broader impact, and provide specific guidelines based on project type and tracking potential. Develop guidelines that support the collection and analysis of qualitative data as well as quantitative, and that expects the articulation of an explicit theoretical and methodological foundation.
Effectively support NSF grant outcomes
Participants argued that evaluation sometimes costs too much of grant, or otherwise don’t have enough funding to do it properly. This may be the first thing that PIs cut when NSF asks for budget reductions.
It would also help for NSF to provide support on developing a less restrictive model of K12 access to education and evaluation data than the current one. A national model might provide support for state-‐by-‐state changes.
NSF could provide a database structure that will help grant writers and grantees with IRB approval and issues of approaching personal data. In addition, some participants were eager to be trained in how to write effective highlights and annual reports, and would be supportive of NSF’s efforts in sharing exemplary ones.
In addition, a “quad” chart (of goals, strategies and outcomes) can be a very efficient summary of findings, and could be an effective communication tool. NSF could ask grantees to create these for dissemination at PI conferences and elsewhere.
31
NSF panels should be encouraged to call out grant proposals that connect with MSIs in order to fulfill some kind of broader impact expectation, but which do not provide any funds for MSIs to do the work.
Collaborating institutions listed in award notices, and four/five co-‐PI limits, large teams cannot provide/earn institutional credit due to the limitations for number of PIs. Where your name is listed on the “list” results in reduced credit, influencing tenure consequences and decreased political capital. Consider ways to expand PI limits.
Disseminate findings, ensure impact, and spur widespread adoption of best practices
Findings are only valuable when they are disseminated, but there are more effective ways to do this than simply through conferences. NSF should help fund the development of new modes of dissemination. For example, participants pointed out that parents are often voters; perhaps grants could collectively sponsor a “community showcase” of grant outcomes with local schools, to communicate more effectively with parents. This would require NSF to provide funding for local community dissemination. The payoff could be significant: parents can impact national policy at the grassroots level if they know what grants and programs are doing and how well it is working.
How, when, and where NSF summarizes and presents data back to grantees and researchers crucial. Can NSF group data under new categories of diverse population groups? For example, “What has been done in BRIDGE programs for ‘this certain population?’” It is not at all clear where grantees and PIs can find the NSF highlights—NSF needs to make this easy and clear. We very much need one portal for all data and best practices that is easily accessible to all grantees where we can share and learn. We want those “dumbed down” congressional highlights also that were spoken of—we need it in that “plain talk” vernacular to present to many audiences of our own who are not specialists and researchers.
The suggestion of an Engineering Education portal or HUB to gather best practices, assessment, and evaluation tools came up repeatedly. Some argued that there is such a portal (CLEERhub.org) and others were completely unfamiliar with it. Therefore, there is NOT a clear, easily accessible portal or HUB that is broadly publicized to grantees. This needs to be done and publicized. Social networking tools can be utilized, and Google linked with industrial program outcomes such as a cloud structure.
If institutions changed faculty evaluation criteria so that education, outreach, and diversity were crucial to faculty advancement and promotion, this would definitely spur widespread adoption of best practices. But how to change the values of institutions? If employment industry could develop an awareness of this new kind of student and of the value of diversity, this could ensure impact. How might we develop this awareness?
It would be wonderful if NSF could provide grantees with the names of REUs for recruitment to graduate school—it could strengthen the REU program, give an incentive for students to participate, and give universities the names of students experienced in research to contact for recruitment purposes.
32
NSF could encourage PIs and researchers to develop new mechanisms for communicating findings to general audiences, such as asking to fund science communications specialists, or funding the development and maintenance of new media such as blogs, vlogs, fee-‐supported social networking platforms, and so on.
33
Graduate Students and Programs: Creating an Emerging Community of Practice for the Next Generation
Facilitators: Don Millard (NSF) James Pembridge (Virginia Tech) Jennifer Turns (U of Washington)
These sessions addressed the issues involved with developing communities of practice for students interested in engineering education. The conversation identified promising work in both research and practice, opportunities for improvement, and areas where NSF can invest time and effort to aid this process.
The discussion was led by an associate professor in Human Centered Design & Engineering at the University of Washington, a PhD candidate from the Department of Engineering Education at Virginia Tech, and an NSF representative. The participants involved in the discussion were primarily graduate students actively engaged in engineering education research while enrolled in engineering education PhD programs or discipline-‐specific engineering programs. Additional participants included faculty and undergraduate engineering students that were involved with REUs.
The discussion over the two sessions addressed the needs of graduate and undergraduate students interested in engineering education. Given the diversity of participants in the sessions, the discussion covered several topics including pathways into engineering education, innovation in teaching, preparing to be educators, and issues related to personal learning experiences.
Pathways into engineering education
A prominent topic in the conversation was the ways that participants are involved in engineering education, and how they came to have their particular involvement. Several paths into engineering education were noted. These pathways included pursuing degrees in engineering education directly as well as pursuing a degree in a traditional engineering field (i.e. mechanical, chemical, industrial, etc.) with engineering education research as a primary component of the dissertation.
Participation in REU experiences was noted as another pathway into engineering education. Participants reported that these opportunities allowed them to explore emerging engineering content and research, which then increased their appreciation for their field and helped them gain practical application-‐based knowledge of the classroom learning. While the undergraduate students in the session did not express interest in teaching in academia, they expressed interest in graduate studies and industry work.
Participants identified increasing awareness of pathways into engineering education as the most apparent opportunity for improvement. Few of the graduate student participants were initially aware of engineering education degree granting programs and even fewer were initially aware that they could conduct engineering education research as part of a more technical
34
engineering degree. The undergraduate REU participants reported similar problems, in that they were not aware of REU’s until a faculty member approached them to be involved in their research. This lack of awareness of engineering education opportunities can be addressed by additional marketing of the opportunities and by getting the information directly to the students.
How innovation is supported and shared: Improving the teaching of engineering
Participants in these sessions noted the limits of lecture-‐based pedagogy and expressed interest in innovative pedagogical approaches. These issues of pedagogy led to the questions such as the following: What practices work well? How can engineering education researchers communicate with practitioners? What are some new innovative approaches to education?
Participants in the sessions expressed interest in studio approaches to education and other approaches related to non-‐traditional engineering education environments. From the perspective of the participants, activities such as reading groups, low stakes activities, and opportunities for discussion and reflection help engineering students develop as engineers and capable learners. When using these approaches, participants noted a variety of challenges and opportunities, such as being able to teach to a broad set of learning styles to meet the needs of all students and taking time to consult with industry so that changing employer short term needs, as well as long term needs, can be taken into account.
Participants noted that preparing students to teach using such innovative approaches would likely include offering graduate students more teaching experiences during their education. Participants wondered what types of opportunities could be made available to support graduate students working with more experienced educators to explore innovations in teaching. Participants also noted that conference and journal papers, while a common means of sharing knowledge, might not be the best way to open the line of communication between the practitioner and researcher.
Preparing to be educators: Skills needed to be future engineering educators
Participants discussed their ideas about being effectively prepared to be an engineering educator and how their current experiences are preparing them, particularly in the area of skills. The discussion of skills needed to be future engineering educators dealt with those that would aid in successful completion of graduate degrees and those that would be used to effectively teach engineering course work. For example, participants indicated that communication and professional skills were important to success, as were having the opportunity to develop those skills.
The graduate students in the groups indicated that much of their time and efforts throughout their graduate studies were spent in a lab or working with a specific research team while being led by one faculty advisor. Within these environments, students work within a specific content area throughout the duration of their academic experiences. This approach to education can force students into a given area early on, not allowing them to explore other research areas and reducing exposure to a variety of pedagogical approaches. Instead of this approach, suggestions
35
were offered that ranged from peer mentoring to working with several groups over the length of a graduate career.
Having and reflecting on personal learning experiences
From the time students enter into elementary education to the time they graduate from graduate school, students experience a variety of teachers, pedagogies, and learning environments. These experiences provide valuable learning experiences for future engineering educators—a future educator can reflect on such learning experiences in order to define his/her own teaching approach.
Several participants in the discussion described experiences as IGERT fellows, and they expressed lessons they learned should they ever apply for an IGERT as a faculty member. These opportunities for learning included assessing the effectiveness of the IGERT experience and developing coursework to meet the needs of their students. One suggestion for NSF’s contribution to the IGERT experience was to offer professional development to prepare future faculty and students that will be involved in the program.
Participants noted that the relationship between the advisor and the student is central to graduate education. Participants reflected on the challenges that they have been experiencing, challenges that suggest opportunities for supporting future graduate students. Such challenges include: choosing an advisor, developing self-‐awareness, assessing readiness for graduation, and finding fields and research of particular personal interest.
Conclusion: Summary and Funding opportunities
The pathways that students take to enter into engineering education are diverse, but no matter how they enter into the field, they bring with them a variety of personal experiences that inform how they will teach when they become faculty. Their learning experiences provide them not only with a set of skills to teach engineering but also introduce them to the community of engineering educators. Specific areas where NSF can support the development of students and establish a community of practice that will aid the innovation of pedagogy include increasing awareness of the pathways available for entering into engineering education, exploring how innovation is supported and shared by the new generation of engineering educators, helping students prepare to be future educators, and encouraging reflection on personal learning experiences.
36
Educating Engineers to be Innovators
Facilitators: Sue Kemnitzer (NSF) Ann F. McKenna (Arizona State) Jill K. Nelson (George Mason)
Introduction
The intent of the panel was to engage participants in a conversation about what it means to educate engineers to be innovators. We structured the panel to enable participants to define what is meant to be an innovator, and what it means to educate this type of person from a knowledge, pedagogical and assessment perspective. The agenda dedicated time to discussing the following questions:
• What is an innovator?
• What knowledge is required for innovation?
• How might you structure a course/curriculum to develop that knowledge?
• What type of pedagogy is necessary to educate innovators?
• How do you assess innovation? And, how do you know if your program is effective in educating innovators?
• What resources are necessary to educate innovators and how might this translate into priority areas for NSF?
Several themes emerged from our discussions and our report is organized around these major themes: 1) the knowledge base and attributes of an innovator, 2) the need for assessment tools and practices, 3) the incompatibility of the engineering education system with educating innovators, and 4) NSF priority areas.
Theme 1: The knowledge base and attributes of an innovator
Our first discussion related to identifying what it means to be an innovator. We discussed what knowledge might be required to engage in the process of innovation, as well as other skills and attributes that are associated with an innovator. We chose to focus on this question first because the responses to this question set some common ground for how we might develop experiences that aim to educate engineers as innovators. That is, we wanted to clarify the educational target, and have some general consensus for what the goals might be for educating innovators.
During both panels the attributes that were most commonly stated to describe innovators were “problem finders,” “risk takers,” “ambiguity lovers,” and “passionate collaborators.” In addition, several noted that innovators possess the characteristic of not only a willingness to fail, but also the ability to learn from failure. These are just a few of the terms mentioned; however, the panel discussions elicited a diverse range of knowledge, skills, attitudes, and dispositions
37
associated with being an innovator. For summary purposes we categorize the attributes that were offered during the panel as affective and cognitive, see Table 1.
Affective/Attitudinal Attributes Cognitive/Knowledge Attributes
• Risk-‐taker
• Collaborative
• Comfortable with ambiguity
• Takes initiative
• Passionate for serving human needs
• Confident
• Always searching for a better thing or way
• Willing to fail
• Persistent
Ability to:
• Synthesize across multiple perspectives
• Identify, define and solve problems
• Deconstruct and redefine problems
• Perform market analysis
• Be creative
• Think critically
• Develop deep disciplinary knowledge
• Be a systems thinker
• Work across disciplinary boundaries
Table 1. Attributes of an innovator.
The panel discussions indicate that the attributes of an innovator are wide-‐ranging. Therefore, any system, curriculum, or experience that aims to educate engineers to be innovators needs to take into account the complex nature of the task. Table 1 can serve as a starting point for defining learning goals and outcomes that would be appropriate for such a system.
Theme 2: The need for assessment tools and practices
Using Table 1 as a basis for articulating potential outcomes for educating an innovator, it is clear that the goals are cognitive as well as affective. The participants clearly stated the need for appropriate assessment tools that would measure this range of innovation competencies. Moreover, the panel discussions noted several issues associated with developing and implementing effective assessments. In particular, assessment tools need to be clear about what they measure, and what counts as evidence. For example, some assessments might measure “learning” or be more cognitive-‐focused, and some might measure self-‐efficacy and measure attitudinal aspects.
The panel also stated that faculty need information about how they might implement various assessment tools, and that additional resources might be needed to implement them effectively. Some pointed out that faculty should also be evaluated using these same tools. That is, since faculty are responsible for teaching students how to be innovators, it is imperative to gauge the knowledge of the faculty in order to identify potential needs for faculty training/development.
38
The participants reflected on the current/typical approach to student assessment and noted that it is primarily exam-‐based. Exam-‐based assessment is not conducive to measuring innovation competencies, so participants suggested switching to a more design-‐focused, project-‐based mechanism for assessing innovation competencies. Since assessment is a major component of what is involved in educating engineers as innovators, it was deemed a priority area for NSF.
Theme 3: Incompatibility of the engineering education system with developing innovators
Much of the discussion of how to educate engineers to be innovators centered around the incompatibility of the current educational structure, particularly in engineering programs, with developing innovators. The panel noted that innovators are people who find problems or opportunities. However, because traditional education focuses on providing students with well-‐defined problems to solve, students are not given the opportunity to develop their problem-‐finding skills.
Participants generally agreed that deep knowledge in one’s technical field is necessary for innovation but noted that many successful innovators of the past did not gain deep knowledge through traditional education. They hypothesized that engineering students’ innovative abilities may suffer because they are too attached to textbooks. Put another way, we teach engineering students that problems have yes/no or black/white answers, but real-‐world problems (and opportunities) are variable and ambiguous. In this way, the curriculum structure opposes innovation, making it difficult if not impossible to encourage students to develop as innovators.
Significant discussion was devoted to the distance between current implementations of teaching innovation and the systemic change that the panel believes must occur to effectively educate innovators. Participants noted that in the current structure, encouraging students to identify needs/opportunities, work in interdisciplinary teams, and solve open-‐ended problems often doesn’t occur until the capstone design project in the senior year. They pointed out that the movement to educating innovators cannot be viewed as an “add on” to the current curriculum but instead must be a cultural change that is diffused throughout the program and the institution. It was suggested that rather than spending two years in math and science courses, students should begin with hands-‐on courses and relevant math should then be introduced in connection with the hands-‐on experiences.
The panel believed that innovators are interdisciplinary, systems thinkers with an inner drive (passion) for problem finding. The segmentation of material into stand-‐alone courses and the use of small “single solution” problems do not encourage these thinking styles. The panel advocated for a focus on active, interdisciplinary forms of education including business games in classes, associations with industry, and significant learning beyond the classroom. This is in contrast to the prevailing existing structure of narrow course topics, lecture-‐based learning, and assessment via traditional (individual) exams.
The panel also identified the importance of the business and marketing elements of innovation, noting that successful marketing (and often profit) are required to move from invention to
39
innovation. They pointed out that business and marketing components are not present in traditional engineering programs and that engineering faculty are generally not qualified to teach in these areas. Evaluating faculty preparedness at a broader level, participants noted that traditional faculty roles don’t encourage innovation (thinking outside the box) and instead encourage adhering to established practices to secure tenure. Faculty with traditional research programs may not have any experience in innovation, particularly the marketing elements, and hence may be ill suited to teach it to engineering students.
On the topic of assessment, participants noted that in the traditional educational structure, there is no reward for innovation. Students are rewarded (in terms of grade) for finding the single correct answer to a problem. To educate innovators, the assessment system must be changed to reward thinking outside the box. In particular, it must be structured such that students are allowed to fail and are rewarded for reflecting on and learning from failures and for iterating on their solutions. Participants observed that students don’t often have time to reflect on their work, nor are they encouraged to do so and that this practice discourages innovation. One recommendation that surfaced was that students’ work be presented and evaluated in a portfolio structure that allows faculty to see the iterations. The goal would be to grade the process, not the outcome. Participants stressed that assessment needs to move away from traditional exams and toward design and process-‐focused evaluation.
Theme 4: NSF Priority Areas
The panel identified several priority areas for NSF funding to encourage education of innovators in engineering programs. Among these were the development of assessment instruments, incubator and scale-‐up projects, the development of next-‐generation instructional materials, and faculty training programs.
Several participants mentioned the importance of developing instruments to measure creativity and innovation in engineering disciplines. They noted that faculty do not have a good definition of what innovation means in an industry setting or what it means to be innovative in different disciplines. Development of instruments to assess effective communication within a group setting was also identified as a need.
Participants pointed to the importance of funding incubators in which faculty could develop new materials and teaching strategies and evaluate their effectiveness. It was suggested that such incubators be structured as multi-‐university initiatives. The panel also recommended that funding be available for replication of results of small research projects, as well as scale-‐up of successful projects to larger populations and institutions.
To ensure that successful approaches are known and adopted, the panel advocated for funding to support broad dissemination of best practices and evaluation of these practices in new settings (similar to scale-‐up). As another approach to disseminating best practices, participants recommended that NSF provide funding for successful innovation instructors to travel and share their approaches with other engineering faculty.
40
Citing the importance of interdisciplinary collaboration for innovation, participants suggested that grants should be available to help faculty develop strategies for teaching students to collaborate in interdisciplinary teams. Several participants also noted the importance of collaborative spaces for teaching and encouraging innovation. They recommended that NSF make funding available for new classroom infrastructure that focused on studios and design environments rather than on lecture halls.
Finally, the panel noted that a critical element of restructuring engineering education to teach and value innovation is the development of next-‐generation instructional materials that move beyond the textbook structure. They recommended that NSF fund the development of such materials.
41
Interdisciplinary Collaboration: Helping Students and Faculty Work Across the Boundaries
Facilitators: Maura Borrego (NSF) Sinead MacNamara (Syracuse U) Michael O’Rourke (U of Idaho)
I. Facilitator Presentations
Maura Borrego, NSF representative. Dr. Borrego focused on interdisciplinary graduate education, supplying a definition of interdisciplinary collaboration rooted in Repko (2008). Her remarks emphasized the need to reflect on teamwork and critical awareness in interdisciplinary, collaborative research, themes developed in detail in NAS (2005). As educators, we must acquire and impart a broad perspective to our students, one that enables them to be effective communicators across interdisciplinary boundaries.
Michael O’Rourke, University of Idaho. Dr. O’Rourke picked up the theme of interdisciplinary communication and focused on work designed to enhance it in the context of collaborative research teams. He argued that communication is the key to successful interdisciplinary research, and failure to communicate well is the root of many challenges that confront scholars crossing disciplines. He devoted much of his time to describing the Toolbox Project, an NSF-‐sponsored effort to enhance communication in cross-‐disciplinary research located at the University of Idaho and Boise State University (Eigenbrode et al. 2007). This effort involves the deployment of a philosophically inspired survey instrument—the “Toolbox”—in a workshop that aims to improve communication by fostering mutual understanding of research assumptions. The dialogue is intended to move collaborative groups from unreasonable epistemic positions to reasonable ones, where those might be either agreement or disagreement. (See http://www.cals.uidaho.edu/toolbox .)
Sinead Mac Namara, Syracuse University. Dr. Mac Namara described an interdisciplinary design program at Syracuse University that combines architecture and civil engineering. After supplying a few historical examples of architects and engineers who bridged this interdisciplinary gap, she focused on the program, which emphasizes creativity and innovation in design. She argued that leveling the playing field by choosing topics relatively unknown to both groups, establishing a common vocabulary, and articulating interdisciplinary design values were critical to its success. One important realization guiding her work in this program concerned the fact that students in undergraduate programs are not as bound by disciplinary boundaries as their instructors are, any many have extra-‐disciplinary educational, personal or career interests. By coaching design sessions, students are made to feel like “experts” and are put in a position to talk across disciplinary boundaries. The resulting designs have been a testament to the power of a program like this to forge interdisciplinary connections for our students.
II. Morning Discussions
A. Critical Issues
42
1. Promising work, in both practice and research
2. Areas for improvement in the field
3. Priority areas for NSF to invest in (based on #1 & 2)
Communication issues are core issues for those engaging in interdisciplinary efforts. These include critical but difficult issues such as variation in vocabulary and communication style across the disciplines. One way to make communication gains is by increasing team cohesion, which is made possible by establishing neutral territory for teambuilding. Of course, neutral territory is possible even though some disciplines may have more importance in certain projects. Among those strategies available for teambuilding include:
• Regularly scheduled meetings
• Development of projects that have a common goal for all of the constituent disciplines
• Establishing a problem so that everybody has an ownership of a piece
• Defining a common language
• Fairness in evaluation process
There will likely be different expectations for different teams; thus, evaluation methods need to be flexible enough to address the diverse team profiles.
Other points made in this session concerned the difficulty of sponsoring and delivering meaningful faculty development opportunities, as well as student team competitions and challenges. It is important for NSF to invest in both faculty development and student competitions, as these will be more popular and meaningful if associated with NSF sponsorship.
B. Impact and Potential
1. How to meaningfully and fairly evaluate NSF grant outcomes
2. Ways to effectively report NSF grant outcomes
3. Ways to disseminate findings, ensure impact, and spur widespread adoption of best practices
Dialogue in this session did not provide balanced coverage of each of the three items under (B); instead, most of the discussion concerned issues related to (3), and in particular, institutional and curricular constraints on dissemination, impact, and broadening participation. We organize the highlights according to these themes.
Institutional Constraints. The participants in this discussion had no issues with NSF’s sponsorship of interdisciplinary collaboration, but felt that institutional support was more problematic, especially for those in the humanities (e.g., philosophers) and the social sciences
43
(e.g., political scientists). Institutions are concerned with 4-‐6 year graduation rates, and interdisciplinary programs can be seen as a threat to those statistics due to the complex nature of the educational experience for the students. In addition, it can be difficult to communicate the value of interdisciplinary work within the institution—it is important to “sell the science”, but that can be challenging if there is institutional resistance. This resistance can reflect different value structures within the institution, both concerning the type of work being conducted (e.g., disciplinary vs. interdisciplinary) and concerning the nature of the ideas involved.
One particularly problematic context within institutions is the department. Departmental support for interdisciplinary projects can be a big concern. Where interdisciplinary work is done across departments within the same school or college there may be fewer problems, but where different schools and colleges and their respective administrative and evaluative processes enter the fray the barriers to interdisciplinary work can become insurmountable. Each department has its own “philosophy”, and many of these champion disciplinary focus over interdisciplinary collaboration, problematizing the latter for those who might be interested in it. This is perhaps nowhere more vexing than in the tenure process, where the by-‐laws may not accommodate interdisciplinary work, or perhaps a departmental representative may not value this sort of work. Given the growing interest and skill in interdisciplinary activity among junior faculty, this is an especially worrying institutional reality. There are ways of building interdisciplinarity into even the most staunchly disciplinary departments, though, such as cross-‐listing courses that have interdisciplinary content in different departments. With the institutional focus on getting undergraduate students through in four years, faculty are being asked to do more and more while institutions are reducing the number of credit hours in programs. These rigid constraints particularly impact graduate students who are trying to do interdisciplinary research by raising administrative issues such as those associated with committee selection. Faculty who are inclined in the direction of interdisciplinary research often deal with these constraints by “riding under the radar”, conducting the research without having it impact their disciplinary commitments.
Curricular Constraints. There is ABET pressure to have curricular balance and many schools may claim this balance when in fact they do not. In engineering programs generally, there is a rigid set of curricular structures built around a model, where past practices, student retention and “throughput” are primary evaluative metrics. Accreditation however can play a large role in terms of getting buy-‐in from departments and faculty for interdisciplinary collaboration. Where interested faculty can show the advantages of a proposed interdisciplinary endeavor in terms of accreditation standards, they are likely to meet with more institutional support. Department practices regarding grading, prize giving, pre-‐ and co-‐ requisites, course numbering and teaching loads are all issues that department heads have given for not supporting interdisciplinary courses. Clearly articulated support from leadership at the school and university level could combat these issues. Two further suggestions were made for loosening curricular constraints: (a) adding a minor (or a certificate program), which may nudge students to take a few extra courses in interdisciplinary area, and (b) creating an interdisciplinary 4th-‐year capstone course.
44
Broadening Participation. NSF can be of real assistance to those interested in interdisciplinary collaboration by working to broaden participation within the academic community. For engineers, broadening participation depends on the ability and interest of engineers to interact in an interdisciplinary and interprofessional manner. Interaction of this type extends scholarly networks and draws people into sponsored research and education from areas not typically represented. One specific thing that NSF could do is recognize relevant interdisciplinary activity in new locations, such as in veterans education.
III. Afternoon Discussions
A. Critical Issues
The “critical issues” discussion in the afternoon addressed undergraduate issues in the main, staying relatively close to the NSF talking points. We use those points to organize this summary.
1. Promising work, in both practice and research
One promising way of introducing undergraduates to interdisciplinary collaboration is through modular interdisciplinary classes, with modules designed to introduce disciplinary content and integrative methodology. A second suggestion focused on “serious games”, i.e., game-‐based approaches to teaching serious course content. This could include “Sim”-‐type environments in which students are presented with engineering problems to solve in simulation. Another suggestion involved redesigning courses so that they included service-‐learning components (e.g., nutrition-‐related outreach, playground safety). This is especially relevant to those interested in interdisciplinary work, since one important driver of interdisciplinary collaboration is the need to solve “big” societal problems. Service learning would put students in contact with these problems as they exist in their own communities, enabling them to acquire the requisite communication skills necessary to be effective interdisciplinary and interprofessional collaborators. Finally, several voices championed team-‐taught, interdisciplinary capstone design courses that could include an interdisciplinary project requirement. These courses can also push students in the direction of entrepreneurship by putting them in collaborative relationships with clients.
2. Areas for improvement in the field
Conversation about this point began with the question, “Can we open people’s minds to interdisciplinary collaboration?” That is, what can be done to excite people about it? It was noted that we are all human beings and so are all engaged in essentially interdisciplinary lives—a fact that should not be lost to us when we step into the classroom. Practical improvements in this spirit will require buy-‐in from faculty and administrators. Institutional and infrastructural barriers can prove to be critical impediments to progress. Important metrics to use in assessing the level of buy-‐in are interest among faculty members in teaching courses with interdisciplinary content and resource allocation by administrators within the institution.
45
Another improvement that can aid the cause here is articulating and developing clear interdisciplinary activity areas. Associated with this is the need to design a framework for assessment to make the interdisciplinary goals clear, perhaps with a map of how the different disciplines fit together. We are asking our students to leave their disciplinary comfort zones, and we should work to make the new interdisciplinary locale seem less foreign and mysterious.
3. Priority areas for NSF to invest in (based on #1 & 2)
Integration of interdisciplinary content into the curriculum is a sure way of exposing undergraduate students to this type of work, and that exposure will generate interest. The NSF is in a position to spur interest in this type of pedagogy by problematizing teaching as an interdisciplinary activity, i.e., as something that can serve as a vehicle of interdisciplinary collaboration. To this end, money could be made available to support creative pedagogical models involving community-‐based service learning and volunteer work that focuses on the social aspects of engineering. Service learning in particular can be used to get buy-‐in from multiple levels in a way that imbeds interdisciplinary principles. These should be courses that put students in close contact with stakeholders, perhaps through substantive projects designed and led by the students. More seed money in general for interdisciplinary pedagogy and public models of engineering education was deemed a critically important step.
B. Impact and potential
Although some attention was paid to interdisciplinary grant activity, this conversation focused primarily on interdisciplinary collaboration in general. Nevertheless, the conversation adhered closely enough to the talking points that we use them to organize the summary.
1. How to meaningfully and fairly evaluate NSF grant outcomes
A variety of evaluative, interdisciplinary metrics were discussed in this session. These include:
• The number of multidisciplinary courses developed, where this can be assessed in terms of where the courses are cross-‐listed and how many students from multiple units are enrolled
• Development of measurable student skills (e.g., teamwork, communication) and meta-‐awareness (e.g., awareness of the existence of interdisciplinary problems)
• The impact the grant has on fostering future, funded faculty collaborations
• Joint publications and presentations, although it is important to note that highly cited journals tend to be discipline based, with certain exceptions (e.g., nanotechnology, energy)
46
It is important that the NSF build long-‐term impact evaluation into grants, when appropriate, and that they commit to implementing changes in interdisciplinary programs and evaluation metrics based on what they learn the evaluation of interdisciplinary activities.
2. Ways to effectively report NSF grant outcomes
When the grant concerns teaching, it is important that student-‐related outcomes be communicated to NSF. These include the impact on students from collaboration with other disciplines (e.g., changes in attitude, skills, and competencies), and specifically the impact this collaboration has on their communication skills (e.g., vocabulary, awareness). Another suggestion that could enhance reporting effectiveness concerned the establishment of criteria for a “NSF approved course”, perhaps with encouragement to institutions to develop “labels” for courses based on NSF funding. The latter could be part of an agency effort to ensure institutional accountability in return for curriculum-‐related funding.
3. Ways to disseminate findings, ensure impact, and spur widespread adoption of best practices
One important point mentioned in connection with this item concerns report format. In particular, it was suggested that the NSF change report formats to allow for data mining at the NSF program level.
IV. Key Terms
Phrase Occurrences Epistemology X Learning mechanisms Learning systems Diversity and inclusiveness Assessment X Research to practice X Innovation cycle in education X Rigorous research X Grand challenges X Pedagogy X
V. References
Eigenbrode, S. D., O’Rourke, M., Althoff, D., Goldberg, C., Merrill, K., Morse, W., Nielsen-‐Pincus, M., Stephens, J., Winowiecki, L., Wulfhorst, J. D., Bosque-‐Pérez, N. (2007) Employing philosophical dialogue in collaborative science. BioScience 57: 55-‐64.
47
National Academy of Sciences, Committee on Facilitating Interdisciplinary Research and Committee on Science Engineering and Public Policy (NAS). (2005) Facilitating Interdisciplinary Research. Washington, DC: National Academies Press.
Repko, A. F. (2008) Interdisciplinary Research: Process and Theory. Thousand Oaks, Calif.: Sage Publications.
