Paper ID #32103

Which prototyping skills should we teach in first-year design? Theanswer is as few as possible

Dr. Matthew Wettergreen, Rice University

Matthew Wettergreen is an Associate Teaching Professor in Engineering at the Oshman Engineering De-sign Kitchen at Rice University.

Joshua Brandel

c©American Society for Engineering Education, 2020

Which prototyping skills should we teach in first-year design? The answer is as few as possible

Matthew A. Wettergreen, Joshua D. Brandel

Oshman Engineering Design Kitchen, Rice University, 6100 Main St, Houston TX, 77005 Abstract Engineering design teams are most successful when members possess a broad range of skills to tackle a project. Instructors of design courses are challenged to select and teach the most important skills they believe will be useful for students now and in the future. Some skills, including teaming and engineering design process skills, can be acquired in a short period of time by applying evidence-based training models. But in terms of prototyping, since there are so many tools and machines available, the question arises of which are truly critical to student success. In first-year design at Rice University, a course that has existed for almost ten years, we aim to teach students only the prototyping skills needed to complete their projects. Students participate in just three workshops that are prototyping related, two of which are required (hand tools and electronics) and an optional third (CAD). By recording student prototyping and measuring experience gains, we have investigated how skills contribute to project completion. The results illustrate that the question for first-year design education is not how many prototyping skills can be taught, but how few an instructor can get away with. Introduction Engineering design is a rich field, and professional engineers must apply a variety of tools and techniques in a process ranging from project identification to the implementation of a final product. Engineers use computation, computer aided design, and an assortment of manufacturing tools to create, prototype, and test their designs. In engineering design education, instructors seek to introduce these techniques and train students up to a working level of proficiency. Meeting these broad course objectives requires careful planning and a suitable educational model. Effective skills training should teach underlying concepts, demonstrate proper usage of tools, dedicate time for learners to practice, and most importantly, provide feedback to the learners [1]. Well-implemented training can lead to safer practices, increased performance, and fewer mistakes. When skills are taught effectively in a class, students see positive results both individually and as teams [2]. When teaching such an extensive field as engineering design, it becomes clear that there are far too many topics to cover in the span of a semester. As a result, instructors face the challenge of deciding which elements to include or omit in their curriculum. This problem is magnified in a first-year engineering design course, where in addition to teaching the engineering design process, an alternate goal is often to provide students fundamental skills that may be useful in

their academic and professional careers. Therefore, first-year design teachers must be highly deliberate in allocating their limited time and resources. Research on cognitive load theory suggests that the way information is presented impacts not only how rapidly students can learn new concepts, but also how long students retain knowledge and how effectively they transfer it to practice [3-5]. Cognitive load theory proposes that above all, cognitive overload should be avoided. Cognitive overload can occur when learners try to process too much new information in a short time period, and it has detrimental effects on the absorption and transfer of knowledge. A prevalent method for preventing cognitive overload is to decrease extraneous cognitive load, which refers to any teaching or activity that does not directly contribute to overall learning objectives [6,7]. This research supports the notion of eliminating all parts of a curriculum that are not strictly necessary for the desired learning outcomes. In this study, we measured the level of experience of students in an introductory engineering design course to evaluate how the skills that were taught were utilized in the class. By interpreting these measurements and examining individual cases where skills had a notable impact on performance, we can draw inferences about effective approaches for course organization and teaching. Experimental Methods This study was conducted in an Introduction to Engineering Design course whose enrollment during the study was 81. In the first half of the class, students teams gathered information and developed initial solutions to real-world design problems. Students filled out a questionnaire reporting their level of experience with several prototyping tools before beginning to build and test physical prototypes of their designs in the second half. The same survey was given at the end of the semester, and a two-tailed t-test was used to assess students’ gains in experience with each tool or skill during the course. Another survey recorded student demographics. In conjunction with these data, the skills and performance of three teams were also examined. These cases are a sampling of the successes and challenges that teams encountered, and their outcomes are evaluated with respect to course objectives. This research is IRB exempt because data was collected using direct observation of students’ work through required course materials. Results and Discussion Student Population The class consisted of 55.6% male and 44.4% female students (n=81). 46.9% of the students were Caucasian, 21.0% were Asian American, 12.3% were Hispanic, 6.2% were African American, and 13.6% were international. The students’ majors are not reported since most had not yet declared their major at the time of data collection. 70 students completed the skills survey both before and after the prototyping unit, allowing comparison of paired responses.

Prototyping Tools/Skills Experience The skills survey evaluated students’ level of experience with a selection of prototyping tools. Some of these are supported in the course via instructional workshops. Others, like manufacturing skills, are useful for high-fidelity prototypes but do not belong in an introductory curriculum. Table 1 reports the statistical significance of changes in the survey responses. It also indicates whether each skill was supported and describes the instructional methods used. Table 1. Prototyping tools/skills used in the course and evaluated via survey.

Tool or skill Supported by course?

^p-value of experience gains

Tool- or skill-specific educational materials available

Hand Tools YES p < 0.01 3 hour guided workshop, required

Physical Prototyping YES p < 0.01 Dedicated lecture and exercise, required; corresponding videos

Power Tools YES p < 0.01 3 hour guided workshop, required

3D Pen Drawing NO* - 1 hour guided workshop, restricted

Post-Processing YES p < 0.05 3 hour guided workshop, required

Hand Drawing/Sketching NO - NONE

Computer Aided Design YES p < 0.01 1 hour guided workshop, optional

Electronics YES - 2 hour guided workshop, required

Laser Cutter NO p < 0.01 NONE**

3D Printer NO - NONE**

Plasma Cutter NO - NONE**

CNC Machining NO - NONE

Molding/Casting NO - NONE

Mill/Lathe NO - NONE

^Survey results and stats are taken from Wettergreen, 2020 [8] *A 3D Pen (3Doodler) was supported for an experimental group of students participating in a related study. It is not considered a core component of the curriculum and is not usually taught. **The maker space in which the class takes place provides resources including lab assistants, online training, public workshops, and step-by-step instructions posted on machines. As reflected in Table 1, statistically significant increases were observed in student experience for each of the skills supported by the course: hand tools, physical prototyping, power tools, post-processing, CAD, and laser cutting. Two exceptions to this trend were electronics, which had a required, dedicated workshop, for the first time during this semester and did not yield significant

experience gains; and laser cutting, where students reported significant experience gains despite the absence of a workshop. Team Collective Prototyping Experience Aggregate prototyping experience for each team can be derived from the surveys which allows analysis of group skill and contribution to the project (Table 2). Generally, considerable portions of the class had prior experience using hand tools, physical prototyping, power tools, and drawing and sketching but most were unfamiliar with manufacturing tools. Initial levels of experience with several other skills, such as 3D printing, varied across the teams. The tools directly taught in the course showed significant experience gains, which indicates the use of appropriate instructional methods. No teams demonstrated experience gains in 3D pen drawing except those who were trained in the tool as part of a separate experimental study. Table 2. Collective prototyping skill experience of teams.

Critical Evaluation of Final Prototypes At the end of the semester the teams delivered final prototypes, which were evaluated based on their level of functionality across various design blocks and the tools/skills necessary to produce them. Additionally, each prototype was assessed on its level of fidelity and whether it performed the core function required to meet project goals. Team prototypes are depicted in Figure 1.

Figure 1. Final prototypes of teams. Left to right, Team BM, Team HT, and Team NN. Teams constructed prototypes from a variety of materials. As can be seen in Table 3, the final prototypes exhibited a range of fidelity, skills used, and functionality. Table 3. Evaluation of final prototypes of teams

Team Pieces/components Skills needed for final prototype

Fidelity of solution

Overall functionality

Core function?

BM wood, u-bolts, brackets,

weights, telescoping metal tubing, fasteners

hand tools, power tools

MEDIUM mostly functional across all design

blocks YES

HT 3D printed parts, PVC, clamps, wood, tape, zip ties, nylon band, velcro

hand tools, power tools, CAD, 3D

printer MEDIUM

some functionality across all design

blocks YES

NN

plywood, metal blades, rubber bands, acrylic sheet, cardboard tube, axle, gears, fasteners

hand tools, power tools, CAD, laser cutter, 3D printer,

plasma cutter

LOW limited to zero functionality

NO

Prototype Success/Failure Due to Collective Prototyping Experience Profile The information in Table 2 allowed us to arrange skills by complexity and instructional support, shown at left in Figure 2. The right side of this map shows simple, quickly adopted skills that are supported by the course and are favorable for prototyping in first-year design. On the left side of the map are skills not supported by the first-year course but taught in higher level design courses. These require significant time investment and prior understanding of complementary techniques.

Figure 2. Aggregate skill experience profiles of three teams. We can interpret how teams’ usage of skills is reflected in the quality of their designs. Team BM had limited experience with most skills at the start of the semester, and they focused on using skills that were supported by the course to deliver a highly functional prototype. Team HT started out with significant experience in several main skills as well as limited prior interaction with laser cutting, 3D printing, and molding/casting. This team used a combination of course-supported skills to produce many iterations of their design and an effective final prototype. Like team HT, Team NN began the prototyping process with considerable experience in several fundamental skill groups. However, this team attempted to apply several complex tools which were not supported by the course, such as the 3D printer and plasma cutter. Allocating time to learn and troubleshoot these advanced manufacturing techniques hindered the team’s progress, and they concluded the semester with a nonfunctional final prototype. Many design instructors wish to furnish their teams with as much information as possible. However, when learners try to acquire too many skills at once, there can be a negative impact on both learning and performance. This paper proposes that rather than teaching every tool they can, instructors should focus on a select few simple, essential skills. In a first-year engineering design course, these may include hand tools, power tools, electronics, and basic CAD. Teaching additional skills requires students to devote time and mental effort that they cannot afford. References [1] E. Salas and J. A. Cannon-Bowers, “The science of training: a decade of progress.” Annual

Review of Psychology, vol. 52, no. 1, pp. 471–499, 2001. [2] E. Salas, S. I. Tannenbaum, K. Kraiger, and K. A. Smith-Jentsch, “The Science of Training

and Development in Organizations,” Psychological Science in the Public Interest, vol. 13, no. 2, pp. 74–101, 2012.

[3] P. Chandler and J. Sweller, “Cognitive Load Theory and the Format of Instruction,” Cognition and Instruction, vol. 8, no. 4, pp. 293–332, 1991.

[4] F. G. W. C. Paas and J. J. G. V. Merriënboer, “The Efficiency of Instructional Conditions: An Approach to Combine Mental Effort and Performance Measures,” Human Factors: The Journal of the Human Factors and Ergonomics Society, vol. 35, no. 4, pp. 737–743, 1993.

[5] J. D. Facteau, G. H. Dobbins, J. E. Russell, R. T. Ladd, and J. D. Kudisch, “The influence of General Perceptions of the Training Environment on Pretraining Motivation and Perceived Training Transfer,” Journal of Management, vol. 21, no. 1, pp. 1–25, 1995.

[6] J. Sweller, “Cognitive Load During Problem Solving: Effects on Learning,” Cognitive Science, vol. 12, no. 2, pp. 257–285, 1988.

[7] J. J. G. van. Merrienboer, Training complex cognitive skills: a four-component instructional design model for technical training. Englewood Cliffs, NJ: Educational Technology Publications, 1997.

[8] M. Wettergreen, “Addressing Challenges of Moving from Low to Medium Fidelity Prototypes by Using the 3Doodler 3D Printing Pen,” American Society for Engineering Education Gulf-Southwest Annual Conference, 2020.