Factors that influence the creativity of engineering students Abstract Many engineering curricula focus on educating engineers who are not only technically competent but also capable of designing innovative engineering solutions. In many of those curricula, creativity is fostered with instruction in design methodology, including ideation techniques such as brainstorming and TIPS/TRIZ. In this paper, factors that influence the creativity of freshman and senior level engineering students are investigated. The results support the hypothesis that freshman engineering students are more creative than senior engineering students and that the introduction of creativity enhancement techniques does not affect this result. The level of enjoyment experienced by the students did not affect the results significantly. Introduction and Background Innovation is said to be the key to keeping the U.S. competitive1. This emphasis has led to a need to teach innovation and creativity as part of standard engineering curricula.2 In most American universities, creativity is currently taught as a set of methods such as brainstorming, TRIZ3, or Design by Analogy4. These ideation techniques have been found to be effective5-8 when compared to unguided concept generation. While creativity methods can be effective, other factors may be even more important in increasing the innovation potential of engineering students. Particularly relevant research indicates that freshman students entering the engineering program are capable of generating more creative designs than senior students.9 This finding is in line with the research in cognitive psychology, which indicates that skill acquisition can decrease a person’s ability to solve ill defined problems that require creative thinking.10-13 In similar research, Guilford11 found that people at lower levels of intelligence do not possess enough domain knowledge and are thus unlikely to be creative. However, people at higher levels of intelligence are not always creative either. While they possess the knowledge, their level of creativity can vary from very low to high. Similar results have been found when investigating experts in the areas of baseball14 and chess.15 It has been suggested that highly skilled individuals may have trouble adapting their thinking in order to produce creative outcomes.10,12 Even teachers of engineering design are sometimes unaware of their own blocks to creativity.16 On the other hand, Ericsson17 has proposed that the development of skill can facilitate creative thinking because a highly skilled person is better able to develop new techniques and skills. Highly skilled individuals are better able to adjust their behavior to demands that arise, suggesting that skill is an important determinant of the capacity to restructure.18 In engineering, this trend could be exemplified by senior engineers approaching a problem breadth first, looking at many design solutions before narrowing on one, as opposed to junior engineers’ less creative depth first strategy19. In this paper, we focus on investigating the differences in creativity between freshman and senior mechanical engineering students. It is assumed that the differences between these two groups are primarily due to their skill acquisition as they proceed through the engineering curriculum. Assuming the engineering students follow skill acquisition models proposed in the literature,20

the freshman students enter the college as novices. They then start acquiring skill and reach the first stage of skill acquisition, the cognitive stage.20 This stage involves encoding a skill or learning a set of facts relevant to the skill. In engineering, this stage includes acquiring basic engineering skills such as calculus or graphical representation. These skills are then used repeatedly in the different engineering classes to solve engineering problems. Eventually the students reach the next stage: the associative stage. This stage involves a transformation from fact-based knowledge to procedural knowledge.21 This stage is characterized by a significant reduction in design errors as students gain practice in solving open-ended design problems. For example, students may become more familiar with appropriate design choices such as typical areas of maximum stress and how to avoid them. Most graduating engineering students are at this level. The final stage would be the autonomous stage, which involves the transformation of procedures from controlled to automatic processes.22 This stage can only be reached after multiple years of practice. Expert engineers are generally performing at this autonomous level.23 Prior work9 with 40 senior and 28 freshman students provided evidence that freshman were more creative than senior mechanical engineering students. For this paper, we expanded the previous study and experimented with 20 additional students to further validate the previous finding. We also exposed both freshmen and seniors to the same creativity method and explored what effect the method had on the students. During the experiment we noticed uneven levels of excitement about the task. While some students dove into the problem at hand and really seemed to enjoy the experiment, others looked more forced to do it. Thus, another potential factor affecting creativity, could be the level of enjoyment associated with the creative task. Previous research has linked enjoyment and motivation together,24,25 and enjoyment has also been found to boost student motivation in engineering.26,27 For example, improvisation comedy has been suggested as an addition to the other common creativity methods.28 In this work, we recorded the observable level of each group's enjoyment of their task and investigated whether it had any effect on the creative output. To summarize, this paper focuses on looking at the effect of skill and level of enjoyment on student creativity by investigating the following hypotheses: Hypothesis 1a) Freshman students produce more creative outputs than their senior counterparts without any creativity method. Hypothesis 1b) Freshman students produce more creative outputs than their senior counterparts when using a creativity method. Hypothesis 1c) Freshman and senior students produce concepts that are of equal technical feasibility. Hypothesis 2) If the students enjoy the exercise, they create more innovative concepts. Research Approach Both senor and freshman level students were divided into subject and control groups for the experiments. The students in both subject and control groups were given the same task - to pretend that they were working for a product design company and that their client has asked them to design a next-generation alarm clock. Both groups of students used a modified 6-3-5

method29 to generate concepts for the next-generation alarm clock. According to the modified 6-3-5 method, six participants sketched three concepts individually and then exchanged those concepts, in a rotational sequence, with the other five participants. Before beginning the 6-3-5 method, both groups of students interacted with two standard alarm clocks. The control groups simply interacted with the alarm clocks freely. The subject groups were introduced to a creativity technique called Empathic Experience Design (EED). The EED exposes the participants to an empathic experience with the product, by altering their senses through the use of disabling devices. In this experiment the disabling devices included blindfolds, ear plugs with ear muffs, and oven mitts. Subject group participants were instructed to interact with the product freely, as long as they tried each of the disabling tools for a brief period. The intent was to open their minds to more alternative ideas. The students in both subject and control groups were given up to 20 minutes to interact with the alarm clocks before beginning the modified 6-3-5 method for concept generation. While interacting with the alarm clocks the students worked together and engaged in discussion. Once the interaction phase ended and the concept development phase began, students were forbidden from any discussion. Students spent 15 minutes developing their original three concepts and 6 minutes modifying other students’ concepts according to the modified 6-3-5 method. Metrics used to measure the creative outcome Since the innovativeness of the product designed by the engineers is generally more important than the sheer creative ability of the engineer, we chose to measure the creativity of the resulting designs. We measure the creative outcomes using two metrics. The first, originality, measures the level of innovativeness of the concepts for hypothesis 1a&b and 2, where as the second, quality, measures the technical feasibility of the concepts for hypothesis 1c. Originality Originality, based on the metric developed by Charyton and Merril30, is estimated at the overall concept level using Table 1. The original metric recommended a 0-10 scale, but we have adapted it to a 5-point scale to keep with Likert scale standards and for coding ease. We tested the shortened scale using a sample of 27 subjects rating 10 products. The Pearson correlation for inter-rater reliability ranged between .71-.96, and the test-retest reliability is good as evidenced by 8/10 products showing no significant differences (p >.42) between the repeated tests and the other two showing only barely significant differences (p =.05). This metric has a significantly higher inter-rater reliability than other common metrics at the overall concept level.31

Table 1 Table used to assess originality of the created concepts Score Five scale originality metric 0 Common 2.5 Somewhat Interesting 5 Interesting 7.5 Very Interesting 10 Innovative

Quality (in terms of technical feasibility) The technical feasibility of the concepts is measured using a quality metric developed by Shah et

al.32-33 Quality is measured as ∑=

=m

jqjjqual SfM

1

, where m is the number of features in the design,

fj is the weight assigned for feature j, and Sqj is the quality score given for feature j. To calculate an average quality score, fj can be replaced with 1/m. Sqj is evaluated by one of the authors. Since we evaluate designs only at the concept stage, the original version of this metric is slightly modified to remove the aspect of time. The evaluator was given a flowchart guideline, as shown in Figure 1 and adapted from Linsey,4 to help in judging the quality.

Is it technically feasible?

Is it technically difficult for

the context?

0

5

10

no

yesyes

no

Figure 1 Flowchart for Analyzing Concept Quality 4

Analysis of sample concepts We next describe sample concepts from each group to illustrate how the above metrics were used. Figures 2-5 are representative concepts from each group, and Table 2 shows the rating of each concept using both metrics.

Figure 2 Freshman Subject Group Sample Concept

Figure 3 Freshman Control Group Sample Concept

Figure 4 Senior Subject Group Sample Concept

Figure 5 Senior Control Group Sample Concept

In order to analyze the concepts above it was first important to look at the concepts as a whole and identify some of the main features of the concepts, which are described in Table 2. Then, each concept can be rated using the 5 point originality scale. The freshman subject clock not only was unique in shape and placement, an ear bud, but also woke the user through subtle waves that could only be heard by the user and would not disturb another person if they were sleeping in the room. Therefore, since this idea possessed a unique shape and layout, as well as a new way of waking a sleeping person without affecting others it was found to be very interesting and received a score of 7.5. The freshman control clock was a wall mount clock which is not entirely unique but not widely seen either; its most interesting feature is the ability to be woken by a smell. Although being awoken by a smell is unique it may not be the most marketable idea for all types of people, leaving this concept to score a 5 for interesting on the originality scale. The senior subject clock was designed to have a fun feature which served as the way of turning off the clock as well as entertainment in the form of a game of Simon which could be played throughout the day as the user saw fit. Although the senior subject clock did not possess a new feature for waking the user, its unique shape and additional gaming feature made this clock interesting leaving it to score a 5. Lastly, the senior control group concept was primarily a standard alarm clock, but it did have the ability to display the date and daily weather information. The daily weather information and alerts separated this clock from common alarm clocks, leading to a score of 2.5 points. In terms of technical feasibility, the first question to be asked of each concept is whether the concept is technically feasible. In this case each of the concepts was considered to be technically feasible since the technology in each clock has been implemented previously in clocks or other mechanical devices. The next question to ask is whether it is technically difficult for the context. Similarly, since the technology already existed in similar electronics it makes it simple for a designer to adapt the ideas and apply them to these clock designs. Therefore, all concepts in this sample set received a score of 10, implying that they are entirely feasible to create.

Table 2 Analysis of sample concepts Features of Clocks

Freshman Subject

Freshman Control Senior Subject Senior Control

Description

Earbud clock that could be recharged, wake up to waves

Wall mount clock with a large snooze button, wake up to recorded music or smell

Game of Simon clock on legs. Display rotates around the clock

Standard clock radio type alarm clock with large display, weather and date information

Concept Level Originality Score 7.5 5 5 2.5

Concept Level Quality Score

10 10 10 10

Results The experiment resulted in a total of 240 concepts that were included in the analysis. The number of concepts generated by each group is shown in Table 3. We used ANOVA to analyze the average originality in each group. In addition we also counted the percentage of high scoring concepts created by each group.

Table 3 Results Freshman Subj. Freshman Ctrl. Senior Subj. Senior Ctrl. Total # of Concepts

52 78 63 47

Total # of Participants

20 28 22 18

Averages Freshman Subj. Freshman Ctrl. Senior Subj. Senior Ctrl. Avg. Orig. Score 3.94 3.14 3.13 2.13 Avg. Quality Score

8.65 8.91 8.97 9.37

Originality Scores Freshman Subj. Freshman Ctrl. Senior Subj. Senior Ctrl.

# of Score 10 0 0 0 0 # of Score 7.5 4 3 2 1 # of Score 5 24 26 22 7 # of Score 2.5 22 37 29 23 # of Score 0 2 12 10 16 Originality Scores (%)

Freshman Subj. Freshman Ctrl. Senior Subj. Senior Ctrl.

% of Score 10 0.00 0.00 0.00 0.00 % of Score 7.5 7.69 3.85 3.17 2.13 % of Score 5 46.15 33.33 34.92 14.89 % of Score 2.5 42.31 47.44 46.03 48.94 % of Score 0 3.85 15.38 15.87 34.04

0

1

2

3

4

5

6

7

8

Age+Group Type Figure 6 Originality metrics for all groups

As shown in Table 3 and Figure 6, the concepts produced by the freshman control groups were more creative than those produced by the senior control groups. The average concept level originality of the freshman control groups (3.14) is significantly higher (p=0.0036) than the average concept level originality of the senior control groups (2.13). This result is further

supported by observing the percentage of high scoring concepts in the two groups. We find that the percentage of concepts scoring at least 7.5 (very interesting) is 3.85% for the freshman control group and only 2.13% for the senior control group. While these percentages are low, it is important to remember that a standard brainstorming sessions may be considered successful if only one or two creative and useful concepts are generated. We conclude that the hypothesis 1a) Freshman students produce more creative outputs than their senior counterparts without any creativity method is supported. The same analysis was repeated for the subject groups (Table 3 and Figure 6 ). We find that the average concept level originality of the freshman subject groups (3.94) is significantly higher (p=0.0218) than the average concept level originality of the senior subject groups (3.13). This result is further strengthened by the difference in the percentage of high scoring concepts in the two groups. As shown in Table 3, the percentage of concepts scoring at least 7.5 (very interesting) is 7.69% for the freshman subject group and only 3.17% for the senior subject group. We conclude that the hypothesis 1b) Freshman students produce more creative outputs than their senior counterparts when using a creativity method is supported. In addition we find that both freshmen and seniors produced more original concepts with the aid of the concept generation method, but even with the aid of the creativity method, the seniors could only reach the level of originality achieved by the freshman control group students without any creativity assistance. The senior subject group produced slightly less original concepts than the freshman control group, but the difference is not statistically significant (p=0.9846). This result had been significant in earlier work.9 The difference is likely due to the change in metric. The originality metric used in this paper has higher inter-rater reliability than the metric used in the previous work. To test the hypothesis 1c) Freshman and senior students produce concepts that are of equal technical feasibility, we calculated the quality of each concept and compared the average quality of all concepts between the groups using ANOVA. Results can be seen in Figure 7. We found that there was no statistically significant difference between any of the groups regarding their quality (technical feasibility) scores. The lowest p-value of 0.0931 was found between the senior control group and the freshman subject group. P-values of 0.2424 and 0.4219 were found between the freshman and senior control groups and between the freshman and senior subject groups, respectively. These results support Hypothesis 1c.

Figure 7 Quality metrics for all groups

As discussed earlier, we also recorded the observable level of enjoyment of each task, i.e. whether the students seemed to have fun with the task or not. As seen in Figures 8 and 9, the level of enjoyment of the activity seemed to have no effect on the originality of the resulting concepts for senior-level engineering students. The effect of enjoyment could only be analyzed for the senior groups. All of the freshman control groups exhibited a high level of enjoyment whereas the freshman subject groups did not. Thus, the level of enjoyment was confounded with the use of a creativity method for freshman students.

-1

0

1

2

3

4

5

6

7

8

Originality

Concept Level

High level

of enjoyment

Low level

of enjoyment

Attitude Figure 8 Originality metrics for senior control groups based on level of enjoyment (p-value

0.8524)

-1

0

1

2

3

4

5

6

7

8

Originality

Concept Level

High level

of enjoyment

Low level

of enjoyment

Attitude Figure 9 Originality metric for senior subject groups based on level of enjoyment (p-value

0.1248)

Originality

Concept Level

Freshman

Control

Senior

Control

Senior

Subject

Figure 10 Originality metrics for students with a high level of enjoyment

We then analyzed the results separately based on the level of enjoyment. Figure 10 shows the originality metrics for students with a high level of enjoyment during the exercise. The freshman control groups created significantly more creative concepts than the senior control groups (p-value 0.0166). No statistical difference was found between the originality of the senior subject group and either of the other student groups. Recall that the senior subject groups performed significantly better than the senior control groups in the earlier analysis in which level of enjoyment was ignored. Repeating the analysis for the groups with low levels of enjoyment, we see no statistical difference between the senior and freshman subject groups (p-value 0.3447), as illustrated in Figure 11. However, both the freshman subject and senior subject groups created more original

concepts when using the creativity method compared to the control group even though they were not having fun with the EED method (p-value 0.0004, p-value 0.0106, respectively).

Originality

Concept Level

Freshman

Subject

Senior

Control

Senior

Subject

Figure 11 Originality metrics for students with a low level of enjoyment

The results indicate that the level of enjoyment of the ideation exercise does not influence the originality of the resulting concepts, but interestingly, the groups with higher levels of enjoyment generate more comparably original designs than the groups with low levels of enjoyment. Overall, we find that Hypothesis 2) If the students enjoy the exercise, they create more innovative concepts is not supported. Conclusions We find support for hypotheses 1a-c. The freshman students are able to produce more creative concepts than senior engineering students, without sacrificing the technical feasibility of the concepts. The results did not depend on whether students use a creativity enhancement method or not. We do not, however, find support that increasing the level of enjoyment of the ideation task results in more original concepts. There could be many explanations for this result. For example, a high level of enjoyment could be observed when students are not taking the task seriously, leading to potentially lower levels of creativity or technical feasibility. On the other hand, high level of enjoyment could relax inhibitions and self criticism and thus aid in original thinking and the production of more creative concepts. Further work is needed to investigate the effect of not only student enjoyment but also student motivation and acceptance of the task on the originality of the resulting concepts. It would also be interesting to broaden the subject pool and repeat the experiments across several universities, perhaps with different student demographics and engineering curricula.

References 1. Council on Competitiveness, N. I. (2006). Innovate America. 2. ASME Council on Education (2004). A vision of the Future of Mechanical Engineering Education. ASME 3. Altshuller, G. (1984). Creativity as an Exact Science. Luxembourg: Gordon and Breach. 4. Linsey, J. S. (2007). Design-by-Analogy and Representation in Innovative Engineering Concept Generation.

Austin, TX: Ph.D. Dissertation, Mechanical Engineering Department, University of Texas at Austin. 5. Mullen, B., Johnson, C., and Salas, E., (1991) "Productivity Loss in Brainstorming Groups: A Meta-Analytic

Integration," Basic and Applied Social Psychology, 12(1) pp. 3-23. 6. Paulus, P. B., and Yang, H. C., (2000)"Idea Generation in Groups: A Basis for Creativity in Organizations,"

Organizational Behavior and Human Decision Processes, 82(1) pp. 76-87. 7. Shah, J. J., (1988). Experimental Investigation of Progressive Idea Generation Techniques in Engineering

Design. ASME DETC Design Theory and Methodology Conference. 8. Weaver, J., Kuhr, R., Wang, D. Crawford, R., Wood, K., Jensen, D., and Linsey J. (2009) Increasing innovation

in multi-functional systems: Evaluation and experimental of two ideation methods for design, ASME DETC Design Theory and Methodology Conference. San Diego.

9. Genco, N., Hölttä-Otto, K., Seepersad, C.C. (2010) An experimental investigation of the innovation capabilities of engineering students, ASEE Annual Conference and Exposition. Louisville, KY.

10. Ericsson, K.A. (1998). The scientific study of expert levels of performance: General implications for optimal learning and creativity. High Ability Studies, 9, 75-100.

11. Guilford, J.P. (1967). The nature of human intelligence. New York: McGraw-Hill. 12. Weisberg, R.W. (2006). Modes of expertise in creative thinking: Evidence from case studies. In K.A. Ericsson,

N. Charness, P.J. Feltovich, & R.R. Hoffman (Eds.), The Cambridge Handbook of Expertise and Expert Performance (pp. 761-787). New York: Cambridge University Press.

13. Yang, M. (2008). Prompt versus Problem: Helping Students Learn to Frame Problems and Think Creatively. 2nd Design Creativity Workshop, Third International Conference on Design Computing and Cognition. Atlanta, GA.

14. Wiley, J. (1998). Expertise as mental set: The effects of domain knowledge in creative problem solving. Memory & Cognition, 26(4), 716-730.

15. Bilalić, M., McLeod, P., & Gobet, F. (2008). Inflexibility of experts – Reality or myth? Quantifying the Einstellung effect in chess masters. Cognitive Psychology, 56, 73-102.

16. Linsey, J., Tseng, I., Fu, K., Cagan, J., Wood, K., & Schunn, C. D. (2010). A study of design fixation, its

mitigation and perception in engineering design faculty. Journal of Mechanical Design, 132. 17. Ericsson K.A. (1999). Creative expertise as superior reproducible performance: Innovative and flexible aspects

of expert performance. Psychological Inquiry, 10, 329-333. 18. Ohlsson, S. (1992). Information-processing explanations of insight and related phenomena. In M. Keane & K.

Gilhooly (Eds.), Advances in the psychology of thinking (pp. 1-44). New York: Harvester-Wheatsheaf. 19. Ball, L., Evans, J. B., Dennis, I., & Ormerod, T. (1997). Problem-solving strategies and expertise in engineering

design. Thinking and Reasoning, 3, 247-270. 20. Fitts, P.M., & Posner, M.I. (1967). Human Performance. Belmont, CA: Brookes Cole. 21. Anderson, J.R. (1983). The Architecture of Cognition. Cambridge, MA: Harvard University Press. 22. Schneider, W., & Shiffrin, R.M. (1977). Controlled and automatic human information processing: I. Detection,

search, and attention. Psychological Review, 84(1), 1-66. 23. Ericsson, K.A., Krampe, R.T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of

expert performance. Psychological Review, 100, 363-406. 24. Schmidhuber, J. (2010) Formal theory of creativity, fun, and intrinsic motivation (1990-2010) IEEE

Transactions on Autonomous Mental Development, 2(3), 230-247 25. Ford, R. C., McLaughlin, F. S., Newstrom, J. W., (2004) Having fun at work. Engineering Management, 14(2),

32-33.

26. VanDeGrift, T. (2007) Encouraging creativity in introductory computer science programming assignments; ASEE Annual Conference and Exposition, Conference. June 24 - 27, 2007.

27. Mann, C.; High, K. A., (2003) A pilot study for creativity experiences in a freshman introduction to engineering course. ASEE Annual Conference and Exposition: Staying in Tune with Engineering Education; June 22 - 25, 2003.

28. Ludovice, P. J., Lefton, L. E. and Catrambone, R., (2010) Improvisation for Engineering Innovation. ASEE Annual Conference and Exposition, Conference June 20- 23, 2010

29. Otto, K., and K. Wood. Product Design: Techniques in Reverse Engineering, Systematic Design, and New Product Development. New York: Prentice-Hall, 2000.

30. Charyton, C & Merrill, J. (2009) Assessing general creativity and creative engineering design in first year engineering students. Journal of Engineering Education, 98(2), 145-156.

31. Srivathsavai, R. Genco, N., Hölttä-Otto, K., and C. C. Seepersad (2010) “Study of Existing Metrics Used in Measurement of Ideation Effectiveness”, ASME IDETC Design Theory and Methodology Conference, Montreal, Canada, Paper Number: DETC2010-28802.

32. Shah, J. J., Kulkarni, S. V., & Vargas-Hernandez, N. (2000). Evaluation of idea generation methods for conceptual design: Effectiveness metrics and design of experiments. Journal of Mechanical Design, 122, 377-384.

33. Shah, J. J., Vargas-Hernández, N., & Smith, S. M., (2003a) Metrics for measuring ideation effectiveness. Design Studies, 24, 111-134.