Formal Assessment of Creativity

Katja Hölttä-Otto

Prof. Product Development

Design Factory / Dept. of Mechanical Engineering

Aalto University

21.6.2016

Measuring Creative Outcome

• Quantity

– Amount of ideas

– Amount of unique ideas

– Amount of high novelty ideas

• Novelty

– Uniqueness in a set

– Originality on an anchored scale

– Originality decision tree

• Variety

– Breadth of idea types

• Feasibility

Originality Metric

0 Common

2.5 Somewhat interesting

5 Interesting

7.5 Very Interesting

10 Exceptional

Does the concept achieve design goals beyond the industry norm?

No

0

Yes

To what extent is the design integrated around its innovation?

Minor, isolated improvements

that are peripheral to

function.

2.5

Moderately integrated

improvements that are essential to

function, but design remains typical.

5

System-level improvements. Entire concept

integrated around

improvements.

Is the design so unique it is unlikely to

be seen again?

No

7.5

Yes

10

A Cross-Sectional and Longitudinal Examination of the Development of Innovation Capability in Undergraduate Engineering Students

Trina C. Kershaw, Rebecca L. Peterson, Molly A.

McCarthy, Adam P. Young, Carolyn Conner

Seepersad, Paul T. Williams, Katja Hölttä-Otto, &

Sankha Bhowmick

Goals and Approach

of the Current Project

• Examine the innovation capabilities of undergraduate engineering students – Concept generation exercise using modified 6-3-5 ideation

method

– Resulting concepts scored for originality

– Cross-sectional and longitudinal analyses of originality

– Technical Feasibility

– Relationships between originality and individual difference variables

• academic performance (GPA)

• engineering design self-efficacy

Participants

• 242 students in undergraduate engineering

courses at UMass Dartmouth participated

as part of a course activity

– 100 freshmen in 2012-2013 academic year (AY)

– 38 sophomores in Spring 2014

– 24 juniors in Fall 2012

– 80 seniors in 2012-2013 AY and Spring 2014

• 569 concepts generated

Cross-Sectional Originality Results,

Individual Level

0

0.2

0.4

0.6

0.8

1

1.2

Fall 2012 Spring 2013

Seniors Freshmen

Longitudinal Originality Results,

Individual Level

0

0.2

0.4

0.6

0.8

1

1.2

1.4

0 0.2 0.4 0.6 0.8 1 1.2 1.4

Senio

rs S

pring 2

014

Juniors Fall 2012

Conclusions

• What impacts students’ innovation capabilities?

– Curriculum: Seniors measured in Spring semester were

more original than seniors measured in Fall semester and

juniors.

• Junior year introduces design framework. Senior design

course focuses on planning in the Fall and prototype

delivery in the Spring.

• No difference between freshmen and seniors, but

freshmen course includes various engineering majors.

– Individual differences: No impact, in contrast to previous

research.

Sample concepts

Novelty: 10,

Feasibility: 4

Novelty: 7.5,

Feasibility: 10

Novelty: 7.5,

Feasibility: 10

Novelty: 7.5,

Feasibility: 10

Novelty: 7.5,

Feasibility: 10

Novelty: 7.5,

Feasibility: 10

Novelty: 5,

Feasibility: 10

Novelty: 0,

Feasibility: 10

Future Directions

• Continue longitudinal analysis

• Global university comparison

– Compare data from 5 universities in EU, US & Asia

• Curriculum changes at UMass Dartmouth

– Freshmen courses now include design instruction

• We will test Spring 2016 freshmen and compare to 2012-

2013 freshmen

• Scale up metrics / analysis

Procedure

Groups

4-6 per group

Task

Design next gen. litter

collection system

Inter-action

5 min with

sample litter

collector

Indiv. Ideation

10 min silent

sketching of 3

concepts

Group ideation

Circulate in group for 5 min (silent)

End

Circulate until

returned to

original owner or

min 3 rounds

Ind Diffs

Eng. design self-

efficacy measure

and report GPA

Modified 6-3-5 ideation