1 Storytelling Alice: Presenting Programming as a Means to the End of Storytelling Caitlin Kelleher Carnegie Mellon University Pittsburgh, PA

1

Storytelling Alice: Presenting Programming as a Means to the End of Storytelling

Caitlin Kelleher

Carnegie Mellon University

Pittsburgh, PA

www.alice.org

2

Shrinking Pool of Computer Scientists

3

There is a strong need for computer scientists

• Three occupational groups are expected to provide 75% of the new jobs between 2004-2014– Health care practitioner and technical

occupations– Education, training, and library occupations– Computer and mathematical science

occupations

4

Projected Science and Engineering Job Openings (new jobs and replacements)

2004-2014

Information Technology

Engineering

Life Science

Physical Science

Natural Science

Managers

5

Other Countries Predict IT Shortages Too

• Canada: the current 35,000 openings per year will grow to 89,000 by 2010. Currently, schools are producing < 7000 per year.

• UK: 50% drop in students since 2000 will lead to hiring difficulties.

6

Computer Science supports progress across many disciplines

7

Simply, we need more computer scientists

…and more diverse computer scientists

8

Broad reach creates a pragmatic need for diversity

9

Lack of diversity can lead to unintentional exclusion

10

Research goal: Design programming

environments that motivate a larger and more diverse group of people to learn to program.

11

Related Work

12

Related WorkSimplify the language:

BASIC

13


BASIC

Prevent syntax errors: Cornell Program

Synthesizer

14


BASIC


Synthesizer

Make programming concrete:

Karel, Logo

15


BASIC


Synthesizer


Karel, Logo

Construct programs with graphical objects:

Lego Mindstorms

16


BASIC


Synthesizer


Karel, Logo

Construct programs with graphical objects:

Lego Mindstorms

Introduce new programming paradigms:

Smalltalk, Pascal

17

No matter how easy something is, people still need a reason to

want to do it.

18

Approach:Develop programming

environments that present programming as a means to an end that the target audience is

motivated to pursue.

19

Solve the harder problem: make programming motivating for an

under-represented group

• There are many important and under-represented groups in Computer Science.

• Women are currently the largest.

20

Female CS pool is shrinking quickly.

1973 2004

21

Critical Time: Middle School

• In 4th grade, the numbers of girls and boys who like math and science are about equal.

• By the end of 8th grade, twice as many boys as girls are interested in math and science.

22

Approach

Present programming as a means to the end of storytelling.

23

Why Storytelling?

• Given a little bit of time, most girls can come up with a story they want to tell.

• Stories are naturally sequential.• Stories provide girls an opportunity to

experiment with different roles, a central activity during adolescence.

• Non-programming friends can readily understand and appreciate an animated story.

24

Developing Storytelling Alice

25

Started with Alice 2

• Alice 2 is a programming environment targeted at college-level intro CS students– Separate learning of programming concepts and

syntax – Enable novice users to create interactive 3D graphics

programs– Formal results show that use of Alice increases

grades and retention of CS students in subsequent Java-based CS courses.

• Alice 2 provides:– drag and drop program construction that

prevents syntax errors– 3D graphics and animation engine

26

Formative Evaluation: Observe middle school girls (and some boys) creating stories in Alice

Design and Implement:Improve Storytelling Alice

Classic HCI Research ParadigmOver 18 months, ~15 iterations

27

Subjects: Formative Evaluation• ~ 60 Campers at

STEM camps• ~ 30 Local home-

schooled kids

• ~ 120 Girl Scouts from within 1.5 hours of Pittsburgh

28

Storytelling Alice Demo

29

Changes to Enable Storytelling

1. Add high-level animations and scene support.

2. Create a story-based tutorial.

3. Provide a gallery of 3D characters that inspire stories.

30





31

Lesson Learned: two-person talk aloud protocol is a poor match for a

creative process like storytelling

32

Storyboarding Process

1. Write the DVD box description

2. Flesh out each scene: setting, purpose, and action

3. Create 6-9 frames/textual descriptions for each scene.

33

Rhetorical Information Helps

• 3 sources for actions:– Users’ descriptions of the action in a given

frame– Differences between drawings of sequential

frames– Users’ descriptions of the action in a scene

(occasionally)

34

People are more important than objects

Kids very rarely animate things.

In Storytelling Alice, humanoids and other non-bipedal characters are a different type and have different methods than non-living “things”.

35

What people can do…

Storytelling Alice:• Say, think• Walk to, Walk offscreen• Walk • Sit On• Lie on• Kneel• Fall Down• Stand Up• Straighten• Look at• Look• Turn to face, Turn away from• Turn• Touch• Keep Touching

Generic Alice:• Move• Turn• Roll • Resize• Play Sound• Move to• Move toward, Move away from• Orient to• Turn to Face, point at• Set point of view to• Set Pose• Stand up• Move at speed, turn at speed, roll at

speed• Constrain to face, Constrain to point at

36

Most character and object interactions can be created using

touch and keep touching

37

Character actions often motivate use of programming constructs like

loops, methods and parameters

38





39

Before: After:

40

Click on the “trevor” tile to see what methods trevor knows how to do.

41

42

Stencils enables users to complete tutorials more quickly and with fewer errors.

Users of a Stencils tutorial:• Learned as well as users of a paper

version of the same tutorial• Made 270% fewer errors (p < .05)• Completed tutorial 26% faster

(p = .057)

• Made 88% fewer requests for human assistance (p = .08)

43





44

Provide custom animations that require explanation in the story

Animations can be incredibly powerful in helping kids to come up with a story idea.

Harold T. Wireton.crazy go nuts

45

Provide characters with clear roles

Hero or Villain Authority Figure

46

Related Work

47

Programming Systems that enable animation

• ToonTalk (Kahn, 1996)

• Squeak Etoys (Kay et al., 1997)

• HANDS (Pane, 2002)

• Scratch (Resnick et al., 2006)

• Kids Programming Language (Morrison Schwartz, Inc., 2006)

48

Present programming in a motivating context

• Algoarena – programming sumo-wrestlers to fight with each other

• Robocode – programming tanks to fight with each other

• Rocky’s Boots – connecting logic gates to solve puzzles

49

Virtual Family

50

Rapunsel

51

There are no formal results showing that any of these

systems are (more) motivating.

52

Evaluating Storytelling Alice

53

Two Questions

• How much do users learn?

• How motivated are they?

54

Two Questions

• How much do users learn?

• How motivated are they?

55

Summative Evaluation

• 88 Girl Scouts from within 1.5 hours of Pittsburgh

• Compare Storytelling Alice with Generic Alice

56

Evaluating Generic Alice vs. Storytelling Alice

Tutorial Build something to show

in Generic Alice

Take programming quiz and attitude survey

Try Storytelling Alice

Choose Generic or Storytelling Alice to take home

Show a world to everyone

Tutorial Build something to show

in Storytelling Alice

Try Generic Alice

2 hrs, 15 min

30 min

Control Group: Experimental Group:

57

Results

58

Similar Learning in Storytelling Alice and Generic Alice

Mean Programming Quiz Scores

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Sequence DoTogether

SimpleLoop

MoreComplex

Loop

MethodCall

Parameter WorldStartsEvent

Mea

n S

core

Generic Alice Storytelling Alice

59

Since it is as educational, which system do girls find

more motivating?

60

Types of Results

• What girls build

• What girls say

• What girls do

61

Three Activities in Alice

1: Scene Layout

2: Editing Programs

3: Running Programs

62

Users of Storytelling Alice spend 42% more time programming

Average % Time Spent on Alice Activities

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

Scene Layout Editing Program Running Program

Generic Alice

Storytelling Alice

p < 0.001

p< 0.001

63

Storytelling Alice motivates the reluctant programmers

Scene Layout vs. Program Editing

0

10

20

30

40

50

60

70

80

90

100

0 10 20 30 40 50 60 70 80 90 100

% Time - Editing Program

% T

ime

- S

ce

ne L

ayo

ut


64

Generic Alice: Arbitrary motion (28/45 = 62%)

A collection of characters that moves in seemingly arbitrary ways.Sometimes whole characters move, sometimes individual body parts.

65

Generic Alice: Character Motion (7/45 = 15%)

The characters put their arms at their sides and the girl on theleft waves hello.

66

Generic Alice: Story-like sequences (7/45 = 15%)

A penguin moves to the lever, the lever turns and the Christmas tree lights up.

67

Generic Alice: Dance routines (3/45 = 7%)

The penguins turn, jump, and look in different directions in sequence and in parallel.

68

Storytelling Alice: Relationship Stories (22/43 = 51%)

A story about a boy named Dave who has been having relationships with three different girls. They find out and kick his legs in retaliation. The story ends with the statement “And that’s why you don’t cheat on girls. It Makes Your Legs Fall Off!”

69

Storytelling Alice: Good vs. Evil (9/43 = 21%)

A story in which the Big Bad Wolf attempts to befriend the three pigs in the hopes of eating them. The pigs get scared and a ninja appears to frighten the wolf away.

70

Storytelling Alice: other (12/43 = 28%)

• 2 stories about finding lost dogs

• 2 stories depicting running or swimming races

• 3 choreographed routines involving cheerleaders or circus characters

71

Types of Results


• What girls say

• What girls do

72

Ease and Entertainment

• No significant differences between Storytelling Alice and Generic Alice users:– Alice’s ease of use– Alice’s entertainment value

73

Users of Storytelling Alice have stronger interest in future Alice use.

Mean Future Alice Use Scores

0

0.5

1

1.5

2

2.5

3

3.5

One time use Use over the next month Interest in a future Aliceclass

Question

Mean

Sco

re

Generic Alice

Storytelling Alice

p < 0.05

74

Users of Storytelling Alice were slightly (but not significantly) more interested in

computer scienceMean Computer Science Interest Scores

-1

-0.5

0

0.5

1

1.5

Create a w orldyou are proud

of

Learnadvanced

Alice features

LearnJava/C++

Interest in HSCS class

Grow up to CS

Question

Mean

Sco

re

Generic Alice

Storytelling Alice

75

Types of Results


• What girls say

• What girls do

76

Alice Version to Take Home

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

Chose Storytelling Alice Chose Generic Alice

Alice Version Selected

% o

f U

sers


Users of Storytelling and Generic Alice prefer Storytelling Alice

p < 0.001

77

Switching Versions

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Show non-main world

Alice Program

Pe

rcen

tag

e o

f U

se

rs


Users of Generic Alice are more likely to show the world they build in 30 minutes.

p < 0.001

78

Sneaking Extra Time

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Show non-main world

Alice Program

Per

cen

tag

e o

f U

sers


Users of Storytelling Alice are more likely to sneak extra time to continue programming.

p < 0.001

79

Summary of Results

• Users of Storytelling Alice– Spent 42% more time programming – Were more than three times as likely to sneak

extra time to work on their programs (51% vs 16%)

– Express stronger interest in future Alice use

• Users of both Storytelling and Generic Alice preferred Storytelling Alice

80

Future Work

81

Research goal: Design programming

environments that motivate a larger and more diverse group of people to learn to program.

82

Other Demographics

• Evaluating and potentially adapting Storytelling Alice for other groups– Boys– Minority students

83

The Real World

• Realistically, most schools will introduce programming in a short unit (if at all)

• Organizations like the Girl/Boy Scouts or a local YMCA will tend towards a small number of sessions.

• Storytelling Alice needs to inspire and enable kids to learn new material on their own.

84

Challenge #1:keep users coming back for more

85

Addictive Educational Software

• Choose and support intrinsically motivating activities.• What extrinsic rewards can reinforce users’ interest in

educational content over long periods of time?– Access to new content?– Positive feedback from peers?

• How do we measure motivation over longer periods?

• Effective extrinsic reward schemes and motivational measures can aid the development of serious games.

86

Challenge #2:motivate users to learn more

programming concepts

87

88

Motivating Use of More Programming Concepts

• More expressive character animations (especially facial and social), to motivate methods and parameterized methods

• Controlling behavior of groups of movie extras to introduce conditionals, iteration, and abstraction.

• Could contribute to systems that enable non-technical users to quickly prototype film pre-visualizations and video games

89

Challenge #3: enable users to teach themselves

90

Using Example Code

• Understanding code: what does this do?

• Re-using code: Copy and modify example code for use in your own program

• Interaction techniques for understanding and re-using example code may lead to improved general-purpose programming environments

91

Caitlin Kelleher

[email protected]

www.alice.org

Questions?

Documents

1 Storytelling Alice: Presenting Programming as a Means to the End of Storytelling Caitlin Kelleher Carnegie Mellon University Pittsburgh, PA