Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Preview:

DESCRIPTION

Listen to session audio while manually viewing slides at: http://ofcoursesonline.com/?p=408. Presentation w/ Rudy McDaniel and Joseph Fanfarelli at 2014 Information Fluency Conference.

Citation preview

Stinking Badges: Why We Need Em’ and How to Use Em’

Rudy McDaniel, Joseph Fanfarelli, and Kelvin ThompsonInformation Fluency Conference

University of Central FloridaFebruary 27, 2014

Outline• Overview • Four Quick Case Studies of Badging Projects• Toward a Badge Design Taxonomy• Discussion / Q&A

• Note: these slides can be downloaded from: http://goo.gl/ezz3DV or http://tinyurl.com/badgesrock

Achievements and Badges• Achievements, or earned tokens of accomplishment,

often encourage players to spend more time within digital systems (esp. videogames) and to alter their playing habits in order to unlock particular types of challenges (e.g., find every coin in a given area or unlock a particular puzzle within a certain amount of time).

• Badges, or visible markers of achievement, have now made the transition from entertainment media to other forms of scholarship and pedagogy, particularly in online learning environments (Jindal, 2011; Bruckman, 2004; Lindgren & McDaniel, 2011; Lindgren, McDaniel, & Friskics, 2011).

Badge graphics courtesy of Matthew Dunn

Badges Are Not New

Video Games

Military Girl Scouts

How Do Badges Work?• Badge = Task-reward system.

– Task - Can present a task to complete.

– Reward – Can also serve as a reward for completing the task.

• Rewards for completing goals can be

– Internal to the system (e.g. Points).– External to the system (e.g. Free or

discounted “stuff”).– The badge, itself.

Badges Are Gaining Widespread Attention• A few familiar names that are actively taking part in

badging:

urdue

What Makes Them So Interesting?• Badges can serve as:

– Goal setters – Motivators – Inspiration to Explore – Creativity Boosters – Progress Trackers

• Connect Badge Criteria to Course Objectives

• Expected vs. Unexpected Badges– Foster different goals. – Expected may help in achieving a specific purpose, while

Unexpected may hurt the purpose (and vice versa).

Implications for Education• Motivate students to do their best work.

– Or additional work.• Help students set goals for clearer routes to

success.• More precisely and creatively track progress, in

comparison to the final grade in a course.• Encourage students to implement creative

thinking to discover unexpected achievements.• We will now discuss some of our

implementations.

Dumb(?) Things I’ve Done with Badges• Badges seen only by recipient• Badges not easily shareable• Badges as “back-handed compliments”• Badges for required activities

CASE STUDY #1: BLENDKIT2012

BlendKit2012

Subject Blended learning

Type Professional Development

Level

Size 1230 enrolled

Badge Source Graphic designer

Badge Platform Developer + Mozilla Framework

Focus Competencies

Grades/Badges Badges only

List/Easter Eggs Badge list

Viewable By Self

Status Complete

CASE STUDY #2: EME5050

EME5050

Subject Ed Tech

Type Academic

Level Grad

Size 15-25

Badge Source Purdue Passport

Badge Platform Credly

Focus 2nd level Competencies

Grades/Badges Grades + Badges

List/Easter Eggs Easter Eggs

Viewable By Class

Status 3rd Iteration Underway

CASE STUDY #3: AEM

Course Structure• 30 different modules to

choose from at 10 different points in the semester.

• A back story involving a media mogul recruiting new students (the “dream job” scenario) is released via four different animations throughout the course.

Example Module Selection

Assessment Approach• Implementation of course technology and

curriculum with ~100 students in Fall 2010 and ~200 students in Fall 2011 was successful

• Badges were added in Fall 2011 version of the course

Sample Badge

Comparing Badges

Assessing the Effects of Badges

• Several components of assessment including student project analysis, focus groups, and comparisons to other courses

• Focus here is on engagement and learning surveys that asked specifically about the badges

• 206 students completed at least one survey• 127 completed both pre- and post-surveys

Student Surveys

To what extent did you believe that having the ability to choose which module to take throughout the course was a positive feature that helped you to learn? Extremely Mostly Somewhat Not at all Positive Positive Positive Positive

I found myself working harder on assignments/projects in order to acquire achievements. Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree

In the last 8 weeks, how often have you discussed ideas from this course outside of class?

Very Often Often Sometimes Never

• Some questions adapted from the 2010 NSSE

General Attitudes

General Attitudes

0

1

2

3

4

5

6

7

Felt Achievement System Had Pos-itive Impact on Course

7 = Strongly Agree

Males FemalesMales Females0

1

2

3

4

5

6

7

Motivated by Other Students Receiving Achievements

7 = Strongly Agree

Males Females

Attitudes - Badges

Variable 1 Variable 2 Pearson Correlation (r)

Significance (p)

Felt Achievement System Was Positive (1 to 7)

Discussed Ideas Outside of Class (1 to 7) .175 .040*

Worked Harder To Receive Achievements (1 to 7)

Commented On Other Students’ Work (1 to 7) .217 .010*

Worked Harder To Receive Achievements (1 to 7)

Amount Of Time Spent Collaborating With Other Students

.242 .004**

Seeing Others Get Achievements Was Motivating (1 to 7)

Total Number Hours Spend On Course Per Week .158 .064

Attitudes - Badges

• Post-Survey: Positive feelings about badge system was correlated with other positive feelings in the course

Assessment Summary

• Importance of “framing” the achievement system at the outset

• Interesting gender patterns – suggests badge systems may be a productive means of targeting female learners

Case Study #4: Two Current UCF Courses

• 1 Semester, 158 Undergraduates• 2 courses: Graphic Design (99 students) and Web

Design (59 students).– 2 sections of each.

» 1 section of each has badges. » 1 section of each does not.

Graphic Design 1

Graphic Design 2

Web Design 1

Web Design 2

Badges Yes No Yes NoStudents 49 50 30 29

Current Courses: Background

• Courses are meant to prepare students for the Adobe Certified Expert (ACE) Exam in the related software.– Web Design = Dreamweaver– Graphic Design = Photoshop

• Completely web-based 16 week courses.• Balanced emphasis on Quizzes / Exams and

Project-based assignments.

Current Courses: Badges

• Badges are:– Unexpected - No list of possible badges can be

found by students.– Private – Students cannot see the badges others

have earned.– Both Objective and Subjective.

• Objective – Named all layers within a Photoshop project.

• Subjective – Helped a classmate succeed

– Awarded both automatically and by the instructor.

Method of Award

• Checkbox in gradebook for each badge simplifies the process

Viewing Badges

• Badges can be found from the course menu, like other important course information.

Viewing Badges• Click a badge to see how it was earned

Current Courses: The Goals

• Improve:– Motivation– Engagement– Academic Performance

• Identify:– Can number of achievements earned be used to

predict grades in a certification preparation course?

Badging Observations

• Each stakeholder determines value

o Issuer, Earner, “Observer,” (Displayer)

• Potential value in each phase of badging:

o Underlying data/record

o Notification email

o Claiming (“Save and Share”)

o Making public

o Linking to specific badges

Unanswered Questions to Ponder

• Why do badges appeal to some but not others?

• Does badging really engage the unengaged?

• What is the right balance of automation and personal attention for course badging?

• What is the relationship between badges and formal credentials?

• What is the right balance of curricular and co-curricular badging at an institution?

Toward a Taxonomy for Badge Design• Subject (e.g., information literacy; educational tech; interdisciplinary; etc.)• Type (e.g., academic; professional development; etc.)• Level (e.g., undergraduate, graduate; etc.)• Tiers (e.g., single tier; multiple difficulty tiers; three cumulative tiers; etc.)• Issued By (e.g., single issuer/multiple; instructor; organization; etc.)• Scale (e.g., course-level; discipline-specific; institution-wide; public; etc.)• Population Size (i.e., to whom badges are available; e.g., 1230; 35; 217; etc.)• Badge Image Source (e.g., graphic designer; badge making template; etc.)• Platform (e.g., Purdue Passport; Credly; etc.)• Focus (e.g., core competencies; off-topic diversion/fun; secondary competencies; etc.)• Grades/Badges (e.g., badges only; badges = grades; grades & badges separate; etc.)• Fixed/Extensible (e.g., defined list of badges; new badges suggested/added on the fly; etc.)• Expected/Unexpected (e.g., published list (“a priori”); discovered Easter eggs; etc.)• Visibility (e.g., issuer; earner; bounded group (“class”); public; etc.)• Status (e.g., complete; interrupted; underway; planning; etc.)

Version 1.1

Thank You!

• Rudy McDaniel, rudy@ucf.edu, @rutang5

• Joey Fanfarelli, joseph.fanfarelli@ucf.edu

• Kelvin Thompson, kelvin@ucf.edu, @kthompso

Badge graphics courtesy of Matthew Dunn

Recommended