Dan Laurence @D4n_

Education & Game Principles:Context, Theory & Application

Senior Learning Designer & Educational Game Designer - La Trobe University

2

Todays Session

3 x 10 minute blocks, followed each by question time (and game time).

Aims: 1. To explore the context of Higher Education & the challenge of student retention.

2. To consider game principles as part of an engagement solution.

3. To demonstrate a specific implementation of game design and it’s outcomes.

3

Higher Education – The Context

A pitched battleground to retain disengaging students.

If the average student fee is around $17,000.

Every 1% decrease in student retention costs…

$4-16 million the same year in lost fees.Double if you factor in cost of acquisition.

…and that is just in the first year.

These dollar values represent peoples dreams & aspirations.

- Adams et al, 2010 Hobson’s retention project

4

How the Government Sees Higher Education

Who are these students and why do they leave?

Many risk factors and correlations:• part-time• mature-age• studying online• first year• 1st in family to attend tertiary study• low socio-economic status • Indigenous• regional• student with a disability

“Student attrition is a wicked problem. It could be 'solved' by excluding from university study everyone except the previously educationally successful, rich, healthy, unencumbered young people without children, who live in cities and study full-time, on campus. But that solution would create some other issues that would be somewhat unpalatable to a developed economy and egalitarian society.”

Marcia Devlin DVC Learning and Quality at Federation University

Some mitigation possible, but factors largely beyond control.

What influences students to discontinue study?

- Adams et al, 2010 – Hobson’s retention project

After enrolment what concerns can we address?

What are students specific complaints?

- The Student Surveys (CSHE, 2004 & AUSSIE, 2008)

• only 50% of students felt that staff were available to discuss student work

• many students perceive that staff are not accessible

• fewer than 30% of students felt that teaching staff took an interest in their progress

• 30% of students reported that they did not receive helpful academic feedback

Students expect: regular constructive feedback.

8

Regular Feedback Also Aids Effective Recall of Facts

The Ebbinghaus Forgetting Curve illustrates the simple point that repetition increases the recall of facts.

9

Student Personas

What are students actually feeling and thinking?

10

Student Journeys

A tool to provide insights into the student experience

“Pain Points”

11

Can’t we just reach out to these students?

• Academics don’t have the time to respond to every student.

• Feedback needs to be of a high quality.

• Feedback needs to be timely.

• Feedback needs to be more regular.

• Feedback isn’t feedback unless students can respond to it.

The only solutions that are practical for large classes are automated.

By Mosborne01 (Own work) CC BY-SA 3.0, via Wikimedia Commons

12

The Formative Assessment Challenge

Tests and quizzes are usually associated with marks.

If assessments aren’t associated with marks, students wont do them, will they?

How can we make unmarked, affordable, automated feedback systems that students will actually want to engage with?

First we should ask what do students actually like doing?

What do students like?

• social media, cats

• games, movies & music

• SMS & other short messaging

• personal email, but rarely school emails.

Modern students access these increasingly via

mobile phone, not desktop PCs, not a website & not a LMS.

14

The requirements of an engagement solution

• Engaging, or put simply • Sense of progress & regular feedback• Social & interactive• Mobile first

15

Block Two: Game Principles as Part of the Solution

We will limit ourselves here to what has become known as “simple gamification”.

Characterised by Points, Badges & Leaderboards

(Werbach’s PBL model, 2012).

What is a Game?Game principles can be observed everywhere in the world.

The animal kingdom, the Legal profession, the business world, religion and yes… education

There are many different definitions and many different types of game.

Engagement & Flow

The state of flow is characterised thus:• There are clear goals every step of the way.• There is immediate feedback to one’s actions.• There is a balance between challenges and skills.• Action and awareness are merged.• Distractions are excluded from consciousness.• There is no worry of failure.• Self-consciousness disappears.• The sense of time becomes distorted.• The activity becomes an end in itself.(Mihaly Csikszentmihaliyi, 1990)

This is the state games aspire towards

17

Types of Rewards

• Points/marks/currency

• Adversarial

• Social/collaborative

• Explorative

(Richard Bartle, 1996)

Bartle’s Player Types

18

Markzewski’s Gamification User Types Hexad

…Intrinsic and Extrinsic MotivationDifferent strokes for different folks

19

Winning is great, unless it isn’t

Many studies and theories align with the broad contention that:extrinsic rewards can sometimes decrease intrinsic motivation & performance!

Only certain types of extrinsic rewards (such as praise) can increase intrinsic motivation.

For complex or creative tasks using extrinsic rewards can lead to poorer performance.

If/then rewards tend to narrow our focus and limit cognitive capacity.

(Ledford, Gerhart, & Fang, 2013)

Gamification/extrinsic rewards increase performance on simple tasks*

*(Arely, Gneezy, Lowenstein & Mazar, 2008)

20

Where are Game Principles Best Used?

Where a knowledge foundation is required - before it can be applied, etc.

• Sciences, Chemistry, Physics, etc

• Accounting, Finance, Economics, etc

• Computer Science, Programming, etc

Creates a point to check & feedback to a student their level of understanding.

theoretical knowledge threshold

knowledgecore concepts abstract

knowledge

Simple tasks

21

Games are about rules

Agreed set of rules and common understandings.

Break the rules and your contract is void.

The architecture of dissatisfaction is:

an unmet expectation, or unfulfilled ideal.

Set clear expectations!Rule breaking results in:

• Decreased extrinsic motivation/engagement

• Low student satisfaction/retention.By Jérôme Dessommes - ÉCRIVAINS CONSULT®, via Wikimedia Commons

22

An Engagement Solution• Clear Expectations• Regular constructive feedback (engagement loops) • Needs to be controllable/curatable (many games aren’t)• Points & badges & leaderboards (PBL model)• Rich media, videos• Social - leaderboards & chat forums• Mobile first - uses ‘push-notifications’• A ‘modern’ interface i.e. Progress trees/bars

Engagement Loops

23

What are the LMS/VLE options?

LMS/VLE’s :

Blackboard 9.x Blackboard Ultra Moodle/MoodleroomsCanvas?

Collaborate Prime Curatr MOOC platforms• Question types: Multiple choice questions, fill in the blank, short answer, equations, • Gamification: Points, badges, leaderboard, levels, progression trees• Analytics: Dashboard, notifications• Student information/enrolment system: Integrated, importable/exportable • Open standards compliance: SCORM, Tin Can API

In our assessment, LMS’s struggle to provide quality mobile interfaces & often lack key functions.

24

What are some other web based options?Web Apps (browser based)Kahoot (hybrid) Quizlet ProProfs GoConqrRiddle Learningpod Socrative That QuizTestMoz Gnowledge QuizStar QuestionProHot Potatoes Quizinator Edpuzzle Khan Academy

Publisher Apps (browser based)

Learning Catalytics (Pearsons)

Cengage - Mindtap

• Question Types: Multiple Choice Questions, Fill in the Blank, Short Answer, Equations, Flashcards• Media Types: Audio, Images and/or Video• Gamification: Points, Badges, Leaderboard, Levels• Social features: Forums, leaderboard• Analytics: Dashboard, notifications• Content Authoring: is it your platform or someone else’s?• Content Bank: OER’s? Personalisation/Branding/Advertising

Mobile feature integration still lacking! - Push Notifications, Geolocation, but mostly about usability

25

What are some mobile first options?

Mobile Apps (iOS & Android):

Q-Stream Duo Lingo MathspaceNearpodQuizling QuizClash iSpring QuizmakerQuestbase

A range of great things about these apps.

The issues we had were:• Inability to curate content• Lack of game elements• Unsustainable cost• Practical implementation issues

26

What went on the wish list?Needs to be both USEFUL and EASY to use:

• Easy to use web portal for question creation

• Well documented

• Import/export external question sets

• Real time learning analytics dashboard

• Import/export use data (currently csv)

27

So after all that here’s what we did…We consulted broadly with game design/programming lecturers, students and graduates.

Using all our research, background data and a teaching grant we created a simple first version.

Featuring:• Timed multiple choice questions• Pre-set badges• Leaderboard

28

And for the Teacher

We created a back-end web portal to create the multiple choice questions:

Complete with basic engagement analytics

29

Research Design

The first iteration performed a simple comparison with previous semesters (year on year).

We wanted control groups, but couldn’t disadvantage those students in the control group.

We sought and received ethics approval.

1. Is there a relationship between introducing a mobile app into the classroom and improved student retention rates?

2. Is there a relationship between the use of the app and improved academic performance?

3. Is student performance within the app related to academic performance?

4. Does the existence of other blended content impact on the effectiveness of the app?

5. Do students respond positively to the introduction of apps?

30

Did gamification make a difference?“The introduction of the app in second semester of 2015 was associated with a positive improvement in student retention (calculated at 12.23%) compared to semester one 2015, a positive improvement of 9.22%...”

“…students who used the app on average achieved marks that were 7.03% higher than students who did not use the app.”

Did gamification make a difference?

31

"I love the competitiveness, ease of access and simple to use."

Strong correlation betweenapp use & academic performance (r .40).

Positive qualitative feedback:

"The app questions were great I really appreciated the daily reminders."

"So much easier to access than the online quiz.”

Negative feedback (during mid-semester break):

“Where have the questions gone, is it broken?”

32

Did it work the next time?The next semester we were given approval to use a control group.

The control group was given an LMS based quiz as a substitute, however due to an unexpected upgrade to Swinburne’s Blackboard installation students also received the BB Student mobile version of the quiz with push-notifications.

This slightly thwarted research design should still allow us to discern the unique contribution of simple gamification.

Control group: No timer, no badges or leaderboard.

Considerations for future research: - Academic performance and engagement after gamification is removed- Game mechanic element studies- Performance on higher order learning tasks and gamification

33

Thankyou. Final Questions?

Follow @D4n_ for updates.

For enquiries or interest in research collaboration contact:

d.laurence@latrobe.edu.au

34

References

Bartle, R. (1996). Hearts, Clubs, Diamonds, Spades: Players Who suit MUDs.

Deci, E.L. Koestner, R. Ryan, R.M. (2001). Extrinsic Rewards and Intrinsic Motivation in Education: Reconsidered Once Again, Review of Educational Research, Spring, Vol. 71, pp 1-27

Ariely, D. Gneezy, U. Lowenstein, G. & Mazar, N. (2008). Large Stakes and Big Mistakes. Federal Reserve Bank of Boston Working Paper No. 05-11, July 2005; NY Times, 20 Nov.

Ledford, G. E. Gerhart, B. & Fang, M. (2013). Negative effects of extrinsic rewards on intrinsic motivation: More smoke than fire.

Csíkszentmihályi, M. (1990). Flow: The psychology of optimal experience. New York: Harper and Row.

Werbach, K. & Hunter, D. (2012). For the Win: How Game Thinking Can Revolutionize Your Business, Wharton School, Philadelphia

Ebbinghaus, H (1885/1913). Memory: A Contribution to Experimental Psychology.