1
References Deterding, S., Sicart, M., Nacke, L., O'Hara, K., & Dixon, D. (2011, May). Gamification: Using game-design elements in non-gaming contexts. Proceedings of the 2011 Annual Conference Extended Abstracts on Human Factors in Computing Systems (pp. 2425-2428). ACM. Gee, J. P. (2003). What videogames have to teach us about learning and literacy. New York: Palgrave Macmillan. Holman, C., Aguilar, S., & Fishman, B. (2013). GradeCraft: what can we learn from a game-inspired learning management system? (p. 260). In LAK '13: Proceedings of the Third International Conference on Learning Analytics and Knowledge, New York, New York, USA:  ACM Martinez R., Martin, C., Harris, S., Squire, K., Lawley, E., & Phelps, A. (2012). Just Press Play: Design Implications for Gamifying the Undergraduate Experience. In C. Martin, A. Ochsner, & K. Squire (Eds.), Proc. GLS 8.0 (pp. 9–113). Pittsburgh, PA: ETC Press. Ryan, R. M., & Deci, E. L. (2000). Self- determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68–78. doi:10.1037/0003-066X.55.1.68 Ryan, R. M., Rigby, C. S., & Przybylski, A. (2006). The motivational pull of video games: A self-determination theory approach. Motivation and Emotion, 30(4), 344-360. Salen, K., Torres, R., Wolozin, L., Rufo- Tepper, R., & Shapiro, A. (2011). Quest to Learn: Developing the school for digital kids. Cambridge, MA: MIT Press Acknowledgements We owe thanks to, Mika LaVaque-Manty who designed and taught the political theory . course, and the students who courageously “played the game.” We also thank the Learning Analytics Task Force at The University of Michigan for their support, the graduate students who assisted in the teaching of the course, and the team behind the design of the gameful LMS GradeCraft. Without their continued efforts this work would not be possible. USE Lab Digital Media Commons Ivory Tower Path analysis (Figure 5) indicates that autonomy is both an outcome of students’ perceived fairness of the assessment system, as well as a direct predictor of student exploration, student effort, and an indirect predictor of final course grade. Grade point average understandably predicted final course outcome, as well as students’ engagement with the LMS. Students’ use of GradeCraft’s “grade predictor” tool also had a direct and positive relationship to autonomy. The core orientation behind much of this game-inspired design work is the willingness to reimagine—to change—the nature of the tasks given to learners, and to do so in a manner that supports their autonomy. Evidence indicates that gameful approaches support student autonomy and provide the “space” for learners to feel encouraged be more effortful which leads to more engagement. Self-Determination Theory provides an appropriate theoretical frame in which to examine game-inspired design. Future studies will include greater attention to how gameful designs can support competence and belonging. Course Grade 1 Effort 2 Grade Point Avg. Predictor Use 3 LMS Logins 4 Fairness Autonomy 5 Exploration 6 .12 .56 .21 .46 .37 .16 .23 .27 -.15 .12 .74 .27 .3 Perceptions grading features GradeCraft system use Academic variables Results Note: only significant paths and their standardized coefficients are shown. The model fit well: χ2(24, N = 226) = 26.6, p = .34, CFI = .99, RMSEA = .02, SRMR = .04, suggesting that the proposed model is consistent with the observed data. “Predictor use” refers to students logging into the GradeCraft LMS and using it to “play” with various configurations to determine their optimal path. Figure 5. Path model indicating the central role autonomy plays in adaptive outcomes Assessment System Design Methodology Game-Inspired Course Design Discussion 60% Student Controlled 40% Instructor Controlled Aendance Reading Quizzes Discussion Secon 20% 5% 15% The grading system supported student autonomy by allowing students to determine how each of four possible assignment types were weighted towards 60% of their final grade (Figure 2). “Badges” were offered as a means to encourage certain academic behaviors (see Figure 3 for two examples). Figure 4 describes one possible assignment configuration within the course. Figure 3. Examples of optional badges “Undertake something challenging to you and do your best, even if you end up failing on it.” Figure 2. Students given control over 60% of their grade. 40% still determined by traditional assessments “Attend office hours more than once (with legitimate questions!)” Promotes Students’ Autonomy Group Project Academic Essay Blogging Assignments Result Weights Solo Project K K K K K K Worth 3x more points Worth 2x more points Worth 1/2 as many points No change Figure 4. One possible distribution of assignment weights. Note: In this example, weights were not assigned to the solo project. Students were given roughly 5 weeks to determine their optimal configuration before being “locked in.” Learning Management System Multiple paths of engagement Mitigates the cost of failure Manage bookkeeping and way for students to “play” with their grades College Students (N = 299) 60% Male 40% Female 50% Freshman 36% Sophomores 11% Juniors 2% Seniors 1% Not Reported Autonomy: “I have more control over my final course grade because of the grading system.” Effort: “The grading system encourages me to work harder than I would in a different kind of grading system.” Exploration: “The grading system encourages me to work on assignment types I would normally avoid.” Fairness: “I believe the grading system is fair to students.” Survey Measures Sample Data were collected through an online survey administered to 299 students at the end of the term; 232 (78%) students completed the survey (see Table 1). A 1-5 Likert scale (Not at all true - Very true) was used on all survey items. Demographic, achievement (e.g., grade point average), and LMS use data were also included and matched to survey data for final analysis. Table 1. Demographic information and items used to measure variables of interest. Figure 1. To “gamify” a course one layers on extrinsic elements. To make a course gameful one must redesign it. Research has shown that videogames can be very motivating (Ryan, Rigby, & Przybylski, 2006), and educators have begun apply game design principles (see Gee, 2003; Deterding et al., 20011) to classroom contexts in hopes of increasing student engagement and motivation. Researchers have also begun to study the effects of these designs (e.g., Salen et al., 2011; Martinez et al., 2012) and also support their development though online Learning Management Systems (e.g., GradeCraft: Holman, Aguilar, Fishman, 2013). “Gameful” courses work because they support student autonomy—an adaptive student outcome (Ryan & Deci, 2000). This approach is different from “gamification,” which simply adds extrinsic rewards to existing learning environments (Figure 1). The objective of the current study is to examine how a particular gameful design supported student autonomy, as well as understand other adaptive student behaviors it fostered. Learning Environment Gamified Learning Environment badges points Gameful Learning Environment autonomy failure choice play

Game-Inspired Course Design Assessment System Design › research › ICM-Final.pdf · used on all survey items. Demographic, achievement (e.g., grade point average), and LMS use

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Game-Inspired Course Design Assessment System Design › research › ICM-Final.pdf · used on all survey items. Demographic, achievement (e.g., grade point average), and LMS use

References

Deterding, S., Sicart, M., Nacke, L., O'Hara, K., & Dixon, D. (2011, May). Gami�cation: Using game-design elements in non-gaming contexts. Proceedings of the 2011 Annual Conference Extended Abstracts on Human Factors in Computing Systems (pp. 2425-2428). ACM.

Gee, J. P. (2003). What videogames have to teach us about learning and literacy. New York: Palgrave Macmillan.

Holman, C., Aguilar, S., & Fishman, B. (2013). GradeCraft: what can we learn from a game-inspired learning management system? (p. 260). In LAK '13: Proceedings of the Third International Conference on Learning Analytics and Knowledge, New York, New York, USA:  ACM

Martinez R., Martin, C., Harris, S., Squire, K., Lawley, E., & Phelps, A. (2012). Just Press Play: Design Implications for Gamifying the Undergraduate Experience. In C. Martin, A. Ochsner, & K. Squire (Eds.), Proc. GLS 8.0 (pp. 9–113). Pittsburgh, PA: ETC Press.

Ryan, R. M., & Deci, E. L. (2000). Self- determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68–78. doi:10.1037/0003-066X.55.1.68

Ryan, R. M., Rigby, C. S., & Przybylski, A. (2006). The motivational pull of video games: A self-determination theory approach. Motivation and Emotion, 30(4), 344-360.

Salen, K., Torres, R., Wolozin, L., Rufo- Tepper, R., & Shapiro, A. (2011). Quest to Learn: Developing the school for digital kids. Cambridge, MA: MIT Press

Acknowledgements

We owe thanks to, Mika LaVaque-Manty who designed and taught the political theory .course, and the students who courageously “played the game.”

We also thank the Learning Analytics Task Force at The University of Michigan for their support, the graduate students who assisted in the teaching of the course, and the team behind the design of the gameful LMS GradeCraft. Without their continued e�orts this work would not be possible.

USE LabDigital Media Commons

Ivory Tower

Path analysis (Figure 5) indicates that autonomy is both an outcome of students’ perceived fairness of the assessment system, as well as a direct predictor of student exploration, student e�ort, and an indirect predictor of �nal course grade. Grade point average understandably predicted �nal course outcome, as well as students’ engagement with the LMS. Students’ use of GradeCraft’s “grade predictor” tool also had a direct and positive relationship to autonomy.

The core orientation behind much of this game-inspired design work is the willingness to reimagine—to change—the nature of the tasks given to learners, and to do so in a manner that supports their autonomy.

Evidence indicates that gameful approaches support student autonomy and provide the “space” for learners to feel encouraged be more e�ortful which leads to more engagement. Self-Determination Theory provides an appropriate theoretical frame in which to examine game-inspired design. Future studies will include greater attention to how gameful designs can support competence and belonging.

CourseGrade

4.6

1

E�ort.22

2

GradePoint Avg.

1

8.3

PredictorUse

-.57

3

LMSLogins

-.88

4

Fairness1

3

Autonomy.31

5

Exploration.47

6

.12

.56

.21

.46

.37

.16

.23

.27

-.15 .12

.74

.27

.3

Perceptions grading features GradeCraft system use Academic variables

Results

Note: only signi�cant paths and their standardized coe�cients are shown. The model �t well: χ2(24, N = 226) = 26.6, p = .34, CFI = .99, RMSEA = .02, SRMR = .04, suggesting that the proposed model is

consistent with the observed data. “Predictor use” refers to students logging into the GradeCraft LMS and using it to “play” with various con�gurations to determine their optimal path.

Figure 5. Path model indicating the central role autonomy plays in adaptive outcomes

Assessment System Design

Methodology

Game-Inspired Course Design

Discussion

60% Student

Controlled

40% InstructorControlled

AttendanceReading QuizzesDiscussion Section

20%

5%

15%

The grading system supported student autonomy by allowing students to determine how each of four possible assignment types were weighted towards 60% of their �nal grade (Figure 2). “Badges” were o�ered as a means to encourage certain academic behaviors (see Figure 3 for two examples). Figure 4 describes one possible assignment con�guration within the course.

Figure 3. Examples of optional badges

“Undertake something challenging to you and do your best, even if you end

up failing on it.”

Figure 2. Students given control over 60% of their grade. 40% still determined by traditional assessments

“Attend o�ce hours more than once (with legitimate

questions!)”

Promotes Students’ Autonomy

Group Project

Academic Essay

Blogging

Assignments ResultWeights

Solo Project

K

K K

K KK Worth 3x

more points

Worth 2x more points

Worth 1⁄2 asmany points

No change

Figure 4. One possible distribution of assignment weights.

Note: In this example, weights were not assigned to the solo project. Students were given roughly 5 weeks to determine their optimal con�guration before being “locked in.”

Learning Management

System Multiple paths of engagement

Mitigates the cost of failure

Manage bookkeeping and way for students to “play” with their grades

College Students(N = 299)

60% Male40% Female

50% Freshman 36% Sophomores 11% Juniors2% Seniors1% Not Reported

Autonomy: “I have more control over my �nal course grade because of the grading system.”

E�ort: “The grading system encourages me to work harder than I would in a di�erent kind of grading system.”

Exploration: “The grading system encourages me to work on assignment types I would normally avoid.”

Fairness: “I believe the grading system is fair to students.”

Survey MeasuresSample

Data were collected through an online survey administered to 299 students at the end of the term; 232 (78%) students completed the survey (see Table 1). A 1-5 Likert scale (Not at all true - Very true) was used on all survey items. Demographic, achievement (e.g., grade point average), and LMS use data were also included and matched to survey data for �nal analysis.

Table 1. Demographic information and items used to measure variables of interest.

Figure 1. To “gamify” a course one layers on extrinsic elements. To make a course gameful one must redesign it.

Research has shown that videogames can be very motivating (Ryan, Rigby, & Przybylski, 2006), and educators have begun apply game design principles (see Gee, 2003; Deterding et al., 20011) to classroom contexts in hopes of increasing student engagement and motivation. Researchers have also begun to study the e�ects of these designs (e.g., Salen et al., 2011; Martinez et al., 2012) and also support their development though online Learning Management Systems (e.g., GradeCraft: Holman, Aguilar, Fishman, 2013).

“Gameful” courses work because they support student autonomy—an adaptive student outcome (Ryan & Deci, 2000). This approach is di�erent from “gami�cation,” which simply adds extrinsic rewards to existing learning environments (Figure 1). The objective of the current study is to examine how a particular gameful design supported student autonomy, as well as understand other adaptive student behaviors it fostered.

LearningEnvironment

Gami�edLearning

Environment

badgespointsGamefulLearning

Environment

autonomy

failure

choice

play