9
Mixed Reality Serious Games and Gamification for smart education Paul Zikas, Vasileios Bachlitzanakis, Margarita Papaefthymiou, Steve Kateros, Stylianos Georgiou, Nikos Lydatakis, George Papagiannakis Foundation for Research and Technology Hellas & University of Crete, Heraklion, Greece [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] Abstract: The appeal of Mixed Reality (MR) digital games arouses interest among researchers and education specialists who since their recent proliferation, they have been trying to introduce their motivating potential in learning contexts. Our main research question in this work focuses on whether MR digital games can, via novel Presence (feeling of ‘being and doing there’ in a virtual or augmented world) and MR gamification (dynamics, mechanics, components) support and foster future learning and teaching, to address a wide variety and variation of educational contexts. To accomplish the above, a consistent computational framework that supports new types of Mixed-Reality Serious Games and Gamification (MRSGs) is established in this work that features MR gesture- based and game-based learning. The introduced term Mixed Reality Serious Games (MRSGs), refers to digital mini game-shells that allow the learners and teachers to sense the feeling of ‘Presence’ experienced under a novel MR educational learning framework, in both Virtual Reality (VR) as well as Augmented Reality (AR) formal and informal learning. The former (VR) allows for the unique feeling of ‘being there’ and ‘doing there’ in the virtual world, that will be transforming the overall game-based learning experience, via latest innovations as well as recent progress in low-cost h/w Head Mounted Displays (HMDs). The latter (AR) blends real and virtual elements so that the 3D virtual element is registered accurately in the real world and interacted freely by the learner via various mobile displays, including smart glasses, natural, gesture-based interaction (mobile RGB and RGB-D), MR virtual characters and gamified learning processes. Game-based learning also involves the incorporation of games into lessons. The principal aim of applying games in education is to increase students’ engagement and motivation. In our case studies we provide two mini MRSGs in VR and AR that accompany the primary school history class and particular the period of the Minoan Civilization, as manifested by the archaeological site of Ancient Knossos in Heraklion, Greece. The AR MRSG implements a desktop-based holographic application using the Meta-AR glasses. This MRSG consist of three mini game shells in which the student has to complete various learning tasks by using gesture-based interaction. These tasks consist of puzzle-based and constructional games tasks. The VR MRSG implements a similar approach of a Virtual tour in Knossos with quests that student need to accomplish to collect rewards. The student has to fully explore the palace, interact with special characters and complete their quests. These MRSGs are a first attempt to formally study latest h/w and s/w advances in Mixed Reality technologies, applied in gesture-based and game-based learning in both formal and informal educational contexts. Moreover we compared the gamification elements for each enabling technology and define the gamification dynamics, mechanics and components that need to be utilized in each MR environment. Keywords: Mixed Reality, Serious Games, Gamification, Game-based Learning, Virtual – Augmented Reality Games, gesture-based learning, game-based learning

Mixed Reality Serious Games and Gamification for smart ... · Mixed Reality Serious Games and Gamification for smart education Paul Zikas, Vasileios Bachlitzanakis, Margarita Papaefthymiou,

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Mixed Reality Serious Games and Gamification for smart ... · Mixed Reality Serious Games and Gamification for smart education Paul Zikas, Vasileios Bachlitzanakis, Margarita Papaefthymiou,

MixedRealitySeriousGamesandGamificationforsmarteducationPaulZikas,VasileiosBachlitzanakis,MargaritaPapaefthymiou,SteveKateros,StylianosGeorgiou,NikosLydatakis,GeorgePapagiannakisFoundationforResearchandTechnologyHellas&UniversityofCrete,Heraklion,[email protected]@[email protected]@[email protected]@[email protected]: The appeal of Mixed Reality (MR) digital games arouses interest among researchers and educationspecialistswho since their recentproliferation, theyhavebeen trying to introduce theirmotivatingpotential inlearningcontexts.Ourmain researchquestion in thiswork focusesonwhetherMRdigital gamescan, vianovelPresence (feeling of ‘being and doing there’ in a virtual or augmented world) andMR gamification (dynamics,mechanics,components)supportandfosterfuturelearningandteaching,toaddressawidevarietyandvariationofeducationalcontexts.Toaccomplishtheabove,aconsistentcomputationalframeworkthatsupportsnewtypesofMixed-RealitySeriousGamesandGamification (MRSGs) isestablished in thiswork that featuresMRgesture-basedandgame-basedlearning.

The introduced term Mixed Reality Serious Games (MRSGs), refers to digital mini game-shells that allow thelearners and teachers to sense the feeling of ‘Presence’ experienced under a novel MR educational learningframework,inbothVirtualReality(VR)aswellasAugmentedReality(AR)formalandinformallearning.Theformer(VR)allowsfortheuniquefeelingof‘beingthere’and‘doingthere’inthevirtualworld,thatwillbetransformingtheoverallgame-basedlearningexperience,vialatestinnovationsaswellasrecentprogressinlow-costh/wHeadMounted Displays (HMDs). The latter (AR) blends real and virtual elements so that the 3D virtual element isregisteredaccurately intherealworldand interactedfreelybythe learnerviavariousmobiledisplays, includingsmart glasses, natural, gesture-based interaction (mobile RGB and RGB-D), MR virtual characters and gamifiedlearningprocesses.

Game-basedlearningalsoinvolvestheincorporationofgamesintolessons.Theprincipalaimofapplyinggamesineducationistoincreasestudents’engagementandmotivation.InourcasestudiesweprovidetwominiMRSGsinVRandARthataccompanytheprimaryschoolhistoryclassandparticulartheperiodoftheMinoanCivilization,asmanifestedbythearchaeologicalsiteofAncientKnossosinHeraklion,Greece.

TheARMRSGimplementsadesktop-basedholographicapplicationusingtheMeta-ARglasses.ThisMRSGconsistof threemini game shells in which the student has to complete various learning tasks by using gesture-basedinteraction. These tasks consist of puzzle-based and constructional games tasks. The VR MRSG implements asimilarapproachofaVirtualtourinKnossoswithqueststhatstudentneedtoaccomplishtocollectrewards.Thestudenthastofullyexplorethepalace,interactwithspecialcharactersandcompletetheirquests.

TheseMRSGs are a first attempt to formally study latest h/w and s/w advances inMixed Reality technologies,appliedingesture-basedandgame-basedlearninginbothformalandinformaleducationalcontexts.Moreoverwecompared the gamification elements for each enabling technology and define the gamification dynamics,mechanicsandcomponentsthatneedtobeutilizedineachMRenvironment.

Keywords:MixedReality,SeriousGames,Gamification,Game-basedLearning,Virtual–AugmentedRealityGames,gesture-basedlearning,game-basedlearning

Page 2: Mixed Reality Serious Games and Gamification for smart ... · Mixed Reality Serious Games and Gamification for smart education Paul Zikas, Vasileios Bachlitzanakis, Margarita Papaefthymiou,

1. IntroductionWhenvisitinganarcheologicalsite,especiallywhenitisnotwellpreserved,thereisalotofinformationthatitishard to assimilate. Facts about the daily habits of the residents, how the buildings looked when they werepopulated and even details considering missing artifacts need a visual representation to improve the visitors’perception.Reflectingon this need for visualization,we introduced thenovel conceptofMRSGs (MixedRealitySerious Games and Gamification) to primary school students to enhance their enthusiasm and interest for thehistorylessontakingasacasestudythearcheologicalsiteofKnossosandtheMinoancivilization.MixedReality(MR)isafieldofgreatinterestspeakingofeducationalgames.TheusercanenjoyinteractiveVirtualenvironments,andamusementriseswhentheexperienceofaudiovisualsimulationisstrong.Ononehand,GoogleCardboard is considered one of themost efficient appliances formobile VR experiences. The combination of ahandyportabledeviceandGoogleCardboardcanofferaninexpensive,userfriendlyVRinteractions.Ineducation,the used apparatus has to be as simple as possible, because the main center of attention needs to be theapplicationitself.Sinceourtargetgroupisprimaryschoolstudents,alowcostVRexperienceseemstobeanidealtool for the lessons of history and Greece’s Mythology. On the other hand, recent Augmented Reality (AR)advances,suchasthosesupportedbygesturebasedinteraction(Billinghurst,2015a,2015b)createanovelgamingplatform ideal fordesktop-basedholographicapplications.WearingtheMeta-ARglasses (orothersimilarRGB-Dsee-throughglassessuchasMicrosoftHoloLens)cangenerateauniqueARgamingexperience.Suchaholographicheadsetwithgesturerecognitionisstudiedforthefirsttimeinthiswork(tothebestofourknowledge)asameansto promote enhanced interaction with the augmented content causing a more integrated experience. ThistechnologyhasthepotentialtoattracttheinterestofstudentsandeducatethemthroughnovelARgaming.Thus our main novelty is the creation of two MRSGs with similar educational content for both VR and ARtechnologiesinordertocomparetheirspecificallyadaptedgamificationelements,thelearningexperienceandthegameplay in general. The gaming elements and gamificationprinciplesneed tobe as close as possible for bothapplicationtohaveaccurateresults.BothgamesweredevelopedusingUnity3D.Thefirstgamewascreatedforthe Google Cardboard VR HMD presenting a Virtual tour in Knossos. The second one refers to the samearcheological site but iswas developedusing theMeta-AR glasses. By comparing our gesture-based and game-basedlearningapproachesintwodifferentplatformsandrealitieswecanextractthegamificationelementsthatneedtobemodifiedorremainintactthroughthesenovelMRSGs.

2. PreviousWorkBased on a previous version of a VR tour in Knossos (Kateros et al, 2015), we enhanced the user’s presence(Papaefthymiouetal,2015)byaddingmoreinteractionfeaturesandavoidingusingquestion-basedtasks.Thiswillincreasethestudents’motivation,promptingthemtoexploreagreaterareaofthearcheologicalsite.Themodelof Knossos we used it’s a realistic representation of the palace as it stands today. To create a better userexperience (UX)we need to present a 3Dmodel of the digitized archeological site as realistic and accurate aspossible(Foni,Papagiannakis,andMagnenat-Thalmann,2010).Aseriousgameforculturalheritage(Andersonetal, 2009) is a successful implementation to enhance the interest of primary school students. Implementing thisideainmobileVR(Psotka,2013)canofferamoreeffectiveleaningenvironment.Anotherapproach(Deterdingetal, 2011) defines the gamification elements and what differentiates serious games from gamified applications.Theseelements(dynamics,mechanicsandcomponents)needtohaveaclearstateinordertopromotelearning.Another study (Freina and Canessa, 2015), analyses the importance of immersive Virtual environments andcompares them with VR desktop based applications. Finally, Sanchez and Pierroux (2015) have developed agamifiedexperienceforanationalhistorymuseum,showingthebasiclearningprinciples.Accordingtosomestudies(Papagiannakis,Singh,andMagnenat-Thalmann,2008),AugmentedRealityenablesanideal visualization of cultural heritage providing certain key edutainment aspects. Additionally, AR features acompletelydifferentaugmentedenvironmentwhichcanbeusedtogamifynewlearningconcepts(Vacchettietal2004)inamoreproductiveandenjoyableway.However,theinteractionbetweenuserandthedeviceneedstobe

Page 3: Mixed Reality Serious Games and Gamification for smart ... · Mixed Reality Serious Games and Gamification for smart education Paul Zikas, Vasileios Bachlitzanakis, Margarita Papaefthymiou,

asnaturalaspossibletoavoidawkwardsituationsinpublicplaces(Carmignianietal,2011).Consideringthis,theMeta-ARglassesintroducearobustgesturerecognitionsystemthatprovidessmoothandefficientinteractionwiththe augmented elements. Moreover, Papaefthymiou et al (2015) proposed a pipeline for robust authoring ofaugmentedcharacterscapableofperforminggestures,objectmanipulation,locomotionandlipsyncing,conceptswithgreatimportanceforrealisticcharactersthattheirprincipleswereemployedinthiswork.

3. VRMRSGgameplaydescriptionThisMRSGis focusedonpromotingthemobileVirtualRealityfeelingaswellaseffortlesseducationoftheuser.Cardboard,iscurrentlylackingtheabilityofactiveinteraction,andtheonlyavailableinputmethodisthetrigger.Howeverwecanoverpassthislimitationbyusinganotherinputdevicesuchasagamepadoranothersmartphone(Papaefthymiouetal,2015b).Theoffered trigger, isused toenableordisableaccordinglyauto-movement. It isimportant to note that the characters use SALSA asset for lip-synching and eyemovement for amore naturalconversation(Yungetal,2011).TheMRSGtakesplaceinthepalaceofKnossosasitstandstoday,andthemainstoryingeneralusesacombinationof real and mythical sceneries. In the opening picture we find ourselves in the archaeological site of AncientKnossosandhave the capability to freely lookaround theenvironment.Whilehaving the first contactwith theVirtualRealityexperience,anarcheologistgreetsyouandasksyoutofindfivelostartifacts.Wehaveimplementeda simple interactive conversation mechanism, in which the player can communicate with other characters bychoosingoneoutoftwopossibleresponses.Itisalsoworthnoting,thatMRSGprohibitstheusageofa2D-GUIasitcan’tbeimplementedinaVRapplication,sotheGUIsaredisplayedinathreedimensionalmanner.Thevirtualsiteis a replicate of the original palace, so the exploration of the scenery can be correlatedwith actually being inKnossos.Whilestrollingaroundtheruins,youcomeacrossothercharacterswhohaveastorytotell,andatasktoappoint. Thementioned story is an introduction to the corresponding task, and sets themoodof theplayer towhatwillfollow.Inaddition,informationthroughconversationandinteractionisgiventotheplayer,andthemostcuriousstudentscouldachieveamoredetailedmaterialonthesubjectbychoosingtheanalogousresponse.TheseMRSGsarebasedonstoriesfromtheGreekMythologythattookplacearoundtheMinoancivilization.ThefeelingofpresencethatVirtualRealityoffers,entails toagamedesignfocusedtomaketheplayerexperienceapersonal view of themyth. By recreating themythological stories, we try to intrigue the student intowillinglydiscoveringthecultureofAncientGreece.Ineachsimulation,theplayerretrievesalostartifactaftercompletingit.Themythsthatthemini-gamesarebasedonare:theLabyrinthofKnossos,IcarusandDaedalus,MountIda,andTheseus.

Figure1:(Upleft)Interactionwithcharacters.(Upright)FindthepiecesofTheseus’whitesail.(Downleft)FlyingtothesunlikeIcarus.(Downright)Thenorthentranceofthepalace.

Page 4: Mixed Reality Serious Games and Gamification for smart ... · Mixed Reality Serious Games and Gamification for smart education Paul Zikas, Vasileios Bachlitzanakis, Margarita Papaefthymiou,

More specifically, for the myth of the Labyrinth we implemented a game where the player is trapped in theLabyrinthofKingMinosand tries toescape. Inorder todo that, the string leftbyAriadnehas tobe foundandwhen you acquire the ball of string a trail that leads to the exit appears. A dark labyrinth seems an idealenvironmentforaVRgameduetotheimmersion.RegardingthemythofIcarusandDaedalus,thespotlightofthestoryiswhenIcaruswrongfullyusedthewingscreatedbyhisfatherDaedalusandflewtooclosetothesun.Thestudent can experience a thrilling flight simulationwith the goal of the game to pass through all twenty ringslocatedacrossthemap.TheintentionoftheentireVRgamecanbeperceivedcharacteristicallythroughthismini-game, because knowledge is acquired through personal experience, while playing as one of the heroes of themyth. Subsequently, in anothermini-game, the player can climbMount Ida to reach the cavewhere Zeuswasborn. Tomake the experiencemore challenging, you have to avoid random rocks, stones and bushes and thecollision with one of these will slow you down. Knowledge is attained about, the notable to many, Zeus andinformationthatfewknowaboutthekingofthegodsofMountOlympus.Finally,thelastrecreatedstoryisbasedonamythwithamodification.ThestudentgetstheroleofasoldierfromtheshipofTheseus,andisaskedtofindinsideatavernfivetornpiecesofwhitesailotherwiseTheseus’fatherwilljumptotheseafromsorrow.Topreventthis,thestudentneedstohelphimconstructawhitesailtoreturnhome.Theimpressionofhelpingothers,andcontributingtoyourowneducationonthemythologicalcontentandstoryisthegoalofthisMRSG.Additionally,beyondthemythswewantedtosecurethat theplayer trulyexplores thepalaceofKnossossoweformed the last task to provoke the students’ interest for the site. Therefore, the final request comes from asecuritypersonnelthatpolitelyasksyoutotakethreepicturesofthemostfamoussites inthepalace.Onceyoureturnwith the pictures, you receive the fifth and final artifact. To complete the experience and finish the VRgame,youneedtoreturnthefiveartifactstotheArchaeologistwhoiswaitingforyouatthebeginning.TheaboveMRSGs,aswellastheinteractionwiththecharacters,wouldn’tbesoeffectivelyimplementedwithouttheuseofVirtualReality.ThereasonisthatthefascinationofthestudentwiththefeaturesofMRSG,couldachieveawell-respectedlearningoutcomethroughamusement.

4. ARMRSGgameplaydescriptionwithMeta-ARglassesFor theARversionof thegameweusedMeta-ARglasses to implementaholographicapplicationable tobeingmanaged with gestures. Augmented reality glasses bring the AR content directly to user’s view by usingstereoscopiccameras,providingabetterexperiencethanconventionalARapplicationsformobiledevices.Meta-AR glasses except for the projection of digital content they have a build-in DepthSense camera which offersgesturerecognition.Withthisfeature,userhastheabilitytocontrolandmanipulatethehologramsheseeswithhand gestures. This ability is better described as the IronMan phenomenon (Pedersen et al, 2012). The basicgesturesprovidedaregrabbing,pinching,grabbingwithbothhands,gazing,pointingandtouching.Thesegesturesconstitute a basic interaction library suitable to manipulate the digital environment. Each gesture has its ownparameters that can be adjusted to match the application’s requirements such as personalize the grabbingmechanism,changethewaypinchingworksoradjustthegrabbingdistance.We used gestures in many different ways, from basic picking and moving digital objects to interaction andcommunicationfeatures.Tobeginwiththegrabbinggesture,MetaSDKaddsvisual indicationstodeterminethepositionofthepalmbyusingtwocircles,oneforthehandpositionandtheotheronetodetectifthefingersareformingafist.Grabbing isusefultomoveobjectsand it is firmlyused inthisapplicationas it isanativegesturethatuserwill perform to interactwithphysicalobjects.Grabbingwithbothhandshas similar featureswith thesinglehandgrabbingbutisusedtorotateanobject.Finally,thetouchinggesturewasusedtoselectspecificpartsofanobjectthathaveaparticularroleforthegame’sflow.Asmentionedbefore,acircleisrenderedtotheuser’sfingertiptomakepickingeasier.Exceptfromgestures,wealsousedmarkerstocontrolbuildingmodelsthatarebettermanipulatedwith this feature.Marker based tracking has the advantage ofmore accurate control overgesturebasedasthemodelisplacedontopofamarkerandbothhandsareusedtomanipulatethedigitalobject.

Page 5: Mixed Reality Serious Games and Gamification for smart ... · Mixed Reality Serious Games and Gamification for smart education Paul Zikas, Vasileios Bachlitzanakis, Margarita Papaefthymiou,

AsthegamestartsthereisabriefexplanationofthearcheologicalsiteofKnossosbythenarrator,consideringitisnotverywidelyknown.ThisisdonenotjustbyasimplerecountingofthefactsbutthenarratorasksthestudentstofindthespecificlocationofGreeceandthentheislandofCretewhereKnossosbuilt.WeusedthismethodtostartthestorytellingofKnossoswithamoreinteractiveapproachandalsotointroducethebasicgesturesstudentsareabletoperformwithMeta-ARglasses.After thisshort introductionthere is thepresentationof theKnossospalaceascurrentlystands.Atthispointit’sthefirsttimeweintroducethemarkerbasedtrackingasstudentscanrotate,zoomandperformvarioustransformationstotheKnossosmodel.

Figure2:(Left)Bullleapingpuzzle,grabbingtileswithbothhands.(Middle)PutthecolumnsofNorthEntrancebackinplace.(Right)Constructingcolordyestorestorethepainting.Themain part of the narration consist of threeMRSGs that tests student’smemory, constructing and learningabilities.Thefirstisapuzzleofthebullleapingwallpainting.Beforeputtinguser’sabilitiestothetest,theoriginalpainting isshown,followedbyanexplanationonthematerialityofbullrituals inMinoancivilization.To interactwiththetiles,thegrabbinggestureisusedasitisthemostsuitableforthissituation.Ontheleftthereisacanvaswith other wall paintings and statuettes foundwith the bull leaping theme that user can navigate through bypressingthebuttonsprovided.Asweareusingholographicheadsetabletoprojectobjectsanywhereuserpointshishead,wewantedtoextendthefocusfromthemainsceneandencouragestudentstoexplorethesurroundingphysical environment in order to findmore augmented elements. The secondMRSG is dedicated to theNorthentranceof thepalace.At thebeginningwehaveapresentationof theNorthentrancewith theuseofmarkerbasedtracking.Afterthat,studentsneedtoreconstructtheNorthentranceasthecolumnsthatadornthefrontside have fallen. The columns have different heights because the building is not well preserved causing eachcolumntohaveonlyonecorrectposition.Tocompletethepuzzlestudentsneedtograbthecolumnsandplacethemintherightorder.ThethirdandfinalquestpresentsthePrincewiththelilies,afrescofromKnossos.ThegoalofthischallengeistomakecolordyesfromnaturalsourcesastheMinoansusedtodoandrestorethepainting.Alltheingredientsneededareprovidedontothestudent’sdesk.Thenarratorgivesinstructionsforthepreparationofeachcolorandthestudenthastoselecttherightingredients,puttheminthemixingbowlandthentakethedyeandpaint the fresco.Aftermanufacturingeachcolor,userhas tograb thedyeandpaint the fresco todisplayacoloredpartofthepainting.ThisisanidealtaskforMeta-ARglassesasitrepresentsamanualworkthatshouldbedoneusinghands.Whenthepaintingisfullyrestored,thenarratorwalksinthescenetocongratulatetheuserandmakethefarewellspeech.

5. ResultsandComparisonofVR-ARgesture-basedandgame-basedlearninginourMRSGmethodologyAlthoughwearereferringintwodifferentMRenablingtechnologies,bothVRandARMRSGshavesomecommongamification principles, concerning the learning elements. There are three main categories of gamificationelements(YDP,2015)eachgamemustfeatureinordertoaccomplishthelearninggoalsanddescribedinthefigurebelow:1)Dynamics,thehighestlevelelements,theyincludeconceptsasthenotionandthegeneralexperienceofthegame(e.g.plot).Theyconstitutethenarrativestyle,theplotandthegoalsofthegamethatshouldbeclarified2)Mechanicsthatdefinethesetofrulesandtheelementsthatdrivetheuserstocontinueexploringthegamee.g.rules. These consist of tasks and quest provided by gamified characters and various challenges or in-gameactivities.3)Componentswhichisthelessabstractcategoryofelementsreferringtospecifictoolssuchaspoints,awardsandvirtualgoodsthatenrichthegamingexperience.Tryingtoapplythesamelearningprinciples intwo

Page 6: Mixed Reality Serious Games and Gamification for smart ... · Mixed Reality Serious Games and Gamification for smart education Paul Zikas, Vasileios Bachlitzanakis, Margarita Papaefthymiou,

differenttechnologiescanbeahardtask.BothARandVRplatformshavetheirownwayofcreatingthelearningexperiencethusweneedtoconsiderwhichparticularfeatureseachplatformperformsbest.

Figure3:Thepyramidofgamificationelements(YDP,2015).5.1Immersive(VR)vsdesktop(AR)gesture&game-basedlearningVirtual Reality is mainly used to create immersive environments in order to have a more realistic gamingexperience.ThefeelingofpresenceinaVRgamecanenrichthelearningelementsasuserwillperformtasksinadigitalenvironmentverysimilartotherealworld.Bycreatingthisexperience,thelearningabilityofagamehaveagreatereffectonuserasthegameplaystimulatesthebraintheexactwayitisdoneinreallifeconditions.ThisisthereasonwhyprofessionalsimulationsuseVRtechnologytoreproducecertainsituationsinwhichtraineesneedto respond in a specificway. On the other hand, in a holographic application the feeling of presence is not asobvious.Forthis,weneedtoconcentrateinotheraspectsofAugmentedtechnologytoproduceabetterlearningexperience.Whenwe start developing both applications, we faced amajor dilemma on how to represent the same gamecontentinbothtechnologies.InVRwehaveacompletelydigitalimmersiveenvironmentbutontheoppositewehavetolimitthedigitalcontenttofitintherealworld.Asitseems,theAugmentedversionofthegamewillbethebasemodelfortheDynamicsoflearningelementsasitislessformativethanafullyVirtualworld.Consideringthis,theAR gamewasdeveloped as desktop-based,meaning all the gameelementswere scaleddown to fit onto adesk.BydoingthiswehadsomeseriousrestrictionsabouttheARgameplayandmorespecificallyabouttheplotandthenarrativestyleofthegame.InVRgamesuserisfreetoexplorethevirtualwordandfindkeyitemsortakespecificquestsfromothercharacterswhichcanbeusedtocontinuetheplotandachievethelearningpurposes.However in a desktop-basedholographic applicationdue to the lack of user’smovementwehave tomake thegamelooklikeapresentation.This isthereasonweintroducedanarratortomediatebetweentheuserandthegame’splot.Despitethelackofalife-sizednarrator(Zikasetal,2016)causedbythelimitedfieldofview,Meta-ARglasses suffers, we managed to create an interactive character able to navigate the user through the game’squests.Ontheotherhand,inaVirtualenvironment,anarratorisnotamandatoryelementoftheplotastheusercannavigatethroughthemapandproceedtothedesirablequestonhisown.Forthisreasonwereplacedtheleadnarratorwithdifferentcharacterswaitingtheuseratthesiteofeachquesttogivethenecessaryinstructions.

Page 7: Mixed Reality Serious Games and Gamification for smart ... · Mixed Reality Serious Games and Gamification for smart education Paul Zikas, Vasileios Bachlitzanakis, Margarita Papaefthymiou,

Figure4:GameplaycomparisonbetweenMeta-ARglasses(left)andVRGoogleCardboard(right).5.2InteractionwithvirtualMRSGelementsAsmentionedbefore,weusedMeta-ARglassesfortheAugmentedversionofthegame.Thisholographicheadsetsupportsgesturerecognitionwhich isundoubtedlythecoreof theapplication’sMechanics.The interactionwithdigitalobjectswasthekeyelementinwhichtheapplicationwasbased.Usingthismethod,wedevelopedquestsandminigamesaccordingtogesturebasedinteraction.Asaresult,thequestsneededtobemanualtasksforuserto complete them with his hands and create a realistic simulation of real object manipulation. A successfulexampleofthis ideaisthelastquestofthegame,comprisingtheconstructionofnaturalcolordyestorestoreadamagedpainting.Thisisaprocessthatisnaturallydonewithhandssoitmatchedperfectlywithourneeds.Thefeedbackgainedfromthisquestexceptfromthemanufactureofcolorpaintwithplantsandnaturalsourceswasthefactthatthiswasahardandtimeconsumingmethodwhichisamplifiedbyreenactingthisprocedure.

Figure5:GesturebasedinteractionwithMeta-ARglasses.Inthisfigureweperformarotationoftheglobebyusingbothhands.Greencirclesindicatethepositionofpalms.In order to have similar game content for both technologieswe decided tomake a pickingmechanism for thecardboard game tomaintain the sameprinciples for both games. This picking feature is activatedwhenuser isapproaching and gazing at a gameobject for two seconds. Being able to interact with objects in a virtualenvironmentprovidesavarietyoftasksausercanhandle,increasingthegame’slearningabilities.ThismethodisusedasasubstituteforgesturebasedpickingprovidedbyMeta-ARglassesbutitisafunctionalabilitywhichdoesnotneedanykeypressing.AtthispointweachievedtocreateaMRSGintwocompletelydifferentplatformsbutstillmaintainabasicgameplaycorebasedonpickingdigitalobjects.

Page 8: Mixed Reality Serious Games and Gamification for smart ... · Mixed Reality Serious Games and Gamification for smart education Paul Zikas, Vasileios Bachlitzanakis, Margarita Papaefthymiou,

5.3BasicgamificationMRSGComponentsTherearegamingelements thathave to remain thesame inboth technologies.Considering theComponentsofthe gamification elements there is notmuchdifferentiationbetween theVR andAR versionof the game. Bothgames have the same components as these values are not affected by the technology shift. Elements such asawards, unlocking quests and collectibles are basic Components both games implement. In the VR game aftercompleting a quest, a special award is given to increase themotivation for further exploration. TheAR versiondoesnothaveavisiblecollectabletotakebutthenarratorcongratulatestheuser forcompletingthetask.Bothversionshaveanawardmechanismimplementedindifferentwaysaccordingtotheplatform’scapabilities.

Figure6:Architecturaldiagramofbothapplications.Thegamificationelementsaselaboratedtofitbothtechnologies.

6. ConclusionsandFutureWorkInthisworkwepresentedacomparisonoftwoMRSGsbetweentwodifferenttechnologies.ThemainconceptwastocompareaVirtualandanAugmentedRealitygamified tour in thearcheological siteofKnossos.Weusedthesamegameprinciplesforbothgamesinordertohavesimilargamecontentandaccurateresults.Weconcludedthat as you procced towards the base of the gamification elements’ pyramid the gaming values remain intactthroughthetechnologyalternation.TheDynamicsofagameareencapsulatedintheusedtechnologyanditisnotfeasibletochangethemwithoutdistortingthecontent.However,MechanicsandComponentsaremorelikelytoabsorbminimal changes andnot affect the game’s core. In order to design a game in different technologies, adeveloperhasto identifywhicharethegamificationelementsthatarecrucialfortheapplicationandwilldefinethe gameplay’s core. This is amajordecision tobemadeas theseelementswill eventually form the gameanddesignate its content. In the future we aim to use the open source OpenGL Geometric Application (glGA)framework(Papagiannakisetal,2014)forimplementingbothversionsofthegame.ConsideringtheARMRSGweplantoaddlifesizedfeaturestooverpassthedesktop-basedlimitationsthisversionhas.AvideoofoursystemandMRSGsiscurrentlyavailableat:https://dl.dropboxusercontent.com/u/43768664/ECGBL2016.mp4

7. AcknowledgementsThe research leading to these results has received funding from the European Union People Programme (FP7-PEOPLE-2013-ITN)undergrantagreementN.608013.SpecialthanksalsotoARdictive(www.ardictive.com)anditsCEO&founderNilsHuebnerforthemetaARglasses.

Page 9: Mixed Reality Serious Games and Gamification for smart ... · Mixed Reality Serious Games and Gamification for smart education Paul Zikas, Vasileios Bachlitzanakis, Margarita Papaefthymiou,

References A.Foni,G.Papagiannakis,andN.Magnenat-Thalmann,“ATaxonomyofVisualizationStrategiesforCultural

HeritageApplications,” ACM Journal on Computing and Cultural Heritage, vol. 3, no. 1, pp. 1–21,Mar.2010.

AndersonEike,F.,McLoughlin,L.,Liarokapis,F.,Peters,C.,Petridis,P.,Freitas,S.(2011),'SeriousGamesforCultural Heritage Technologies'. VAST: International Symposium on Virtual Reality, Archaeology andIntelligentCulturalHeritage-ShortandProjectPapers.TheEurographicsAssociation.

Billinghurst,M.,Clark,A.,andLee,G.'ASurveyofAugmentedReality',FoundationsandTrendsinHuman- ComputerInteraction.Vol.8,No.1(2015)1–202,2015bBillinghurst,M.(2015a)'GestureBasedARInteractionResearchAtTheHITLabNZ'.PresentationPedersen,I.andSimcoe,L.(2012),‘TheIronManPhenomenon,ParticipatoryCulture,&FutureAugmentedReality

Technologies'.InCHI’12ExtendedAbstractsonHumanFactorsinComputingSystems,CHIEA’12,pages291–300,NewYork,NY,USA.

Deterding,S.,Dixon,D.,Khaled,R.,andNacke,L.(2011),'Fromgamedesignelementstogamefulness:Defining"gamification"'. In Proceedings of the 15th International Academic MindTrek Conference: EnvisioningFutureMediaEnvironments,MindTrek’11,pages9–15,NewYork,NY,USA.

E.SanchezandP.Pierroux,“GamifyingtheMuseum:ACaseforTeachingforGamesBasedLearning,”ECGBL2015- 9thEuropeanConferenceonGame-basedLearning,2015.G.Papagiannakis,G.Singh,andN.Magnenat-Thalmann,“Asurveyofmobileandwirelesstechnologiesfor augmentedrealitysystems,”Comp.Anim.VirtualWorlds,vol.19,no.1,pp.3–22,2008.G.Papagiannakis,P.Papanikolaou,E.Greasidou,andP.Trahanias.glga:anopenglgeometricapplication

Framework for a modern, shader-based computer graphics curriculum. Eurographics 2014, pages 1–8,2014.

J.Carmigniani,B.Furht,M.Anisetti,P.Ceravolo,E.Damiani,andM.Ivkovic,“Augmentedrealitytechnologies, systemsandapplications,”MultimedToolsAppl,vol.51,no.1,pp.341–377,2011.Kateros,S.,Georgiou,S.,Papaefthymiou,M.,Papagiannakis,G.,andTsioumas,M.(2015),Acomparisonof

gamified, immersive VR curationmethods for enhanced presence and human- computer interaction indigitalhumanities.InternationalJournalofHeritageintheDigitalEra,4(2):221–233.

L.FreinaandA.Canessa,“ImmersivevsDesktopVirtualRealityinGameBasedLearning,”ECGBL2015-9th EuropeanConferenceonGame-basedLearning,2015.P.Zikas,M.Papaefthymiou,V.Mpaxlitzanakis,G.Papagiannakis”Life-sizedGroupandCrowdsimulationinMobile AR”,ComputerAnimationandSocialAgents(CASA2016),MIRALab,UniversityofGeneva.Psotka,J.(2013),'EducationalGamesandVirtualRealityasDisruptiveTechnologies.’EducationalTechnology& Society16(2),69-80Papaefthymiou,M.,Feng,A.,Shapiro,A.,andPapagiannakis,G.(2015)'Afastandrobustpipelineforpopulating

mobile AR scenes with gamified virtual characters'. In SIGGRAPH Asia 2015 Mobile Graphics andInteractiveApplications,SA’15,pages22:1–22:8,NewYork,NY,USA.

Papaefthymiou,M.,Plelis,K.,Mavromatis,D.,Papagiannakis,G.,“MobileVirtualRealityfeaturingasixdegreesoffreedom interaction paradigm in a virtual museum application”, Technical Report, FORTH-ICS/TR-462,2015b

Vacchetti,L.,Lepetit,V.,Papagiannakis,G.,Ponder,M.,Fua,P.,Thalmann,D.,andMagnenat-Thalmann,N.,“Stablereal-time AR framework for training and planning in industrial environments,” Virtual Reality andAugmented Reality Applications in Manufacturing, Ong, S. K., Nee, A.Y.C. (eds), ISBN: 1-85233-796-4,Springer-Verlag,pp.125–142,Jan.2004.

Y.Jung,A.Kuijper,D.Fellner,M.Kipp,J.Miksatko,J.Gratch,andD.Thalmann,“Believablevirtualcharactersin human-computerdialogs,”Eurographics2011-StateofTheArtReport,2011.YDP, Young Digital Planet, “The Book of Trends in Education 2.0,” pp. 1–340, http://www.ydp.eu/trendbook-download/,2016.