An Interactive Mobile Augmented Reality Magical Playbook: … · 2017-01-18 · learning numbers using an old folklore literature, The Thirsty Crow, via mobile augmented reality application

Procedia Computer Science 25 ( 2013 ) 123 – 130

1877-0509 © 2013 The Authors. Published by Elsevier B.V.Selection and peer-review under responsibility of the programme committee of the 2013 International Conference on Virtual and Augmented Reality in Educationdoi: 10.1016/j.procs.2013.11.015

ScienceDirect

2013 International Conference on Virtual and Augmented Reality in Education

An Interactive Mobile Augmented Reality Magical Playbook: Learning Number With The Thirsty Crow

Azfar Bin Tomia, Dayang Rohaya Awang Ramblib* a,bUniversiti Teknologi PETRONAS, Bandar Sri Iskandar, Tronoh, 31750 Perak, Malaysia

Abstract

This paper presents the development of an interactive mobile augmented reality magical playbook for preschool children in learning numbers using an old folklore literature, The Thirsty Crow, via mobile augmented reality application and interactive physical book interface design. Augmented Reality (AR) refers to a technology that gives the ability to the user the sense of the real world while interacting with the virtual and physical object. Mobile refers to portability usefulness of the application itself, thus mobile AR application can be referred as a portable AR application. The real world can be enhanced by AR through augmented virtual object into real environment and providing some additional information for users. By applying this concept to AR storybook, the physical book (the real world) will be enhanced by augmenting the virtual object (3D models, animations, and sounds) viewed over a mobile device. This paper specifically highlights an innovative development of the interfaces for providing an AR storybook that enhanced story reading and learning experience for preschool children via mobile AR application and highly interactive physical book interface design of the AR book. Findings from the initial observational study based on developed prototype are also presented. For future work, there will be a series of interactive mobile AR magical playbook using other old folklore story.

© 2013 The Authors. Published by Elsevier B.V. Selection and/or peer-review under responsibility of the programme committee of the 2013 International Conference on Virtual and Augmented Reality in Education.

Keywords: Augmented Reality; mobile AR application; interactive physical book; preschool storytelling

1. Introduction

It is important for children to have early literacy education. This is essential for future education that establishes the foundation of reading and writing skill. Early literacy education can develop children’s emotion,

* Corresponding author. Tel.: +60-195-721-995. E-mail address: [email protected].

Available online at www.sciencedirect.com

© 2013 The Authors. Published by Elsevier B.V.Selection and peer-review under responsibility of the programme committee of the 2013 International Conference on Virtual and Augmented Reality in Education

Open access under CC BY-NC-ND license.

Open access under CC BY-NC-ND license.

http://creativecommons.org/licenses/by-nc-nd/3.0/

http://creativecommons.org/licenses/by-nc-nd/3.0/

124 Azfar Bin Tomi and Dayang Rohaya Awang Rambli / Procedia Computer Science 25 ( 2013 ) 123 – 130

the way they think and the use of language [1]. Based on Chew and Ishak et al, literature program in preschool is aimed to create a positive attitude among the children toward literature which give a greater impact for children’s’ future. Recently, old folk, stories tend to disappear among children of the newer generation due to advancement of technology nowadays. Thus, including them as a literature lesson for preschool can revive the traditional storytelling.

However, traditional storytelling often uses a physical book which allows many interactions and sensory feedbacks between the reader and the book, for example, speech through the narration, vision through pictures and touching through the turning and pointing at the pages [2], but, these activities can be a monotonous activity. Monotonous activity can be referred as actions that are usually descriptive based and illustrated through static graphics.

Then, the electronic book (e-book) was introduced. E-book provides various multimedia supported and successfully attract user’s attention and motivate them, especially children. However, Zhou et al stated that user interaction is limited on graphical user interface (GUI) in interaction with the content of the book. For example, users are limited to the use of mouse and keyboard to control the e-book running on desktop computer. Furthermore, the used of physical book will decreased and traditional storytelling will disappear among newer generations.

Augmented reality (AR) has the ability to give the user the sense of the real world while interacting with the virtual and physical object [3]. AR technology has promising potential that can be used in enhancing a physical storybook by augmenting the book pages with graphic, animations and audio [4]. Dayang Rohaya Awang Rambli et al stated that a static story reading experience will be converted into a dynamic, fun and engaging reading experience for the children by integrating AR concept into a physical book.

In early development, most of AR book setups are based on static desktop computers with stationary cameras or on Head Mounted Displays (HMDs) [4][5][6][7][8]. However, AR book setup based on desktop computer is not efficient in context of reading book due to serious drawbacks such as high costs, low social appeal and limitations of user’s dexterity, which avoid them from targeting a wide audience of non-technical users especially children [9]. Thus, moving toward mobile AR in implementing an AR book is highly potential in enhancing a physical storybook and learning experience. Furthermore, mobile devices are widely used among newer generation no matter what age.

Interactive learning environment can turn passive students into active learners that are fully engaged [10][11]. Furthermore, engaging students in learning process can translate into greater learning. Moreover, Information and Communication Technology (ICT) has the potential in enhancing and promoting the traditional folklore stories [1], also creating an interactive, interesting and engaging learning environment [12][13][14][15]. In this paper, we present the development of an interactive AR based edutainment story book for preschool students to learn numbers. The story was based on a popular folklore tale: “The thirsty crow”.

Conceding the importance of creating an engaging learning experience via interactive learning, the design focuses on two aspects of the book: the physical book and GUI of the virtual view of the book via mobile AR application. The physical book was designed in such a way to increase user interactivity while reading the book. The GUI, which also aim to increase user interaction with the system, allows user to control the AR book with ease as well improve the presentation of the virtual view of the book using mobile AR application. Besides entertaining, the storybook could be used as a fun, motivating and engaging teaching and learning tool for preschool and lower primary schools. Related work on AR book will be presented in Section 2 followed by the overall design in section 3. The results of the observational study on the prototype will be presented in section 4. Section 5 concludes the overall paper with recommendation for future work.


2. Related Work

In previous work, Billinghurst et al introduced one of the earliest AR book called “MagicBook” [5]. Based on his work, this book allows the seamless interaction between real and virtual environment [5][16][17]. User can read the physical book as a normal book. User also can see the virtual 3D object on the physical book via handheld display. Other AR books have been implemented based on MagicBook but in different context and aspect. Medina et al introduced an AR book for biochemistry [7]. Mohd Zainuddin et al work based on AR book in science learning for deaf students [8]. AR book for geometry is introduced by Kaufmann and Schmaltieg et al [6]. Dayang Rohaya Awang Rambli et al introduced an interactive AR edutainment storybook for preschool using old folklore story [4]. The AR book is based on engaging method in learning number via a highly interactive edutainment children storybook using old Malay folklore story, “Sang Kancil and the Crocodiles”. Dunser and Hornecker et al stated that AR book is efficient and effective as instructional tool for children 5- 7 [18]. It is fun and engaging learning environment for children due to 3D animation and physical interaction of the book.

However, Grasset et al stated that most of AR book using the physical book as a container of marker which leads to lack of the effectiveness of pictures and text content of the book itself [16]. All of AR books above are implemented using desktop based computer that leads lack of efficiency in context of reading a normal book.

3. Interface Design

Fig 1 General structure of the book

The prototype concept of AR book presented in this paper is shown in Figure 1. It is based on general AR

application set up that consists of four main components which are a camera to capture the real environment and tracking, computer and display devices for virtual augmentation (mobile device), real environment (in this case, physical book) and markers as tangible interaction devices as well as tracking target [19]. The overall book design divided into story page and marker page. Story page covered the story line and illustration that illustrated the story within that current page. Marker page consists of marker that representing each 3D character with animation within that current page. In AR application, there are three main display methods which are HMD, handheld display such as mobile devices, and through the use of spatial display such as monitor based display [20][19]. This prototype is using handheld display (mobile device) for viewing the augmentation of the book because handheld display will help user to experience the AR concept while maintaining the context of reading normal book. Furthermore, mobile devices nowadays advanced in


computing power and also in 3D graphic processing with the introduction of embedded Graphic Processing Unit (GPU).

Compared to previous AR book setup which needed some initial setup before usage, this mobile AR book concept can be used directly as a normal user reading a normal book. Thus, it will reduce the learning hassle as well as increasing the interactivity between user and book. This so called playbook is mainly focused on two parts which consist of physical book and mobile application.

3.1. Physical Book

Fig 2 Physical book

Figure 2 shows the physical book of the prototype. The richness and aesthetics of the pictorial content of the physical book playing an important role for first impression in grabbing user’s attention [16][8][4]. It will also give a playful learning environment targeted to attract children into learning process with the content of the book. Previous AR books are mostly used the container of the marker to represent each of augmentation object. In this AR books, marker page is designed to be fully ‘invisible’ to user. Refer to figure 1 and 2, we can see that marker page is designed based on AR marker criteria such as high contrast and detailed image feature [3] but we also include richness and aesthetics aspect of the picture itself as it is blend together with the overall page.

Natural interaction between user and the physical book should be included in AR book to maintain the context of normal user reading normal book. As AR book, it is aimed to enhance the traditional book, but not to replace the entire book [4]. Thus, the normal interaction with the book such as pointing will be presence in AR book. Tangible User Interface (TUI) in AR is the interaction that uses a physical environment to be tangible while interacting with the AR system [19]. In this prototype, finger can be used to interact with the AR book. Figure 4 shows the interaction with the AR book using finger.


Fig 3 the interaction with the AR book using finger

3.2. Mobile Application

Mobile AR application has gaining popularity nowadays due to mobile technology advancement. As mention early, mobile devices advanced in computing power and also in 3D graphic processing. This is the key to unlock the potential of developing an interactive mobile AR edutainment book for children. Some of work in mobile AR such as Mobile Maze [21], AR Tennis [22], Penalty Kick [23], Vidente [24], Art of Defense (AoD) [25] and etc. are being referred in developing mobile application for this AR book. This mobile application developed in Android platform. Figure 4 shows the mobile application installed in Android device.

Fig 4 Mobile application for AR book installed in Android device

The mobile application included with the ability to augment the 3D character and animation (the crow) with

audio onto the marker page of the book. Figure 5 shows the user using the mobile application with the book.


Fig 5 The user uses the mobile application via mobile device to augment the character of the book

This application is not only augmenting the 3D character, animation and audio, but it also provides the users with narrator. The narrator is aimed to help children as guidance for them to read throughout the storyline. The learning number part is shows in figure 6.

Fig 6 Learning number part

Based on figure 6, learning number part is highly interactive designed for children. The user will interact with the book using their finger to count together with animation of the augmented 3D character on the book, leaving the natural means interaction between the user and the book. From here, the concept of engaging the student within learning environment in the learning process is fully applied. Furthermore, highly interactive design of the mobile AR application within AR book will create a fun, motivating and engaging teaching and learning tools.


4. Observational study

The observational study of the prototype is based on user feedbacks. User feedbacks are gathered from visitors during exhibition. The users who tried the prototype were from different background and age. Targeted user feedbacks were gathered from parents and young children. Based on the observation, most of targeted users excited with the AR technology concept applied on the book. AR concept of the book successfully grabs their attention. Informal observation of the systems’ use among users elicits very encouraging response from them; indicating it’s potential as an engaging and fun learning tool. Users were likely to use this mobile AR application with the book without being helped even young children know how to use it. This observation shows that this mobile AR application with the book is easy to use especially for young children. Most parents asked whether the book, together with mobile AR application is already available at the moment in the market. Even they were willing to buy the book if it’s already available for commercialization.

Furthermore, the TUI element in this book is the main attraction of the book. Users would likely to interact with the 3D character on the book especially young children. The excitement of interacting with the book’s character using their finger can be shown based on their reaction. In learning number section, users were excited to count together with the 3D character while interacting with the book. Thus, it shows the users were fully engaged with learning process in joyful learning environment.

Based on user feedbacks, more activity and learning part are needed to improve the efficiency of the book as an interactive edutainment playbook.

Fig 7 Users interacting with the prototype during exhibition

5. Conclusion and Future Work

This paper presented the development of an interactive mobile augmented reality magical playbook for preschool children in learning numbers using an old folklore literature, The Thirsty Crow. This playbook consists of mobile AR application and interactive physical book interface design. Based on observational study on the prototype, this playbook turns reading old folklore story from traditional book into a new joyful, interactive and engaging learning experience. Based on encouraging response from observational study, it motivates the development of the AR playbook series for preschool education as future work.


References

[1] Chew FP, Ishak Z. Malaysia folk literature in early childhood education. In world Academy of Science, Engineering and Technology 66, 2010. pp. 557-563.

[2] Zhou ZY, Cheok AD, Tedjokusumo J, Omer GS. wIzQubesTM – A novel tangible interface for interactive storytelling in mixed reality. The international journal of virtual reality, 2008, 7(4):9-15.

[3] Azuma R. A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments 1997; 6( 4):355-388. [4] Dayang Rohaya, AR, Wannisa Matcha, SS, Mohd Yunus N. Design and Development of an Interactive Augmented Reality

Edutainment Storybook for Preschool, IERI Procedia, Vol 2, 2012, pp. 802-807, ISSN 2212-6678. [5] Billinghurst M, Kato H, Poupyrev I. The MagicBook: a transitional AR interface. Computers and Graphics 2001; 25 (5):745-753. [6] Kaufmann H, Schmalstieg D. Mathematics and Geometry in Education with Collaborative Augmented Reality. In ACM SIGGRAPH

2002 Conference, New York. 2002. pp. 37-41. [7] Medina E, Chen YC, Weghorst S. Understanding biochemistry with Augmented Reality. In C. Montgomerie & J. Seale (Eds.),

Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2007, pp. 4235-4239. [8] Zainuddin, NMM, Zaman HB, Azlina A. A participatory design in developing prototype an augmented reality book for deaf students,

in: Proceedings of the 2nd International Conference on Computer Research and Development 2010 IEEE, pp.400-404. [9] Mulloni A. A collaborative and location-aware application based on augmented reality for mobile devices, Master Thesis, Universita

degli Studi di Udine, 2007. [10] Moore AH, Fowler SB, Watson CE. Active Learning and Technology: Designing Change for Faculty, Students, and Institutions.

EDUCAUSE Review 2007; 42(5): 42-61. [11] Dufresne RJ, Gerace WJ, Leonard WJ, Mestre JP, Wenk L. Classtalk: A Classroom Communication System for Active Learning.

Journal of Computing in Higher Education 1996; 7: 3-47. [12] Beeland WD. Student Engagement, Visual Learning and Technology: Can Interactive Whiteboards Help? Learning, in: Proceedings of

the Annual Conference of the Association of Information Technology for Teaching Education. 2002. [13] Johnson A, Roussos M, Leigh J, Vasilakis C, Barnes C, Moher T, The NICE Project: Learning Together in a Virtual World, in:

Proceedings of the Virtual Reality Annual International Symposium 1998 IEEE, pp. 176-183. [14] Davies CHJ. Student engagement with simulations: a case study. Computers & Education. 2002; 39(3):271-282. [15] Monahan T, McArdle G, Bertolotto M. Virtual reality for collaborative e-learning. Computers & Education.2008; 50(4):1339-135. [16] Grasset R, Dünser A, Billinghurst M. Edutainment with a Mixed Reality book: a visual augmented illustrative children's book. In

Advance in computer entertainment technology 2008, Yokohama, Japan. ACM, pp. 292-295. [17] Grasset R, Dünser A, Billinghurst M. The design of a Mixed-Reality Book: is it still a real book?, in: Proceedings of the Int.

Symposium on Mixed and Augmented Reality IEEE ISMAR, 2008. Cambridge, UK. pp. 99-102. [18] Dünser A, Hornecker E. Lessons from an AR book study, in: Proceedings of the 1st international conference on Tangible and

embedded interaction. ACM, 2007. p. 179-182. [19] van Krevelen DWF, Poelman R. A Survey of Augmented Reality Technologies, Applications and Limitations, International Journal

of Virtual Reality 2010, 9(2):1-20. [20] Bimber O, Raskar R. Spatial Augmented Reality Merging Real and Virtual Worlds. A K Peters Wellesley Ltd., Massachusetts. USA,

2005. [21] Bucolo S, Billinghurst M, Sickinger D. Mobile maze: a comparison of camera based mobile game human interfaces, in: Proceedings of

the 7th international conference on Human computer interaction with mobile devices & services. ACM, 2005. p. 329-330. [22] Henrysson A, Billinghurst M, Ollila M. Face to face collaborative ar on mobile phones, in: Proceeding of the Fourth IEEE and ACM

International Symposium on Mixed and Augmented Reality, ISMAR 2005 IEEE, pp. 80-89. [23] Rohs M. Marker-Based Embodied Interaction for Handheld Augmented Reality Games. Journal of Virtual Reality and Broadcasting

2007; 4(5). [24] Schall G, Reitinger B, Mendez E, Junghanns S, Schmalstieg D. Handheld geospatial augmented reality using urban 3d models, in:

Proceedings of the Workshop on Mobile Spatial Interaction, ACM International Conference on Human Factors in Computing Systems (CHI 07). 2007.

[25] Huynh, Duy-Nguyen Ta, et al. Art of defense: a collaborative handheld augmented reality board game, in: Proceedings of the 2009 ACM SIGGRAPH symposium on video games. ACM, 2009. pp. 135-142.

Documents

An Interactive Mobile Augmented Reality Magical Playbook: … · 2017-01-18 · learning numbers using an old folklore literature, The Thirsty Crow, via mobile augmented reality application