Research on Augmented Reality Interactive Games

Rong Fan, Ying Liu Sino-Korean School of Multi-media

Shanghai University of Engineering ScienceShanghai, China

fanrongsh@126.com Abstract – Augmented Reality (Augmented Reality, referred to as AR for short) is one of the research hotspots of many renowned foreign universities and research institutions in recent years. AR technology has a wide range of applications not only in similar application fields to VR technology, such as research and development on sophisticated weaponry and aircraft, visualization of data model, virtual training, entertainment and arts and other fields, but also in medical research and anatomy training, manufacturing and maintenance of precision equipment, navigation of military aircraft, engineering design and remote control on robot and other areas. Besides, because of its characteristic enhancing the display output of real environment, it has more pronounced advantages than VR technology in those latter areas. Interactive games based on AR technology are games of developmental direction in the future. It can unite virtual and reality, which can bring sense of deeper immersion to the players, thus greatly increasing the interest and attraction of game. Such games used in the field of education, compared to the traditional gaming species, will greatly enhance the player's memory, which can better achieve the entertaining and educational effect.

Key words– Augmented Reality, Face recognition, Interactive games, 2010 World Expo, New Energy Pavilion, Entertaining and educational

I. INTRODUCTION

Augmented Reality (Augmented Reality, referred to as AR for short) is one of the research hotspots of many renowned foreign universities and research institutions in recent years. AR technology has a wide range of applications not only in similar applications fields to VR technology, such as sophisticated weaponry, research and development on aircraft, visualization of data model, virtual training, entertainment and arts and other fields, but also in medical research and anatomy training, manufacturing and maintenance of precision equipment, navigation of military aircraft, engineering design and remote control on robot and other areas. Besides, because of its characteristic enhancing the display output of real environment, it has more pronounced advantages than VR technology in those latter areas. Interactive games based on AR technology are games of developmental direction in the future. It can unite virtual and reality, which can bring sense of deeper immersion to the players, thus greatly increasing the interest and attraction of game. Such games used in the field of education, compared to the traditional gaming species, will greatly enhance the player's memory, which can better achieve the entertaining and educational effect.

II. CURRENT DEVELOPMENTAL STATUS AND APPLICATION FIELDS OF AUGMENTED REALITY AT HOME AND ABROAD

A. Brief introduction of Augmented Reality technology Augmented Reality (Augmented Reality) is also called Mixed Reality. It has virtual information applied in the real world through computer technology, thus making the real environment and virtual objects overlay in real time in the same screen or space then coexist. Augmented reality provides information different from which human can sense in general condition. It not only demonstrates the information of real-world but also displays virtual information at the same time, thus having two types of information complementing and superimposing each other. In visualized augmented reality, the user uses the helmet display to manage the multiple synthesis of the real world and computer graphics, thus making us can see the real world around it. Augmented Reality uses computer graphics technology and visualization technology to generate the virtual objects which don’t exist in real environment and accurately "place" the virtual objects in the real environment through sensing technology. It has virtual objects integrated with the real environment through the display device and presents to the user a new environment with true sensory effects. Therefore, the Augmented Reality system has new features of combination of virtual and reality, real-time interactive and three-dimensional registration B. Current developmental status of Augmented Reality at home and abroad Augmented Reality technology is a research hotspot among computer vision, computer graphics and other technologies in recent years. It brings users real feelings by adding 3D pictures to real scenes. Currently, the most widely used Augmented Reality technology is Augmented Reality technology “based on manual marked points”. With a simple arrangement of manual printed patterns in the scene, such as a square or several triangles with internal encoding, etc., you can add a variety of virtual 3D objects in the scene. Boeing’s Computer Services Research and Technology group apply Augmented Reality technology in the connection of power cable and assembly of connector in aircraft manufacturing; Sony's latest PS3 game "Eye of Judgment" is also developed by using Augmented Reality technology “based on manual marked points”. The game realistically has rendered 3D characters such as monsters in the real environment around the players, thus being widely praised. C. Application fields of Augmented Reality With the development of computer technology, Augmented Reality technology is becoming one of the main

978-1-4577-0856-5/11/$26.00 ©2011 IEEE

developmental directions of the next generation of human-machine interface technology. Augmented Reality technology enables users to easily access to the microscopic world people should have been unable to enter and the special environment people cannot directly enter. And it builds an ideal interactive environment for people to study and work, such as Augmented Reality Systems for observation of molecular three-dimensional structure, exploration of nature of the gene chip and so on. Augmented Reality technology combined with computer graphics, human-machine interactive technology and network multimedia technology has been widely used in medical, military, industrial, entertainment and other fields. In the medical field: doctors can use Augmented Reality technology for easy pinpoint of surgical site. In the military field, the troops can use Augmented Reality technology for the recognition of orientation, thus getting access to the geographic data of current location and other important military data. In the field of historic rehabilitation and digital cultural heritage preservation, the information of cultural heritage has been provided to visitors in the way of Augmented Reality. The user can not only see the introductory text of monuments but also the virtual reconstruction of missing parts of the site. In the field of network video communications: the system uses Augmented Reality and face tracking technology to overlay hats, glasses and other virtual objects in real time in front of callers while in a call, thus greatly improving the fun of network video dialogue. In the television broadcast field, the auxiliary information can be added to the screen in real time in the broadcast of sports through the Augmented Reality technology, thus making viewers can get more information. In the field of entertainment and gaming, augmented reality games allow players from different locations in the world come together into a real natural scene and then play and fight online in the form of a virtual avatar. In the field of municipal construction planning, the use of Augmented Reality technology can bring direct access to planning effects by overlaying the planning effects in the real scenario. With the development of human society technology, Augmented Reality technology will definitely obtain more and more extensive applications.

III. EXAMPLE PRODUCTION OF PROJECT OF AUGMENTED REALITY INTERACTIVE GAMES IN 2010 SHANGHAI EXPO NEW

ENERGY PAVILION

A. Introduction of 2010 Shanghai Expo New Energy Pavilion Expo New Energy Pavilion, based on the theme “Low-carbon City-Expo is Underway, Shanghai is in Actions”, is a platform for collectively displaying new energy application in China, achievement and result in new energy practice obtained by Shanghai as well as efforts made by the Expo Site in promotion and application of new energy technology. The whole section adopts paper-free displaying method, and uses multimedia instead of traditional displaying method to carry more information. A number of interactive experiencing systems have been set in the displaying items. There are four

theme sites: Shanghai in Actions, Expo Underway, Expo Site in Changing and Future Calling. The first thing that the visitors can see after entering New Energy Pavilion is a 3.5 meters high and 7 meters wide curved screen replaying a large number of practical application cases of low-carbon technologies in the modern city. The image projected by two sets of high-lumen projector is formed into a large one with seamless integration after processed by the image processing equipment, thus bringing the audience shocking visual effects. In front of the curved screen there is a seamless sand table to vote at the length of 5 meters and the width of 2.5 m. The energy consumption and emissions of carbon dioxide and other data of the selected pavilions will be displayed in real time if the tourists "click on" the different icons of Expo Pavilions on the sand table with the laser pointer. Two sets of independent interactive projection systems are designed in the center of the walls of the pavilion. When the audiences pass through the front of the screen, the screen will appear the images of the viewers. And there will be a lot of sunflowers around the images to convey the hopeful vision of the greater use of solar energy and other low Carbon energy to the audiences. The audience will also experience a lot of fun while being educated at the same time. There is a 1.5 m high pyramid specially designed in the center of the pavilion. In the center of the 4 pieces of glass in the pyramid there replays the pavilion introduction of low-carbon technologies applied in the Expo. On the top of the pyramid there are four digital displaying screens echoing the contents of replaying on the glass of tower. It is worth mentioning that the picture on the glass of tower is reflected back to four different surfaces of the tower through a piece of glass after processed by four 6500-lumen projectors and image processing equipment. B. Needs of project of augmented reality interactive games in New Energy Pavilion The construction al party of New Energy Pavilion hopes to set an area in the pavilion, thus achieving the purpose of people’s establishing the concept of "energy saving and emission reduction" in the way of an easy digital entertainment. New energy and development of new technology are closely related, so this kind of digital entertainment way must make visitors feel novel and never seen before, to achieve the entertaining and educational effect. Based on the above requirements, the project implementation party decided to use the form of Augmented Reality interactive games to achieve this purpose. The game is an integrated application and development of computer Augmented Reality technology and computer facial recognition technology, which is used at home for the first time and also the only one in the 2010 Shanghai World Expo. C. The key technology used in New Energy Pavilion augmented reality interactive games 1)Face recognition technology Face recognition is one of the most challenging subjects in the fields of machine vision and pattern recognition, which is also with more wide application meaning. Face recognition

technology is a very active research field; it covers digital image processing, pattern recognition, computer vision, neural networks, psychology, physiology, mathematics and many other contents of subjects. Now, although research in this area have made some promising results, but in the practical application FRT is still facing a very serious problem because the distribution of human facial features are very similar, and the face itself is a flexible object. The ever-changing of facial expressions, gestures or hair and make-up all has brought the correct identification considerable trouble. How to correctly identify a large number of people and meet the real-time requirements is an urgent problem needed to address. Based on face detection, the face detection of key features attempts to detect the location of main facial features of people and shape information of major organs such as eyes and mouth. Eyes are the most important feature of face and their precise positioning is the key to identify. The eyes positioning technology based on regional growth takes full use of the feature that the eyes are in the gray valley of upper left and upper right area in the center of the face to quickly and accurately fix the position of the center of pupils of eyes on the basis of face detection technology. The algorithm uses a search strategy based on regional growth, which estimates the initial location of the nose and then define two rectangles of initial search respectively growing from the approximate location of left and right eyes in the general framework of human face given by face detection algorithm. The algorithm uses search of the rectangle to find the edge of eyes on the basis of the feature that human eye is significantly lower than face in terms of gray scale and finally locate the center of the pupil. Experiments show that the algorithm have a strong ability to adapt for changes of face size, posture, and lighting, but in the case of heavy eye shadow, there will not be allowed accurate positioning. Wearing black-rimmed glasses will also affect the positioning results of the algorithm. 2) Augmented Reality technology Perhaps one day when we put on special 3D glasses while walking in the Old Summer Palace, we can enjoy the magnificent imperial garden style of the Old Summer Palace at that time. This is not a movie set, but a wonderful experience that "Augmented Reality" can give us. The research project led by Professor Wang Yongtian of Beijing Polytechnic University is committed to the completion of virtual reconstruction of the Old Summer Palace by Augmented Reality technology. While in the 2010 Shanghai Expo New Energy Pavilion the Augmented Reality technology is also tried to increase the interest and attraction of interactive games. When people enter the area within the identification area of camera, the computer calculates the height of the head, and then paly the pre-stored animation movie above the head of the real visitors on the screen, such as the growth of sunflower, or random graphical energy-saving formula. The animation movie moves slowly as the head move around, up and down. The visitors will see above their real image on the screen there are virtual animations where they can interact with themselves, thus resulting in strong interest.

Fig. 1 New Energy Pavilion augmented reality interactive games

D. Analysis on social and economic benefits of New Energy Pavilion augmented reality interactive games This project is the latest developed modern multi-media displaying technology and thus currently needs more innovation of technology details. The application of this project will bring the users more real experiences and interactive effects. With the technology matures completely, its application prospects will be very broad. Meanwhile, due to its relatively high technical level, its social and economic benefits will be very significant.

CONCLUSION

With the development of augmented reality technology and face recognition technology gradually matures, the multimedia databases using virtual reality and based on content retrieval and other advanced technologies to build interactive display has become a trend. In the future, these technologies will be widely applied in research and development on sophisticated weaponry and aircraft, visualization of data model, virtual training, entertainment and arts and other fields and have good prospects for development

REFERENCES [1] http://my.oschina.net/xiaoke/blog/3982 [2] http://pavilion.expo.cn/z0012/ssize/cn/index.html [3] http://news.davinfo.com/detail/www/17132_1.shtml