5
CARMa: Content Augmented Reality Marker Mohammed Hossny Centre for Intelligent Systems Research (CISR) Deakin University, Australia [email protected] Mustafa Hossny Faculty of Computers and Information (FCI) Cairo University, Egypt [email protected] Saeid Nahavandi Centre for Intelligent Systems Research (CISR) Deakin University, Australia [email protected] Abstract The current marker-based augmented reality (AR) rendering has demonstrated good results for online and special purpose applications such as computer-assisted tasks and virtual training. However, it fails to deliver a solution for off-line and generic applications such as augmented books, newspapers, and scientific articles. These applications feature too many markers that imposes a serious challenge on the recognition module. This paper proposes a novel design for augmented reality markers. The proposed marker design employs multi-view orthographic projection to derive dense depth maps and relies on splats rendering for visualisation. The main objective is to interpret the marker rather than recognising it. The proposed marker design stores six depth map projections of the 3D model along with their coloured textures in the marker. Author Keywords Marker-based Augmented Reality; Content-based Marker Introduction Augmented Reality (AR) is a relatively new interdisciplinary field that applies image processing, pattern recognition and computer graphics Extended Abstracts of the IEEE International Symposium on Mixed and Augmented Reality 2013 Science and Technology Proceedings 1 - 4 October 2013, Adelaide, SA, Australia 978-1-4799-2869-9/13/$31.00 ©2013 IEEE

[IEEE 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) - Adelaide, Australia (2013.10.1-2013.10.4)] 2013 IEEE International Symposium on Mixed and Augmented

  • Upload
    saeid

  • View
    212

  • Download
    0

Embed Size (px)

Citation preview

Page 1: [IEEE 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) - Adelaide, Australia (2013.10.1-2013.10.4)] 2013 IEEE International Symposium on Mixed and Augmented

CARMa: Content AugmentedReality Marker

Mohammed HossnyCentre for Intelligent SystemsResearch (CISR)Deakin University, [email protected]

Mustafa HossnyFaculty of Computers andInformation (FCI)Cairo University, [email protected]

Saeid NahavandiCentre for Intelligent SystemsResearch (CISR)Deakin University, [email protected]

Copyright is held by the authors. This research was fully sup-

ported by the Centre for Intelligent Systems Research (CISR)

at Deakin University.

AbstractThe current marker-based augmented reality (AR)rendering has demonstrated good results for onlineand special purpose applications such ascomputer-assisted tasks and virtual training. However,it fails to deliver a solution for off-line and genericapplications such as augmented books, newspapers,and scientific articles. These applications feature toomany markers that imposes a serious challenge on therecognition module. This paper proposes a noveldesign for augmented reality markers. The proposedmarker design employs multi-view orthographicprojection to derive dense depth maps and relies onsplats rendering for visualisation. The main objective isto interpret the marker rather than recognising it. Theproposed marker design stores six depth mapprojections of the 3D model along with their colouredtextures in the marker.

Author KeywordsMarker-based Augmented Reality; Content-basedMarker

IntroductionAugmented Reality (AR) is a relatively newinterdisciplinary field that applies image processing,pattern recognition and computer graphics

Extended Abstracts of the IEEE International Symposium on Mixed and Augmented Reality 2013Science and Technology Proceedings1 - 4 October 2013, Adelaide, SA, Australia978-1-4799-2869-9/13/$31.00 ©2013 IEEE

Page 2: [IEEE 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) - Adelaide, Australia (2013.10.1-2013.10.4)] 2013 IEEE International Symposium on Mixed and Augmented

technologies in order to perceive the reality and amplifyit with more information. The idea started in firstshooter 3D computer games where the player utilisesmany on screen gadgets highlighting targets, path andthe terrain map [1]. This technology has then beenpromoted in military applications for situationalawareness combats [2]. Researchers have classifiedAR applications as marker-less and marker-basedapplications. While military are main motivator ofmarker-less applications; marker-based applicationshave taken further steps ahead due to the existence ofits tools of marker recognition, pose estimation, and 3Drendering. Despite of its simplicity, this frameworkimposes recognition errors on the AR system. It limitsthe number of markers to be memorised by the systemas well as levels of detail.

This paper proposes a new design scheme for ARfiducial markers. The new marker design aims to storethe 3D information in the printed tag rather thangenerating unique tags to select unique 3D modelsfrom an xml file and upgrades the recognition phase inmarker-based augmented reality to an interpretation. Asample of the proposed marker of the famousStanford’s bunny 3D model is presented in Fig. 1.

Related WorkThe main deriving module in marker-based renderingis the recognition module. In fact, the overallperformance of the system depends mainly on thequality of used cameras, feature extraction,training/recognition, training data sets/schemes, andthe design of fiducial markers. The very first AR markeris the barcode. A barcode is an opticalmachine-readable representation of data, which showscertain data on certain products. A 2D version of

barcodes that resembles cross word puzzles is alsoavailable and known as Semacode and ARTags [4, 3].barcodes that resembles cross woravailable and known as Semacode

(a) 3D Model (b) CARMa Marker

Figure 1: Stanford’s bunny transformed into CARMa marker.

Recently, barcodes and semacodes evolved intofiducial markers standardised the design of ARmarkers with a black on white boundary to facilitateobtaining the orientation matrix of the marker in therecognition system. Fiducial markers have to be inblack and white and must feature the black on whitebounding box. In order to increase the number of levelsof detail, Tateno et al. used of nested fiducial markersin [7]. Nested markers demonstrated a huge potentialin two main future trends. The, first trend is allowingaugmenting 3D objects progressively in a fashion thatreduces response time and allows very high resolution3D models for interested observers having close-upexaminations. The second trend is using thesub-markers to recognise and fetch primitive 3Dobjects and bind them together with computationalsolid graphics (CSG) boolean operators. The mainchallenge with this trend is the limitation of CSG to

Page 3: [IEEE 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) - Adelaide, Australia (2013.10.1-2013.10.4)] 2013 IEEE International Symposium on Mixed and Augmented

represent a real life 3D tessellated mesh. In otherwords, as the 3D object becomes more complicatednumber of nested levels increase exponentially. In [5],Mooser et al. designed a tricodes. Tricodes aretrinary-based bar code using a grid of triangles [5]. Themotivation behind tricodes is to facilitate the creation,organisation, and identification of large, dynamicfiducial sets.

New fiducial markers such as Tricodes [5], ARTag [3],nested markers [7] facilitated recognition of asignificantly high number of fiducial markers as well asmaintaining check sum validation. Yet, they still lacks intwo parts; 1) relying mainly on a consistently onlinedatabase; and 2) having object ID digitally encodedwhich renders the marker useless during occlusion.

CARMa: Content AR MarkerThe main motivation behind the development ofContent AR Marker (CARMa) is to overcome theocclusion problem in fiducial markers and replace therecognition/fetch cycle with an actual interpretation ofthe observed marker. The design of the proposedmarker is inspired from multi-view orthographicprojection and relies on splats rendering.

Design

As demonstrated in Fig. 2-a, the marker is divided into12 equally sized squares. Six squares encode thedepth map of the point cloud projected on the six cubefaces. In order to eliminate lighting challenges, depthimages are presented using jet colour map scaled from0 to 100 for front projections and -100 to 0 for backprojections. The other six squares are used to storetexture images for each view. The coloured depth mapmake it easier to provide significant variation between

different depth values at different projections. Atrendering, point cloud generated from each depth mapare then registered together to eliminate anyredundancy and fill any occlusions. The renderingmodule for the proposed marker relies on QSplats [6].It requires a dense textured point cloud and relaxes themeshing constraints. Splats does sacrifice meshquality in order to provide a very high frame rate anddynamic levels of details.

Recognition and Rendering

Marker rendering relies on the same procedure ofother AR markers. It starts by locating the the markerin the acquired video stream. The next step is locatingthe negative depth maps (back, front and leftprojections) for pose estimation. The final step isrendering and augmenting the 3D object at the rightlocation and the right pose of the marker in theacquired stream. Yet the rendering of the proposedmarker does not require looking into a database to findthe proper 3D object. Both depth and textured data areembedded into the marker. In order to preserve a highrendering frame rate, the rendering module abandonsmeshing information and uses QSplats rendering [6]. Asample of intermediate point cloud and the renderedsplats is presented in Fig. 2.

Marker Generation

Transforming a CARMa marker from a 3D model isquite simple. First, 3D model is tested for havingenough point cloud. If the point cloud is not denseenough a subdivision is then applied to maintain adense enough mesh (minimum of 50K vertices). Thesecond step is rendering the mesh at top, bottom, left,right, front and back views. The six generated imageswill serve as texture for point cloud colouring. The next

Page 4: [IEEE 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) - Adelaide, Australia (2013.10.1-2013.10.4)] 2013 IEEE International Symposium on Mixed and Augmented

Left Texture

Right Texture

Front Texture

Top Texture

Back Texture

Bottom Texture

Back Depth Map

Bottom Depth Map

Front Depth Map

TopDepth Map

Right Depth Map

Left Depth Map

(a) Marker Template

0 50 100 1500 20 40 60 80 100 120

−40

−30

−20

−10

0

10

20

30

40

(b) Point Cloud (c) QSplats Rendering

Figure 2: a) CARMa Template, b) Intermediate point cloud obtained form the CARMa marker, and c) Stanford’s bunny reduced tothe density obtained from a CARMa marker in Fig. 1 and rendered using splats.

step is extracting the point cloud form the 3D modeland project its depth on top, bottom, left, right, frontand back views.

ApplicationsThe proposed marker facilitates a wide range ofapplications. The growing interest and anticipation ofthe release of Google Glass, as the first off-the-shelfintegrated AR system, will boost the printed digitalcontent and revive the newspaper industry. Theredundant non-identifying information encoded in theproposed marker allows AR systems to have stablerecognition and visualisation of 3D illustrations in smartbooks and newspapers. The simple jet colour codingallows recognition and visualisation of the rough shapefrom a distance and allows progressive refining of thedetected model as the user get closer. The efficiencyof the splats rendering allows rendering 60 frames per

second for a full stereo visualisation at a very lowprocessing cost.

ConclusionThis paper demonstrated a proof of concept of a novelAR marker design. The proposed marker facilitatesoffline recognition and visualisation and allows therepresentation of theoretically infinite set of 3D models.CARMa works well with available head mounteddisplays. In order to unleash the full potential ofCARMa, better camera sensors will be required todifferentiate between different colour levels from adistance. Future advancement to CARMa is beingimplemented to employ a generic multi-vieworthographic projection to accommodate complexshapes for artistic and scientific visualisation.

Page 5: [IEEE 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) - Adelaide, Australia (2013.10.1-2013.10.4)] 2013 IEEE International Symposium on Mixed and Augmented

AcknowledgementsThis research was fully supported by the Centre forIntelligent Systems Research (CISR) at DeakinUniversity.

References[1] Azuma, R. A survey of augmented reality. In

Presence-Teleoperators and Virtual Environments,

International Conference on, vol. 6 (1997),355–385.

[2] Azuma, R., Baillot, Y., Behringer, R., Feiner, S.,Julier, S., and MacIntyre, B. Recent advances inaugmented reality. IEEE Computer Graphics and

Applications 21, 6 (2001), 34–47.[3] Celozzi, C., Paravati, G., Sanna, A., and Lamberti,

F. A 6-dof artag-based tracking system. Consumer

Electronics, IEEE Transactions on 56, 1 (2010),

203–210.[4] Fiala, M. Designing highly reliable fiducial markers.

Pattern Analysis and Machine Intelligence, IEEE

Transaction on 32, 7 (2010), 1317–1324.[5] Mooser, J., You, S., and Neumann, U. Tricodes: A

barcode-like fiducial design for augmented realitymedia. In Multimedia and Expo, IEEE International

Conference on (2006), 1301–1304.[6] Rusinkiewicz, S., and Levoy, M. Qsplat: A

multiresolution point rendering system for largemeshes (2000). 343–352.

[7] Tateno, K., Kitahara, I., and Ohta, Y. A nestedmarker for augmented reality. In Virtual Reality,

IEEE International Conference on, Sherman, Wand Lin, M and Steed, A, Ed. (2007), 259–262.