Upload
saeid
View
212
Download
0
Embed Size (px)
Citation preview
CARMa: Content AugmentedReality Marker
Mohammed HossnyCentre for Intelligent SystemsResearch (CISR)Deakin University, [email protected]
Mustafa HossnyFaculty of Computers andInformation (FCI)Cairo University, [email protected]
Saeid NahavandiCentre for Intelligent SystemsResearch (CISR)Deakin University, [email protected]
Copyright is held by the authors. This research was fully sup-
ported by the Centre for Intelligent Systems Research (CISR)
at Deakin University.
AbstractThe current marker-based augmented reality (AR)rendering has demonstrated good results for onlineand special purpose applications such ascomputer-assisted tasks and virtual training. However,it fails to deliver a solution for off-line and genericapplications such as augmented books, newspapers,and scientific articles. These applications feature toomany markers that imposes a serious challenge on therecognition module. This paper proposes a noveldesign for augmented reality markers. The proposedmarker design employs multi-view orthographicprojection to derive dense depth maps and relies onsplats rendering for visualisation. The main objective isto interpret the marker rather than recognising it. Theproposed marker design stores six depth mapprojections of the 3D model along with their colouredtextures in the marker.
Author KeywordsMarker-based Augmented Reality; Content-basedMarker
IntroductionAugmented Reality (AR) is a relatively newinterdisciplinary field that applies image processing,pattern recognition and computer graphics
Extended Abstracts of the IEEE International Symposium on Mixed and Augmented Reality 2013Science and Technology Proceedings1 - 4 October 2013, Adelaide, SA, Australia978-1-4799-2869-9/13/$31.00 ©2013 IEEE
technologies in order to perceive the reality and amplifyit with more information. The idea started in firstshooter 3D computer games where the player utilisesmany on screen gadgets highlighting targets, path andthe terrain map [1]. This technology has then beenpromoted in military applications for situationalawareness combats [2]. Researchers have classifiedAR applications as marker-less and marker-basedapplications. While military are main motivator ofmarker-less applications; marker-based applicationshave taken further steps ahead due to the existence ofits tools of marker recognition, pose estimation, and 3Drendering. Despite of its simplicity, this frameworkimposes recognition errors on the AR system. It limitsthe number of markers to be memorised by the systemas well as levels of detail.
This paper proposes a new design scheme for ARfiducial markers. The new marker design aims to storethe 3D information in the printed tag rather thangenerating unique tags to select unique 3D modelsfrom an xml file and upgrades the recognition phase inmarker-based augmented reality to an interpretation. Asample of the proposed marker of the famousStanford’s bunny 3D model is presented in Fig. 1.
Related WorkThe main deriving module in marker-based renderingis the recognition module. In fact, the overallperformance of the system depends mainly on thequality of used cameras, feature extraction,training/recognition, training data sets/schemes, andthe design of fiducial markers. The very first AR markeris the barcode. A barcode is an opticalmachine-readable representation of data, which showscertain data on certain products. A 2D version of
barcodes that resembles cross word puzzles is alsoavailable and known as Semacode and ARTags [4, 3].barcodes that resembles cross woravailable and known as Semacode
(a) 3D Model (b) CARMa Marker
Figure 1: Stanford’s bunny transformed into CARMa marker.
Recently, barcodes and semacodes evolved intofiducial markers standardised the design of ARmarkers with a black on white boundary to facilitateobtaining the orientation matrix of the marker in therecognition system. Fiducial markers have to be inblack and white and must feature the black on whitebounding box. In order to increase the number of levelsof detail, Tateno et al. used of nested fiducial markersin [7]. Nested markers demonstrated a huge potentialin two main future trends. The, first trend is allowingaugmenting 3D objects progressively in a fashion thatreduces response time and allows very high resolution3D models for interested observers having close-upexaminations. The second trend is using thesub-markers to recognise and fetch primitive 3Dobjects and bind them together with computationalsolid graphics (CSG) boolean operators. The mainchallenge with this trend is the limitation of CSG to
represent a real life 3D tessellated mesh. In otherwords, as the 3D object becomes more complicatednumber of nested levels increase exponentially. In [5],Mooser et al. designed a tricodes. Tricodes aretrinary-based bar code using a grid of triangles [5]. Themotivation behind tricodes is to facilitate the creation,organisation, and identification of large, dynamicfiducial sets.
New fiducial markers such as Tricodes [5], ARTag [3],nested markers [7] facilitated recognition of asignificantly high number of fiducial markers as well asmaintaining check sum validation. Yet, they still lacks intwo parts; 1) relying mainly on a consistently onlinedatabase; and 2) having object ID digitally encodedwhich renders the marker useless during occlusion.
CARMa: Content AR MarkerThe main motivation behind the development ofContent AR Marker (CARMa) is to overcome theocclusion problem in fiducial markers and replace therecognition/fetch cycle with an actual interpretation ofthe observed marker. The design of the proposedmarker is inspired from multi-view orthographicprojection and relies on splats rendering.
Design
As demonstrated in Fig. 2-a, the marker is divided into12 equally sized squares. Six squares encode thedepth map of the point cloud projected on the six cubefaces. In order to eliminate lighting challenges, depthimages are presented using jet colour map scaled from0 to 100 for front projections and -100 to 0 for backprojections. The other six squares are used to storetexture images for each view. The coloured depth mapmake it easier to provide significant variation between
different depth values at different projections. Atrendering, point cloud generated from each depth mapare then registered together to eliminate anyredundancy and fill any occlusions. The renderingmodule for the proposed marker relies on QSplats [6].It requires a dense textured point cloud and relaxes themeshing constraints. Splats does sacrifice meshquality in order to provide a very high frame rate anddynamic levels of details.
Recognition and Rendering
Marker rendering relies on the same procedure ofother AR markers. It starts by locating the the markerin the acquired video stream. The next step is locatingthe negative depth maps (back, front and leftprojections) for pose estimation. The final step isrendering and augmenting the 3D object at the rightlocation and the right pose of the marker in theacquired stream. Yet the rendering of the proposedmarker does not require looking into a database to findthe proper 3D object. Both depth and textured data areembedded into the marker. In order to preserve a highrendering frame rate, the rendering module abandonsmeshing information and uses QSplats rendering [6]. Asample of intermediate point cloud and the renderedsplats is presented in Fig. 2.
Marker Generation
Transforming a CARMa marker from a 3D model isquite simple. First, 3D model is tested for havingenough point cloud. If the point cloud is not denseenough a subdivision is then applied to maintain adense enough mesh (minimum of 50K vertices). Thesecond step is rendering the mesh at top, bottom, left,right, front and back views. The six generated imageswill serve as texture for point cloud colouring. The next
Left Texture
Right Texture
Front Texture
Top Texture
Back Texture
Bottom Texture
Back Depth Map
Bottom Depth Map
Front Depth Map
TopDepth Map
Right Depth Map
Left Depth Map
(a) Marker Template
0 50 100 1500 20 40 60 80 100 120
−40
−30
−20
−10
0
10
20
30
40
(b) Point Cloud (c) QSplats Rendering
Figure 2: a) CARMa Template, b) Intermediate point cloud obtained form the CARMa marker, and c) Stanford’s bunny reduced tothe density obtained from a CARMa marker in Fig. 1 and rendered using splats.
step is extracting the point cloud form the 3D modeland project its depth on top, bottom, left, right, frontand back views.
ApplicationsThe proposed marker facilitates a wide range ofapplications. The growing interest and anticipation ofthe release of Google Glass, as the first off-the-shelfintegrated AR system, will boost the printed digitalcontent and revive the newspaper industry. Theredundant non-identifying information encoded in theproposed marker allows AR systems to have stablerecognition and visualisation of 3D illustrations in smartbooks and newspapers. The simple jet colour codingallows recognition and visualisation of the rough shapefrom a distance and allows progressive refining of thedetected model as the user get closer. The efficiencyof the splats rendering allows rendering 60 frames per
second for a full stereo visualisation at a very lowprocessing cost.
ConclusionThis paper demonstrated a proof of concept of a novelAR marker design. The proposed marker facilitatesoffline recognition and visualisation and allows therepresentation of theoretically infinite set of 3D models.CARMa works well with available head mounteddisplays. In order to unleash the full potential ofCARMa, better camera sensors will be required todifferentiate between different colour levels from adistance. Future advancement to CARMa is beingimplemented to employ a generic multi-vieworthographic projection to accommodate complexshapes for artistic and scientific visualisation.
AcknowledgementsThis research was fully supported by the Centre forIntelligent Systems Research (CISR) at DeakinUniversity.
References[1] Azuma, R. A survey of augmented reality. In
Presence-Teleoperators and Virtual Environments,
International Conference on, vol. 6 (1997),355–385.
[2] Azuma, R., Baillot, Y., Behringer, R., Feiner, S.,Julier, S., and MacIntyre, B. Recent advances inaugmented reality. IEEE Computer Graphics and
Applications 21, 6 (2001), 34–47.[3] Celozzi, C., Paravati, G., Sanna, A., and Lamberti,
F. A 6-dof artag-based tracking system. Consumer
Electronics, IEEE Transactions on 56, 1 (2010),
203–210.[4] Fiala, M. Designing highly reliable fiducial markers.
Pattern Analysis and Machine Intelligence, IEEE
Transaction on 32, 7 (2010), 1317–1324.[5] Mooser, J., You, S., and Neumann, U. Tricodes: A
barcode-like fiducial design for augmented realitymedia. In Multimedia and Expo, IEEE International
Conference on (2006), 1301–1304.[6] Rusinkiewicz, S., and Levoy, M. Qsplat: A
multiresolution point rendering system for largemeshes (2000). 343–352.
[7] Tateno, K., Kitahara, I., and Ohta, Y. A nestedmarker for augmented reality. In Virtual Reality,
IEEE International Conference on, Sherman, Wand Lin, M and Steed, A, Ed. (2007), 259–262.