Availability of Mobile Augmented Reality System for Urban Landscape Simulation
Tomohiro Fukuda, Tian Zhang, and Nobuyoshi Yabuki
Division of Sustainable Energy and Environmental Engineering,
Graduate School of Engineering, Osaka University, Japan
Contents
1. Introduction
2. Developed Mobile AR
3. Comparative verification of landscape simulation methods 1. Experimental Outline
2. Differences between Cloud-VR and mobile AR in Evaluation
3. Results and Discussion
4. Conclusion
2
1. Introduction
2. Developed Mobile AR
3. Comparative verification of landscape simulation methods 1. Experimental Outline
2. Differences between Cloud-VR and mobile AR in Evaluation
3. Results and Discussion
4. Conclusion
3
Contents
1.1 Motivation -1
In recent years, the need for landscape simulation has been growing. A review meeting of future landscape is carried out on a planned construction site in addition to being carried out in a room.
It is difficult for stakeholders to imagine concretely such an image that is three-dimensional and does not exist. A landscape visualization method using Computer Graphics (CG) and Virtual Reality (VR) has been developed.
However, this method requires much time and expense to make a 3D model such as the present terrain and artificial material in addition to the subject of the landscape assessment. Moreover, since consistency with real space is not achieved when using VR on a planned construction site, it has the problem that a reviewer cannot get an immersive experience.
4 A landscape study on site VR capture
1. Introduction
1.1 Motivation -2
In this research, the authors focus Augmented Reality (AR) which can superimpose an actual landscape acquired with a video camera and 3DCG. When AR is used, a landscape assessment object will be included in the present surroundings. Thereby, a drastic reduction of the time and expense involved in carrying out 3DCG modeling of the present surroundings can be expected.
A smartphone is widely available on the market level.
5
Sekai Camera Web http://sekaicamera.com/
Smartphone Market in Japan
1. Introduction
モバイル型景観ARの進化
6 ©2012 Tomohiro Fukuda, Osaka-U
2006
Image Sketch (2005)
1.2 Previous Study
In AR, realization of geometric consistency with a video image of an actual landscape and CG is an important feature
1. Use of physical sensors such as GPS (Global Positioning System) and gyroscope. To realize highly precise geometric consistency, special hardware which is expensive is required.
1. Introduction
1.2 Previous Study
7
Yabuki, N., et al.: 2011, An invisible height evaluation system for building height regulation to preserve good landscapes using augmented reality, Automation in Construction, Volume 20, Issue 3, 228-235. artificial marker
2. Use of an artificial marker. Since an artificial marker needs to be always visible by the AR camera, the movable span of a user is limited. Moreover, to realize high precision, it is necessary to use a large artificial marker.
1. Introduction
1.3 Aim
The authors have developed and verified SOAR (Sensor Oriented Mobile AR) system which realizes geometric consistency using GPS, a gyroscope and a video camera which are mounted in a smartphone [1]. The authors have also developed and verified GOAR (GIS Oriented Mobile AR) system which uses GIS to obtain position data instead of GPS [2]. A low cost AR system with high flexibility is realizable.
In this research, the availability of landscape simulation method of a mobile AR is considered, comparing with a photo montage and VR which are existing methods.
8
1. Fukuda, et al.: SOAR: Sensor oriented Mobile Augmented Reality for Urban Landscape Assessment, Proceedings of the 17th International Conference on Computer Aided Architectural Design Research in Asia (CAADRIA2012), pp. 387-396 (2012)
2. Fukuda, et al.: GOAR: GIS oriented Mobile Augmented Reality for Urban Landscape Assessment, 4th International Conference on Communications, Mobility, and Computing (CMC 2012), pp. 183-186 (2012)
1. Introduction
1. Introduction
2. Developed Mobile AR
3. Comparative verification of landscape simulation methods 1. Experimental Outline
2. Differences between Cloud-VR and mobile AR in Evaluation
3. Results and Discussion
4. Conclusion
9
Contents
2.1 Developed Mobile AR System
Standard Spec Smartphone: GALAPAGOS 003SH (Softbank Mobile Corp.)
Development Language: OpenGL-ES(Ver.2.0),Java(Ver.1.6)
Development Environment: Eclipse Galileo(Ver.3.5)
Location Estimation Technology: GIS includes Google Maps API and Digital Elevation Model (DEM) which is 10 m mesh size (GOAR)
10
OS Android™ 2.2
CPU Qualcomm®MSM8255 Snapdragon® 1GHz
Memory ROM:1GB RAM:512MB
Weight ≒140g Size ≒W62×H121×D12mm
Display Size 3.8 inch
Resolution 480×800 pixel
Spec of 003SH
003SH
Video Camera
2. Developed Mobile AR
2.2 System Flow -1
Distortion
Calibration of the video camera using Android NDK-OpenCV
While the CG model realizes ideal rendering by the perspective drawing method, rendering of a video camera produces distortion.
Calibration
11
Definition of landscape assessment 3DCG model
Selection of 3DCG model
Calibration of a video camera
Activation of AR system
Starting of Google Maps
Position information acquisition
Activation of gyroscope
Angle information acquisition
Definition of position and angle information on CG virtual camera
Superposition to live video image and 3DCG model
Display of AR image
Activation of video camera
Capture of live video image
Save of AR image
Input of DEM
2. Developed Mobile AR
2.2 System Flow -2
3DCG model allocation file
Geometry, Texture, Unit
3DCG model name, File name, Position data (longitude, latitude, orthometric height), Degree of rotation angle, and Zone number of the rectangular plane
Number of the 3DCG model allocation information file, Each name
3DCG Model
3DCG model arrangement information file
12
Definition of landscape assessment 3DCG model
Selection of 3DCG model
Calibration of a video camera
Activation of AR system
Starting of Google Maps
Position information acquisition
Activation of gyroscope
Angle information acquisition
Definition of position and angle information on CG virtual camera
Superposition to live video image and 3DCG model
Display of AR image
Activation of video camera
Capture of live video image
Save of AR image
Input of DEM
2. Developed Mobile AR
2.2 System Flow -3
13
GUI of the Developed System
Definition of landscape assessment 3DCG model
Selection of 3DCG model
Calibration of a video camera
Activation of AR system
Starting of Google Maps
Position information acquisition
Activation of gyroscope
Angle information acquisition
Definition of position and angle information on CG virtual camera
Superposition to live video image and 3DCG model
Display of AR image
Activation of video camera
Capture of live video image
Save of AR image
Input of DEM
2. Developed Mobile AR
2.2 System Flow -4
14
Coordinate System of Developed AR system
yaw
roll pitch
Definition of landscape assessment 3DCG model
Selection of 3DCG model
Calibration of a video camera
Activation of AR system
Starting of Google Maps
Position information acquisition
Activation of gyroscope
Angle information acquisition
Definition of position and angle information on CG virtual camera
Superposition to live video image and 3DCG model
Display of AR image
Activation of video camera
Capture of live video image
Save of AR image
Input of DEM
2. Developed Mobile AR
2.2 System Flow -5
15
Definition of landscape assessment 3DCG model
Selection of 3DCG model
Calibration of a video camera
Activation of AR system
Starting of Google Maps
Position information acquisition
Activation of gyroscope
Angle information acquisition
Definition of position and angle information on CG virtual camera
Superposition to live video image and 3DCG model
Display of AR image
Activation of video camera
Capture of live video image
Save of AR image
Input of DEM
1. The user tap the current location on Google Maps
2. The position data (longitude, latitude) on the current location is obtained
3.Altitude is created using position data (longitude, latitude) and DEM
2. Developed Mobile AR
2.2 System Flow -6
16
Definition of landscape assessment 3DCG model
Selection of 3DCG model
Calibration of a video camera
Activation of AR system
Starting of Google Maps
Position information acquisition
Activation of gyroscope
Angle information acquisition
Definition of position and angle information on CG virtual camera
Superposition to live video image and 3DCG model
Display of AR image
Activation of video camera
Capture of live video image
Save of AR image
Input of DEM
2. Developed Mobile AR
1. Introduction
2. Developed Mobile AR
3. Comparative verification of landscape simulation methods 1. Experimental Outline
2. Differences between Cloud-VR and mobile AR in Evaluation
3. Results and Discussion
4. Conclusion
17
Contents
3.1 Experimental Outline -1
The landscape simulation method of a mobile AR was verified through comparative experiments using photo montage and VR, which are existing methods. In order to use the same conditions as mobile AR, a cloud computing type VR (cloud-VR) which can run Android OS was applied.
Experimental Methodology 1. A 3D model of a virtual project was created. In this research, a high-rise
building (width: 40m, depth: 40m, height: 150m) and a wind power generator (height:
104m) were selected at varying distances (100m and 1200m) from a viewpoint. Moreover, the Tokyo Sky Tree (height: 634m) was selected at a distance of 1500m from the viewpoint.
2. The operation of photo montage, Cloud-VR, and mobile AR was explained to the subjects.
3. The subjects carried out the landscape study using photo montage for about two minutes, using Cloud-VR for about five minutes, and using a mobile AR for five minutes, in that order.
4. After the experiment, a questionnaire about the three landscape simulation methods was implemented. The themes of the questionnaire were the reproducibility of the landscape, the operability of the system, and cost.
3. Comparative verification of landscape simulation methods
19
3.1 Experimental Outline -2
Experimental photos and outputs
Photo montage Cloud-VR Mobile AR
3. Comparative verification of landscape simulation methods
20
21
3.1 Experimental Outline -4
The viewpoint was the West Park (longitude: 34.672501111,
latitude: 135.20194833, altitude: 4m) in Port Island, Kobe city.
Present state
Regulation of building heights
viewpoint
3. Comparative verification of landscape simulation methods
22
3.1 Experimental Outline -5
There were 28 subjects, of which 75% were male (N=21) and 25% were female (N=7).
Regarding age, 50% were in their 20s (N=14), 14% were in their 30s (N=4), 22% were in their 40s (N=6), and 14% were in their 50s (N=4).
54% subjects (N=15) had experience of using photo montage and/or VR for landscape study before and 46% subjects (N=13) had no such experience.
14, 50%
4, 14%
6, 22%
4, 14% 0, 0%
20代
30代
40代
50代
60代
3. Comparative verification of landscape simulation methods
S
S
S
S
S
The question items on the reproducibility of a landscape were "reality", "reproducibility", "scale grasp", "immersion", and "intuitiveness". The question items on operability were "easiness", "feedback", and "interactivity". The question items on cost were "expense", "creation time".
The questionnaire result was scored using a 5-point scale. Five points was the best value. An independent t-test was performed according to simulation methods.
Question items Large classification Small classification
Reproducibility
Reality Reproducibility Scale grasp Immersion Intuitiveness
Operability Easiness Feedback Interactivity
Cost Expense Creation time 23
3. Comparative verification of landscape simulation methods
3.1 Experimental Outline -6
24
3.2 Differences between Cloud-VR and mobile AR in Evaluation
In regard to operability, mobile AR acquires the position data of CG virtual camera by GPS or GIS, and acquires the angle data of one with a gyroscope in real-time. Cloud-VR defines beforehand the position data and the angle data of view-points. Features such as fly-through, walk-through, parallel translation, rotation, etc. are operated via a GUI (Graphical User Interface) on a screen.
The screen size of the Cloud-VR is 10.1 inches, and the screen size of the mobile AR differs from 3.8 inches. However, the subjects considered the screens to be the same size.
At the time of the experiment, although texture mapping was used in the Cloud-VR, it was not used in the mobile AR. Since it is technically possible, the mobile AR was evaluated as if the texture mapping had been used.
3. Comparative verification of landscape simulation methods
25
3.3 Results and Discussion
As for the mobile AR, all the user groups gave a score of four or more points for "scale grasp", "immersion", "intuitiveness", "easiness", "feedback" and "interactivity". The score of 3.2 or more points was given for "reality", "reproducibility", "expense" and "creation time" which were the remaining items.
The items, “immersion", "feedback", and "interactivity” of photo montage and the items "expense", "creation time" of Cloud-VR were lower than three points. That is, mobile AR was given a high evaluation for all items.
3. Comparative verification of landscape simulation methods
26
3.3 Results and Discussion: AR vs. Photo Montage
In all the user groups, a significant difference was obtained for the items "immersion", "easiness", "feedback", and "interactivity". In the experienced subjects, a further significant difference was obtained for "intuitiveness".
Why "feedback" and "interactivity" were given a high evaluation is considered. Both photo montage and mobile AR create a 3DCG model superimposed on a photo or live video. A photo montage is a two-dimensional picture and cannot respond to changes in the viewpoint position or direction during study. On the other hand, mobile AR can change the position and direction of the viewpoint corresponding to the user's intention.
3. Comparative verification of landscape simulation methods
△/▼: significant difference 5%, △△/▼▼: significant difference 1%, △△△/▼▼▼: significant difference 0.1%, △: Left conditions have a large value., ▼: Right conditions have a large value.
Comparison
Reality
Reproducibility
Scale grasp
Immersion
Intuitiveness
Easiness
Feedback
Interactivity
Expense
Creation
time
Whole (N=28)
AR PM △△△ △△ △△△ △△△
AR VR ▼ △ △△ △ △△△ △△△ Experienced
(N=15)
AR PM △△△ △ △△ △△△ △△△
AR VR ▼ △△ △△ △△
Inexperienced
(N=13)
AR PM △ △ △△△ △△△
AR VR △ △△△ △△△
27
3.3 Results and Discussion: AR vs. VR
In all the user groups, a significant difference was obtained for the items "expense" and "creation time". VR needs to create all 3DCG models. AR creates only the subject in the 3DCG model. Therefore, when an object for landscape assessment created using a 3D model is not large, the cost performance of AR is high.
About "reproducibility", the reason the significant difference was obtained for the Cloud-VR may be associated with a problem of the optical integrity of AR. Since VR is created using a full 3DCG model, optical integrity is realized within the VR virtual space. On the other hand, AR differs in the influence of light on the 3DCG model and live video, and also differs in shade expression.
3. Comparative verification of landscape simulation methods
△/▼: significant difference 5%, △△/▼▼: significant difference 1%, △△△/▼▼▼: significant difference 0.1%, △: Left conditions have a large value., ▼: Right conditions have a large value.
Comparison
Reality
Reproducibility
Scale grasp
Immersion
Intuitiveness
Easiness
Feedback
Interactivity
Expense
Creation
time
Whole (N=28)
AR PM △△△ △△ △△△ △△△
AR VR ▼ △ △△ △ △△△ △△△ Experienced
(N=15)
AR PM △△△ △ △△ △△△ △△△
AR VR ▼ △△ △△ △△
Inexperienced
(N=13)
AR PM △ △ △△△ △△△
AR VR △ △△△ △△△
1. Introduction
2. Developed Mobile AR
3. Comparative verification of landscape simulation methods 1. Experimental Outline
2. Differences between Cloud-VR and mobile AR in Evaluation
3. Results and Discussion
4. Conclusion
28
Contents
4.1 Conclusion
For mobile AR, which is used as a smartphone platform, a score of 3.2 or more points was obtained for reproducibility of a landscape, operability, and cost. When comparing it with existing methods, mobile AR is evaluated as being better than equivalent.
When mobile AR was compared with photo montage, a significant difference was obtained for "immersion" and "intuitiveness" of landscape reproducibility, and for "easiness", "feedback" and "interactivity" of operability. This was because mobile AR can respond to changes in the user's viewpoint position or orientation, whereas photo montage cannot.
When mobile AR was compared with Cloud-VR, a significant difference was obtained for "expense" and "creation time" of cost. VR needs to create all 3DCG models. AR creates only the subject using a 3DCG model. Therefore, when an object for landscape assessment created using a 3D model is not large, the cost performance of AR is high.
29
4. Conclusion
4.2 Future Work
A future work should attempt to improve the optical integrity of the AR system.
30
4. Conclusion
Thank you for your attention!
E-mail: Twitter:
Facebook: Linkedin:
[email protected] fukudatweet Tomohiro Fukuda Tomohiro Fukuda