RESEARCH ARTICLE

Tangible mixed reality fore

ve

have been several well recognized interactive digitaltabletop systems, either for collocated design groups or

novel and exciting approaches for humans to interact withvarious scenarios.

Wang and Dunston Visualization in Engineering 2013, 1:8http://www.viejournal.com/content/1/1/8also been reported, addressing issues, such as co-designSchool of Built Environment, Curtin University, Perth, AustraliaFull list of author information is available at the end of the articlefor networked collaborative design groups such asthe TeleDesign interface of Shu and Flowers (1992)and the WebShamn (Tuikka and Salmela, 1999;Merrick et al., 2011; Singh et al., 2011). Interactive digitaltabletop systems have been widely applied to manyaspects of human life such as education (Khandelwal and

Computer-supported collaborative design review hasbeen widely used Mavrikios et al., (2011). The collaborativeconceptual design has been a major area of research work,mainly addressing several web- and agent based approachesto support collaboration during the early stages of productdevelopment, conceptual design tools and frameworks,conflict resolution, and team/project management forconceptual design (Wang et al., 2002). Extensive researchwork on collaborative computer-aided design (CAD) has* Correspondence: x.wang@arch.usyd.edu.au1Australasian Joint Research Centre for Building Information Modelling,involved parties. Distributed design collaboration is made possible by advances in networking techniques. Atangible Mixed Reality (MR)-based virtual design prototype was created as a distributed virtual environment (DVE)for the purpose of improving remote design review collaboration. This tangible MR system has been developed toa point that experimental evaluation is necessary in order to understand the strengths and weaknesses of variousfeatures of the system.

Methods: In this paper, we evaluated the tangible MR system against a commercial 3D distributed design reviewsystem in three aspects: the investigation of how users experienced virtual models in the tangible MR systemas compared with the commercial system, the measurement of the users attitude towards the effectiveness ofthe tangible MR system, and the discoveries of usability issues involved in the tangible MR interface throughusability studies.

Results: The findings from user feedback suggest that the tangible MR system may facilitate problem-solving andthe quantity of work in a given amount of time and that virtual design displayed in the mixed scene was a usefulaid in the design error detection task.

Conclusion: These findings are useful for the improvement of future generations of the MR system. Also thesuggestions can be further generalized to become usability guidelines for the MR developers in other applicationsand domains.

Keywords: Mixed reality; Tangible interface; Usability

BackgroundDistributed design collaboration is made possible byadvances in networking techniques. For instance, there

Mazalek, 2007), office (Wigdor et al., 2007), military(DiGiovanni et al., 2005) entertainment (Cowell et al.,2004) and games (Nilsen and Looser, 2005). They offereda study understanding usacceptanceXiangyu Wang1* and Phillip S Dunston2

Abstract

Background: The design review process is often expensi 2013 Wang and Dunston; licensee Springer.Commons Attribution License (http://creativecoreproduction in any medium, provided the origOpen Access

remote design review:r perception and

due to the need for face-to-face meetings between theThis is an Open Access article distributed under the terms of the Creativemmons.org/licenses/by/2.0), which permits unrestricted use, distribution, andinal work is properly cited.

Wang and Dunston Visualization in Engineering 2013, 1:8 Page 2 of 15http://www.viejournal.com/content/1/1/8systems and feature/assembly based representations,web-based visualization, 3D representations for web-based applications, and 3D streaming over networks(Khan et al., 2006). Several synchronous and asynchronousco-design systems, providing collaborative modelingfunctionality, have been developed. Most of them arebased on the clientserver model, while recently,some systems providing real-time online collaboration,based on a peer-to-peer network, have also beenpresented (Chen, 2001). The integration of differentcommercial client CAD systems into a co-design platformhas been demonstrated in some cases (Huang et al., 2010).Virtual reality (VR) based systems for collaborativeproduct modeling were also suggested in the past(Chryssolouris et al., 2009). Shared product visualizationand collaborative design review has been another majorarea of research and development work (Dong and Kamat,2013). Methods of sharing virtual product representationsover the web and a number of CAD-integrated sharedworkspaces have been presented in the scientific literaturefor distributed design review (Hou et al., 2008; Hren andJezernik, 2009). Shared VR-based environments have alsobeen used to support interactive collaboration in productdesign review (Escorpizo et al., 2011; Liang et al., 2013).Most of the reported research activities focus on providingcollaboration tools for specific phases of the productdevelopment process. Research works on internet-basedproduct information sharing and product data manage-ment (PDM) related applications have also been widelydocumented (Hu et al., 2010). Apart from product designissues, several researchers have also worked on the devel-opment of methods and tools to support real-time collab-oration for distributed activities related to manufacturingand product assembly (Zhen et al., 2010; Wu et al., 2012).The development of web-based manufacturing systemshas also been extensively investigated (Lan, 2009).Mixed Reality (MR) is a powerful user interface paradigm

enhancing a users perception by incorporating thecomputer-generated information into the real world.Augmented Reality (AR), which is a sub-mode ofMixed Reality and sometimes interchangeable with MR incertain cases, can insert digital information into a predom-inantly real environment (Dunston and Wang, 2005).Verlinden et al. (2009) suggested the concept of augmentedprototyping that projects the perspective images of theproduct on the physical object made by rapid prototypingtechniques (Verlinden et al., 2003). Greenberg and Fitchett(2001) presented toolkits called Phigets that allow designersto explore a tangible user interface (TUI) for interactiveproduct design. In addition to the toolkit, powerful toolsincluding stereoscopic display systems, head mounteddisplays (HMD), data gloves and haptic devices have been

introduced (Burdea and Coiffet, 2003) and combined toconstruct Virtual and Augmented Prototyping systems thatprovide realistic display of products in a simulated environ-ment and offer various interaction and evaluation means(Wang et al., 2009). For instance, Bochenek et al. (2001)compared the performance of four different VR displays ina design review setting and mentioned that the bestapproach for design review activities could be a combinedtechnology approach. As for AR adoptions for designreview, Moeslund et al. (2003) and Sidharta et al. (2006)both employ optical see-through Head Mounted Displaysto inspect designs, but little support is provided to the actof reviewing. Verlinden (2003) attempts to improve designreviews in the domain of Industrial Design Engineering byproviding a specific prototyping and annotation deviceemploying physical mockups. The IMPROVE projectexplicitly addresses design reviews for product design;its main focus lies with hardware and algorithms forphoto-realistic rendering (Santos et al., 2007). Theannotation facility is minimal.Although various ways above have been proposed to

support collocated Augmented Prototyping and designreview of industrial and architectural products, moreresearch is still needed in the following aspects. AR shouldbe made available and accessible to geographically distrib-uted people in product design evaluation. The prototypingenvironment should be available at low cost without strongrestriction of its accessibility to remote users. Applicationsof AR have been predominantly focused on industrial andarchitecture product design, however, there is very littlenoted research on AR adoptions in the area of buildingservices, which this paper is presenting. In this paper,we address these aspects by proposing a tangibleMixed Reality-based distributed virtual environment(DVE) MRDVE for distributed concurrentcollaboration, which provides a platform which allowsgroups of people to share the same work and communica-tions space over a network. There have been severalnoted research efforts that investigated the applica-tions of distributed Mixed and Augmented Realitytechnologies for collaborative interior design applica-tions (Ahlers et al., 1995), videoconferencing systems(Barakonyi et al., 2004), tutoring sessions in teachingmulti-variate calculus (Orozco et al., 2006). Other examplesinclude the collaborative augmented reality systemStudierstube (Schmalstieg, 2002), SharedSpace (Billinghurstand Kato, 1999), and mediaBlocks (Ullmer et al., 1998).The development of MR/AR technologies for the architec-ture, design, and engineering industry should be guided byresearch aimed at adapting the features of the MR/ARsystems to the different demands of the various situationsin which it may be used. The user acceptance and usabilityissue are thus of great importance. There have beenseveral attempts to evaluate the effectiveness of Virtual

Reality technology in design and design collaboration.Usability testing has been widely implemented in Virtual

Wang and Dunston Visualization in Engineering 2013, 1:8 Page 3 of 15http://www.viejournal.com/content/1/1/8Environments (Hix et al., 1999; Bowman et al., 1999;Gabbard et al., 1999; Hinckley et al., 1997). Although therehas been much speculation about the potential of MR/AR,considerable work needs to be done in usability assessmentof these mixed environments (Dias et al., 2003; Regenbrechtand Schubert, 2002; Tang et al., 2003). The common featureof these noted research efforts is the attempt to usewell-designed systematic usability tests to create detailedknowledge about the usability aspects of the various inter-action and information presentation techniques involvedin remote MR/AR systems. Very few empirical studieshave been done in the area of MR/AR, which creates aknowledge gap. Wang and Dunston (2008) implementedan experimental evaluation that was intentionally gearedand implemented to investigate how users experiencedvirtual models in the MR system and to measure the users(1) experience, (2) satisfaction, and (3) attitude towardsthe effectiveness of viewing the 3D models through theMR system as compared with viewing those 3D modelson paper.The most state-of-the-art of commercial design review

software provide environments for remote collaborativereview. Since industrial construction practitioners arefamiliar with such remote design review tools, positiveresults from comparing the MR tool to one suchcommercial tool may persuade practitioners to movetoward the adoption of MR/AR to change the waythey work. Based on this motivation, experimentationon the tangible and distributed MR-based virtualspace system is presented, a follow-up study to theone by Wang and Dunston (2008) mentioned above.The purpose of this experimentation is to gather andgenerate detailed knowledge, through well-designedempirical tests, about the usability aspects of the variousinteraction and information presentation techniquesinvolved in the MR-based virtual space system from apractical perspective. Specifically, the experimentationassessed the effectiveness of the tangible MR system ascompared with a representative commercially availableremote collaboration system. Although the experimenterschose the mechanical design review as the context forevaluating the MR system, the results and learned lessonscould possibly be lent to other similar types of designreview activities.

Methods: System Prototyping and ExperimentalEvaluationPrototyping the tangible mixed reality-based distributedvirtual environmentsFigure 1 depicts the generic system architecture of theMR-based virtual space system. This prototype wasbased on the previous version of an MR prototype

introduced by Wang and Dunston (2008) for face-to-facedesign review collaboration but is a distributed versionwith extra remote functions and features. CommercialNetMeeting software is used for audio communicationonly. The hardware has been operated on a Pentium 4 PCwith a 1.6 GHz processor (central server) and on differentPCs and laptops (local clients) with head-mounteddisplays (HMDs). The components or devices consist ofthe central server, network, client computer, HMD,markers and arbitrary real objects. The central server isconfigured with a CAD application. The central sever cancontrol what information is sent to each client. Thenetwork configuration connects server and clients, wherean IP network with multicast support was realized on topof the network infrastructure. Client computers areconfigured with the same MR software, which may receivereal-time online data packets for graphics objects.Multiple-marker recognition was used as the trackingtechnology where a high contrast tracking marker isessential. By manipulating the cube (real objects) to con-trol the virtual tracking ball, users can naturally interactwith the virtual object in the virtual graphic world, theusers can tell the location of the real cube with referenceto the virtual object. Therefore, by using the tracker cube,the user can track his/her hands position relative to theviewed virtual model by knowledge of the spatial relation-ship between the virtual model and the tracker ball. In thefollowing sections, the tangible interface and virtual net-work computing components are elaborated because theyare essential for implementing and understanding the useracceptance study presented in the latter part of the paper.

TangibilityTangible interfaces enable the integration of convenientarbitrary objects into the real collaboration environmentfor effective object manipulation. Tangible interactiontechniques applied in the system are incorporated viatracking markers for Mixed Reality registration. The userscan manipulate (rotate, tilt, move around) the virtualmodel design by manipulating the physical trackingmarker underneath it. Video-based fiducial recognitionwas used as the tracking technology where a high contrasttracking marker is essential. The computer performsimage processing on the video image captured from thecamera to find the specially marked tracking marker(s).Once an appropriate marker is recognized, a virtual modelis then registered to the position of the marker. Theresultant composite image reveals the insertion of the 3Dvirtual model into the users real world view. Wheneverthe spatial relationship between the camera and the track-ing marker is changed, the virtual model moves in concertwith the marker. The other key tangible interaction tech-nique applied is a tracker cube used for interpreting thespatial relationship of virtual and real entities. The tracker

cube is actually a specially marked, small-size real cubewith a virtual ball overlaid on it (see Figure 2).

Wang and Dunston Visualization in Engineering 2013, 1:8 Page 4 of 15http://www.viejournal.com/content/1/1/8View windowsThe system design is based on a distributed shared

Figure 1 System architecture of the MR-based virtual space.Mixed Reality scene window. When communicating themodel with others over the network and transferringattention back and forth between discussion with otherpeople and observation of his/her own 3D models, theuser must periodically share the perspective and focus withthe other collaborators in order to ensure consensus intheir decisions. Such consensus can enable collaborator Ato keep track of the series of actions taken by collaboratorB from the latters viewpoint. For example, collaborator B

Figure 2 Illustration of the virtual tracking ball for positioningthrough manipulation of a real cube (Wang and Dunston 2008).asks A to navigate to a target by passing three walls in frontand then turning 60 degrees left and then passing another

five walls forward in a virtual environment. Collaborator Amight be confused by such verbal instructions and mightfail to reach the target. An effective collaboration isthus difficult to accomplish. Effective perspective sharingthrough certain groupware software can reduce suchcognitive load that is required otherwise.

Experiment setupAs stated above in Section 1, the entire experimentationwas implemented using the procedure/method that wasset by the authors in a study comparing face-to-facecollaboration facilitated by paper and MR media(Wang and Dunston, 2008). This procedure was usedin that instance to measure the users perspective ona tabletop Mixed Reality collocated application. Theremaining discussion details the evaluation of theMRDVEs using the same method. The experimentaimed to collect data for investigating the subjectsexperience with the MR system, gauge the subjectsattitude towards the MR system, and discover usabilityissues. Three questionnaires were administered to individ-ual participants in an experiment where pairs of subjectsworked collaboratively using both the MR-based virtualspace system and a leading commercial design reviewapplication to detect design errors: the first questionnairewas devised to measure the users attitude towards the

effectiveness of the MR system; the second questionnaireinvestigated how users experienced virtual models inthe MR system as compared with the commercial sys-tem; the third questionnaire aimed to reveal usabilityissues involved in the MR system interface through ausability study.

Comparison benchmark: vommercial3D designreview toolThe commercial design review application chosen forcomparison, NavisWorks Roamer, is sophisticatedcommercial 3D design review software which enablesgeographically distributed users to review 3D designs(see Figure 3). For interaction with models inNavisWorks Roamer, there are nine navigation modesto control how you move around the main navigationview six camera-centric modes (walk, look around,zoom, zoom box, orbit, fly) and three model-centricmodes (pan, examine, turnable). In a camera-centricmode, the camera moves within the scene, whereas ina model-centric mode, the model moves inside the scene.NetMeeting software is incorporated into NavisWorksRoamer as a built-in functionality. It was the experi-menters responsibility to install the marker in its precise

not lose the tracking image, the marker was set still in thespace. The human subjects only needed to familiarizethemselves with the MR interface and how to interactwith the MR information.For collaborative design review, NavisWorks Roamer

essentially supports asynchronous collaboration, whereone collaborator finds the suspicious error, redlines it,saves the current viewpoint, and then sends the informationto the other collaborator. The other collaborator thenreceives the file and loads the saved viewpoint forreview. However the MR interface support synchronouscollaboration and thus allows real-time view slaving andsharing. Based on the collected data, the time spent onsending views in NavisWorks was quite short, averagingless than half a minute, which almost constitutessynchronous collaboration. This situation is similar tosending and receiving emails between two personswho are waiting at both ends for each exchange.Furthermore, during the experiment, the experimenterintentionally excluded recording the time that wasspent on file transfer in order to make its comparisonwith MR fundamentally fair. A stop watch was usedto capture the task duration without file transfer.When the subjects filled in the questionnaires, the

Wang and Dunston Visualization in Engineering 2013, 1:8 Page 5 of 15http://www.viejournal.com/content/1/1/8position and calibrate the MR system. The markers finalrelevant position was not subject to be altered afterwards.For example, in order to guarantee that the camera didFigure 3 Screenshot of NavisWorks Roamer with redlining and interaexperimenter alerted them to not consider the file transfertime as a drawback/effect when rating/evaluating theNavisWorks performance.ction functions demonstrated.

cognition and reasoning, any brain damage, or eye(vision) problems that were not corrected by standardeyeglass prescription. It is important to note thatthere was no cross-over between participants in thisexperiment and the earlier experiment by Wang andDunston (2008) comparing the MR approach to thepaper-based method. Thus learning curve effects andrelated biases were avoided as well.Industry participants in design reviews typically want

to view and browse the design independently. Therefore,the experimental task was to review and inspect thedesign model independently and then figure out thedesign errors. The error detection task was chosenbecause it emphasizes design model comprehension and

Wang and Dunston Visualization in Engineering 2013, 1:8 Page 6 of 15http://www.viejournal.com/content/1/1/8HypothesesAs noted above, post-experience questionnaires were usedto collect data for investigating the subjects experiencewith the MR system, gauge the subjects attitude towardsthe MR system, and discover usability issues. Ourprior experience in testing applications of MR technologyhas demonstrated that user issues of visual quality,maintaining spatial awareness, and functional utility(i.e., comfort and ease of use) are important to potentialusers embracing such a system (Wang and Dunston,2006; Wang and Dunston, 2008). Therefore, the firstquestionnaire implemented was designed to evaluatethe subjects experiences of using the tangible MR-basedvirtual space system and NavisWorks Roamer in terms ofthe following five hypotheses:

Hypothesis 1 asserts the following: When comparedto NavisWorks Roamer, the MR-based virtual spacesystem will involve significantly better quality ofvisual presentation;

Hypothesis 2 asserts the following: When compared toNavisWorks Roamer, the MR-based virtual spacesystem will induce significantly higher physical comfort;

Hypothesis 3 asserts the following: When comparedto NavisWorks Roamer, the MR-based virtual spacesystem will involve a significantly more natural andintuitive interaction with the model;

Hypothesis 4 asserts the following: When comparedto NavisWorks Roamer, the MR-based virtual spacesystem will involve a significantly higher level ofimmersion;

Hypothesis 5 asserts the following: When comparedto NavisWorks Roamer, the MR-based virtual spacesystem will give users significantly more ability tomaintain sense of location and orientation.

Subjects, tasks, materials and methodSixteen (12 men, 4 women) graduate student partici-pants were recruited for the study (8 groups with 2in each group). The participants ages ranged from 25to 33. Important to the backgrounds of those graduatestudents was their lack of any 3D computer-aided drawingand animation experience, which practically eliminatedexperience-based bias. While selecting the subjects, ascreening was conducted before the experiment in theformat of a simple oral interview. All participantswere pre-screened to ensure they had no sophisticatedexperience with CAD, VR, and MR applications. Themodules for design review in the experiments were alltaken from real construction industry designs and thus aproper level of complexity in the modules was ensured.These precautions can make the experience factor

insignificant to bias the results. Furthermore, none ofthe participants had any known impairment in spatialopportunities for collaboration and it involves a highdemand of mental interpretation and maintaining aviable spatial mental model. Real 3D CAD models,obtained from an industry partner, were used as designsto be reviewed using the two methods. Small-scalecluttered ductwork models were used for collaborationbetween two individuals in separate rooms but connectedover the network. Before their collaborations, each teamof participants was briefed consistently on the typesof errors that might be found. Two ductwork modeldesigns were confoundedly used for the two treat-ments (MR and NavisWorks Roamer) respectively(see Figures 4 and 5). Both designs had a similarspatial layout but different locations of the errors.Three questionnaires were designed to be completedby the subjects with the purpose of investigating theusers experience with the MR system, measuring usersattitude towards the MR system, and implementingusability studies.

TreatmentsThe Treatment 1 was to perform the review using com-mercial NavisWorks Roamer where the pair of subjectsFigure 4 Design Layout of AutoCAD Design I for thetwo treatments.

could interact with the 3D ductwork design in a desktopwindow environment (see Figure 6a) in two differentrooms. The Treatment 2 was to use the MR system(user can move his/her marker in hand and view themodel from different angles via HMD) where the subjects

ductwork model floating above the corresponding realtracking marker via their HMD (see Figure 6b). Subjectscan change viewpoints by either moving around ormanipulating the real tracking marker underneath thevirtual ductwork model.

Experiment processEach experiment session lasted three hours, including train-ing session, actual experiment session, and post-experimentquestionnaire session.

Training session: The subjects were instructedto use two treatments/methods (MR andNavisWorks Roamer) to review a design. Theywere assigned enough time to practice how touse the different platforms.

Actual experiment session: In the realexperiment, each team consisted of two subjects(collaborators): one in office A that owns PartI of the design and one in office B that ownsPart II. Each collaborator reviewed the designof the combined models. Each group performeda trial with each method (treatment) to checkthe two respective sets of the model in a

Figure 5 Design layout of AutoCAD design II for thetwo treatments.

Wang and Dunston Visualization in Engineering 2013, 1:8 Page 7 of 15http://www.viejournal.com/content/1/1/8in two different rooms were able to see their own virtualFigure 6 Screen Shots of two Visualized Collaboration Treatments. (aspecified sequence. During this process, the) NavisWorks View and (b) MR View.

subjects checked the design errors (see Figure 7for an example).

Post-experiment questionnaire session: At the end ofthe session, each user or group was asked for theircomments and observations on the system, as wellas presented a set of questionnaires on thecomparison of the MR system and NavisWorksRoamer regarding various features

The participants performances were measured bysubjective matrices, such as questionnaires, as well asobjective matrices, such as task achievement andduration. It was generally observed that the subjectsusing the MR system completed the task much fasterthan the subjects using NavisWorks Roamer. It is believedthat the perception and spatial cognition advantages ofthe MR system resulted in an offloading of some mentalprocessing to the MR system and thus enabled thesubjects to complete the task in less time. Further detailsof the time comparison are described elsewhere by Wang(2005). Furthermore, special useability questionnaires and

Development of questionnairesThe first questionnaire was developed to evaluate thesubjects individual experiences of the virtual model inthe MR system as compared with how they experiencedthe design in the NavisWorks Roamer environment. Thequestionnaire ratings were on a five-point scale, with avalue of 5 representing excellent and 1 representingpoor. The questions and a summary of the results arepresented in Table 1, including average ratings andstandard deviations for each of the methods features.Aspects covered include quality of visual presentation,physical comfort, naturalness and intuitiveness of inter-action with the model, level of immersion, ease of naviga-tion, ability to maintain sense of location and orientation,overall suitability for making decisions and performingtasks on design models.The second questionnaire was developed to evalu-

ate the subjects experiences of the virtual model inthe MR-based virtual space system for conductingcollaborative work compared with their experiencesin the NavisWorks Roamer environment. In order tominimize the questionnaires bias and/or order

Wang and Dunston Visualization in Engineering 2013, 1:8 Page 8 of 15http://www.viejournal.com/content/1/1/8associated data collection strategies like interviewsand observations were developed in order to assessthe participatory process and certain features of theMR space. Finally, a statistical model was also developedto arrange experimental sessions and collect data. Astatistical analysis parameter (p-value) was used totest inter-factor correlations for reliable results.Figure 7 An example of design error pattern: interference between tweffects on the results, half of the subjects evaluatedthe MR system relative to the NavisWorks Roamer,as in Questionnaire I, while the other half evaluatedthe NavisWorks Roamer relative to the MR system,as in Questionnaire II (see Table 2). The five-scalerating is from totally agree to neutral to totallydisagree.o rectangular ducts highlighted in red.

The third questionnaire aimed to investigate furthersome of the questions not fully addressed by the previousquestionnaires. The questionnaire was used to explore theusability issues involved in the current version of the MR

in using the MR system. Comfort was measured in theareas of arm strain, neck strain, dizziness, and nausea.

Results and Discussion

Table 1 Evaluation of the subjects experiences of using the MR system and the NavisWorks Roamer, rated from 1(poor) through 5 (excellent)

No. Questions NavisWorksRoamer

MR F Value p-value SignificantDifference?

Mean (S.D.) Mean (S.D.)

1 Quality of visual presentation 3.1 (0.99) 3.6 (0.74) 2.14 0.1543 No

2 Physical comfort 3.27 (1.22) 2.73 (0.83) 2.09 0.1602 No

3 Naturalness and intuitiveness of interaction with model 3.26 (1.03) 3.87 (0.64) 3.66 0.0661 No

4 Level of immersion 2.73 (0.88) 4.37 (0.81) 27.78 0.05and F (1, 14) = 2.14, which does not support Hypothesis 1.This is different from the results obtained from the previous

nions on the value of the MR system for collaborative

Questionnaire II1. I felt that 3D interactivity in the NavisWorks Roamer aided designcomprehension.

2. Overall, compared with AR, the NavisWorks Roamer better facilitatesdesign collaboration tasks.

3. The NavisWorks Roamer better facilitated communication.

4. The NavisWorks Roamer better facilitated creativity.

5. The NavisWorks Roamer better facilitated problem-solving.

6. The NavisWorks Roamer increased the overall quality of output fromthe collaboration.

7. The NavisWorks Roamer better facilitated the quantity of work I couldcomplete in a given amount of time.

8. The NavisWorks Roamer increased the quality of my contribution tothe project.

9. The NavisWorks Roamer increased my satisfaction with the outcomeof the project.

10. The NavisWorks Roamer increased understanding between mycollaborator and me.

or

of y

tsid

on

e w

o

s a

ble

ery

Wang and Dunston Visualization in Engineering 2013, 1:8 Page 10 of 15http://www.viejournal.com/content/1/1/8experiment by Wang and Dunston (2008) where an MRsystem was proved to provide more realistic and vividvirtual models than a corresponding 3D color drawing onpaper. The virtual model in the MR-based virtual spacesystem possessed a similar level of fidelity as the 3D modelin NavisWorks Roamer, however, the current quality of thevirtual model in the MR system is somewhat limited(bottlenecked) by the resolution of the head-mounted

Table 3 Means (S.D.s) for usability questions

No. Questions

1 Did the MR visual display (head-mounted-display) create difficulties f

2 Did you often feel disoriented?

3 Did you have a natural perspective that gives you a compelling sense

4 With the MR system, are you isolated from and not distracted by ou

5 Did the surrounding real background help your spatial comprehensi

6 Was the FOV (field of view) appropriate for supporting this activity?

7 Does visual output have adequate stability of the image as you mov

8 Does visual output have acceptable degree of response delay with n

9 Is the MR display effective in conveying convincing scenes of model

10 Can you predict responses to your actions?

11 Did you have satisfactory control over the system?

12 Is the MR system comfortable for long-term use?

13 Is tracking marker lightweight, portable, non-encumbering, and comforta

14 Did you experience excessive eye fatigue?

15 Did you experience nausea during your interaction with MR system?a The question response format was a scale with steps from 1 (very little) to 5 (vdisplay. The current virtual models in use were convertedfrom original 3D AutoCAD models through the PolyTransConverter, where certain rendering information is lost.A more sophisticated converter might preserve morerendering data, therefore obtaining a higher level offidelity in the MR virtual model. Some subjects commentedthat more shading and shadow rendering might furtherimprove the visual quality and thus enhance the usersspatial cognition.The subjects felt more physical comfort using NavisWorks

Roamer (Mean, 3.27; S.D., 1.22) than the MR system(Mean, 2.73; S.D. 0.83). However, the significance wasnot attested with only a p-value (0.1602) > 0.05. Thisdoes not support Hypothesis 2. The users neck andback strain (approximately 1020 minutes of use) inthe MR system was induced by the long-time uncomfort-able poses for inspecting details of the model and thephysical weight of the HMD.Interaction with the virtual model via the nine

functions in the NavisWorks Roamer seemed to be asnatural and intuitive as in the MR system accordingto the ratings (NavisWorks: Mean, 3.26; S.D., 1.03;MR: Mean, 3.87; S.D., 0.64). The difference is insignificantaccording to the p-value > 0.05, and therefore Hypothesis3 is not supported. This similarity can be explained as thetrade-off between the unnaturalness of MR due to thedemand of more physical effort and the intuitiveness ofinteraction with the virtual model via the virtual trackingball. The subjects assessed the level of immersion for theMR system at an average rating of 4.37 (S.D., 0.81)compared with 2.73 (S.D., 0.88) for the NavisWorksRoamer with a significant p-value indication, which

Mean (S.D.)a

performing? 4.31 (0.96)

2.93 (1.18)

our hand-motion while manipulating the tracking cube? 3.38 (0.88)

e activities? 3.75 (1.13)

of the model? 3.56 (1.15)

2.63 (1.2)

ith no perceivable distortions in visual images? 2.81 (1.17)

perceivable distortions in visual images? 3.31 (1.14)

ppearing as if in the real world? 3.88 (0.88)

4.0 (0.57)

3.38 (0.88)

2.06 (1.28)

thereby avoiding issues of limited your mobility and fatigue? 3.94 (0.85)

3.5 (1.09)

1.75 (1.0)

much).strongly implies that the MR system provides the users agreater sense of being present with the model. ThusHypothesis 4 is well supported.Subjects did not identify much difference in the ease

of navigation between these two treatments, rating bothof the tools as providing a mediate ease of navigation(MR: Mean, 3.87; S.D., 0.99; NavisWorks Roamer: Mean,3.2; S.D., 1.15). However, for inspecting the details ofthe design model, the MR system enabled the usersto move more easily to a more advantageous view-point (closer and directly in front of the target) tocomplete the error verification more quickly. Thisadvantage was observed by the experimenter in alarge number of trials. The subjects had learned thelayout of the 3D design space and expressed moreconfidence in the error detection task without furthereffort to interpret the occlusion and depth cues amongthe objects because they were reviewing the design modelin a 3D manner.Hypothesis 5 is well supported because subjects

using the MR system had higher ability to maintainsense of location and orientation as they navigatedthe model than when using NavisWorks Roamer(MR: Mean, 4.1; S.D., 0.7; NavisWorks Roamer:

Wang and Dunston Visualization in Engineering 2013, 1:8 Page 11 of 15http://www.viejournal.com/content/1/1/8Mean, 3.2; S.D., 1.16). The difference is regarded assignificant with a p-value (0.0305) < 0.05. Many subjectscommented that there was easy loss of spatial orientationin using NavisWorks Roamer. NavisWorks Roamer pro-vides the subjects with the birds-eye view (exo-centric)and close-view (ego-centric) for review. As the subjectstried to switch between these two views through theviewing functionalities (e.g., zoom out from a smallobject to the complete view of the design model),they had to mentally transform the view in their mindto accommodate the change of these two reference frames.Therefore it was often observed and reported that thesubjects became lost or disorientated, especially after usingthe interactive viewing techniques to switch between thesetwo views. This disorientation and re-accommodation wasan additional mental burden to the cognitive attentionswitching. Some subjects using the NavisWorks Roamerreported that they felt frustrated because it was hard toeven know the previous viewpoint of the design and theyhad to spent additional time to learn the model again fromthe new perspective. In contrast, fewer cases were reportedfrom the users of the MR system since MR provided asmooth and natural way to transit back and forth. Thesubjects using the MR system could offload this centricitytransformation task to the intuitive viewing mechanism ofthe MR system. When the subject leaned back allowing thecamera mounted on the HMD to capture the completedesign, a birds-eye view (exo-centric) was created. Forinspecting the details of design model, the MR systemenabled the users to move their viewpoint to a moreadvantageous position (closer to the target, with the targetdirectly in front of them) to complete the task morequickly. When the subjects approached a target closelyenough, an ego-centric view was created. The extra timespent in using the NavisWorks Roamer to accommodatethe switch of reference frames was certainly mostlycognitive overhead. Therefore, the frequent loss oforientation could be interpreted as the inefficiencymargin of the NavisWorks Roamer against the MR systemin the subjects spatial orientation activity.The final point of comparison asked for a general

impression of the two methods. Subjects believed thatthe MR system was more suitable for making deci-sions and performing tasks on design models thanthe NavisWorksRoamer system.

Results of questionnaire 2For the convenience of interpreting the data from thetwo questionnaires in Table 2, it was rationalized thatthe respondents who totally agree with a statement ofQuestionnaire I were regarded to totally disagreewith the corresponding statement of Questionnaire II.

Therefore, the percentages for these responses wereadded together. A similar summation of results wasapplied to the other four levels of the response scale.The data from the two questionnaires was thus talliedwith respect to attitudes regarding the MR systemand is visually presented in Figure 8.As indicated in the Figure 8, about 69% of the respon-

dents felt that 3D interactivity in the MR system aideddesign comprehension in total or partial agreementwith the statement. Question 2 asked if the MR systemoverall better facilitates design collaboration tasksthan the NavisWorks Roamer, with which about 69%of respondents agreed. Most of the respondents(63%) believed that the MR system better facilitatedcommunication and creativity and a total of 65% ofthe respondents totally or partially agreed with thestatement that the MR system better facilitatedproblem-solving. The increase in the overall qualityof output from the collaboration is marginally positive(57%), however, the belief that the MR system better facili-tated the quantity of work in a given amount of time andincreased the quality of the users contribution to theproject is more marked (70% and 70% respectively). It issurprising that these impressions are not stronger in lightof the clear time benefits that were observed. The beliefthat there has been an increase in self-satisfaction from thecollaboration as a result of using the MR system iswell supported by 63% of the users. Only one-third ofrespondents, however, believed that the MR systemincreased understanding between their collaboratorand them. One-third held a neutral attitude and one-thirddisagreed with the statement. A summary of the resultsconvey the conclusion that the MR system both aidsindividual performance and also supports the collab-oration, but there are implications that improvementsare yet desirable.The MR system was rated to offer the advantage of

improved collaboration as expected. By sharing the sameviewpoint continuously and synchronously between tworemote subjects via VNC platform, one subject could main-tain the same changing of viewpoints and frame of refer-ence along with the other subject who led the discussionand was in charge of navigating in the MR environment.On the contrary, the subject using the NavisWorks Roamerhad to iteratively go through the process of finding the sus-picious error, redlining it, saving the current viewpoint, andthen sending the information to the other subject. Theother subject then received the file and loaded the savedviewpoint for review. Therefore, the receiving subjecthad to execute additional mental transformation toadapt to the static viewpoint saved by the sendingsubject for creation of a discussion context. The subjectstask performance using the MR system was thereforeimproved by reducing the frequency of such electronic

exchanges of the static red-lining comments and savedviewpoint between subjects.

Wang and Dunston Visualization in Engineering 2013, 1:8 Page 12 of 15http://www.viejournal.com/content/1/1/8Results of questionnaire 3This section explored some of the usability problems iden-tified from Table 3 and possible redesign recommendationsas given for the future generations of MR-based virtualspace systems. The following discussion and interpretationsare based on the selected questions from Table 3.Questions 1, 2, 4, 6, 7, 8, 9, 14, and 15 deal with

the usability of the HMD. From the ratings in thefirst question, the respondents reported that the MRvisual display did create difficulties for performing(Mean, 4.31; S.D., 0.96). The difficulty mainly resultedfrom having to wear the cumbersome and uncomfortableHMD. The fact that one hand of the user needed to holdthe HMD to maintain a stable view of the mixed scene,increased the arm muscle fatigue. Also, the screen in theHMD was designed to make the user feel that the screenwas actually placed several feet away from the eyes. Thefaraway and low resolution display resulted in a low clarityof the virtual model compared with a standard computermonitor. The fidelity of the HMD is totally dependent on

Figure 8 Plot of Responses to Attitude Questionnaire.the advance of technology in the industry. More advanceddisplays might be a promising option for exploration anddevelopment. HMDs are well-suited for applications(e.g., architectural and interior design) where completevisual immersion or absence of distractions is required.From the relatively high rating in question 4 (Mean, 3.75;S.D., 1.13), it seemed that the users did not feel muchdistracted by outside activities (distractions), whichimplied that the HMD was useful in focusing the usersminds on the task at hand. This result is consistent withthe subjects assessment of the level of immersion for theMR system at an average rating of 4.37 in Table 3.For question 6, the rating for appropriateness of theFOV for supporting this activity was relatively low(Mean, 2.63; S.D., 1.20). The broader the FOV, thebetter sense the user should have for the environmentand communication with other users. This sense ismuch reliant on the HMD and video camera as thesubjects FOV is constrained to the governing FOV of thedisplay or the camera. The limited field-of-view of thevideo camera currently used, constrained the usersview to some extent. The visual output had moderatestability of the image as far as allowing the subjectsto move with no perceivable distortions in visualimages (Mean, 2.81; S.D., 1.17) as indicated by Question 7.The stability of the vision tracker (fiducial recognition) isstill insufficient for ensuring stable Augmented Realityapplications. Thus improved or alternative methods arerequired for maintaining a proper registration. The furthersubstantiation of the tracked six-degree-of-freedom databy redundant tracking methods can be a feasible solution.Hybrid technology tracking systems should be examinedfor applicability and performance regarding stability. Inresponse to Question 8, respondents thought the MRsystem had a relatively acceptable degree of responsedelay with no perceivable distortions in visual images(Mean, 3.31; S.D., 1.14). Thus, the system lag was tolerableand did not affect the perception of the visual image by

users and therefore did not hinder their performance. Thesubjects experienced a rather convincing scene of modelsappearing as if in the real world (Mean, 3.88; S.D., 0.88)as reported for Question 9. The virtual model lookedconvincingly to be floating in the air of the real envir-onment, which was due to the stereoscopic feature ofthe MR display with two display panels creating highobject presence effects. The relatively high ratingimplied that the combination of virtual model andreal world reached seamlessness to some extent. It isinteresting to note regarding Question 14 that therating (Mean, 3.5; S.D., 1.09) implies that some subjectsmight have experienced light eye fatigue. The subjectsnormally wore the HMD for only 2030 minutes for theentire experiment. If a user has to wear it for a longertime, eye fatigue might emerge more prominently asa negative. On the contrary, regarding the last question,the subjects experienced very little nausea during their

Wang and Dunston Visualization in Engineering 2013, 1:8 Page 13 of 15http://www.viejournal.com/content/1/1/8interaction with the MR system (Mean, 1.75; S.D., 1.0).These issues need to be further examined and addressed iflong-duration tasks are to be performed. It was alsorevealed from some additional short-answer questionsin the questionnaire that about 87.3% of respondersbelieved that they would not be resistant to using MRor similar MR systems in the future. About 81.3% believedthat they would embrace the opportunity to use the MRsystem again in the future. Almost none of the usersexperienced blurred vision, dizziness, nausea,difficulty focusing, or loss of vertical orientationafter exposure to the MR system.Questions 3 and 13 deal with the naturalness of inter-

action. In response to the question whether the subjectshad a natural perspective that gave them a compelling senseof their hand-motion while manipulating the trackingmarker (Question 3), the rating (Mean, 3.38; S.D., 0.88)indicates a mild agreement. From Question 13, the subjectsrevealed a belief that the tracking marker was light-weight, portable, non-encumbering, and comfortablethereby avoiding issues of limiting their mobility and fatigue(Mean, 3.94; S.D., 0.85). These results are consistent withthe rating (Mean, 3.87; S.D., 0.64) from Item 3 in Table 1.Questions 5, 10, 11, and 12 deal with the issues

regarding the overall control of the MR system. Thesurrounding real background seemed to marginallyhelp the subjects spatial comprehension of the model(Mean, 3.56; S.D., 1.15) as evidenced by responses toQuestion 5, an expected result. The real backgroundcould have served as a stationary landmark referencefor orientation of the virtual model, especially while userswere turning the tracking marker. Question 12 responsesindicate almost all the subjects had a fairly uncomfortableexperience with long-term use of the MR system, with avery low mean rating of 2.06 (S.D., 1.28). This result is inaccordance with the high physical discomfort rating in thefirst questionnaire discussed earlier. The bulky HMD isthe most significant usability issue of the MR system.Because the design review collaboration task mightrequire long-duration, intensive work, the MR systemwould require moderately high levels of ease of useand user comfort. It was found that using the MRsystem produced neck strain in some users. Thetracking and interaction techniques in future generationsof the MR system must be easy, comfortable, and intuitiveto the user, so that the focus can be on the task and noton the system itself.

ConclusionBased on the experimental evaluation of the MR-basedvirtual space system, several conclusions were noted. Itwas observed that the subjects using the MR system

completed the task much faster than the subjects using theNavisWorks Roamer commercial visualization software. Itappears the increased 3D visual perception, spatial cogni-tion, and spatial layout of the design offloaded the requiredmental processing to the MR system and thus enabled thesubjects to complete the task in a shorter time. Subjectsfelt more physical comfort using NavisWorks Roamer thanthe MR system. The subjects rated the level of immersionfor the MR system, on average, much higher than for theNavisWorks Roamer application, which implies that MRdefinitely provides the users a sense of being present orisolated with the model, more than with the static modelin NavisWorks Roamer. The subjects believed that the MRsystem was more suitable for making decisions andperforming tasks on design models than the NavisWorksRoamer. MR seemed to better facilitate communication,creativity and problem-solving. The increase in the overallquality of output from the collaboration is marginallypositive, however, the belief that the MR system betterfacilitated the quantity of work in a given amount of timeand increased the quality of the users contribution to theproject is more marked. Perhaps more importantly,there is a belief that there has been an increase inself-satisfaction from the collaboration as a result ofusing MR.The usability studies implemented with the third

questionnaire revealed usability issues and meanwhilehelped to produce possible remedies. These suggestionsare useful for the improvement of future generationsof the MR system and summarize below Also thesesuggestions can be further generalized to becomeusability guidelines for the MR developers in otherapplications and domains.

Participants suggested a zoom feature to facilitateinspection of some tiny objects inside dense modelstructures.

Participants expected a quicker response in VNC forthe real-time collaboration. There was still somelatency (lag) in synchronizing the MR scenesbetween the two collaborators because the VNCconveyed the video stream which occupied toomuch of the available computing resource. Theperformance was heavily dependent on thecomputing resource and the network bandwidth.

Participants suggested a better user-friendly display.The current HMD was not comfortable enough forlong term use. The HMD kept falling and neededone hand to hold it to a correct position, whichcreated extra hand occupation and distracted theusers from focusing on the work at hand.

Participants requested a camera with an automaticfocusing function. If the user pulled his or her headback farther, the camera failed to recognize the

tracking marker even though it was actually in theFOV of the user.

(pp. 96103). IEEE.

Wang and Dunston Visualization in Engineering 2013, 1:8 Page 14 of 15http://www.viejournal.com/content/1/1/8 Participants suggested adding shadows and shadinginto the virtual design to help the users obtain moredepth cues for better apprehension of the spatialrelationships.

Even though these experimental results are preliminary,they proved useful in deriving guidelines and furtherdevelopments and experimentation. Learning from thepreliminary results of the usability study, the authors nowhave to refine the MR interface to tap into more potentials(Wang et al., 2012). One important factor not addressedspecifically in the experiment was the subjects level of ex-pertise. The testing of expert users could produce quitedifferent results. There are very few potential users ofmost MR applications who could be considered experts,so the current results are useful, but an understanding ofhow performance is affected by greater experience withconventional tools would have an added value. Futurework will hopefully lead to the implementation of the MRsystem on real construction projects. The real improve-ments in performance and productivity with MR can bethen measured and quantified with site assembly activitiesin a real project context. The transfer of the animated MRsystem from laboratory-based applications to real con-struction applications has higher potential for systemflexibility and tracking, e.g. to enable assembly in a limitedway on construction sites. Using portable MR devicessuch as wireless HMD or cameras will enable more stabletracking (images will not be lost when occluding the pathbetween tracking targets and camera). Where the trackingtargets (markers) are not able to be posted, markerlesstracking techniques such as tracking using the salientgeometric features of real spatial objects might be adopted.Since tracking is one of the most essential elements indetermining whether the MR system is effective or not inreal projects, future work should focus on the integrationof the current tracking technologies and develop morerobust tracking methods.

Competing interestsBoth authors declare that they have no competing interests.

Authors contributionsXW prototyped the tangible mixed reality system package, designed theexperiments and evaluation methods, and carried out the experiments. PDco-authored the system package, and co-designed the experiment. Bothauthors read and approved the final manuscript.

AcknowledgementThis research has been funded by National Science Foundation grant CMS-0239091. Opinions, findings, conclusions, or recommendations presented inthis paper are those of the authors and do not necessarily reflect the viewsof the National Science Foundation. Models used in the experiment wereprovided by BMW Constructors, Inc. of Indianapolis, Indiana, U.S.A.

Author details1Australasian Joint Research Centre for Building Information Modelling,School of Built Environment, Curtin University, Perth, Australia. 2PurdueUniversity, School of Civil Engineering, West Lafayette, India, USA.Hou, J, Su, C, Zhu, L, & Wang, W. (2008). Integration of the CAD/PDM/ERP systembased on collaborative design. Paper presented at the Computing,Communication, Control, and Management, 2008. CCCM'08. ISECSReceived: 7 February 2013 Accepted: 24 June 2013Published: 13 August 2013

ReferencesAhlers, KH, Kramer, A, Breen, DE, Chevalier, PY, Crampton, C, Rose, E, & Greer, D.

(1995). Distributed augmented reality for collaborative design applications.Paper presented at the Computer Graphics Forum, The EurographicsAssociation and John Wiley & Sons Ltd.

Barakonyi, I, Fahmy, T, & Schmalstieg, D. (2004). Remote collaboration usingaugmented reality videoconferencing (Paper presented at the Proceedings ofGraphics interface 2004, pp. 8689). Waterloo, Ontario, Canada: CanadianHuman-Computer Communications Society School of Computer Science,University of Waterloo.

Billinghurst, M, & Kato, H. (1999). Collaborative Mixed Reality (Paper presented atMixed Reality Merging Physical world and Virtual World, proceedings ofthe International Symposium on Mixed Reality (MR99)). Yokohama, Japan.

Bochenek, GM, Ragusa, JM, & Malone, LC. (2001). Integrating virtual 3-D displaysystems into product design reviews: some insights from empirical testing.International Journal of Technology Management, 21(3), 340352. IndersciencePublishers.

Bowman, DA, Johnson, DB, & Hodges, LF. (1999). Testbed Evaluation of virtualenvironment interaction techniques (Paper presented at the Proceedings ofthe ACM symposium on Virtual reality software and technology, pp. 5975).New York, NY, USA: ACM.

Burdea, G, & Coiffet, P. (2003). Virtual Reality Technology. Presence: Teleoperators& Virtual Environments, 12(6), 66364.

Chen, M. (2001). Design of a virtual auditorium (Paper presented at theProceedings of the ninth ACM international conference on Multimedia,pp. 1928). New York, NY, USA: ACM.

Chryssolouris, G, Mavrikios, D, Pappas, M, Xanthakis, E, & Smparounis, K. (2009). Aweb and virtual reality-based platform for collaborative product review andcustomisation. In Collaborative design and planning for digital manufacturing(pp. 137152). Springer.

Cowell, A, May, R, & Cramer, N. (2004). The Human-Information Workspace(Hi-Space): Ambient Table Top Entertainment (pp. 13361). EntertainmentComputingICEC 2004.

Dias, M, Jorge, J, Carvalho, J, Santos, P, & Luzio, J. (2003). Usability Evaluation ofTangible User Interfaces for Augmented Reality. Paper presented at theAugmented Reality Toolkit Workshop, 2003 (pp. 5461). IEEE International.

DiGiovanni, C, Bowen, N, Ginsberg, M, & Giles, G. (2005). Quarantine StressingVoluntary Compliance. Emerging Infectious Diseases, 11(11), 1778.

Dong, S, & Kamat, VR. (2013). SMART: scalable and modular augmented realitytemplate for rapid development of engineering visualization applications.Visualization in Engineering, 1(1), 1.

Dunston, PS, & Wang, X. (2005). Mixed Reality-based Visualization Interfaces forthe AEC Industry. Journal of Construction Engineering and Management,American Society of Civil Engineers (ASCE), 131(12), 13011309.

Escorpizo, R, Reneman, MF, Ekholm, J, Fritz, J, Krupa, T, Marnetoft, S-U, & Tucki, G.(2011). A conceptual definition of vocational rehabilitation based on the ICF:building a shared global model. Journal of occupational rehabilitation, 21(2),126133.

Gabbard, JL, Hix, D, & Swan, JE. (1999). User-Centered Design and Evaluation ofVirtual Environments. Computer Graphics and Applications, IEEE, 19(6), 210.

Greenberg, S, & Fitchett, C. (2001). Phidgets: easy development of physicalinterfaces through physical widgets (Symposium on User Interface Softwareand Technology, pp. 209218).

Hinckley, K, Tullio, J, Pausch, R, Proffitt, D, & Kassell, N. (1997). Usability analysis of3d rotation techniques (Paper presented at the Proceedings of the 10thannual ACM symposium on User interface software and technology). NewYork, NY, USA: ACM.

Hix, D, Swan, JE, Gabbard, JL, McGee, M, Durbin, J, & King, T. (1999).User-Centered Design and Evaluation of a Real-Time Battlefield VisualizationVirtual Environment. Paper presented at the Virtual Reality, 1999. ProceedingsInternational Colloquium.Hren, G, & Jezernik, A. (2009). A framework for collaborative product review. The

International Journal of Advanced Manufacturing Technology, 42(7-8), 822830.

387396. International Council for Research and Innovation in Building andConstruction (CIB), Rotterdam, Netherlands.

Wigdor, D, Perm, G, Ryall, K, Esenther, A, & Shen, C. (2007). Living with a tabletop:Analysis and observations of long term office use of a multi-touch table(Paper presented at the Horizontal Interactive Human-Computer Systems,2007. TABLETOP'07, pp. 6067). Newport, RI.

Wu, D, Zhen, X, Fan, X, Hu, Y, & Zhu, H. (2012). A virtual environment for complexproducts collaborative assembly operation simulation. Journal of IntelligentManufacturing, 23(3), 821833.

Zhen, X-J, Wu, D-L, Hu, Y, & Fan, X-M. (2010). A real-time simulation grid forcollaborative virtual assembly of complex products. International Journal ofComputer Integrated Manufacturing, 23(6), 500514.

doi:10.1186/2213-7459-1-8Cite this article as: Wang and Dunston: Tangible mixed reality forremote design review: a study understanding user perception andacceptance. Visualization in Engineering 2013 1:8.

Wang and Dunston Visualization in Engineering 2013, 1:8 Page 15 of 15http://www.viejournal.com/content/1/1/8Khan, S, Peng, Y, Steinbach, E, Sgroi, M, & Kellerer, W. (2006). Application-drivencross-layer optimization for video streaming over wireless networks.Communications Magazine, IEEE, 44(1), 122130.

Hu, Y, Zhou, X, & Li, C. (2010). Internet-based intelligent service-oriented systemarchitecture for collaborative product development. International Journal ofComputer Integrated Manufacturing, 23(2), 113125.

Huang, C-H, Hsiung, P-A, & Shen, J-S. (2010). UML-based hardware/softwareco-design platform for dynamically partially reconfigurable network securitysystems. Journal of Systems Architecture, 56(2), 88102.

Khandelwal, M, & Mazalek, A. (2007). Teaching table: a tangible mentor for pre-kmath education (Paper presented at the Tangible and embedded interaction:Proceedings of the 1 st international conference on Tangible and embeddedinteraction, pp. 191194). New York, NY, USA: ACM.

Lan, H. (2009). Web-based rapid prototyping and manufacturing systems: areview. Computers in Industry, 60(9), 643656.

Liang, JS, Chao, K-M, & Ivey, P. (2013). VR-based wheeled mobile robot inapplication of remote real-time assembly. The International Journal ofAdvanced Manufacturing Technology, , 115.

Mavrikios, D, Alexopoulos, K, Xanthakis, V, Pappas, M, Smparounis, K, &Chryssolouris, G. (2011). A Web-Based Platform for Collaborative ProductDesign. Review and Evaluation Springer, City.

Merrick, K, Gu, N, & Wang, X. (2011). Case Studies Using Multiuser Virtual Worldsas an Innovative Platform for Collaborative Design. Journal of InformationTechnology in Construction, 16, 165188. International Council for Researchand Innovation in Building and Construction (CIB), Rotterdam, Netherlands.

Moeslund, TB, Stoerring, M, Broll, W, Aish, F, & Liu, Y. (2003). The ARTHUR system:an augmented round table. Computer, 1(1), 277282.

Nilsen, T, & Looser, J. (2005). Tankwar-Tabletop war gaming in augmented reality(Paper presented at the 2nd International Workshop on Pervasive GamingApplications, PerGames).

Orozco, C, Esteban, P, & Trefftz, H. (2006). Collaborative and DistributedAugmented Reality in Teaching Multi-Variate Calculus. Paper presented at theIASTED International Conference on Web based Education (WBE 2006).Puerto Vallarta, Mexico.

Regenbrecht, H, & Schubert, T. (2002). Measuring presence in augmented realityenvironments: design and a first test of a questionnaire (Paper presented at theProceedings of the Fifth Annual International Workshop Presence 2002, pp. 911).

Santos, P, Stork, A, Gierlinger, T, Pagani, A, Arajo, B, Jota, R, Bruno, L, et al. (2007).Improve: Collaborative Design Review in Mobile Mixed Reality. Virtual Reality,54353.

Schmalstieg, D, & Hesina, G. (2002). Distributed applications for collaborativeaugmented reality (Paper presented at the Virtual Reality, pp. 5966). VirtualReality, 2002. Proceedings. IEEE.

Shu, L, & Flowers, W. (1992). Groupware experiences in three-dimensionalcomputer-aided design (Paper presented at the Proceedings of the 1992 ACMconference on Computer-supported cooperative work, pp. 179186).New York, NY, USA: ACM.

Sidharta, R, Oliver, J, & Sannier, A. (2006). Augmented reality tangible interface fordistributed design review (Paper presented at the Computer Graphics, Imagingand Visualisation, pp. 494470). Sydney, Qld.

Singh, V, Gu, N, & Wang, X. (2011). A Theoretical Framework of a BIM-basedMulti-disciplinary Collaboration Platform. Automation in Construction, 20(2),134144.

Tang, A, Owen, C, Biocca, F, & Mou, W. (2003). Comparative effectiveness ofaugmented reality in object assembly (Paper presented at the Proceedings ofthe SIGCHI conference on Human factors in computing systems, pp. 7380).New York, NY, USA: ACM.

Tuikka, T, & Salmela, M. (1999). Webshaman: Shared Virtual Prototypes for ProductDesigners in the World Wide Web. ACM SIGGROUP Bulletin, 20(1), 5155.

Ullmer, B, Ishii, H, & Glas, D. (1998). mediaBlocks: physical containers, transports,and controls for online media (Paper presented at the Proceedings of the25th annual conference on Computer graphics and interactive techniques,pp. 37938). New York, NY, USA: ACM.

Verlinden, JC, De Smit, A, Peeters, AWJ, & Van Gelderen, MH. (2003). Development ofa Flexible Augmented Prototyping System. Journal of WSCG, 11(1), 496503.

Verlinden, J, & Horvth, I. (2009). Analyzing Opportunities for Using InteractiveAugmented Prototyping in Design Practice. AI EDAM (Artificial Intelligence forEngineering Design, Analysis and Manufacturing), 23(3), 289.Wang, L, Shen, W, Xie, H, Neelamkavil, J, & Pardasani, A. (2002). Collaborativeconceptual designstate of the art and future trends. Computer-AidedDesign, 34(13), 981996.Submit your manuscript to a journal and bene t from:

7 Convenient online submission7 Rigorous peer review7 Immediate publication on acceptance7 Open access: articles freely available online7 High visibility within the eld7 Retaining the copyright to your articleWang, X. (2005). Applications of Mixed Reality in Architecture, Engineering, andConstruction| Specification, Prototype, and Evaluation. PURDUE UNIVERSITY.

Wang, X, & Dunston, PS. (2006). Compatibility Issues in Augmented RealitySystems for Aec: An Experimental Prototype Study. Automation inconstruction, 15(3), 314326.

Wang, X, & Dunston, PS. (2008). User Perspectives on Mixed Reality TabletopVisualization for Face-to-Face Collaborative Design Review. Automation inconstruction, 17(4), 399412.

Wang, X, Chen, J, & Zhou, H. (2009). Experimentally Exploring the Benefits ofAugmented Reality Technology in Design Conceptualisation. InternationalJournal of Design Engineering, 1(3), 278299. Inderscience Publisher.

Wang, X, Love, P, Klinc, R, Kim, M, & Davis, P. (2012). Integration of E learning2.0 with Web 2.0. Journal of Information Technology in Construction, 17, Submit your next manuscript at 7 springeropen.com

AbstractBackgroundMethodsResultsConclusion

BackgroundMethods: System Prototyping and Experimental EvaluationPrototyping the tangible mixed reality-based distributed virtual environmentsTangibilityView windows

Experiment setupComparison benchmark: vommercial3D design review toolHypothesesSubjects, tasks, materials and methodTreatmentsExperiment process

Development of questionnairesResults and DiscussionQuestionnaires

Results of questionnaire 2Results of questionnaire 3ConclusionCompeting interestsAuthors contributionsAcknowledgementAuthor detailsReferences

/ColorImageDict > /JPEG2000ColorACSImageDict > /JPEG2000ColorImageDict > /AntiAliasGrayImages false /CropGrayImages true /GrayImageMinResolution 300 /GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 300 /GrayImageDepth -1 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages true /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict > /GrayImageDict > /JPEG2000GrayACSImageDict > /JPEG2000GrayImageDict > /AntiAliasMonoImages false /CropMonoImages true /MonoImageMinResolution 1200 /MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict > /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile (None) /PDFXOutputConditionIdentifier () /PDFXOutputCondition () /PDFXRegistryName () /PDFXTrapped /False

/CreateJDFFile false /Description > /Namespace [ (Adobe) (Common) (1.0) ] /OtherNamespaces [ > /FormElements false /GenerateStructure true /IncludeBookmarks false /IncludeHyperlinks false /IncludeInteractive false /IncludeLayers false /IncludeProfiles true /MultimediaHandling /UseObjectSettings /Namespace [ (Adobe) (CreativeSuite) (2.0) ] /PDFXOutputIntentProfileSelector /NA /PreserveEditing true /UntaggedCMYKHandling /LeaveUntagged /UntaggedRGBHandling /LeaveUntagged /UseDocumentBleed false >> ]>> setdistillerparams> setpagedevice