Advance Computer Graphics • Spring 2014

A Framework for Furniture Visualization in WebGL:A User Study Based Extension

Max Espinoza

Rensselaer Polytechnic Instituteespinm2@rpi.edu

Abstract

As an extension of current research done in architectural day-lighting simulation I provide a framework for the visual-ization furniture in user sketched spaced in WebGL for ease of use and accessibility. We also provide a naive furnitureoptimization using a simulated annealing approach. Using a texture output for lighting values of a given space, asimplified cost function, and a specified number of furniture items we find an approximate arrangement in an arbitraryshaped room. The framework provided, can be extended to use a Metropolis-Hasting state-search step to solve the optimalfurniture arrangement for light, and other ergonomic properties.

Above is shown the pipeline of the online interface. We start with a sketching interface where you can draw wallprimitives and then save them into the server, in which daylighting simulation is computed. A visualization is shownon middle image is the resulting scene with textures on a 3D model. On the right is the furniture visualization after

running our naive optimization for 20 iterations.

I. Background and Related Works

Research done on the Virtual Heliodon [4] offersa tangible user interface to create and build roomlayouts and display accurate illumination of natu-ral lighting back onto user created physical sketchesthrough virtual reality augmentation. The processof sketching spaces and displaying daylighting in-formation back on the scene is faster and come withless overhead then traditional methods. Motivation

behind this research is to offer architects the abilityto consider use of natural lighting earlier in their de-sign phase in order to save energy in lighting cost.Furthermore architects with tangible user interfacessuch as the Virtual Heliodon can collaborate andshare ideas within a common space and get inter-active feedback on changes to physical sketches.

User evaluations on the Virtual Heliodon [3] havebeen done and have offered us key insight into howthe tangible user interface is used, in addition to

1

Advance Computer Graphics • Spring 2014

possible extensions that would aid users in under-standing daylighting in interior spaces. The evalua-tions discovered that both novice and architects alikedidn’t have a firm grasps on natural lighting con-tribution and penetration into a room. The studyalso revealed that users had difficulties detectingwhere problem areas with glare would occur. Thislead to further research in glare visualization tokenswhich offered a solution this problem. Also the userstudied revealed that adding furniture into the Vir-tual Heliodon might further provided a better under-stand of problem areas caused by over illumination.Overall the benefit of the user study is that it giveuser feedback that can be used to refine and improveresearch done on this project.

While the previous study offered insight into fur-ther extensions and problems with the system, moreuser feedback is required to gain a better under-standing of what direction to take future research.In addition increased feedback offers confirmationthat solutions to previous problem have been suc-cessfully fixed. Informative feedback requires aplanned user study and scheduling participants tocome physically interact with the Virtual Heliodon.This limits our number of users who we can test ina given amount of time. However, some parts of thesystem can be studied in a wide spread manner byoffering a simplified online alternative that allowsthe creation and exploration of room layouts.

For this project our goal is to develop the frame-work and extension to expand our user studies toa much larger number of participants. We want tomake a easy to navigate and use online applicationthat will aid in our collection of user sketches andfeedback about daylighting in architectural roomdesigns. Borrowing from previous research donethroughout the semester on this online design tool,I extend it to offer furniture visualizations and anoptimization.

The project is an exploration into what can bedone using the online interface and daylighting in-formation generated by Virtual Heliodon. Just asARmy [1] was an exploration into the application ofthe physical system, this is an exploration of whatcan be done with the daylighting output generated

by the Virtual Heliodon.Also the incorporation of furniture and furniture

optimization was done as extra feature that wouldmotivate users to create and edit more wall mod-els based on results generated by key algorithms inthe Virtual Heliodon. Similar to Real-time DrawingAssistance Through Crowdsourcing [?] applicationdone by Limpaecher at the Mircosoft research group.An iPhone application was used to collect the mini-mal amount of lines needed to draw a recognizableface, by designing their data collection in the form ofa iPhone game they got a large number participantsto contribute data for research purposes.

Lastly, it was a learning experience and devia-tion from OpenGL used in class to what was possi-ble given current online libraries and tools in con-ductions with WebGL. It would give the researchteam an idea of continuing this simplified online ap-proach was worthwhile, and offer us an existing ex-tendible framework to continue development.

II. Methods

Our criteria for creation of the successful frameworkwas easy integration into the current online systemwe provided, extendability to add more features,and fast enough to run on modern hardware. Therewere other choices considered, and choices that re-sulted in bottlenecks that made the furniture visual-ization non-interactive.

I. WebGL

Given that we needed to reach a wider amount ofpeople for this user study there were alternatives toa website considered for development of this furni-ture optimization. Java was considered for it abil-ity to use OpenGL and portability, as well as C++which could have much easily been extended sincea stable current framework for OpenGL was pro-vided for us in homework assignments. Howeverboth of these approaches require additional installa-tion and would discourage users from participatingin the user study. Despite choices that offer betterperformance , WebGL offered a solution that works

2

Advance Computer Graphics • Spring 2014

in modern web browsers. WebGL can make use ofGPUs specialization for rending computer graphics.Modern browsers like Firefox and Google Chromehave become ubiquitous giving us a much wider au-dience who can participate. In addition, librariesmade to simplified usage of WebGL such as Three[2] can increase the speed of development of fea-tures to motivate users to participate. Also WebGLlike OpenGL allow, can be used to write shader pro-grams that can increase performance. Lastly, beingonline hosted our website would allow the creationof a database to store all user created floor designsand feedback. There were downsides to working inWebGL however, it generally did not perform withthe same frame rates as OpenGL, our unfamiliaritywith web development and WebGL, and large bot-tlenecks in JavaScript programing that needed eitheroptimization or moved server-side.

II. Full WebGL and Javascript Based Ap-poach

The choice to begun development entirely inJavascript and WebGL was based on visualizationdemos that used the Three JavaScript library. WhileThree [2] offered a great set of tools to set up an ini-tial scene, including a room, camera, and ambientlighting, it failed to fully support the set of featureswithin it’s documentation. Three was a tool for basicscenes but extending it handle much more complexinteractions proved to be a difficult due to lack ofdocumentation.

For example a lot of effort was spent on gettingaverage texture from triangles underneath each deskin the scene. Three library had consistently missingdocumentation, and functions that were not imple-mented entirely that caused issues with this WebGLand Javscript based approach. Using Three I man-aged to raycast from the center of each desk itemdownwards into the triangles beneath it. HoweverI quickly ran into a problem with how Three han-dles meshes loaded in from OBJ files. It offered noway to get the textures pixels or iterate through themwithin WebGL. Instead I got the UV texture coordi-nates and converted it absolute texture coordinates,

created a separate html canvas used that to querypixels in the floor texture.

There was a massive draw back to this approach.Despite running with over 60 fps on high end per-formance computers, it wouldn’t run smoothly on amodern laptops, such as mid-range performance ma-chines such as Thinkpad laptops. This approach asthen discarded since it would be a barrier for entryfor participants in the user study. Code in Javascriptproved to be costly in that it had to run all calcu-lations on user web browsers which created a largebottleneck.

III. Server Side and Texture Based Ap-proach

An alternative considered afterwords was a serverside based approach in that we would use a separateprogram on the server to run the optimization anddecide where furniture should go by absolute po-sition relative to the floor texture. Communicationbetween these offline scripts and our visualizationin WebGL would occur via state files, which savethe position and angle of each item in the scene.Server-side these scripts can create state files thatwill update the online visualization as to where eachdesk at specific iterations of an optimization func-tion. Just as in INSERT PAPER TITLE HERE we canreduce the optimization into two dimensional prob-lem, since items are assumed to be supported by theroom’s floor. Since we are given a room’s floor tex-ture in which indoor spaces with lighting are repre-sented with white pixels of varying brightness, andblack for spaces with no light and spaces outsidethe room are shown by pure black pixels on the tex-ture. The texture being a 2D representation of theroom was chosen over the 3D representation in thattexture look up is fast and provided us the lightingvalues we could use for finding the interior spacesand where to place our furniture intuitively.

On the other hand using only the floor textureresulted in problems finding a furniture angles rela-tive to the arbitrary space. Since in the texture rep-resentation we loose the precise points a wall beginsand end, it makes finding the angle difficult through

3

Advance Computer Graphics • Spring 2014

standard graphics methods. In addition the textureitself is limited in the 512 x 512 pixels used to rep-resent the floor. This leads to sampling issues if Iwere to use 2D ray cast to find values needed for theoptimization and cost function. While in the paperwe base this project on they use basic square rooms,we are given user created room of arbitrary shapethat can support dividing walls and multiple roomdesigns with use of user created wall files.

Despite setbacks the server-side approach willprobably be continued and extended such that theonline visualization of furniture arrangements ismore then just a visualization offering much more in-formation and manual editing for furniture arrange-ment.

IV. Framework

The WebGL and server-side framework worksthrough state files, in which we use to store rele-vant information that needs to be shared betweenserver-side scripts and the online visualization. Cur-rently the framework only supports desk, but find-ing and loading other OBJ models to work is straightforward and can be easily extended. Manual inter-vention however is required for the optimization, itis important we know which side of an furnitureitem’s bounding box is considered its backside. TheBackside is defined as the side of each furniture itemopposite of where person would sit or use that item.Like stated previously our current framework savesthe furniture item number requested from the onlinevisualization to state file, that an offline script readsand generated several output state files. Each repre-senting a frame / iteration of a simulated annealingmethod to find lighting maxima for a specific num-ber of desk.

Left: Bounding boxes that define accessible space. Right:Boxes that define where an item is viewed from.

V. Optimization

Our optimization was based on the simulated an-nealing algorithm, presented in Make it Home [5]paper, to find best use of lighting in a given space.Plans were made to take into consideration muchmore then just lighting conditions through a costfunction that penalized unergonomic furniture ar-rangements. I planed to use some of the ergonomicvalues in presented in the Make it Home paper suchas accessible space and viewing frustum. The algo-rithm considers items based on their position pi andtheir Θi which represents the angle between a rayfrom back side of an object and ray to nearest wall.The ability to get ray to nearest well proved to bedifficult using only textures and hindered by abilityto implement the their cost function, however withuse of other data provided in the system this anglecan be found.

Accessible space is defined by four 2D boxesaround an object that define the space needed tonavigate around that object. For example in the fig-ure provided accessible space around a sofa wouldbe need to wide enough for a person to fit throughwithout trouble. The cost function for the accessiblespace is defined below. It punishes items that enterthe accessible space of another object. If object i over-laps the accessible space boxes k of object j the costis calculated as bellow. Where bi is the diagonal ofthe bounding box around item i.

Ca(Θ) = ∑i ∑j ∑k max[0, 1 − ||pi−ajk ||

bi+adjk

]Another element from the Make it Home paper

we wanted to implement was the viewing frustum.It is a series of rectangles defined in front of an ob-ject that are there to define what space shouldn’t betaken up by another object. Like accessibility how-ever with viewing in mind. Such as a monitor hasa viewing frustum directly in front of it and shouldbe taken up by another desk. It cost is also calcu-lated identically, however each triangle that makes

4

Advance Computer Graphics • Spring 2014

up the viewing frustum would have decreasing over-all penalty, for example items really close with in thesmallest viewing frustum bound would be penalizedat more then those that intersect the outer viewingbounds.

Cv(Θ) = ∑i ∑j ∑k max[0, 1 − ||pi−vjk ||

bi+vdjk

]

This is a floor texture generated by algorithms in theVirtual Heliodon. We use it as a texture to find interior

and exterior spaces.

Daylighting was the most important element Iwanted to incorporate into the cost function and wasimplemented for the live demo. To make the bestuse of natural lighting within a given space I ap-proximated light received by a surface as the areaof their bounding box in 2D. While this doesn’t takeinto account occlusion between objects, it wouldn’teffect low laying furniture such as desks, that aretypically low to the ground standing only a fewfeet high assuming desk didn’t cluster too closely.I wanted to penalize any desk that were laying incomplete darkness by a value large enough valuethat would discourage any solution that consideredsuch an arrangement. This was a naive greedy so-lution for handle daylighting and is intended to re-placed with a more sophisticated cost function thatwill take into consideration more then just boundingbox of an item and sum of pixels underneath.

The overall cost function is computed as a sumof these ergonomic other cost function, with weightsmultiplied to choose which of these properties aremore important. It is important to note that myimplementing only takes into consideration lightingand a bounding box cost that incorporates accessi-bility by making a bounding box larger on either thesitting side of the desk and side opposite of that oneconsidered the backside.

The solution was also implemented as describedby the Make it Home paper, the simulated annealingwas used to move furniture items between arounda given room. This can be best described as mov-ing molecules in a heated medium. As that mediumcools the amount the particles move is reduced un-til the medium loses all heat. In this case we simu-late heat by a heat factor, which is set low for ourimplementation. We then simulate particles movingas the desk moving around within the given space,as long as the desk movement doesn’t push it out-side the interior region and the cost function of thenew arrangement is lower we move that desk. TheMake it Home paper describes this same methodwith the addition of the swaps, in which two itemsare swapped if lower cost is achieved, however be-cause we only have a single furniture item swapswouldn’t alter the results. We plan implementingswaps after we consider different furniture items.

VI. Challenges

As stated previously there are many challenges inthis exploration away from traditional OpenGL pro-gramming. With WebGL there is the additional bot-tleneck of JavaScript programming. Inefficient codecan cause large reductions in the frame rates of thevisualization. The current implementation has beenoptimized to run on the RPI Laptop under GoogleChrome and Firefox with reasonable frame rates.Previous attempts had such large bottlenecks thatmade it unusable to those without high end ma-chines.

Besides the challenge of optimization, there wasthe challenge of working the Three Javascript libraryin that its documentation as incomplete at times. Italso had functions defined that would only work forsimple demos and required round about mannersto get information about texture coordinates in myfirst attempt at getting lighting information from thescene. None of these challenges however cannot beovercome with future work and time.

III. Results

5

Advance Computer Graphics • Spring 2014

Above are results gathered by our optimization.Top row: Simple case testing if we can keep desk inside arbitrary made spaces. Iterations 0,1,5,19

Middle row: Adding a inner wall to test if desk collide with it during any iteration. Iterations 0,5,19Bottom row: More complex layout, with many walls, it can be seen desk can phase through walls Iterations 0,1,5,19

For this project I wanted to test if the current im-plementation of the framework would be able to first,be able to infer between indoor and outdoor spaceusing the output texture generated for the floor. Sec-ond be able handle interior walls, and motion be-tween walls of furniture objects. Lastly we wantedto test the implementation of simulated annealing tosee if we could find the light, and if our visualizationwould show correct sequences to the light source.

In the above image in the first row we show theresults of testing out ability to find interior and exte-rior spaces with furniture objects using texture lookups. We then discount any position that find a spe-cific threshold of black pixels underneath a furnitureitem. While other parts of the framework will re-quire reworking, the texture look up approach works

well and fast for this application.Above in the second row we make models that

have interior walls and see if there is collision be-tween the walls and furniture items placed inside.Because of our texture based look up and way thetextures are made in the Virtual Heliodon wall haveno floor texture defined underneath them, whichmeans that any desk that get moved into a wall hassome black pixels which pass the interior/exteriorthreshold and considers that item to be outside thegiven space. In thus making sure no furniture itemsget moved into walls.

In the last example in the third row we see thatwhen running the simulated annealing algorithmthat our furniture items can move between rooms,which can either be desirable if users have no prefer-

6

Advance Computer Graphics • Spring 2014

ence items populate specific rooms. Because of theway simulated annealing works, the initial heat vari-able allows items to make large movements, specif-ically 100 pixel movements in either the x or y di-rection of a 512x512 pixel floor texture. This widemovement allows desk to move between walls andsettle in room with the most light.

This is the angle that was difficult to calculate usingonly floor textures. Edge detection of walls wouldhave be implemented if only the floor texture wasconsidered. Fortunately we have other data that

contains a manner to collect this necessaryinformation.

This is the angle that was difficult to calculate using onlyfloor textures. Edge detection of walls would have beimplemented if only the floor texture was considered.

Fortunately we have other data that contains a mannerto collect this necessary information.

Due to the limitation of not being able to find theangle between the backside of each desk and near-est wall with texture look up alone, the cost func-tion and simulated annealing application will onlysolve for optimal light, and avoid self collisions. Fu-ture work is aimed at a full implementation usingthe framework provided from this project.

IV. Discussion

As explained before this project was three-fold asan exploration, motivation to get participants, andlearning experience. In the results shown, I con-cluded that exploration justifies further research intoa full implementation of this furniture arrangement.We have the framework and tools developed to takeinto consideration other items and any floor layout

generated by users from our previously made inter-face.

Secondly, I hypothesise that with more furnituremodels we can create an furniture arrangement ap-plication, in which we ask users to design roomsand quantity lighting within the room. This applica-tion can be created into a game, with an competitiveaspect that will further allow us to get more feedback and data to valid the algorithm behind infer-ring closed and open spaces, users lack of intuitionin architectural daylighting, and eventually the bene-fit of a quick and easy way to visualize lighting within an interior space.

Lastly, I can say this project was a learning ex-perience for me, I learned about web developmentwith WebGL and libraries available to help renderand load meshes into our user generated scene. Inaddition to JavaScript optimization and other webdevelopment concepts. More importantly I learnedhow I can use WebGL to accomplish my goal of cre-ating an effective user study that will be usable toanyone with a modern laptop.

V. Future Work

There is a lot of future work to be done, howevermost of the daunting work setting up the beginningsof this project is complete. Future work to be doneis on the full implementation of the furniture opti-mization, in addition to a user interface that offersmore visual information to the user regarding theoptimization happing, such as the heat function anddisplay of the current cost at each iteration. Moreintuitive controls are also needed, the controls madewere setup for convince rather then simplicity andintuitiveness.

References

[1] Andrew Dolce, Joshua Nasman, and BarbaraCutler. Army: A study of multi-user interactionin spatially augmented games. 2012.

[2] mrdoob. Three javascript library.

7

Advance Computer Graphics • Spring 2014

[3] Joshua D. Nasman and Barbara Cutler. Evalua-tion of a tangible interface for architectural day-lighting analysis. In Proceedings of the ACM SIG-GRAPH Symposium on Interactive 3D Graphics andGames, I3D ’12, pages 207–207, New York, NY,USA, 2012. ACM.

[4] Yu Sheng, Theodore C. Yapo, Christopher Young,and Barbara Cutler. Virtual heliodon: Spatially

augmented reality for architectural daylightingdesign. In VR, pages 63–70, 2009.

[5] Lap-Fai Yu, Sai-Kit Yeung, Chi-Keung Tang,Demetri Terzopoulos, Tony F. Chan, and Stan-ley J. Osher. Make it home: Automatic opti-mization of furniture arrangement. ACM Trans.Graph., 30(4):86:1–86:12, July 2011.

8