FAVE Framework Architecture for Virtual Environments Use case; visualisation of online CFD simulations

Daniel Patel, Inge Kristian Eliassen, Tor Langeland

Christian Michelsen Research P.O. Box 6031 Postterminalen

N-5892 Bergen, Norway {daniel, ingekr, tor}@cmr.no

We present a highly configurable framework for virtual environments. The framework consists of a toolbox of classes which are instantiated into objects as dictated by our own XML specification language. The result is that application behavior is specified with XML at runtime instead of in source code at compile time. This enables development by nonprogrammers familiar with the XML specification language and facilitates rapid prototyping. The process of creating a GUI is automated through use of XML transformations. The GUI designer need only create an XML data document containing the values to be presented in the GUI and an XML document describing a mapping from each data element to its GUI representation. This creates a clear-cut separation between the GUI and the application logic where the GUI communicates with the application logic through updating the data document. These mechanisms have been used to create an application for visualisation of online CFD simulations. Keywords: Configurable framework, rapid prototyping, XML related technologies, visualisation of online CFD-calculations, Virtual Reality.

1 The FAVE framework Immersive Virtual Reality (VR) is an aid to mastering spatial complexities by facilitating improved understanding of 3D data and offering new modes of interdisciplinary collaboration. Immersive VR is gaining usage within a range of application areas, e.g. design, medicine and oil and gas exploration and production. A typical hardware configuration for an immersive VR application consists of backprojected walls, glasses for stereoscopic viewing, a mouse with 6 degrees of freedom (DOF) and a 6DOF sensor mounted on the glasses. CMR has developed a framework called FAVE [1] for creating VR applications. The FAVE framework is written in C++ and runs on an SGI ONYX 3200 under the IRIX operating system. The framework is built around a certain type of objects called controllers. They have in common their inheritance from the controller class. These objects all offer some kind of high level functionality in the VR domain, and not low level algorithmic or datastructure objects. They usually represent some graphical part of VR and can for instance implement basic GUI building blocks like buttons, menus and sliders (widgets), functionality for navigating around in the 3D world or specific visualisation techniques to present data.

1

.

Figure 1 A possible configuration for a CAVE. Picture from http://eve.hut.fi/

1.1 The controller building blocks FAVE consists of a collection of different controller classes. This collection can be looked upon as a toolbox for creating a VR application. What is special about the framework is that the classes in the toolbox are not combined with each other in ordinary C++ fashion using aggregation and inheritance; i.e. creating new classes with controllers as members or by overloading their functions. In fact you don't need to know any programming language to create an application in terms of the objects in the toolbox. We have developed our own simple language in XML to express which controllers to be instantiated and how they will communicate with each other. You don’t need to look at the code in the controllers to use them. A controller’s core functionality is to offer functions for other controllers to call and to create events, typically after a user interaction on the controller, for instance clicking on a button controller.

2

PresentationManager Function Event LoadScenario(string)UnloadScenario(void)StartAnim(void)StopAnim(void)SetSpeed(float)SetTime(float)StepForward(void)StepBackward(void)StartSim(void)PauseSim(void)StopSim(void)…..

none

Button Function Event

none Pressed (void)

ToggleButton Function Event

none Toggled (bool)

ArrowSlider Function Event

SetVal(float) NewVal (float)

MovableObject Function Event

SetPos(3*float) MovedTo(3*float)

TextWindow Function Event SetText(string) SetTextSize(float) SetHeader(string)

none

Interpolator Function Event

SetValue(float) none

Environment Function Event ExitApp(void) SetSkyColor(3*float) SetCulling(bool) SetShading(bool)

none

Light Function Event SetOn(bool) SetDist(float) SetAzimuth(float) SetElevation(float)

none

VolumeRenderer Function Event SetVar(string) SetPlaneCount(int) SetTransparency(float) SetLut(string)

none

3

Figure 2 Some controllers from the toolbox with their functions and events. Some controllers have no functions and can only be used to send data to functions of other controllers. Others have no events and can only offer their functions to other controllers.

Controller objects can be compared with Microsoft Visual Basic's ActiveX components. In Visual Basic (VB) a typical programmer may drag some widgets onto a form, fill out their attributes and fill out functions on widgets that create events. These functions will then typically call functions on other VB objects, this might include objects that were dragged onto the form.

Figure 3 A VB Button. From left; its function, its attributes and the toolbox it was dragged from.

1.2 XML configuration Creating an application with FAVE can be compared with the VB approach described above. However, instead of instantiating an object by dragging it onto a sheet, the user instantiates a controller by writing an XML section in a document in which the controller’s type and name is specified followed by a set of attributes. A controller type has one set of attributes that is common for all controllers, and one part specific to configure that particular controller type. The common attributes are

• The controller type. • The controller’s name. • Its appearance and position specified by referring to a VRML file. • Its tooltip-text. • If it should be highlighted when pointing at it; so that objects that can be interacted with

can be distinguished from static 'environment' objects. • Context menu specified by referring to another controller, for instance a menu controller. • State symbol; a VRML file defining the symbol shown on the wand when the controller has been selected. • Subscription list; a mapping from other controllers' events to functions in this controller.

VB objects associate a function to every event the object creates. For instance a button object associates a function to the event created when the button is pressed. FAVE has a more restrictive approach to this; an event can not be associated with a function that the user can define ‘on the fly’ as in VB, an event can only be associated with an already implemented function in an instantiated controller. This is specified in the subscription-list attribute.

4

The subscription-list attribute specifies which events in other controllers will result in a function call in this controller. If the event contains arguments, the arguments will be input to the function. This is specified in a list where each element consists of the name of a controller that sends an event and the name of a function in this controller that wants the event (if the sending controller sends more than one event, the event name must also be specified). We then say that the function in this controller listens to the sending controller. If a function in a controller needs to listen to many different controllers it is not necessary to list up all of them, one can instead specify in their attributes that they are part of a common group and specify that the function listens to that group. Visually, the subscription-list will define lines connecting outports from controllers with inports of other controllers if they were drawn as in Figure 2.

Figure 4 An XML setup. The NavigationController listens to (the event sent from) a walk and a fly button.

1.3 Creating new controller types In case the user wants an event to call a function that is not already implemented in a controller, a new controller must be implemented with that function, or an existing controller must be extended with that function. This might seem awkward, but it forces the programmer to think object oriented. The easiest thing would be to extend an existing controller, so the programmer first checks if the new function naturally belongs in any of the existing controllers, if not a new controller must be created. Doing this is easy. By inheriting from the controller class all the behaviour from the

5

common arguments will be inherited, the programmer need only implement the new function.

1.4 The XML - C++ separation. This design results in that classes are implemented in C++, but instantiated, configured and connected with each other in an easy to use XML language. There is a clear separation between implementing the building blocks and configuring and connecting them with each other. The latter can be performed by anyone familiar with the XML language and the interface of the controllers, and it does not require recompilation after changing the configuration. Although the XML language has a restricted expressability compared to C++, it offers surprisingly good control over the system’s behaviour and almost full control over configuring the GUI. The number of controller types in the toolbox and the expressiveness in the controllers’ attributes will of course influence the configurability of the system. Alternatively to set this up in C++ would require more lines of code and create a larger and more unmanageable project with more classes than when using XML, and confusion might arise when different programmers do this in slightly different ways and in different parts of the code. Since the configuration is written in XML, we can make use of different XML related technologies. We have created XML schemas describing each controller’s XML syntax resulting in a verification of the XML configuration files thus eliminating the need for the programmer to create C++ code to perform structural checking of the XML files being parsed. In addition we made an XML transformation converting the XML configuration file to a HTML file readable by people not familiar with the XML syntax and an XML transformation to a graphical language [2] (the dot language in AT&T’s Graphviz software) visualising the connections between the controllers. Finally, the configuration files can be created using visual XML editors that present an editable GUI document based on the XML schema for the configuration language.

Figure 5 A graphical representation of controllers connected with each other made by an XML tranformation into the dot language in AT&T’s Graphviz open source software for visualising graphs.

6

2 FAVE used in a VR application for improved safety and risk communication The project to be presented in this section was initiated by Norsk Hydro and Statoil as an attempt to better communicate safety risks and to create a training simulator for gas explosions and fire drills. Two CFD (Computational Fluid Dynamics) simulators are used by these companies and others in the oil industry during construction and planning of structural modules on platforms to simulate how the structures will react on explosions and fire situations. The FLACS simulator [3] developed at GexCon, a subsidiary of CMR, is specialized on simulating explosions, Kameleon [4] developed at ComputIT is tailored for fire simulations. Both these simulators are being used in this project. A common interface was implemented for both of these codes for communicating with a VR environment.

2.1 The simulator interface This section described the common interface of the simulators.

The simulator input A bit simplified one can say that input to the simulators is a 3D geometry describing the structure in which the simulation takes place and a scenario description describing additional parameters such as dynamic items in the structure which may change during the simulation. Some of the items specified in the scenario are

• Placement and size of doors and their opening factors. • Placement and size of pressure relief panels that will fall out at a specified pressure. • Placement, direction, flow rate and gas components of gas leakage points. • Placement and ignition-time for ignition points.

The scenario description is specified in XML.

Figure 6 A simplified scenario file. This file describes the position, size and opening of a door and the position, flow rate, temperature and direction of a gas leakage. Typically a scenario file will contain more items, and also more elements and attributes for each item.

<scenario> <DOOR itemNumber="1"> <POSITION x="0" y="0.0002" z="0"/> <SIZE x="0" y="12.5005" z="6.25"/> <OPENING>0</OPENING> </DOOR> <LEAK itemNumber="1" > <POSITION x="3" y="5.25" z="9"/> <OUTLET> <MASSFLOW>0</MASSFLOW> <VELOCITY> 0 </VELOCITY> <TEMPERATURE>20</TEMPERATURE> <DIRECTION_COSINES x="1" y="0" z="0"/> </OUTLET> </LEAK> </scenario>

7

The simulator output Without going into too many details, one can roughly say that a simulation is initiated by invoking the simulator program with the geometry and the scenario as parameters. The simulator will then start calculating results for discrete timesteps. Each timestep consists of a 3D grid surrounding the geometry with values in each gridpoint. The values in a gridpoint can describe physical measures such as temperature, fuel and pressure among others. The timesteps are written to a file. Timesteps are typically calculated about every second second, the time it has calculated for is varying, from 0.01 to 1 second. It has been discussed to parallelize the simulator codes so they can give results closer to realtime thereby making it possible for realistic emergency and fire drills in VR.

Changing the input parameters while the simulator is running Attributes of the scenario items described above can be changed while the simulator is running, like for instance increasing the leak rate or opening a door after the gas has ignited. The scenario is changed by sending update messages to the simulator. An update message consists of a path to the attribute in the XML document to be changed and its new value.

2.2 The VR environment The VR environment contains control for initiating a simulation, updating it while it is running and visualising the results in a realistic and informative manner. The VR application is built using the FAVE framework. The VR application must support the following functionality for it to interface with the simulators.

• Start, stop and pause the simulator. • Present the scenario items in a GUI so they can be interacted with. • Translate the GUI interactions to messages to the simulator. • Visualise results from the simulation.

These issues will be discussed in the next sections.

Automatic generation of the scenario GUI for the VR environmenty The scenario XML description that is input to the simulator must also be visualised in VR. Different simulations require different scenario files. Therefore the scenario elements can not be hardcoded into the VR application, but must dynamically be generated from the scenario file. The straightforward way to do this would be to express the different scenario items in terms of C++ classes (a class for Door, Leak, Ignition and so on) for instance with constructors that take the arguments of the scenario items and create a graphical object that can be interacted with and that encapsulates the communication with the simulation code. We selected a different approach much inspired by the FAVE XML language. We realized that all the scenario items could be created by combining existing controllers with each other. These controllers can be described in FAVE XML. Therefore all that would be needed was an XML transformation from the scenario XML file to the FAVE XML language. This approach has some

8

advantages over the first alternative; •We are creating the GUI for the scenario items by using the controller building blocks. •The blocks are assembled into ‘aggregate objects’ using XML, not by creating new C++ classes. •The blocks are created at runtime, changing their looks doesn’t require recompilation. •To add new or change existing scenario items is done by changing the transformation.

Figure 7 The controllers and configuration needed to represent a pressure relief panel. Gray ellipses represent controllers. The two arrows from the menu controller point to the two controllers the menu consists of. Dashed boxes represent events. The figures below controller mp, sl and dd represent theirgraphical appearance. This setup can be described in terms of FAVE XML.

9

Figure 8 The yellow wall represents the pressure relief panel in VR. Its menu contains 4 arrowsliders.

Changing the transformation The XML transformation transforming the scenario XML file to FAVE XML describes how each scenario item will end up looking like in VR. If one wants to tweak or change the look one needs to edit the transformation file. This requires knowledge of the XML transformation language and of the structure of that particular transformation. To make it easier to configure how scenario items will look like in VR it was decided to generate the transformation file from a more understandable high level XML description of the representation of the items. Thus a new transformation was created that took the high level description and transformed it into a transformation. This is then finally applied to the scenario XML file resulting in FAVE XML.

10

Figure 9 Scenario.xml is the scenario description, gui.xml describes the mapping from xml attributes in the Scenario.xml file to widgets. The result is a file FAVE.xml with the description of controllers that represent a GUI of the scenario file. The xslt squares represents an xml transformation process.

The result of this approach is that nonprogrammers can decide how scenario items will look like and that changing the high level description is easy and requires no recompilation (only retransformation).

Communication of scenario changes from VR to simulation The missing part is now the ability of the controllers to communicate their values to the simulator. Editing the scenario objects in VR must end up with the simulators scenario being updated with the same values. We designed a simple protocol such that change messages are sent to the simulator as (XPath,value) pairs. The XPath refers to some element or attribute in the scenario document and the value describes the item’s new value. Editing a scenario object in VR must thus result in such a pair being created and sent to the simulator. This is handled by letting each value-holding controller store in itself an XPath describing what part of the document its value describes. A global controller listens to all value-holding controllers and sends their update messages to the simulation through a network object. To make this possible the Controller class was extended with a new attribute storing the XPath(s) that a controller’s value(s) logically refers to. Finally the transformations make sure that controller XML definition have their XPath attributes equal to the attribute or element in the scenario XML file they describe.

The GUI – data separation By generalizing this process one can create a workflow where the GUI designer only needs to create an XML data document containing the underlying values for the GUI and an XML document describing a mapping from each data element to its widget representation. The objects that contain the application logic which process the values in the widgets will receive the widget data through (XPath,value) pairs, i.e. in terms of the XML data document. This creates a clear separation between the GUI and the application logic where the GUI communicates with the application logic through updating the data document.

Visualising simulator results in VR One of the main objectives for this project was to present the simulation results in VR. The data

11

from the simulators is ultimately visualised by one of two controllers; the isosurface visualiser or the direct volume rendering visualiser. They use different graphical techniques to visualise the data. The isosurface visualiser creates an opaque surface through all points that have a certain user specified value for some physical measure. The volume renderer creates a semitransparent volume where all points on the grid are coloured according to their value for some physical measure. In addition to choosing between visualisation techniques, the user can choose whether the newest calculated timestep is to be shown, or if the program should loop through all calculated timesteps creating an animation at a user defined speed. If the simulators were calculating results in near realtime the system could be used as a training simulator with real time response on user actions such as reducing the leakage, opening or closing doors or using a fire distinguisher on a fire. Another method for performing training sessions, is to precalculate a set of scenarios corresponding to different possible user actions.

Figure 10 Outside view of a gas explosion with volume rendering of combustion products.

12

Figure 11 Internal view of a gas leakage. Both visualisation techniques shown.

3 Conclusions This paper presents FAVE and a VR application built upon it. The toolbox of controllers together with the XML specification language offers a strong degree of configurability even by nonprogrammers. A higher level of configurability has been achieved by creating XML transformations from an XML document describing the looks of scenario items into the XML specification language describing these items in terms of the controllers in the toolbox. An application communicating with CFD simulators and presenting their results has successfully been made with the FAVE framework. Links: [1] http://eve.hut.fi/cavews2003/favePerfAppPaper.pdf [2] http://www.research.att.com/sw/tools/graphviz/ [3] http://www.computit.no/ [4] http://www.gexcon.com/index.php?src=flacs/flacs.html

13