Creating User Interface for Interactive Simulations

Jeetinder SinghCenter for IT in Education

International Institute of Information Technology,Hyderabad, A.P, India, 500032

Email: jeetinder@research.iiit.ac.in

Jayanthi SivaswamyCenter for IT in Education

International Institute of Information TechnologyHyderabad, A.P, India, 500032

Email: jsivaswamy@iiit.ac.in

Abstract—Interactive simulations encourage active/discoverylearning in students. But developing simulations is a timeconsuming task. This along with the usability and scalabilityissues are the bottleneck in wide availability of such tools.The developer spends most of his/her time in tailoring theuser interface (UI) to meet multiple constraints like pedagogy,teacher satisfaction and student learning styles. Providing anefficient way to develop UI can greatly speed up the develop-ment cycle and facilitate widespread access to high qualityresources. In this paper, we describe how UI developmentcan be separated from the simulation program to facilitateeasy development of such visualization tool. We also discussdesign and development of Graphical interface constructionKit (GicK) which helps in creating UI and combining it withsimulation program for creating interactive simulations. Aconstructive physics simulation is developed as an exemplarof proposed framework. In the end we reported evaluation ofconstructive physics simulation by 43 higher school studentsand our conclusion.

Keywords-Simulation, adaptable user interface, virtual envi-ronment, design

I. INTRODUCTION

Interactive simulations which fall in the mid ofinstructionism-constructionism spectrum [1] can provide ameaningful learning environment where active/discoverylearning is encouraged and supported. The interactive sim-ulations are standalone single purpose tools and aimed forshort and focused learning (examples are applets, flash-baseddemos, animations). They are meant for passive viewing orsupport very few forms of input. These standalone toolsare easy to use and can be easily integrated into classroomteaching. The use of interactive simulation in and outside theclassroom has gained considerable interest [2], [3], [4],[5].Common belief is that traditional theoretical/text basedapproach alone is often inadequate for describing complexconcepts and motivating students and interactive simulationscan be used as supplementary material in classroom to fillthis gap[6].

A recent study shows that, teachers spend a lot of time insearching for the right content and adapting them to suit theircurriculum. So teachers prefer to create their own contentto use in classroom. If a teacher could develop the contentin minimum time, then this would encourage them to usemore simulations in their classrooms [7]. The three key

obstacles to widespread adoption of interactive simulationsin classroom are:

• From the students perspective, the tool may not bemotivating.

• From the teachers perspective, the tool may not beeducationally beneficial.

• From the developers perspective, adapting tool to meetindividual teaching style and pedagogy will be difficultand ideally hard to achieve.

Availability of interactive simulations that utilize preferredteaching and learning style has become more importantto integrate such visualization tool in the classroom. Idealcondition will be where teachers have most of the requiredcontent at one place and they just drag and drop the contenton canvas to create instructional simulations. Hence one canthink of authoring tool which can offer flexibility in terms ofthe range of visualizations that can be developed for differentconcepts but at the cost of a minimum learning phase for theinstructors. Intelligent authoring tool are one such example[8], [9], [10] which are proving to be increasingly effective.However, they are difficult and expensive to build [11]. Soit would be of interest to explore a way to minimize thetime and expertise needed to build standalone interactivesimulations.

An interactive simulation is usually composed of an user-interaction layer tightly coupled with a simulation program.The tight coupling makes it difficult to separate a tool fromits particular use. One of the solutions we are proposing inthis paper is separating the user-interaction layer from thesimulation program. User-interaction layer can be created orcontrolled by the instructional designer such as developersand domain experts such teachers; software developers canwork on building intelligent simulation engine to supportvarious behaviors. As a result, one can develop widerrange of standalone interactive simulation tools. Moreover,development hours spent in building standalone interactivesimulations can be reduced. Since standalone visualizationtool are easy to use, the time to learn the new tool will bereduced.

In this paper first, we will review relevant literaturein the area of interactive simulations. Following this, we

2010 IEEE International Conference on Digital Game and Intelligent Toy Enhanced Learning

978-0-7695-3993-5/10 $26.00 © 2010 IEEE

DOI 10.1109/DIGITEL.2010.25

38

will describe the difficulty in creating and adopting suchvisualization tool in classroom. Then we will discuss theproposed framework to explore the possibility of creatinguser interface of a program to adapt to a particular use.We will then present the design process itself, as wellas implementation of Graphical interface construction Kit(GicK) and constructive physics simulations. By constructivesimulation is meant a visualization tool where students areprovided with few objects to create possible combinationof objects and observe their behavior controlled by thesimulation program. Such a visualization tool hence neednot be a complete authoring tool and can lie in betweeninteractive simulation and simulation based authoring tool.Finally we will evaluate the constructive physics simulationand report the conclusion.

II. MOTIVATION

In general, the challenge lies in adapting technology tocreate an educational tool where teachers, students andinstructional designer have appropriate control over instruc-tional educational content without increasing the complexityof the system [11] [12]. In order to provide flexibility toinstructor/teacher a large amount of effort has been put incontent representation using authoring tool. Authoring toolsoffer flexibility in terms of the range of visualizations thatcan be developed for different concepts but at the cost of asteeper learning curve for the teacher/student. The work bySIGCSE group [13] indicates that 72% of teachers preferusing static visualization almost daily in their classroomand 54 % of the teachers reported that they use dynamicvisualization only a few times per term. The reason forthis usage pattern is possibly that teachers face substantialimpediments when they try to use such visualization tools intheir teaching [14]. Hence, an effective way to increase theuse of dynamic visualization in classrooms is to developand disseminate standalone interactive simulations whichare suited to a curriculum [7]. This would facilitate easyand widespread access to high quality educational resourcessuch as PhET [4], MyPhysicsLab [15], cITe [16]. However,development cost for such interactive simulations is highin terms of time and effort. To eliminate development timeinvolved in creating interactive simulations we examine thecomponents of simulation-based educational tools with anaim to separate a simulation tool from its particular use. Thegoal is to find a way to fabricate the interactive simulationsand adapt them to the way the concepts are presented in theclassroom by teachers. Furthermore, this needs to lead toactive learning (by doing) in students.

In traditional simulation tool design the user interface(UI) is tightly coupled with simulation program. Typicallyconsiderable effort is spent in interface building of anapplication to provide flexibility to user to perform differenttask [17]. An alternate approach can be where the interfaceitself can be created or adapted by the instructional designer.

Such an interactive simulation where complete UI can bedeveloped or tailored without programming, will be ofinterest. This will enable the developer to focus on addingnew functionalities to computer programmed model withminimum effort in UI development. Additionally, it willreduce the time spent by teachers on learning to use thetool.

Given this analysis, next we propose a method for design-ing interactive simulations such that the UI can be changedaccording to needs of the user.

III. PROPOSAL

Figure 2. Gick Architecture

A simulation tool can be characterized by three compo-nents: 1) Content, which defines different kind of resourcesand their characteristics. 2) Simulation program 3) Interfaceor View layer. The interface layer is a very critical aspectof the overall software design process. It acts as the linkbetween the user and simulation program for any appli-cation. In traditional designs, these 3 are tightly coupled.We propose to separate the last component from the tool.This will enable the integration of the interface layer withdifferent simulation programs.

UI for any application can be thought of as an virtualenvironment composed of graphical objects where eachobject can act as active object. These active objects canplay different role (navigation menu, input controls, sim-ulation body) in virtual environment. Normally interactivesimulation tools do not let the end user choose what controlsand menus will be displayed. Even if it is configurable,this is generally via the preferences section and hence ishidden. The real time rendering of graphical object providesflexibility to configure the interface model with differentneeds. To create or adpat user interface for an interactivesimulation we have designed and developed a Graphicalinterface construction Kit (GicK). The proposed architecturefor creating interactive simulations using GicK is shown inFigure 2.

39

Figure 1. (a) Sample discription file format

(a) (b)

(c) (d)

Figure 3. (a)Virtual Lab(VL) with buttons as input control (b) VL with graphical object as input control (b) Creating a virtual background for anexperiment (d) Showing visual response to the user. (Enlarge the image to see instruction text and navigation menu on the top of the screen)

A. Graphcial interface construction Kit (GicK)

To create or adpat user interface for an interactive simu-lation we have designed and developed Graphical interfaceconstruction Kit (GicK), Figure 2. The GicK is designed topermit an interface to be constructed with a set of verbalinstructions and configuration parameters that are specifiedin an interface description file. These instructions and pa-rameters allow users to change the interface, decide which

controls would be displayed, what objects would be availablein toolbox, and what feedback would be provided. Themain components in GicK are a description file; controller;xml processing; virtual environment composition engine andaction shell.

• Description file: A description file as shown in Figure1 is used to create a semantic representation which willthen be converted into a hierarchical structure of objects

40

to be rendered. The user can specify objects in thedescription file that should be present in the interface,their size, position, orientation, surface color, controlsand toolbox items that would be visible. The user candefine the description file similar to a XML document.

• XML processor: The XML processor processes thedescription document and tries to extract understandingfrom the text information recognized at each node.Because understanding is a mapping process, a databaseis used to preprocess the input text information tosupplement the analyser’s knowledge. The XML pro-cessing output normalized DAG dependency tree whichis an arrangement of visual objects in a tree structurecalled as scene graph that completely specifies thecontent of the a virtual environment, and how it is tobe rendered.

• Virtual environment composition engine: The virtualenvironment composition engine creates the layout ofthe user interface from the structure generated by thexml processing engine as shown in Figure 2. Everyobject in the interface is placed in relationship withother objects or ground and their positions are deter-mined accordingly. Constraints are applied to facilitatethe placement of the visual objects. Below are the typeof placement constraint:

– Surface constraint: Surface constraint define onwhat surface an object is to be placed. The shape ofobject relative to which other object has to placedcan be 1) open 2) semi-closed and 3) closed. Bydefault each body is assumed closed surface, incase body is define as open or semi-closed a flagindicating 3D placement location on the surface isused for placement of object.

– Proximity constraint: A proximity constraint indi-cates how close the object should be placed relativeto other object. These constraints are encapsulatedas an integral function to map spatial relationshipto approximate the position.

– Semantic constraint: In the real world, we gener-ally make some inherited assumptions about whereobjects are defined in relationship with others. Theplacement of one object on a surface determineshow to place the other object on the same surface.This is can be best explained by the case where ’Cup and Lamp are on Table ’ result into a samescene as ’Cup on Table and Lamp is Left/Rightof the Cup’. In addition their might be the casewhere shape of two object, in our case ’Cup’ and’Lamp’, are differ and approximate position hasto be determine correctly when Lamp is define inrelationship with ’Cup’ as in case of sentence ’Cupon Table and Lamp is Left/Right of the Cup’. Therelationship of ’Lamp’ with ’Cup’ is not enough

to place the object correctly on the table.In such a scenario, one possible solution is to usepseudo-physics to find out the final position of anobject. To avoid the computation cost needed toapply the pseudo-physics, each object position isdetermined by its relationship with specified objectin the scene graph and the object on which it hasto be placed.

• Action shell: This component processes event/requestand combines it with response defined in descriptionfile. It identifies events and associates responses tothem. These events and responses can be provided inthe description file. Events are inputs to the system.Responses can be audio, verbal to visual feedback tospecific events. Feedbacks can be at two levels: oneat the application level that developers can use toguide novice users, and the other at the experimentlevel, where instructors use a medium of choice toprovide instructional content. They can use audio andtext feedback to provide media-rich educational contentto the learners.

• Controller: The controller provides a centralized pointof control. The controller is responsible for interpretingthe request from user and pass it to the other componentfor the appropriate action. After the action has beentaken on the request, the controller is responsible fordirecting the appropriate visual response to the user viaUI.

B. Simulation Program

The simulation program can be a simple computationsuch as solving the simple harmonic motion equation to acomplex engine such as a physics engine. In general, it em-bodies the computational engine of an interactive simulation.For illustrative purpose, we will use a physics simulationdeveloped using the Open Dynamics Engine (ODE) [18]which has been used for developing games. We use ODEto develop a constructive physics simulation tool for highereducation. The ODE is combined with other componentssuch as graph plotter data recording etc, to develop a virtualphysics lab.

The Figure 4(a) shows the constituent components of oursimulation program. This simulation program helps createa virtual workplace to author new experiments with a setof experimental (domain) objects which can be combinedto form experimental structures of interest. The physicsengine ODE controls the behavior of the domain objectsby implementing basic physics laws. The physics simulationengine helps in simulating the physical environment once arequired structure has been constructed. Users can changeobject attributes or environment properties.

To create a complete virtual physics lab, other programsare integrated with ODE to show graphs at runtime, recordthe measurement data in a separate data file and save the

41

(a) (b)

Figure 4. (a) Simulation program (b) Constructive simulation tool created by combining simulation program with GicK

running simulation as a movie. These are intended to helpa learner to observe the dynamic behavioral changes frameby frame after running the simulation.

By defining different description file for an experimentinstructive designer can create standalone demos with lim-ited controls. The Figure 5 (a) show how one can openan experiment with restricted controls on UI to createinstructive simulation. The learners can play and pause theexperiment to observer the changes.

C. Content Object Format

Visual information can be presented in text, images orvideo, which can be easily represented by graphical objects.Defining 3D graphical objects as basic form of contentmodel helps in reducing the complexity which incur inhandling different media formats. But it necessitates havinga database of 3D graphical objects for all form of visualobjects. Number of objects needed depends on the appli-cation domain. Objects can also be grouped together byusers to create complex objects. Most importantly databaseof geometry model for 3D objects make it possible to recordthe running simulation without compromising the renderingrates. The figure 5 (c) and (d) shows recorded simulationexperiment which can be edited to add text instructions andimages. Creation of complex objects can be made simplerfor a user if the functionality of GicK is extended to handlenatural language descriptions as input as shown in Figure 5(b)

IV. EVALUATION

We evaluated GicK by creating different interactive simu-lations. We created standalone simulations for Newtons lawof motion, inclined plane, projectile, pendulum, bouncingball and other experiments that are possible with the ODE.

We did a pilot study where users can construct a simu-lation by arranging objects in a virtual space, connect themtogether, modify their properties and observe the behavior.

There are lot of interative physics simulation but we didnot find any constructive virtual physics lab for highereducation. By constructive we mean it provides a virtualworkplace to create new experiment. A set of experimental(domain) objects are made available to the user to constructexperiments within the virtual environment space. Interactivesimulation are different form the construtive simulation inthe way that in the interactive simulations experiment arepre-set by the developer and user are provided by definedset of parameters to change the dynamics of simulation. Wetried find how such a system will be accepted by studentsin classroom. Next, we will describe our pilot study.

A. Pilot Study: User Interaction with constructive physicssimulation

We conducted an evaluation of our constructive virtualphysics tool with 43 users. We were interested in studyingthree factors:

• Utility of system feedback in learning a new tool• Effect of prior exposure to other software applications,

on the nature of interaction with the tool• The usability of such constructive simulation tool

B. Material

Users evaluated two versions of our simulation tool. Oneversion provided the users with maximum feedback and theother limited feedback. In the latter, audio feedback was as-sociated with only few controls and incomplete instructionswere provided at different stages (experiment construction,parameter settings for objects and graph selection). Systemfeedbacks consisted of visual dynamic graphics, audio andverbal feedback. The verbal feedback consisted of hintsgiven to user about previous action and next accepted actionfrom the user.

In both versions, users could select an object from thetoolbox and place it into simulation environment and thenchange its parameters. Additionally, they can record mea-surements and plot graph for the selected measurements.

42

(a) (b)

(c) (d)

Figure 5. (a) Incline plane interactive simulation (b) Creating a image from text input (b) Opening saved simulation as sequence of frame (d) Editing theselected frame by adding text and image.

Users can construct and change the settings of simulation inpause mode only. Two different views of graph was shown tothe user. In the pause mode, graph is zoomed and appears tobe behind the simulation area. In the play mode, the graph,if selected, is shown above the simulation environment.

C. Users

There were 3 groups of users. Group I had 7 users fromgraduate and undergraduate computer science students whowere very familiar with computers. Group II and Group IIIhad 18 users each. These were students from a high schoolwho had very little access and familiarity with computers.Group I and II were given the authoring tool with limitedfeedback while Group III was given the version whichprovided maximum feedback.

D. Rationale behind the experiment

The two prototypes were created to test the importanceand need for different dimensions of system feedback. Theintention of making minor changes in the versions was toillustrate that a simple system with few controls on userinterface are easily accepted whereas exposure of too manycontrols simultaneously, distracts the users.

We took the classical example of pendulum experimentwhere one need to understand the relationship between timeperiod and length of the rod attached between pivot and thebob. A list of tasks were given to the user: 1) construct thependulum 2) Set the bob parameter, 3) Plot the positiongraph for bob 4) Record the instantaneous velocity andposition values of bob. Sufficient time was given to completethe task. No initial instructions were given on how to usethe tool. We then received feedback from them on the taskthey were able to complete, amount of time they took tocomplete them and their comfort level with the tool.

E. Results

All participants were able to complete the task. Themaximum and minimum time taken to complete the taskwas approximately 15 min and 2 mins, respectively. Theresults are tabulated in Table.I. The result shows that novicecomputer users can also perform as good as advanced usersif given enough feedback. Users of Group III found verbal,visual feedbacks very useful to complete the task. This isevident from the average time taken to complete the task byGroup II and III. Group I relied on their past experience tocomplete the task. They avoided the verbal instructions and

43

Group Users Experience Feedback *AvgTime

(mins)I 7 Advanced Limited 7II 18 Novice Limited 12III 18 Novice Maximum 7

Table IEXPERIMENT PARAMETERS AND OBSERVATIONS,*Avg Time is time

taken to complete the given task.

consequently took more time to complete the task.1) Observations: Though no initial instructions being

given, most students were able to work out how to use thetool. They found visual feedbacks and instruction at eachstage very useful to complete the task. Once they had a hangon the tool, we found that they started experimenting. Theyvaried the initial parameter values, position of the objectsand observed the behavior. Such experimenting would havetaken a lot of time in a physical lab. But they were able todo this in less than 15 minutes with the virtual lab. They alsotried to interpret the graph drawn by the tool. The immediatecause and effect relation they were able to see encouragedthem. We got a very positive feedback on their experiencewith the tool.

We found that the system feedback are important part ofthe overall user interface design process to reduce the burdenof learning of the tools. These feedback are also essentialfor guiding the learners in constructive teaching approach.General wisdom is that navigational components shouldbe included in the authoring tool design to overcome theimpediment of learning to use these tools. However, since nostandard has been defined on tool design, users tend to relyon their past experience with other tools and applications tolearn any new tool given to them. Our experimental resultsalso reveal this to be the case.

Even after completing the give task we found studentswere trying different combination and arrangements of ob-jects. This shows that there is scope for such constructivevirtual environment in student learning. Clearly it would ofinterest to see how these constructive simulations can beeffective in active learning. In future, we are planning toconduct a controlled experiment to quantitatively evaluatethe improvement in student’s learning with such type of toolin comparison to interactive simulation.

V. FUTURE WORK

Currently the GicK design meets the needs of pedagogy-oriented systems more than performance-oriented systems.Future improvement of GicK may include adding high leveldisplaying functions for graphics objects and framework foreasy integration of different simulation engine. In additiona visual language can be developed for instructor/developeras well as teacher to create description files. Another area ofinterest is to build interactive virtual learning environment

similar to the digital games by turning individual simulationinto educational games as shown in Figure 6.

VI. CONCLUSION

Figure 6. Basket Ball Game.

In this paper, we propose a solution to enable UI designto be in the control of a user rather a developer. This was byseparating the interface layer from the simulation programand content model layer. An interface construction kit waspresented as an way to implement this idea. The kit was usedto develop interactive simulations. Adaptable user interfacescan speed up the creation, adaptation, and utilization ofinteractive simulations.

REFERENCES

[1] P. Dawabi and M. Wessner, “Combining instructionist andconstructionist learning in a virtual biotech lab,” in C. Mont-gomerie and J. Viteli (Eds.),World Conference on EducationalMultimedia, Hypermedia and Telecommunications, 2001, pp.369–373.

[2] M. Magee, “State of the field review :simulation in edu-cation,” Final Report:Alberta Online Learning Consortium.Calgary AB, 2006.

[3] R. L. Bell and L. K. Smetana, “Using computer simulationsto enhance science teaching and learning.” National ScienceTeachers Association, 2008, pp. 23–32.

[4] W. K. Adams, S. Reid, R. LeMaster, S. B. McKagan, K. K.Perkins, M. Dubson, and C. E. Wieman, “A study of educa-tional simulations part i - engagement and learning,” Journalof Interactive Learning Research, pp. 397–419, 2008.

[5] J. H. Kim, S. T. Park, H. Lee, K. Yuk, and H. Lee, “Virtualreality simulations in physics education,” Interactive Multi-media Electronic Journal of Computer-Enhanced Learning-IMEj, 2001.

[6] D. Kennepohl, “Using computer simulations to supplementteaching laboratories in chemistry for distance delivery,”Distance Education, pp. 58–65, 2001.

44

[7] T. Naps, S. Cooper, B. Koldehofe, C. Leska, G. Rling,W. Dann, A. Korhonen, L. Malmi, J. Rantakokko, R. J. Ross,J. Anderson, R. Fleischer, M. Kuittinen, and M. McNally,“Evaluating the educational impact of visualization,” in ACMSIGCSE Bulletin, vol. 35. ACM, 2003, pp. 124 – 136.

[8] T. Murray, “Authoring intelligent tutoring systems: An analy-sis of the state of the art,” in International Journal of ArtificialIntelligence in Education, vol. 10. Springer, 1999, pp. 98–129.

[9] J. Zumbach, S. Schmitt, and P. Reimann, “Learning lifesciences: Design and development of a virtual molecularbiology learning lab,” Journal of Computers in Mathematicsand Science Teaching, vol. 25, pp. 281–300, 2006.

[10] W. R. V. Joolingen and T. D. Jong, “Simquest: Author-ing educational simulations,” in T. Murray, S. Blessing, S.Ainsworth: Authoring Tools for Advanced Technology Learn-ing Environments: Toward cost-effective adaptive, interactive,and intelligent educational software, 2003, pp. 1–31.

[11] T. Murray, “Design tradeoffs in usability and power for ad-vanced educational software authoring tools,” in EducationalTechnology, 2004, pp. 10–16.

[12] S. A. T. Murray, S. Blessing, Authoring Tools for AdvancedTechnology Learning Environments: Toward cost-effectiveadaptive, interactive, and intelligent educational software.Springer, 2004.

[13] T. L. Naps, G. Rling, V. Almstrum, W. Dann, R. Fleis-cher, C. Hundhausen, A. Korhonen, L. Malmi, M. McNally,S. Rodger, and J. ngel Velzquez-Iturbide, “Exploring therole of visualization and engagement in computer scienceeducation,” in ACM SIGCSE Bulletin, vol. 35. ACM, June2003, pp. 131 – 152.

[14] J. Buzhardt and L. Heitzman-Powell, “Stop blaming theteachers: The role of usability testing in bridging the gapbetween educators and technology,” Electronic Journal forthe Integration of Technology in Education, pp. 1–19, 2005.

[15] E. Neumann, “Myphysicslab: Physics simulation with java,”http://www.myphysicslab.com/.

[16] CITE, “Viplab,” http://new.iiit.ac.in/research/cITe/index.html.

[17] W. K. Adams, S. Reid, R. LeMaster, S. B. McKagan, K. K.Perkins, M. Dubson, and C. E. Wieman, “A study of ed-ucational simulations part ii interface design,” Journal ofInteractive Learning Research, pp. 551–577, 2008.

[18] R. Smith, “Ode,” http://www.ode.org/.

45