Knowledge-based Representation of 3D Content Behavior in aService-Oriented Virtual Environment

Jakub FlotyńskiPoznań University of Economics and Business

Niepodleglości 10Poznań, Poland 61-875

�otynski@kti.ue.poznan.pl

Krzysztof WalczakPoznań University of Economics and Business

Niepodleglości 10Poznań, Poland 61-875

walczak@kti.ue.poznan.pl

Figure 1: An example VR application with reasoning on high-level behavior description.

ABSTRACTSemantic representation of interactive 3D content gains increasingattention due to possibilities of high-level domain-speci�c contentdescription, inference of hidden knowledge as well as explorationof content on request with the use of semantic queries. However,the available approaches mostly address representation of static 3Dobjects and scenes, which do not evolve over time, thus disregard-ing the current state of dynamically changing content. The paperpresents knowledge-based representation of 3D content behavior.The representation covers 3D content features at di�erent points intime, enabling reasoning and exploration of dynamic content. Thepossibility of using di�erent vocabularies enables application ofthe approach to di�erent domains. The representation can extendthe available static semantic 3D content representations, and itcan be used together with 3D modeling tools and game engines.The proposed representation has been implemented in the Prologdeclarative language and deployed within a service-oriented virtualenvironment with the Unity game engine.

CCS CONCEPTS• Information systems → Multimedia content creation; Se-mantic web description languages; Multimedia databases; REST-ful web services; • Computing methodologies → Virtual Real-ity; • Theory of computation → Semantics and reasoning;

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor pro�t or commercial advantage and that copies bear this notice and the full citationon the �rst page. Copyrights for components of this work owned by others than theauthor(s) must be honored. Abstracting with credit is permitted. To copy otherwise, orrepublish, to post on servers or to redistribute to lists, requires prior speci�c permissionand/or a fee. Request permissions from permissions@acm.org.Web3D ’17, Brisbane, QLD, Australia© 2017 Copyright held by the owner/author(s). Publication rights licensed to ACM.978-1-4503-4955-0/17/06. . . $15.00DOI: http://dx.doi.org/10.1145/3055624.3075959

KEYWORDS3D web, semantic web, behavior modeling, event calculusACM Reference format:Jakub Flotyński and Krzysztof Walczak. 2017. Knowledge-based Represen-tation of 3D Content Behavior in a Service-Oriented Virtual Environment.In Proceedings of Web3D ’17, Brisbane, QLD, Australia, June 05-07, 2017,10 pages.DOI: http://dx.doi.org/10.1145/3055624.3075959

1 INTRODUCTIONVirtual reality (VR) and augmented reality (AR) applications be-come increasingly popular in various application domains, such aseducation, training, tourism, medicine, entertainment, social me-dia and cultural heritage, signi�cantly enhancing possibilities ofpresentation and interaction with complex data and objects. Theprimary element of any VR/AR application is interactive 3D content.Creation, modi�cation and customization of interactive 3D contentare much more complex and challenging tasks than in the case oftypical web resources. Components of 3D web content are struc-turally more complex and dependencies between such componentsencompass several aspects – visual, spatial, temporal, structural,logical and behavioral – making it di�cult to manage such contenteven when using high-level component-based content manage-ment systems. Despite signi�cant research and development e�ortin this domain, di�culty associated with creation of interactive 3Dcontent is still one of the main obstacles limiting wider use of 3Dtechnologies on the web.

Modeling of 3D content behavior is possible with a numberof well-established technologies, which may be divided into sev-eral groups. The �rst group encompasses libraries (e.g., Away3D(Away3D 2017), Java3D (Oracle 2017) and Direct3D (Microsoft2017)), which enable programming of 3D content behavior with im-perative languages, such as ActionScript, Java and C++. The second

Web3D ’17, June 05-07, 2017, Brisbane, QLD, Australia J. Flotyński, K. Walczak

group includes declarative 3D content formats, such as VRML (W3C2017b), X3D (W3C 2017a) and XML3D (DFKI 2017), which enablecontent animation based on key frames and interpolation as well asdescribed by snippets of imperative scripts. The use of key framesand interpolation is also permitted by tools for visual modelingsuch as Motion Builder (Autodesk 2017b) and SketchUp (Trimble2017), which form the third group. The last group comprises en-vironments that enable both manual modeling of animations andprogramming of behavior, e.g., Blender (Foundation 2017) and 3dsMax (Autodesk 2017a).

Modeling of 3D content with the aforementioned approachesdoes not conceptually conform to the current trends in the develop-ment of the web. In particular, the following requirements shouldbe met to enable wider dissemination of behavior-rich 3D content.

(1) The content modeling process requires the involvementof domain experts, who are usually not IT-specialists. Inaddition, the same users are often both content produc-ers and consumers (prosumers). However, the availableapproaches either require modeling of content based onconcepts speci�c to 3D graphics or restrict designers touse a domain-speci�c tool.

(2) The 3D content representation should be understandableto non-programmers. In particular, it should focus on high-level goals to be achieved (like in declarative programming)instead of the steps that must be completed to achieve thegoals (like in imperative programming).

(3) To liberate 3D content authors from determining all con-tent features, the content creation process should enableinference of tacit knowledge on the basis of the explicitlystated facts and rules, e.g., to infer assignment of objects toclasses based on property domains and ranges. However,the available approaches require speci�cation of all 3Dcontent features to be presented.

(4) Created 3D content representations should be suitable for�exible exploration with high-level queries, e.g., by searchengines and analytical tools. This is not enabled by theavailable 3D content formats and representations based onprogramming libraries, which are di�cult for automaticprocessing.

These requirements can be satis�ed by using the semantic webstandards for representing the created 3D content. The research onthe semantic web aims at evolutionary development of the currentweb towards a distributed database linking structured content. Se-mantic description of web content makes it ”understandable” forboth humans and computers, achieving a new quality in buildingweb applications that can interpret the meaning of particular com-ponents of content as well as their relations, leading to much bettermethods of content creation, management and presentation.

However, the available semantic 3D content representations aremainly static – they do not evolve over time even if the contentdoes, i.e. even if a semantic representation speci�es changes ofcontent features, it is not altered after the changes occur. Therefore,dynamic (time-dependent) nature of semantically represented 3Dcontent cannot be subject to semantic content exploration. Suchexploration would enable precise querying of dynamic content. Forexample, a running 3D engine could be queried about the states

and fuel consumption at di�erent points in time; 3D football gamescould be queried about the goals scored by a player in a period oftime; a 3D chess board could be queried about the possibility to�nish the game at a particular time.

The main contribution of this paper is knowledge-based repre-sentation of interactive 3D content behavior. The representationis based on semantic facts and rules, which enable expression ofdomain-speci�c concepts in a form that permits reasoning and isconvenient for automatic exploration with queries. In addition, thesyntax of rule-based descriptions can be more understandable tonon-IT-specialists in comparison to imperative instructions. Therepresentation is based on the event-calculus (Kowalski and Sergot1989), which is one of the main approaches to declarative descrip-tion of dynamic (time-dependent) domains. The representationhas been implemented in the Prolog declarative language withina service-oriented 3D virtual environment based on Unity (Tech-nologies 2017). However, also other languages for logic program-ming could be used, e.g., RuleML (Inc. 2017), Prova (Kozlenkov andPaschke Kozlenkov and Paschke) and SWRL (W3C 2004).

The remainder of this paper is structured as follows. Section 2provides an overview of the current state of the art in semanticrepresentation of 3D content and the event calculus, which is thefoundation of the proposed representation of 3D content behav-ior. In Section 3, the motivations of the proposed approach arediscussed. Section 4 introduces the new knowledge-based repre-sentation. In Section 5, the architecture and the implementation ofthe service-oriented 3D virtual environment based on the proposedrepresentation are described. An example dynamic knowledge-based representation of the 3D chess game is presented in Section6. Finally, Section 7 concludes the paper and indicates possibledirections of future research.

2 RELATEDWORKS2.1 Semantic Representation of 3D Content

2.1.1 Domain-independent 3D Content Representation. A num-ber of works have been devoted to semantic representation andmodeling of 3D content. In (Kalogerakis et al. 2006), an ontologyproviding elements and properties equivalent to elements and prop-erties speci�ed in X3D has been proposed. Moreover, a set of seman-tic properties have been proposed to enable representation of 3Dscenes with domain knowledge. In (Pittarello and De Faveri 2006),integration of X3D and OWL using scene-independent ontologiesand semantic zones have been proposed to enable semantic repre-sentation of 3D content. In (De Troyer et al. 2007), an approach togenerating virtual environments upon mappings of domain ontolo-gies to 3D world ontologies has been proposed. Another approachto semantic modeling of 3D content has been proposed in (Flotyński2014; Flotyński and Walczak 2013a, 2014; Flotyński and Walczak2015; Walczak and Flotyński 2014; Walczak et al. 2014). The ap-proach enables separation of concerns between domain-experts,developers and graphic designers. In addition, 3D content can becreated with queries to semantic scenes. In (Wiebusch and Latoschik2012), a framework for decoupling components in real-time interac-tive systems with ontologies, and the concept of semantic entitiesin VR applications have been discussed.

Knowledge-based Representation of 3D Content Behavior Web3D ’17, June 05-07, 2017, Brisbane, QLD, Australia

2.1.2 Domain-specific 3D Content Representation. An ontologyfor representing virtual humans has been described in (Gutiérrezet al. 2007). A virtual human is linked to geometrical descriptors(covering vertices and polygons), structural descriptors (coveringlevels of articulations), 3D animations of face and body and be-havior controllers (animation algorithms). In (Lugrin and Cavazza2007), abstract semantic representation of events and actions in AIsimulators has been presented. In (Jiang et al. 2009), semantic rep-resentation of 3D crowd simulation is proposed. Data semantics isdescribed in three respects: the structure, topology and height coor-dinates of 3D scenes. In (Chaudhuri et al. 2013), semantic attributesare selected by users to describe 3D models with di�erent strengths,e.g., animals may be scary to varying degrees. In CityEngine, mod-eling of 3D content is based on semantic rules that are CGA doc-uments specifying elements of the content, such as shapes andtextures, as well as operations on the content, such as extrude andsplit (Hu et al. 2013). In (Pellens et al. 2005a,b, 2009), an approachfacilitating modeling of content behavior has been proposed. Theapproach provides temporal operators and enables expression ofprimitive and complex behaviors. In (Rabattu et al. 2015), an ap-proach to spatio-temporal reasoning on semantic descriptions ofevolving human embryo has been proposed. The approach lever-ages RDF, OWL and SPARQL as well as an ontology describingstages, periods and processes.

Moreover, several works provide an overview of semantic de-scription of 3D content in arti�cial intelligence and web-basedsystems—(Latoschik and Blach 2008), (Van Gool et al. 2007), (Tutenelet al. 2008), (Spagnuolo and Falcidieno 2009) and (Catalano et al.2011). A comprehensive review of representation and modeling of3D content with semantic web ontologies has been presented in(Flotyński and Walczak 2017).

2.2 Event calculusThe issue of reasoning on temporal data has been extensively stud-ied in the domains of knowledge representation and—later—in thesemantic web. The research has lead to the development of multiplesolutions. Some of them are: the situation calculus (Kowalski andSadri 1994), the event calculus (Kowalski and Sergot 1989; Shana-han 1999) as well as temporal description logics, Temporal RDF,versioning of ontologies and 4D-�uents (Batsakis et al. 2009). Thiswork focuses on object and scene behavior, for which, in general,rule-based descriptions are more expressive than description logics.Moreover, in complex 3D scenes local states of di�erent objectsare associated with (possibly) parallel time intervals designated byconcurrent events, which may be independent and not necessarilyrelated to global situations of the scenes. Therefore, the event cal-culus speci�ed within the �rst-order logic (FOL) has been selectedas the framework for semantic behavior description in this work.

The event calculus enables reasoning on events and time. Thebasic concept in FOL, which is also the foundation of the con-cepts introduced in the event calculus is predicate. A predicate is aboolean-valued function that may be speci�ed on zero or multiplevariables, which can be values or other predicates:

predicate(X1,X2, ...,Xn ) ⇒ {true, f alse}.The main concepts of the event calculus are (Fig. 2):

(1) point in time;

(2) event occurring at a particular point in time;(3) period of time, which is started by an event and �nished

by an event;(4) �uent, which is a predicate whose value changes over time,

e.g., f alls(ball) is false for a ball running on a table as longas this is not on the edge.

after(e1)=before(e2)after(e0)=before(e1) after(e2)=before(e3)

timet0 t1 t2 t3

eventse0 e1 e2 e3

periods

in(t4, after(e2))t4

Figure 2: The main concepts of the event calculus.

The event calculus consists of several application and domain-independent predicates that describe ful�llment of application ordomain-speci�c �uents in some points and periods of time. Themost important predicates of the event calculus are the following:

(1) time(Event , Point) – the Point in time at which Event oc-curred.

(2) after(Event) – the period of time started by the occurrenceof Event.

(3) before(Event) – the period of time preceding the occur-rence of Event.

(4) start(Period,Event) – Period is started by the occurrenceof Event.

(5) end(Period,Event) – Period is �nished by the occurrenceof Event.

(6) in(Point , Period) – Point in time within Period.(7) holds(f luent(X1,X2, ...,Xn ), Period) – an application or

domain-speci�c �uent is satis�ed (true) for the variablesX1,X2, ...,Xn within Period. For instance,

holds(f alls(ball),af ter (reached_edдe))means that a ball is falling down since it reached the edge.

(8) holdsAt(f luent(X1,X2, ...,Xn ), Point) – �uent is satis�edfor the variables X1,X2, ...,Xn at Point in time. For in-stance,

holdsAt(inside(bullet ,body),point)means that the bullet is inside the body at the point in time.

3 MOTIVATIONThe available semantic 3D content representations do not cover dy-namic changes of 3D content properties and relations between con-tent components. When semantically represented content evolvesover time, new properties and relations between components arenot re�ected in the content representation. The main contributionof this paper is an approach to knowledge-based representation of3D content behavior. The approach separates 3D content from thehigh-level semantic representation of the content behavior.

Knowledge-based behavior representation can be easier and bet-ter understandable to users who are not IT-specialists, and moreconvenient for software for automatic analysis. In addition, thesimpler declarative syntax supports development of graphical no-tations and tools for developers. Moreover, it leads to the creationof semantically explorable 3D content, which is di�cult to achievewhen using other approaches.

Web3D ’17, June 05-07, 2017, Brisbane, QLD, Australia J. Flotyński, K. Walczak

Knowledge Base ManagerVR/AR Application

3D object behavior descriptions3D contentInteraction

triggersContent

manipulators

Figure 3: The separation between the low-level 3D scenemanagement logic and the high-level behavior logic in theproposed approach.

In general, two main elements of a VR/AR system compliantwith the approach may be distinguished (Fig. 3).

• A VR/AR Application, which enables 3D content presenta-tion and manipulation by users. The application may bebuilt using various 3D development technologies, such asprogramming languages and libraries, 3D modeling toolsand game engines. The application implements the low-level part of the overall system logic, with the focus onthe activities that do not need to be subject to semanticexploration and do not require complex implementation,in particular:(1) triggers in interaction between users and objects (pos-

sibly without the e�ects of such interaction), e.g.,touch action, capturing marker events in AR appli-cations;

(2) getting and setting values of properties of 3D objectsand scenes (possibly without determining the values),e.g., set object position to a given value.

• A Knowledge Base Manager (KBM), which enables event-based semantic representation of 3D content behavior. Im-plementation of behavior representation in a KB has im-portant advantages over implementing behavior directlyin the application, in particular in the cases when the logic:(1) needs to be a subject of semantic exploration with user

queries in collaborative web-based environments, e.g.,search for 3D football games with goals in the secondhalf;

(2) requires the use of high-level domain-speci�c con-cepts with well-de�ned meaning, e.g., avatars jump,rise hands and lift something up;

(3) requires speci�cation of a large number of contentproperties and relations that could be concisely ex-pressed with a lower number of more general rules,e.g., every object that is a cannon in a battle�eld isrunning in a direction within a period of time;

(4) can be expressed by constraints on content propertiesthat may be satis�ed in di�erent ways, e.g., in a battle-�eld di�erent knights �ght using di�erent weapons(it does not matter how a particular knight �ghts);

(5) requires complex implementation in the imperativeapproach (oriented on the steps to be done), which

can be simpli�ed in the declarative approach (orientedon the goals to be achieved), in particular searching incomplex recursive structures and checking multiplecombinations of elements, e.g., search for all possiblekilling moves in a checkers board.

Separation between the activities to be implemented in the ap-plication and the activities to be implemented in the KBM is theconcern of the system developer, who can take into account theaforementioned rules.4 EVENT-BASED CONTENT BEHAVIORThe event-based representation of 3D content behavior proposedin this paper consists of two ontologies, which are sets of facts andrules.

A fact is a predicate whose some variables may be set to par-ticular values. A fact is unconditional – its ful�llment does notdepend on the ful�llment of other facts. In the paper, variables andconstants are denoted with capital and small letters, respectively.A chess example fact:

holds(pos(piece(C), 1,Y ), t0)means that every individual that is a piece has the x coordinateequal to 1 and any y coordinate at the initial time.

A rule is an implication, whose head is true if the body is true,thus in contrast to facts, rules express conditions. Both head andbody are logic statements on facts. A speci�c type of rule is a Hornclause, in which the head consists of only one fact and the body isa conjunction of facts. The large advantage of Horn clauses overarbitrary rules is the availability of a polynomial-time inferenceprocedure (Russell et al. 2009). An example Horn clause:

opponent(Piece1, Piece2) ←color(Piece1,Color1), color(Piece2,Color2),Color1! = Color2

means that two pieces are opponents is they di�er in colors. Inthe proposed knowledge representation, facts and Horn clauses areused. The two main representation ontologies are:

(1) The event calculus ontology (ECO), in which the domain-independent predicates of the event calculus (cf. Sec. 2.2)are de�ned. The ECO is common to all VR/AR applicationsthat use the proposed knowledge-based behavior represen-tation.

(2) Domain-speci�c ontologies (DSOs), in which predicates spe-ci�c to a particular application or domain are de�ned. ADSO consists of the following groups of facts and rules.(a) Classi�cation facts, which assign 3D objects or their

components (e.g, materials and textures) to particularclasses. A classi�cation fact has the form class(object).

(b) Event de�nition facts, which indicate moments in timein which events occur. Events change 3D objects andtheir components in a dynamic scene. An event de�-nition fact has the form time(event, moment).

(c) Period de�nition facts, which indicate events that startor �nish periods of time. Properties of 3D objects andtheir components are immutable within periods. Aperiod de�nition fact has the form start/end(period,event).

(d) Fluent de�nition facts and rules, which specify condi-tions that must be satis�ed to make the �uent true. If

Knowledge-based Representation of 3D Content Behavior Web3D ’17, June 05-07, 2017, Brisbane, QLD, Australia

no conditions are given, the �uent is true at the par-ticular time or within the particular period. Fluentsspecify properties of objects and their componentsthat change over time. A �uent de�nition rule has theform holds/holdsAt(�uent(...), time/period)← ....

(e) Predicate de�nition facts and rules, which are similarto �uent de�nitions except they do not change overtime. Predicates specify properties of objects and theircomponents that do not change over time. Thereforethey have the form of a typical predicate (cf. Sec. 2.2).

In addition, the closed world assumption is used, thus 3D contentproperties that are not known to be true are considered false. Also,negation as failure is used to explicitly indicate the presence or thelack of content properties.

5 SERVICE-ORIENTED 3D ENVIRONMENT5.1 ArchitectureA service-oriented 3D virtual environment has been implementedto enable the use of the proposed knowledge-based behavior repre-sentation in di�erent VR/AR applications. The overall architectureof the environment is depicted in Fig. 4. The environment consists ofthree components—VR/AR Applications and the two elements of theKBM: Knowledge Management Service and Knowledge Base. VR/ARApplications and the Knowledge Management Service implementapplication logic. Distinction of these two system components en-ables separation and independent execution of time-consumingtasks—tasks speci�c to a particular VR/AR Application (3D process-ing) from tasks common to all applications (semantic processing).The third component—the Knowledge Base—is comprised of multi-ple ontologies. Below, the implementation of the three componentsis shortly described.

(1) A VR/AR Application is a component providing 3D scenesand 3D objects, which can be manipulated by users. In theenvironment, di�erent applications for di�erent domainscan be hosted. The application implemented in the proto-type system is the chess game built using the Unity gameengine. However, VR/AR applications used in the environ-ment can be based on di�erent tools providing networkcommunication capabilities. Every application has a sep-arate DSO, which includes facts and rules speci�c to theparticular domain.

(2) The Knowledge Management Service is the component re-sponsible for processing facts and rules describing appli-cation behavior. The service is shared by multiple VR/ARapplications and retrieves data from the KB. The proposednetwork communication between applications and the ser-vice is based on concise JSON messages sent through HTTPaccording to the REST approach. Upon receiving a requestfrom an application, the service makes reasoning to infernew facts to generate a response to the user’s query. Theinference is performed on the union of the DSO associ-ated with the particular VR/AR application and the ECO.Since the hosted applications have dynamic 3D contentthat changes over time, the service also adds new eventde�nition facts, period de�nition facts and �uent de�nitionfacts to the KB. Such additional data is associated with a

Users

VR/AP Applications

(Unity)

Knowledge Base(Prolog)

ECO

User group 3

User group 1

User group 2

Knowledge Management Service

(WCF, C#Prolog)

HTTP, REST, JSON

GUI interaction

DSO3

DSO2

DSO1App1

App2

App3

DSO-Ext-App1-

Session1

DSO-Ext-App1-

Session2

DSO-Ext-App1-

Session3

DSO-Ext-App2-

Session1

DSO-Ext-App2-

Session2

DSO-Ext-App2-

Session3

DSO-Ext-App3-

Session1

DSO-Ext-App3-

Session2

DSO-Ext-App3-

Session3

Figure 4: Service-oriented 3D environment architecture.

VR/AR application and the users’ sessions, and it is referredto as a DSO-Extension. In the prototype, the service hasbeen implemented in C# using the WCF library (the RESTnetwork interface) and the C#Prolog library (Pool 2017)(reasoning).

(3) The Knowledge Base (KB) is a storage comprised of multipleontologies that conform to the event-based representationof 3D content behavior proposed in Section 4. The KB hasbeen implemented in the Prolog declarative programminglanguage. It includes three sub-components:(a) The ECO including event calculus facts and rules that

de�ne predicates described in Sec. 2.2. Since thesepredicates are domain-independent, this ontology iscommonly used in knowledge processing for all VR/ARapplications hosted in the environment.

(b) DSOs including classi�cation facts, event de�nitionfacts, period de�nition facts, �uent de�nition facts andrules as well as predicate de�nition facts and rules,which are speci�c to a particular VR/AR application.A DSO is commonly used in knowledge processingfor all users interacting with a particular application.

(c) DSO-Extensions including event de�nition facts, pe-riod de�nition facts and �uent de�nition facts that aredynamically (on-request) added to the KB during theevolution of the 3D scene in a particular application.Since changes of 3D scenes in the applications canbe di�erent for di�erent users (depend on users’ ses-sions), every DSO-Extension is individually createdfor the usage of a particular application by a particular

Web3D ’17, June 05-07, 2017, Brisbane, QLD, Australia J. Flotyński, K. Walczak

group of users and re�ects the state of the interactionof the users with the application. For example, a groupof users can collaboratively design machine behavior.

5.2 InteractionInteraction between a user and the system involves a VR/AR Ap-plication, the Knowledge Management Service and the three on-tologies: the ECO (common to all applications), the DSO associatedwith the application, and a DSO-Extension dynamically created andmaintained for the particular interaction session between a groupof users and the system. The subsequent steps of interaction (Fig.5) are as follows.

Knowledge BaseUser

ECO

DSO

DSO-Ext-App-Session

(3) Add new facts

(4) Get all relevant knowledge

(2) Send request when scene manipulated

(5) Update scene according to

reasoning results

Knowledge Management Service

VR/AR Application

(1) Scene manipulation

Figure 5: The interaction schema in the environment.

(1) The user manipulates the 3D content of the VR/AR Appli-cation, e.g., transforms a 3D object.

(2) The action performed initiates a request from the VR/ARApplication to the Knowledge Management Service. Therequest is encoded in the JSON format and it includes:• if the action does not modify the scene—a query com-

prised of �uents for which {true, false} responses areexpected—if the only parameters of the �uents arevalues, or possible values of variables are expected—ifsome parameters of the �uents are variables, or

• if the action modi�es the scene—new event-, period-and �uent-de�nition facts—since an event occurred,which has �nished the previous and started a newperiod of time, and a property of the scene has beenchanged since that moment.

(3) In case of 3D content manipulation, the facts are extractedfrom the request and added to the DSO-Extension for thetuple (VR/AR Application, session) in the Knowledge Base.

(4) The union of the ECO, the DSO and the DSO-Extension isretrieved by the service for reasoning.

(5) The reasoning is performed and its results are conveyed tothe VR/AR application. If the request has included a query,the 3D scene is appropriately altered.

6 EXAMPLE APPLICATIONA 3D chess game has been implemented as an example applicationfor the proposed approach. The environment covers a VR Applica-tion based on the Unity game engine, the Knowledge Management

Service implemented in C# using the C#Prolog library (Pool 2017)and the Knowledge Base implemented in the Prolog declarativelanguage. The KB consists of the ECO, a DSO and dynamicallygenerated DSO-Extensions.

6.1 The Event Calculus OntologyAn excerpt of the ECO is presented in Listing 1. In lines 1-2, the startand end predicates are associated with the after and end predicates(cf. Section 2.2)—an event starts the period following it and ends theperiod preceding it. The unsaid predicate, which is out of the scopeof the event calculus, checks whether a predicate is true, if not—unsaid is satis�ed (lines 4-5). In the de�nition of unsaid, negationas failure has been used. A point in time is in a period, if the pointis between two events: starting and ending the period, or after theevent starting the period if the period is not �nished yet (lines7-8). A �uent holds at a particular point in time if the point is in aperiod within which the �uent holds (line 10). The complementingnotHoldsAt predicate has been de�ned using negation as failure(lines 11-12).

Listing 1: A fragment of the event calculus ontology1start(after(Event), Event).

2end(before(Event), Event).

34unsaid(Predicate) ← Predicate , !, fail.

5unsaid(Predicate).

67in(Point , Period) ← start(Period , Event1), end(Period , Event2),

time(Event1 , Point1), time(Event2 , Point2), Point >=Point1 ,

Point <Point2.

8in(Point , Period) ← start(Period , Event1), unsaid(end(Period ,

Event2)), time(Event1 , Point1), Point >= Point1.

910holdsAt(Fluent , Point) ← holds(Fluent , Period),in(Point , Period).

11notHoldsAt(Fluent , Point) ← holdsAt(Fluent , Point), !, fail.

12notHoldsAt(Fluent , Point).

6.2 The Domain-speci�c OntologyAn excerpt of the DSO is presented in Listing 2. In lines 2-5, classi-�cation facts assign the particular pieces on the board to di�erentclasses. The classes will be later used to check the possible movesof the pieces on the board. In addition, the numbers 1 to 8 are classi-�ed as coordinates on the board (line 7). For both axes, coordinatesbeing numbers have been established. Initial events of putting thepieces on the board are speci�ed in lines 9-12. Every initial eventoccurs at the time 0. In the game, every move of a piece is an indi-vidual event, which starts a new period for the piece. Hence, everypiece has its own set of events and periods. Periods started by theinitial events are speci�ed in lines 15-18. The color property, whichdistinguishes opposite pieces, is set in lines 21-24. In contrast topiece color, the positions of pieces change over the game, thus theyhave been modeled as �uents. The position of a piece is constantwithin a period of time determined by two subsequent events ofmoving the piece on the board (lines 27-30). The rule in line 33,which is independent of time, designates opponents as pieces withdi�erent colors. The most complex are the �uent de�nition rules.The possPos �uent speci�ed for a rook (lines 35-36) is satis�ed atthe point in time, if there is a free path along the x or y axis (lines38-40) and the destination �eld is free (lines 42-45) or is occupiedby an opponent piece. The possible position of the other types of

Knowledge-based Representation of 3D Content Behavior Web3D ’17, June 05-07, 2017, Brisbane, QLD, Australia

pieces is veri�ed in a similar way, respecting the appropriate gamerules for them. Finally, a piece becomes dead, if its �eld starts tobe occupied by another piece that has not occupied this �eld atthe previous moment (line 47). In such a case, the DeadEvent isgenerated, and it starts the endless period within which the pieceis out of the game.

Listing 2: A fragment of the domain-speci�c ontology1 ---Classification facts-------

2 rook(white_rook1).

3 rook(black_rook1).

4 king(white_king).

5 king(black_king).

6 ...

7 coord (1)..coord (8).

8 ---Event definition facts-----

9 time(event_white_rook1_0 , 0).

10 time(event_black_rook1_0 , 0).

11 time(event_white_king_0 , 0).

12 time(event_black_king_0 , 0).

13 ...

14 ---Period definition facts----

15 start(after(event_white_rook1_0), event_white_rook1_0).

16 start(after(event_black_rook2_0), event_black_rook2_0).

17 start(after(event_white_king_0), event_white_king_0).

18 start(after(event_black_king_0), event_black_king_0).

19 ...

20 ---Predicate definition facts-

21 color(white_rook1 , white).

22 color(black_rook1 , black).

23 color(white_king , white).

24 color(black_king , black).

25 ...

26 ---Fluent definition facts----

27 holds(pos(white_rook1 ,1,1), after(event_white_rook1_0)).

28 holds(pos(black_rook1 ,1,8), after(event_black_rook1_0)).

29 holds(pos(white_king ,4,1), after(event_white_king_0)).

30 holds(pos(black_king ,4,8), after(event_black_king_0)).

31 ...

32 ---Predicate definition rules-

33 opponent(Pawn1 ,Pawn2) ← color(Pawn1 ,C1), color(Pawn2 ,C2), C1\=C2.

34 ---Fluent definition rules----

35 holdsAt(possPos(Piece ,X,Y2), Point) ← rook(Piece), holdsAt(pos(

Piece ,X,Y1), Point), coord(Y2), holdsAt(freePathY(X,Y1,Y2),

Point), holdsAt(freeField(X,Y2), Point).

36 holdsAt(possPos(Piece ,X,Y2), Point) ← rook(Piece), holdsAt(pos(

Piece ,X,Y1), Point), coord(Y2), holdsAt(freePathY(X,Y1,Y2),

Point), opponent(Pawn2 , Piece), holdsAt(pos(Pawn2 ,X,Y2),

Point).

37 ...

38 holdsAt(freePathY(X,Y1,Y2), Point) ← coord(Y3), Y3>Y1, Y3<Y2,

holdsAt(notFreeField(X,Y3), Point), !, fail.

39 holdsAt(freePathY(X,Y1,Y2), Point) ← coord(Y3), Y3<Y1, Y3>Y2,

holdsAt(notFreeField(X,Y3), Point), !, fail.

40 holdsAt(freePathY(X,Y1,Y2), Point).

41 ...

42 holdsAt(freeField(X,Y), Point) ← coord(X), coord(Y), holdsAt(pos(

Piece ,X,Y), Point), !, fail.

43 holdsAt(freeField(coord(X),coord(Y)), Point).

44 holdsAt(notFreeField(X,Y), Point) ← coord(X), coord(Y), holdsAt(

freeField(X,Y), Point), !, fail.

45 holdsAt(notFreeField(coord(X),coord(Y)), Point).

4647 holds(dead(Piece), after(DeadEvent)) ← time(KillingEvent , Point),

holdsAt(pos(Piece ,X,Y), Point), holdsAt(pos(KillingPawn ,X,Y),

Point), opponent(Piece ,KillingPawn), notHoldsAt(pos(

KillingPawn ,X,Y), Point -1), holds(pos(KillingPawn ,X,Y),

Period), start(Period , KillingEvent), DeadEvent is "event_" +

Piece + "_dead", assertz(time(DeadEvent , Point)).

6.3 InteractionThe implemented chessboard with pieces and an example sequenceof interaction steps is presented in Fig. 6-10. The 3D chess models

from http://www.blendswap.com/blends/view/40175 have been used.The following steps correspond to basic activities performed byusers in the chess VR Application as presented in Fig. 6-10.

(1) Select piece (Fig. 6). Since any piece can be selected and theselection involves only a mouse event and a color change,no reasoning is necessary. Hence, the VR Application doesnot use the Knowledge Management Service for this pur-pose.

(2) Verify selected move (Fig. 7 and 8). The VR Applicationcaptures the selected destination position and generatesa message in the JSON format, which speci�es the �uentand indicates the potential destination �eld as well as apoint in time, which is an integer incremented with everyevent generated:{ fluent: possPos ,

piece: white_rook_1 ,

field: [2, 5], point_in_time: CurrTime }

Since the Knowledge Management Service is stateless, theVR Application includes information about the currenttime in every request for a �uent. Such approach allows theservice to be generic and shared by various VR applicationshosted in the environment. The request is transformed bythe Knowledge Management Service to a Prolog query:holdsAt(possPos(white_rook_1 ,2,5), T)

Next, the query is veri�ed against the KB. The moves areprohibited by the possPos �uent since the source and thedestination �elds have no common axis (Fig. 7a, Listing 2,line 35) or the �eld is already occupied by a friendly piece(Fig. 7b, line 36). The JSON response message delivered tothe VR Application is the following:{ response: no }

The moves presented in Fig. 8 are permitted since they leadto free �elds along an axis. The requests and the responsesare analogous to the previous ones.

(3) Move piece (Fig. 9). Once a permitted �eld is chosen, arequest is sent from the VR Application to the KnowledgeManagement Service to update the KB. The sent JSONmessage has the following form:{ activities: [

{ predicate: time , event: event_rook_id_NUMBER ,

point_in_time: CurrTime },

{ predicate: end , event: event_rook_id_NUMBER ,

point_in_time: CurrTime },

{ fluent: pos , piece: rook_id , field: [X,Y],

event: event_rook_id_NUMBER } ] }

The verify move andmove piece requests are sent every timea move is performed by a player. The service transformsevery update request to an event de�nition fact (a newevent occurred), a period de�nition fact (the event starts anew period) and a �uent de�nition fact (within the period,the rook has a new position). Unique identi�ers of eventsmust be maintained during the game, e.g., by generatingnumbers using a sequencer.time(event_rook_id_NUMBER , CurrTime ).

end(after(event_rook_id_[NUMBER -1]), event_rook_id_NUMBER ).

holds(pos(rook_id ,X,Y), after(event_rook_id_NUMBER )).

Web3D ’17, June 05-07, 2017, Brisbane, QLD, Australia J. Flotyński, K. Walczak

It would be possible to add consistency checking betweenthe new piece position and the potentially possible posi-tions at the Knowledge Base level by adding appropriaterules. This would validate the conformance of the selecteduser’s move to the game rules in the KB (Listing 2, lines35-36). The facts are added to the DSO-Extension associ-ated with the current game session. A DSO-Extension isalways created at the beginning of the game for two oppo-nent players. The following response, which con�rms therequired action, is delivered to the VR Application:{ response: yes }

(4) Kill piece (Fig. 10). In case of killing a piece, the messagesent to the Knowledge Management Service has the sameschema as in case of piece move. The new position of theblack rook in Fig. 10 is permitted. As a result of killing, aDeadEvent is generated, and it excludes the white rook fromthe game by marking it as dead. The response delivered tothe application:{ response: yes , deadPiece: white_rook1 }

indicates which piece should be removed from the 3Dscene.

Figure 6: Selection of a piece.

a) b)

Figure 7: Prohibited moves of rooks: a) neither along the xnor along the y axis; b) to a �eld occupied by a piece withthe same color.

6.4 Semantic Game Log ExplorationAfter the completion of all steps leading to the end of the game,the KB includes the full log of all events and states that occurredduring the game. Such a comprehensive log can be in particularused to:

a) b)

Figure 8: Permitted moves of rooks along axes.

a) b)

Figure 9: Moves of rooks.

Figure 10: Killing the white rook.

(1) check if a move was possible at a moment in time duringthe game, e.g., if a player could win;

(2) check a possible scenario of the game at a moment in timeby sending extra queries simulating the opponent moves;

(3) query for all speci�c situations in the game, e.g., for teach-ing purposes;

(4) query for possible consequences of the desired player’smoves, e.g., for beginners.

A web-based repository of various game sessions could be cre-ated to provide tutorials and examples, which would be semanti-cally explorable with user queries, in particular, to complete theaforementioned use cases.

7 CONCLUSIONS AND FUTUREWORKSThe development of the current web towards the semantic webbeing a large distributed database of interconnected and explorablecontent, requires new methods and tools for building VR/AR ap-plications. The use of semantic web technologies gains increasingattention in the research community. However, the available meth-ods and tools do not enable creation of 3D content with dynamic

Knowledge-based Representation of 3D Content Behavior Web3D ’17, June 05-07, 2017, Brisbane, QLD, Australia

semantic representations, which could evolve over time togetherwith the represented content.

In this paper, a new knowledge-based representation of 3D con-tent behavior has been proposed. The representation leveragesthe event calculus, which is an important approach to creatingdeclarative applications in various domains such as AI and robot-ics. However, it can also be successfully used in web-based VR/ARenvironments.

The proposed representation can cover content behavior at thedomain-speci�c level and enables knowledge inference and explo-ration. This enables development of tools for domain experts whoare not IT-specialists. Moreover, in some cases it can reduce e�ortin 3D modeling by inferring new content properties and relations,and make the content suitable for precise and extensive searching.The proposed representation has been used in a service-oriented3D virtual environment to implement 3D chess game.

The possible directions of future research encompass severalaspects. First, the proposed representation can be combined withsemantic representation of the other VR/AR content features, suchas geometry, structure, space, appearance and markers—at the low(Flotyński and Walczak 2013b; Flotyński and Walczak 2014; Ru-miński and Walczak 2014a,b, 2017) and high (Flotyński and Walczak2013, 2014, 2016; Walczak and Flotyński 2015) levels of abstraction.This would provide a comprehensive 4-dimensional 3D contentrepresentation. Second, the approach could enrich the available3D content representation standards (e.g., VRML and X3D), e.g., inthe form of external declarative scripts describing behavior. Third,methods and tools enabling 4-dimensional manipulation of 3D con-tent (including time) with user queries can be developed. Next,the development of user-friendly graphical modeling tools couldbene�t from the syntax of the representation, which is relativelysimple in comparison to the syntax of imperative languages. Suchtools could liberate users from programming, which could furtherimprove the overall dissemination of 3D content among domainexperts. Finally, the representation should be evaluated in terms ofthe size and complexity of 3D content as well as the performanceof query execution and knowledge base manipulation.

ACKNOWLEDGMENTSThis research work has been supported by the Polish NationalScience Centre (NCN) Grant No. DEC-2012/07/B/ST6/01523.

REFERENCESAutodesk. 2017a. 3ds Max. (2017). http://www.autodesk.pl/products/

autodesk-3ds-max/overviewAutodesk. 2017b. Motion Builder. (2017). http://www.autodesk.com/products/

motionbuilder/overviewAway3D. 2017. Away3D. (2017). http://away3d.com/Sotiris Batsakis, Euripides Petrakis, Ilias Tachmazidis, and Grigoris Antoniou. 2009.

Temporal representation and reasoning in OWL 2. Semantic Web (2009), 1–20.http://www.semantic-web-journal.net/system/�les/swj855.pdf

Chiara Eva Catalano, Michela Mortara, Michela Spagnuolo, and Bianca Falcidieno.2011. Semantics and 3D media: Current issues and perspectives. Computers &Graphics 35, 4 (2011), 869–877.

Siddhartha Chaudhuri, Evangelos Kalogerakis, Stephen Giguere, and ThomasFunkhouser. 2013. Attribit: Content Creation with Semantic Attributes. In Proceed-ings of the 26th Annual ACM Symposium on User Interface Software and Technology(UIST ’13). ACM, New York, NY, USA, 193–202.

Olga De Troyer, Frederic Kleinermann, Bram Pellens, and Wesley Bille. 2007. Con-ceptual Modeling for Virtual Reality. In Tutorials, posters, panels and industrialcontributions at the 26th Int. Conference on Conceptual Modeling - ER 2007 (CRPIT),

John Grundy, Sven Hartmann, Alberto H. F. Laender, Leszek Maciaszek, and John F.Roddick (Eds.), Vol. 83. ACS, Auckland, New Zealand, 3–18.

Intel Visual Computing Institute DFKI, Computergraphics Lab of the Saarland Univer-sity. 2017. XML3D. (2017). http://xml3d.org/

Jakub Flotyński. 2014. Semantic modelling of interactive 3D content with domain-speci�c ontologies. Procedia Computer Science 35 (2014), 531–540. 18th Interna-tional Conference on Knowledge-Based and Intelligent Information & EngineeringSystems.

Jakub Flotyński and Krzysztof Walczak. 2013. Conceptual Semantic Representationof 3D Content. Lecture Notes in Business Information Processing: 16th InternationalConference on Business Information Systems, Poznań, Poland, 19 - 20 June, 2013 160(2013), 244–257.

Jakub Flotyński and Krzysztof Walczak. 2013a. Semantic Modelling of Interactive 3DContent. In Proceedings of the 5th Joint Virtual Reality Conference. Paris, France.

Jakub Flotyński and Krzysztof Walczak. 2013b. Semantic Multi-layered Design ofInteractive 3D Presentations. In Proceedings of the Federated Conference on ComputerScience and Information Systems. IEEE, Kraków, Poland, 541–548.

Jakub Flotyński and Krzysztof Walczak. 2014. Conceptual knowledge-based modelingof interactive 3D content. The Visual Computer (August 2014), 1–20. DOI:http://dx.doi.org/10.1007/s00371-014-1011-9

Jakub Flotyński and Krzysztof Walczak. 2014. Multi-platform Semantic Representa-tion of Interactive 3D Content. In Proceedings of the 5th Doctoral Conference onComputing, Electrical and Industrial Systems. April 7-9, Lisbon, Portugal.

Jakub Flotyński and Krzysztof Walczak. 2014. Semantic Representation of Multi-platform 3D Content. Computer Science and Information Systems 11, No 4, October2014 (2014), 1555–1580.

Jakub Flotyński and Krzysztof Walczak. 2015. Ontology-based Creation of 3D Contentin a Service-Oriented Environment (Lecture Notes in Business Information Processing,18th International Conference on Business Information Systems). Springer Verlag.

Jakub Flotyński and Krzysztof Walczak. 2016. Customization of 3D content withsemantic meta-scenes. Graphical Models (2016), in print. DOI:http://dx.doi.org/10.1016/j.gmod.2016.07.001

Jakub Flotyński and Krzysztof Walczak. 2017. Ontology-Based Representation andModelling of Synthetic 3D Content: A State-of-the-Art Review. Computer GraphicsForum (2017), 1–25. DOI:http://dx.doi.org/10.1111/cgf.13083

Blender Foundation. 2017. Blender. (2017). http://www.blender.org/Mario Gutiérrez, Alejandra García-Rojas, Daniel Thalmann, Frédéric Vexo, Laurent

Moccozet, Nadia Magnenat-Thalmann, Michela Mortara, and Michela Spagnuolo.2007. An Ontology of Virtual Humans: Incorporating Semantics into Human Shapes.Vis. Comput. 23, 3 (Feb. 2007), 207–218.

Xiaoxia Hu, Xuefeng Liu, Zhenming He, and Jiahua Zhang. 2013. Batch modeling of3D city based on ESRI cityengine. In Smart and Sustainable City 2013 (ICSSC 2013),IET Int. Conf. on. IET.

RuleML Inc. 2017. RuleML Home. On-line. (2017). Retrieved Feb 28, 2017 fromhttp://wiki.ruleml.org/index.php/RuleML_Home

Hao Jiang, Wenbin Xu, Tianlu Mao, Chunpeng Li, Shihong Xia, and Zhaoqi Wang.2009. A semantic environment model for crowd simulation in multilayered complexenvironment. In Proceedings of the 16th ACM Symposium on Virtual Reality Softwareand Technology. ACM, 191–198.

E. Kalogerakis, Stavros Christodoulakis, and Nektarios Moumoutzis. 2006. CouplingOntologies with Graphics Content for Knowledge Driven Visualization. In VR ’06Proceedings of the IEEE conference on Virtual Reality. Alexandria, Virginia, USA,43–50.

Robert Kowalski and Marek Sergot. 1989. A logic-based calculus of events. In Founda-tions of knowledge base management. Springer, 23–55.

Robert A Kowalski and Fariba Sadri. 1994. The Situation Calculus and Event CalculusCompared.. In ILPS, Vol. 94. 539–553.

Alex Kozlenkov and Adrian Paschke. Prova Rule Language. https://prova.ws/Marc Erich Latoschik and Roland Blach. 2008. Semantic Modelling for Virtual Worlds

– A Novel Paradigm for Realtime Interactive Systems?. In Proceedings of the ACMVRST 2008. 17–20.

Jean-Luc Lugrin and Marc Cavazza. 2007. Making Sense of Virtual Environments:Action Representation, Grounding and Common Sense. In Proc. of the 12th Int. Conf.on Intelligent User Interfaces (IUI ’07). ACM, New York, NY, USA, 225–234.

Microsoft. 2017. Direct3D 11.1 Features. (2017). https://msdn.microsoft.com/en-us/library/windows/desktop/

Oracle. 2017. Java3D. (2017). Retrieved Feb 28, 2017 from http://www.oracle.com/B. Pellens, O. De Troyer, W. Bille, and F. Kleinermann. 2005a. Conceptual Modeling of

Object Behavior in a Virtual Environment. In Proceedings of Virtual Concept 2005.Springer-Verlag, Biarritz, France, 93–94.

B. Pellens, O. De Troyer, W. Bille, F. Kleinermann, and R. Romero. 2005b. An Ontology-Driven Approach for Modeling Behavior in Virtual Environments. In Proceedings ofOn the Move to Meaningful Internet Systems 2005: Ontology Mining and Engineeringand its Use for Virtual Reality (WOMEUVR 2005) Workshop, R. Meersman, Z. Tari, andP. Herrero (Eds.). Springer-Verlag, Springer-Verlag, Agia Napa, Cyprus, 1215–1224.

Bram Pellens, Frederic Kleinermann, and Olga De Troyer. 2009. A DevelopmentEnvironment Using Behavior Patterns to Facilitate Building 3D/VR Applications.

Web3D ’17, June 05-07, 2017, Brisbane, QLD, Australia J. Flotyński, K. Walczak

In Proc. of the 6th Australasian Conf. on Int. Entertainment (IE ’09). ACM, Article 8,8 pages.

Fabio Pittarello and Alessandro De Faveri. 2006. Semantic Description of 3D En-vironments: A Proposal Based on Web Standards. In Proceedings of the EleventhInternational Conference on 3D Web Technology (Web3D ’06). ACM, New York, NY,USA, 85–95.

John Pool. 2017. C#Prolog - A Prolog interpreter written in managed C#. (2017).Retrieved Feb 28, 2017 from https://sourceforge.net/projects/cs-prolog/

Pierre-Yves Rabattu, Benoit Massé, Federico Ulliana, Marie-Christine Rousset, DamienRohmer, Jean-Claude Léon, and Olivier Palombi. 2015. My Corporis Fabrica Embryo:An ontology-based 3D spatio-temporal modeling of human embryo development.Journal of Biomedical Semantics 6, 1 (2015), 36. DOI:http://dx.doi.org/10.1186/s13326-015-0034-0

Dariusz Rumiński and Krzysztof Walczak. 2014a. Semantic Contextual AugmentedReality Environments. In The 13th IEEE International Symposium on Mixed andAugmented Reality (ISMAR 2014), ISMAR 2014. IEEE, 401 – 404. DOI:http://dx.doi.org/10.1109/ISMAR.2014.6948506

Dariusz Rumiński and Krzysztof Walczak. 2014b. Technological Innovation for Collec-tive Awareness Systems, in: IFIP Advances in Information and Communication Tech-nology, Luis M. Camarinha-Matos, Nuno S. Barrento, and Ricardo Mendonça (Eds.),Vol. 432. Springer, 183–190. DOI:http://dx.doi.org/10.1007/978-3-642-54734-8_21

Dariusz Rumiński and Krzysztof Walczak. 2017. Semantic Model for DistributedAugmented Reality Services. In Proceeding Web3D 17 Proceedings of the 22th Inter-national Conference on 3D Web Technology, Brisbane (Australia), June 5 - 7, 2017,Accepted for publication. ACM New York.

Stuart Russell, Peter Norvig, and Arti�cial Intelligence. 2009. A modern approach.Pearson.

Murray Shanahan. 1999. The event calculus explained. In Arti�cial intelligence today.Springer, 409–430.

Michela Spagnuolo and Bianca Falcidieno. 2009. 3D Media and the Semantic Web.IEEE Intelligent Systems 24, 2 (2009), 90–96.

Unity Technologies. 2017. Unity. On-line. (2017). Retrieved Feb 28, 2017 from http://unity3d.com/5

Trimble. 2017. SketchUp. (2017). http://www.sketchup.com/Tim Tutenel, Rafael Bidarra, Ruben M Smelik, and Klaas Jan De Kraker. 2008. The

role of semantics in games and simulations. Computers in Entertainment (CIE) 6, 4(2008), 57.

Luc Van Gool, Bastian Leibe, Pascal Müller, Maarten Vergauwen, and Thibaut Weise.2007. 3D Challenges and a Non-In-Depth Overview of Recent Progress. In 3DIM.118–132.

W3C. 2004. SWRL. On-line. (2004). Retrieved Feb. 28, 2017 from http://www.w3.org/Submission/SWRL/

W3C. 2017a. Getting Started with X3D. (2017). http://www.web3d.org/getting-started-x3d

W3C. 2017b. VRML Virtual Reality Modeling Language. (2017). https://www.w3.org/MarkUp/VRML/

Krzysztof Walczak and Jakub Flotyński. 2014. On-Demand Generation of 3D ContentBased on Semantic Meta-Scenes. In Lecture Notes in Computer Science; Augmentedand Virtual Reality; First International Conference, AVR 2014, Lecce, Italy, September17-20, 2014. Springer International Publishing, 313–332.

Krzysztof Walczak and Jakub Flotyński. 2015. Semantic Query-based Generationof Customized 3D Scenes. In Proceedings of the 20th International Conference on3D Web Technology (Web3D ’15). ACM, New York, NY, USA, 123–131. DOI:http://dx.doi.org/10.1145/2775292.2775311

Krzysztof Walczak, Dariusz Rumiński, and Jakub Flotyński. 2014. Building ContextualAugmented Reality Environments with Semantics. In Proocedings of the 20th Inter-national Conference on Virtual Systems & Multimedia, Hong Kong, 9-12 September.

Dennis Wiebusch and Marc Erich Latoschik. 2012. Enhanced Decoupling of Com-ponents in Intelligent Realtime Interactive Systems using Ontologies. In SoftwareEngineering and Architectures for Realtime Interactive Systems (SEARIS), proceedingsof the IEEE Virtual Reality 2012 workshop.