Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
INFO310 0 Advanced Topics in Model-Based Information Systems
Kandidat 108
Oppgaver Oppgavetype Vurdering Status
Introduction Dokument Automatisk poengsum Levert
Plagiarism and Declaration Dokument Automatisk poengsum Levert
1 Essay Filopplasting Manuell poengsum Levert
INFO310 0 Advanced Topics in Model-Based Information Systems
Emnekode INFO310Vurderingsform INFO310Starttidspunkt: 03.11.2016 14:00Sluttidspunkt: 09.11.2016 14:00Sensurfrist Ikke satt
PDF opprettet 16.11.2016 16:36Opprettet av Andreas Lothe OpdahlAntall sider 19Oppgaver inkludert NeiSkriv ut automatisk rettede Nei
1
Seksjon 1
1 OPPGAVE
Essay
Upload your file here. Maximum one file.
BESVARELSE
Filopplasting
Filnavn 9066477_cand-3238752_9157556
Filtype pdf
Filstørrelse 1088.801 KB
Opplastingstid 08.11.2016 15:03:31
Neste side
Besvarelsevedlagt
Kandidat 108
INFO310 0 Advanced Topics in Model-Based Information Systems Page 2 av 19
INFO310 – Advanced Topics in Model-Based Information Systems
Utilizing the data from biofeedback-capable gaming equipment Candidate: 108
Words: 5950 | Pages: 16
08.11.2016
Kandidat 108
INFO310 0 Advanced Topics in Model-Based Information Systems Page 3 av 19
1
Contents Introduction ............................................................................................................................................. 2
Affective gaming and biofeedback mechanisms ..................................................................................... 3
Real-time usages...................................................................................................................................... 4
Games (affective games) ..................................................................................................................... 4
E-sports ................................................................................................................................................ 5
Health .................................................................................................................................................. 6
Posterior usages ....................................................................................................................................... 6
Games .................................................................................................................................................. 7
E-sports ................................................................................................................................................ 7
Health .................................................................................................................................................. 7
Big data ................................................................................................................................................... 7
Personal data........................................................................................................................................ 8
Vitals data ............................................................................................................................................ 8
Game data ............................................................................................................................................ 8
Environmental data .............................................................................................................................. 9
Technological data .............................................................................................................................. 9
Ontologies ............................................................................................................................................... 9
Medical .............................................................................................................................................. 10
SS4RWWR .................................................................................................................................... 10
OBO Foundry’s ontologies............................................................................................................ 10
VSO ............................................................................................................................................... 10
MEDDRA ...................................................................................................................................... 11
Personal ............................................................................................................................................. 11
Game ................................................................................................................................................. 11
Framework ............................................................................................................................................ 12
Privacy ................................................................................................................................................... 13
Conclusion ............................................................................................................................................. 13
Literature ............................................................................................................................................... 14
Resources .............................................................................................................................................. 16
Illustrations ........................................................................................................................................ 16
Ontologies ......................................................................................................................................... 16
Social media contribution ...................................................................................................................... 16
Kandidat 108
INFO310 0 Advanced Topics in Model-Based Information Systems Page 4 av 19
2
Introduction It is over a half decade since what we know as the internet of things (IoT) was born, by the estimations
of Cisco (Evans, 2011). For some people the abbreviation IoT will make them think of their car, with
all the sensors that have been implemented over the years. For others, it can be smart TVs, smartphones,
coffee-machines, refrigerators, wearables or smart homes. A field that haven’t been affected too much
by the IoT yet, is the world of video games. E-sports, gaming communities and the number of people
playing and watching games are growing each day. It has grown such that several large sports clubs
around the world has started to pick up E-sports teams, as illustrated by Mike Kent’s article of
professional sports teams with e-sport teams (2016).
The last few years have given the gamers some technologies to play with, VR headsets are being released
one after the other, Oculus Rift, HTC Vive and the PlayStation VR are some examples. AR-glasses is
also worth mentioning, namely Google Glass and Microsoft HoloLens, although neither have been
released for the public masses yet. A new product that recently made its entrance to the world, will
hopefully pave the way for some exiting techniques and technology that the world of gaming and the
health sciences might benefit from, and drive VR-gaming to a whole new level.
The product is the NAOS QG (NQG), a gaming
mouse produced by the Swedish company Mionix.
Mionix is not new to the area of producing gaming
mice, but there is something special about the NQG,
where the QG stands for Quantified Gaming
(Mionix, 2015). What is special about it is that it has
additional sensors built into it, as seen in illustration
1, in addition to the normal optical laser sensor and
standard buttons. The NQG has a sensor for detecting
heart-beat and pulse and a sensor that notices
electrical properties in the skin, with a GSR
(Galvanic Skin Response) sensor. GSR captures
conductive changes in the skin, which changes based
on sweat (Peuscher, 2012).
Keyboards, mice and controllers are devices with
sensors and buttons. The “actuators” of the computer
makes use of the information given from these
devices, and translates it to navigating or performing
an action within the operating system, or sending the
information along to another application. These are
basic input-output (I/O) operations, that have existed
for many years. Extend these operations to utilize
input of other types of information and we can get
much more immersive experiences in gaming, and
contribute to e-health and science in the medical
fields. The data devices like the NQG produce, and
could in the future produce, what can we do with
them? And how do we utilize them in the best
manner possible?
Illustration 1. The Mionix NAOS QG
Kandidat 108
INFO310 0 Advanced Topics in Model-Based Information Systems Page 5 av 19
3
Affective gaming and biofeedback mechanisms To fully comprehend some of the usages of the data produced by devices like NQG, it is essential to
understand the concepts of affective gaming and biofeedback mechanisms.
Allanson et al defines affective gaming as “where the player’s current emotional state is used to
manipulate gameplay” (2005). In addition to the player’s conscious decisions and choices being utilized
to make changes in the game, we also utilize the unconscious emotional states of the player. The way
we access these unconscious emotional states is through what is called biofeedback mechanisms, which
means that we are monitoring the player’s vitals and physiological measurements. An important thing
Allanson et al mentions is that by using biofeedback mechanisms simply as another method of input as
a controller, it is merely a biofeedback-game. An affective game needs to utilize the biofeedback to
manipulate the game itself based on the emotional states of the user, just as if the game truly understood
the users emotional state and reactions.
Torres explains biofeedback in his master thesis as: “Biofeedback is the ability of self-regulating a
person’s biological or physiological functions by gaining greater awareness of them with the user of
instruments that provide information on those same systems” (2013). He then goes on explaining that
biofeedback can be divided into two subgroups: indirect and direct biofeedback: “Direct biofeedback
consists on conscious physiological function such as constraining a muscle, and indirect biofeedback
corresponds to an unconscious body action such as heart rate or respiration.” The indirect biofeedback
is what seems most suitable for affective gaming to utilize, as this is where the unconscious changes are,
which means that this is where the emotional states are possible to analyze.
Champion and Dekker made some interesting research on the topic of biofeedback mechanisms almost
ten years ago, which identified potential, problems and issues with the technology (2007). They used
sensors clipped to the user’s fingertips, and fed the biometric data into the game to dynamically change
visuals of the game, effects like screen shake and to generate the spawn of the NPC’s (Non-playable
character) which in this case was zombies. Additionally, they changed the speed of the playable
character based on heartbeat, and the volume of sound based on the user’s skin response, and more.
Kalyn et al did further work based on amongst other Champion and Dekker, and published a paper on
direct and indirect physiological control in games (2011). They basically made parts of the body replace
controller-input that changed things in the game, using both direct controllers like muscles to perform
actions, and indirect like heartbeat. The results they produced clearly showed them that indirect
controllers are not viable as a controller of actions in the game, which wasn’t surprising. The indirect
measurements should rather be used behind the scenes, changing scenery and other things that the player
isn’t in control of. The direct controllers have potential usages, for instance as being used for controlling
the player’s character.
In both projects the sensors had to be attached to the user, one could argue that most people wouldn’t
deal with the hassle of strapping on a bunch of sensors to access this “next level” of gameplay, and this
is where the NQG and similar devices make a big difference. These devices are also why it is time for
the themes of biofeedback mechanisms and affective gaming to re-emerge and make its entrance to the
world of video games. Major players in the game industries have been loking into these fields, for
instance Valve has been investigating how biofeedback mechanisms impacts cooperative player
experiences on major titles like Portal 2 and Left 4 Dead 2 (Perron and Schröter, 2009). There are likely
several reasons why affective gaming hasn’t been introduced to video games on a full scale yet, one of
which clearly is solved to some degree by the NQG. Mionix has also made it easy with a free and open
API, that developers could utilize (Mionix, 2015). The other reasons revolve around implementations,
mechanics, logic and how to best utilize the data. We will look closer at how to utilize the data, what to
use them for and how to use them in the most useful manner.
Kandidat 108
INFO310 0 Advanced Topics in Model-Based Information Systems Page 6 av 19
4
There are lots of ways to utilize and handle the data produced by these kinds of products, and there are
two main groups that will be in focus. Firstly, the real-time usages, what you can do with the data right
there and then. Secondly, the posterior usages, which is more of the analytic usages and what one can
do with the data generated from the real-time usages.
Real-time usages
Games (affective games) The game developers can take advantage of all these sensors to change the user-experience within the
game. It can be used to change the storyline, get the users emotional state to fire events, make the
NPC/AI characters adapt to the user and make the games optimize the gameplay further for the current
user. The game developers can build onto the standard I/O operations, with physical sensor giving input
and the game producing some output.
For the storylines, the developers could make use of the user’s reaction to a certain event, and to more
deeply understand how he/she responded and what emotional impact it had, with that information they
would be able to tailor the storyline in a direction the user might enjoy more. The game could build a
profile on that user, while constantly learning more information and adapting the game in a more
sophisticated way. There are also usages which doesn’t need to change the whole story of the game, for
instance one could also benefit from the sensors and fire events with perfect timing based on the actual
emotional state the user is in, which is more in the style of what for instance Champion and Dekker did.
In a horror game, the developers could program the game to fire off that jump-scare with perfect timing.
If the game has learned that the user’s average BPM (beat per minute) is around 70, it might fire off a
scary event when the user is approaching 120 BPM. This is just one example of how the game could
fire off certain events, and make the NPC/AI characters adapt to the user.
These aspects build on what the game-industry is already doing a lot of today, giving the users choices
to make in-game, and then change the story based on their choices. Which is a good thing, as it adapts
the game to the playstyle of the user. The user is although able to manipulate it to his liking, which is
hard to do with this sensor based game manipulation. With the game utilizing both conscious choices
and unconscious reactions/emotions, one could see even further personalized and unique gameplay.
Kandidat 108
INFO310 0 Advanced Topics in Model-Based Information Systems Page 7 av 19
5
E-sports This is what Mionix seemed to have in mind, when they released their campaign to make the mouse.
Illustration 2 shows us an image from their Kickstarter campaign, with a HUD displaying the vitals of
a team. It does not illustrate affective gaming, but regular biofeedback within a game.
Ingo Froböse is a professor at the German Sport University Cologne, and he has spent some time
studying e-sports, and has found that professional e-sports players are “real athletes” in the sense that
they experience a lot of the same physical strains that athletes in traditional sports do. For instance, they
sometimes achieve pulse rates close to what of a person running a marathon, and cortisol levels at the
level of a race-car driver (Schütz, 2016). In traditional sports on professional level the athletes often
have a support team, to help them improve their performance. Doctors, coaches, psychologists, personal
trainers, lifestyle coaches, there is a long list of professions that can help the athletes. The point is that
to improve most efficiently the athletes need help, and quantified gaming are a tool the “electronic
athletes’” can use for their improvement. The idea here is that on-screen while in the game, they have a
HUD (head-up display) that presents the player information about himself, and potentially his
teammates. So, they can see their own and the team-members’ physiological states. Which might result
in better team play, as one of the team members can assist one that is stressing out. On a personal level
the players can also identify scenarios that stresses them out, which could help them work on remaining
calmer in similar situations later. Stressing out can often lead to “hot-headed” actions and decisions,
which might produce negative results.
Hand in hand is the casting of e-sports events, they are able to benefit from these sensors as well, as they
are able to analyze deeper aspects of the game. They can see when a player is stressed and prone to make
mistakes, it adds another layer of information for them to serve to the audience. This has been done at
for instance Dreamhack (Swedish LAN party), where they showed the pulse rates, as can be seen in
illustration 3.
Illustration 2. How Mionix envisions a team being able to see each other’s vitals and stress levels.
Kandidat 108
INFO310 0 Advanced Topics in Model-Based Information Systems Page 8 av 19
6
Health There are of course also health applications to this technology, which both the medical field and the
players can benefit from. For instance, the games could incorporate some health functionality, in that
they could warn users of abnormalities from their averages, as well as monitor them over longer periods
of time. The horror games could even choose not to scare the player too much, if the person for instance
has a heart-condition or something similar. This also goes for operating systems; in that they could also
incorporate this technology. Additionally, the fact that players are able to monitor themselves could also
help them take breaks when they get too stressed out, and taking brakes from the computer in general
as most people in the medical fields recommend.
Posterior usages The data produced has the potential of being used for more than showing HUD’s and serving real-time
information, what can be extracted of the information posterior to the playing? For instance, for games
it would be necessary to implement some functions that would save the information, preferably both the
information given by the NQG, but equally important information about the game-states. As the data
about our vitals is itself not enough to utilize some of the concepts that will be described. But the vitals
data, alongside game-data, possibly combined with personal data and possibly others as well, will help
us puzzle together more pieces of the story.
If one were to use the data for game development optimization by learning how the users react to the
game, it would be key to identify the different aspects that are happening when a game is being played,
to fully utilize the potential.
Illustration 3. Heart rates visualized for the audience at Dreamhack in a StarCraft II match
Kandidat 108
INFO310 0 Advanced Topics in Model-Based Information Systems Page 9 av 19
7
Games The posterior-data could give detailed analytics to the developers, so that they could make improvements
to the existing game. They could learn about how to deal with problems and how to best solve them for
future games as well. The potential could be huge for the companies that release a lot of games, they
could create a huge game-knowledgebase generated by the games they make.
As mentioned earlier, one could make the games learn about the user and tailor the rest of the story to
player’s liking. Machine learning is also a relevant topic for the posterior data, this might be a tad far-
fetched for today’s technology, but image in a few years; Games that uses the posterior data could learn
not to only improve the local story for the local user, but also self-improve the game itself for all users.
E-sports For e-sports it is pretty much all about the analytics, how to improve the teams or the players
performance. Over time with the posterior data they could see improvements/decrease in performance,
analyze what types of teams they perform badly against, and why that is. As mentioned, all the answers
will not be given by simply utilizing the data about the player’s vitals, but it should be combined with
several other key elements.
Health E-health is no new term, dating back to the end of the last millennium. According to Eysenbach, it was
already in 1999 a “buzzword” for generally anything that could relate to computers and medicine (2001).
And some years later, a group found 51 different definitions of the word in different published works
(Enkin, et al., 2005).
In the future, people will most likely not have to go to their general practitioner to find some details
about their health, most likely our vital data will be sent directly to a machine or system for analysis.
The machine could then notify our doctor if there were something that needed attention, or it could send
an overview that our doctor would only need to brief through. In this scenario, the doctors would of
course have access to the data to see for themselves, at least that seems appropriate in the early stages
of the technology. As one might not make these systems analyze the data perfectly, in the early stages.
E-health-systems will be thought of as a system with parts where some device (A) monitors a person
(B) and sends the information along to system (C), and then a doctor/medical professional (D) can look
at it. (D) can then be the person’s general practitioner. There are already existing systems that does
similar things, for instance Qualcomm’s 2net system (Qualcomm, 2016).
Big data There are other sensors that in the future could be implemented, and taken advantage of. One could
utilize the camera to analyze facial expressions, maybe the microphone could be used for analyzing
respiratory rates by listening to the breath? A thermometer could help us better understand the user’s
surroundings, for instance. Without going into too much detail, as this is a somewhat fresh field and
identifying all the potential use cases and instruments that could benefit the various systems would likely
require teams of cross-discipline experts. I will mention some types of data that could be used, in the
upcoming paragraphs, merely to illustrate some of the potential. The game-knowledgebase mentioned
earlier, is an example of why these data could be used for big data generation, and then one could create
knowledgebases with the big data as a fundament. That there are enough amounts of data to generate so
called big data, is taken for granted, as this is hypothetical in the sense that “in some years, the sensors
will be implemented in all equipment and most applications will utilize the data.”
Kandidat 108
INFO310 0 Advanced Topics in Model-Based Information Systems Page 10 av 19
8
The question arises, what kind of data should we collect? It depends on the goal of what one wants to
use it for, game and health related big data is the two that might be closest to collecting the same data
as illustrated in illustration 4. The illustration shows some categories of data that is likely to be included
for learning about games, if one were to remove for instance the game data part, then it might look closer
to something to be used in a pure health perspective.
Personal data Data about the player, one might argue that it would be easiest for the game to link to some other
platforms to access what it needs. Age, height, weight, the general personal data, are examples of data
that could be collected, there is no doubt the data is powerful in discovering patterns and trends, and
generally being useful in the scenario of getting answers. It does although raise some potential privacy
concerns, so the player should be able to opt out, and the data collected shouldn’t be too specific.
Vitals data Data about our vitals from NQG-like devices could play a big role alongside wearables, for big data
generation for usages in the medical sciences. It involves heart rates, pulse, skin response and all these
things that we potentially can measure and build into devices. For the generation of big data solely
around health, it would likely be statistics/averages consisting of measurements at certain points,
mapped with other data, like timestamps, information revolving the measurement and such. Within a
game data system, it would likely be mapped with events and specific points within the game.
Game data Data about the game, its states, possibly generated player profiles based on choices the player has made
and the playstyle. Information about events mapped together with the vitals data, to more deeply
understand the effects the game has on the user.
Illustration 4. Some types of data that could be used for the purposes of big data generation
Kandidat 108
INFO310 0 Advanced Topics in Model-Based Information Systems Page 11 av 19
9
Environmental data Environmental data is another type of data that could be useful to look at for other purposes as well.
Environmental data is here thought of as things concerning the user’s environment, like the dates, times,
weather, temperatures and all these types of data. It could be useful to look at to get more context and
details that could explain why the user had that specific reaction, if for instance it were an unusual
reaction compared to the majority for an event within the game.
Technological data Data from sensors that is utilized, hardware, maybe also data about the software could prove itself useful
to collect. It could certainly be useful to see where the data is coming from, one could for instance
identify sensors that are not doing an optimal job. And come up with co-relations between the sensors
and the measurements they do, we will also see in the next section that sensors, like many electronic
devices, are prone to errors and might malfunction, which could be problematic.
The details around the types of data is somewhat vague, which is alright to illustrate potential data one
could collect, and how one should reason around why exactly that kind of data should be collected.
Specifying all the data that is important to collect is difficult without having any knowledge and
experience within neither the fields of game development or health. This is where one could bring groups
of experts from various fields together, with the fields depending on the usage. This group could then
work together to identify what data should be collected and how to best utilize it.
The big data itself can answer questions, but there is no reasoning with it. To be able to answer complex
questions, and do reasoning on the data, the big data collections should be mapped towards appropriate
ontologies.
Ontologies To most efficiently utilize the data collected, an appropriate ontology with which we could map the data
is essential. What types of ontologies are suiting for these kinds of data? The medical field has already
utilized semantic technologies for some time, and therefore have some useful parts to utilize. In the next
sections, we will see different existing ontologies that is applicable to some degree, now it doesn’t
necessarily mean that these are the best or only ontologies that are usable. It depends on what one wishes
to achieve, if it is within the medical fields, game development, psychology or social sciences. One can
tailor an ontology and reuse from these ontologies to represent what is most beneficial to one’s project.
Since ontologies have been used in the medical field for some time already, it is certainly the field with
the most available resources, and the wearable wave that came a few years ago haven’t decelerated the
growth. NQG-like devices have many similarities to wearables, although the users aren’t wearing the
mouse, it is still sensors built into a device which we “interact” with, most publicly available wearables
today take the shape of a watch, earbuds or similar. Wearables like the smart-watch is great for
monitoring everyday health, fitness, sleeping habits, and has a lot of real world appliances like
navigation, exercise motivators and so on. The appliances of devices like the NQG are to a degree
defined by the ecosystem in which they take a part. The NQG then takes the part of the PC and gaming
environment, naturally it is a part of the PC environment as it takes form of a computer mouse and it is
a part of the gaming environment, as it was intended to.
The point being, the data produced by wearables like the watch, who can be on you at most times, is for
instance highly valuable in the medical fields. The data produced by the NQG, can also be valuable for
Kandidat 108
INFO310 0 Advanced Topics in Model-Based Information Systems Page 12 av 19
10
the medical field as those who use it the most, might also be the persons in the more vulnerable positions
when it comes to health and fitness. One can argue that people who are concerned about health and
fitness are more likely to buy a smart-watch to monitor their vitals, than someone who isn’t as concerned
about health and spends more time in front of the computer monitor. In that way, NQG-like devices fills
an important gap where the smart-watches and fitness-bands might not reach a customer group.
In both cases, there are enormous amounts of data that could be produced, and semantic technologies
can help us make sense of it all. Drira et al wrote that the IDC (International Data Corporation) stated
in 2013 that less than 5% of information in the digital universe was analyzed, because one knew little
about the data. Drira wanted to illustrate that semantic enrichment can help us with that problem (2015).
Medical Fitting, usable and potential ontologies to use for the kind of data produced is majorly consisting of
ontologies for e-health, biology, physiological and medical fields. Remember that e-health was a
buzzword back in the start of this millennia, so there has been a lot of work in the medical fields, with
information technologies. Therefore, I will present some of the ontologies that seems appropriate to
potentially utilize for different projects.
SS4RWWR
In Ortuño and Rojas collection, Banos et al has an article revolving an ontology for sensor selection in
wearable activity recognition (2015). In the article, they argue about the error margins for these kinds
of sensors. And how most solutions and applications are created in such a manner that every tiny sensor
must function, which ideally they would, but that isn’t necessarily always the case. The sensors is often
small and to some extent fragile, and Banos et al presents an ontology which they named SS4RWWR
(Sensor Selection for Real-World Wearable Activity Recognition) to manage information about the
sensors, where are the sensors located on the body, what kind of data do they produce. They argue that
the ontology will help the system for instance provide an adequate type of data if one sensors were to
fail, from one of the other sensors that might capture something similar, or possibly even the same data.
Surely something of the like would be useful for the medical field, but also game developers could
borrow some of the features.
OBO Foundry’s ontologies
The OBO Foundry is a collective of ontology developers that produce several ontologies, with focus on
it being open and the ontologies sharing the same principles. A list of all their ontologies can be found
at obofoundry.org, as well as more information about their organization. The HDO (Human Disease
Ontology) and CDO (Cardiovascular Disease Ontology) are for instance two interesting ontologies,
which could be utilized to potentially warn the user about potential dangers or just contribute as a part
of an e-health-system like 2net. Another relevant ontology is the emotion ontology, which can describe
various emotions, moods and affective phenomena, as stated at their GitHub page (Hastings, 2015). The
ontology would be a great addition as to map the recorded vitals and related information to actual
emotional states.
In more general the OBCS (Ontology of Biological and Clinical Statistics) looks like a candidate to be
used for instance to generate Big Data for the Watson Project, or 2net-like systems. IBM has a project
where they allow hospitals to access Watson’s power, to diagnose patients (Otake, 2016). Surely the
collected data in these kinds of projects could be put to good use, with or without the OBCS in specific.
VSO
VSO (Vital Sign Ontology) covers blood pressure, body temperature, respiratory rates and pulse rates,
with the possibility to build further on it. It seems like a nice foundation for the type of data we can
collect now, and the data we could potentially collect in the future. Used with the emotion ontology, one
Kandidat 108
INFO310 0 Advanced Topics in Model-Based Information Systems Page 13 av 19
11
would perhaps be a few steps into a project that could help us understand the users’ reactions to the
game. Arbandi et al explains that VSO also follows the OBO Foundry’s guidelines, and interoperates
with OGMS (Ontology for General Medical Sciences) (2011).
MEDDRA
MEDDRA (Medical Dictionary for Regulatory Activities) is an ontology which revolves around
medical concepts, and contains such entities as heart rate, heartbeats increased and similar concepts. It
could prove to be useful for instance to make concepts a little easier to understand to non-medical
persons, as for instance the VSO are more specific about which artery the pulse was monitored from.
Personal Most ontologies revolving around personal information, seems to be more of the kinds that tracks
information for the user, to make various digital assistants. For instance the PIMO (Personal Information
Model Ontology), which has entities like things, tasks, events, locations and similar concepts. So
DBpedia’s “person” is what seems most appropriate to use with personal information, and one could
add additional information if there are concepts not covered. The privacy issues mentioned in the big
data part, is also existent here. One could co-relate the personal information with for instance medical
information and psychologic information. Some personal information should be stored, so the game
developers can ask questions on a deeper level concerning the users, and to do that one needs at least
some information about the users. For the medical fields, it could also be useful in seeing patterns
amongst users who share some attributes.
Game The GOP (Game ontology project) has a lot of usable information on describing, analyzing and studying
games (Whitehead, 2007), although it seems not much work has been done in the last few years. It
appears solid enough so that it is certainly possible to reuse, at the very least some of the parts and
concepts. It could also be utilized in the mechanics in affective games, as it contains inputs, game rules,
entity manipulations and things that could be adapted to the player. If one looks towards big data, one
would although need to define a sort of game event in addition, to map certain event or points in the
game towards the users emotional and biological states. For this one could utilize parts of the VGO
(Video Game Ontology) whose main goal is to “capture knowledge about events that happen in video
games and information about players” (Garijo et al, 2014). VGO capture some of what GOP doesn’t,
and VGO have defined different types of evens, which would be required to learn the most about the
gathered data. Both ontologies contain relevant information, that could be utilized.
Kandidat 108
INFO310 0 Advanced Topics in Model-Based Information Systems Page 14 av 19
12
Framework
Illustration 5 shows a diagram of a simple flowchart of what the data extraction could like in a
framework for a game-knowledgebase generation and how one could incorporate some of the
ontologies. The game would have to record its various events, and each time an event happened it would
have to send the data along. To make the most use out of the medical ontologies, Hodges et al showed
how medical ontologies could benefit from being able to share the information they model, and can
integrate that with wearable devices (2014). In this data extraction process, it is thought of as if the
ontologies could share that information.
The green circles are health related, and firstly we figure out what is being monitored, and where, then
the VSO can help us define it, and then MEDDRA can output a “something changed”, all the appropriate
data from each element is sent along. The GOP and VGO tells us what was going on in the game at the
time. The blue ones are about “the real world”, what devices are available, what is the user’s
surroundings/environment? Information about the person is the standard personals, but as this goes
through the event cycles in the game, one could build more detailed analysis of/profile on the user each
time. The emotion ontology would have to get the available information in order to analyze the users
emotional state, and then respond. From there one could generate a knowledgebase, and add onto it each
time, in order to later do reasoning on it. Or one could generate big data, and get statistics, visualizations
and so on. This is merely an example of what one of the parts of a framework for utilizing the data could
potentially resemble, if one wanted to know about how the user reacts to certain aspects within a game.
Illustration 5. Diagram of the simple data extraction flow description, what it might look like
Kandidat 108
INFO310 0 Advanced Topics in Model-Based Information Systems Page 15 av 19
13
The tricky part is to draw the lines of what data to collect, what data proves to be useful and what
doesn’t? For a full-scale project, you would likely need experts from a handful of different areas, in
order to clearly be able to define exactly what you want to utilize. A good starting point for further work
would although be to utilize the medical and game ontologies.
Privacy As in many other topics relating data collection, privacy is an essential issue, and it is also existent here.
Now there are certainly work-arounds that will make this less of a problem, for instance one could give
the users the options to opt out of giving personal data at all, or to allow the game to use it in real time,
but not save the data for usage in further analysis with big data. Asking the user for what data that the
user is willing to share, is what Android is doing, when the application asks for permission to read
information on the phone, similar concepts could be implemented in the games. It is also important to
remember to anonymize the data, very detailed information isn’t necessarily needed. For the e-health
systems like 2net, the doctor would benefit from knowing who the patient is, although it could be
implemented in such a way that the data is anonymized. Then, when the patient sees the doctor, the
doctor will get the appropriate data with for instance comments that he or some other doctor has made.
The summoning of the patient could be done electronically, so that the doctors schedule is automatically
generated for these types of situations.
Conclusion For now, we should build games as we always have, but adding the layers of affective gaming,
biofeedback mechanisms and data gathering on top of the game. In this way the game will function even
if the user does not have any of the required devices to utilize affective gaming. The technology and
techniques described will probably, need some time in to be realized. That could change over time, but
the first problem as of now is firstly that the NQG has just recently been publicly released. Secondly,
the equipment is more expensive due to the included sensors, so many might not see the usages yet and
will not buy it. Thirdly it might take some generations before all potential/appropriate sensors has been
incorporated into the mouse, with very accurate measurements and durable sensors. Additionally,
affective gaming might not become a natural part of games until VR and AR have a few generations of
products and see the use for incorporating biofeedback mechanisms, for instance.
Affective gaming with affective feedback would require some extra effort of the game developers, but
is certainly doable when it comes to implementations. The issue here might depend more on the available
technology and devices, and potential standards of the data transmitted. Implementing mechanics to
handle data from hundreds of different NQG-like devices, with different data inputs could prove to be
difficult.
Biofeedback gaming is a reality already, the NQG works at least for one person. Further work is to for
instance making it so that whole teams can watch each other’s vitals. And incorporate the tools needed
for them to analyze their games, for broadcasting/streaming many of the same or similar tools will help
them analyze the games they cast in much more detail.
Semantic technologies have the potential to improve normal and affective gaming, and biofeedback-
capable gaming equipment can help us on our way to these improvements. It can help game developers
understand the players better, and in general make better games. It could help psychologists better
understand our minds. It could improve our lives through e-health systems, and big data for medical
science. There are more usages for the data, that is guaranteed, the question remains of what these usages
are.
Kandidat 108
INFO310 0 Advanced Topics in Model-Based Information Systems Page 16 av 19
14
Literature
Allanson, J., Dix, A., Gilleade, K. (2005) Affective Videogames and Modes of Affective Gaming: Assist Me,
Challenge Me, Emote Me [Internet] Digital Games Research Association
Available from: http://www.digra.org/wp-content/uploads/digital-library/06278.55257.pdf
[Downloaded 25.10.16]
Arabandi, S., Brochhausen, M., Goldfain, A., Hogan, W., Smith, B. (2011) Vital Sign Ontology [Internet]
University of Buffalo
Available from: http://ontology.buffalo.edu/smith/articles/Vital_Sign_Ontology.pdf
[Downloaded 14.10.16]
Champion, E., Dekker, A. (2007) Please Biofeed the Zombies: Enhancing the Gameplay and Display of a
Horror Game Using Biofeedback [Internet] Digital Games Research Association
Available from: http://www.digra.org/wp-content/uploads/digital-library/07312.18055.pdf
[Downloaded 25.10.16]
Drira, K., Exposito, E., Mezghani, E., Silviera, M. (2015) A Semantic Big Data Platform for Integrating
Hetereogenous Wearable Data in Healthcare [Internet] Springer
DOI: 10.1007/s10916-015-0344-x
[Downloaded 14.10.16]
Enkin, M., Jadad, A., Oh, H., Rizo, C. (2005) What is eHealth(3): A Systematic Review of Published
Definition [Internet] Journal of medical internet research
DOI: 10.2196/jmir.7.1.e1
[Downloaded 21.10.16]
Evans, D. (2011) The Internet of Things [Internet] Cisco
Available from: http://www.cisco.com/c/dam/en_us/about/ac79/docs/innov/IoT_IBSG_0411FINAL.pdf
[Downloaded 28.10.16]
Eysenbach, G. (2001) What is e-health? [Internet] Journal of medical internet research
DOI: 10.2196/jmir.3.2.e20
[Downloaded 21.10.16]
Garijo, D., Parkkila, J., Poveda, M., Radulovic, F. (2014) The Video Game Ontology [Internet] Linkeddata.es
Available from: http://vocab.linkeddata.es/vgo/
[Downloaded 19.10.16]
Hastings, J. (2015) Emotion Ontology [Internet, GitHub page] Github.com
Available from: https://github.com/jannahastings/emotion-ontology
[Downloaded 22.10.16]
Hodges, J., Kritzler, Michahelles, F., Lueder, S., Wilde, E. (2014) Ontology alignment for wearable devices
and bioinformatics in professional health care [Internet] The Semantic Web Journal
Available from: http://www.semantic-web-journal.net/system/files/swj826.pdf
[Downloaded 14.10.16]
Kalyn, M., Lough, C., Mandryk, R., Nacke, L. (2011) Biofeedback Game Design: Using Direct and Indirect
Physiological Control to Enhance Game Interaction [Internet] University of Saskatchewan
Available from: http://hci.usask.ca/uploads/202-Nacke-Biofeedback_Game_Design.pdf
[Downloaded 24.10.16]
Kent, M. (2016) Full List of Professional Sports Teams in Esports [Internet] Dexerto.com
Available from: http://www.dexerto.com/news/2016/07/06/full-list-of-professional-sports-teams-in-
esports/ [Downloaded 21.10.16]
Kandidat 108
INFO310 0 Advanced Topics in Model-Based Information Systems Page 17 av 19
15
Mionix (2015) Quantified Gaming – know more, game better [Internet] Kickstarter.com
Available from: https://www.kickstarter.com/projects/206180815/quantified-gaming-know-more-game-
better [Downloaded 04.10.16]
Ortuño, F., Rojas, I. (2015) Bioinformatics and Biomedical engineering Switzerland: Springer
Collection of articles, article used:
Banos, O., Pomares, H., Rojas, I., Villalonga, C. Pages: 141 -152
An Ontolgoy for Dynamic Sensor Selection in Wearable Activity Recognition
Otake, T. (2016) IBM big data used for rapid diagnosis of rare leukemia case in Japan [Internet] Japan
Times
Available from: http://www.japantimes.co.jp/news/2016/08/11/national/science-health/ibm-big-data-
used-for-rapid-diagnosis-of-rare-leukemia-case-in-japan/#.WA9A-OiLQox
[Downloaded 05.10.16]
Perron, B., Schröter, F. (2009) Video Games and the Mind, Essays on Cognition, Affect and Emotion. North
Carolina: McFarland & Company, Inc., Publishers
Peuscher, J. (2012) Galvanic Skin Response (GSR) [Internet] Twente Medical Systems Internationa
Available from:
http://www.tmsi.com/products/accessories?task=callelement&format=raw&item_id=43&element=fe0c95f3-
af08-4719-bc51-36917715660d&method=download
[Downloaded 25.10.16]
Schütz, M. (2016) Science shows that eSports professionals are real athletes [Internet] Deutsche Welle
Available from: http://dw.com/p/1I4sT
[Downloaded 12.10.16]
Torres, V. (2013) Development of Biofeedback Mechanisms in a Procedural Environment Using Biometric
Sensors [Master Thesis]. University of Porto
Available from: https://arxiv.org/ftp/arxiv/papers/1310/1310.2102.pdf
[Downloaded 14.10.16]
Whitehead, J. (2007) An Ontology for Computer Games [Internet] University of California, Santa Cruz
Available from: https://classes.soe.ucsc.edu/cmps080k/Winter07/lectures/game-ontology-overview.pdf
[Downloaded 24.10.16]
Qualcomm (2016) 2net™ Platform [Internet] Qualcomm.com
Available from: http://www.qualcommlife.com/2net-platform
[Downloaded 27.10.16]
Kandidat 108
INFO310 0 Advanced Topics in Model-Based Information Systems Page 18 av 19
16
Resources
Illustrations Illustration 1: https://mionix.net/wp-content/uploads/2016/09/Naos-QG-Top-w-icons-2000px.png
Illustration 2: https://ksr-ugc.imgix.net/assets/002/992/678/d246a5617bec861b33101b137c2e425a_original.png
Illustration 3: http://quantifiedgaming.org/wp-content/uploads/2014/12/ujsdw.jpg
Illustration 4: Self-created
Illustration 5: Self-created
Ontologies Mentioned ontologies and where one can look at them:
MEDDRA: http://bioportal.bioontology.org/ontologies/MEDDRA
VSO: https://bioportal.bioontology.org/ontologies/VSO
EMOTION ONTOLOGY: https://github.com/jannahastings/emotion-ontology/tree/master/ontology
CVDO: https://raw.githubusercontent.com/OpenLHS/CVDO/master/cvdo.owl
HDO: http://www.disease-ontology.org/
GO: https://www.mindmeister.com/324669511/game-ontology-project
DBpedia person: http://dbpedia.org/ontology/Person
PIMO: http://www.semanticdesktop.org/ontologies/2007/11/01/pimo/
VGO:
https://github.com/dgarijo/VideoGameOntology/blob/master/GameOntologyv3.owl
http://vocab.linkeddata.es/vgo/
Social media contribution
There was no existing Wikipedia article on Affective gaming, so I made one:
https://en.wikipedia.org/wiki/Affective_gaming
Kandidat 108
INFO310 0 Advanced Topics in Model-Based Information Systems Page 19 av 19