22
Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge, L. P. M. Roelofs, F. Rümelin Student Industrial Design Eindhoven, the Netherlands {b.v.dijk, j.t.brugge, l.p.m.roelofs, f.rumelin}@student.tue.nl INTRODUCTION According to Rosekind et al. (Rosekind et al., 2010), people suffering from insomnia and insufficient sleep are less productive, proving the importance of a good night of sleep. Also, as Tsuno, N et al. (Tsuno, Besset, & Ritchie, 2005) state that approximately 9 out of 10 people suffering from depression state have sleep quality issues as well. It is stated that people feel more tired throughout the day after snoozing, raising the question why we snooze. Caused by sleep inertia, our ability to make rational decisions is not optimal (Ritchie et al., 2017). This might be the cause why a large portion of society is tempted by the snooze button. Naturally, a lot of design- and business opportunities arise in this field. Smartphone applications (e.g. “Runtastic Sleep Better: Sleep Cycle & Smart Alarm – Apps on Google Play,” n.d., “Sleep as Android - Apps on Google Play,” n.d., “Sleep Cycle alarm clock - Apps on Google Play,” n.d.) awash the consumer market with sleep monitoring apps, and alarms. Most of these application rely on algorithms that have not been validated by science (Ong & Gillespie, 2016). However previous work by Kim (Kim, Kientz, Patel, & Abowd, 2008) showed that sharing sleep data leads to a more intimate bonding between people, which might be a promising solution. Also, Kim (Kim et al., 2008) states that personal reflection upon sleeping behaviour might improve someone's sleep quality. Also based on personal experiences, snoozing behaviour is less occurring in social contexts. This might be a great approach to improve snoozing behaviour, and thus the perceived quality of sleep. This study addresses this design opportunity and evaluates a resulting design using various methods. The study has been conducted for the Eindhoven University of Technology course ‘User Experience Theory and Practice’. A small photo elicitation interviewing method was used in order to obtain insights in the ‘Fabric of Everyday Life’. Based on these insights, combined with the opportunity described above, a small design process was conducted. A concept was developed and evaluated using various tools. The tools were evaluated on their worth, and analysed based on their conclusions. Concluding, a tool is proposed which could have helped during the stated process. The tool might be relevant for other processes as well, as it is designed to provide insights in possible design flaws. THEORETICAL BACKGROUND As stated, there is a large amount of related work available in the field. Physical alarms have been around since approximately 400 BC (John W. Humphrey, 2003; Kim et al., 2008). Consequently there is already an enormous amount of solutions to wake up available. Currently, an increased amount of people use their smartphone to wake up using various alarm and sleep monitoring applications. When competing in this market, which will be a result of this process, the user experience should be optimal and suit the context well. As stated by Hassenzahl (Hassenzahl, 2010), experience should be a design objective as it has the ability to energise our behaviour, make us happy and its self-defining nature. This first point, energising our behaviour, will influence our goal: “a more energised daily feeling” positively. This addresses the importance of User Experience on the defined context. According to the design force, used in design processes. The why; what; how, respectively are based on science; design; engineering. To properly profound, (social) science(s) on sleep will be required to properly define a concept. Also as waking up is a daily event, “Experience Patterns” will occur. If the concept would rely on an element of surprise for example, this should also be included in the experience to minimise the novelty effect, which only lasts for a certain amount of time. As Roto V et al. state, a subset of user experience is co-experience (Hassenzahl, 2010; Roto, Law, Vermeeren, & Hoonhout, 2011). In some cases, multiple perspectives upon an experience influence the overall experience. As the design will tend towards social influences, it is important to keep these co-experiences into account. Something else that came to light by the work of Roto V et al. relates to the influence of time on user experience. As changes are made upon habitual behaviour, the experience over time might

Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,

Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge, L. P. M. Roelofs, F. Rümelin

Student Industrial Design Eindhoven, the Netherlands

{b.v.dijk, j.t.brugge, l.p.m.roelofs, f.rumelin}@student.tue.nl

INTRODUCTION According to Rosekind et al. (Rosekind et al., 2010), people suffering from insomnia and insufficient sleep are less productive, proving the importance of a good night of sleep. Also, as Tsuno, N et al. (Tsuno, Besset, & Ritchie, 2005) state that approximately 9 out of 10 people suffering from depression state have sleep quality issues as well. It is stated that people feel more tired throughout the day after snoozing, raising the question why we snooze. Caused by sleep inertia, our ability to make rational decisions is not optimal (Ritchie et al., 2017). This might be the cause why a large portion of society is tempted by the snooze button. Naturally, a lot of design- and business opportunities arise in this field. Smartphone applications (e.g. “Runtastic Sleep Better: Sleep Cycle & Smart Alarm – Apps on Google Play,” n.d., “Sleep as Android - Apps on Google Play,” n.d., “Sleep Cycle alarm clock - Apps on Google Play,” n.d.) awash the consumer market with sleep monitoring apps, and alarms. Most of these application rely on algorithms that have not been validated by science (Ong & Gillespie, 2016). However previous work by Kim (Kim, Kientz, Patel, & Abowd, 2008) showed that sharing sleep data leads to a more intimate bonding between people, which might be a promising solution. Also, Kim (Kim et al., 2008) states that personal reflection upon sleeping behaviour might improve someone's sleep quality. Also based on personal experiences, snoozing behaviour is less occurring in social contexts. This might be a great approach to improve snoozing behaviour, and thus the perceived quality of sleep. This study addresses this design opportunity and evaluates a resulting design using various methods. The study has been conducted for the Eindhoven University of Technology course ‘User Experience Theory and Practice’. A small photo elicitation interviewing method was used in order to

obtain insights in the ‘Fabric of Everyday Life’. Based on these insights, combined with the opportunity described above, a small design process was conducted. A concept was developed and evaluated using various tools. The tools were evaluated on their worth, and analysed based on their conclusions. Concluding, a tool is proposed which could have helped during the stated process. The tool might be relevant for other processes as well, as it is designed to provide insights in possible design flaws.

THEORETICAL BACKGROUND As stated, there is a large amount of related work available in the field. Physical alarms have been around since approximately 400 BC (John W. Humphrey, 2003; Kim et al., 2008). Consequently there is already an enormous amount of solutions to wake up available. Currently, an increased amount of people use their smartphone to wake up using various alarm and sleep monitoring applications. When competing in this market, which will be a result of this process, the user experience should be optimal and suit the context well. As stated by Hassenzahl (Hassenzahl, 2010), experience should be a design objective as it has the ability to energise our behaviour, make us happy and its self-defining nature. This first point, energising our behaviour, will influence our goal: “a more energised daily feeling” positively. This addresses the importance of User Experience on the defined context. According to the design force, used in design processes. The why; what; how, respectively are based on science; design; engineering. To properly profound, (social) science(s) on sleep will be required to properly define a concept. Also as waking up is a daily event, “Experience Patterns” will occur. If the concept would rely on an element of surprise for example, this should also be included in the experience to minimise the novelty effect, which only lasts for a certain amount of time. As Roto V et al. state, a subset of user experience is co-experience (Hassenzahl, 2010; Roto, Law, Vermeeren, & Hoonhout, 2011). In some cases, multiple perspectives upon an experience influence the overall experience. As the design will tend towards social influences, it is important to keep these co-experiences into account. Something else that came to light by the work of Roto V et al. relates to the influence of time on user experience. As changes are made upon habitual behaviour, the experience over time might

Page 2: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,

change. The momentary user experience might not be great, but as it results in a great cumulative user experience this might be worth the previous experiences. As we currently live in a knowledge economy, as described by Brand, R. and Rocchi S, the strength of peer-to-peer help might be valuable for the design (Brand & Rocchi, 2011). This might especially be relevant for acquainted people. People tend to invest in personal relationships and connections, even more than in professional environments.

Rational decision making The related work research was extended by provided literature about behaviour of choosing and rational decision making which will be displayed here. Since it is not possible to make rational decisions when someone is just awake, this is of significant interest for this paper. Ajzen (Ajzen, 1985) states that human behaviour is goal directed and thus it is mostly planned ahead. The paper provides a theory of planned behaviour which looks for behaviour under volitional control. This is one of the main factors on whether a subject will behave in a way their intention is telling them. The subjective norm which inter alia includes whether a subject believes the behaviour will lead to social prestige or similar. The paper shows that the theory of reasoned action can be directly applied to volitional behaviour. This can be elaborated by Ouellette and Wood (Ouellette & Wood, 1998) who state that there are types of habits: volitional and automatic, which both have three phases; first intention for a certain action, initiation of the action and finally the termination of the action. The way of volitional behaviour becoming a volitional habit is by repetition since it is a well-practiced behaviour according to Ouellette and Wood. As an actions is proved to be quick and easy, these actions are repeated, making in an efficient method to obtain a certain result. Bad habits are particularly interesting for this research. They are caused by results or outcomes of an action or habit that were valued previously, like the valued status of smoking in the past. Snoozing could be valued by the idea of having extra sleep and rest in the morning while this is actually not the case. Next to this, Ouellette and Wood state that the behaviour of a subject is stable and repetitive and thus predictable as long as the context is stable which indicates that changing the behaviour or a habit the provided design has to change the context. In addition to Ouellette and Wood, Schütz (Esser, 1993) states that “behaviour is a result of a choice between alternatives which are equally within reach”. In the comparison of alternatives, knowledge or expectations and motives or evaluations can play a role. Schütz states that subjective expectations can determine behaviour through weighting alternatives and those expectations are based upon knowledge of previous actions. The paper is an analysis on how rational choices are made according to Schütz and the RCT (Rational Choice Theory), which is an evaluation of alternative courses of action on the basis of

the value of goals and estimation of subjective probabilities to achieve a goal. A variant of RCT is the SEU (Subjective Expected Utility) theory, in which Alternatives are compared and the alternative course of action is chosen which is most likely to achieve a highly valued goal. Similar to this theory, Schwartz (Schwartz et al., 2002) states that “People go through life with all their options arrayed before them” and people can compare options with one another on a single scale of preference, value or utility. However in contrary to the theory of Schütz, Schwartz states that after making comparisons people make the choice to maximise their preference. The paper of Schwartz focuses on the two different kind of people, those who aim to maximise his or her outcomes in that domain and those who aim to satisfice. The paper addresses four different studies, to test the outcomes of the different ways of decision making. The outcomes of these studies specify that there are significant positive correlations between maximisation and regret, perfectionism, and depression, and significant negative correlations between maximisation and happiness, optimism, satisfaction with life, and self-esteem. Maximisers are “more inclined to engage in social comparison than satisficers”. The studies suggest that maximisers may in general achieve better outcomes than satisficers as a result of their higher standards and more extensive research, however they are likely to experience these outcomes as worse. Schütz in his theory uses the theory of Leibniz which seems similar to the theory of Schwartz. According to Leibniz’s Theory of Volition “will has various phases”. In the first phase positive weights of alternatives are considered and in the second phase negative weights are considered as well. This would reflect on a maximiser trying to maximise his or her outcome. Schütz also states that routines are satisfactory and cheap and the information about alternatives is normally costly. This states that the search for best solutions is ended if relatively satisfactory routines are established which would reflect on a satisficer. The paper and its knowledge is relevant to this process as the morning routine is mainly conducted as maximizers: being as efficient as possible.

Emphatic design To ensure that this concept includes emphatic design, the provided papers and their contributions are explained here. Smart (Smart, 2013) introduced a design challenge which taught him about the design process and about the power of empathy to foster innovation. The design challenge was to solve 50 problems in 50 days using design thus having “24 hours to observe a problem, attempt to solve it and then communicate the solution.” Smart states in his paper that a designer can miss real empathy if they do not experience it themselves but only through observations or interviews. He states that analysing people’s behaviour and problems is not enough, but a designer has to experience the situation by himself. According to Smart, Empathic research helps

Page 3: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,

designers to understand the users’ needs better and even beyond the functional. This would enable a designer to develop more appropriate design outcomes. He states that “great designers understand people.” Finally Smart concludes that empathetic understanding is a vital tool in fostering innovation. If the targeted users are understood better, the decisions towards the designed solution for them will be better, and thus the design itself and the results and implementation as well. To innovate, a designer should go beyond analysing people’s experiences and try to make them their own. This theory is useful for the concept, as the domain relates to an activity which is experienced everyday by the researchers as well: getting out of bed. This emphasized the clear understanding of the problem by the researchers. According to Zaki (Zaki, 2014) “Empathy is a vital emotional force that scaffolds everything from close relationships to large-scale cooperation”. Zaki states that “Empathy features a tension between automaticity and context dependency.” There are two characteristics of empathy, on the one hand people take on the internal states reflexively of others and outside of awareness, on the other hand, empathy shifts with characteristics of empathizers and situations. The key role of motivation in driving people to avoid or approach engagement with others’ emotions reconcile these two characteristics of empathy. Zaki names three phenomena which motivate people to avoid empathy and three which motivate people to approach empathy namely “Suffering, material costs and interference with competition” and “positive affect, affiliation, and social desirability” respectively. From this, it can be concluded that in order for people to approach others, a designer should engage positive affect, affiliation and social desirability. As Zaki concludes “Incorporating these motives into research and theory can restructure the way scientists conceive empathy, opening numerous new directions for understanding when empathy fails and how to maximise empathy when it does not come naturally but is needed the most. With this he implies that empathy can be forcibly maximised when it does not come naturally by incorporating these motives into research. In a group discussion is however questioned if this is an ethically responsible and acceptable way of designing. The paper implies that the designer can force users into experiencing empathy which could lead to users making different decisions by being influenced by their emotions. The conclusion of the discussion stated that the use of forcibly designed empathy can be used in cases where only good for the user is desired by the designer however if the goal is to gain money from the user by creating empathy it is ethically questionable. Here is decided that in this concept, empathy is used as an option for all the users, but it will not be forcibly imposed to ensure that the concept will make the users act through their own, not modified emotions.

To create empathic involvement from the user with the concept while not finishing the final design, the method of co-constructing stories of Buskermolen and Terken (Buskermolen & Terken, 2012) is used. The paper introduces this participatory design technique to gain “early formative concept evaluation to elicit in-depth user feedback and suggestions, revealing attitudes and motivations of users.” The technique consists of two main phases, one focused on recollecting past experiences related to the context and one focused on an envisioned future experienced. Both are established through storytelling, used by the designer to set the stage and present the concept. Storytelling is used by the user to communicate their past experiences and anticipate the future situation. The technique gives insights and feedback on a concept while still being in the design phase. A designer could identify possible barriers within the concept which could stop the user from using it for example the distribution of the pains and gains within a concept. This technique is used to gain feedback on the concept in this paper.

STUDY

Setup In order to obtain insights in how activities in daily life come about, a study was conducted using photo elicitation interviewing (PEI) (Meo, 2010). The study was selected as it would allow participants to easily log their daily activities, which could then be used later during the interview. Despite the fact that photo elicitation interviewing is more time consuming (Meo, 2010), it has proven to be more valuable than traditional interviewing as described by Thompson and Oelker (Thompson & Oelker, 2013). Four participants were selected, and asked to take photographs of their daily activities for a day. When the photos were collected, an unified meeting was planned among three of the participants. The interviews lasted each 15 - 20 minutes, with a short analysis on insights afterwards of approximately five minutes. During such a round, the participant would tell about their day and give possible remarks, after which both of the researchers were allowed to ask questions allowing them to hitch in upon questions. After the interviews, all insights were collected and discussed, and mapped among projected value and interest.

Insights As the above described theory states, everyday activities are influenced by difference aspects. The formation of habits influence daily choices, since habits are a big part of the fabric of our life. Habits are things we do on a regular basis and this means they are important to us and our life. As stated by Schütz, the origin of habits lies in efficiency, as the action resulted to be time and result efficient; and thus it is executed again if a similar result is desired (Esser, 1993; Zaki, 2014). Human values are also important in our day to day life.

Page 4: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,

Values are what forms you as a human and shapes the interaction you have with other persons. This obviously is a big part of our life, and it influences the choices we make, for example, when someone is shy and introvert, they will not make the choice to go to crowded places when it is not necessary. If someone makes a decision according to their human values, they might be pursuing a goal. The basic human needs are an important aspect of surviving and thus this is a necessary part of making choices. If someone makes a choice on the aspects of basic human needs this is because they are trying to satisfy a need. These needs can relate to the what and why of decision making through “Do”- (required tasks, expected functionality) and “Be” (personal goals, human needs and relevance) goals (Buskermolen & Terken, 2012; Hassenzahl, 2010). Choice behaviour, the way you decide things, is also of influence to making choices. The outcome of a choice can vary a lot if you are for example a maximiser or a satisficer. The study resulted in various insights amongst various domains within daily life. Some of the most interesting insights are listed below.

Insights general

Eetlijst/who’s at home? The social aspect of knowledge about who is at home at that moment in time is given through various designs. With ‘eetlijst’ (A website for students to indicate whether they are eating at home and whether they can cook that evening (“Eetlijst.nl,” n.d.). It can also be seen through participation of meals who will be home in the evening and sometimes in the coming days. ‘Eetlijst’ is the more efficient alternative, because you already sign up for dinner and with this you can see who will be home. The in-order-to-motive to sign in is really evident, you want to be able to join the dinner. The motive to sign in at the home design is socially influenced, you want to sign up because other people can see your home (Ajzen, 1985).

Bike The participant takes her bike, because it gives a feeling of independence. From the alternatives in reach, this is her best alternative. The alternatives within reach are the bus and bike, because she does not have a car. The bus gives less independence in traveling and this is one of her motives to take the bike.

Headphones One of the participants showed a picture of his headphones, these headphones are used when listening to music. The headphones are wireless. When he connects his headphones he always checks if the music is really playing through his headphones and not the speakers before he turns up the volume. The other participants recognized this behaviour even if they were using other headphones or earphones. The headphones are useful when travelling on the train, but

they are not convenient while working at the university. At the university you might need to communicate with other students, social normative beliefs influence this inconvenience of interaction (Ajzen, 1985). The participant chose to buy these specific headphones because of past experiences. He first used iphone earphones which would break easily and were of less high quality. To achieve the goal of listening to high quality sound he chose the alternative to buy the more expensive headphones (Esser, 1993). The headphones can be used with bluetooth and wire. In past experiences the wires got tangled up, so bluetooth could be useful, but when the battery dies you can still use it with the wire (convenience).

Washing machine All housemates use the same setting of the laundry machine. The social influence of the housemates has an effect on choosing the same setting, if everyone uses that setting it should be the best option (Ajzen, 1985). When using the laundry machine, it is usually on the “correct” settings, however the participant always checks the display to see if the settings are correct. The first setting which can be seen on the display is the heat, which is really important for her (convenience).

Insights chosen domain: Morning Routine A number of valuable insights were gained in the analysis of the results of the photo-elicitation interviews. During the discussions, a domain of preference was chosen within everyday-life: The morning routine. Topics within this domain were analysed more thoroughly, concluding in our top 2 insights which are analysed and described in detail.

Fridge next to breakfast table In order to be able to set her table more efficiently, one of the interviewees explained that she placed a refrigerator in her room which is located on the top floor of a student house. She did this because it enables her to quickly and efficiently set the breakfast table without having to go downstairs to retrieve food for an extensive breakfast from the fridge at the bottom floor (which relates to convenience and efficiency). Having an extensive breakfast is a habit that originates from her parental home which makes it a subjective norm (Ajzen, 1985). She is also making use of a thermos flask as a tool to make the enjoyment of drinking tea more convenient. It is kept warm instead of having to wait for the water in the kettle to boil and having to set tea several times. Keeping the water warm makes it more time-efficient.

Coffee One interviewee stated that she would consume coffee in three different scenarios. In the first one she would consume it mainly for the taste in which this physio-pleasure would matter the most and accordingly the quality and the way it was made. E.g. in the morning she uses a percolator

Page 5: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,

because it is the best alternative, it is sufficient for her needs in this situation to search for another alternative (Esser, 1993). In a second scenario coffee would be consumed as part of a social activity, in this case the consumption of the beverage is secondary to the motivation of interacting with a friend/colleague. In the third scenario coffee would be consumed for its caffeine in which in comparison to scenario one the physio-pleasure is not concerning the taste but rather the stimulating effect of the beverage, the taste becomes so unimportant that the taste has to be no better than ‘bearable’.

Curtains One of the interviewees explained how he had built a mechanism that makes it possible to open the curtains simultaneously in order to strive for maximum efficiency in the morning. The influences of efficiency on morning behaviour/habits is a recurring phenomenon (Ouellette & Wood, 1998).

Choosing shoes The interviewee described the process in which he was choosing his shoes before leaving the house, the shoes needed to be appropriate to the activity that he was going to attend. He explained that he did not always have shoes that would fit every specific situation, in the process he was weighing physical and social circumstances of the activity against each other to make a choice. Example situations would be going to a party, attending a formal dinner or a walk in the field. This insights does relate to social influences as well, as the presence of different (groups of) people influence the choices made.

Detailed analysis: Electric toothbrush Two of the interviewed participants indicated to use an electric toothbrush. The choice to use an electric toothbrush over a regular one, is based on two things. First, the feeling of cleaner teeth after brushing with an electric one (past experiences). And secondly, the advice of an expert: the dentist. These reasons to use an electric toothbrush do not hold for all of the participants, as the other participant indicated to have the feeling of cleaner teeth with a regular toothbrush. The electric toothbrush is always used, also when going on a trip or sleepover. However, when the trip is expected to take longer than the battery life of the toothbrush the choice is made to use a regular one. The participants stated that they did not like to bring the charger. This relates to the quality of the tool used to obtain clean teeth, as better battery life would increase its usage as well as its convenience. If the charger would have been smaller, they might bring it along on holidays. The importance of the qualitative aspect of the toothbrush also show in its user experience. When people are sleeping near the projected place of brushing teeth, the electric function is not used as

the users are scared to wake them up. The physio pleasure aspect of the toothbrush shows in the difference between different electric toothbrushes. One participant indicated to prefer a certain type, namely ultrasonic, of electric toothbrush as regular electric toothbrushes caused an itchy nose due to its vibrations.

Detailed analysis: Alarm During the interviews all participants stated to have problems with waking up. Multiple alarms amongst different devices are required to wake up in time. This might relate to the quality of the product, despite the fact that they indicate to not have experienced a faulty alarm. They indicated that when other people are in the room (either at home or at someone else’s place) this phenomenon does not occur, and only one alarm is required to become fully awake, indicating a social aspect relating waking up. This might be due to past experiences, as one participant indicated. It was stated that the participant had experienced a sleepover where the other snoozed for over three hours. This might be the origin of the courtesy to skip snoozing when someone else is present. This mainly relates to weekdays, as during the weekend participants have alternating stories. Some stated that they were able to easily get up during the weekend, whereas others indicate to find it just as difficult. Participants indicated that alternating alarm sounds helped to get up (e.g. an radio-alarm versus a beep).

Conclusion From the analysed insights it is shown that a lot of people make choices to optimise the time. Especially in the morning they want to spend their time as efficient as possible. They do take other options into account. In their morning routine, most people are behaving like maximisers when they have to take decisions in their morning routine. From the analysis it is learned that many people have trouble with waking up and getting up in the morning. This causes people to snooze their alarms. This causes people to be late, more sleepy and take way longer than needed. This is in contrary to the rest of their morning routine, because as shown before, the rest of their morning routine is chosen to be as efficient as possible. This raises the question why everything is made to be as efficient as possible in the morning routine except for the actual waking up and getting out of bed. A possible answer for this is because people do not want to get up early and they prefer sleeping over getting up. That causes a hurry to get in time to their necessary daily things, which causes the rest of the morning routine to be in a hurry. This requires the morning routine to be as efficient as possible, so they can sleep as long as possible. However, this on its turn raises the question, why do people snooze, and do they not put their alarm somewhat later and sleep for longer. This is attempted to be solved in the first iteration.

Page 6: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,

DESIGN

First iteration The insights were used to make a first concept iteration. The insights show that people would rather snooze and plan the rest of their morning routine more efficiently, than to sleep longer, get up immediately, and have a more relaxed morning routine. This is only preferred on the instance of waking up, as snoozers indicate that they feel more tired during the day and regret their lazy behaviour. In order to tackle this problem, design process was conducted in order to find solutions which stimulates people to get up in the morning instead of snoozing continuously. There is an enormous amount of smartphone applications that try to tackle this problem. For example, Alarmy (“Alarmy (Sleep If U Can) - Alarm clock – Apps on Google Play,” n.d.) is an application which tries to prevent the user from snoozing by limiting the user to only be able to turn off the alarm by solving equations. Despite this fact, the described study showed that these solutions do not offer a complete solution, as often the snoozing behaviour returns in the form of multiple alarms. Concluding from the previously described study and brainstorms, a new method was designed to prevent snoozing behaviour. The concept relied on gamification, and more importantly social factors. As it was stated that snoozing behaviour barely occurs in social environments. The concept is a smartphone application in which your snooze data can be shared with your friends. These friends can be added to your account through usernames or you can invite friends through social media (e.g. Facebook). The alarm allows users to set their alarm and their snoozing interval (2 – 10 minutes). As the user decides to snooze, he or she is allowed to as normally. However, the amount of times snoozed and alarms is counted and results in a final score, consisting of the amount of alarms that have rang in total. The score is presented on a leaderboard in which you can compete with your friends. This can result in a sloth of

Figure 1. Phone application mockup of the first iteration.

the day, week and month. Also, the scores are also used to keep track of your own ‘snoozing progress’, where you can track your own improvement. A calendar with scores and a total average is shown in the progress section. In this section, it is easy to see how your last score affected your total score and how your snoozing behaviour develops over time. The concept has been evaluated on the improvement of snoozing behaviour by social influences, since this was the desired goal of the concept. The concept was based on social pressure of a too large group, which was thought to cause a lack of feeling of responsibility. Also, the snoozer might feel less motivated as he/she can see that he is not the only one snoozing, counteracting the goal of the concept. The concept slightly hinted towards a knowledge economic product (Brand & Rocchi, 2011). However during its evaluation, it was thought that the concept would offer too little “mass participation, sparked continuous activity and interest, and offered accessible possibilities” (Brand & Rocchi, 2011) which are required factors in order to be a successful knowledge economy product. In order to tackle these aspects of the design, a new concept was developed with a focus on personal social pressure. Using the social monitoring of people who are emotionally connected and/or related was used to provide a similar experience as when sleeping in a social context (e.g. with friends) and thus would be more effective.

Second iteration Based on the evaluation of the concept, a new design iteration is proposed. Two emotionally related people form a team, in which two roles are allocated in the concept: A helper (awake), and a snoozer (in the middle of the snoozing process). The roles are interchangeable, as one can be a snoozer one day and a helper on the other. This might be dependent on the relative moment of waking up. The application has the following features:

● Both parties are able to set their alarm tone, volume, vibration and time as normally.

● The helper will receive a notification when the snoozer snoozes.

● After the third time of snoozing, the helper is able to record a custom alarm tone which will be played when the next alarm of the snoozer rings (e.g. “Good morning lazybones, get up!”).

● After the fifth time of snoozing, the helper is able to directly talk to the snoozer. Similar to a baby intercom.

● Both parties are able to see their snoozing statistics, to monitor their overall progress.

Page 7: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,

Figure 2. Mockup of the second iteration.

EVALUATION 1: HEURISTIC EVALUATION To evaluate the current concept an heuristic evaluation was conducted. The Heuristics were devised from the heuristics each of the group members had drawn up. To devise these heuristics, an inductive thematic analysis was used. In this inductive analysis, themes are generated from raw data. In this analysis tables are used in which the first step are the codes which are discussed and then the themes are generated. After this the validity of the themes is considered to determine if the themes represent the codes well (Nowell, Norris, White, & Moules, 2017). This analysis helped to combine the different heuristics into one set of heuristics which could be used for the evaluation of the concept. The defined heuristics from the thematic analysis are explained.

With this set of heuristics the concept was evaluated individually by every group member. The different evaluations were combined and discussed to get to the most complete result.

Heuristics Consistency The same action always gives the same result Aesthetics/Clearity Aesthetic and minimalist design which limits distractions, communicating data clearly, with a clear difference between actions and clear feedback. Flexibility/ Level of control If you desire more control over your actions, then you put more effort in it. How you achieve a certain goal. Efficiency Get quickly to the desired result. Errors & help Clear errors and providing of useful help which supports the user in solving the problem. Go back/ undo If you do something wrong, you can still undo it. System status visibility You can see what the system is doing, for example busy with sending, recording, thinking etc…. Feedforward & Feedback User should obtain sufficient amount of feedforward to guide them to the desired result. And get sufficient feedback whether an action was successful. (personal)Motivation Good-self-image, self-esteem, becoming who you really are. Feel like your successful.

Results From the evaluation was found that there is not yet a help function and this should be implemented. In the current iteration the design does not have a go-back option, this would be a desirable function. Also the user cannot see whether messages have been sent and received, the user should be able to see this. The concept could implement some more “success” screens to communicate to the user whether a send message has been received. This also relates when calling live, the user should be notified when a connection has been established. The application will stimulate the connection to a caring other, improving behaviour together. An improvement for

Page 8: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,

the concept would be to implement awards and achievements. This could be in the form of physical, real life, rewards. For example, the snoozing person could cook dinner for the helping person if they would improve their snoozing behaviour by a certain amount.

If someone does not improve, or maybe even gets worse, the application should still give a motivating image, instead of negative feedback only.

EVALUATION 2: CO-CONSTRUCTING STORIES After evaluating the concept with the heuristics which were defined through an inductive thematic analysis, users were included in the process, the co-constructing stories method was used to get insights from the user. This user study consisted of eight participants who participated in co-constructing story session which lasted 45 minutes. In Table 1. the setup of these sessions is described.

Activity Explanation

Introduction A short explanation about the co-constructing story session. Discuss the planning.

Presenting the sensitizing story

Go over the sensitizing storyboard with the participant, no need to put extra attention at the snoozing. Let the participant react to the story.

Eliciting stories of real-life experiences

Discuss with the prompt materials the experiences of the participants. The prompt materials can be used as support, they can draw or write on it but are not required to do so.

Presenting the visionary story

Go over the visionary storyboard and use the app visualizations if needed to explain. Let the participant react to the story.

Talking about first impressions

What does the participant think of the concept?

Eliciting stories about the imagined use of the concept

How would the participant use the application?

Comparing the envisioned experience to the experience now

How does the concept change the experience

Table 1. Setup co-constructing story sessions.

Results From the analysis of the insights of the different sessions some new important themes were derived.

Privacy Does the concept keep the privacy of the users safe? How much of the privacy related data is necessary to keep? Does the user want to share data, and what is being shared? Have all users agreed to share data?

Alternative solutions Are there already alternative solutions for the problem? Are these alternatives valued more than the concept.

Burden of the helper It is not the helpers problem, but they do carry a lot of the pains. The helper has to put a lot of effort in the product, but what does the helper get in return?

Benefits of the helper The benefits of the helper are based on a personal connection with the snoozer. This connection makes it fun for the helper to put in some effort.

General burdens The time of the use could affect the effectivity and some will still be stressed in the morning, because they will just put an alarm at the latest time possible.

General Benefits Less stress and hurrying in the morning, will create a safer environment. Less stress in traffic, more awake during meetings in the morning, being on time, knowing if someone will be on time.

Snoozing behaviour The alarm is not nice for others. You think you sleep more, but you sleep less. Snoozes daily. No appointments so snoozes. Does not experience snoozing as something annoying, but does have the idea to suffer from it.

Target group Concept does not have a purpose for people living together. The most notification get send to his girlfriend before she leaves the house.

Motivation For the helper it would take too much effort to use it for more than a week. It would only be fun to help snoozers you know. Some snoozers might not see the relevance of the problem, because snoozers will keep snoozing. Why does the helper need to record it at that specific time, why not use pre-recorded recordings. The system would not work if the messages are always the same, it would get boring. Adding milestones to turn the improvement into something physical, like a meal to celebrate. Approach the problem more positive, do not show how many times a snoozer has snoozed, but how many friends are already up.

Market When the product fulfils its purpose it will not be used anymore. The product will only be fun for a short period of time. For a boss, coworking relationship, there would be a market opportunity.

Page 9: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,

Design The colours of the design could be ‘happier’, making waking up more ‘joyful’. The design is nice and clear and does not have unnecessary features.

Misuse The colours of the design could be ‘happier’, making waking up more ‘joyful’. The design is nice and clear and does not have unnecessary features.

Experience It seems like a therapy. It could evoke stress and a bad feeling about yourself.

Social Social pressure could probably help with getting up, but how can a helper get out of this if a friend still snoozes after two weeks. Maybe this concept could affect friendships negatively.

App functionality The live calling and recordings will sometimes take too much effort of the helper. And in some situation the helper will not be able to record or speak directly. If the helper is sleeping he/she would be disturbed. Do not let snoozers snooze at all, if you want them to stop snoozing. The freedom of being able to choose between alarms is nice. Not a lot changes for the snoozer, while the helper has to put in a lot of effort into the product in the morning. If I would snooze and I would have a live connection, I would turn it off immediately and get up (But it should not be possible to turn it off, so keep that in mind)

DISCUSSION HEURISTICS AND CO-CONSTRUCTING STORY METHOD From comparing the results from the heuristic evaluation and the co-constructing stories method, we concluded that the co-constructing stories sessions gave us new insights. The heuristic evaluation gave us more insights about the design details of the concept and the co-constructing story sessions gave a more holistic view of the concept. In these co-constructing story session not only the design and functionality of the concept was evaluated, but also the user experience and usability. Conducting the co-constructing stories method, showed us that the pains and gains were not evenly divided over the different stakeholders. These pains and gains are really important to predict if the target audience will buy or use the product. The helper in the scenario had more pains than gains, and it is not the helpers problem. This means that the helper will probably not have a lot of motivation to buy or use the product. This problem in the concept was not predicted by the heuristic evaluation. This is why we wanted to develop a tool which could easily compare pains and gains. These comparisons are made for every individual stakeholder and can also be easily compared to each other. The tool really looks into detail at the different individuals who use the concept to make a comparison since this would give a holistic view of the

pains and gains of all the users of the concept. This tool could help to predict the use of a concept better, and not neglecting the gains of a part of the stakeholders.

EVALUATION TOOL From the Heuristic evaluation and mainly from the co-constructing stories evaluation is concluded that the pains and gains are not evenly divided in the anti-snoozing concept. This could have been analysed a lot earlier in the design process if the right tool was provided. To prevent this from happening in other design processes a tool is created. This tool focuses on creating a clear overview of the pains and gains, showing connections and detecting possible problems in a concept. From user testing, a lot of valuable information is gathered. The difficulty is to find all the useful connections between the different feedback and create a holistic view of the pains and gains of a concept. To design a tool to find a possible uneven balance in the pains and gains between different users of a concept, previous and current projects are taken into consideration. The first design choice is to compare the pains and gains within one user next to each other. Next to this, the situation before and after the concept is introduced has to be evaluated. Finally the desire is brought up to compare the pains and gains between different users. All these variables and comparisons could give valuable feedback about the concept and it could help detecting possible barriers for people to use the concept. These requirements are brought to the design phase. When designing the tool, it was important to gain a clear overview which would show possible connections.

The final design of the tool is shown here (see Figure 3).

Figure 3. The Pain and Gain analysis designed tool

The tool is based on basic keywords gained from the thematic analysis after the co-constructing stories test: “Privacy, Alternative solutions, Burden of the helper, Benefits of the helper, General burdens , General Benefits, Target group, Motivation, Market, Design, Misuse,

Page 10: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,

Experience, Social, App functionality.” These keywords will be on a A4 paper as shown on the figure with three words as an example. Underneath every keyword there are two points: Old situation and New situation. The user can fill in the boxes based on their findings on the concept through a co-constructing stories method. After this, the researcher can compare the gained feedback in three different ways; The old versus the new situation with the concept implemented in both the pains and the gains, the pains versus the gains within one user, and also the pains versus the pains of different users or the gains versus the gains between users [Figure 4]. This could give insights in if the pains and gains are divided equally between different users, possibly with different roles in the concept. Besides this, the researcher can validate any differences before and after the concept is implemented. By using this tool, the concepts pains and gains can be adapted to fit all the users before the concept is finalized and tested and it gives a better prediction if a target audience is going to find motivation to use the concept. The tool provides a clear overview of all the gathered data so that no connections or possible barriers in using the concept is overseen.

Figure 4. The between-users evaluation using the designed tool

CONCLUSION AND DISCUSSION With the diary study and the photo elicitation method (Meo, 2010) and the evaluation of the results, a valid problem was identified in the “morning-routine” domain. The interesting part about the design opportunity was that it was addressing a situation which was changed through a social component, meaning that the process of snoozing was influenced by the presence of a peer or partner. In order to find out how successful the product might be a heuristic evaluation was executed. The result thereof were showing that the design was missing a lot of detail but did not show any major

design flaws in the core of the concept. Bringing the concept into contact with possible users through the co-constructing stories method showed insights in a bigger picture and gave a better understanding on how the design would be integrated into its environment and possible usage habits, and the mismatch in gain and pain division. In the case of the design which was analysed by this group the response of the participants showed that there was a discrepancy between the two user groups. The helper would have to put unproportionally much work into assisting the snoozer to stop his habit, while the latter did not need to make a lot of effort. While for this design it may have been better to first make use of the co-constructing stories method, and use the heuristics on an improved version of the design, the same might have happened for any other design which is building on the interaction of two or more people. The heuristic evaluation did not assess the social component of the design well enough, although an own set of categories was used in order to not only comply with heuristic frameworks which were set up for the evaluation of applications on screens. Nevertheless the heuristic evaluation can be helpful during different steps of the design process, while the categories of evaluation might need adjustments depending on the state and purpose of the design. The new tool which was proposed is aiming to survey participants in an organized way about key areas which were identified as important during the thematic analysis of the co-constructing stories method. It offers a pains and gains analysis that compares not only the situation without and with the implemented design but also allows to assess whether burdens and benefits are relatively equal for the users. Looking forward, the developed tool will need to be tested and evaluated in practice.

REFERENCES Ajzen, I. (1985). From Intentions to Actions: A Theory of Planned Behavior. In Action Control (pp. 11–39). Berlin, Heidelberg: Springer Berlin Heidelberg.

Alarmy (Sleep If U Can) - Alarm clock – Apps on Google Play. (n.d.). Retrieved June 20, 2018, from https://play.google.com/store/apps/details?id=droom.sleepIfUCan

Brand, R., & Rocchi, S. (2011). Rethinking value in a changing landscape. A Model for Strategic Reflection and Business Transformation. A Philips Design Paper. Retrieved from https://pdfs.semanticscholar.org/c34a/3e300f1b9d1d4eb45e2af3cf7e2aa3d0344b.pdf

Buskermolen, D. O., & Terken, J. (2012). Co-constructing Stories: A Participatory Design Technique to Elicit In-depth User Feedback and Suggestions About Design Concepts. In Proceedings of the 12th Participatory Design Conference:

Page 11: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,

Exploratory Papers, Workshop Descriptions, Industry Cases - Volume 2 (pp. 33–36). New York, NY, USA: ACM.

Eetlijst.nl. (n.d.). Retrieved June 20, 2018, from http://eetlijst.nl/

Esser, H. (1993). The Rationality of Everyday Behavior: A Rational Choice Reconstruction of the Theory of Action by Alfred Schütz. Rationality And Society, 5(1), 7–31.

Hassenzahl, M. (2010). Experience Design: Technology for All the Right Reasons. Morgan and Claypool Publishers.

John W. Humphrey, J. P. O. A. A. N. S. (2003). Greek and Roman Technology: A Sourcebook: Annotated Translations of Greek ... - Andrew N. Sherwood, Assistant Professor Department of Classics Milo Nikolic, Milo Nikolic, John W. Humphrey, John P. Oleson - Google Books (p. 648). Routledge.

Kim, S., Kientz, J. A., Patel, S. N., & Abowd, G. D. (2008). Are you sleeping?: sharing portrayed sleeping status within a social network. In Proceedings of the 2008 ACM conference on Computer supported cooperative work (pp. 619–628). ACM.

Meo, A. I. (2010). Picturing Students’ Habitus: The Advantages and Limitations of Photo-Elicitation Interviewing in a Qualitative Study in the City of Buenos Aires. International Journal of Qualitative Methods, 9(2), 149–171.

Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic Analysis: Striving to Meet the Trustworthiness Criteria. International Journal of Qualitative Methods, 16(1), 1609406917733847.

Ong, A. A., & Gillespie, M. B. (2016). Overview of smartphone applications for sleep analysis. World Journal of Otorhinolaryngology-Head and Neck Surgery, 2, 45–49.

Ouellette, J. A., & Wood, W. (1998). Habit and intention in everyday life: The multiple processes by which past behavior predicts future behavior. Psychological Bulletin, 124(1), 54–74.

Ritchie, H. K., Burke, T. M., Dear, T. B., Mchill, A. W., Axelsson, J., & Wright, K. P. (2017). Impact of sleep inertia on visual selective attention for rare targets and the influence of chronotype. Journal of Sleep Research, 26(5), 551–558.

Rosekind, M. R., Gregory, K. B., Mallis, M. M., Brandt, S. L., Seal, B., & Lerner, D. (2010). The Cost of Poor Sleep: Workplace Productivity Loss and Associated Costs. Journal of Occupational and Environmental Medicine / American College of Occupational and Environmental Medicine, 52(1), 91–98.

Roto, V., Law, E., Vermeeren, A., & Hoonhout, J. (2011). User Experience White Paper. Retrieved from http://www.allaboutux.org/files/UX-WhitePaper.pdf

Runtastic Sleep Better: Sleep Cycle & Smart Alarm – Apps on Google Play. (n.d.). Retrieved June 20, 2018, from https://play.google.com/store/apps/details?id=com.runtastic.android.sleepbetter.lite

Schwartz, B., Ward, A., Monterosso, J., Lyubomirsky, S., White, K., & Lehman, D. R. (2002). Maximizing versus satisficing: happiness is a matter of choice. Journal of Personality and Social Psychology, 83(5), 1178–1197.

Sleep as Android - Apps on Google Play. (n.d.). Retrieved June 20, 2018, from https://play.google.com/store/apps/details?id=com.urbandroid.sleep&hl=en

Sleep Cycle alarm clock - Apps on Google Play. (n.d.). Retrieved June 20, 2018, from https://play.google.com/store/apps/details?id=com.northcube.sleepcycle&hl=en

Smart, P. (2013, May 27). 50 Design Problems In 50 Days: Real Empathy For Innovation (Part 1). Retrieved June 20, 2018, from https://www.smashingmagazine.com/2013/05/50-problems-50-days-part-1-real-empathy-innovation/

Thompson, M., & Oelker, A. (2013). Use of Participant-Generated Photographs versus Time Use Diaries as a Method of Qualitative Data Collection. International Journal of Qualitative Methods, 12(1), 624–637.

Tsuno, N., Besset, A., & Ritchie, K. (2005). Sleep and Depression. The Journal of Clinical Psychiatry, 66(10), 1254–1269.

Zaki, J. (2014). Empathy: A Motivated Account. Psychological Bulletin, 140(6). Retrieved from http://ssnl.stanford.edu/sites/default/files/pdf/zaki2014_motivatedEmpathy.pdf

Page 12: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,

APPENDIX A: REFLECTIONS

Reflection B. van Dijk As stated in my Personal Develop I stated that I wanted to improve my methodology around user involvement in design processes. The course stimulated me to first learn about methods through papers, allowing me to get familiar to the theory and its fundamentals on which it is based. After this first acquaintance with a method, we had the opportunity to execute the method in practice. This was a valuable experience as it allowed me to experience the strengths and weaknesses of each method after a proper execution of the method. The evaluation of methods is important prior to its execution, as not every method will give insight into every aspect of the design. For example, with the heuristic analysis we oversaw different flaws of our design as the method gave to much in depth insights. The co-constructing stories method allowed us to get a birds-eye-view on the user experience, allowing us to obtain insights into the mismatch of gains and pains prior to further concept development.

The course taught me to look more critically at what scientific papers I read. Before this course, I always stated every paper as the truth in every period of time. Because I was obligated to describe my most important points of discussion, I was stimulated to think of new views or insights. This is especially relevant while reading old papers, as their theory might not hold for phenomenons of today. This critical attitude towards literature is something I will take with me, together with its method to do so: writing down important statements and raised questions.

Normally in design processes, I assume that I need a functional prototype to test its projected user experience. I learned that with the help of several methods (e.g. storyboards, co-constructing stories, heuristics) concepts can be evaluated prior to their realization sufficiently to obtain valuable insights. This insight will help me in future processes by preventing the development of concepts in depth unnecessarily.

As a personal goal I stated that I wanted my meetings to be more goal oriented and efficient. I wanted to do this through clear to do lists and a shared calendar. I suggested this and implemented this method in our meetings and workflow. This worked really well, and was stated to have had a good influence on our collaboration. During future processes this is a valuable experience and skill, as especially during the last busy weeks a lot of work has to be done while everyone is busy. The method helped us to keep working in a structured way, also when working apart.

Reflection J. ter Brugge In my projects I think it is really important to include the user. This is why I chose this elective, to get a better

understanding about user experience and how to get information about the user experience.

The methods I learned in this elective helped to give me new tools to include the user in my design process, I think co-constructing stories could really help the user empathize with a situation and help the user give more feedback on the concept. Our own tool could also be of help, in predicting if my concept will be successful. Looking at the pains and gains of every user and comparing the old situation to the new could help me get a good perspective of the concept.

Because I am really interested in design for dementia the paper about “adoption of innovations” was really interesting for me. Every target group consists of individuals with a different likeliness to adopt a new innovation. This knowledge could be used how you approach a person. In my current project I noticed that some people, especially in care facilities are really protective against new innovations and research. But you also have people who are really interested in the new innovations and want to give their input to improve the concept. For the development of the concept, I think it is better to first approach early adopters to get a lot of input. And when looking at marketing you could focus on the later adopters too, by giving a lot of background information about the new innovation.

Reflection L. P. M. Roelofs In this course I have learned a lot about how to design a good User Experience, but I have learned even more about evaluating your design and concept in the early stages of the process. It all started with the “Fabric of everyday life.” I have learned that it gives really good insights in all the important things of the everyday life and there can be interesting and unknown problems or similarities between a lot of people. The next assignment, evaluating the heuristics of our prototype, gave me insights in the outcomes you can get by critically looking at your own concept. It can be really useful if you look at your own concept through the eyes of the user and look evaluate it according to certain set heuristics. This could give you insights in what has to change or what has to improve for the concept to be useful or feasible. After this, the co-constructing stories are a great addition, since the heuristics go really into depth into some details, while the co-constructing stories is a global evaluation technique. I had conducted the co-constructing story some time before and thus the technique was not new to me, however it again proved its usefulness in this project. I have learned that it is a good technique to give vital feedback about the concept before entering the prototype phase. Finally I have learned that it can be really useful to create a tool after your project. Looking back at a project it is easily detected what problems occurred or what could have been prevented to come to a better end concept or have a more efficient design process. However while in a design process this is not as clear and obvious. This is why

Page 13: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,

it can by really useful to create a tool to prevent what went wrong in your design process in a future project. This is something for me to definitely keep in mind because having a lot of tools kept in reserve, and gaining knowledge about what can go wrong and how to prevent it can really improve my overall skill as a designer and improve the efficiency of how I go through my design process. This is my most important learning point of this elective.

Reflection F. Rümelin I have been learning and practicing user centred design in my previous studies as well as other design approaches. When taking a user centred approach we are often trying to understand the user’s needs and possible problems they are running into, consequently these problems are solved in the form of a design. Now I believe that not all successful or famous designs necessarily solve a problem, some just add an experience or a value to already existing products. Other designs fail because their creator has never thought about what experience the user would have when actually using them. This is why I wanted to learn more about how to use knowledge from literature and practice in design, and chose for this course. My main goal was to learn from reading literature and listening to lectures that are complimentary, for the practical design part I wanted to work on a physical or digital prototype and learn about how to evaluate the experiences of users. I do think that I have learned a lot reading the papers, although it cost me a lot of time to read them, which left less time for the design which I think we could have made good use off to make a more useful and developed design and to do some more testing and application of methods that can be used in UX.

I liked the discussions in groups, but I think that some lectures would have been nice as well, which doesn’t mean that there cannot be a discussion afterwards.

I was a bit disappointed about the fact that the design was used as a vehicle to practice methods, while there was not really time to find out if the concept was any good. Having a brief from the beginning might have helped, or maybe even just in the observation phase to focus on one specific activity. That approach would not only have saved time but resulted in data which would have been easier to compare.

In the end I would also have liked to test out the tool we created to see if it was useful at all, like this I am not sure if I will ever try it because I don’t know if it will help me to assess my design properly, or if it will be as misleading as the heuristics which we came up with. Which had backing from other sources but did not predict that the product was undesirable for the “helper” side.

Since I am also focusing on developing my aesthetic skills it would have been nice if time would have allowed to go

more in depth with the design to a point where aesthetics would make a difference in testing with users, and not merely showing them a rough concept.

Reflection group The group work for this course went very well. After each group meeting, the work was divided equally. We agreed on what had to be finished prior to the next meeting, and what we wanted to do and finish during the next meeting. This helped us to work efficiently as everybody was motivated to work during the meeting as we had a clear goal. As everybody finished their tasks as agreed, the collaboration was professional and pleasant to work with. Also, during the meetings we looked ahead at what had to be finished when over the course of the next weeks. This allowed us to look ahead and tackle timing issues in time (e.g. allowing time to analyse research results). All in all, a great group effort which was well-structured for everybody in our team.

Page 14: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,

APPENDIX B: FINDINGS HEURISTICS

Page 15: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,
Page 16: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,
Page 17: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,

APPENDIX C: CO-CONSTRUCTING STORIES STORYBOARDS

Page 18: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,
Page 19: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,
Page 20: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,

APPENDIX D: FINDINGS CO-CONSTRUCTING STORIES Privacy I don’t want anyone to see how many times I snooze. I do not want other people to see my “mistakes”. I do not want someone to call me in the morning, this is my own time.

- I see no problems with privacy reasons if they both agree on it. Alternative solutions The insight that you do not achieve anything by delaying the moment of getting up. If you want to snooze, put your alarm earlier so you have enough time. Noticing that it does not work if you snooze, negative experiences through oversleeping. Burden of the “helper” It takes a lot of effort for the “helper”, but it is not the “helpers” problem. The one who doesn’t have the problem does the most work. If my friend doesn’t get up it’s not my problem unless we have an appointment. After some time it will be annoying for the “helper”. Can i link my lazy colleague directly to my boss so it has an impact on the impression he makes there?

- It should really not take too long for it to be solved. - The problem should be over in a week. - I would be sick of it really soon. For once it would be fun. After that I wouldn’t wanna do it anymore.

Benefits of the “helper” Nice if you know each other, less nice if you don’t. Thought it was funny that he overslept. Fun to wake someone with my screams. Fun if you have a emotional bond with your buddy.

- If you’re the boss or colleague of someone, it is nice to know if they’re on time or not, and if not, why they’re too late.

- If you’re a good friend, it is nice that you could help. General burdens

- I wouldn’t stop snoozing. - I might be late and stressed anyway because I put my alarm at the latest time possible. - If you get over a certain point of shame, it wouldn’t help anymore.

General Benefits

- It would be much safer, people would be less stressed and hurried in the morning. - People would be coming on work less stressed, be less stressed in traffic and be more clear and awake in meetings. - You would know if someone would be coming on time or not.

Snoozing behavior The alarm is not nice for others. You think you sleep more, but you sleep less. Snoozes daily. No appointments so snoozes. Does not experience snoozing as something annoying, but does have the ideo to suffer from it.

- Does somebody know how much they snooze? - Does it matter how much you snooze if you will be on time anyway. - If I would snooze and I would have a live connection, I would get up immediately. - I wouldn’t do it to get rid of my snoozing behavior. I don’t see it as a bad thing.

Target group Doesn’t have a purpose for people living together. The most notification get send to his girlfriend before she leaves the house.

Page 21: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,

Motivation I would not use it more than a week, to much effort. It seems fun, but I do not see the relevance of the problem, people who snooze are going to keep snoozing. I would only want to help people I know. Why do you need someone else to make recordings at that exact moment, pre-record. It would get boring if the messages are always the same. I would turn it off, because I do not want to hear the whining. Adding milestones to turn into something physical, like a meal to celebrate. More positive approach, don’t show how many times snoozer has snoozed, but show snoozer how many friends are already up.

- I would not use this to get rid of my snoozing habit. I would mostly just do it for fun. Market

- “What kind of channel would you use to market this” - “ How to implement this in the market? It seems to be a product that abolishes itself if it fulfills its purpose as

intended. How do you keep earning money with it?” - “After some time it’s not new and funny anymore. Then it’s merely waking someone up” - For a boss, coworking relationship, I would see a market.

Design

- “ The design is nice and clear it doesn’t have any unnecessary features, maybe you can add some ‘happy’ colours to make waking up more joyful. “

- Misuse

- “Could end up being a communication tool and not fulfill its job to stop snoozing” - “This is more about the social component, but would it then not make some people snooze on purpose for

attention?” - I would use it for fun. Just keep snoozing so I could tell them in a live connection that I would be late. - I would keep snoozing, even when I’m up already, just to mess with them.

Experience

- “Seems like a therapy” - “The experience seems to evoke stress and a bad feeling about yourself”

Social

- “ Social pressure possibly helps with getting up” - “How can I (as a helper) get out of this if my friend still snoozes after 2 weeks - “This would probably not completely mess up my friendships but I think it could for other people. Is that worth it?” - “The helper may get a negative image of the snoozer, especially if he is disturbed by the snoozer all the time” -

App functionality

- “ A live call is too much work and not handy when on the way” - “A live call won’t happen, the recorded messages will be used” - “How do you know that someone really got up and didn’t just switch up his alarm?” - “Would you really still record a message or call if you’re already at work?” - “The helper would only use the record a message function” - “If the helper is sleeping he would be disturbed.” - “Record messages beforehand” - “Can I record messages beforehand for the next few days? I cannot be bothered to do this every morning”

Page 22: Creating a multilateral evaluation tool for UX Design ...€¦ · Creating a multilateral evaluation tool for UX Design based on an exemplary design process B. van Dijk, J. ter Brugge,

- “Don’t let people snooze at all if you want them to stop. Don’t give them the chance to snooze three times before getting up”

- “It’s nice to choose between alarms” - “ Is the call automatically on speaker?” - “Not much changes for the snoozer, while it is a lot of work for the helper” - “ Only the recorded message will be used” - I would want to be able to record a message the first time they snooze. I think I really would want the possibility to

immediately show that they have to get up. - If I would snooze and I would have a live connection, I would turn it off immediately and get up (But it should not

be possible to turn it off, so keep that in mind)