MAIN FIELD: User Experience

AUTHOR: Justyna Sedkowska

SUPERVISOR: Domina Robert Kiunsi

JÖNKÖPING 2020 May

How does the user experience of a progressive web application compare to native application?

A case study on user's attitude in the context of social media.

Postal address: Visiting address: Phone: Box 1026 Gjuterigatan 5 036-10 10 00 551 11 Jönköping

This final thesis has been carried out at the School of Engineering at Jönköping University within [see main field on previous page]. The authors are responsible for the presented opinions, conclusions and results. Examiner: Bruce Ferwerda Supervisor: Domina Robert Kiunsi Scope: 15 hp (first-cycle education) Date: 2020-05-17

Abstract Purpose – The purpose of this thesis paper was to measure differences in the user experience provided by the Progressive Web App technology in comparison to native application in a context of social media. The study investigated users attitude towards different user interfaces implemented in the PWA and the native application. Method – This paper is qualitative research in the field of user experience. Usability tests paired with semi-structured interview were chosen as research methods for this study. The study has been conducted on Twitter's PWA and Android application. Ten participants took part in the study. Findings – The results have shown that most of the participants can identify differences in the user interfaces of both apps. Although both applications provided the same functionalities, the different UI solutions were the major factor influencing users feelings. On top of that, different user interfaces did not change the user's perception of the platform's consistency. As a result, more participants preferred the PWA over the native app. Moreover, results suggest that PWA can provide native-like experience in the context of social media. Implications – The main contribution of this bachelor thesis is a comparison of user experience on PWA and native app technology in the context of social media. This research opens a discussion about the need for companies to broader their digital presence by implementing PWA technology as it can bring benefit to users and businesses. Limitations – The main contribution of this bachelor thesis is a comparison of user experience on PWA and native app technology in the context of social media. This research opens a discussion about the need for companies to broader their digital presence by implementing PWA technology as it can bring benefit to users and businesses. Keywords – User Experience, Progressive Web Application, User Interface, Native Mobile Application, Attitude

Table of contents

1 Introduction .............................................................................. 1

1.1 BACKGROUND ....................................................................................................................... 1

1.2 PROBLEM STATEMENT .......................................................................................................... 2

1.3 PURPOSE AND RESEARCH QUESTIONS .................................................................................... 3

1.4 SCOPE AND DELIMITATIONS .................................................................................................. 3

1.5 DISPOSITION ........................................................................................................................ 3

2 Theoretical framework ................................................................... 4

3.1 LINK BETWEEN RESEARCH QUESTIONS AND THEORY ............................................................. 4

3.2 USER EXPERIENCE ............................................................................................................... 4

3.3 NATIVE MOBILE APPLICATIONS ............................................................................................ 5

3.4 PROGRESSIVE WEB APPS (PWA) .......................................................................................... 5

3 Methods ..................................................................................... 6

2.1 LINK BETWEEN RESEARCH QUESTIONS AND METHOD ............................................................ 6

2.2 WORK PROCESS ..................................................................................................................... 6

2.3 APPROACH ............................................................................................................................ 6

2.4 CHOICE OF PLATFROM .......................................................................................................... 6

2.5 DESIGN ................................................................................................................................. 8

2.6 DATA COLLECTION ................................................................................................................ 9

2.7 DATA ANALYSIS .................................................................................................................... 9

2.8 VALIDITY AND RELIABILITY ................................................................................................... 9

4 Empirical research .................................................................. 11

4.1 PARTICIPANTS… .................................................................................................................. 11

4.2 SUMMARY OF INTERVIEWS .................................................................................................. 11

4.3 AFFINITY DIAGRAM ............................................................................................................ 13

5 Analysis .................................................................................. 15

5.1 RESEARCH QUESTION 1 ....................................................................................................... 15

5.2 RESEARCH QUESTION 2 ....................................................................................................... 15

6 Discussion and conclusions ................................................... 17

6.1 FINDINGS .......................................................................................................................... 17

6.2 IMPLICATIONS .................................................................................................................... 17

6.3 LIMITATIONS ...................................................................................................................... 17

6.4 CONLUSIONS AND RECOMMENDATIONS ............................................................................... 17

6.5 FURTHER RESEARCH ........................................................................................................... 18

References .................................................................................. 19

Appendices ................................................................................. 22

1

1. Introduction The chapter provides a background for the study and the problem area the study is built upon. Further, the purpose and the research questions are presented. The scope and delimitations of the study are also described. Lastly, the disposition of the thesis is outlined.

In 2007, Apple introduced the first smartphone, which changed the everyday lives of many people. It has been estimated that by 2020 there will be 6.1 billion smartphones existing in the world (Lunden, 2015). According to the Liftoff report, most users in South East Asia connect with the Internet through mobile devices (Liftoff, 2020), which proves how important role smartphones play in the digital landscape. Moreover, mobile phones allow users to comfortably complete a variety of different tasks such as shopping online, checking social media, online banking, reading news and more (Statista, 2020).

The arrival of a new type of device built a demand for a more convenient way to browse the Internet on mobile phones. A revolution in building websites began, and Responsive Web Design (RWD) was introduced. RWD is described as "The design of a Web site in such a way that it can take account of the device that it is being displayed on; this can vary from a small mobile phone to a large computer." (Ince, 2013). Websites began to be "mobile-first" meaning designing and developing websites for small screens became the priority (Glassman and Shen, 2014).

The arrival of smartphones came with another revolution - native mobile apps. Mobile apps have many advantages over mobile websites. Native apps can achieve better functionalities and user experience because they can use features of its operating system, such as a microphone, camera or GPS (Glassman and Shen, 2014) and are designed for mobile space.

However, compared to responsive websites, app development and maintenance can be more challenging. Native apps require frequent updates and need to be developed separately for each operating system (Glassman and Shen, 2014) (Malavolta, Ruberto, Soru and Terragni, 2015). This is costly for companies in terms of time and money.

Since the development of native apps requires many resources, cross-platform development became a popular alternative among businesses (Biørn-Hansen, Majchrzak and Grønli, 2017). Many frameworks, such as Ionic Framework, PhoneGap, React Native, and Xamarin has been in use. Such cross-platform frameworks allow developers to assemble one code that will function on different operating systems (Palmieri, Singh and Cicchetti, 2012).

Another approach of making developing across platforms easier came to life with the introduction of Progressive Web Apps(PWA). Until the introduction of PWA, mobile web failed to keep up with native and cross-platform apps (Biørn-Hansen, Majchrzak and Grønli, 2017). The new technology seems to be very promising, bridging the gap between native-like experience and responsive web design (Biørn-Hansen, Majchrzak and Grønli, 2017). Moreover, PWA can work across all mobile operating systems. Therefore they are easier to maintain (Kinsbruner, 2018).

1.1 Background Although Progressive Web Apps are a novelty, data proving its effectiveness already exists. Google has conducted several case studies on technology's impact. In one of these cases, Alibaba's PWA was investigated. Results have shown that conversion rate grew significantly on the mobile web since the PWA was introduced. Moreover, the service gained more monthly active users (Google Developers, 2020). PWA's benefits companies and developers. The technology is easy to maintain and works across different platforms. But what is this new technology bringing to users? Progressive web apps are as available for users as native mobile apps since they can be accessed from the home screen and work regardless of the Internet connection. Since PWA's offer functionalities like camera access, data storage, GPS, motion sensors, and push notifications, they provide a native-like experience (Kinsbruner, 2018). Moreover, a study has shown that PWA requires less memory storage and has faster launch time than hybrid apps developed with Ionic Framework and integrated apps developed with React Native (Biørn-Hansen, Majchrzak and Grønli, 2017). Due to its unique features and native-like experience, PWA's are believed to

2

deliver a user experience as good as native apps. This paper is going to investigate and describe differences in user experience among PWA and corresponding native apps in a social media context with the emphasis on users attitude.

1.1 Problem statement Progressive Web Apps are a new technology. Therefore the amount of available research on this topic is still limited. Among the first ones to introduce PWA's to the scientific community were Andreas Biørn-Hansen, Tim A. Majchrzak and Tor-Morten Grønli. In their paper "Progressive Web Apps: The Possible Web-native Unifier for Mobile Development" researchers analyzed differences between interpreted apps, progressive web apps, hybrid apps and native apps. As a result, research proved that PWA could provide "native-like" experience, although they do have certain limitations compared to a hybrid, interpreted and native apps. For instance, progressive web apps can not access features like a calendar or contact list. On the other hand, researchers concluded that progressive web apps outperform hybrid and interpreted apps when it comes to the size of installation and launch time (Biørn-Hansen, Majchrzak and Grønli, 2017). A bachelor thesis was written by Rebecca Fransson, and Alexandre Driaguine also investigates the technical capabilities of PWA's (Fransson and Driaguine, 2017). In their research, they tried to describe hardware access limitations of PWA's compared to native apps. As a result, only five areas that PWA's could not access have been identified - Local Notifications, Proximity sensor, Geofencing, Wake lock and Contacts. On top of that, PWA's performance of accessing hardware was measured and compared with the performance of a native Android app. Results have shown that Native Android Apps were much faster in accessing cameras. However, PWA's performed better than native apps when it came to accessing geolocation. Another group of researchers investigated the impact of service workers on the energy efficiency of PWA's (Malavolta, Procaccianti, Noorland and Vukmirovic, 2017). The study was conducted on different PWA's with the use of two different smartphones. Researchers were unable to prove a significant difference in energy consumption among PWA's with service workers and the correspondent PWA's without it. When variables of different network connections were introduced to the study, researchers noticed that PWAS running on smartphones connected to WiFi used less energy. However, researchers concluded that it is caused by a higher speed of internet connection which means that assessing data takes less time, therefore less energy. Most of the existing research focuses on PWA's performance and technical capabilities. During literature research, a paper concerning aspects of user experience has been found (de Andrade Cardieri and Zaina, 2018). Researchers carried out an analysis of user experience on PWA, mobile website and a native Android app of TripAdvisor. Both users perspective and human interaction specialist perspective have been taken into account. The study concentrated on user's interactions with specific UI elements such as search input, calendar, select, map, tab menu and table. Participants were asked to conduct a set of tasks on all three platforms and rate their experience in a questionnaire according to SAM's dimension. Later on, HCI specialist analyzed users' facial expressions. Results have shown that PWA brought similar user experience to a native app. When native app and mobile website were compared, results have shown that mobile web provided the better experience in most cases. Comparison between PWA and mobile website has not been made. This bachelor thesis will focus on comparing user experience on a PWA and a corresponded native Android App. Since emotions play an important role in understanding user experience (Partala and Saari, 2015), this research will look into users attitude towards different user interfaces (UI) implemented in a PWA and a native app. Moreover, the study will investigate whether these differences were notable by users and whether they affected the user experience. Unlike the study conducted by de Andrade Cardieri and Zaina, more qualitative methods will be applied to measure the differences in user experience provided by the two technologies. Furthermore, the study will investigate the user experience in the context of social media to see if similar findings are true for a different type of platform.

3

1.2 Purpose and research questions Drawing on the problem statement, it is evident that very little has been done to measure user experience on PWA's. Furthermore, the main focus of the existing research has been on the functionality and capabilities of PWA's and how they could be advantageous for businesses and developers. The use of PWAs and its influence on user experience in a social media context also has not been thoroughly investigated. Therefore, the purpose of this study is to identify PWA's benefits for users in comparison with a native app in social media context with the emphasis on the user's feelings. To be able to fulfil the purpose, it has been broken down into two questions. Q1: What UI differences are implemented in a PWA and a native app? Q2: How do the differences implemented in the UI's impact on users' attitudes? The first research question will investigate whether user's notice differences in the user interface on a PWA and an Android app. The second question will look into users attitude and feeling on the different UI's. To answer the questions and thereby to fulfil the purpose, a case study will be conducted at Jönköping University, Sweden.

1.3 The scope and delimitations The scope of this study is to identify whether differences between user experience delivered by PWA's and native apps exist and whether different user interfaces influence the UX in the social media context. This bachelor thesis will not study PWA's performance and technical capabilities. Moreover, this study only investigates native Android apps. Furthermore, this study focuses on the users' attitude. The main limitation of this study is the low number of participants taking part in the usability test. Due to a difficult situation caused by the global pandemic of Covid-19, limited time and a requirement of social distancing, it was impossible to conduct the study on a larger scale. Those reasons also factored the choice of convenient sampling, where users were selected based on their availability.

1.4 Disposition

This thesis paper is broken down into six chapters: Introduction, Theoretical Framework, Methodology, Empirical Research, Analysis, Discussion and conclusion. The first chapter introduces readers to the topic of this paper and presents the purpose and the research question. The theoretical framework explains relevant terms such as User Experience, the role of attitude in UX, Progressive Web Apps and native mobile apps. The third chapter presents the methodology used in the study. The data collected during the study is included in Empirical Research chapter and analyzed in the next one. The last chapter includes the findings of this study, together with implications, limitations, conclusions and further research suggestions.

4

2. Theoretical framework The chapter presents the theoretical foundation for the study.

2.1 Link between research questions and theory In the following chapter, the theory providing the foundation for answering the research questions is described. To give a theoretical foundation to the first research question "How different are UI's implemented in a PWA and a native app?" and the second research question "How differences implemented in the UI's impact users attitude?" the following areas are described in the theoretical framework: User Experience, Native Mobile Applications, and Progressive Web Applications. User Experience is covered because it is the field in which the study is positioned. Therefore it is crucial for the readers to understand the concept of it. Native Mobile Applications and Progressive Web Applications are covered because they are the subjects of the study.

2.2 User Experience User experience (UX) has many definitions. The International Organization for Standardization defines user experience as "a person's perceptions and responses that result from the use or anticipated use of a product, system or service." (FDIs, I.S.O., 2009.). According to the authors of "Measuring the user experience: collecting, analyzing, and presenting usability metrics." user experience includes three attributes. First of all, a user needs to be involved. Second of all, "user is interacting with a product, system, or really anything with an interface" and lastly "The users' experience is of interest, and observable or measurable" (Albert and Tullis, 2013.). The field of UX aims to describe a feeling that occurs when users use a product (Kraft, 2012.). This means that user experience, unlike the usability, takes a broader perspective and interprets users interaction with a product, including a person's perception of it (Albert and Tullis, 2013.). Usability, often confused with UX, is a part of the user experience term. However, usability concentrates on the performance of a product (Bevan, 2009) and the ability of a user to complete tasks successfully (Albert and Tullis, 2013.).

Role of attitude and emotions in UX Attitude can be referred to feelings one may have towards a person, a product, a company or an experience. In the field of UX user's attitude plays a major role as good user experience should evoke positive feelings like happiness or satisfaction, and all negative emotions should be avoided (Kraft, 2012). According to the Tripartite Model of Attitude, in short, ABC model, attitude can be composed out of three components: cognitive, affective, and conative. Cognitive refers to beliefs, affective refers to feelings, and cognitive refers to intentions. Each of these components correlates with each other and can be measured using rating scales like SUS, SUPR-Q or UMUX (Sauro, 2019). In "Technology as Experience" authors introduced a framework that analyzes the experience with technology. The framework is build of four Threads of Experience (compositional, sensual, emotional and spatio-temporal) (See

Figure 1 Threads of experience (McCarthy and Wright, 2004)

5

Figure 1) which are influenced by The Six Sense-making Processes (McCarthy and Wright, 2004). According to McCarthy and Wright, emotions play a major role in the experience as humans tend to summarize experience by feelings like excitement or frustration. Feelings were investigated in many other frameworks and models. In the Technology Acceptance Model 3, perceived enjoyment and computer anxiety was proven to relate to easy of use (Venkatesh and Bala, 2008). A different study investigated how emotions that users experience at the early stage of implementation of a new IT solution in the company relates to its use later. As a result, researchers found direct and indirect connections between emotions and IT use (Beaudry and Pinsonneault, 2010) and prove the role of users feelings in technology acceptance.

2.3 Native Mobile Applications Today's app market offers a variety of native apps as well as cross-platform apps. Since the research is investigating native mobile applications, this part will only focus on describing native apps. Native mobile applications are developed using a programming language specific to a platform, for instance, Java for Android and Objective-C for iOS (Que, Guo and Zhu, 2016). This means that the same application needs to be developed separately for each operating system. One of the characteristics of native code is that it is compiled, meaning it is faster than interpreted programming languages like Javascript (Charland and Leroux, 2011). Another characteristic of the native apps is their ability to access build-in features of its operating system such as a microphone, camera, contacts, GPS etc. (Glassman and Shen, 2014). Thru this powerful functionalities, native apps can provide a great user experience, which can be extended outside of an app (Charland and Leroux, 2011). Native apps can be installed on the device by downloading it from a respective application store, for instance, Play Store for Android and App Store for iOS. Moreover, they require frequent updates. When it comes to a user interface, native apps can attain great looks by using a platform's widgets and offer outstanding user interaction (Charland and Leroux, 2011). They are easily accessible as well since users can enter an app from a home screen.

2.4 Progressive Web Apps (PWA) As a solution to problematic native app development, Google has introduced new technology in 2015 - Progressive Web App. PWA's are constructed of three components. First one, an app shell, which uses HTML, CSS and Javascript and can be cached offline (Kho, 2018). The second component, service workers, which allow developers to handle functionalities like push notifications even when there is no connection to the Internet (Kinsbruner, 2018). Last component, an app manifest, makes it possible to install PWA from a browser and provides elements characteristic to native apps like home screen icon and splash screen (Kho, 2018) (Kinsbruner, 2018). PWA's can be characterized by six principles (de Andrade Cardieri and Zaina, 2018) (See Figure 2).

PWA Principles

I responsive

II connectivity independent

III similar to native applications

IV re-engageable through push notifications

V installable

VI self-updating

Figure 2 Principles of PWA (de Andrade Cardieri and Zaina, 2018)

6

3. Methods and implementation The chapter provides an overview of the work process of the study. Further, the approach and design of the study are described as well as the data collection and data analysis. The chapter ends with a discussion about the validity and reliability of the study.

3.1 Link between research questions and methods The following chapter describe the chosen methods for data collection and data analysis as well as how they answer the research questions. To answer both research questions, usability testing has been conducted. Usability tests are defined as "the activity that focuses on observing users working with a product, performing tasks that are real and meaningful to them" (Barnum, C.M., 2011). After the usability test on each type of platform, study participants have been interviewed. The goal of the interviews was to identify users feeling about their interaction with the PWA and the native Android app, using the interview allowed for gaining a deeper understanding of the topic.

3.2 Work process

The study began with preparations of the methods. During that stage, usability tests were prepared—this included selection of a suitable platform, preparations of a scenario and an interview. Later on, the methodology has been tested with one participant and adjusted to allow for the best possible performance known to me. During the preparations, stage participants have been recruited. During a conduct stage, participants took part in the usability tests, which have been recorded and transcripted. At the end of this study, a thorough analysis of gathered research material has been done. The process is illustrated in a figure (See Figure 3).

3.3 Approach A case study was chosen as an approach for this bachelor thesis. According to Blomkvist and Hallin, a case study has four characteristics. First of all, a case study chooses to investigate one or more cases to draw a picture of the phenomenon. Secondly, the phenomenon is being studied in real life. Third of all, "the method incorporates gathering a sufficient amount of the information" which can be used to explain the phenomenon that the study is investigating. Lastly, the research has to be conducted systematically (Blomkvist and Hallin, 2015). To knowledge and understanding, this thesis paper fulfils all the characteristics mentioned above.

3.4 Choice of platform Given the time constraints, only one type of social media platform has been used in the usability test. Twitter was selected for this study because it had a PWA with a corresponding native Android application. Moreover, the PWA and the native app had different UI's (See Figure 4 and Figure 5).

Figure 3 The work process of the study

7

After the analysis of both platforms, few differences in UI's were noted. Overall, the native app seems to have smoother interactions than the PWA, especially noticeable when entering the message view when the "create a Tweet" button animates into "write a message" button. Changes in the UI occur in places such as notification settings, display settings, creation of a Tweet or side menu.

Figure 4 Twiter's UI on PWA

Figure 5 Twiter's UI on native application

8

3.5 Design To answer the research questions, participants were asked to complete a set of two tasks on the PWA and the native Android app of Twitter. Tasks were formulated based on the platform, the user's goals and the UI's analysis (See Figure 6).

User’s Tasks

I Mute a keyword that you find offensive or uninteresting.

II Create a Tweet and add image to it.

All of the participants were asked to complete the tasks on Twitter PWA and Twitter native app. During the usability tests, participants were presented with a scenario to help them navigate towards the tasks (See Figure 5). Same tasks were done on the PWA and the native app.

Task Scenario

Mute a keyword that you find offensive or uninteresting.

Please, imagine that you have been annoyed by notifications that mention Donald Trump. You would like to mute this keyword “Donald Trump” and stop receiving notifications about it. Could you please try to find out how to do it. When you have completed this task, let me know that you are done.

Create a Tweet and add image to it. Please, imagine that you really excited about your family meeting and want to share it and show a picture you too that day. I would like you to create a Tweet about it with a picture. When you have completed this task, let me know that you are done.

After completing the user test on the PWA and the native app, participants were interviewed. This way, it was possible to collect data which allowed for analysis and comparison of users experience. For the interviews, a semi-structured interview technique has been chosen. It allowed for more flexibility and gathered insight through open-ended questions. Interview questions were constructed to answer the research questions (See Figure 7). Some of the interview questions were based on the UEQ measuring scale. Factors from the scale that were most relevant for this study such as friendliness, attractiveness, consistency and pleasingness were chosen. On top of that, additional follow up questions were asked during the interview if more in-depth information was needed.

Interview Question

I How often do you use Twitter?

II Have you noticed any differences in the user interface when you were conducting the first task (muting a keyword)? If yes, what were they?

III What are your thoughts on these differences?

Figure 6 Tasks for usability test

Figure 6 Tasks for usability test

9

IV

Did you find conducting the first task more or less complex on one of the platforms?

V

Have you noticed any differences in the user interface when you were conducting the second task (create a tweet)? If yes, what were they?

VI What are your thoughts on these differences?

VII Did you find conducting the second task more or less complex on one of the platforms?

VIII Do you find one of the platforms less or more inconsistent?

IX Do you find one of the platforms more or less friendly?

X Do you find one of the platforms more or less unattractive?

XI Do you find one of the platforms more or less pleasing?

XII If you were to use Twitter in the future, would you rather use PWA or a native app?

3.6 Data collection Data has been collected from 10 participants. Due to the difficult situation caused by the global pandemic of Covid-19 and limited access to participants, a convenient sampling method was chosen. Therefore, the age of participants varies from 18 to 48. Moreover, participants selected for this study had a different level of experience with Twitter which has been taken into account in the analysis. During the tests, all participants were using the same Android device connected to WiFi. Moreover, for the study and to avoid using personal accounts of participants, a special account on Twitter was created. All interviews have been recorded and transcripted, observations made during the usability tests were noted. The tests were conducted in person in a controlled environment.

3.7 Data analysis In the field of user experience, it is advised to conduct preliminary analysis on collected data following each usability test and adjust the test (Courage and Baxter, 2005). However, for this thesis paper, the data was analyzed after all usability tests have been done. This way standardization across the usability tests was guaranteed. However, there has been done a preliminary test of methods which allowed for adjustments and changes in how the usability tests were initially conducted. Since the study collected qualitative data, an affinity diagram was used for its analysis. Affinity diagrams help in grouping similar patterns and identifying trends in the data (Courage and Baxter, 2005). This technique allowed for fast and effective analysis. Moreover, the interviews have been summarized.

3.8 Validity and reliability In the field of user experience, it is advised to conduct preliminary analysis on collected data following each usability test and adjust the test (Courage and Baxter, 2005). However, for this thesis paper, the data was analyzed after all usability tests have been done. This way standardization across the usability tests was guaranteed. However, there has been done a preliminary test of methods which allowed for adjustments and changes in how the usability tests were initially conducted.

Figure 7 Interview Questions

10

Since the study collected qualitative data, an affinity diagram was used for its analysis. Affinity diagrams help in grouping similar patterns and identifying trends in the data (Courage and Baxter, 2005). This technique allowed for fast and effective analysis. Moreover, the interviews have been summarized.

11

4. Empirical data The chapter provides an overview of the empirical domain which forms the basis of this study. Further, a description is given of the empirical data that has been collected to answer the research questions.

4.1 Participants

Participant Number

Age Gender Occupation

1 18 Male Student

2 21 Male Student

3 18 Female Student

4 23 Male IT specialist

5 19 Male Student

6 22 Male Student

7 21 Male Student

8 48 Female Teacher

9 21 Female Student

10 21 Female Student

4.2 Summary of interviews This section summarizes all of the interviews which were conducted in English. Since some of the interviews were long, summarizing was essential. Consequently, a summary of the interview only includes essential and relevant information.

Interview 1 The participant began the tests by conducting the tasks on the PWA, and as he stated, he had no previous experience with the platform. When asked whether he noticed any differences in the UI while conducting the first task (mute a keyword), he pointed out that the second app he tested (native app) was nicer and more useful. When asked about specifics, he answered, "I can't remember what they were, but it was different.". The user was able to point out a difference in the process of creating a tweet and indicated that different methods of adding images were implemented on the apps. When asked about the complexity of completing the tasks, answered that both apps were easy for him to use. Overall, the participant showed a more positive attitude towards the native app and answered that the PWA was less attractive, friendly and pleasing to him. Lastly, he pointed out that in the future, he would rather use the native app, and it suited him the most.

Interview 2 This participant began the test by conducting the tasks on the native app first. He stated that he uses Twitter occasionally. The user pointed out differences in the UI, such as less visible advanced settings on the native app when muting a keyword. He also pointed out that the overall interface was in his opinion more "modern" on the native app and described the PWA's

Figure 8 List of participants

12

as "a few years older application". In the interview, he also talked about how the process of adding images was different and that he considered the native app as less complex to create a tweet and easier to select images. Overall, the user gravitated towards the native app and implied that he sees it as more friendly and pleasing and his preferable choice. However, he remained neutral when asked to rate inconsistency and attractiveness of the two apps.

Interview 3 The participant began the test on the PWA. The user was very quick with conducting all given tasks, although she had never used Twitter before. When asked about differences between the two presented apps, she could not point any differences and stated that both apps were very similar to her. The user was very neutral and did not show any particular preference or attitude. The only difference she notices was the shape of home screen icons and her preference for PWA's round icon.

Interview 4 The user began the usability test with the native app and declared himself as a frequent user. When asked about differences while conducting the first task, he pointed out that the native app had a bigger font and less visible options. The PWA was for him more minimalistic and "intelligently designed with a lot more things to customize". When asked about visible differences while conducting the second task, he noticed a different way of adding images to a tweet. The user showed a more positive attitude towards the PWA and answered that it is more attractive. However, he showed a neutral attitude when asked about complexity, consistency and friendliness. When asked about future use, he stated that he would prefer to use the PWA and it appears to be faster.

Interview 5 The participant began the usability test with the PWA and had no previous experience with the platform. He pointed out differences in layout or available options that he noticed while muting the keyword. When asked about his thought about it, he stated: " I think that they are both very similar; however, the second option looks more friendly (Native app), because it looks better for the user than the first one (PWA).". He also noticed a difference in how he could upload a photo to a tweet. On top of that, he mentioned that "the second one (native app) it was more clicking but more visually appealing while on the first one it was faster but kind of less attractive.". He remained neutral when asked about the consistency of apps. However, he strongly showed his preference for the native app and claimed it to be more friendly, attractive and pleasing than PWA. When asked about future use, he stated that he would prefer to use the native app because the interface was nicer.

Interview 6 This user began the usability test with the native app. He stated that he uses Twitter once a week or less. To complete the first task on the native app, the participants entered settings through the hamburger menu and successfully muted a keyword. In the case of PWA, he took different steps through notifications view and accessed the notification settings from there. Therefore when asked about differences that he noticed while conducting the first task, the user mentioned that he had to take different steps. Nevertheless, he indicated " I think it was easier for me in the first app". Moreover, the participant was able to identify differences in how images could be added and said that how photos were displayed on the native app was more friendly. When asked about complexity, he pointed out that it was harder to conduct the tasks on the PWA. The user showed a strong positive attitude towards the native app and declared he would rather use it in the future.

Interview 7 This participant began with the PWA and declared he had never used the platform before. He was able to identify differences in the UI when conducting the first task. "The first difference I noticed was the graphics..." he said and added that it was easier to notice more advanced settings on the PWA. While creating a tweet, the user noticed the difference in how one can add an image and said: "... in the second one (PWA) you could not realize where to write the text or where you need to take a photo". Overall, the participant showed a more positive attitude towards the PWA and rated it more friendly, attractive and consistent than the native app. However, when asked "Do you find one of the platforms more or less pleasing?", he pointed

13

out that the native app was more pleasing because it had bigger font. In the end, he declared he would rather use the PWA in the future.

Interview 8 This participant began with the native app first and had no previous experience with the platform before. When asked "Have you noticed any differences in the user interface when you were conducting the first task (muting a keyword)? If yes, what were they?", she pointed out that she had to use different steps to achieve her goal. Just like participant number six, she used two different ways to complete the task. She thought that the native app is for more advanced users and found it harder to use. She also pointed out that the two apps used a different way of selecting images when creating a tweet. She did not have strong opinions on apps attractiveness or friendliness and found both apps to be consistent. In the end, she declared that she would rather use the PWA in the future.

Interview 9 The user began the test with the PWA. She had no previous experience with Twitter. She was able to identify the differences in both apps after conducting the first task. She pointed out that the layout was different and said that the PWA was more readable to her. She felt the differences were not "drastic"; however, she liked the PWA more. When asked about differences while conducting the second task, she said that the differences were small, but "...it was different where to add text to it.". The participant did not have a strong opinion about the attractiveness of two apps and rated them as equal. She also did not think any of them was more pleasing or less inconsistent. She pointed out that the PWA was more friendly and would rather use it in the future.

Interview 10 The participant began the usability test with the native app and had no previous experience with the platform. She was not able to present any differences in the UI after conducting the first task and had no thoughts about the two apps. After conducting the second task, the user pointed out the difference in adding images to the tweet and that she could not access the gallery from the PWA. When asked about her opinion, she said: "No, I can not say I have any special thoughts about these differences, because it is nothing big.". Later, she pointed out that the native app was more attractive to her. When asked why, she said, "I can't really explain it, it is just a feeling.". She also rated the native app as more pleasing and declared she would rather use it in the future.

4.3 Affinity Diagram This part is presenting an affinity diagram based on interviews conducted with the participants. Affinity diagram has been chosen, since it allows to group the recurring patterns in the data, add structure and gain clarity on the problem efficiently. Moreover, the affinity diagram technique is also recommended by authors of "Understanding your users: A practical guide to user requirements methods, tools, and techniques" for analyzing data collected during focus groups, diary studies, interviews, or field studies (Courage and Baxter, 2005).

Users about differences in the UI’s when adding a photo to a tweet.

● "...when adding a picture, there was a difference."

● "In the first app (native app), the latest pictures were just already preloaded in the

app..."

● "...the second one (PWA) I had the option to pick up the camera..."

● "In the second one (native app) when you wanted to add a photo, you had a

gallery and the camera together."

● "...in the first one (PWA) you had to choose either to use the gallery or the camera

only.",

● "In the first platform (native app), I had the list of the pictures already..."

● "...the pictures were displayed differently."

● "Maybe the second time (PWA) I made a picture by myself, and that was a difference because I could not find the photo library."

14

Users about differences in the UI’s when muting a keyword.

● "...when adding a picture, there was a difference."

● "In the first app (native app), the latest pictures were just already preloaded in the

app..."

● "...the second one (PWA) I had the option to pick up the camera..."

● "In the second one (native app) when you wanted to add a photo, you had a

gallery and the camera together."

● "...in the first one (PWA) you had to choose either to use the gallery or the camera

only.",

● "In the first platform (native app), I had the list of the pictures already..."

● "...the pictures were displayed differently."

● "Maybe the second time (PWA) I made a picture by myself, and that was a

difference because I could not find the photo library."

Users on if they find one of the platforms inconsistent

● “They are equal to me.” ● “...they both felt very similar and consistent” ● “No, I don’t think so. They are both consistent.” ● “I don’t think I find any of them to be inconsistent.” ● “To me both of them were consistent.” ● “No, I think they both worked well and were comparable to each other.” ● “No, for me no.”

15

5. Analysis The chapter answers the research questions by processing the collected empirical data and the theoretical framework.

5.1 What UI differences are implemented in a PWA and a native app? This section answers the first research question "What UI differences are implemented in a PWA and a native app?". After conducting all the usability tests and gathering all the interviews, it is clear that most of the participants were able to notice differences in the UI's of the native app and the PWA. Only participant number three was not able to identify any differences while conducting the first and second task. Participant number ten was not able to identify differences while conducting the first task, but she noticed changes in the UI while doing the second task. Two participants, number six and eight, took two different paths to mute a keyword on each app and pointed out that as the major difference in the UI's. However, since both apps have two ways of completing the first task, either through a side menu or notifications view, these differences were irrelevant for this study. While doing the first task, most participants pointed out that the layout of the two applications was different. Many noticed that PWA had better visibility of advanced settings in muting the keyword section. Some also noticed that the font was bigger on the native app. Participant number two implied that the native app had more "modern look". On the other hand, participant number four said about the PWA that it is "more minimalistic and more intelligent designed". While doing the second task, participants pointed out different UI solutions implemented in the apps that enabled them to add images. They identified that the native app was displaying images from the phone's gallery within the app's interface and had camera access incorporated in the application. When using the PWA, most of them noticed they had to pick the source of the image first - whether it was a camera or a gallery. Few found a difference in how they added a text to a tweet.

5.2 How do the differences implemented in the UI's impact on users' attitudes?

This section answers the second research question "How do the differences implemented in the UI's impact users' attitudes?". As stated above, most of the participants had been able to notice differences in the UI's, and some of them showed a strong opinion on the different user interface solutions. Participant number two pointed out that the native app was more friendly for him, explaining that "...the first app (native app) seemed more modern and everything I needed was visible". He also said the native application was more pleasing. Participant number one also showed a more positive attitude towards the native app and gave the looks of the interface as the reason for it. Participant number five had a similar feeling about the native app and rated it more friendly, attractive and pleasing than the PWA. When asked "Did you find conducting the second task more or less complex on one of the platforms?" he responded that although PWA seemed faster for him, it was "less visually appealing". The last participant declared she would rather use the native app in the future and found it more pleasing and attractive. When asked why she found the native application more attractive, she responded: "I can't really explain it, it is just a feeling that next time I would prefer to use the first app (native app).". Some of the participants found the interface of the PWA better. Participant number four was very found of the PWA and liked the fact that it offered more settings while muting a keyword. He also found it easier and faster to conduct both tasks. The user thought that the PWA was more attractive and according to him, it was "more minimalistic and more intelligent designed". Probably because he was a more frequent user of Twitter, he liked the fact that PWA allowed him to advance the settings. However, it must be pointed out that the native app also allowed for same advanced settings while muting a keyword. Participant number seven also showed a more positive attitude towards the PWA. He pointed out that the advanced settings

16

were more visible while conducting the first task. Moreover, he preferred how images were added on the PWA and said that it was "more clear when I wanted to post something". He found the native app less attractive and pointed out that the advanced settings while muting a keyword were significant to him because he thought it would help him filter out unreliable sources and keep up with authors he liked. He also gave visibility of advanced settings as a reason he would prefer to use the PWA in the future. Participant number eight did not have such strong feelings about the PWA. Nevertheless, she said she would rather use it in the future because it was easier for her. She struggled with the given tasks since she had no experience with the platform. Therefore the reason for her preference might be caused by her understanding the platform better after a first try. Her response to one of the questions was "For me, the second one seemed easier, but I also knew what to look for. I think the first app is for more advanced users." which suggests that. Participant number nine also preferred the PWA and justified its choice by saying that "It seems to me that it is more friendly and looked a bit better.". After analyzing the interviews, it was easy to see that participants who showed a more positive attitude towards one of the applications, also preferred to use it in the future. This does not come as a surprise since this phenomenon has been already researched. A study has proven that higher satisfaction leads to higher loyalty (Joel and Carmina, 2008). Another study has shown that users emotions have a direct and indirect connection to future IT use (Beaudry and Pinsonneault, 2010). One of the interesting observations of this study was that most of the users perceived both platforms as equally consistent despite different UI solutions implemented in them. When asked to rate the consistency of both platforms, participant number four said that both apps felt very similar and consistent. Participant number five, six, eight, nine and ten shared similar opinions and did not find any of the applications more or less inconsistent. Participants number two and one remained very neutral and responded to this question indirectly saying that consistency might be perceived differently depending on the individual needs of other users. Although some users prefer one platform over the other, they did not think that PWA was any different application than the native app. Participant number three did not see any differences in the apps except for a different shape of the home screen icons. Only participant number two said about the PWA "It looks like it is not a mobile app, but a browser loaded Twitter.". Interestingly, more participants declared they would rather use the PWA in the future, which gives hope that the new technology has the potential to gain popularity among users.

17

6. Discussion and conclusion This chapter summarizes the results of the study. Further, it describes the implications and limitations of the study. It also describes the conclusions and recommendations of the study. Suggestions for further research are given at the end of the chapter.

6.1 Findings This paper presents a qualitative study over progressive web apps and native Android application. The thesis investigated how differences in the user interface implemented in both apps determine users attitude and whether PWA technology can provide a native-like experience in the context of social media. First and foremost, this study showed that most of the participants can identify differences in UI's and that they matter for them. Although both applications provided the same functionalities in given tasks, the different UI solutions were the major factor influencing users feelings. From the results of the study, it is clear that when participants developed a more positive attitude towards a platform, they also preferred to use in in the future. One of the most surprising findings of this study is that most of the participants rated both platforms equally consistent, despite noticing differences in the user interfaces. This suggests that although PWA is built with a different technology than native applications, it can provide native-like experience in the social media context. Moreover, most of the participants did not imply that PWA is not a native app and saw them as equal. Furthermore, most of the participants preferred to use PWA in the future, which suggests that the technology has the potential to gain acceptance among users and expand its territory in the digital landscape. Given other advantages of PWA's such as low memory storage and easier development, the technology has the potential to gain popularity among users and companies.

6.2 Implications One of the main contributions of this thesis is a comparison of PWA and native app technology in the context of social media. The study showed that the PWA could keep up with the native application even in a demanding environment of social media, where users have high expectations towards services and its interactivity. This study opens a discussion about the need for companies to implement the PWA technology for it can be beneficial both for business and end-users. The results of the study have shown that the new technology does not influence the integrity of the service, for most of the participants did not perceive the consistency of the platform to be influenced by the different technology or user interface. Furthermore, most of the study participants preferred to use Twitter's PWA in the future, which gives hope that technology has the potential to gain popularity among users and return the investment of companies.

6.3 Limitations The main limitation of this study is the low number of participants taking part in the usability test. Due to a difficult situation caused by the global pandemic of Covid-19, limited time and a requirement of social distancing, it was impossible to conduct the study on a larger scale. Those reasons also factored the choice of convenient sampling, where users were selected based on their availability.

6.4 Conclusions and recommendations As stated above, this study has shown that the PWA technology can provide native-like experience in the social media context. Therefore, it is recommended for companies to reconsider the PWA technology as it is not only beneficial for them but also the users

6.5 Further research Since the topic of PWA technology is a novelty, and little has been done in the scientific community to investigate it, there is plenty of space for potential research. First of all, it would be interesting to see the impact of PWA installation process on users. Such a study would add

18

another dimension to how PWA's are received by users. It would be beneficial to compare PWA's and mobile websites, especially from the perspective of users. Research concerning how PWA corresponds with native apps in a context of retail, online banking or gaming would be beneficial for the industry and the scientific community.

19

7. References Agar, J., 2013. Constant touch: A global history of the mobile phone. Icon Books Ltd.

Albert, W. and Tullis, T., 2013. Measuring the user experience: collecting, analyzing, and presenting usability metrics. Newnes. Barnum, C.M., 2011. Usability testing essentials: ready, set--test/Carol Barnum. Burlington, MA: Morgan Kaufmann Publishers. Beaudry, A. and Pinsonneault, A., 2010. The other side of acceptance: studying the direct and indirect effects of emotions on information technology use. MIS quarterly, pp.689-710. Bevan, N., 2009, August. What is the difference between the purpose of usability and user experience evaluation methods. In Proceedings of the Workshop UXEM (Vol. 9, pp. 1-4). Biørn-Hansen, A., Majchrzak, T.A. and Grønli, T.M., 2017, April. Progressive Web Apps: The Possible Web-native Unifier for Mobile Development. In WEBIST (pp. 344-351). Blomkvist, P. and Hallin, A., 2015. Method for engineering students: Degree projects using the 4-phase Model. Studentlitteratur AB.

Charland, A. and Leroux, B., 2011. Mobile application development: web vs. native. Communications of the ACM, 54(5), pp.49-53. Courage, C. and Baxter, K., 2005. Understanding your users: A practical guide to user requirements methods, tools, and techniques. Gulf Professional Publishing. de Andrade Cardieri, G. and Zaina, L.M., 2018, October. Analyzing User Experience in Mobile Web, Native and Progressive Web Applications: A User and HCI Specialist Perspectives. In Proceedings of the 17th Brazilian Symposium on Human Factors in Computing Systems (pp. 1-11). FDIs, I.S.O., 2009. 9241-210 (2009). Ergonomics of human system interaction-Part 210: Human-centered design for interactive systems (formerly known as 13407). International Organization for Standardization (ISO). Switzerland. Fransson, R. and Driaguine, A., 2017. Comparing progressive web applications with native android applications: an evaluation of performance when it comes to response time. Glassman, N. and Shen, P. (2014). One Site Fits All: Responsive Web Design. Journal of Electronic Resources in Medical Libraries, 11(2), pp.78-90. Google Developers. (2020). Alibaba | Web | Google Developers. [online] Available at: https://developers.google.com/web/showcase/2016/alibaba [Accessed 20 Feb. 2020]. Ince, D. (2013). A dictionary of the Internet. Oxford: Oxford University Press.

20

Joel, E. and Carmina, F., 2008. Consumer satisfaction, A key factor of Consumer Loyalty and Buying Intention of a Product Developing Organization for Food Product. Kho, N.D. 2018, "EVERYTHING YOU NEED TO KNOW ABOUT PROGRESSIVE WEB APPS", EContent, vol. 41, no. 2, pp. 20-24. KINSBRUNER, E., 2018. How progressive web applications (PWAs) are revolutionizing user experience. InfoWorld.com Kraft, C., 2012. User experience innovation: User centered design that works. Apress.

Liftoff. (2020). 2019 Southeast Asia App Engagement Report. [online] Available at: https://liftoff.io/resources/report/southeast-asia-app-engagement-2019/?submissionGuid=25d603de-a423-4b74-bb13-67e97efb80a2#benchmarks [Accessed 7 Feb. 2020]. Malavolta, I., Procaccianti, G., Noorland, P. and Vukmirovic, P., 2017, May. Assessing the impact of service workers on the energy efficiency of progressive web apps. In 2017 IEEE/ACM 4th International Conference on Mobile Software Engineering and Systems (MOBILESoft) (pp. 35-45). IEEE. Malavolta, I., Ruberto, S., Soru, T. and Terragni, V., 2015, May. Hybrid mobile apps in the google play store: An exploratory investigation. In 2015 2nd ACM international conference on mobile software engineering and systems (pp. 56-59). IEEE. McCarthy, J. and Wright, P., 2004. Technology as experience. interactions, 11(5), pp.42-43.

Natarajan, T., Balasubramanian, S. and Kasilingam, D. (2020). Understanding the intention to use mobile shopping applications and its influence on price sensitivity.

Palmieri, M., Singh, I. and Cicchetti, A., 2012, October. Comparison of cross-platform mobile development tools. In 2012 16th International Conference on Intelligence in Next Generation Networks (pp. 179-186). IEEE. Partala, T. and Saari, T., 2015. Understanding the most influential user experiences in successful and unsuccessful technology adoptions. Computers in Human Behavior, 53, pp.381-395. Que, P., Guo, X. and Zhu, M., 2016, December. A comprehensive comparison between hybrid and native app paradigms. In 2016 8th International Conference on Computational Intelligence and Communication Networks (CICN) (pp. 611-614). IEEE. Sauro, J., 2019. Why You Should Measure UX Attitudes? Measuring Usability LLC.Available at: https://measuringu.com/ux-attitudes/ Statista. (2020). Leading mobile internet activities by device 2017 | Statista. [online] Available at: https://www.statista.com/statistics/249761/most-popular-activities-carried-out-on-mobile-internet-devices/ [Accessed 7 Feb. 2020]. Lunden, I., 2015. 6.1 b smartphone users globally by 2020, overtaking basic fixed phone subscriptions. Tech Crunch, 2.

21

Venkatesh, V. and Bala, H., 2008. Technology acceptance model 3 and a research agenda on interventions. Decision sciences, 39(2), pp.273-315.

22

8. Appendices

8.1 Transcript of interviews

Participant 1 The participant began the tests by conducting the tasks on the PWA and secondly on the native app. The participant had no issues with completing the tasks. The person conducted the first task by accessing settings located in the hamburger menu in both cases. When it comes to adding a tweet, the participant created the post by tapping on the round blue icon at the right bottom corner in both cases and select the images already existing in the phone gallery. How often do you use Twitter? Never use twitter. Have you noticed any differences in the user interface when you were conducting the first task (muting a keyword)? If yes, what were they? Yes, the second one was nicer, more useful. But have you noticed any specific differences? I can't remember what they were, but it was different What are your thoughts on these differences? The second app (Native) was nicer than the first one. Did you find conducting the first task more or less complex on one of the platforms? Both were the same easy for me. Have you noticed any differences in the user interface when you were conducting the second task (create a tweet)? If yes, what were they? Yes, I noticed that when adding a picture, there was a difference. What are your thoughts on these differences? I think it looks better. But which one do you think looks better? On the second application (Native). Did you find conducting the second task more or less complex on one of the platforms? For me, it was all very easy. Do you find one of the platforms less or more inconsistent? Oh well… The first platform had it so that when I entered the settings to mute the keyword, there was privacy and safety while the second one had mute the keyword right away and that was more practical and better for me. Does what you said means that you think one of the platforms was more inconsistent? Yeah, the second one was better. Do you find one of the platforms more or less friendly? The second one was more friendly to me. Do you have any reason for that? Yeah, because of the view. Do you find one of the platforms more or less unattractive? Well, I think they were similar but the second one was more attractive. Do you find one of the platforms more or less pleasing? The second one seemed more satisfying for me because of the interface. If you were to use Twitter in the future, would you rather use PWA or a native app? I would rather use the second one (Native) because it is nicer for me.

Participant 2 This participant began the test by conducting the tasks on the native app first. The participant struggled with finding the option for muting the key world while conducting it for the first time on the native app. After a while, he was given a hint that helped him quickly navigate to the right place. As he admitted, he did not understand that the keywords are supposed to be muted in notifications and was trying to look for it in content settings. He completed this task by accessing notification settings in the hamburger menu in both cases. When the participant was creating a tweet, he used the round blue icon at the right bottom. He was confused with the

23

number of options available to him when he wanted to add a photo and did not know which source (gallery, camera or documents) to use on PWA. Eventually, he selected already existing images in the phone's gallery in both cases. How often do you use Twitter? Occasionally I use it, but not too often. Have you noticed any differences in the user interface when you were conducting the first task (muting a keyword)? If yes, what were they? So the first app that you asked me to use (Native App) seems a little more user friendly, it had less visible options, but the options that were visible were the ones I would probably use. So there were a little bit fewer things that I could do, but all the things that I needed were visible. And the second app ( PWA) it seemed a little more complicated, but I could do more like muting a keyword, I could choose for how long I want it muted and whose tweets I don't want to see. I didn't really see that option in the first app. Maybe it was there but not visible for me. And when posting a tweet, they were very similar, and they were just looking a bit different. The interface on the first app was modern, and the second app it was like, I would say it was a few years older application, but adding a picture was not a problem. Do you have any more thought on it? No, I think I said it all. Nothing comes to my mind right now. Did you find conducting the first task more or less complex on one of the platforms? It was a little bit complicated for me because I don't use Twitter every day, so I didn't know where to look for it first. But when I got the hint to search for it in the notifications menu, it was easier. In the first app, it was easier because there were fewer options that I could choose, just mute the word, and it was done. In the second, when there are more options, so it was more complex because of that, but it also wasn't hard when I had to look for it. So you also talked about the differences you noticed while doing the second task, so I am going to ask you whether you find conducting the second task more or less complex on one of the platforms. It was more complex on the second platform because I had to choose where do I want the picture to be uploaded from, like do I want to take a picture right now or is it from my documents or if have to search for it on my phone. In the first app, the latest pictures were just already preloaded in the app so I could choose one of them. The pictures I would most probably want to upload were already prepared for me, so it was easier. Do you find one of the platforms less or more inconsistent? It actually depends. If I was a new user, I would probably like more the first app, and it is better for me because it seemed a bit easier. On the other hand, for someone who is a bit more into Twitter and is more advanced, the second app offered more things. Do you find one of the platforms more or less friendly? Yes, the first one is definitely more friendly, the second one was less friendly for me. Do you have any reason for that? I think I already said it, the first app seemed more modern, and everything I needed was visible. Do you find one of the platforms more or less unattractive? It all depends on people, but I think the first one is modern and the second one is a bit raw. I look like it is not a mobile app, but a browser loaded Twitter. Some people might like it. Do you find one of the platforms more or less pleasing? I would say the modern one (Native) is actually more pleasing today because the graphic design is more user friendly. If you were to use Twitter in the future, would you rather use PWA or a native app? Based on these two exercises I probably would use Twitter just for myself like browsing thru other posts and not creating tweets I would see the first app because it is more friendly, it seems that I would find the things I am interested in quicker than it would take less time.

Participant 3 The participant began the test on the PWA. The participant was very quick with conducting all given tasks. She muted the keyword by accessing settings from the notification view in the app in both cases. The person added tweets by tapping on the round blue icon at the right bottom in both cases. She was a bit confused while having to choose a source of the image on the PWA and decided to take pictures on the spot.

24

How often do you use Twitter? I have never used it before. Have you noticed any differences in the user interface when you were conducting the first task (muting a keyword)? If yes, what were they? Not really, because both had the same way of getting to the muting function. I just had to write a word, so it was not a big difference. What are your thoughts on these differences? Honestly, not really. Did you find conducting the first task more or less complex on one of the platforms? I don't think so … Would you say they were equal for you? Yes. Have you noticed any differences in the user interface when you were conducting the second task (create a tweet)? If yes, what were they? I don't think so … Not really. What are your thoughts on these differences? Unfortunately, I did not have any special feelings about it. Sorry. Did you find conducting the second task more or less complex on one of the platforms? As I said before, they were the same to me. Do you find one of the platforms less or more inconsistent? They are equal to me. Why do you think so? They looked similar. Do you find one of the platforms more or less friendly? No. Do you find one of the platforms more or less unattractive? Maybe just the home screen icon… I liked the round one more (PWA). Do you find one of the platforms more or less pleasing? No Are you having any special feelings about it? No, it doesn't matter to me. If you were to use Twitter in the future, would you rather use PWA or a native app? I would say the first one because I like the round logo, but I don't have any special feelings about it.

Participant 4 This user began the usability test with the native app. The participant did not have any issues with completing given tasks. He completed the first task by accessing settings in the hamburger menu. He was surprised how little time it took for him as he said: "I have never done that before.". He took similar steps to complete the first task on PWA as well. To fulfil the second task, he tapped on the round blue icon at the right bottom in both cases. He did not have any issues with adding images to the tweet in any case and took images on the spot instead of selecting them from the gallery. How often do you use Twitter? At least a couple of times a week. Have you noticed any differences in the user interface when you were conducting the first task (muting a keyword)? If yes, what were they? The first app (native app) was I think the keywords looked bigger and there were not as many options for certain stuff, whilst the second one was more minimalistic and more intelligent designed with a lot more things to customize. In the first app, I had to read a lot to find what I wanted to, while in the second one it was way easier and there was not as much stuff on the screen to block my view from what I wanted. What are your thoughts on these differences? I personally would use the second one because it just feels faster. Did you find conducting the first task more or less complex on one of the platforms? About the same, I would say, but the second app seems more visually easier.

25

Have you noticed any differences in the user interface when you were conducting the second task (create a tweet)? If yes, what were they? To be honest not that much, but there were few differences for sure. In the first one, I had to go to images had to find the camera, so it was more clicking to do, whilst with the second one I had the option to pick up the camera already and take a selfie quickly. What are your thoughts on these differences? I think I do not have anything to add after what I said. Did you find conducting the second task more or less complex on one of the platforms? It was more complex and took more time on the first one. Do you find one of the platforms less or more inconsistent? I think I would have to use it more to find out, but they both felt very similar and consistent. Do you find one of the platforms more or less friendly? I mean the first one would be more oriented to older people, I would say because the fonts were bigger. But the second one would be preferable for me. Do you find one of the platforms more or less unattractive? I find the second one more attractive than the first one. Do you find one of the platforms more or less pleasing? I was very pleased with doing both tasks on both platforms, so I don't think so. If you were to use Twitter in the future, would you rather use PWA or a native app? The second app (PWA).

Participant 5 This user began the usability test with the PWA. To complete the first task, the user went to settings in the hamburger menu and muted a keyword successfully in both cases. To fulfil the second task, he tapped on the round blue icon at the right bottom in both cases. He did not have any issues with adding an image to the tweet in any case and took a picture on the spot. How often do you use Twitter? I don't use it at all. Have you noticed any differences in the user interface when you were conducting the first task (muting a keyword)? If yes, what were they? The layout was a little different. I think there were more options in the first one (PWA). The method of doing the task was similar, but not exactly the same. What are your thoughts on these differences? I think that they are both very similar. However, the second option looks more friendly (Native app), because it looks better for the user than the first one. Did you find conducting the first task more or less complex on one of the platforms? No, I don't think so... Have you noticed any differences in the user interface when you were conducting the second task (create a tweet)? If yes, what were they? Yes, in the second one when you wanted to add a photo, you had a gallery and the camera together whilst in the first one you had to choose either to use the gallery or the camera only. What are your thoughts on these differences? I don't have any special thoughts about it. Did you find conducting the second task more or less complex on one of the platforms? No, on the second one it was more clicking but more visually appealing while on the first one it was faster but kind of less attractive. Do you find one of the platforms less or more inconsistent? No, I don't think so. They are both consistent. Do you find one of the platforms more or less friendly? The second one is definitely more friendly to me. Do you find one of the platforms more or less unattractive? The second one was more attractive to me. Do you find one of the platforms more or less pleasing? The first one was less pleasing to the eye. If you were to use Twitter in the future, would you rather use PWA or a native app? I would choose the second one because the interface looks nicer.

26

Participant 6 This user began the usability test with the native app. To complete the first task on the native app, the participants entered settings thru the hamburger menu and successfully muted a keyword. In the case of PWA, he took different steps and entered the notifications view and accessed the notification settings from there. When it comes to the second task, the user used the round blue icon at the right bottom in both cases. In the first case, the person chose to add an already existing image from the gallery, whilst on the PWA, he took a picture himself. How often do you use Twitter? Maybe once a week, sometimes less. Have you noticed any differences in the user interface when you were conducting the first task (muting a keyword)? If yes, what were they? Yes, so in the first app (native app) I had to get into the settings first and then to notification settings, and on the second app, I went to notifications view and then the settings of the notifications. What are your thoughts on these differences? Well, I think it was easier for me in the first app, because if I want to change something I look for settings and than I look for what I want to change and not another way around. Did you find conducting the first task more or less complex on one of the platforms? I think it was more complex on the second platform. Have you noticed any differences in the user interface when you were conducting the second task (create a tweet)? If yes, what were they? No, I don't think I noticed anything… Or I did. The difference was in the way I was able to put a picture in a tweet. In the first platform, I had the list of the pictures already. In the second one, I had to look for it. What are your thoughts on these differences? It seemed more friendly to have all the images already in place. Did you find conducting the second task more or less complex on one of the platforms? It was more complex for me in the second one, because I had to look for how to add an image. Do you find one of the platforms less or more inconsistent? I don't think I find any of them to be inconsistent. Do you find one of the platforms more or less friendly? I think I was more comfortable with the first one. Do you find one of the platforms more or less unattractive? No, for me the esthetics doesn't matter. Do you find one of the platforms more or less pleasing? Not really. They both seemed to be equal. If you were to use Twitter in the future, would you rather use PWA or a native app? The first one (native app). Can you tell me why? Yes, because it felt a little bit more natural, maybe because it was kind of similar to what I already know from other apps.

Participant 7 This participant began with the PWA. He conducted the first task (mute a keyword) by entering settings in the hamburger menu in both cases. For the first time, he struggled with this task since the concept of Twitter was new to him. Therefore, he took the time to get to know the app by browsing thru different views. The user created a tweet by tapping on the round blue icon at the right bottom of the screen and took pictures at a spot in both cases. He didn't have any major issues with completing the second task. How often do you use Twitter? I don't have Twitter, and I have never used me before. Have you noticed any differences in the user interface when you were conducting the first task (muting a keyword)? If yes, what were they? The first difference I noticed was the graphics. For me, it did not bring any difference in usage, just had different looks. I think on the first one (PWA) it was easier to noticed more advanced options like the time of muting and what to mute. It was easier to notice that. I think if you are just angry with some words and you don't want to read anything about it, the second app would

27

be better, but if you think some people write clever things about it and would like to filter those, the first one would be a little bit better. Is it because the setting was more advanced on the first app to you? I think it was just easier to notice (settings) on the first app because advanced options were clearer. But if you don't want to use more advanced options, there is no significant difference. Do you have any more thoughts on these differences? Not after just first time using. Did you find conducting the first task more or less complex on one of the platforms? I think maybe for the first time user it is difficult to understand the platform, but I think when you use it more often, you learn so I don't think any of the apps were harder to use. Maybe just the first app was easier to add more advanced options. Have you noticed any differences in the user interface when you were conducting the second task (create a tweet)? If yes, what were they? I think the first one was more clear when I wanted to post something, especially when adding a photo because in the second one you could not realize where to write the text or where you need to take a photo. What are your thoughts on these differences? Maybe for the first uses, the first app is more friendly, but after that, you get used to it. This interface is quite similar to other apps like Facebook like Instagram, and I could learn it in one day. And do you think the second app was not as easy to learn? I think that both of them are easy to learn, but the first one is a little bit more friendly. Did you find conducting the second task more or less complex on one of the platforms? No, they were equal to me, but maybe it is because I don't use Twitter and I don't see any big difference. Using Twitter is for me, something new. Maybe if I use it more frequently, the differences would be more significant. Do you find one of the platforms less or more inconsistent? Maybe the first was more consistent. Could you explain to me why? As I said, it was easier to find more advanced settings. Do you find one of the platforms more or less friendly? I think the first one is a little bit more friendly, but it is like a not significant difference. Do you find one of the platforms more or less unattractive? For me the second one is less attractive because probably I would be using this function to mute notification I wouldn't like to mute all notifications with that word, and I would like to read posts from the authors that I like, and I think are reliable and to mute other sources. Do you find one of the platforms more or less pleasing? When we talk about pleasing, I would say the second one was more pleasing. I don't know why, just that the graphics were more pretty. It had bigger letters, and it is important to me because I have a bad defect of vision and have to wear glasses. If you were to use Twitter in the future, would you rather use PWA or a native app? I would choose the first one because all advanced settings were more clear. Even though I was using the app for the first time, finding settings from whom you want the notifications to block was easier. I did not need to seek it.

Participant 8 This participant began with the native app first. She needed a lot of clues to understand the concept of the platform at the beginning since it was her first-time experience with it. To conduct the first task, the user entered notification settings from the hamburger menu on the native app. In the case of the PWA, she completed the task by accessing notifications view and tapping on setting button in the right top corner. However, both cases seemed to be challenging for the user. To complete the second task, the participant tapped on the round blue icon at the right bottom of the screen and took pictures at a spot in both cases. When she was adding a tweet on the native app, she was confused whether she did it correctly for the first time and repeated the process again. How often do you use Twitter? Oh my god, never. Have you noticed any differences in the user interface when you were conducting the first task (muting a keyword)? If yes, what were they?

28

The first app seemed to be more complicated than the second app. I also had to use different steps and click on different buttons. What are your thoughts on these differences? For me, the second one seemed easier, but I also knew what to look for. I think the first app is for more advanced users. Did you find conducting the first task more or less complex on one of the platforms? The first app was harder for me, and it took me more time. Have you noticed any differences in the user interface when you were conducting the second task (create a tweet)? If yes, what were they? Yes, the pictures were displayed differently. I don't know how to explain it like in the first one they showed all pictures. What are your thoughts on these differences? I don't know. To me, the second app seemed easier, less complicated. Did you find conducting the second task more or less complex on one of the platforms? Like I said before, the second one was easier for me to use. Do you find one of the platforms less or more inconsistent? To me, both of them were consistent. I think once you learn how to use Twitter, both can be consistent. Do you find one of the platforms more or less friendly? No, they were looking very similar. The differences were small. Do you find one of the platforms more or less unattractive? No, they were similar. Do you find one of the platforms more or less pleasing? If I learn how to use Twitter, I would say they are comparable. Maybe the second one, because I completed my tasks faster. If you were to use Twitter in the future, would you rather use PWA or a native app? It depends … Maybe the second one since it was easier for me.

Participant 9 This participant began the usability test with the PWA. To complete the first task, the user entered the notification view and tapped on the settings icon in the right top corner. She had some difficulties finding a save button after writing the keyword. In the case of a native app, she used the same path to complete the task. However, she did not have any trouble adding the keyword as she noticed the save button immediately. The user had no problem conducting the second task and did it thru tapping on the round blue icon at the right bottom of the screen in both cases. In both cases, she took a photo on the spot and added it to the tweet. How often do you use Twitter? I don't use it. I think I never even tried. Have you noticed any differences in the user interface when you were conducting the first task (muting a keyword)? If yes, what were they? The layout was different. The first app the items were organized just as they were only written, and the items were not organized like in tiles. It is hard to explain, but the first one was more readable to me. What are your thoughts on these differences? They are not drastic. I don't know. I think I liked the first app more. Did you find conducting the first task more or less complex on one of the platforms? I feel like the second app was harder, I had to enter deeper in the structure, and it was not so obvious to me. Have you noticed any differences in the user interface when you were conducting the second task (create a tweet)? If yes, what were they? I think there were some small differences but how it worked was, in general, the same. Could you name those differences? I think how you add images were very the same, but it was different where to add text to it. What are your thoughts on these differences? I think they both worked well, but somehow I would prefer the first app. Did you find conducting the second task more or less complex on one of the platforms?

29

I think the second app was harder when it came to muting the keyword, but creating a tweet was exactly the same. Do you find one of the platforms less or more inconsistent? No, I think they both worked well and were comparable to each other. Do you find one of the platforms more or less friendly? I would still say that the second app seems heavier to use. Do you find one of the platforms more or less unattractive? No, they seemed almost the same when it comes to esthetics. Do you find one of the platforms more or less pleasing? Not really. I think that for an average person, it does not matter if you have to click a button a bit higher or lower. They gave me similar satisfaction. If you were to use Twitter in the future, would you rather use PWA or a native app? Definitely, the first one. Could you sum up why? It seems to me that it is more friendly and looked a bit better.

Participant 10 This participant began the usability test with the native app. She showed some confusion at the beginning of the usability test as she pointed out the concept of the platform is new to her. To complete the first task, she entered the notification setting thru notification view in both cases. To complete the second task, the user tapped on the round blue icon at the right bottom of the screen in both cases. In the native app case, she added the image from the gallery and quickly completed the task. In the case of PWA, she had an issue with finding a way to add an image from the gallery. Therefore she took an image on the spot. How often do you use Twitter? I never use Twitter, and I do not have an account. Have you noticed any differences in the user interface when you were conducting the first task (muting a keyword)? If yes, what were they? I think I was too focused on completing the task to even notice if the apps were different. I didn't know twitter. It was new to me. I was looking for settings and how to do your task. But the second time when I was doing it I am not sure I noticed anything. So you did not notice anything different? Maybe in the first moment… I am not sure, and I think not. But it was easy to work on it. Did you find conducting the first task more or less complex on one of the platforms? Of course, when I was doing it for the second time, it was easier, because I already knew what to do, but I was not frustrated using any of them. Both apps were useful to me. Have you noticed any differences in the user interface when you were conducting the second task (create a tweet)? If yes, what were they? Maybe the second time I made a picture by myself, and that was a difference because I could not find the photo library. What are your thoughts on these differences? I think each app was okay for me, but now I am thinking maybe… No, I can not say I have any special thoughts about these differences, because it is nothing big. Did you find conducting the second task more or less complex on one of the platforms? No. I think for the second time it was easier because of my experience. Do you find one of the platforms less or more inconsistent? No, for me no. Do you find one of the platforms more or less friendly? Maybe the first one was more friendly. I don't know, the second app I wanted to choose a picture from the library but I could not so I decided to take it. Do you find one of the platforms more or less unattractive? I think the first one was better, more attractive to me. Could you tell me why? Maybe because of this library, I did not know how to do it on the second one. But I can't really explain it, it is just a feeling that next time I would prefer to use the first app. Do you find one of the platforms more or less pleasing? The first app, it was more pleasing for me. If you were to use Twitter in the future, would you rather use PWA or a native app?

30

I choose the first. Would you like to explain more why you prefer it? No, I have nothing more to add than I already said.

31

9.