Embed Size (px)
Citation preview
IN DEGREE PROJECT COMPUTER SCIENCE AND ENGINEERING,SECOND CYCLE, 30 CREDITS
, STOCKHOLM SWEDEN 2020
DOES WEB USABILITY AFFECT BRAND PERCEPTION? A COMPARATIVE STUDY ON THE “ILIAD ITALIA S.P.A.” CORPORATE WEBSITE
GABRIELE GULLÌ
KTH ROYAL INSTITUTE OF TECHNOLOGYSCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE
II
III
Abstract The concept of brand nowadays goes beyond the simple logo printed or displayed on a product or service. Branding is an important element for companies, and the communication channels used to deliver brand messages play a fundamental role. The advancement and innovation in web technologies made web communication artefacts crucial channels for companies and a key aspect of these artefacts is their usability. In this project a re-design process has been carried on the “Iliad Italia S.p.A.” corporate website from scratch, introducing innovative approaches such as user-centered design and value-based design. The developed website has been then used to perform an innovative comparative study aiming of verifying whether or not web usability affects brand perception. Adopting two different innovative procedures, brand perception has been measured on test subjects after using the old website and the new one. Analyzing the gathered data, a visible correlation has been found between usability and brand perception, indeed, using a website with higher usability resulted in a higher brand perception and the usage of a website with lower usability resulted in a decrease in brand perception in test subjects. The project introduced innovative research tools and procedures in a real case scenario, illustrating how they can be adopted in the development of web-communication artefact in companies, moving web development and marketing a step closer each other.
Keywords: web usability, corporate website, e-branding, brand perception, web
development.
IV
V
Sommario Il concetto di brand oggi va oltre il semplice logo stampato o esposto su un prodotto o servizio. Il branding è un elemento importante per le aziende, e i canali di comunicazione utilizzati per veicolare i messaggi del brand giocano un ruolo fondamentale. Il progresso e l'innovazione nelle tecnologie web hanno reso gli artefatti di comunicazione web cruciali per le aziende, e un aspetto chiave di questi artefatti è la loro usabilità. In questo progetto è stato portato avanti un processo di re-design del sito web aziendale di "Iliad Italia S.p.A." partendo da zero, introducendo approcci innovativi come lo user-centered design e il value-based design. Il sito web sviluppato è stato poi utilizzato per effettuare uno studio comparativo innovativo volto a verificare se l'usabilità di un sito web influisce o meno sulla percezione del brand negli utenti. Adottando due diverse procedure innovative, è stata misurata la percezione del brand successivamente all'uso del vecchio e del nuovo sito web. Analizzando i dati raccolti, è stata trovata una visibile correlazione tra usabilità e percezione del brand, infatti, l'utilizzo di un sito web con una maggiore usabilità ha portato a una maggiore percezione del brand e l'utilizzo di un sito web con una minore usabilità ha portato a una diminuzione della percezione del brand nei soggetti del test. Il progetto ha introdotto strumenti di ricerca e procedure innovative in uno scenario di uso reale, illustrando come questi strumenti e procedure possono essere adottate nello sviluppo di artefatti di comunicazione web nelle aziende, avvicinando lo sviluppo web al marketing.
Parole chiave: web usability, corporate website, e-branding, brand perception,
web development.
VI
VII
Sammanfattning
Begreppet märke går idag utöver den enkla logotypen som skrivs ut eller visas på en produkt eller tjänst. Varumärke är ett viktigt inslag för företag, och kommunikationskanalerna som används för att leverera varumärkesmeddelanden spelar en grundläggande roll. Utvecklingen och innovationen inom webbteknologier gjorde att webbkommunikationsföremål är viktiga kanaler för företag och en viktig aspekt av dessa föremål är deras användbarhet. I detta projekt har en ny designprocess genomförts på "Iliad Italia S.p.A." företagets webbplats från grunden och introducerar innovativa metoder som användarcentrerad design och värdebaserad design. Den utvecklade webbplatsen har sedan använts för att utföra en innovativ jämförande studie som syftar till att verifiera om webbanvändbarhet påverkar varumärkesuppfattningen eller inte. Antagande av två olika innovativa förfaranden, varumärkesuppfattning efter användningen av den gamla och nya webbplatsen har mäts på testämnen. Genom att analysera de samlade uppgifterna har en synlig korrelation hittats mellan användbarhet och varumärkesuppfattning. Att använda en webbplats med högre användbarhet resulterade faktiskt i en högre varumärkesuppfattning och användningen av en webbplats med lägre användbarhet resulterade i en minskning av varumärkesuppfattningen i testpersoner. Projektet introducerade innovativa forskningsverktyg och förfaranden i ett verkligt fallsscenario, som illustrerar hur de kan användas för att utveckla webbkommunikationsartiklar i företag, flytta webbutveckling och marknadsföring ett steg närmare.
Nyckelord: web usability, corporate website, e-branding, brand perception,
web development.
VIII
IX
Acknowledgments
I would like to thank the company “Iliad Italia S.p.A.” for the opportunity they gave me to work on this thesis project. Special thanks to Francesco Como and Roberto Barbadoro that supervised me during the entire project in the company. I would like to extend a special thanks to Prof. Garzotto, Prof. Gulliksen and Claudio Panariello that supported me regarding several aspects of the master thesis. Last, but not least, I want to thank my family, Alessandra and all the people who have been close to me in these years and that helped me to achieve this goal.
X
Contents
1. Introduction .................................................................................................. 1
1.1 Web usability .................................................................................................. 1
1.2 Corporate websites .......................................................................................... 2
1.3 Brand perception ............................................................................................. 2
1.4 e-Branding ....................................................................................................... 3
1.5 Iliad Italia S.p.A. ............................................................................................ 4
1.6 Goal ................................................................................................................. 4
1.7 Research question ............................................................................................ 5
1.8 Ethics and sustainability ................................................................................. 5
1.9 Outline ............................................................................................................ 6
2. State of the art ............................................................................................. 8
2.1 Usability evaluation methods .......................................................................... 8
2.2 Value-based design ......................................................................................... 12
2.3 Brand perception evaluation methods ............................................................ 13
2.4 Web usability and brand perception .............................................................. 15
3. Case study .................................................................................................. 19
3.1 “Iliad Italia S.p.A.” corporate website ............................................................ 19
3.2 Requirements elicitation ................................................................................. 22 3.2.1 User study .................................................................................................................. 22 3.2.2 Marketing requirements .............................................................................................. 25
3.3 Design process ................................................................................................ 26 3.3.1 Sitemap ...................................................................................................................... 26 3.3.2 Paper prototypes ........................................................................................................ 26 3.3.3 High fidelity prototypes .............................................................................................. 28
3.4 Implementation .............................................................................................. 39 3.4.1 System overview ......................................................................................................... 39 3.4.2 Database design and implementation ......................................................................... 39 3.4.3 Back-end implementation ........................................................................................... 40 3.4.4 Front-end implementation .......................................................................................... 41
XI
3.5 User testing .................................................................................................... 51
3.6 Inspection-based usability evaluation ............................................................. 52
4. Brand perception evaluation: Research method ......................................... 61
4.1 Research hypothesis ....................................................................................... 61
4.2 Participants .................................................................................................... 61
4.3 Procedure ....................................................................................................... 62
4.4 Data collection ............................................................................................... 64
5. Brand perception evaluation: Results ......................................................... 68
5.1 Main findings ................................................................................................. 68
5.2 Discussion ....................................................................................................... 71
6. Lessons learned ........................................................................................... 76
6.1 Theoretical framework critique ...................................................................... 76
6.2 Marketing perspective .................................................................................... 77
7. Conclusions and future works ..................................................................... 79
Bibliography ....................................................................................................... 81
Appendix A – Questionnaire answers ................................................................ 86
XII
List of figures
Figure 1: Brand Identity and Brand Image ......................................................... 3 Figure 2: Phases of the framework, source: adapted from [20] ........................... 14 Figure 3: Aaker’s Brand Identity Model, source: adapted from [21] p. 177. ...... 15 Figure 4: Results from previous works, source: adapted from [23] ..................... 17 Figure 5: "CHI SIAMO" page ............................................................................ 20 Figure 6: "COMUNICATI STAMPA" page ...................................................... 20 Figure 7: "DICONO DI NOI" page ................................................................... 21 Figure 8: "LAVORA CON NOI" page .............................................................. 21 Figure 9: Sitemap ............................................................................................... 26 Figure 10: Paper prototypes of different design alternatives .............................. 27 Figure 11: "CHI SIAMO" page high-fidelity prototype ..................................... 30 Figure 12: "MEDIA" page high-fidelity prototype ............................................. 33 Figure 13: "LAVORARE IN ILIAD" page high-fidelity prototype ................... 36 Figure 14: "ARCHIVIO" page high-fidelity prototype ...................................... 37 Figure 15: "COMUNICATO STAMPA" page high-fidelity prototype .............. 38 Figure 16: System overview ............................................................................... 39 Figure 17: ER Diagram ...................................................................................... 40 Figure 18: Backend overview ............................................................................. 41 Figure 19: "CHI SIAMO" page implemented website ........................................ 44 Figure 20: "MEDIA" page implemented website ............................................... 46 Figure 21: "LAVORARE IN ILIAD" page implemented website ...................... 48 Figure 22: "ARCHIVIO" page implemented website ......................................... 49 Figure 23: "COMUNICATO STAMPA" page implemented website ................ 50 Figure 24: Usability evaluation mean score per dimension ................................ 58 Figure 25: Heuristic usability scores .................................................................. 59 Figure 26: Correlation between usability and brand perception ........................ 66 Figure 27: Brand perception outcome "Testing H1" .......................................... 68 Figure 28: Brand perception outcome "Testing H2" .......................................... 69 Figure 29: Brand perception procedures combined ............................................ 70 Figure 30: Correlation "Testing H1" .................................................................. 72 Figure 31: Correlation "Testing H2" .................................................................. 73 Figure 32: Heuristic Usability & Brand Perception ........................................... 74
XIII
XIV
List of tables
Table 1: User needs ............................................................................................ 23 Table 2: User goals ............................................................................................ 24 Table 3: Requirements elicited from user study ................................................. 24 Table 4: Requirements elicited from marketing team ........................................ 25 Table 5: Old website usability evaluation scores ............................................... 55 Table 6: New website usability evaluation scores .............................................. 57 Table 7: Assigned scenarios for the experiment ................................................. 63 Table 8: Brand values ........................................................................................ 65
XV
Acronyms
UX User Experience
HCI Human-Computer Interaction
WOM Word-of-Mouth
UEM Usability evaluation method
GOMS Goals Operators Methods and Selection
IDM Interactive Dialog Model
CMS Content Management System
HR Human Resources
ERD Entity-Relationship Diagram
ORM Object Relational Mapping
DRF Django REST Framework
REST Representational State Transfer
CRUD Create Read Update and Delete
XVI
1
1. Introduction 1.1 Web usability
Modern websites and web applications are nowadays involved in most of our daily activities, they are becoming day by day more complex and richer of functionalities. An important aspect that should be taken into consideration while developing modern web application is their usability. Usability can affect how people interact with web applications. According to ISO 9241-11, usability is defined as “extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” [1]. Intuitively, considering the context of web communications, usability can be defined as the easiness of use perceived while using a website or a web application.
Web usability is an important topic in the field of Human-Computer Interaction (HCI) and it is an umbrella term used to group several aspects of a website, namely:
- Content - Information architecture - Navigation and interaction - Semiotics - Graphics and layout - Operations - Errors management
Usability can be evaluated on a website using different methods and heuristics. The goal of usability evaluation methods and procedures is to detect possible problems and inconsistences among the different webpages or interactions.
An important role, in usability evaluation, is played by user testing. User testing is a procedure in which real users interact with the web application completing tasks in order to discover possible issues and errors.
2
1.2 Corporate websites
The web is currently considered the main communication channel thanks to the possibility to be reached by a huge amount of people. According to Internet Live Stats there are more than 1.5 Billion websites on the web [2] and they can be classified in different categories. The category of websites that is considered in this project is corporate websites.
The term “corporate website” is used to refer to a website that aims of representing a company or institution on the Internet. Typically, a corporate website is used by the company to deliver brand messages, press releases, news and make people aware of open positions in the company.
There is no unique trend in how a corporate website is implemented. Based on the size of the company and the content that need to be delivered, companies tend to choose between creating a completely separate website or just include a few pages in a website already available to customers. A study performed by Tilley and Gustavsen [3] also pointed out that interactivity should be present to obtain a good level of public relations communication through these websites.
1.3 Brand perception
According to the American Marketing Association “A brand is a name, term, design, symbol or any other feature that identifies one seller’s good or service as distinct from those of other sellers” [4]. The brand is an important aspect that companies take into consideration since it is considered one of the main aspects that creates differentiation. Building a strong brand indeed, helps a company in being different from others and leads to long-term profitability. [5] The goal of branding activities performed is to create in the mind of consumers rational and emotional associations.
3
To understand what brand perception is, it is important to define the concepts of brand identity and brand image:
- Brand identity: set of elements regarding physical looking, personality, culture, relationships, reflection and self-image that represent what a company want its brand to be.
- Brand image or brand perception: set of beliefs, emotions, attitudes and mental association that a person has while thinking about a specific brand.
Figure 1: Brand Identity and Brand Image
It is important to notice that the concept brand is different from concepts such as usability or user experience in the way it relates to the user. Indeed, usability and user experience take place on the interaction level that is different from the relationship level considered by brands and branding activities. [6] [7]
1.4 e-Branding
The importance gained by the web in recent years has increased the awareness around e-branding for companies. With the term “e-Branding” we define a particular type of branding activities aimed to create and manage a brand image through the web. Differently from traditional marketing, the main communication channels used by this type of marketing are social media and websites [8]. The e-Branding has several characteristics:
- Constant presence: differently from traditional advertising that is delivered once by TVs and Radios, through the internet the content is always accessible by the users.
4
- Interactivity: brand messages are delivered to users that can immediately provide their feedbacks.
- Speed: the process of creating and delivering content is faster and can rely on word-of-mouth (WOM) mechanisms through social media sharing.
- Build trust: using social media users can express their feedback that affects the brand perception. It is important to create a community around the brand to build trust.
1.5 Iliad Italia S.p.A.
Iliad Italia S.p.A. is the telecommunication carrier that revolutionized the mobile network market in Italy. Their strategy focuses on truth and trust, their offers rapidly became famous due to their low price, no constraints and no hidden costs.
Iliad Italia S.p.A. came into the Italian mobile network market in 2018 with offers guaranteed forever. The company innovated also in the field of retail stores introducing into the market the so called “Simbox”, an automatic distributor of SIM cards that let users subscribe to an offer in just 3 minutes.
Iliad Italia S.p.A. headquarter is located in Milan (Italy), where this project will be carried on. Other offices are present in Rome and in other 10 cities all over Italy.
1.6 Goal
The goal of the project is to completely re-design the corporate websites of Iliad Italia S.p.A. and analyze how this re-design process affects the users brand perception.
To reach this goal several activities have been carried out:
5
- Complete re-design of the website following specific requirements and guidelines from the marketing team and specific requirements coming from needs and goals of different categories of users
- Front-end and back-end website implementation - Database design and implementation - Usability study and user testing - Analysis and data gathering on brand perception with test subjects
1.7 Research question
Usability is considered the main aspect affecting the user experience of a web communication artefact and intuitively we associate that experience to the entity behind that artefact. A common thought, for instance, can be that the use of a website that has a low usability can reduce trust in the company behind that artefact. Even if this correlation seems intuitive for us, there is a lack of quantitative data supporting these hypotheses. The process of re-design carried on in this project introduces the possibility to test these hypotheses on a real case example, so the research question that is investigated is:
- Does usability of a web-based communication artefact affect brand perception, i.e., the set of beliefs, emotions, attitudes, or qualities that people mentally associate to the entity behind that artefact?
1.8 Ethics and sustainability
This project carries several ethics and sustainability issues. Indeed, the process of redesign of a corporate website can increase awareness in users on how a communication artefact such as a website can alter their perception of a brand. The focus on usability in this project is important to help users in achieving their
6
goals on the website: the more the website is usable the less we expect is the time spent by a user to complete a task. This results in an increase in the overall productivity.
Another important issue to be taken into consideration is that with this re-design process and the improvements made in the communication section, users are now able to retrieve more official information avoiding potential problems related to fake news. Indeed, according to a study performed by Hunt Allcott, Matthew Gentzkow and Chuan Yu [9], from 2015 to 2017 the engagements and shares of fake news increased significantly on Facebook and Twitter. Even if on Facebook, due to new policies, these engagements have been drastically reduced from 2018, on Twitter the shares of fake news are still raising demonstrating how this problem is still present in our society. The introduction of more official communication channel can have a strong positive impact on this ethical problem, reducing engagements and shares of fake or unofficial information.
1.9 Outline
The rest of the thesis is organized as follows:
- Chapter 2 presents the actual state of the art of the usability evaluation methods, value-based design and brand perception evaluation methods and it describes the theoretical frameworks used in the project.
- Chapter 3 describes the user study, the design process, the implementation process, the user testing and the usability study.
- Chapter 4 describes the research method used in the project to analyze the brand perception. The focus of this chapter is on the participants involved in the experiment, on the description of procedure and on data collection.
- Chapter 5 illustrates the main findings of the analysis and provide a discussion built upon these data.
- Chapter 6 presents the lessons learned during the project, a critique on the theoretical framework both from a personal and a marketing perspective.
- Chapter 7 presents the conclusions and future works that can be carried on.
7
8
2. State of the art 2.1 Usability evaluation methods
As defined by Fernandez, Insfran and Abrahão in an analysis made on the available usability evaluation methods, “a usability evaluation method is a procedure which is composed of a set of well-defined activities for collecting usage data related to end-user interaction with a software product and/or how the specific properties of this software product contribute to achieving a certain degree of usability” [10]. According to their studies, usability evaluation methods can be divided in five main categories:
1. Testing involving users 2. Inspection methods 3. Inquiry methods 4. Analytical modeling 5. Simulation methods
Considering testing involving users, this category is composed of the UEMs involving the interaction between the artefact and the user as the main source of feedback. The UEMs in this category are Think-Aloud Protocol, Question-Asking Protocol, Log Analysis and Remote Testing.
In the category of inspection methods, we can insert all those methods involving inspections made by usability experts such as Heuristic Evaluation, Cognitive Walkthrough, Perspective-based inspection and Guideline review.
The category of inquiry methods refers to all those methods that use subjective data gathered from users as main source of feedback. Under this category we have: Questionnaires, Interviews, and Focus Groups.
Analytical modeling is a category of UEMs in which certain elements of the websites are modeled in order to analyze specific issues. Examples are Cognitive Task Analysis, Task environment analysis and GOMS analysis.
UEMs play an important role in this project since they are responsible in attributing a degree of usability to the website. The level of usability is needed
9
as parameter to condense and consider the differences between the old and the redesigned website. This parameter is the key element that can be used in the relation between brand perception and usability.
An advanced usability evaluation method that will be used in this project is the MILE+ method [11] designed by F. Garzotto and D. Bolchini. The choice of this evaluation method has been made based on the fact that it is considered, as described by the authors, the best choice for novice usability evaluators with poor or limited experience. This evaluation method can be considered the combination of different evaluation methods, indeed MILE+ is based on the idea that the usability of a web application can be evaluated on two different perspectives:
- Requirements-independent perspective: the elements under analysis are independent from the content, domain and goals of the users, such as information architecture, layout, graphics, navigation and interaction mechanisms.
- Requirements-dependent perspective: the analysis focuses on understanding if the needs of defined users in specific context are satisfied.
The requirements-independent perspective is evaluated with a technique named technical inspection. In this procedure an evaluator examines the web application checking from a design perspective a set of heuristics that are independent from the context of use. The original method is composed by 82 heuristics divided in different categories, the terms used in this set of heuristics are based on the Interactive Dialog Model (IDM) [12] [13], a well-known approach in hypermedia applications design.
For the scope of this project and considering the limited experience of the evaluator, not all the 82 heuristics are applicable. A subset of heuristics has been selected based on the specific case under study:
Content
- Text errors: the written text should not present grammatical errors. - Multimedia consistency (images, audio, videos...): all the multimedia files
must be consistent with the subject of the page.
10
Navigation
- Landmarks: the access to the main sections of a web site is given by a number of landmarks. Using the landmarks the user can access easily and quickly all the macro-sections of the application. Therefore, the landmarks should be well highlighted in every page.
- Consistency: all the web applications have a general navigation architecture that supports the navigation of the user. This navigation has to be consistent among the different parts of the application. In this sense, it is very important that this “general” architecture emerges in a satisfactory way: the user has to comprehend how the general navigation works.
- Accessibility: accessibility refers to ensuring that content is accessible, ie. ensuring that content can be navigated and read by everyone, regardless of location, experience, or the type of computer technology used.
- Orientation: different websites are designed with a tree structure. In this site, the orientation of the user become fundamental both when the user explores a branch (section) of the tree and when he passes from a branch (section) to another. The user should be aware when a change of context happens.
- Backward navigation: when the user navigates within a tree (in particular when he passes from a section to another) one of the most difficult things to manage is related to the navigation to the previous visited pages. The application should support this action without the use of back functionality offered by the browser.
Interface Design
- Information overloading: a single page is composed by a set of different messages, each having a precise meaning. The quantity of the messages and their degree of heterogeneity could request an excessive effort for a first time/web novice to understand the whole page.
- Scannability: users do not “read” the page until they find what they are interested in (a link, a text, an image). First of all, they “scan” it, basing
11
on the structure of the page and how different messages are grouped and organized (in terms of macro areas).
- Visual identity: lack of coordination with the visual identity of the company who run the site (if present).
- Use of a chromatic code: the correct use of colors in a website is very important for many reasons and helps the users in the navigation. Colors can identify sections or subsections of the site; colors can reinforce the visual identity of the site; colors can attract the attention of the users on different elements of the pages (titles, links...); The set of the colors of the site creates the look and feel of the site.
- Background contrast: the use of strong colors for the background or not suitable pictures can damage the readability of the contents of the website. Some matches of colors can be very difficult to read especially for people with visual disabilities.
- Font size: all fonts work at large sizes, problems start at smaller sizes. Text on the screen must be easy to read. Choosing the right font size is important to make it readable.
- Font color: the colors used for screen texts must be accurately designed.
The requirements-dependent perspective in the MILE+ method is evaluated using an approach named user experience inspection. The user experience inspection is a procedure based on user testing within a scenario [14]. The focus of this approach is on defining the category of users, the context of use, the main goal and a set of tasks that enable the user to achieve the main goal. Once the scenario has been completed, the approach includes the interview of the user in order to gather subjective data on different aspects such as content, navigation and operational flow. With this procedure it is possible to discover possible problems and also gather insights from the users.
The user experience inspection of the MILE+ will not be used. User testing will be of course carried out within scenarios to discover problems and gather qualitative data, but without following specific and strict guidelines from the MILE+ method.
12
2.2 Value-based design
The redesign process of a corporate website moves the focus of the background literature also on newly design methods based on brand values. During the entire project, it has been important to have clear the brand messages and the values the company wants to deliver, indeed, brand values introduce a new category of requirements that has to be taken into consideration while designing the website.
The goal value-based design is trying to achieve in the field of web design, is the use of a web communication artefact as an additional manner to differentiate among brands. To do so the artefact must consider additional requirements, indeed, taking into consideration broader requirements related to the brand can increase the brand awareness in the users of the website. The web offers the possibility to clarify the brand image, looking to modern web application, the concept of value-based design is strongly used by a lot of companies.
According to studies performed by Paolini, Garzotto and Bolchini [15], to properly elicit requirements coming from the brand, it is important to understand what are the communication goals of the company. As communication goals they define the messages that the company wants to deliver and each of them can be translated into one or more requirements for the website. Requirements engineering typically define two different types of requirements, functional and non-functional. Requirements coming from the brand can be classified accordingly into a particular type of the two different categories. The case study presented in the mentioned article shows how these requirements at the end results in requirements affecting the content, graphic design, information architecture and interactions.
To help designers and web developers in the requirement elicitation process, a modeling framework named AWARE+ has been developed by Garzotto et al. in [15] and [16]. This framework, in addition to goals and needs of the target users, takes into consideration also brand values and communication goals in the process of elicitation of requirements, resulting in a more structured and more comprehensive approach.
13
2.3 Brand perception evaluation methods
How the brand is perceived by customers is an important aspect companies care about. To evaluate brand perception several methods have been proposed in the past. Dubberly proposed a model [17] aiming to unpack all the characteristics of brand in order to evaluate each of them. Gregory [18] identified three main dimensions in which brand efficacy can be measured:
- Position: how the brand is positioned within the different categories of brands
- Reach: the amount of people that recognize the brand in specific geographic areas
- Reputation: the feelings and emotions associated to the brand by people
Another interesting analysis [19] has been performed by Young and Rubicam, they created a brand asset evaluator aiming at evaluating the brand value on four different dimensions:
- Differentiation: being different from other brands. This dimension is based on the concept of uniqueness.
- Relevance: importance perceived in a specific market segment. - Esteem: actual perception of the consumers about the brand and its
potential popularity. - Knowledge: awareness of the brand identity and characteristics in
consumers.
According to the study presented in [20], researchers identified that all the previously described methods present the following problems:
- Non-repeatable method: different methods assess the brand perception of the users but do not provide a repeatable procedure.
- No clear parameters: most of the methods do not clarify what should be checked to understand the brand perception of a user.
- No user differentiation: the available methods do not differentiate based on the category of user.
14
To get rid of these problems, a new brand perception evaluation method has been accurately created by Bolchini, Tang and Garzotto [20]. This method is trying to resolve the mentioned problems introducing a clear procedure, defining the parameters and considering different categories of users.
Figure 2: Phases of the framework, source: adapted from [20]
As illustrated in Figure 2, the method is composed by two different phases: the preparatory phase and the execution phase. In the preparatory phase the researcher identifies the target users and the brand values the company wants to deliver. In the execution phase a pre-test questionnaire is submitted to the users to assess their brand perception before the use of the website, then a scenario is proposed to the user that has to complete some tasks on the website, to complete the procedure the post-test questionnaire is submitted to the user to discover possible variations in the brand perception after the interaction with branded artefact. This method seems to be the most complete and easy to use to assess variations of the brand perception in users.
15
2.4 Web usability and brand perception
As discussed before, the definition of brand nowadays goes beyond the logo. Brand is a much wider concept containing personality aspects related to the company behind the product or service. To properly define the concept of brand, the most used framework is the “Aaker’s Model” [21].
Figure 3: Aaker’s Brand Identity Model, source: adapted from [21] p. 177.
According to this model, the process of brand management can be defined as three different phases: Strategic Brand Analysis, Brand Identity System and Brand Identity Implementation System. In the Strategic Brand Analysis customers, competitors and the company itself are analyzed and the inputs for the second phase of the process are created. In the Brand Identity System, the main aspects of the brand are defined such as the value propositions, the brand identity, customer relationships and credibility. In the last phase, the Brand Identity Implementation System, the communication goals are defined and implemented to achieve the desired brand position.
16
In the last phase the web communication artefacts play an important role. The communication goals need a channel to be achieved, and this channel can be implemented using websites or web applications.
To analyze the importance of these channels, a study from Müller and Chandon [22] illustrated how actually the exposure of users to a website affects their perception of the brand behind that website. Interesting results highlighted how the exposure alters users perception in a way that perception results more sincere and reliable.
Previous studies in the field of usability have been conducted by professor Bolchini, Garzotto, Sorce [23] to prove how brand perception can be affected by web usability and the article has been considered the main source of inspiration for the entire degree project.
The researchers studied whether or not there is a correlation between the usability of a website and brand perception. A repeatable experiment was designed, the brand perception was examined before and after the use of a website starting from the hypothesis that a website with a higher usability will result in a higher brand perception, and that a website with lower usability will result in a lower brand perception. To implement the experiment, the researchers altered different websites improving or decreasing the level of usability and measuring the corresponding level of brand perception on different subjects. The level of usability has been measured using the technical inspection approach of the MILE+ method described in the section 2.1, while the brand perception has been measured using the framework developed by the same researchers and described in the section 2.3. They proceeded as follow:
- Pre-usage brand perception assessment - Use of the website in a scenario - Post-usage brand perception assessment
In that case, the structure of the question used by the framework to assess brand perception was fixed and just some keywords were changed. The question asked for each brand value was: “How much does Brand X inspire you Value Y?” [23]. Participants in the study could rate the value on a scale going from 0 ("not perceived") to 5 ("strongly perceived").
17
This procedure was used for each version of the website, modified or not, and the results proved researchers' initial hypothesis.
Figure 4: Results from previous works, source: adapted from [23]
Their results show how in the case of the Ryanair website and Verbania website, the increase in terms of usability corresponds to an increase in terms of brand perception. Instead in the case of USI website and Disney website a decrease of the usability results in a decrease in brand perception.
Although this result can be considered predictable, the analysis' outcome is valuable since it is the first empirical evidence of the correlation between usability and brand perception. A strong knowledge contribution by the researchers is also the repeatable research methodology ideated: this procedure results important to check the effectiveness of the redesign process of different typologies of websites.
18
19
3. Case study 3.1 “Iliad Italia S.p.A.” corporate website
The entire project has been built upon the “Iliad Italia S.p.A.” corporate website. The website has been developed by the company in the past using a Content Management System (CMS) in this specific case WordPress [24], so a redesign from scratch was needed. The advantages of building a website from scratch, instead of using a CMS, are mainly based on the freedom you have in creating a website that looks exactly as you want without any limitation related to the CMS you are using [25].
The website under analysis was reachable by the users through the links available in the footer of the consumer website of the company “www.iliad.it”. Since the main audience of the website is Italian speaking, the marketing team decided to build the original website only in Italian. This design choice has been also taken for the new corporate website.
The old website has been used by the company for the classical functions of a corporate website: brand messages, press releases, news and open positions in the company. It is composed by four main sections:
- “CHI SIAMO” (Who we are): in this section the company was delivering brand messages and exposing the manifest of company (Figure 5).
- “COMUNICATI STAMPA” (Press releases): in this section the company published all the press releases divided in different categories, allow users to search for items, subscribe to the newsletter and have a look at some pictures of the offices (Figure 6).
- “DICONO DI NOI” (What they say about us): in this section there was a collection of news from external websites in which the authors were discussing about “Iliad Italia S.p.A.” (Figure 7).
- “LAVORA CON NOI” (Work with us): in this section the company was proposing a link redirection to an external platform in which the Human Resources (HR) department was managing and publishing all the open positions in the company (Figure 8).
20
Figure 5: "CHI SIAMO" page
Figure 6: "COMUNICATI STAMPA" page
21
Figure 7: "DICONO DI NOI" page
Figure 8: "LAVORA CON NOI" page
22
3.2 Requirements elicitation 3.2.1 User study
The requirements elicitation process started with an analysis of the potential users of the website. To understand who the users of website are, a study process has been carried out over 9 corporate websites provided as inspiration by the marketing team:
- TIM (https://www.telecomitalia.com/tit/it.html) - VODAFONE (http://www.vodafone.it/portal/Vodafone-italia) - WIND TRE (https://www.windtre.it/it/Home.aspx) - FASTWEB (https://www.fastweb.it/corporate/) - ENEL (https://corporate.enel.it/it/media) - OPEN FIBER (https://openfiber.it/corporate/) - IMILLE AGENCY (https://www.imille.agency/en/work/case-
studies/enel-people) - FACEBOOK (https://about.fb.com/company-info/) - APPLE (https://www.apple.com/it/leadership/ ,
https://www.apple.com/it/newsroom/)
A study carried on over these corporate websites allowed the identification of the following categories of users:
- Journalists: person who writes, collects and shares current information to the public.
- Job applicants: person who is searching for a job. - Potential partners: person representing a company who wants to create a
partnership with the company owner of the website. - Students and researchers: person who carries out academic or scientific
research. - Potential new customers and already customers: person who is already a
customer or who is not a customer, but he can be in the future.
23
Following a user centered design approach [26], needs, goals and corresponding requirements have been identified for each category of users. The results of the analysis are presented in Table 1 and 2:
Category of users Need
Journalists N1 - Improved channel to find information about the company
N2 - Improved procedure to contact the company press office
Job applicants N3 - Improved solution to find open positions in the company
Potential partners N1 - Improved channel to find information about the company
Students and Researchers
N4 - Improved content and information related to the company
Potential new customers and already customers
N5 - Lack of channel where to find events and initiatives from the company
Table 1: User needs
Category of users Goal
Journalists G1 - Find information and press releases from the company
G2 - Contact the company for specific needs
Job applicants G3 - Discover the open positions in the company
Potential partners G1 - Find information and press releases from the company
24
Students and Researchers
G4 - Gather sufficient information about the company
Potential new customers and already customers
G5 – Find events and initiatives from the company
Table 2: User goals
After analyzing the needs and goals of different categories of users and considering specific suggestion from the marketing team, the requirements in Table 3 have been elicited:
Requirement Needs Goals
R1 - The website should display information related to the company and press releases
N1, N4 G1, G4
R2 - The website should provide a channel to contact the press office of the company
N2 G2
R3 - The website should allow users to discover open positions in the company
N3 G3
R4 - The website should allow users to discover all the events and initiatives from the company
N5 G5
Table 3: Requirements elicited from user study
25
3.2.2 Marketing requirements
After several brainstorming session with the marketing team, a value-based design approach [15] has been adopted and all their different suggestions and requests have been analyzed and translated into corresponding requirements (Table 4):
Requirement
MR1 – The website should have a strong visual impact highlighting the importance of specific words
MR2 – The website should allow a fluid navigation within the different sections
MR3 – The website should include just three main section in the navigation bar
MR4 – The website should allow the user to have a seamless experience in switching between the consumer and the corporate website
MR5 – The website should use the “CHI SIAMO” page as homepage
MR6 – The website should present in the “CHI SIAMO” page the brand values
MR7 – The website should include animation to create a more fluid experience
MR8 – The website should alternate in the different pages white sections with sections containing pictures
MR9 – The website should have a different tint for each section
Table 4: Requirements elicited from marketing team
26
3.3 Design process
In the design process of the corporate website, several artefacts have been created to illustrate and discuss functionalities, navigation and interaction in the several brainstorming sessions. The following artefacts will illustrate the latest version of the website, highlighting and modeling specific characteristics.
3.3.1 Sitemap
After the analysis of the websites given as inspiration by the marketing team and after several iterations the following site map illustrates the structure of the new website. In the version below all the section names have been translated from Italian to English.
Figure 9: Sitemap
3.3.2 Paper prototypes
Different paper prototypes have been developed as low fidelity prototypes. The goal of these artefacts is to easily sketch possible solutions in order to evaluate rapidly alternatives [27]. During the design process several alternatives have been developed and the following figures illustrate some of them.
27
Figure 10: Paper prototypes of different design alternatives
28
3.3.3 High fidelity prototypes
After several iterations and taking into consideration all the requirements elicited in the user study and from the marketing team, a high-fidelity prototype has been developed responsible of showing the aesthetic, navigation, interaction and architecture of the final solution.
“CHI SIAMO” (Who we are)
29
30
Figure 11: "CHI SIAMO" page high-fidelity prototype
The page “CHI SIAMO” (Who we are) is the homepage of the website and it is composed of several sections that can be accessed by scrolling the page. At the beginning, the manifest of the company is presented then the mission, the brand values and additional description of the company profile. The page finishes with a section related to where the different offices are located and a footer containing useful links. (Figure 11)
31
“MEDIA” (Media)
32
33
Figure 12: "MEDIA" page high-fidelity prototype In the “MEDIA” (Media) page a section dedicated to press releases is positioned at the beginning of the page. A picture used as a divider separates the press releases from the section related to events and initiatives. Another divider aims to divide between events and the button that can be used to contact the press office of the company by the users. (Figure 12)
“LAVORARE IN ILIAD” (Working in Iliad)
34
35
36
Figure 13: "LAVORARE IN ILIAD" page high-fidelity prototype
The page “LAVORARE IN ILIAD” (Working in Iliad) is composed by different sections aiming to attract potential new job applicants presenting the company. The latest open positions are displayed with a button redirecting to the external platform used by the HR team. The page finishes with a section dedicated to interviews to the employees of the company. (Figure 13)
37
“ARCHIVIO” (Archive)
Figure 14: "ARCHIVIO" page high-fidelity prototype
The “ARCHIVIO” page is the page responsible for the search of items in the websites, here it is possible to retrieve past press releases and past events and initiatives. It is composed by an additional navigation bar suggesting to the user which category of content is displayed and the list of the retrieved content. (Figure 14)
38
“COMUNICATO STAMPA” (Press release)
Figure 15: "COMUNICATO STAMPA" page high-fidelity prototype This page is intended as a template for the different press releases and also the page dedicated to events and initiatives. Each specific press release or event or initiative will have a specific structure but following this template. (Figure 15)
39
3.4 Implementation 3.4.1 System overview
Figure 16: System overview
The website has been developed following a classical three-tier architecture [28]. Each of the different layers has been developed using some of the most advanced technologies and frameworks available in the field of web development, as illustrated in Figure 16.
3.4.2 Database design and implementation
Considering the type of website developed, and the amount of data that need to be managed, a relational database structure has been chosen. The database design process has been carried out creating an Entity-Relationship Diagram (ERD):
40
Figure 17: ER Diagram
The designed schema has been implemented using MySQL [29] a relational database developed by Oracle. The implementation process has been carried out using the Object Relational Mapping (ORM) functionalities provided by Django. These functionalities improve the efficiency of the SQL queries, making the database management more object-oriented and portable [30].
3.4.3 Back-end implementation
The back-end part has been developed using Django [31], one of the most advanced technologies available on the market, used by a lot of companies such as Instagram, Spotify, Dropbox and YouTube [32]. Django is a web framework using Python to develop fast and reliable web applications.
In this project, Django has been enhanced using Django REST Framework (DRF) [33] to add Representational State Transfer (REST) API calls management. Using
41
DRF it has been possible to manage all the HTTP requests sent by the client to the server.
Figure 18: Backend overview
The main resources managed by the backend of the developed website are press releases, events and interviews. For each of them, Create, Read, Update and Delete (CRUD) operations have been mapped on REST methods POST, GET, PUT and DELETE and adequately managed by the server to perform the corresponding operations on the underlaying database.
3.4.4 Front-end implementation
The front-end part of the project has been implemented using Vue.js a progressive JavaScript framework [34]. Nuxt.js has been used to enhance the performances of Vue.js adding features such as modular structure, server-side rendering and static site generation [35]. The layout and graphics of the website have been developed mainly using the grid system provided by Vuetify [36] adding some custom CSS.
42
The communication between front-end and back-end has been implemented using HTTP requests over an OpenAPI 3.0 [37].
The final implemented solution, that has been used for the experiment, is now presented:
“CHI SIAMO” (Who we are)
43
44
Figure 19: "CHI SIAMO" page implemented website
45
“MEDIA” (Media)
46
Figure 20: "MEDIA" page implemented website
47
“LAVORARE IN ILIAD” (Working in Iliad)
48
Figure 21: "LAVORARE IN ILIAD" page implemented website
49
“ARCHIVIO” (Archive)
Figure 22: "ARCHIVIO" page implemented website
50
“COMUNICATO STAMPA” (Press release)
Figure 23: "COMUNICATO STAMPA" page implemented website
51
3.5 User testing
To gather qualitative data on usability and discover possible problems on the website, user testing has been carried out. To gain as much feedback as possible, according to the study [38] performed by Jakob Nielsen, 5 users can be considered enough to capture most of the possible issues. In this specific phase of the project, the goal was to reach the highest level of usability possible, for this reason the selected sample for user testing was composed by people not coming just from the categories of users the website has been designed for.
Due to 2020 Coronavirus pandemic in Italy [39], user testing has been carried out remotely. Since the evaluator and the user performing the test were in two different locations, the user accessed the website using AnyDesk [40] a remote desktop software that easily gave control of the evaluator laptop. For the communication between the evaluator and the user, Cisco Webex Teams [41] has been used. The user testing procedure was composed by three different scenarios:
SCENARIO 1:
Goal: Read the latest press release of the company.
- Task 1: Find the “media” page - Task 2: Find the press releases section in the media page - Task 3: Select the latest press release
SCENARIO 2:
Goal: Discover the latest event or initiative organized by the company
- Task1: Find the “media” page - Task2: Find the section related to events and initiatives - Task3: Select the latest event or initiative
52
SCENARIO 3:
Goal: Read what employees say about the company
- Task 1: Find the page “lavorare in iliad” - Task 2: Find the section related to the employees’ interviews - Task 3: Check the available interviews
Each user was introduced to the test explaining orally the purpose of the website and the possibility to quit was given. The “thinking aloud” procedure [42] was adopted and explained to the users. The goal of this procedure was to gather their intentions, reactions and problem-solving strategies. During the different testing phases the evaluator was responsible for taking notes and gather feedbacks. Once the test finished, some informal questions were asked to the participant to gather more general information and thoughts about the website.
The main feedbacks gained from the users are:
- Clear and readable - Easy to navigate - Few sections made the navigation easier - The cards, in the “Who we are” page, are really cool - The division between press releases and events in the archive is really
useful to understand the available contents - The messages delivered by the websites are clear - Animations are beautiful - The website is clearly following the brand
3.6 Inspection-based usability evaluation
An inspection-based usability evaluation has been carried out. In this phase the old and the new website have been evaluated in terms of usability to obtain a value summarizing the advancement resulting from the re-design process.
53
To evaluate the two versions of the website the evaluator adopted the subset of metrics previously described in the section 2.1 coming from the technical inspection of the MILE+ method.
For each heuristic the corresponding action provided by the MILE+ method has been carried out and a numeric score from 0 to 5 with a step size of 1 was given. The score aims to verify whether the usability problem object of the heuristic has been solved or not:
- 0: the usability problem object of the heuristic has not been solved at all - 5: the usability problem object of the heuristic has been completely solved
After the inspection of the old and the new website the results of the evaluation are shown in Table 5 (old website evaluation) and in Table 6 (new website evaluation).
Heuristic Score Dimension Notes
Text errors 5 Content There are no visible text errors.
Multimedia consistency
3 Content The different elements have a fixed cover, not taking into consideration appropriately the content. In total, just three images were used as covers.
Landmarks 5 Navigation Landmarks to navigate among different sections always accessible.
Consistency 5 Navigation The navigation architecture is clear, based on the closable menu available in the navbar.
Accessibility 3 Navigation Navigation items often are cut off of the screen. Some elements on mobile are too small and not easily clickable.
54
Orientation 2 Navigation Since the menu after click will close, the user cannot properly understand just looking at the page in which section he is in that moment. To understand, the user should always open the navigation menu.
Backward navigation
0 Navigation The page related to a press releases or news that seems to be positioned deeper in the navigation architecture, do not provide this functionality.
Information overloading
2 Interface Design
Pages such as ‘comunicati stampa’ contain a lot of functionalities that do not seems strictly related and this create confusion.
Scannability 4 Interface Design
The different sections in the pages are delimited and easy to find even if on mobile result in longer pages.
Visual identity 1 Interface Design
The logo is always visible, the colors are respected but the entire website is not matching all the advertising and graphics present on other channels of the company.
Use of a chromatic code
5 Interface Design
The colors for title, subtitles and content are used adequately in a consistent way.
55
Background contrast
4 Interface Design
Overall the background makes the text readable. A problem is visible in the section ‘lavora con noi’ where with the selected background, moving the cursor on the available link, the text is not properly readable.
Font size 3 Interface Design
There are some pages where the font size tends to be too small.
Font color 5 Interface Design
The color chosen for the text makes it readable.
Table 5: Old website usability evaluation scores
Heuristic Score Dimension Notes
Text errors 5 Content There are no visible text errors.
Multimedia consistency
5 Content Each element has now a different cover, properly designed based on the content.
Landmarks 5 Navigation Landmarks to navigate among different sections always accessible.
Consistency 5 Navigation The navigation architecture is clear, based on the different sections always present in the navbar.
56
Accessibility 5 Navigation Elements always accessible even on mobile.
Orientation 4 Navigation On lager devices the user can understand in which section of the website he is just looking at the navbar that is providing a visual feedback based on which page is selected. On mobile devices due to their smaller screens, it is necessary to open the navigation menu to access this feedback.
Backward navigation
5 Navigation The pages related to single items such as press releases and event provide this functionality.
Information overloading
4 Navigation The messages the different pages are trying to deliver are limited. Sometimes the text content of some items such as press releases, in the dedicated pages, can result long. But considering the purpose of the website and the importance of this items, this does not impact much the overall experience.
57
Scannability 5 Interface Design The different sections in the pages are delimited and easy to find.
Visual identity 5 Interface Design Logo and colors are respected. The entire website follows the marketing campaigns of the company performed on other channels.
Use of a chromatic code
5 Interface Design The colors for title, subtitles and content are used adequately in a consistent way.
Background contrast
5 Interface Design The text is always readable.
Font size 5 Interface Design The font size used seems always appropriate.
Font color 5 Interface Design The color chosen for the text makes it readable all the pages.
Table 6: New website usability evaluation scores
58
Figure 24: Usability evaluation mean score per dimension
Figure 24 compares the scores (mean values) of the main usability dimensions for the old website and the new one. The major improvements are in the navigation and interface design.
A unique score to compare the old and the new website can be defined as ‘heuristic usability’ that corresponds to the average among all scores obtained in the different heuristics on a specific website, and can be computed as follow:
!"# =%ℎ'#(
)
'*+
where:
- w is the website on which the heuristics has been computed. It can be the old website or the new one.
- n is the cardinality of the set of heuristics - ℎ'# is the corresponding score obtained in the heuristic i on the website w
0 1 2 3 4 5
Content
Navigation
Interface Design
Mean score per dimension
Old website New website
59
Figure 25: Heuristic usability scores
Figure 25 shows the heuristic usability value for both the old and the new website: it is possible to notice an overall improvement from the old website to the new one. The two HU scores can now be used to investigate the correlation between usability and brand perception.
3,357142857
4,857142857
0 1 2 3 4 5
HU
Heuristic Usability
New website Old website
60
61
4. Brand perception evaluation: Research method
4.1 Research hypothesis
The goal of the project is to analyze the impact of the re-design process on the brand perception of the users, investing the correlation between web usability and brand perception.
Considering the ‘heuristic usability’ (HU) as the score obtained in the inspection-based usability evaluation on the two different version of the website, defined in section 3.6, the study is based on the following hypotheses:
- H1: Increase in HU impacts positively the brand perception change - H2: Decrease in HU impacts negatively the brand perception change
The hypothesis H1 states that since the HU of the new website is higher compared to the HU of the old website, this will result in positive change in brand perception. Vice versa the hypothesis H2 states that since the HU of the old websites is lower than the HU of the new websites this will result in a negative change in brand perception.
4.2 Participants
The study on brand perception has been performed on the specific categories of users the website is addressing, defined in section 3.2:
- Journalists - Job applicants - Potential partners - Students and researchers - Potential new customers and already customers
62
The experiment was conducted on a sample of 12 people aged between 22 and 53. To obtain most accurate results possible, half of the sample were men and the other half women. Concerning the categories, participants were:
- 2 Journalists - 2 Job applicants - 2 Potential partners - 4 Students or researchers - 2 Already customers or potential new customers
Since there are two different hypotheses to be tested, the sample has been divided in two different groups of the same size. Each user has been personally contacted and asked, if they were willing to participate to a research project related to a master thesis. Once gathered their permission, a meeting has been scheduled.
4.3 Procedure
Due to 2020 Coronavirus pandemic in Italy [39], also the experiment to evaluate brand perception changes has been conducted remotely using Zoom [43] and its remote control functionality. The questionnaire evaluating the perception was summited to the users using Google Forms. To perform the experiment, a link to a Zoom meeting has been sent to each participant. After the test subject entered into the meeting, a link to the questionnaire has been sent in the meeting chat. Before starting the experiment, the purpose of the website and of the experiment was orally explained to each user. The “thinking aloud” procedure [42] was adopted and it was explained to the users.
The use of the two different versions of the website was performed within scenarios granting the remote-control access. Each user has been asked to perform one scenario on each version of the website. The scenario to be performed was chosen among the following list based on the category of user:
- Scenario 1: Read the latest press release of the company. - Scenario 2: Discover the latest event or initiative organized by the company
63
- Scenario 3: Read what employees say about the company - Scenario 4: Find all the open positions in the company
Since the new website is offering additional functionalities, it was not possible to perform all the above scenarios in the old website. Based on the categories of users they have been assigned as follow:
User category Old website New website
Journalists Scenario 1 Scenario 1
Job applicants Scenario 4 Scenario 3
Potential partners Scenario 1 Scenario 1
Students and researchers
Scenario 1 Scenario 1
Potential new customers and already customers
Scenario 1 Scenario 2
Table 7: Assigned scenarios for the experiment
To test the changes in brand perception on the two groups, two different procedures have been used:
Testing H1: Increase in HU impacts positively the brand perception change
1. Questionnaire on brand perception, before any interaction with the website 2. Use of the old website to perform the defined scenario 3. Questionnaire on brand perception 4. Use of the new website to perform the defined scenario 5. Questionnaire on brand perception 6. Interview to gather qualitative data
64
Testing H2: Decrease in HU impacts negatively the brand perception change
1. Questionnaire on brand perception, before any interaction with the website 2. Use of the new website to perform the defined scenario 3. Questionnaire on brand perception 4. Use of the old website to perform the defined scenario 5. Questionnaire on brand perception 6. Interview to gather qualitative data
4.4 Data collection
To gather quantitative data needed to investigate the correlation between usability and brand perception, participants were asked to fill in a survey several times. For each brand value of the company, a question has been proposed to the user following the structure:
“How much does ‘Brand X’ inspire you ‘Value Y’?”
Since the audience of the website is made up of Italian speaking people, all the questions have been opportunely translated into Italian following this structure:
“Quanto ‘Brand X’ trasmette a te ‘Value Y’?”
The brand values identified by the company that need to be delivered to the users are:
English Italian
BV1 Truth Verità
BV2 Transparency Trasparenza
BV3 Simplicity Semplicità
65
BV4 Trust Fiducia
BV5 Innovation Innovazione
Table 8: Brand values
To express their perception, for each question a score between 0 and 5 with a step size of 1 has been assigned by test subjects:
- 0: The brand value in the question is not perceived at all - 5: The brand value in the question is strongly perceived
The gathered quantitative data have been computed to identify a “brand perception” score. The level of brand perception of a user i at a certain time t has been computed as follow:
,-(/)' =%,(/)'12
3
1*+
where:
- t is the time when the evaluation is computed: it is 0 before any interaction with the website, 1 after the interaction with the first website and 2 after the interaction with the second website.
- m is the cardinality of the set of brand values.
- ,(/)'1 is the score assigned by the user i to the brand value j at time t.
To compute a unique score that can be directly compared with heuristic usability, the overall level of brand perception at a certain time t has been computed as follow:
45(/) =%,-(/)'(
)
'*+
where:
- n is the sample size.
66
Thanks to the computation of this score, it has been possible to create a correlation with the usability using the HU defined in section 3.6.
Figure 26: Correlation between usability and brand perception
Once the data gathering on brand values was completed, an unstructured interview at the end of each procedure aimed at collecting qualitative data and obtain some feedback about the new website.
67
68
5. Brand perception evaluation: Results
5.1 Main findings
The gathered data from the questionnaires have been opportunely computed to obtain the following outcomes. The results are presented based on the procedure they are referring at, and then combined to illustrate some overall outcomes.
Considering the data gathered following the procedure “Testing H1”, the different values assigned to t have the following meanings:
- t = 0: Before any interactions with the websites - t = 1: After the interaction with the old website - t = 2: After the interaction with the new website
The level of brand perception, at those different points in time, has been computed obtaining values in Figure 27:
Figure 27: Brand perception outcome "Testing H1"
In the procedure “Testing H2”, the values of t assume different meanings:
- t = 0: Before any interactions with the websites
4,07
3,93
4,2
0 1 2
Brand perception outcome "Testing H1"
BP(t)
69
- t = 1: After the interaction with the new website - t = 2: After the interaction with the old website
The outcomes of this procedure are illustrated in Figure 28:
Figure 28: Brand perception outcome "Testing H2"
The results obtained in the two different procedures can be combined to obtain additional interesting outcomes. Merging the scores of the two procedures obtained at t=0 to a single average, it is possible to have a look at the overall initial brand perception of test subjects. If we consider merging the scores obtained in the procedure “Testing H1” at t=1 with the scores in the procedure “Testing H2” at t=2, it is possible to understand the overall brand perception obtained after the use of the old website, vice versa merging the scores of the procedure “Testing H1” at t=2 with the scores of the procedure “Testing H2” at t=1 will result in the overall brand perception after the use of the new website. These results are illustrated in Figure 29.
3,93
4,23
3,77
0 1 2
Brand perception outcome "Testing H2"
BP(t)
70
Figure 29: Brand perception procedures combined
With the previously described procedures it has been possible to gather also important qualitative feedback. The adopted “thinking aloud” approach [42] allowed us to verify that all the tasks proposed to test subjects have been fully understood. No major difficulties have been encountered by test subjects in dealing with the assigned tasks. Their problem-solving strategies always matched the designed navigation path allowing them to complete the task in few clicks.
The interviews at the end of each session resulted important to understand their ideas regarding the new website. The main feedbacks are:
- The new website is well structured. - The brand colors in the new website are more visible. - The values of the company are clearer and better delivered in the new
website. - The new website was more engaging. - The presence of a section related to events and initiatives in the new
website, for a potential partner, is important since can understand that its logo can appear on that page in a specific post increasing the awareness of the brand.
- The font size used made reading experience easier.
4 3,854,22
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
5
BP
Brand perception precedures combined
Initial After the use of the old website After the use of the new website
71
- The way the open positions in the company is structured in the website, is a strong call to action to discover if there is something also for you.
- Looking at the employees and their positions is truly motivating. - All the sections are clearly reachable - The filter in the archive should be more visible - The dates are well highlighted - The presence of a text preview for the different press releases is really
helpful
5.2 Discussion
The trend obtained from the two different procedures has been analyzed to identify possible correlations between usability and brand perception.
Considering Figure 27 the initial brand perception of the users was 4.07. This value is relatively high, and it can be related to strong marketing campaigns implemented by the company on the different communication channels. After the use of the old website their perception weakly decrease to 3.93, this trend illustrated how the brand values maybe were not transmitted in the old version of the website. Moving the focus on their perception after the use of the new website, here we have an increase that set the level of brand perception to 4.2. This score is even greater than the initial one, highlighting how the messages delivered in the new website increased their awareness and perception.
Considering the usability of the two websites it is possible to analyze the correlation between usability and brand perception:
72
Figure 30: Correlation "Testing H1"
Looking at Figure 30, the increase in terms of heuristic usability (HU) results in an increase of brand perception. With this result it is possible to confirm the hypothesis H1.
If we consider instead Figure 28, the initial brand perception was 3.93. The interaction with the new website increased their perception to 4.23, this result can be addressed to the messages delivered by the website, that added to their prior idea about the brand, strengthened their perception. The use of the old version of the website after the use of the new one resulted in a strong decrease in their perception, reaching a value of 3.77.
Including usability in this analysis, results in the following correlation:
3,934,2
3,36
4,86
Old website New website
CORRELATION "TESTING H1"
BP HU
73
Figure 31: Correlation "Testing H2"
Figure 31 illustrates that for a decrease in terms of heuristic usability there is a decrease in brand perception. This result aims to confirm the hypothesis H2.
It is interesting to analyze the differences obtained in these two different procedures. In Figure 27 the delta in terms of brand perception score in switching from the old website to the new one is 0.27 compared to 0.46 in Figure 28. A possible explanation is that moving from the old website to the new one, the brand is more perceived but not strongly. On the other hand, considering the delta obtained in moving from the new website to the old one, we can assume that the presence of strong brand messages in the new website has been perceived by users as normal and that switching to the old website, where there is a lack of those messages, has been strongly perceived in a way that their brand perception decreased.
Once analyzed the two procedures, looking the results obtained merging these values, in Figure 32 we obtain a clear picture of the correlation between usability and brand perception:
4,233,77
4,86
3,36
New website Old website
CORRELATION "TESTING H2"
BP HU
74
Figure 32: Heuristic Usability & Brand Perception
Considering our results, we can conclude that usability of a web-based communication artefact does affect brand perception. An additional knowledge contribution is that the brand perception is most affected when switching from a website with higher usability to another one with a lower usability. Switching from a website with lower usability to one with higher usability affects brand perception as well but to less extent.
To conclude this discussion, another interesting analysis can be performed moving the focus on a higher perspective. In other words, with these outcomes we highlighted the correlation between usability and brand perception, but another important aspect has been also tested: the interaction with a branded artefact affects the brand perception of a user. This result has been intrinsically tested in the experiment.
3,36
4,86
3,854,22
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
5
Old website New website
Heuristic Usability & Brand Perception
HU BP
75
76
6. Lessons learned 6.1 Theoretical framework critique
Analyzing the two procedures adopted to gather quantitative and qualitative data, looking at the results obtained, they have been a good tool to test the research hypotheses. Some critical aspects emerged analyzing the users' behavior. At first sight the pre-usage questionnaire about brand perception seemed a bit difficult to be answered since most of them were not familiar with marketing concepts and brand. They encountered major difficulties in understanding how actually those brand values were perceived.
The adopted procedures have been built to evaluate the re-design process performing a comparative study. These procedures compared to the one adopted in previous literature result longer and more structured since two websites should be tested. A possible drawback of this insight is that it has been more difficult for the evaluator to recruit users since it required about 30 minutes for each user to perform the experiment.
The remote-control functionalities adopted due to Coronavirus pandemic, influenced both positively and negatively the results obtained in this work. Indeed, in a limited number of test subject presented some delay problems in the animations even if overall test subjects stated that the websites were fully usable. On the other hand, the possibility to perform the experiment remotely helped in recruiting test subjects of specific categories. Considering the journalists category, indeed, probably in normal conditions performing an experiment in presence with them may result in time delay due to their busy schedules.
The use of this theoretical framework on a real case scenario has been important to test whether or not it can be effectively used to analyze the improvements made in a re-design process. The results obtained suggest a positive answer that allow us to suggest the adoption of this framework to other re-design processes.
77
6.2 Marketing perspective
From a marketing perspective, this experiment introduces interesting insights. The first tangible impact of the experiment is that the number of users on which the website has been tested within real scenarios increased of 12 people (the sample size of the experiment). The total number is now 17 people considering 5 users coming from the user testing procedure performed before the experiment. It can be considered a good achievement since, as previously stated in section 3.5, it is true that 5 users can be considered enough but with an higher number additional feedback can be gathered.
The feedback gained during the experiment have been added to the set already gathered during user testing. These feedbacks are important since will be used for further development to improve the website.
Having tested the website with the categories of users the website has been designed for, helped in understanding whether or not the messages are delivered correctly. An additional innovative aspect introduced by this master thesis, indeed, is the use of this theoretical framework in a real case development environment. Testing brand perception before actually releasing the website to public helped in understanding if there were possible improvements in e-branding that can be performed. This is a not so common practice in companies, and it can be seen as a step forward in branding and web development.
78
79
7. Conclusions and future works
In this master thesis project, a full re-design process and evaluation of the results have been performed. The whole argumentation has been built around the concepts of web usability, corporate websites, e-branding and brand perception. The research question investigated has been deeply considered while developing this project, trying to build a research method having procedures that can clearly answer and test the research hypotheses. Interesting ethics and sustainability issues has been pointed out hoping that this research work can contribute also in those aspects.
The literature analyzed aimed to introduce the discussed topics and present the actual state of the art of the theoretical framework used within the research project. Available usability evaluation methods have been illustrated, describing the characteristics of the chosen one. The value driven design has been presented explaining their major innovations. A deep analysis of the available literature has been performed to identify previous works on the correlation between usability and brand perception, the outcomes of this analysis pointed out the presence of prior analyses that in this project allowed us to build important knowledge contribution.
The key element around which the whole research project has been built is the re-design process of the “Iliad Italia S.p.A.” corporate website. An iterative design process has been performed together with the marketing team, based on the user needs, goals and requirements. In the development process, with the help of the software development team, new skills have been developed allowing the creation from scratch of a corporate website with the most advanced web technologies available on the market.
The research project aimed to verify a possible correlation between usability and brand perception, following some works in the field but trying to integrate innovative aspects in the research method. Two major research hypotheses have been tested obtaining interesting results. To test these two research hypotheses two different innovative research procedures have been designed and implemented. The outcomes of the analysis allowed us to confirm both hypotheses
80
and to obtain some additional interesting feedback that can be used for further development. Indeed, according to results, brand perception is affected by web usability in both directions.
The results obtained in this research project are important also for other types of organizations. Indeed, these results highlighted how each company that is trying to build a strong brand in this digital era should care about usability of their web-based communication artefacts. The importance of this work is also in the described structured procedure that can be used by companies to assess brand perception of their target users.
The historical moment in which this master thesis has been carried on forced the research to be performed remotely. Due to the dramatic situation in Italy, people were lockdown in their houses, and it has been impossible to perform user testing in presence. This is the reason why the possible future work can be to perform the same analysis with user testing and the experiment based on the real presence of test subjects. This is a proposal that can give us the possibility to gather new insights that can be added to the analysis performed in this master thesis. This new approach will allow us to avoid possible delay problems encountered in the procedures used in this project.
To conclude this research project, it is important to highlight how thanks to this master thesis innovative research tools have been introduced in a real case scenario. The way these procedures and tools have been used can be seen by other companies as an inspiration for their future projects, allowing them to move branding and web development a step closer each other.
81
Bibliography
[1] ISO/IEC, "9241-11 Ergonomic requirements for office work with visual display terminals (VDT)s - Part 11 Guidance on usability.," 1998.
[2] "Total number of Websites - Internet Live Stats," [Online]. Available: https://www.internetlivestats.com/total-number-of-websites/. [Accessed 21 February 2020].
[3] Gustavsen and Tilley, "Public Relations Communication through Corporate Websites: Towards an Understanding of the Role of Interactivity," PRism OnLine PR Journal, 2003.
[4] AMA, "Definitions of Marketing," [Online]. Available: https://www.ama.org/the-definition-of-marketing-what-is-marketing/. [Accessed 21 February 2020].
[5] S. Goodson, "Why Brand Building Is Important," Forbes, 27 May 2012. [Online]. Available: https://www.forbes.com/sites/marketshare/2012/05/27/why-brand-building-is-important/.
[6] K. Kaplan, "Brand Is Experience in the Digital Age," 3 July 2016. [Online]. Available: https://www.nngroup.com/articles/brand-experience-ux/.
[7] K. Flaherty, "User Experience vs. Customer Experience: What’s The Difference?," 9 June 2019. [Online]. Available: https://www.nngroup.com/articles/ux-vs-cx/.
[8] M. Grzesiak, "E-BRANDING vs. TRADITIONAL BRANDING," Modern Management Review, 2015.
[9] H. Allcott, M. Gentzkow and C. Yu, "Trends in the Diffusion of Misinformation on Social Media," 2019.
82
[10] A. Fernandez , E. Insfran and S. Abrahão, "Usability evaluation methods for the web: A systematic mapping study," Information and Software Technology, 2011.
[11] D. Bolchini and F. Garzotto, "QUALITY AND POTENTIAL FOR ADOPTION OF USABILITY EVALUATION METHODS: AN EMPIRICAL STUDY ON MILE+," Journal of Web Engineering, vol. 7, no. 5.
[12] D. Bolchini and P. Paolini, "Interactive dialogue model: a design technique for multichannel applications," IEEE Transactions on Multimedia, 2006.
[13] D. Bolchini and F. Garzotto, "Designing Multichannel Web Applications as “Dialogue Systems”: the Idm Model," Web Engineering: Modelling and Implementing Web Applications Human-Computer Interaction Series, vol. 12, 2007.
[14] J. Carroll, "Making Use – Scenario-based design of Human-Computer Interactions," MIT Press, 2002.
[15] D. Bolchini, F. Garzotto and P. Paolini, "Branding and Communication Goals for Content-Intensive Interactive Applications," 15th IEEE International Requirements Engineering Conference (RE 2007), 2007.
[16] D. Bolchini, F. Garzotto and P. Paolini, "Design Requirements for Communication-Intensive Interactive Applications," Lecture Notes in Business Information Processing Design Requirements Engineering: A Ten-Year Perspective, 2009.
[17] H. Dubberly, "A Model of Brand," 2001. [Online]. Available: http://www.dubberly.com/concept-maps/a-model-of-brand.html.
[18] A. Marcus, "Branding 101," Interactions, vol. 11, no. 5, pp. 14-21, 2004.
[19] Young and Rubicam, "Summary of the BAV Model of Young and Rubicam," [Online]. Available: https://www.valuebasedmanagement.net/methods_brand_asset_valuator.html.
83
[20] D. Bolchini, T. Yang and F. Garzotto, "Evaluating the communication design of branded websites," Proceedings of the 27th ACM international conference on Design of communication - SIGDOC 09, 2009.
[21] D. A. Aaker, Building strong brands, New York: NY: The Free Press, 1996.
[22] B. Müller and J. Chandon, "The Impact of Visiting a Brand Website on Brand Personality," Electronic Markets, vol. 13, no. 3, pp. 210-221, 2003.
[23] D. Bolchini, F. Garzotto and F. Sorce, "Does Branding Need Web Usability? A Value-Oriented Empirical Study," Human-Computer Interaction – INTERACT 2009 Lecture Notes in Computer Science, 2009.
[24] Wordpress.com, "Wordpress," [Online]. Available: https://wordpress.com.
[25] S. Davis, "The pros and cons of using a CMS to build your website," 24 May 2017. [Online]. Available: https://flickerleap.com/pros-cons-using-cms-build-website/.
[26] I. D. Foundation, "User Centered Design," [Online]. Available: https://www.interaction-design.org/literature/topics/user-centered-design.
[27] A. Crespo, "5 Reasons to use Paper Prototypes in UX," 15 December 2015. [Online]. Available: https://www.nomensa.com/blog/2015/5-reasons-use-paper-prototypes-ux.
[28] JReport, "3-Tier Architecture: A Complete Overview," [Online]. Available: https://www.jinfonet.com/resources/bi-defined/3-tier-architecture-complete-overview/.
[29] O. Corporation, "MySQL Database," [Online]. Available: https://www.mysql.com.
[30] "ORM Framework uses advantages and disadvantages of the post-frame," 2018. [Online]. Available: https://topic.alibabacloud.com/a/orm-framework-uses-advantages-and-disadvantages-of-the-post-frame_8_8_20284586.html.
84
[31] D. S. Foundation, "Django, The web framework for perfectionists with deadlines.," [Online]. Available: https://www.djangoproject.com.
[32] D. Stars, "10 Popular Websites Built With Django," [Online]. Available: https://djangostars.com/blog/10-popular-sites-made-on-django/.
[33] "Django REST Framework," [Online]. Available: https://www.django-rest-framework.org.
[34] E. You, "Vue.js, a progressive javascript framework," [Online]. Available: https://www.google.com/search?client=safari&rls=en&q=vue.js&ie=UTF-8&oe=UTF-8.
[35] "Nuxt.js, the progressive Vue.js framework," [Online]. Available: https://nuxtjs.org.
[36] "Vuetify, material design component framework," [Online]. Available: https://vuetifyjs.com/en/.
[37] "OpenAPI Specification," [Online]. Available: https://swagger.io/specification/.
[38] J. Nielsen, "Why You Only Need to Test with 5 Users," 18 March 2000. [Online]. Available: https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/.
[39] "2020 coronavirus pandemic in Italy," 2020. [Online]. Available: https://en.wikipedia.org/wiki/2020_coronavirus_pandemic_in_Italy.
[40] A. S. GmbH, "AnyDesk, Remote Desktop Software," [Online]. Available: https://anydesk.com/en.
[41] Cisco, "Cisco Webex Teams," [Online]. Available: https://www.webex.com/team-collaboration.html.
[42] J. Nielsen, "Thinking Aloud: The #1 Usability Tool," 15 January 2012. [Online]. Available: https://www.nngroup.com/articles/thinking-aloud-the-1-usability-tool/.
85
[43] I. Zoom Video Communications, "Zoom," [Online]. Available: https://zoom.us.
86
Appendix A – Questionnaire answers
Testing H1:
87
88
89
90
Testing H2:
91
92
93
www.kth.se
TRITA -EECS-EX-2020:455
