View
214
Download
0
Category
Tags:
Preview:
Citation preview
Usability Test – UNESCO.org
Full Report June 8 2006
Laetitia Giannettini [giannettini@axance.fr] Usability SpecialistCyril Alegre [alegre@axance.fr] Usability Specialist
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
2Table of Contents
1. Study Background and Objectives
2. Methodology
3. Executive Summary
4. Detail of Results
5. Main Recommendations
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
4Background of the Study and Objectives
The United Nations Educational, Scientific and Cultural Organization (UNESCO) is a specialized United Nations agency that promotes peace through Education, Social and Natural Science, Culture and Communication programmes across the world.
UNESCO’s Bureau of Public Information (BPI) is the service in charge, among other functions, of public relations and providing audiovisual media. Its also in charge of the administration and editorial coordination of the UNESCO web portal: www.unesco.org.
The web portal aggregates a great amount of distinct UNESCO websites with different contents and look and feel. BPI wishes to create a standard setting and a capacity building process to harmonize standardize these websites.
A framework has been developed to migrate all the websites’ pages into a new look and feel essentially based on usability and accessibility criteria and previous user tests. The home page and the Worldwide and Organisation sections have been migrated so far.
In addition, the UNESCO portal now provides content in 6 languages: Arabic, Chinese, Russian and Spanish (the four other official languages) were added to French and English (the Organization’s working languages languages).
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
5Background of the Study and Objectives
BPI wished to assess the perception and understanding of the portal by the representatives of the UNESCO Executive Board
The study’s objectives were to assess:
• Perception and understanding of the home page and its main items (access, tools, navigation…)
• Perception and understanding of pages in the main sections (Themes, Worldwide, Organisation, Services)
• Opinions toward the online version of the UNESCO Courier
• Opinions toward the availability of multilingual contents
• Opinions toward the legitimacy of BPI in coordinating the portal
The applied methodology was user testing (see section 2. Methodology)
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
6Background of the Study and Objectives
This actual user testing follows a previous one, which took place in February 2004
The main objectives of the first study were to assess the utility (usefulness) and usability of the portal by UNESCO’s target public.
18 users divided into 6 profiles (3 participants per profile) were consulted for the first study:
• Researchers/Scientists/Academics
• Media
• Teachers/Educators/Trainers
• Decision-makers/Civil Servants in Supervising Ministry/Governing bodies
• Development agents (NGOs, IGOs, UN)
• Students/Young people
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
8Background of the Study and Objectives
User testing is mostly a qualitative method based on observational and oral data
It assesses users’ experience through two major criteria:
• usability (usefulness) of the website: is the website easily and efficiently usable?
• utility of the website: does the website meets users’ needs and expectations (contents, tools, etc.)?
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
9Methodology > Usability Testing
Individual sessions in three steps:
• Step 1: Pre-test questionnaire
– User’s profile and their use of the UNESCO web portal
• Step 2: Website browsing
– perception and understanding of the home page
– perception and understanding of pages in the main sections
– opinions toward the online version of the UNESCO Courier
• Step 3: Post-test questionnaire
– General opinion of the website
– Performance and satisfaction rating (questionnaire for performance indicators)
Duration of Sessions : variable – between 10 and 45 minutes according to the participants availability
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
10Methodology > User Profiles
User profiles (cf. Excel spreadsheet for details) :
• 55 participants to sessions of the UNESCO’s Executive Board
• 22 female and 33 male
• From 39 Permanent Delegations (cf. list on next slide)
• Age 22 to 66
• 30 users saw the portal in English and 25 users saw the portal in French
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
11Methodology > User Profiles
Participants to this test are from the following Countries:
Algeria (2) Argentina Austria Bahrain (2) Bangladesh Canada Cambodia China
Czech Republic (2)
Egypt Ethiopia European Commission (2)
Greece Guatemala Holy See (2) India (2)
Indonesia Italy (2) Kazakhstan Kuwait Luxembourg (4)
Madagascar Netherlands Nigeria
Norway Palestine (2) Poland Portugal (3) Romania Russian Federation
Slovenia (2) Sri Lanka (2)
Switzerland Thailand (2) Uganda United Kingdom
Uruguay Yemen Zimbabwe
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
13Strong Points
Plenty of information available – the portal is a useful working tool
Some information is available in several languages
The home page layout is clear
The work done in order to add some consistency across the portal (the Framework) was noticed and appreciated
The “Worldwide” section is found useful (the possibility to find information about members states, its programs)
The online UNESCO Courier was appreciated
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
14Weak Points
Guidance needs some improvement
The portal is rather expert-oriented – outsiders and new comers need some time to understand it
Some contents are outdated or missing
Users have difficulty finding certain information and documents and the search engines are not efficient enough
There is still a lack of consistency between the different sections – this hurts understanding and learnability
Users expect more multilingual content
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
16Results Details – Severity ratings
The results are classified according to their severity. This classification should also be seen as an emergency rating – the higher the number the sooner rework should be done:
Graphical issue
Minor usability issue
Medium usability issue
Major usability issue
Good point
2
4
3
1
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
17Results Details – Table of contents
A. Pre-test questionnaire – Users’ profiles and web habits
B. Browsing observation – Perception and understanding of the home page
C. Browsing observation – Perception and understanding of the “Worldwide” section
D. Browsing observation – Perception and understanding of the UNESCO Courier
E. Browsing observation – Transversal issues
F. Post-test questionnaire – General opinions
G. Post-test questionnaire – Performance and satisfaction rating
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
18A. Pre-test questionnaire
The age spread was as shown in the graphic opposite
The average age was around 41
Half of the participants had spent more than 2 years working for UNESCO
The average seniority was around 3.5 years
Seniority
5+16%
2-537%
1-225%
0-122%
Age spread
20-2920%
30-3917%
40-4937%
50-5917%
60+9%
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
19A. Pre-test questionnaire
Most participants were web experts: 83% of them have been using for more than 5 years – on average for 10 years and for 16 hours per week nowadays
Spreads as shown in the graphic opposite
Internet expertise (years)
10+23%
0-10%
1-24% 2-5
13%
5-1060%
Hours spent using Internet every week
20+16%
5-1022%
10-2054%
0-58%
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
20A. Pre-test questionnaire
Almost all participants had already browsed UNESCO.org.
Most often, participants started using the portal as they started working at UNESCO. On average, they have been using it pretty often (on a daily basis for most respondents) for 4.5 years
Spread is as shown in the graphic opposite
UNESCO portal expertise (years)5+9%
0-115%
1-228%
2-548%
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
21A. Pre-test questionnaire
Mostly, the participants browse the portal to find information and documents. The most visited sections are:
• Themes (mentioned by 22 participants) to get information about UNESCO’s programs and activities
• Organization (22) to collect information and documents from the Executive Board, the General Conference (meetings, resolutions, speeches…)
• Services (20) to find some documents and publications (conventions, resolutions…) and information about legal instruments
• News (14) and Events (11) to check what’s happening (UNESCO’s activities, exhibitions, conferences…)
• Communities (7) to find contacts
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
22A. Pre-test questionnaire
The main strong points reported by the participants according to their own experience were:
• The great amount of information availableThe portal is seen as a useful source of information to keep up to date about UNESCO news, about “what’s currently going on at UNESCO”. Furthermore, the UNESCO portal is often seen and used as a regular working tool: some users, for example, use it to collect information in order to prepare Executive Board meetings
• The overall ease of use and easy access to the information
• The contents available in 6 languages
Strong points
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
23A. Pre-test questionnaire
The main weak points reported by the participants were:
• Overall complexity, especially for outsiders or newcomers – the portal was often said to take some understanding of UNESCO structure to be efficiently used
• Difficulty in finding certain information and documents: all the participants said they could find most of what they were looking for but many also reported some problems in finding some information
• Outdated contents: many users deplored the slowness – or lack, at worst – of updates
• Lack of multilingual contents
Weak points
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
24A. Pre-test questionnaire
Most participants were aware that BPI was in charge of the UNESCO website
Most of them also had some general knowledge of BPI activities:
“Press information, Public Relations” (Holy See)
“Publishing, Communications” (China)
“Public information about UNESCO’s activities” (Italy)
“Promotion of UNESCO’s image” (Sri Lanka)
Other main activities reported: website administration, content feed, updates and coordination
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
25Results Details – Table of contents
A. Pre-test questionnaire – Users’ profiles and web habits
B. Browsing observation – Perception and understanding of the home page
C. Browsing observation – Perception and understanding of the “Worldwide” section
D. Browsing observation – Perception and understanding of the UNESCO Courier
E. Browsing observation – Transversal issues
F. Post-test questionnaire – General opinions
G. Post-test questionnaire – Performance and satisfaction rating
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
26B. Perception and understanding of the home page
Most users found the home page layout clear and user friendly
They understood the 3-column layout and the left and right menus
The sections’ organisation and the wordings were well understood
“It’s clear, one can find almost everything he needs” (China)
“It’s practical, I can find what I want” (Indonesia)
The home page is clear
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
27B. Perception and understanding of the home page
The languages available on the home page were well perceived and the work done to provide multilingual contents highly appreciated
“They are very useful” (Italy)
The tools are well perceived – the search engine is the most attractive
“Practical, I find what I want” (Indonesia)
The home page is clear
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
28B. Perception and understanding of the home page
The home page is seen as being kept up to date (with changing pictures, themes, news and events), which gives the website a dynamic look that was appreciated
The home page is regularly kept up to date
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
29B. Perception and understanding of the home page
Users found the home page had too much information and links on it, especially in the right menu
“Too many things – I can’t get any information” (Holy See)
“It’s compact, there are too many things on it, which distracts one’s attention – even more for someone browsing it for the first time” (India)
“The right menu offers too many things – nothing shows up” (Switzerland)
High informational density3
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
30B. Perception and understanding of the home page
A few users did not understand the wording “Communities” nor anticipate the content of this section
The right menu is too heavy for non-UNESCO users – it takes some good knowledge of the UNESCO organisation to understand the menu
The high density in the right menu restrains the visibility of the links of interest – they do not attract attention (e.g. “General Conference” and “Executive Board”)
High informational density3
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
31B. Perception and understanding of the home page
Hide the submenus (under “Secretariat”) – and make it available by a click on “Secretariat”?
By default, the Organisation menu should only offer “About UNESCO”, “General Conference”, “Executive Board”, “Director-General” and “Secretariat”
“The home page should focus: it should only give access to UNESCO’s main sectors and organs” (India)
“The ‘General Conference’ and ‘Executive Board’ links should show up more” (Switzerland)
This 3 column layout is more adapted for 1024 resolution.
High informational density3
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
32B. Perception and understanding of the home page
A few users deplored the fact that the home page like the whole website did not convey efficiently UNESCO’s mission, values and messages
“I would prefer a global picture in the central area, a more universal look to it” (Kazakhstan)
“The home page fails at showing what UNESCO is” (Canada)
The home page does not communicate UNESCO’s spirit enough
3
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
33B. Perception and understanding of the home page
The home page content is rather insider-oriented: non-UNESCO users may not get a good insight of what UNESCO is and does
A baseline on top of the page could provide a good introductory overview of UNESCO’s definition missions:
“UNESCO - the United Nations Educational, Scientific and Cultural Organization (UNESCO) was founded on 16 November 1945. For this specialized United Nations agency, Education, Social and Natural Science, Culture and Communication are the means to a far more ambitious goal : to build peace in the minds of men”
The home page does not communicate UNESCO’s spirit enough
3
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
34B. Perception and understanding of the home page
The central area does not give an overview of UNESCO’s core activities (i.e. key international programs)
The central area is not well perceived, it doesn’t catch UNESCO people’s attention a lot – insiders are more attracted to the menus
“I don’t browse the central area” (Holy See)
“It doesn’t catch my attention” (Kazakhstan)
3The home page does not communicate UNESCO’s spirit enough
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
35B. Perception and understanding of the home page
The central area layout is not clear:
• The articles in the “News” and “Events” boxes are separate with white lines and look like they’re discrete boxes – therefore, the second article, for example, does not seem to be part of the news
The boxes should be united
The central boxes’ layout is confusing3
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
36B. Perception and understanding of the home page
The themes tabs on top of the page were not well perceived by some users – the lack of consistency across the portal restrains learnability of the navigation:
• Different graphical aspect
• Different tabs – some sites offer an “The Organisation” tab or a “Site map” tab, which could be extended to the whole portal
Some accesses are not perceived2
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
37B. Perception and understanding of the home page
Some users deplored the fact that the home page did not mention the ongoing Executive Board (Palestine, European Commission)
The home page does not mention some current events2
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
38B. Perception and understanding of the home page
The tab’s icon and label are so far from each other that they don’t seem to be linked together
The site map icon needs some graphical readjustment
The French “Plan” tab is not well designed2
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
39Results Details – Table of contents
A. Pre-test questionnaire – Users’ profiles and web habits
B. Browsing observation – Perception and understanding of the home page
C. Browsing observation – Perception and understanding of the “Worldwide” section
D. Browsing observation – Perception and understanding of the UNESCO Courier
E. Browsing observation – Transversal issues
F. Post-test questionnaire – General opinions
G. Post-test questionnaire – Performance and satisfaction rating
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
40C. Perception and understanding of the “Worldwide” section
The “Worldwide” access was well perceived by most participants
It was also well understood: most participants had a good insight of what the section offers – local information about UNESCO organisation and programmes
The Worldwide section available from the left menu was generally appreciated as a useful source of local information
The “Worldwide” access was well perceived and appreciated
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
41C. Perception and understanding of the “Worldwide” section
The “Worldwide” access was easily used by most users – they had no difficulties accessing the page dedicated to their country
The “Worldwide” access is mostly clear
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
42C. Perception and understanding of the “Worldwide” section
A few users got confused by the “Worldwide” access, which follows UNESCO’s own logic in aggregating regions of the world, which does not match the common logic about world’s continents and therefore may confuse non-UNESCO people – e.g. Europe and North America form a single region and the Russian Federation and Turkey are listed twice, both in the “Europe and North America” and the “Asia and the Pacific” regions.
The “Worldwide” access is also confusing2
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
43C. Perception and understanding of the “Worldwide” section
Several interpretations were given as for UNESCO’s own logic in aggregating regions of the world:
• Regions are voting groups
• Regions follow a “political” logic
The “Worldwide” access is also confusing2
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
44C. Perception and understanding of the “Worldwide” section
The availability of country-specific information (themes) was mostly appreciated
The cross-referencing of information principle was appreciated
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
45C. Perception and understanding of the “Worldwide” section
Some users did not notice that some more content was available in the lower area of the page
The page layout should allow users to notice the content below – the browser should “cut” the page where some content is partly visible in order to suggest some more content
3The themes were not easily perceived by users
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
46C. Perception and understanding of the “Worldwide” section
A few users deplored some difficulties anticipating the contents “behind” the themes links on the country-specific pages
The links are not clear enough, not understood as being contextual, country-specific
There is no clear introduction
Explanatory sentences should introduce the themes’ content (what about the type of links is)
The themes’ content was not easily anticipated by users 2
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
47C. Perception and understanding of the “Worldwide” section
Many users did not see the “Organisation” menu on the country-specific pages
• The background colour and the search engine and “UNESCO at 60” picture above the menu make it “secondary” (Austria)
• The “organisation” menu lacks consistency
• “The Organization” label is not country-specific and the “UNESCO in this Country” label is not well located – it applies only to the 2 links underneath it
The “Organisation” menu lacks visibility3
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
48C. Perception and understanding of the “Worldwide” section
The menu should be moved upward in the place of the search engine and picture and gain consistency from one page/section to another
The menu label should be more specific
The “Organisation” menu lacks visibility3
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
49C. Perception and understanding of the “Worldwide” section
According to the users, some contents are poor or information is missing or not enough emphasised :
• The countries’ main sectors of activity and the key programmes (Luxembourg, Greece, Holy See, Poland, Slovenia, Ethiopia, etc.)
• The statistics should come along with interpretations (Indonesia)
• The National Commission programme (Netherlands)
• A link to the country website from the National Commission page (Argentina, Kazakhstan)
Content was sometimes assessed not to be full-accomplished
2
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
50C. Perception and understanding of the “Worldwide” section
Some contents are not up to date – and some pages are still empty (Slovenia)
Some users (Slovenia, Luxembourg) suggested the National Commissions should be involved and provide BPI with information – it could be through a form –, given the National Commissions are in charge of defining their status, composition principles and main sectors of activities
Some country-specific pages content is outdated2
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
51C. Perception and understanding of the “Worldwide” section
Some users found the pictures could be more attractive (India, Luxembourg) and linked to the content
Some country-specific pages pictures are disliked2
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
52C. Perception and understanding of the “Worldwide” section
The text is compact – its format should be improved in order to ease the reading:
The main words and sentences should be written in bold font
Text should be divided into different paragraphs
There should be more pictures on both side of the central area
Text format is not adapted2
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
53Results Details – Table of contents
A. Pre-test questionnaire – Users’ profiles and web habits
B. Browsing observation – Perception and understanding of the home page
C. Browsing observation – Perception and understanding of the “Worldwide” section
D. Browsing observation – Perception and understanding of the UNESCO Courier
E. Browsing observation – Transversal issues
F. Post-test questionnaire – General opinions
G. Post-test questionnaire – Performance and satisfaction rating
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
54D. Perception and understanding of the UNESCO Courier
Most users would likely read the UNESCO Courier online
Most of them would like to print it out – be it in PDF format or directly from their Internet browser
The UNESCO Courier was appreciated
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
55D. Perception and understanding of the UNESCO Courier
All the users would likely participate in the diffusion of the UNESCO Courier, mostly via email – as a link, as a PDF document or using a “send this article to a friend” feature
Users would diffuse the UNESCO Courier
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
56D. Perception and understanding of the UNESCO Courier
The website offers several accesses to the UNESCO Courier
Several ways to access the UNESCO Courier
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
57D. Perception and understanding of the UNESCO Courier
The UNESCO Courier box was not perceived: many users missed it although they had scrolled the page down to the bottom
The UNESCO Courier access should be easier: it should show up more, either visually (a specific colour, layout) and/or by its location (to be moved upward, in a menu?)
The title “The UNESCO Courier” should be clickable
The UNESCO Courier central access is not well perceived3
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
58D. Perception and understanding of the UNESCO Courier
The first paragraph is a general introductory mention to the online version of the UNESCO Courier
As such, it should be located on top of the page, above the first picture and first article
The “Free subscription” link should be located right under the first paragraph
The right menu is not clear enough: the first item is not clearly related to the Courier’s main theme – water
Put an introductory mention like “Read more (about this theme)” instead of “Features”
The UNESCO Courier layout needs improvement2
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
59Results Details – Table of contents
A. Pre-test questionnaire – Users’ profiles and web habits
B. Browsing observation – Perception and understanding of the home page
C. Browsing observation – Perception and understanding of the “Worldwide” section
D. Browsing observation – Perception and understanding of the UNESCO Courier
E. Browsing observation – Transversal issues
F. Post-test questionnaire – General opinions
G. Post-test questionnaire – Performance and satisfaction rating
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
60B. Transversal issues
The standardization work done so far and to be continued was appreciated for adding consistency to the portal and bringing better ease of use to the user experience: layout and navigational principles
The Framework was appreciated
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
61B. Transversal issues
Many users reported difficulties finding what they want (mostly documents):
• The website was often reported as requiring some learning – the content is seen as huge and the structure as complex
• Searches by keywords and themes were reported to give poor results
• The search engines don’t give acceptable results often enough – some participants use Google to find documents in the UNESCO portal
• A French-speaking participant (Algeria) deplored the fact that the advanced search feature is only available in English
Search issues3
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
62F. Perception and understanding of other main sections
The “Executive Board” search engine suggests that only searches by document codes can be run here – some users weren’t aware that a search by keywords was also possible here
A few users wished they could run searches by themes
Document-finding difficulties3
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
63B. Transversal issues
Some information on the website is outdated – especially in the “Worldwide” and “Themes” sections
Some documents are uploaded and available on the website too late vis-à-vis UNESCO users’ needs during the Executive Board – these late updates add some difficulties in finding the desired information or documents
“The information is brought out to late” (Algeria)
“The meetings’ outcomes are not published right after the meeting is over” (Portugal)
Late updates3
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
64B. Transversal issues
The lack of consistency across the portal (different layouts) restrains learnability and understanding from one section to another
Lack of consistency across the portal3
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
65B. Transversal issues
Page titles and layout fail to provide efficient guidance due to a lack of a logical and consistent principles in building titles:
The page title building principle should be the same through the whole portal
The page title building principle should always refer to the relevant menu building principle – i.e. the items’ organisation and order
The title of a page should always refer to the link that brought to the page and vice versa
Lack of consistent and efficient guidance principles3
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
66B. Transversal issues
On the first page shown on the left the subtitle doesn’t refer to any item in the menu on the right
On the second page, “About UNESCO” should remain visible as the page title while “The Organisation’s History” should be the subtitle
The titles and subtitles should be the same format – titles either bigger or smaller than the subtitles – through the whole portal
Lack of consistent and efficient guidance principles3
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
67B. Transversal issues
The last page of the breadcrumbs does not always refers to the last clicked link and the current page title
The last page of the breadcrumbs should always match the last clicked link and the current page title
Therefore the last page of the breadcrumbs should not be clickable
Lack of consistent and efficient guidance principles3
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
68B. Transversal issues
The breadcrumbs are usually located on top of the central area rather than above the left menu
Moreover, the breadcrumbs format (font type and size, background colour) is the same as the menu below, which reduces its visibility
Breadcrumbs should start reading above the central area in order to offer the best visibility possible
The background should be white
Lack of consistent and efficient guidance principles3
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
69B. Transversal issues
Some UNESCO sites do not permit going back to the portal home page
• e.g. the World Heritage web site has the same UNESCO logo as the portal although this one leads to the World Heritage home page
• The UIS web site offers no link to the portal
Lack of consistent and efficient guidance principles3
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
70B. Transversal issues
The “Organisation” menu changes (content, format, location) from one page or section to another, which is confusing
Lack of consistent and efficient guidance principles3
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
71B. Transversal issues
The home page links – top tab, breadcrumbs and top left logo – were not well perceived
Most participants used the browser’s “Back” button
Home page links not well perceived3
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
72B. Transversal issues
A few users deplored the lack of explanations about the acronyms used on the website
Acronyms are not clear2
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
73B. Transversal issues
A few users wished the themes landing pages would provide users with a baseline mentioning UNESCO’s key goals and programmes in the relevant field
Expectations – Themes landing page
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
74B. Transversal issues
Some users wished the website offered:
• Yearly and monthly meetings schedules
• Daily meetings summary
Expectations – Some extra content
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
75Results Details – Table of contents
A. Pre-test questionnaire – Users’ profiles and web habits
B. Browsing observation – Perception and understanding of the home page
C. Browsing observation – Perception and understanding of the “Worldwide” section
D. Browsing observation – Perception and understanding of the UNESCO Courier
E. Browsing observation – Transversal issues
F. Post-test questionnaire – General opinions
G. Post-test questionnaire – Performance and satisfaction rating
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
76G. General opinions
The main strong points reported by the participants were:
• A great amount of useful information available
• Numerous accesses on the home page
• The standardization process
• Country-specific information
• 6 languages
Strong points
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
77G. General opinions
The main weak points were:
• Overall complexity, especially for outsiders or newcomers
• Problems to find certain information and documents
• Outdated contents
• Heavy home page
Weak points
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
78H. Post-test questionnaire – Performance and satisfaction rating
Performance indicators: 5 criteria rated from 1 to 5
• Attractiveness / Website utility
• Browsing/Locating throughout the site (guidance)
• Efficiency / Ease of access to information
• Content structure (Logic in content organization)
• Understanding / Learnability
Criteria assessed through a questionnaire (inspired from the Wammi questionnaire): participants are asked to tell whether they agree or not (from 0 – not at all – to 5 – totally) with 20 statements (4 statements per criterion)
• E.g.: “This website has much that is of interest to me”
Fully disagree • • • • • Fully agree
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
79G. General opinions
Utility has a very good score (4.2) – it might refer to the rich content (compared to 3.5 for the first study)
Browsing and Understanding/Learnability could be better (resp. 3.3 and 3.2) – it might refer to the difficulties in finding certain contents (compared to 2.6 for the first study)
Efficiency/Ease of access has a good score (3.8) – maybe due to the expertise of most of the participants (compared to 3.3 for the first study)
Content structure has also a good score (3.6) (compared to 2.9 for the first study)
Scores for the performance indicators were pretty good
Performance indicators
1,0
1,5
2,0
2,5
3,0
3,5
4,0
4,5
5,0 Attractiveness/ Websiteutility
Browsing/locatingthroughout the site
Efficiency/Ease ofaccess
Content structure
Understanding/Learnability
19/04/23 Axance User Experience Design 5, rue Régnault 93500 Pantin France
81Main recommendations
The standardization work must be continued to offer a better overall user experience
• Strong and efficient guidance principles should be applied across the portal
The portal targets need to be clearly defined and the website be adapted to them: is it a portal for everyone or a working tool for the insiders?
• The right column offers too many links, specially for the general public
The home page needs more clarity, especially the layout of the central area
The work done to offer more multilingual contents should also be continued as it was very appreciated
The updates need to be done more regularly – especially for a working tool
The “Worldwide” section should offer more contents and be updated more regularly
The search engines need improvement
Recommended