An iPad usability study constructed during my HCI course at the University of Edinburgh.
HCI Report s1055940 Sebastian Borggrewe
Table of C ontent ABSTRACT INTRODUCTION THE IPAD RESEARCH PROPOSAL MOTIVATION RECENT RESEARCH WORKING PLAN HYPOTHESIS PLANNING EQUIPMENT USER TEST USER TASKS TASK 1: CNN.COM TASK 2: AMAZON.CO .UK TASK 3: FACEBOOK .COM SETUP RESULTS U SER DEMOGRAPHY AND PRECONDITIONS TIME CONSUMPTION OF THE TASKS ERROR TYPES AND FREQUENCY ADVICE FOR WEBSITE DESIGNERS RESTRICTIONS TO MY RESULTS FUTURE RESEARCH 2 3 3 3 3 3 4 4 5 5 6 6 6 6 7 7 8 8 8 9 11 11 12
HCI Report s1055940 Sebastian Borggrewe
AbstractThis report deals with the efficiency of the iPad as a web-browsing device and points out the main reasons why the iPad is slower than conventional computing devices (desktop/notebook). Since Apple claims that the iPad is the best way to experience the web [..] i I was highly motivated to find out whether it is also the most efficient way to browse the web. By constructing a user experiment for two iPad users and a control group of two notebook users containing three common websites (cnn.com, amazon.co.uk and facebook.com), I found out that the iPad is only significantly slower if heavy keyboard is involved in the interaction. However there was another error that can actually be avoided: read-tap-asymmetry1 . The results were taken to create a first checklist, that I advise all website designer to follow. These proposed improvements include especially reducing keyboard input for iPad-optimized sites. I.e. search suggestions and increasing font and link size, as well as spacing between links.
Users are able to read a link, but fail to tap it accurately.
HCI Report s1055940 Sebastian Borggrewe
IntroductionThe iPadApple introduced the iPad on 03/04/2010 and over three million have been sold to date, which makes it relevant in the mobile computing market. It is designed to be a netbook replacement in tablet form, which is almost 100% controllable via touch screen. Apple claims that the iPad is the best way to experience the web [..] ii and a recent studyiii indeed shows that already 38% of iPad users prefer using their iPad for browsing the web over their notebook and mobile phone.
Research ProposalAlthough the iPad has quite a high resolution for a mobile device, it is still limited due to the lack of keyboard and mouse, in addition to other restrictions that mobile devices in general suffer from when it comes down to the web. These are, for instance, the smaller screen and a potentially slower Internet connection. In addition the iPad is the pioneer in a rejuvenated computing category: the tablet. Therefore it is eligible to ask whether current websites are usable with the iPad. Consequently, my main idea is an investigation of the way users browse the Internet with this new device and what consequences this has for website design regarding their usability.
MotivationSince I am highly interested in both mobile and web technologies and work with both technologies quite intensely, I find the combination of the two even more appealing and exciting. Moreover I am really keen to research whether current websites are suitable to be browsed by the iPad.
Recent ResearchThe iPad is a fairly new product and therefore there have been only a few articles and research papers published until now. However the iPad is, as stated above, still a mobile device and therefore general rules for the usage of the mobile web can be partly applied as well. The Norman Nielsen Group did one of the first researches on Apples iPad. In their 103 page report, called Usability of iPad Apps and Websitesiv, they give an overview of usability issues that occur along apps and websites, when used on the iPad and conclude recommendations for iPad specific app and website design. For example: Bigger font-size to reduce the frequency of zooming and bigger links to counteract the read-tap-asymmetry. This article is especially relevant, because it addresses mobile web usability issues directly related to the iPad. As far as mobile web in general is concerned, I found a couple of useful papers, which help understanding of its usage and what has to be done, to improve the concept. In 2008, a US study on why people access the mobile web was published in A Framework for Understanding Mobile Internet Motivations and Behaviourv. This study tries to explain the reasons people have for using the mobile web and how they use it. This study is therefore able to give an overview of when people access certain services on the web and why. These findings help designing mobile websites in the way they are actually used and can therefore improve user experience on mobile platforms in general, including the iPad. The main findings the author
HCI Report s1055940 Sebastian Borggrewemakes are that users motivation and behaviours can be grouped into patterns i.e. Awareness (stay up to date) and Social Connection (socializing online). Understanding these patterns is especially helpful, when designing a suitable user test. Moreover, I took the article Web Content Adaptation for Mobile Handheld Devicesvi into consideration, which was published in 2007 by Dongsong Zhang. The author proposes multiple approaches to adapt web content for mobile devices. Although this article is mainly aimed at handheld devices, the parallels you can draw from handhelds to the iPad cannot be denied. This article is particularly interesting because it combines the ideas of mobile web being differently used from desktop web and the consequences for design that derive from this behaviour. The author suggests three different approaches that can be combined to adapt web content for mobile devices: Selective Content Delivery The user doesnt need all the information at once, Adaptive Content Presentation - the website has to be design differently for mobile devices and finally Adaptation of Multimedia Content Mobile devices are bound to different formats, filesize in order to show multimedia content (the iPad doesnt support flash).
Working PlanIn my report I want to evaluate users interaction with websites on the iPad. In order to do so, I will keep track of the time users take and the errors they make, when browsing the Internet using an iPad.
HypothesisMy underlying hypothesis claims that a novice user, after using the iPad for one day for the purpose of web browsing, is not able to navigate to a specific information on an, not specifically iPad-optimized, website as fast as a notebook user and produces more errors due to faulty interaction with the website (for example: clicking the wrong link due to its small size). The results shall help to give advice on how to design usable iPad-optimized websites.
MethodologyIn order to determine the actual usability of websites using the iPad as a web-browsing device, I want to setup a qualitative study containing 4 users. Two of them will use the iPad for browsing the web; the other two are my control group that will use a notebook instead (In case the website suffers from some general usability issues). Each of them has to fulfil three information retrieval tasks that will be identical for each user. The three tasks will differ from each other in the way the user has to interact with the website. I will use a news website: cnn.com, where the user only needs to navigate through the website, an Internet shop: amazon.co.uk and a social network: facebook.com, where the user needs to type and point with either finger or pointing device. All actions of the user will be recorded while fulfilling the tasks, using a screen recording software, and they will be asked to comment on their actions after the experiment. During the experiment I will observe the user, paying attention to two criteria: First of all the time it takes the user to fulfil the task and secondly, the types of errors he/she makes while fulfilling his/her task.
HCI Report s1055940 Sebastian BorggreweThere will be a clear focus on the questions: Which part of the task took longer on the iPad (for example, typing in the search term) and why? What kind of errors where made and what are the reasons for those errors according to a) the user and b) my observations.
PlanningThe development of the task the users will need to fulfill will take about one hours to complete. I will need an additional 30 minutes to setup the experiment. I plan to spend a maximum of 45 minutes with each user to execute the experiment described above. The whole review and evaluation process will take approximately 6 hours. After the experiment I plan to spend another two hours aggregating the results to a reasonable number of figures and tables for this will make the report more graphic and readable. Six hours will be used for actually writing the report. Since I have another 1,5 hours left, I will have a reasonable amount of backup time in case one or two parts of my plan take longer, than I calculated it to be. I want to add, that the duration of every task was already generously calculated with a reasonable amount of backup time. This way I ensure the plan to be carried out in utmost 20 hours (cf. Table 1: Time calculation). Task Task development Experiment setup User test Evaluation Result Aggregation Writing the report Backup time TotalTable 1: Time calculation
Time 1h 0.5h 3h 6h 2h 6h 1.5h 20h
This plan will be executed according to the following timetable: Time window 08/10/2010 14/10/2010 15/10/2010 22/10/2010 Action One day time for each of the two iPad users to become familiar with the device. User experiments. I calculate a whole week, since some users might be busy on certain dates. Evaluation Result Aggregation Report Writing + use of backup time, if necessary.
24/10/2010 25/10/2010 27/10/2010 28/10/2010Table 2: Timetable
EquipmentIn order to execute my plan I need the following equipment: iPad and Notebook
HCI Report s1055940 Sebastian Borggrewe Screencast Software (notebook) / Webcam (iPad) and Microphone A table and two chairs I have already organized the equipment that I need and therefore it wont consume any time to get a hold on it.
User TestUser TasksThe User tasks were designed to cover both pointing and keyboard input. In order to simulate everyday life tasks, I chose three websites that cover a representative cross section of human interaction with the Internet: News, Shopping and Social Networking. In the following I will describe the three tasks. Task 1: cnn.com The User is supposed to navigate to the Top Story in Europe and get to the comment section of the story. This task will involve using the navigation bar, in-text links and scrolling.
Task 2: amazon.co.uk The user has to add the book Senior Nice: Straight Life from Wales to South Africa to his/her shopping basket, by searching for the Author Howard Marks. Afterwards the user shall proceed to the checkout, without logging in and without actually ordering it. This task will involve using the website search (keyboard input), in-text links /buttons and scrolling.
HCI Report s1055940 Sebastian BorggreweTask 3: facebook.com The user is supposed to login to facebook with their own credentials and send me Sebastian Borggrewe a message with the Subject: Done and a message body that says: I am done with this experiment! They are already friends with me on facebook, so the task will be about the same for everyone (except from the length of their credentials). This task will involve a rather strong use of the key input for login and message composing, as well as the use of various buttons integrated in the facebook interface.
SetupAs already suggested in my Working Plan, the usability setup consists of a table and two chairs. One for the tester, one for me. This setup will basically look the same for iPad and Notebook. The device itself determines the only difference. The device will be given to the user in order to be placed on the table or lap or how he/she would use this device at home. The screen recording software will record the screen and the microphone remarks users make, while fulfilling their tasks.
Figure 1: Test setup. Internal Notebook camera is used for recording the iPad screen
HCI Report s1055940 Sebastian Borggrewe
ResultsUser demography and preconditions The four users that participated in the test are between 22 and 25. Three of them have a high level of technical experienced (four on a scale from one to four) and one of them is less experienced (two on a scale from one to four). All of the users were male and the two iPad users had iPhone experience for at least one year. They were given the iPad for one day to explore its browsing capabilities. Time consumption of the tasks As suggested in my hypothesis iPad users were in fact slower fulfilling the task, than notebook users. However the results are still surprising (cf. Figure 2). If no, or little, keyboard interaction was involved, the iPad users performed nearly as well as the notebook users. The average time differences in the first two tasks are between 0.65 and 1.65 seconds. As soon as heavy keyboard input is involved in solving the task, the average difference rises up to over 40 seconds. This already gives a first hint as to which kind of interaction takes significantly longer on the iPad: Typing.90 80 70 60 50 42.15 40 40.5 30 cnn.com Figure 2: user test - time
iPad 49.65 49 40.95 Notebook
In fact the results show that keyboard input is the main reason why iPad users are slower in task three (cf.
Text input purpose Login Search for User Write Message Total
iPad 18.2s 1.2s 36.2s 56.6s
Notebook Difference 7.3s 1.1s 9s 17.4s 10.9s 0.1s 27.2s 38.2s
Table 3: time differences due to text input
HCI Report s1055940 Sebastian BorggreweTask 3: facebook.com). Table 3 shows the keyboard input times for task three, without accounting for the delay times due to keyboard appearance (iPad) or switch between touch pad and keyboard (notebook). As you can see the difference that derives from the mere keyboard input is already 38.2s, which reduces the difference deriving from clicking operations down to 8.55s. Again the browsing speed is not significantly slower than on a notebook. Another thing that is quite striking is that the Search for User time difference is hardly noticeable. In fact all four users didnt type more than the two letters Se. Apparently little text input is not an issue on the iPad and during the user experiment the two iPad users didnt have a problem typing little text (approx. text input