Sensemaking on the Web: How Users ‘Get it’

ABSTRACT The volume and complexity of information available on the web has grown enormously, confronting users with a large amount of heterogeneous and often unstructured content . When users encounter an unfamiliar web site for the first time, they are facing the most basic task of all: making sense of what they see. However, relatively little is known about this process and the web elements that are significant for sensemaking. We describe an exploratory study in which we examined the process of sensemaking on the Web and some of the factors that affect it. The study was a controlled factorial experiment with 13 participants, which investigated how people made sense across two web sites and compared a sensemaking task with a straightforward search task. Eye tracking technology was used to study participants’ viewing behaviour through a range of quantitative measures, notably the number of fixations on specific elements and the time to first fixation. In addition, qualitative data from post-task interviews were collected to support the analysis. The study revealed the importance of descriptive text and other web elements for sensemaking but also highlighted the fact that people’s responses did not always align with their actual viewing behaviour. The study provides greater understanding of users’ viewing behaviour on the web and contributes to the overall debate on sensemaking as an emerging field in HCI.

Categories and Subject Descriptors H.5.3 [Group and Organization Interfaces]: Web-based interaction.

General Terms Human Factors

Keywords Sensemaking, eye tracking, viewing patterns, information visualisation

1. INTRODUCTION Understanding how users make sense of information on the

web has immense practical implications for both commercial and academic endeavours. The volume and complexity of information available on the web has grown enormously , confronting users with a large amount of heterogeneous and often unstructured content. When users encounter an unfamiliar web site for the first time, they are facing the most basic task of all: making sense of what they see. Yet there is evidence that most users rapidly make a decision as to whether a site satisfies their needs. According to PARC Research “How information is organized, presented, integrated and controlled directly affects how easily and thoroughly users will analyse and understand it – especially in mission-critical and time-pressured situations” [10]. The burgeoning area of research known as Sensemaking is trying to address these questions and is attracting increasing interest from HCI and other communities. Since 2005, the CHI conferences have offered workshops on the subject , exploring how people make sense of complex sets of information, how visual representations can support the sensemaking of individuals or groups, how sensemaking across groups can be coordinated and facilitated, and how sensemaking fits into other knowledge work such as information gathering and decision making. However, to date there has been relatively research into sensemaking on the web. How do people look at web pages when they are trying to make sense of them? Is there a common pattern during this process we call sensemaking, and if so, how is this pattern different from that resulting from a more focused task such as a search task with a defined ending and clear measure of success? Which clues do people believe are important to their sensemaking? In this paper we describe an empirical study to investigate how users make sense on the web. The study used eye tracking with detailed analyses of the eye tracking data and qualitative interview methods to help us understand the common viewing patterns in sensemaking and the key clues that users rely on in the process.

2. RELATED WORK 2.1 Sensemaking Definitions of sensemaking vary depending on the context in which the term appears (e.g., Organisational Research, Information Management, Library Science, and Decision Science). Sensemaking has been discussed in context of collaborative working practices and social computing [13], [17], but also studied as an individual activity [4], [16]). It is the latter aspect of sensemaking that is explored in this study.

Furnas and Russell [5] describe sensemaking as a “core professional task in itself, […] arising when we change our place in our world, or when the world changes around us”. To him sensemaking is a function of insufficient knowledge of a subject or a situation, and “requires users finding pattern or structure in a seemingly unstructured situation”. Weick [18] has explored organisational sensemaking suggesting that the (social) process of sensemaking “...is less about discovery and more about invention.. .to construct, filter, frame...and render the subjective into something more tangible”. Yi et al [19] put this simply, saying that when sensemaking “we construct interpretations of the information we find”. This creative aspect of sensemaking also shows when we explore data looking for its inherent structure. Russell et al [15] define sensemaking as cyclic process “of encoding retrieved information to answer task specific questions”, and calls this the Learning Loop Complex. Like many cognitive tasks in the real world, sensemaking is a top down and bottom up process: it involves deducting from knowledge structures we have learned previously (internal representations, top down) and inductively interpreting facts we gather from the world around us (external representations, bottom up), as well as borrowing from the sensemaking structure of others [14]. In sensemaking, internal and external representations form an integrated cognitive system [7]. The design of the representation is vital to the sensemaking activities. The form of representations can profoundly affect the effectiveness of the search for information and the cost and gain of the task operations [15]. The external representations we encounter during sensemaking can put more or less heavy cognitive demands on us [15]. Pirolli and Card at the Palo Alto Research Center (PARC) extensively study sensemaking in order to understand the cognitive and perceptual aspects of information work (cognitive task analysis), how people perceive, organise and use information. PARC’s studies range from eye tracking to fieldwork, and include micro and macro structures of human-computer interaction. Recent work has started to look at sensemaking on the web. This has been described as involving multiple tasks undertaken by a single individual when challenged with an unfamiliar situation, in this case, a yet unknown web page [10]. These tasks include: the searching, filtering and extracting of content on a page relevant to the understanding, rapidly reading and absorbing meaning. This involves identifying relationships between page elements (‘connecting the dots’) and detecting patterns, in order to ultimately form a mental model of the site as a whole. Building on this work, a key question for us is which page elements do people use in practice when making sense of web pages?

2.2 Eye tracking The use of eye tracking in usability research is a fairly new phenomenon. Since the Poynter Institute released their first study in 2000 investigating how users view online news websites, a number of studies have reported eye tracking as a useful tool for the usability evaluation of user interfaces of websites ([1], [2], [6], [8], [16]). These studies have shown that eye tracking can offer supplementary, objective and specific data regarding a user’s visual attention and

perception, something that cannot be measured by means of think-aloud protocols, interviews or questionnaires . Russell’s study [16] on a range of e-commerce websites found that eye-movement data supplements what users verbally report when they view a website for the first time. Going beyond a typical usability evaluation, which would examine whether the design effectively communicates the purpose of the site, he was able to employ eye tracking to find out where on the interface users were looking to make sense of the site , thereby bridging the two so far separate areas of usability evaluation and sensemaking.

Several high profile studies have used eye tracking to investigate how users view web pages. For example, Nielsen [9] claimed that users read web pages in an ‘F’ pattern, while a study of the web search behaviour of Google users [3] reported a ‘Golden Triangle’ at the top of search results pages. These studies concentrated on reading and scanning behaviour, looking at the overall pattern of eye movements as visualized in heatmaps. In contrast, our interest is in sensemaking and in the process of sensemaking. In this context, it is not sufficient to consider overall patterns of eye movements . Rather, we are interested in finding out what elements people attend to (and ignore) when sensemaking, irrespective of where these elements occur on a given web page.

3. SENSEMAKING STUDY 3.1 Overview The aim of this exploratory study was to investigate how users make sense of web sites and what they attend to during the sensemaking process. To this end, we undertook an experimental study which compared a sensemaking task with a search task so that we could determine not just what happens during sensemaking but also how this might differ from other tasks.

The study was a controlled within-subjects experiment with 13 participants. All participants were asked to undertake both sensemaking and search tasks. We collected two kinds of data for each task: firstly we used eye tracking technology to study participants’ viewing behaviour through a range of quantitative measures; secondly, we undertook a qualitative investigation where we used interview questions to collect information about which elements participants believed were relevant for their sensemaking

3.2 Participants Thirteen participants (6 female and 7 male) ranging in age from 20-50 years old, all with a similar level of web experience, took part in the study. In recruitment, we controlled for language, age, gender, cultural background, education level and computer literacy, in order to ensure that these factors did not affect the results. Participants had to be frequent web users with a good understanding of social networking sites. All participants had normal or corrected-to-normal vision. None of them had experience using an eye-tracker.

3.3 Apparatus The experiment was performed with a Tobii X50 (Tobii 2006) eye tracking system, which was connected to a Pentium based

PC (with Windows XP) running Tobii Studio analysis software version 2.11. Two 21' flat screen monitors were attached to the computer, one displaying the websites to the participant, the other showing the eye movements simultaneously to the facilitator. A webcam, two keyboards and two mice were attached to give control to both participant and facilitator.

3.4 Experimental design Two websites were selected as stimuli. Both were social networking sites, however they varied in the complexity of their layouts; we refer to the simpler design as the “basic layout” and the more complex one as the “busy layout”. Figure 1 shows the sites in miniature and summarises their key differences. We used pages within the sites as the stimuli rather than the home pages (because the home pages contained clear statements of purpose).

The type of task was the independent variable for this study and its two values, sensemaking and search, were distinguished as follows:

Sensemaking: Sensemaking is a process by which an individual creates an understanding of a web page, so that he or she can act in an informed manner.

Search: Search is the process of looking for a known item. (This is synonymous with known-item search or direct search and is based on the assumption that a user knows what they want and what words to use to describe it.)

The sensemaking task presented to participants was the same for both sites and was formulated as:

“A friend has sent you a link to the following site. Spend a little time making yourself familiar with the site and I'll tell you when to stop”.

In contrast, it was necessary to formulate the search task differently for each of the two sites, due to the different domains. The search tasks were:

“Find an article about carrots” (Basic layout)

“Find a plumber in London W1, who is rated with 5 stars on the site” (Busy layout)

Each participant was asked to carry out two tasks on each website: the sensemaking task and the appropriate search task. The ordering of tasks was varied between participants to counterbalance any order effects, such as fatigue or learning.

Multiple dependent variables were measured, in order to obtain rich information about participants’ behaviour. The key dependent variables were all eye tracking metrics: Number of Fixations (on the display as a whole), Number of Fixations on each Area of Interest (AOI) and Time to First Fixation (TTFF). In addition, heatmaps were generated from this data to provide an overview of viewing patterns for each of the two experimental conditions and the qualitative interview data was analysed to reveal participants’ perspectives on their own sensemaking.

Basic layout: www.mytinyplot.com Busy layout: Problemsolved.co.uk

• Article page titled “10 Jobs for March” • 2 column layout, built to 800x600px screen resolution • Top bar is simple, consisting of small logo and main

navigation • Wide central column with main content , i.e. large image

and body of text; narrow right hand column with smaller modules of related information

• Body text follows typical structure for blogs including form field for visitor comments

• Basic in content and functionality • Low density

• Detailed page for specific company • 3 column layout, built to 1024x768px screen resolution • Top bar is complex with prominent logo, main navigation,

search boxes, additional links and banner ad • Wide central column with main content; narrow columns on

either side containing related information. • Feature rich • Higher density - large number of different page elements in

close proximity

Figure 1. The two test sites

3.5 Procedure Each participant session started with the facilitator providing information about the study and obtaining the participant’s consent. A short questionnaire was administered to obtain demographic information and data regarding the participant’s use of and experience with the web. This was followed by a brief introduction to the eye tracker and eye tracker calibration.

Participants were then asked to undertake the 4 tasks: sensemaking and search on the basic layout, sensemaking and search on the busy layout, with the order of tasks counterbalanced. Participants were given approximately 30 seconds to work on each task at their own pace. Participants were eye tracked while undertaking these tasks and audio and video recorded.

After each task on each site, the experimenter replayed the eye tracking session with the participant and encouraged them to verbalise their sensemaking and search strategies (this data is not presented here). A semi-structured interview was also conducted after each task designed to obtain qualitative data about users’ sensemaking and search strategies. Participants’ responses to the interview questions were collected in real time through audio recording and manual logging by the facilitator.

Six interview questions were asked after each sensemaking task in order to determine how successful participants had been in their sensemaking (because this was not observable) and their perceptions of the page elements that contributed to their sensemaking:

1) What do you think this site is about?

2) How did you work that out?

3) Was there any element in particular that made you click, i.e. get it?

4) What did you think of the site’s layout?

5) What was the structure of the site – how did you work out the site’s structure?

6) How would you do [activity x] on this site?

Success in the search task was observable (if a participant found one of the required results and accurately identified it as such) and therefore just two interview questions were asked after each search task:

1) What did you do when you searched?

2) Was there any element in particular that helped / hindered your searching?

4. RESULTS Eye tracking yields a large amount of very detailed quantitative data regarding where a user is looking on a display (the fixations) and the movement of the eyes inbetween these relatively fixed points (the saccades). Participants took varying lengths of time to complete the tasks, hence we selected the first 25 seconds of data from each task to use in the analysis. We explored several complementary eye tracking measures derived from the raw fixation and saccade data in order to understand how users go about sensemaking and how sensemaking differs from direct search. In our analysis, we compare these eye tracking measures for the sensemaking and search tasks on the basic layout and, separately, for the sensemaking and search tasks on the busy layout, but we do not compare across layouts. Factors such as different placement and design of ostensibly similar items (e.g. the main navigation) on the two sites introduced additional variables that would have reduced the validity of any such comparison.

4.1 Number of Fixations The first measure we investigated was the total number of fixations made by each participant during each task i.e. fixations that occurred anywhere on the web page. The total number of fixations and the mean number of fixations across the 13 participants are given in Table 1, organized according to task and layout.

Paired 2-tailed t-tests were conducted for each layout. These revealed a significant difference between the number of fixations in the sensemaking and search tasks for both layouts at the 0.05 level (p<0.05); the busy layout was also significant at the 0.01 level (p<0.01). This significantly greater number of fixations in sensemaking as compared to search supports the commonsense view that sensemaking is a more exploratory, and therefore naturally less ‘efficient’, activity – not unlike browsing – whereas (known-item) search is a much more directed activity. The study also suggests that efficiency in sensemaking could be aggravated by a complex layout. The number and complexity of page elements and their arrangement in the layout negatively affected people’s sensemaking efforts . This fact relates to studies by Goldberg & Kotval who claimed that ‘large numbers of fixations indicated less efficient search perhaps resulting from poor visual design.’

Table 1. Number of fixations on specific layout by task

Number of Fixations on Display Basic layout

Sensemaking Search

Sum 723 463

Mean 55.615 35.615

Number of Fixations on Display Busy layout

Sensemaking Search

Sum 873 620

Mean 67.154 47.692

4.2 Number of Fixations on AOIs Secondly, we investigated the number of fixations that participants made on specific page elements. Each page was divided into Areas of Interest (AOIs), where an AOI corresponded to a page element of interest. In doing this, we sought to identify specific page elements that were common to the two layouts, thereby to facilitate comparison. For example, both sites contained a logo (although obviously appearing in a different location and with a different design); and hence we defined a logo AOI. We defined a total of 10 AOIs that appeared in both sites (Table 2). Having divided the layouts into their constituent AOIs, it was then possible to determine how many fixations occurred in each AOI.

Table 3 presents the number of fixations on AOIs for sensemaking in the basic layout and Table 4 presents the number of fixations on AOIs for search in the basic layout. Figure 2 provides a visual comparison. In the basic layout, there was a significant difference at the 0.05 level (p<0.05) between the two tasks for the number of fixations on the following AOIs: COM, DESC, MENU and TITLE and at the 0.01 level (p<0.01) for COM.

Table 2: AOIs defined for both layouts AOI Name Description

LOG Logo

MENU Main navigation menu

SEA Search field

TITLE Page title

IMG Main image

DESC Description, main body of text

COM Visitor feedback, blog comments

REL Related links, tags, archives

AD Ad panel, Amazon ad

LATE Latest updates

BUT Action buttons (busy layout only)

Table 5 presents the number of fixations on AOIs for sensemaking in the busy layout and Table 6 presents the number of fixations on AOIs for search in the busy layout. Figure 3 provides a visual comparison. In the busy layout, there was a significant difference at the 0.05 (p<0.05) level between the two tasks for the number of fixations on the following AOIs: LOG, DESC, COM, MENU, TITLE. There was also a significant difference for TITLE at the 0.01 level.

These results indicate that the main body of text (the DESC AOI) played a vital role in people’s sensemaking activities . The DESC AOI received a remarkably high fixation count in each of the tested layouts pointing to the fundamental importance of this element for people’s sensemaking, irrespective of a specific layout situation. This is not to say that people are reading all the description text; indeed other studies would suggest that people may be scanning rather than reading.

The second most important element for making sense of the basic layout appeared to be the list of blog comments below the main article on the page, followed by the navigation menu and the latest updates (a Twitter module located in the top right hand corner below the search field). In the busy layout the study found a relatively high Number of Fixations on the logo and the page title. This suggests that participants – when confronted with a more complex layout and elements within – were possibly seeking orientation and additional clues for their sensemaking from these two elements.

In addition the physical size of these two elements was substantially larger in the busy layout. The busy layout had been designed to a larger screen width and page elements were given more space. The logo was double in size in the busy layout, the page title was around three times are big as the equivalent title in the basic layout. Also, the design of the elements was more complex, in structure and colour and might have taken participants a few attempts to understand.

Table 3: Number of Fixations (Total and Mean) in basic layout by AOI during sensemaking task

Number of Fixations – Basic layout – Sensemaking

LOGO MENU SEA TITLE IMG DESC COM REL AD LATE

Sum 37 81 11 47 52 193 145 32 33 64

Mean 2.847 6.231 0.847 3.616 4 14.847 11.154 2.462 2.539 4.924

Table 4. Number of Fixations (Total and Mean) in basic layout by AOI during search task

Figure 2. Comparison of Number Of Fixations by AOI for both tasks in basic layout

Table 5: Number of Fixations (Total and Mean) in busy layout by AOI during sensemaking task

Number of Fixations – Busy layout – Sensemaking

LOGO MENU SEA TITLE IMG DESC COM REL AD LATE BUT

Sum 112 62 52 105 13 173 62 47 56 29 18

Mean 8.616 4.77 4 8.077 1 13.308 4.77 3.616 4.308 2.231 1.385

Table 6. Number of Fixations (Total and Mean) in busy layout by AOI during search task

Number of Fixations – Basic layout – Search

LOGO MENU SEA TITLE IMG DESC COM REL AD LATE

Sum 13 59 44 24 37 67 32 31 33 81

Mean 1 4.539 3.385 1.847 2.847 5.154 2.462 2.385 2.539 6.231

Number of Fixations – Busy layout – Search

LOGO MENU SEA TITLE IMG DESC COM REL AD LATE BUT

Sum 50 21 153 44 9 91 15 37 41 39 26

Mean 3.847 1.616 11.77 3.385 0.693 7 1.154 2.847 3.154 3 2

Figure 3. Comparison of Number Of Fixations by AOI for both tasks in busy layout

4.3 Time to First Fixation (TTFF)It was important to determine not just which page elements participants had looked at during the sensemaking and search tasks but also the order in which they had done so. To do this , we examined a measure known as Time to First Fixation (TTFF): this was the elapsed time between a participant starting the task and first fixating on the specific AOI.

Table 7 and Figure 4 show the Mean Time to First Fixation on each AOI across all participants on the basic layout for both the sensemaking and search tasks. Table 8 shows the same for the busy layout. Figures 4 and 5 make visual comparisons between the means. Many of the participants never fixated on some of the AOIs, e.g. only one AOI (DESCR) was fixated upon by all 13 participants during the search task on the basic layoutand only 1 AOI (IMG) was fixated upon by all participants during sensemaking on the same layout. These gaps in the data meant that it was not feasible to undertake a pairwise comparison of the TTFF data for the sensemaking and search tasks.

However, we can still get a broad overview of what was happening from looking at the means. In the sensemaking process, the logo and page title stand out as equally critical across layouts, in the sense that participants looked at them early in the process. Other elements that were attended to early in the sensemaking task were the menu, the image and the latest updates (for the basic layout) and the description, the search field and the ad banner (for the busy layout). Participants’ attention might have been caught by an effective design or placement of these elements. Or they might have supported people’s existing sensemaking strategies.

The study also found other common patterns in sensemaking for both layouts. For example, any elements that were (physically) less central to layout (such as those placed below the fold) were looked at relatively late in the process and weren’t available immediately for sensemaking. In the busy layout, however, there was a relatively large number of AOIs with similar TTFFs between tasks indicating a less strong effect of task on the busy layout compared to the effect on the basic layout.

Table 7: TTFFs (mean) on basic layout by AOI for specific task

Table 8: TTFFs (mean) on busy layout by AOI for specific task

TTFF (Mean) – Basic layout

LOGO MENU SEA TITLE IMG DESC COM REL AD LATE

Sense- making 1.046 1.001 6.651 4.163 1.113 5.419 13.356 14.331 7.396 3.293

Search 5.917 3.079 5.454 3.368 1.844 3.247 13.138 8.981 3.497 1.765

TTFF (Mean) – Busy layout

LOGO MENU SEA TITLE IMG DESC COM REL AD LATE BUT

Sense- making 1.387 5.314 4.306 3.384 8.115 2.614 16.242 8.57 5.23 11.062 8.595

Search 5.081 2.549 3.402 5.01 8.067 4.018 11.603 9.398 9.597 12.222 11.142

Figure 4: TTFFs (mean) on basic layout by AOI for specific task

Figure 5: TTFFs (mean) on busy layout by AOI for specific task

Table 9. AOIs in order of attention in sensemaking, ranked using mean TTFFs (1 – looked at first, 10 looked at last)

1 2 3 4 5 6 7 8 9 10 11

Basic layout MENU LOGO IMG LATE TITLE DESC SEA AD COM REL –

Busy layout LOGO DESC TITLE SEA AD MENU REL BUT LATE IMG COM

In order to gain some insight into the sensemaking process, we wanted to get a feel for the viewing order – the sequence in which participants looked at AOIs. Table 9 shows the viewing order by ordering the AOIs according to their mean TTFFs for the sensemaking task on each layout. Figure 6 visualises the viewing order by means of a new measure which we have termed the ‘collective scanpath’. The collective scanpath sets out to establish a collective viewing order (across predefined AOIs) by a group of people. The measure is based on the average time it takesa group of participants to fixate on a specific AOI from entering the web page. The ‘collective scanpath’ was created by

• Ordering all relevant AOIs in order of their collective (mean) TTFF (from shortest to longest TTFF) • Assigning a number from 1 to 10 (11 for busy layout) to the AOI depending on their position in the order – 1 indicating the element that was looked at first, and 10 the element that was looked at last • Visualising the resulting order by means of a diagram (based on the original layout), with a line drawn between consecutive AOIs The emerging ‘collective scanpath’ can then be described with regards to its complexity, density, degree of extension, number of crossings (e.g. of certain areas in a layout) for example.

Looking across the tasks, the page title was the single most important element which people referred to both in sensemaking and search: if we look at the first five elements in the viewing order, it was the only common element across layouts and tasks. The logo (LOGO) and the menu (MENU) were other recurring AOIs amongst the first five in sequence. These commonalities in people’s viewing patterns – beyond the effectiveness of the elements themselves – could point to similarities in the nature of both tasks. There might be overlapping aspects (of search) inherent in sensemaking and search activities, indicating crossovers between the task strategies themselves.

Figure 6. Simplified ‘collective scanpath’ during sensemaking

4.3 Heatmap analysis We also generated heatmaps (Figure 7) from the ‘number of fixations’ data to visualize the areas of the overall web pages that were and were not attended to by participants during the tasks. All the heatmaps showed distinct areas of ‘heat’ and ‘cool’ clearly related to specific page elements. However, these visualizations are a rather blunt instrument; it is difficult to discern the significant differences in numbers of fixations that we reported earlier. However, the study suggested that sensemaking might be directly affected by the complexity of a layout. The busy layout showed a chaotic, and almost fragmented viewing pattern, while the basic layout lead to a more focussed pattern, consisting of fewer and larger hotspots . The study showed that sensemaking might be distinctly affected by the degree of complexity of a layout, as differences between tested layouts directly impacted on participant’s viewing patterns. Important areas, which people predominantly attended to in their sensemaking were the menu, the page title, the description, the logo and the latest updates (Twitter). Heatmap analysis indicated that sensemaking might be a very exploratory activity. Participants ventured into the lower part of the layout and scrolled the page for additional clues. Sensemaking behaviour extended to a wide number of elements spread across a large page estate. (In search the main task activity took place above the page fold.) Each heatmap pattern also reflected the underlying layout structure and specific page elements within this structure.

Figure 7. Heatmaps showing effects of task

Basic layout Busy layout

4.4 Interview results Overall the majority of participants had successfully made sense of the both layouts, basic and busy, and understood their subject matter, purpose and scope. According to participants , important clues in the basic layout were the pictures, main article header, and body of content and overall layout structure. No overarching single clue emerged as the most vital to all participants in this layout. In the busy layout, three clues, reviews, name of site and pictures were considered useful clues, with the reviews being named as the most vital clue here. Asked for their personal judgement on each layout, participants made more positive remarks about the basic layout than about the busy layout. People’s awareness of site sections in a layout was mainly on the main and secondary navigation menu, the about us link, and the fact that they were looking at a specific article or company page. Search strategies varied depending on the type of layout, on the complex layout, participants said they would resort to use the search box straight away, while on the basic layout they would rather explore the page for keywords first. The obvious location of the search box was considered helpful in the search in both layouts.

5. DISCUSSION We set out to investigate how people look at web pages when they are trying to make sense of them. Which elements provide clues in people’s sensemaking? Is there a common pattern during this process we call sensemaking, and if so, how does this differ from that resulting from a more focused task such as search?

The results of this study suggest that the main body of text (description) in a layout is a vital clue for people’s sensemaking. When confronted with a basic layout of a blog, users also heavily rely on the comments, in order to confirm their assumptions about the site – i.e. they rely on an element that is typical of this kind of site. They also relied (to a slightly lesser degree) on the navigation menu in order to confirm their assumptions about the site. If the layout is relatively complex, the logo and the page title seem to become additional important clues for understanding the site. The complexity of the layout affected people’s viewing order, however in both layouts, the menu, the description and the page title attracted people’s attention early, irrespective of the layout’s complexity. Despite these common clues, sensemaking is to a degree a relatively individual activity , possibly depending on factors such as personality, experience and habits. Sensemaking is also a relatively exploratory in nature and easily affected by a disorganised layout. Not surprisingly, page elements placed above the fold are most effective for sensemaking. However, the page fold is no longer considered an obstacle in viewing a website if users are looking for useful clues.

The study found many correspondences between people’s own reporting of what they had done and their viewing behaviour measured by the eye tracker. For example all participants had identified the basic layout successfully as a web log and reported that the description, page title and (blog) comments had been vital clues to their sensemaking. These reports were

backed up by analysis of numbers of fixations, which showed high fixation counts on these areas.

However, there were exceptions. The study also found instances where people’s responses did not align with their viewing behaviour. When asked for important clues that helped them make sense of each layout, seven people (in each layout) responded ‘the pictures’. Heatmap analysis however showed only very few fixations on any of the pictures across the layouts, suggesting that people predominantly had used the available text elements for their sensemaking. In much the same way, when asked for the single most vital element in their sensemaking, the majority of respondents (5) named the ‘reviews’ (in the busy layout). Interestingly heatmap patterns show very few fixations on the actual review listings, but do confirm that participants have picked up on the additional clues (action button, small ad for reviews, link in secondary navigation). This mismatch of people’s visual attention and their actual reporting indicates both a weakness and a strength of eye tracking as a stand-alone usability technique.

6. CONCLUSIONS In summary, this study has successfully identified elements of web pages that people used in their sensemaking of these particular websites and has offered insight into the viewing order of these elements. The importance of elements such as the main text description was revealed by the eye tracking results and supported by users’ own reporting. It is also apparent that sensemaking behaviour does not result in the common viewing patterns reported by Nielsen [9] or Eyetools [3], but is influenced by the structure of the visual layout. Heatmap patterns tend to be more fragmented and fuzzy for busy layouts, and relatively focused for basic layouts, indicating that users’ attention can be distracted if the layout is complex or not organized optimally to facilitate understanding.

As an exploratory study, the emphasis was on undertaking a detailed analysis of the data from a relatively small number of participants. We believe applying this approach to greater numbers of participants on a range of sites would contribute further to our understanding of how we can facilitate sensemaking on the web. The study provides greater understanding of viewing behaviour on the web and is relevant for web designers in their attempt to successfully engage and direct users’ attention through good design. It also contributes to the overall debate on sensemaking as an emerging field in HCI.

7. REFERENCES [1] Bojko, A. 2006 Using Eye Tracking to Compare Web

Page Designs: A Case Study. Journal of Usability Studies, 3, 1 (May 2006), 112-120.

[2] Ellis, S., Candrea, R., Misner, J., Craig, C.S., Lankford, C.P. and Hutshinson, T.E. 1998. Windows to the soul? What eye movements tell us about software usability . Proceedings of the UPA Conference 1998, 151−178.

[3] Eyetools Research and Reports, Google Search’s Golden Triangle, Retrieved May 27t h, 2008, from eyetools.com/research_google_eyetracking_heatmap.html

[4] Furnas, G.W. 2008 Representational Change in Sensemaking. Sensemaking Workshop, Conference on Human Factors in Computing Systems (CHI 2008).

[5] Furnas, G.W. and Russell, D.M. 2005 Making Sense of Sensemaking. In Proceedings CHI 2005, 2115-2116.

[6] Goldberg, J.H., Stimson, M.J., Lewenstein, M., Scott, N. and Wichansky, A.M. 2002 Eye tracking in web search tasks: design implications. Proceedings of the 2002 Symposium on Eye Tracking Research & Applications.

[7] Hutchins, E. 1995 Cognition in the Wild. Cambridge, MA. MIT Press.

[8] Jacob, R. and Karn, K. 2003 Eye tracking in human-computer interaction and usability research: Ready to deliver the promises. In Radach and Deubel (Eds.), The Mind’s Eye: Cognitive and Applied Aspects of Eye Movement Research. Elsevier, Amsterdam, 573-605.

[9] Nielsen, J. 2006 F-Shaped Pattern for Reading Web Content. Alertbox, April 17, 2006. Retrieved May 20t h, 2008, from www.useit.com/alertbox/reading_pattern.html

[10] PARC Research 2007 Sensemaking. Retrieved June 10th, 2008, from http://www.parc/research/projects/sensemaking/default.html

[11] PARC Intelligent Systems Laboratory 2008 Glossary of Sensemaking Terms. Retrieved August 10th, 2008, from www2.parc.com/istl/groups/hdi/sensemaking/glossary.htm

[12] Pirolli, P. 2003 Exploring and finding information. In J. Carroll, (Ed.), HCI Models, Theories and Frameworks. San Francisco, Morgan Kaufmann.

[13] Pirolli, P.L. 2008 Social information foraging and sensemaking. Sensemaking Workshop, Conference on Human Factors in Computing Systems (CHI 2008).

[14] Qu,Y. and Furnas, G.W. 2005 Sources of structure in sensemaking. Proceedings CHI 2005 Conference on Human Factors in Computing Systems, Extended Abstracts, April 02-07, 2005, Portland, OR, USA.

[15] Russell, D.M., Stefik, M.J., Pirolli, P. and Card, S.K. 1993 The cost structure of sensemaking. Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems, 269-276.

[16] Russell, M. 2005 Using Eye-Tracking Data to Understand First Impressions of a Website, Usability News 7.1.

[17] Stefik, M.J. 2008 Social indexing. Conference on Human Factors in Computing Systems (CHI 2008), Florence, Italy, April 2008.

[18] Weick, K.E. 1995 Sensemaking in Organisations. Sage Publications,Thousand Oaks, CA.

[19] Yi, J., Kang, Y., Stasko, J.T. and Jacko, J.A. 2008 Understanding and characterizing insights: how do people gain insights using information visualization? Proceedings BELIV 2008 Workshop, Conference on Human Factors in Computing Systems (CHI 2008), Florence, Italy, April 2008.