A UNIFIED INTERFACE FOR VISUAL AND INTERACTIVE WEB SEARCH

Orland Hoeber and Xue Dong YangDepartment of Computer Science

University of ReginaRegina, Saskatchewan, Canada

email:{hoeber, yang}@cs.uregina.ca

ABSTRACTThe interfaces used by the top Web search engines havechanged very little since the early days of Web search.These interfaces follow the traditional model of informa-tion retrieval in which users first formulate their queries andthen evaluate the search results. Little support is providedfor the users’ tasks of crafting and refining their queries,and subsequently exploring the search results. In this pa-per, we describe HotMap+WordBars, a next-generation in-teractive Web search interface that allows users to take anactive role in their Web search tasks through visual explo-ration and manipulation features.

KEY WORDSWeb search, user interface, interactive query refinement, in-teractive search results exploration.

1 Introduction

In recent years, significant research has gone into the im-provement of Web search engines, both in the sizes of theirindexes, as well as in the algorithms used to match users’queries to these indexes. A prime example of these ad-vances are those made by Google [2, 3]. In the course ofthese improvements, Web search engines have become theprimary tool people use to find information on the Web.

Web search engines generally perform very wellwhen users are able to craft good queries that accuratelycapture their information needs. In these cases, there areoften many relevant documents within the first few pagesof the search results. Presented in a list-based format, theusers can easily view the top search results one-by-one anddecide the relevance of the documents with respect to theinformation they are seeking.

However, when users are unable to effectively crafta query, or when their information needs are inherentlyambiguous or poorly defined, Web search engines pro-vide little support to assist the users in their primary Websearch tasks. In these cases, the results of an initial searchwill often contain many non-relevant documents. The op-tions available to the user are to either spend time lookingthrough many pages of search results, or try to construct abetter query.

Evidence of the inability of users to craft effectivequeries has been highlighted in studies on user behaviour

with respect to Web search engines [9, 14, 15]. A recurringtheme in these studies is that search queries commonly con-sist of only one to three terms, and that users seldom makesubsequent modifications to their queries, even when thesearch results are poor. Part of the difficulties that users ex-perience can be attributed to the lack of support providedby the Web search engines for the task of crafting a query.Often, queries must be modified manually, based on infor-mation the users may have learned by viewing the resultsof the current or previous searches.

A related problem identified in these studies is thatusers seldom venture past the third page of search results[14, 15]. Even when users are able to craft a high qual-ity query that effectively captures their information needs,the search results may be a mixture of relevant and non-relevant documents. The static list-based representationthat is common among Web search engines requires theusers to evaluate the search results one-by-one, and to somedegree, in the order provided.

Given the shortcomings of the interfaces of the topWeb search engines, an opportunity exists to use interac-tivity and visualization as a means for supporting the usersin their primary Web search tasks. Our approach pro-vides visual representations of features of the initial set ofsearch results, supporting the users as they interactively re-fine their queries and interactively explore the search re-sults sets. This represents an example of what we believewill be the next generation of Web search interfaces: sys-tems which allow the users to become actively involved intheir Web search tasks through the visual exploration andmanipulation of Web search results.

2 HotMap+WordBars

Our prior research in the area of interactive Web searchinterfaces has resulted in a number of systems to supportusers in performing their Web search tasks. Two such sys-tems are HotMap [8] and WordBars [7]. User evaluationsof HotMap have shown that the visual and interactive fea-tures it provides can effectively support users in finding rel-evant documents [6]. A user evaluation with WordBars wasalso very positive (the results are currently under review forpublication).

Two observations about how some people search theWeb have guided this research. The first observation is

that users sometimes scan the search results looking fordocuments which make frequent use of their query terms.This observation led to the exploration of methods for visu-ally displaying query term frequencies from the top searchresults (HotMap), and subsequently, the investigation ofmethods for visually displaying the frequencies of otherterms that also appear in the top search results (WordBars).The second observation was that there are commonly manyrelevant documents buried deep in the search results list;accessing these documents often requires the users to con-sider many non-relevant documents in the process. Fromthis observation, techniques were explored for re-sortingthe search results to bring these relevant documents to thetop of the list, a feature that is present in both HotMap andWordBars.

There are benefits to combining the positive featuresof these two systems into a single system that supports bothinteractive query refinement and interactive search resultsexploration in a single interface. HotMap is an effectivetool for the visual exploration and interactive manipulationof the search results set when a high-quality query is used.However, it provides no support for crafting or refining aquery. The WordBars features address this shortcoming,and provides additional methods for exploring the searchresults. As a result, the users of theHotMap+WordBarssystem can take advantage of the features of both systemsin a seamless unified interface that supports both funda-mental Web search tasks.

In this paper, we describe the features drawn fromHotMap and WordBars, along with the benefits providedby combining these features into a single Web search in-terface. A usage scenario is provided to illustrate thesebenefits. The subjective reaction results from user stud-ies highlight the ease of use and the degree to which usersare willing to accept the visual and interactive features ofHotMap+WordBars.

3 HotMap

The primary goal in the development of HotMap was tosupport the users in exploring the search results and ulti-mately finding relevant documents. After a user submitsa query, a set of the top search results are obtained fromthe underlying search engine (the Google API [4]), and thefrequencies of the query terms are counted within the titleand snippet of these search results. The search results arepresented in a scrollable document list. Along with eachdocument in this list, a group of colour-coded indicatorsare provided (one for each query term) that visually repre-sent the frequency of the query terms.

This supplemental information about the query termfrequencies is present both in thedocument list, whichshows the details of a subset of the search results (see Fig-ure 1a), and in theoverview map, which shows a com-pact and abstract representation of all the documents in thesearch results set (see Figure 1b). Coordinated scrollingbetween these two views of the search results allows the

users to easily remain aware of the location of documentsthey are currently viewing with respect to the entire set ofdocuments retrieved.

In the original implementation of HotMap, the searchresults were presented in thedocument listin a compactformat, where 20 to 30 documents could be viewed at onetime. This compact format only showed the title of the doc-ument in a persistent manner; the snippet and URL of thedocument were available as a tool tip. Although many par-ticipants in our user studies indicated that they preferreda compact representation which promotes scanning withinthe titles, others indicated that they did not like the compactrepresentation.

In the development of WordBars, we departed fromthis compact representation, and provided the title, snippet,and URL in a format similar to that used by the major Websearch engines. In the combined HotMap+WordBars sys-tem, the user has a choice between the expanded and com-pact format, and can easily switch between the two viewsas as they perform their search tasks.

In theoverview map, a compact visual representationof the set of retrieved search results is displayed. Thisincludes colour-coded boxes to represent the query termfrequencies in the search results, as well as lines whoselengths are proportional to the lengths of the titles of thedocuments. A grey box indicates which documents are cur-rently being displayed in the document list. These featuresprovide perceptual cues that make the relationship betweenthe overview map and the document list apparent to the user[1].

Although the overview map can easily represent sev-eral hundreds of documents at a time, the current imple-mentation limits the search results set to 100 documents.This limit represents 10 pages worth of search results, andwas chosen as a balance between two constraints: collectmore documents than users would normally have the pa-tience to view; and return the search results set to the usersin near real-time. Collecting fewer documents may result inmissing some highly relevant documents buried deeper inthe search results; collecting more documents may resultin the user waiting longer for the search results to finishloading.

The features of HotMap provide two interactionmethods to support users as they explore the search re-sults. First, users can visually inspect the overview map toidentify documents which make frequent use of the queryterms. A simple glance at the overview map provides userswith an immediate impression of the use of their queryterms within the search results set. The users can jumpto the location of documents of interest by clicking on theoverview map, which scrolls the document list to the cor-responding location in the search results. The documentscan then be investigated further in the document list andconsidered for relevance.

A second method for interacting with the search re-sults is through a nested sorting feature in the documentlist. Each query term is provided at the top of a column in

(a) Document List (b) HotMap overview map (c) WordBars histogram

Figure 1. Zoomed-in views of the three main features of HotMap+WordBars.

the document list, below which are the colour-coded queryterm frequency indicators. Clicking on the query term col-umn header sorts the search results based on the frequencyof this term. Control-clicking can add additional queryterms in a nested sorting fashion. This allows the usersto easily specify their order of preference for their queryterms, resulting in a re-sorting of the top search results.

Since the re-sorting of the search results is an impor-tant feature in this system, a method for keeping track ofwhich documents have been viewed is necessary. The defacto standard for Web link colours is employed, with bluebeing used for the links of documents that have not yet beenviewed, and purple being used for the links of viewed doc-uments. Accessing a document simply requires a click ofthe title link of the document, resulting in the documentbeing opened in a new browser window.

Through these two interactions methods, the users caninteractively and visually explore the set of search resultsreturned from the query. Doing so can lead the user todiscover relevant documents buried deep in the search re-sults that would have been difficult to view using the staticlist-based representation that is commonly employed bythe top search engines. User evaluations of HotMap indi-cate that the visual inspection of the overview map and thenested sorting in the document list window based on queryterm frequencies can allow the users to become more effi-cient and effective in finding relevant documents within thesearch results [6].

4 WordBars

The goal in the development of the WordBars features wasto provide support to users both in their tasks of craftingbetter queries, and their tasks of evaluating and exploringthe Web search results. The technique used is to count thefrequencies of all the terms that appear in the title and snip-pet of the top search results, and presents the top 20 of these

terms in a vertically-oriented colour-coded histogram (seeFigure 1c).

Rather than using exact term matches, Porter’s stem-ming algorithm [12] is used in the counting process so thatwords with the same root are matched as the same term.The terms displayed in the histogram use the format of thefirst occurrence of the term from the search results. Com-mon terms, and those that are less than three characterslong are ignored, since they are often words with little de-scriptive value for the Web search process.

By visually inspecting this histogram, users are ableto determine the general makeup of the top search results.The commonly used terms are clearly identifiable, as arethe relative differences in frequency of use. The terms thatappear in this histogram can assist the users in determiningwhether their query is very specific (with many of the termsbeing relevant to the information need) or very vague (witha wide range of terms on a number of different topics).

The user interaction within the histogram was de-signed to be very easy to use. Clicking on a term in thehistogram selects it, and causes the search results to be re-sorted based on the frequency of this term. Clicking multi-ple terms in this manner causes the search results to be re-sorted based on their collective use of the selected terms.Double-clicking adds or removes the target term from thequery, allowing the user to submit a newly refined query tothe underlying search engine.

Providing a list from which the users can choose addi-tional terms to add to their query is not a new technique. Acommonly cited method [5] provided three simple lists ofterms from which the users could choose. However, it hasbeen suggested that users may have difficulties making ef-fective selections from these lists when given the task of ex-panding their queries [13, 10]. In WordBars, the histogramrepresentation allows the users to easily see the relative fre-quencies of the terms in the top search results. In addition,the users can experiment with re-sorting the search resultsbased on some of these terms before committing to adding

them to the query. These additional features make the util-ity of the interactive WordBars histogram superior to a sim-ple list of terms.

The histogram containing potential terms to add orremove from the query allows the users to recognize termsthat are relevant to their information need, rather than re-quiring them to recall all the relevant terms.Recognitionrather than recallis a primary usability principle, suggest-ing that users should be able to see the information theyneed as they conduct their tasks, rather than having to re-member the information [11]. As a result, users can startwith a query consisting of a few relevant terms, and theneasily identify additional terms to add or remove from thequery.

A new feature in HotMap+WordBars is a colour-coded indicator in the document list which visually repre-sents the frequencies of the set of selected terms in Word-Bars for each of the documents in the search results set.These colour codes use a yellow-green-blue colour scaleso as not to interfere with the HotMap colour scale. Theycommunicate to the users the relative differences in thefrequencies of the terms selected from the WordBars his-togram within the search results set.

It is interesting to note that simply selecting the mostfrequently used terms to add to the query may not producebetter search results. Often, the significant words are thosethat appear in the middle frequencies [16]. WordBars as-sists the users in recognizing terms that may be good de-scriptors of their information needs, and allows the usersto easily see how these terms are being used in the searchresults before choosing to add them to their query.

5 Combined Benefits

Although both the HotMap and WordBars features pro-vide visual representations of the properties of the Websearch, they do so in support of two different user tasks.The primary benefit of the WordBars features is to supportthe users in interactive query refinement processes. By re-sorting the search results, the users can consider the poten-tial relevance of candidate terms, and easily add or removethese from the query. By contrast, the primary benefits ofthe HotMap features are to explore the search results (bothvisually and through interactive nested re-sorting) once ahigh-quality query has been provided. Together, these fea-tures support the users in their Web search task within aunified user interface.

The single interface is an important aspect ofHotMap+WordBars. Within the Web search process, oncethe user is satisfied that they have constructed a qualityquery for their information need, they can readily turn theirattention to the HotMap features as they explore the searchresults. Similarly, if a user decides that they would like toreturn to the query refinement stage, they can do so withoutthe cognitive overhead of initiating a change of state in theinterface. All they need to do is direct their attention backto the WordBars histogram.

6 Usage Scenario

The HotMap+WordBars system supports the users in theirWeb search tasks through visual representations of featuresof the current search, along with methods for interactivelymanipulating both the query and the search results set. Toillustrate how the this system can support the users as theyconduct their Web search tasks, a typical usage scenario isprovided. Suppose a user wishes to search for informationon the same topic as the previous example: “internationalart crime”. The results of the first stage of this search aredepicted in Figure 2a. By inspecting the top few searchresults, the user can see that there are a lot of non-relevantdocuments mixed in with the relevant documents. As such,the user may wish to refine their query further.

Inspecting the WordBars histogram, the user can seethat there are a number of terms that appear in the searchresults that are relevant to their information need. By se-lecting a few of these, those documents that make use ofthese terms the most are moved to the top of the search re-sults, as shown in Figure 2b. These documents can readilybe considered for relevance.

With the query refined based on terms recognized asrelevant in the WordBars histogram, the user can retrievea new set of search results by clicking the search button.The system presents a new set of search results, along witha new WordBars histogram and a new HotMap representa-tion, both based on the new search results provided by theGoogle API. The results of this refined search are displayedin Figure 2c.

At this point, the user may wish to explore the top 100search results by visually inspecting the HotMap. Withinthis compact and abstract representation, the user can eas-ily identify documents that make frequent use of the queryterms. Clicking or scrolling to the vicinity of one such doc-ument allows the user to view the corresponding documentin the document list and determine its relevance. For exam-ple, consider document 52 in Figure 2d.

Alternately, the user may wish to browse the searchresults in the document list in a more compact manner. Se-lecting the “compact list” option, the search results are pro-vided with one document title per row. Tool tips are used toaccess the snippet and URL of the document, as illustratedin Figure 2e.

In addition to the visual inspection, the user can alsore-sort the search results based on the importance theyplace on the terms in our query. For example, the user maywish to explore the documents that make frequent use of theterms “international” and “theft”. A nested sort is specifiedby control-clicking on these terms in the column header ofthe document list, the results of which are illustrated in Fig-ure 2f.

From this scenario, the support this system providesto the user in interactively refining their query, as well asinteractively exploring the search results is clear. In thequery refinement process, the user was able to recognizeadditional terms that might be relevant to their informa-

(a) Initial search forinternational art crime. (b) Selecting relevant terms in the WordBars histogram.

(c) Refining the query based on the information in the WordBarshistogram.

(d) Visual inspection of the HotMap

(e) Compact representation of search results list. (f) Re-sorting the search results based on query terms.

Figure 2. Screenshots from HotMap+WordBars illustrating a usage scenario.

Figure 3. Responses to subjective reaction questions for HotMap and WordBars features.

tion needs, and experiment with bringing the documentsthat make use of those terms to the top of the search re-sults list. Adding new terms to the query simply requireda double-click. Once a good query was developed, theHotMap representation supported the user in visually ex-ploring the search results, as well as re-sorting the searchresults based on the importance placed on the query termsby the user. Because each of the operations that re-sortsthe search results in the document list does so instantly, theuser was able to interactively explore the search results. Avideo depicting a similar scenario is available on the au-thor’s Web site1.

7 Subjective Reactions

User studies were conducted to evaluate the effectivenessof the HotMap and WordBars features separately. Prelim-inary results from HotMap were reported in [6]; here wereport the subjective reactions with respect to the variousinterface features from the extended studies. There were28 participants in the HotMap evaluation, and 24 partici-pants in the WordBars evaluation.

Figure 3 illustrates the subjective reactions of the par-ticipants to the various features of both HotMap and Word-Bars after having completed a number of assigned tasks us-ing the interfaces. Clearly, many of the participants foundthe features easy to use and useful in their Web search tasks.

When asked to rank their preference for a Web searchinterface, 68% of the participants ranked HotMap higherthan Google. In the WordBars study, 96% of the partic-

1http://www.cs.uregina.ca/˜hoeber/HMWB/

ipants ranked the search results generated using the fea-tures of WordBars as preferable to the original set of searchresults provided by the Google API. These subjective re-actions to the visual and interactive features present inHotMap+WordBars indicate that users are willing to acceptand use new tools to support their Web search tasks.

8 Discussion

Wise et al. noted that “the need to read and assess largeamounts of text that is retrieved through even the most effi-cient means puts a severe upper limit on the amount of textinformation that can be processed by any analyst for anypurpose” [17]. This statement provides a clear motivationfor the use of visualization methods to support Web searchtasks. Allowing the searchers toseethe information with-out having toread the information may allow this upperlimit to be exceeded.

Visual representations of the features of a Web searchcan be further exploited through interactive methods.While a static visual representation may be of some value inevaluating the search results, an interactive visual represen-tation allows the users to take an active role in the informa-tion retrieval process. Systems such as HotMap+WordBarscan provide support for users in their tasks of interactivequery refinement and interactive search results exploration.

The primary drawback in this work is that there is lit-tle ability to support the users in their Web search taskswhen a very poor initial query is provided. If no rel-evant documents are present within the top search re-sults, the ability to explore the search results or refinethe query based on frequently used terms is of little value

to the user. However, if at least some of the documentsthat are returned from the initial search are relevant, thenHotMap+WordBars can be very beneficial in supportingthe users as they conduct their Web searches, as illustratedby the usage scenario in this paper.

9 Conclusion

In this paper, the methods by which HotMap+WordBarsmakes use of interaction and visualization features to sup-port Web searchers in their tasks of interactive query refine-ment and interactive search results exploration are high-lighted. The system allows the users to take an active rolein their information seeking tasks, and interact with the in-formation present in the search results both to build betterqueries, and to find relevant documents that are buried deepin the search results. All of the interaction occurs within asingle unified interface, allowing the users to easily switchbetween their two primary tasks of interactive query refine-ment and interactive search results exploration.

HotMap+WordBars represents an example of whatwe believe will be the next generation of Web search in-terfaces: tools that provide visual representations of fea-tures of the search, and allow the users to interact withinthe search process in order to improve their abilities andperformance. Our future plans include further evaluationsof this work in real-world Web search settings, exploringother metrics which may assist the users in selecting rel-evant terms which are of value for query refinement andsearch results exploration, as well as investigating the util-ity of personalization features to support the users as theyconduct their Web search tasks.

References

[1] Michelle Q. Wang Baldonado, Alliston Woodruff,and Allan Kuchinsky. Guidelines fo using multipleviews in information visualization. InProceedings ofthe ACM Advanced Visual Interfaces, 2000.

[2] Sergey Brin and Lawrence Page. The anatomy of alarge-scale hypertextual web search engine. InPro-ceedings of the Seventh International World Wide WebConference, pages 107–117, 1998.

[3] Sanjay Ghemawat, Howard Gobioff, and Shun-TakLeung. The Google file system. InProceedings ofthe Nineteenth ACM Symposium on Operating SystemPrinciples, pages 29–43, 2003.

[4] Google. Google web API. www.google.com/apis/,2005.

[5] Donna Harman. Towards interactive query expansion.In Proceedings of the ACM SIGIR Conference onResearch and Development in Information Retrieval,pages 321–331, 1988.

[6] Orland Hoeber and Xue Dong Yang. A comparativeuser study of web search interfaces: HotMap, Con-cept Highlighter, and Google. InProceedings of theIEEE/WIC/ACM International Conference on Web In-telligence, 2006.

[7] Orland Hoeber and Xue Dong Yang. Interactive webinformation retrieval using WordBars. InProceedingsof the IEEE/WIC/ACM International Conference onWeb Intelligence, 2006.

[8] Orland Hoeber and Xue Dong Yang. The visual ex-ploration of web search results using HotMap. InPro-ceedings of the International Conference on Informa-tion Visualization, pages 157–165, 2006.

[9] Bernard J. Jansen and Udo Pooch. A review of websearching studies and a framework for future research.Journal of the American Society for Information Sci-ence and Technology, 52(3):235–246, 2001.

[10] Mark Magennis and Cornelis J. van Rijsbergen. Thepotential and actual effectiveness of interactive queryexpansion. InProceedings of the ACM SIGIR Con-ference on Research and Development in InformationRetrieval, 1997.

[11] Jakob Nielsen. Enhancing the explanatory power ofusability heuristics. InProceedings of the ACM Con-ference on Human Factors in Computing Systems,1994.

[12] Martin Porter. An algorithm for suffix stripping.Pro-gram, 14(3), 1980.

[13] Ian Ruthven. Re-examining the potential effective-ness of interactive query expansion. InProceedingsof the ACM SIGIR Conference on Research and De-velopment in Information Retrieval, 2003.

[14] Craig Silverstein, Monika Henzinger, Hannes Marais,and Michael Moricz. Analysis of a very large websearch engine query log.SIGIR Forum, 33(1):6–12,1999.

[15] Amanda Spink, Dietmar Wolfram, B. J. Jansen, andTefko Saracevic. Searching the web: The public andtheir queries. Journal of the American Society forInformation Science and Technology, 52(3):226–234,2001.

[16] C. J. van Rijsbergen.Information Retrieval. London:Butterworths, 1979.

[17] James A. Wise, James J. Thomas, Kelly Pennock,David Lantrip, Marc Pottier, Anne Schur, and VernCrow. Visualizing the non-visual: Spatial analy-sis and interaction with information from text docu-ments. InProceedings of IEEE Information Visual-ization, 1995.