Adaptive HyperMan:A Customizable Hypertext System for Reference Manuals

Joshua RabinowitzCaelum Corp.

NASA Ames Research Center

MS 269-2, Moffett Field,CA 94035, U. S. A.

joshr @ ptolemy.arc,nasa.gov

Nathalie MatheSan Jose State Univ. FoundationNASA Ames Research Center

MS 269-2, Moffett Field,CA 94035, U. S. A.

mathe @ ptolemy, arc.nasa, gov

James R. ChenRecom Technology, Inc.

NASA Ames Research CenterMS 269-2, Moffett Field,

CA 94035, U. S. A.j chen@ ptolemy, arc.nas a.gov

Abstract

We are interested in facilitating informationaccess from large volume of referenceinformation contained in technical andoperational manuals. When a large volume ofinformation is used on an everyday basis toperform one’s job, the problem for users is tobuild and maintain an accurate cognitive modelof the information in order to access it quickly.The problem is not to perform a search todiscover new information, since they havealready learned the information during theirtraining. We have developed an adaptivehypertext system to help Space Shuttle flightcontrollers access operations documents inmission control center. We describe thisintelligent system, called Adaptive HyperMan,which lets users incorporate their representationof the content and organization of documentsover time. It provides sophisticated annotationsand hyperlinking capabilities to end-users, andintegrates an adaptive indexing and retrievalengine for managing annotations. This novelfeature lets users assign topics to annotations,retrieve annotations by topics, and providerelevance feedback over time. Besidesmemorizing user inputs, the indexing engine alsolearns to generalize user inputs in order tofacilitate retrieval from similar topics. Wedescribe the Adaptive HyperMan system, thenshow how it provides a virtual "goody book"facility to flight controllers, and supportscollaborative work.

1 The Problem

We are interested in facilitating access to large volumeof reference information contained in technical andoperational manuals. This type of information is used inmany fields by professionals like airplane mechanics,pilots, astronauts, Space Shuttle flight controllers, powerplant controllers, lawyers, doctors, etc. These personsneed to use of large amount of technical information to

perform their everyday job. However, as opposed to moretraditional information retrieval tasks, they do not oftenneed to search for information to answer their queries.They rather already know most of the information andwhere it is stored (it is part of their training), and justneed to access it quickly to use it as a back-up to theirmemory (to prevent human errors).

For example, we are working with Space Shuttleflight controllers. It takes years of training to become aflight controller in the Space Shuttle Mission ControlCenter. As part of this training, people learn to use alarge corpus of documentation to solve problems. Theydevelop a deep knowledge of the organization andcontent of these manuals in order to access the propersections as quickly as possible.

This knowledge, or cognitive map, is context- anduser-dependent [Boy, 1991]. However, it does not persistvery long unless the same information is accessed oftenunder the same context. Users therefore develop artifactsto support their memory, like annotations, Post-It notes,bookmarks; or they create condensed representations ofimportant information like cue cards, quick access guide,abbreviated checklists. For example, each flightcontrollers develops a personal "goody book", which is apersonal collection of selected information from existingShuttle operations documents.

Beside the volume of information to remember,another problem is due to the frequent updates andrevisions of technical manuals. Users have to constantlyrevised what they know, as well as manually update theirpersonal annotations and goody books. This is timeconsuming, and creates a potential safety risk. Currentlythe operational documentation used by flight controllersis paper-based, and goody books are paper copies ofpages organized and annotated together in a binder.

Thus, the problem for technical users is to build andmaintain an accurate cognitive model/map of theinformation in order to quickly access relevantinformation. We describe an intelligent system, calledAdaptive HyperMan, which lets users incorporate theirrepresentation of the content and organization ofdocuments over time. It provides sophisticatedannotations and hyperlinking capabilities to end-users,

ii0

From: AAAI Technical Report FS-95-03. Compilation copyright © 1995, AAAI (www.aaai.org). All rights reserved.

and integrates an adaptive indexing and retrieval enginefor managing annotations. This novel feature lets usersassign topics to annotations, retrieve annotations bytopics, and provide relevance feedback over time (toupdate the relevance of annotations to topics). Besidesmemorizing user inputs, the indexing engine also learns

to generalize user inputs based on relevance feedbackand previous retrievals, in order to facilitate newretrievals. We briefly describe the Adaptive HyperMansystem, which builds upon the features of the HyperMansystem, then show how it provides a virtual "goody book"facility to flight controllers.

................. i ............................. . ....

S H UTFL E 0 P 8 ~i::~ii~B~DPS/JSC-18819 3II4/94: BASIC, BEV B 8CP 5.8

NOTE:-~_...

BUSESLOST

HNRHNBHNQ

HNQ FPCI

HPCIQPC4

HNB FPC2

HPC2QPC5

HNC FPC3

QPCG

CNIL QB2 CNTL RB3

DOUBLE BUS LOSS MATRIXQLTERNQTE QPPROflCH IF STQNgQRg 9IQGONQL NOT EQSILY IHF

LRU’sT~ F FF F"1 2

,--_...NNB XHNC XHNC

HNB FPC2 XHNC FPC3CNTL RB2

QB3 XQB3

HNB HPC2ESS 2CA

MNB ~IPC5HNC FIPC6CNTL RB2

CA2CQ3

NNC FPC3 XCNIL FIB1

QB2 XBC3 XCfll

ESS IREHNQ QPC4HNC QPCGCNTL QB"1

BC2CNTL BCI

BC2 XCQ2

HNR RPC4NNB APC5CNTL BCI

BC2X

FFFFFFPPPFFQQQQLLL34123412T

BX X X

X X XX XX

XX

XX

XX

XX X

XX

XX X

X X

XX

XX X

X i~ ~-X X

XX

X X

Figure l: The HyperMan book window. The annotation tools palette is shown at the top left, below the menu bar. Anexample of a marker, the highlighted text "BUS LOSS TABLES", is shown at the top right of the page.

iii

2 The HyperMan Viewer

HyperMan 2.0 is a software tool for document viewingand parsing, developed as part of the ElectronicDocumentation Project (EDP) at the NASA JohnsonSpace Center (JSC) [NASA JSC-26679, 1994]. Thegoal of the EDP project is to provide an electroniccapability to support authoring, distribution, viewing,and controlled revision of crew and ground controlleroperations documents, for use in the new ControlCenter Complex and in their office environment. EDPintegrates the state-of-the-art hypertext documentviewer, HyperMan, with JSC flight planning andscheduling tools, and commercial workflow automationtools. Starting with a literal representation of thecurrent paper-based system, HyperMan extends thatmetaphor with hypertext capabilities. HyperMan is afull blown wysiwyg PDF (Portable Document Formatfrom Adobe) viewer designed for hundreds ofsimultaneous users’ accesses to operations documents.HyperMan is being used by flight controllers in supportof Shuttle missions since July 1995. The current librarycontains the following books: Flight Data Files (FDF),Flight Rules, Operator Console Handbooks, and SpaceShuttle System Handbook Drawings (a total ofapproximately 100 books.)

To answer flight controllers’ need for intensivecustomization of documents, HyperMan provides theability for end-users to create and store various types ofannotations (Figure 1). Users can create visualmarkers on a page as color highlight (background colorchange of selected text or graphical region), as coloredtext (foreground color change), as icon anchor, or bookmark; and users can create hyperlinks betweenany markers (both inter- and intra-book). To supportcollaborative work, users can publish their annotations,and subscribe to published annotations from varioususer groups.

Other useful features include full text search,automatic hyperlinking at parsing time, version control(to retain user annotations between versions ofdocuments), and transportable annotations (each user’sannotations are stored separately from the documents,so that the document itself is never altered by thecreation or deletion of markers or hyperlinks). We donot have space to describe the HyperMan system inmore details, and will focus in the next sections on theadaptive modifications we have made.

3 Adaptive HyperMan

Adaptive HyperMan (AHM) builds on the strengths HyperMan by allowing users to assign subjectiveindexing information, called topics, to markers indocuments, and to perform searches for markers thathave been associated with particular topics1. It alsolets users provide relevance feedback over time (toupdate the relevance of annotations to topics). The

1 This supplements the full text search ability, as markers

may have been indexed under topics that never appear aswords on that page, or may be associated with graphics.

system utilizes an adaptive indexing and retrievalengine designed to learn from similar queries andrelevance feedback [Chen and Mathe, 1994; Matheand Chen, 1994].

3.1 Indexing

To help users build a cognitive map of the information,AHM provides the ability to index markers by topics.From the indexing engine point of view, a topic can beany word or sequence of words2. The indexing enginememorizes the exact set of topics defined by a user fora selected marker, so as to provide accurate retrievallater on. It also generalizes the indexing to subsets ofthe topics set, in order to facilitate retrieval withsimilar queries later on (similar queries are defined assharing common topics). This indexing information isstored in a network structure, called user profiledatabase.

3.2 Retrieval

To facilitate quick access to information, AHMprovides the ability to retrieve markers by topics. Theuser specifies a set of topics as a conjunctive query.The engine retrieves a list of markers by exact match,if the exact same set of topics was previously assignedto some markers (memorization); or by derivation fromprevious queries (memorized or generalized) whichcontain subsets of the current query.

3.3 Learning

To help users maintain an accurate cognitive model, asthe information is updated or the user knowledgeevolves, AHM offers users the ability to give relevancefeedback in order to modify the relevance of markers togiven topics over time (users can always add newtopics through the indexing capability). When usersgive positive or negative feedback to markers retrievedwith given topics, the adaptive engine automaticallyadjusts its relevance measures for these markers andtopics, and propagates feedback to all proper topicssubsets (generalization). The adaptive engine alsolearns to improve its retrieval performance by usage. Itautomatically analyzes co-occurrence statistics oftopics over a collection of previous queries, thenselects and memorizes sets of topics with high co-occurrence value to improve the generalizationprocess.

3.4 Collaborative work

To facilitate sharing of information among a group ofusers (e.g., flight controllers with the same consoleposition), and to facilitate training of novice users,AHM combines inputs from all users in a separatesystem profile database (in addition to each user’sprofile database). This supports the creation of corporate memory over time, which can be shared byall users, or used as a starting point to their individualindexing.

2We do not currently represent semantic relationship betweentopics, but plan to do it in our future work..

112

7: il .............................................. ............. @i

iiii iii:!:ii:ii

7I 7iiiiii{iiiiiiiiiii" " ...............

Figure 2: The Customize Marker window. The user selects from the list of topics on the right and assembles a list of topicsto associate to the marker shown on top.

4 The User InterfaceThis section describes how users interact with our system.An explicit design goal was to change the HyperManuser interface as little as possible, so as to not confusenovice HyperMan users. The only visible difference wemade to the book window is the additional User Profilemenu (Figure 1), which provides access to four newwindows: Customize Marker (Figure 2), MarkerRetrieval (Figure 3), Marker Retrieval Results (Figure4), and Markers Basket (Figure 5). The features Adaptive HyperMan fall into three categories:

1) Assigning topics to markers:The user can associate any list of topics, either pre-defined (called author topics) or user-defined topics, to selected marker (Figure 2). The user can also assigntopics to a marker by clicking on the "success" button(Figure 4), specifying that the selected marker relevant to the last topic search.

2) Retrieving markers by topics:

The user can search all books in the current library formarkers relevant to a list of selected topics (Figure 3).This search can be performed on either the user profile orsystem profile databases. In addition, the user canspecify the maximum number of markers wanted,whether to use full match (match all topics) or partialmatch (at least one topic) algorithms. The list retrieved markers is displayed in the Marker RetrievalResults window (Figure 4).

3) Setting aside interesting markers:

The Markers Basket is a place where users can store alist of markers belonging to any book. We envision thatusers will use this to recall markers that they wish toassign topics to later; but it can also be used like ahotlist in Netscape or Mosaic to store a list of markers forquick reference.

The Customize Marker window (Figure 2) allows users assign list of pre-defined topics (Author Topics) andstrings of free text (User Topics) to markers. This performed by first selecting a marker in the HyperManbook window, then choosing the Customize Markeroption from the User Profile menu. The CustomizeMarker window will be opened or brought to theforeground with the selected marker in the top line 3. Theuser selects from the list of Authors topics on the rightand assembles a list of topics to associate to the selectedmarker. To assign topics that aren’t expressed by any ofthe Author Topics, users can type them into the EnterUser Topic field. The selected marker will then beautomatically indexed under the list of topics specifiedby the user, and this customization will be stored in boththe user profile database and the system profile database.

3 The current implementation displays only the marker internalID, the next implementation with display the marker name,book, and page location.

113

II F.::::::ii:::: ::::::.::i:. F. ii ! ii¢~ i¢ ~¢ !!~!~:!!!!!!!i~! ~!i!i!:i~i!i~:.:.:+~:.:.:+:.:.:.:.:.: ~.: ~.: ~.: ~ ~ ~.:

iii!!i!iiii!ii!! i! i!i i ii]i!!i!!i! l!il®ii

!i!i !i i i i i i i iiiiiiiiiiiiii!i!iii!!iii!i

I!II!!I! ...........................................................................................!i!i!ili! ..................i li~!iiiili¸I!~!!!!!!!!!!!~!~!!!!!~!!!!!~!!!!!~!!~!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!::~i~i~i~i~i~i~i~i~i~i~ii~iiii~i~!!iii~i~ii!~i~i~i!ii!~iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii!ii~

Figure 3: The Marker Retrieval window. The user assembles a list of topics describing the markers they are looking for.

The Marker Retrieval window (Figure 3) allows user search all books in the current library for markersrelevant to a list of topics. The user assembles a list oftopics describing the markers they are looking for. Thiswindow gives several options specifying how toperform the retrieval. The matching mode specifieswhether to retrieve markers that have been assignedevery topic in the list or those that include at least one.

The profile database specifies whether to considerinput learned from all users relevant to the search(System), or only input which has been learned fromthat particular user. The user can also specify themaximum number of markers to be retrieved (upperlimit is set to 1024). After submitting the search, theMarker Retrieval Results window will display a list ofrelevant markers.

The Marker Retrieval Results window shows a list ofretrieved markers relevant to the selected set of topics.The user can select a marker and directly go to thismarker in a book. The list of markers is ranked byrelative relevance to selected topics (most relevantfirst). The user can adjust the ranking of a particularmarker by giving relevance feedback (Success andFailure buttons), which will move the marker up/downthe list. The user can also associate additional topicsto a marker using the Customize button, which sets upthe Customize Marker window to customize thatmarker.

ii ~:ii !i!!~;!!ii: i~i:iiiiiiiiiii:iiii:iii~iiiiiiiiiiii!i!!~:~:~i iii:::i~ ........................................ ~iiiii!iiiiiiii!iiii-

Figure 4: The Marker Retrieval Results windowshows the results of a search for markers associatedwith the retrieval topics listed on top.

The Markers Basket window is used to set aside/store aset of interesting markers (belonging to any book in thecurrent library). It can be seen as the equivalent of theHot List in Netscape or Mosaic (World Wide Webbrowsers). This can be used as a quick access tool todirectly go to markers attached to specific pages inbooks. This can also be used as a tool to set asideinteresting markers, to be indexed or worked on later

114

on (flight controllers might not have the time to indexmarkers during a mission)

! I warkea o zs : :::: i-iii:~ :. i i~::~ ~i:::O~::i~O~i i~ :.::!i:::i if: :~ i.i.::i: i~:!~i ii~iiii ii !ii iiii~i t~ ~?:-

ji:il

Figure 5: The Markers Basket window stores a listof markers for quick access or future indexing.

5 Virtual "Goody Book"Adaptive HyperMan was tested in April 1995 withthree flight controllers. Their response to the additionalmarker indexing and retrieval capabilities describedabove was very positive. In fact, they requested to beable to use these capabilities for the next Shuttle flight.The reason for this success is based on the idea of avirtual "goody book". A paper "goody book" is apersonal collection of selected pages from existingShuttle operations documents, assembled by eachflight controller to perform their task more efficiently.In order to ensure using only up to date information,mission control management decided to have onlyversion controlled operations books be accessible inHyperMan, and to not allow the parsing of any userassembled "goody book". A simple solution would beto provide the equivalent of a Netscape hierarchicalhotlist in HyperMan, but this forces the classificationof markers under unique headings. Instead, flightcontrollers create markers for multiple purposes: quickaccess (goody book),’highlighting critical information,writing down comments during a mission (post-flightactions), which are later used in collaborative reviewsto revise documents. Therefore being able to indexmarkers under multiple topics is very valuable, as itgives them capabilities to retrieve markers for a givenflight or simulation, for a particular problem to share incollaborative review, for a particular system, and tocombine these topics to narrow down the list ofmarkers. It gives them a virtual "goody book" which isdynamically computed and always up to date.

6 ConclusionAdaptive HyperMan was designed to facilitateinformation access from large volume of technical andoperational manuals, by helping users build andmaintain an accurate cognitive model of theinformation. The system provides sophisticatedannotations and hyperlinking capabilities to end-users,

115

and lets them assign topics to annotations, retrieveannotations by topics, and provide relevance feedbackover time. Besides memorizing user inputs, theunderlying indexing engine also learns to generalizeuser inputs in order to facilitate new retrievals. Webriefly described the Adaptive HyperMan system, itsuser interface, and showed how it provides a "virtualgoody book" facility for Space Shuttle flightcontrollers.

We plan to enhance the system in two majordirections. First we will support collaborative work byproviding a publish/subsCribe mechanism to developusers group indices for markers. Second, we arestudying the idea of a virtual goody book beingdisplayed as a book (with pages to flip) instead of list of markers, and of providing an authoring facility tolet users edit this virtual book (adding their own newpages, or reordering pages). Finally, we aredeveloping a World Wide Web Hotlist Organizer basedon our adaptive indexing and retrieval engine.

AcknowledgmentsMany thanks to all members of the ElectronicDocumentation Project team at NASA/JSC and toShuttle flight controllers for their support.

References[Boy, 1991] Boy, G. A. 1991. Indexing Hypertext

Documents in Context. In Proc. of the ThirdConference on Hypertext, San Antonio, Texas,51-61.

[NASA JSC-26679, 1994] ElectronicDocumentation Project (EDP) Level BRequirements. NASA JSC-26679, June 30, 1994[ a 1 s o a thttp://www.j sc.nasa.gov/cssb/HyperMan/edp.html].

[Chert and Mathe, 1994] Chen, J. R., and Mathe, N.1994. A Contextual Relevance Model forAdaptive Information Retrieval. In Proc. of theAAAI-94 Workshop on Indexing and Reuse inMultimedia Systems, Seattle, WA, 175-186.

[Mathe and Chen, 1994] Mathe, N, and Chert, J. 1994.A User-Centered Approach to Adaptive Hypertextbased on an Information Relevance Model. InProc. of the Fourth International Conference onUser Modeling, Hyannis, MA, 107-114 [also athttp://ic-www.arc.nasa.gov/ic/projects/aim/um94/cidum94.html].