Pergamon

Computers ind. Engng Vol. 29, No. 1-4, pp. 437---441, 1995 Copyright © 1995 Elsevier Science Ltd

Printed in Great Britain. All rights reserved 0360-8352/95 $9.50 + 0.00

0360-8352(9b')00113-1

AN EVALUATION OF DIFFERENT NAVIGATIONAL TOOLS IN USING HYPERTEXT

Manu Gupta and Anand K. Gramopadhye

Department of Industrial Engineering Clemson University

Clemson, SC 29634-0920

Abstract

Hypertext is a widely used tbrm of intbrmation presentation which has gained wide populari .ty in the last l~w years due to the ease with which intbrmation can be portrayed and accessed. However, the very flexibili .ty of h~lgertext creates problems of user disorientation and cognitive overhead. Designers in the past have tried to combat these problems by providing users with a myriad of navigational tools, with the most widely employed tools being a map and an index. The primary objective of this study was to evaluate the uselhlness of the above mentioned navigational tools in alleviating the problems associated with using a hypertext system. For the purpo~ of this study, a hypertext package was developed using Hypercard on "Ancient Civilizations." The experiment explored three variables: types of navigational tool available to the user (Map, Index and Combination of map and index): hypertext size (Small stack, Large stack) and trials (Belbre, After). The results of the study indicate significant dift~rences in the use and eli~ct of the navigational tools.

INTRODUCTION

Hypertext has quickly become an established paradigm in the design of information systems. The success of products in the software market, evident benefits as reported by users, and the flowering of related research activity all attest to the significance and staying power of hypertext information systems. Hypertext has been defined as a non-linear organization of information (Jonassen, 1988), which involves linking together discrete blocks (chunks) of information to create a network of information. It can also be looked upon as a non sequential method of presenting and accessing information wherein the users feel that they can move freely through the information according to their own needs (Marchionini, 1988). Often the terms "hypertext" and "hypermedia" are used interchangeably, but they are not s)aaonymous. In the case of hypertext systems, information is only in text format; however, in hypermedia the information is multimedia (text, graphics, animation, audio).

Hypertext and hypermedia systems have found extensive use for various applications ranging from browsing to training. For instance, Jonassen and Gabringer (1990) list many examples of the use of hypermedia in instructional tools to language learning, science teaching and browsing in encyclopedias. In another context, Christensen, et al. (1993) developed a hypermedia-based instructional tool for teaching the design of hypermedia systems and Koshy, et al. (In Press) developed a hypermedia version of a maintenance manual for diagnostic training. Thus, h)qaertext and hypermedia systems, with their potential to enhance learning, provide an unparalleled opportunity to be useful for various learning and training applications.

PROBLEMS WITH HYPERTEXT

The strengths ofhypertext arise from its flexibility, in storing and retrieving "knowledge. Any piece of information can be linked to any other piece of information, thus constituting for an unstructured network of information. Not only do these systems make it possible to collect wide realms of information with implicit or explicit interconnections in a variety of rapid-access peripherals, but more importantly hypertext systems provide a nonconstrictive environment for choosing educational processes. This environment

437

438 17th International Conference on Computers and Industrial Engineering

allows the users ample controlling power to select paths identified by explicit connections or to navigate freely in tune with their individual capabilities and needs. However, this kind of environment obliges the learners to make decisions continually and to assess constantly their state of progress, forcing them to use high-level intellectual processes. Thus, the hypertext's capability to create environments endowed with high quality information leads to some problems precisely because of the overwhelming amount of information which can be freely navigated and because of the high-level of learner control provided. It is difficult to maintain a sense of where things are in a relatively unstructured network of information (Valdez et al., 1988). In many ways, the problems of hypertext stem from the very flexibility that is its chief advantage and justification. Research conducted in hypertext has identified essentially two classes of problems: those with the current implementation and those that seem to be endemic to hypertext (Conklin, 1987). The problems in the first category include delays in the display of referenced material, restrictions on names and other properties of links, and deficiencies in browsers. However, the problems of disorientation and cognitive overhead found in the second category are more challenging than these implementation shortcomings and may, in fact, ultimately limit the usefulness of hypertext.

Disorientation, the tendency to lose a sense of location and direction in a hypertext document, can become a problem since users must know where they are in the network or how to get to some other place that they know (or think) exists in the network. Thus, one of the main problems in using hypertext is the user's risk of "losing himself' (Roselli, 1991), a risk proportional to the size of the hypertext document. The other fundamental problem with using hypertext is cognitive overhead, the additional effort and concentration necessary to maintain several tasks or trails at one time (Conklin, 1987). This problem can be further compounded if the users have at their disposal a plethora of choices and navigational tools to guide them. The series of alternate choices or the paths to follow and/or ignore, presented in hypertext, lead to the problem of cognitive overhead.

Designers of hypertext systems have tried to combat the problem of disorientation by providing users of hypertext with myriads of navigational tools like browsers, indexes, maps and oven,iew diagrams (Nielsen, 1990). Unfortunately, these tools do not always prove to be effective since they can clash with a number of factors (e.g., the number of nodes and connections and users' lack of orientation towards visual processing) (Conklin, 1987). If designers ofhypertext systems are to develop superior systems (reduction of disorientation and cognitive overhead), they need a set of guidelines.

OUR STUDY

Guidelines for hypertext designers can be established by conducting more studies which will compare the effectiveness of different navigation tools for hypertext systems of different sizes. These studies will then help to determine which navigational tools should be provided to combat the problems of user disorientation and cognitive overhead. This paper reports on a comparison study conducted to deal with these issues and to evaluate the effect of using different navigational tools in hypertext systems.

The specific navigational tools evaluated were a map, an index and a combination of map and index. Although users perform a wide variety of tasks within hypertext, ranging from browsing to authoring, the most frequently performed task is that of information retrieval. Hence, this study focussed on the task of retrieving specific target information to answer questions. To conduct the study, a hypertext system on "Ancient Civilizations" was developed.

Description of the Hypertext System: "Ancient Civilizations"

Hypercard 2.1 was used for the construction of the two packages of different sizes. The stacks were classified as Small or Big based on the number of cards in each stack. The Small stack had 37 cards while the Big stack had 77 cards.

Stack Versions

For both the Small and the Big stack, four versions were created as described below:

1. No Aid: This hypercard stack did not have any special navigational aid. To navigate in this stack, the subject used the content's buttons to switch between sections. The arrow buttons were used to travel between nodes inside a section, and buttons on different topics could be used to cross-reference between and

17th International Conference on Computers and Industrial Engineering 439

inside the sections. 2. Map: In this stack, the subjects were provided with a navigational aid in the form of a localized spatial map. Also known as a "fish-eye view," the map showed each card as a node in the tree structure. Subjects could reach the appropriate node by simply clicking on the node name. 3. Index: In this stack, the subjects were provided with a scrollable index which offered them a listing of all the cards listed by card name. 4. Combined: This stack had both the map and the index.

Subjects

The subjects used for this experiment were 24 graduate and undergraduate students at Clemson University. Subjects were tested for 20/20 vision and randomly assigned to the following four groups:

1. Control group: This group was not provided with any aid on the After trial. 2. Map group: This group was provided with a map as a navigational aid on the After trial. 3. Index group: This group was provided with an index as a navigational aid on the After trial. 4. Combined group: This group was offered both map and index as navigational aids on the After trial.

Experimental Design

The experiment used a "combined within subjects and between subjects" design. The 4 x 2 x 2 design consisted of four groups (Control, Index, Map and Combined) with six subjects nested under each group, two trials (Before and After) and two stack sizes (Big and Small) with the latter two factors treated as repeated measures. All the subjects performed the tasks on both stacks on the Before trial without any aid, but the Index, Map and the Combined Groups were provided with appropnate navigational tools on the After trial. The task set for the subject was an information seeking task wherein the subjects had to locate specific information and fill in the missing blanks to 30 questions.

Data Collection

Data was collected on speed and accuracy measures. The measures related to speed are defined below: 1. Total time to complete the task: The time the subject took to answer all the 30 questions was defined as the total time to complete the task. 2. Total effective task time: On the second trial, Groups 2, 3 and 4 used a navigational aid to complete the task. The total effective task time was the total time to complete the task less the time spent in using the navigational aid. 3. Time spent per node: The total time spent by the subject, divided by the total number of nodes visited, gives the average time spent by the subject at a node.

Accuracy was defined by the following measures: 1. Total number of nodes visited: The total number of nodes visited by the subject in performing the task (for answering all 30 questions) was recorded. 2. Number of nodes - Nodeop,: This measure was obtained by summing the difference between the number of nodes the user took to locate an answer for all 30 questions less the optimum number of nodes it should have taken to locate the answers to all the 30 questions.

RESULTS AND DISCUSSION

For simplicity purposes we will summarize the results as changes in accuracy and performance measures.

Changes in Speed Measures

For both the stacks, all groups required less time to complete the task on the After trial. On analyzing the percentage differences in total task time between the Before and After trials, it was observed that Groups 2 and 3 showed significantly greater improvements than Groups 1 and 4. However, in the case of Group 3, reduction in total task time was significantly greater on the Small stack compared to the Big

440 17th International Conference on Computers and Industrial Engineering

stack. The larger improvements in speed shown by Groups 2 and 3 were as a result of their usage of the navigational aids provided to them. Based on the results obtained it can be hypothesized that the effectiveness of a map in reducing total time does not depend on the size of the stack, unlike the effectiveness of an index. The reduced effectiveness of an index on the Big stack can be explained based on the operational features of an index. To use index, subjects had to scroll through a list of all node names and then make a selection. For the Big stack, the list got longer and hence subjects had to spend more time scrolling which increased the total task time.

To obtain a better understanding of the actual time spent on the task, the effective task time was determined. Group 3 showed maximum percentage improvement on both the Small and Big stacks. On comparing the performance improvements on total task time and effective task time, a shift in performance improvement from Group 2 to Group 3 was observed. This shift in performance can only be hypothesized as the additional time required by the subjects in the index group to use the aid, which added to the total task time on the After Trial. Although, Group 4 was provided with both the tools, it did worse than Groups 2 and 3. On different questions, the map led to a more optimal route than the index and vice versa. The subjects often switched between the two aids in search of a more optimal route to the answer which resulted in an increase in total task time. This finding was also corroborated from the verbal protocols of the subjects in the Combined group who indicated that they spent significant amounts of time switching between the aids and in doing so often got lost, causing them to spend a large portion of their time back- tracking.

Accuracy

All the groups visited significantly fewer nodes on the After trial than on the Before trial. When the groups' performances on the Before trial were compared with performances on the After trial (sum of number of nodes visited for completing tasks using Big and Small stacks), Group 3 showed the maximum percentage decrease followed by Group 4, Group 2 and Group 1. When asked what problems they faced when they started performing the task on the Before trial, the subjects talked about not being sure what information would present itself or what would happen if they clicked on a button. This uncertainty was one reason why the subjects tried to stick to a familiar route on the Before trial rather than experiment with routes which could conceivably reduce their travel within the stack.

Groups 2 and 3 showed significantly large improvements in (nodes - nodeop,) scores on the After trial as compared to Groups 1 and 4. The performance improvements for Group 3 were comparable for both the big and small stack, unlike the performance of Group 2, which showed twice as much improvement on the Small stack as compared to the Big stack. The results discussed above are consistent with those obtained by Webb and Krarner (1987)and Zellweger (1989). Zellweger (1989) concluded that maps or overviews of paths, particularly situated overviews (such as thumbnails or a similar representation), are helpful for small data structures. Webb and Kramer (1987) found that subjects who received a spatial map of a data structure outperformed others. However, they hypothesized that for large and complex data structures, it was possible that the benefit of a spatial map would decrease as the size of the map increased. A similar trend was observed in our study. The marginal improvement in the performance of Group 4 was contrary to expectations. Subjects in Group 4 reported that while switching between the two aids, they would often get lost and have to retrace their path. Two subjects in Group 4, after trying to use both the map and the index, settled on one of the two aids and showed a higher improvement in performance as compared to the other subjects in the same group. Kozma et al. in their research concluded that the use of navigational tools can on occasion improve student performance but on some occasions can actually hurt performance. Tools can diminish performance when students have insufficient domain -knowledge or experience with the tool to take advantage of its capabilities. In these situations, the use of the tool competes for limited cognitive resources that would otherwise be used for the primary task. Such an effect was especially observed in the case of Group 4 wherein they had to learn how to make use of two available tools.

CONCLUSIONS

Analysis of user performance and verbal protocol revealed that different navigational tools appear suitable in different circumstances.

17th International Conference on Computers and Industrial Engineering 441

The important findings of the research are summarized below: 1. The effectiveness of navigational tools is dependent on the size of the stack and the tools are differentially sensitive to different performance measures, which are dependent on stack size. 2. The map is a more effective navigational tool in reducing the total travelling time and, based on current findings, seems less sensitive to stack size. On the other hand, the index is sensitive to the size of the stack. The index is more effective in reducing the total time for the small size stacks. 3. The results with respect to accuracy measures recorded indicate that the index is equally efficient in reducing the number of nodes visited for both the small and the big stack. However, the efficiency of a map is dependent on stack size. A map is more efficient for use in stacks of smaller size. 4. User performance on both speed and accuracy measures degraded when both the index and the map were provided simultaneously. Analysis of performance measures and protocols recorded that the subjects were often inefficient in their usage of the tool, when both the tools were simultaneously available.

The results of this study suggest that the designers of hypertext systems need to bear in mind the size of the system and the tasks users will be seeking to accomplish when they design hypertext systems.

REFERENCES

Christensen, M., Giamo, M and Jones, T. (1993) Support for teaching the design and implementation of multimedia systems. ,STGSE Bulletin, 25(1): 242-246.

Conklin, J. (1987) Hypertext: an introduction and survey. IEEE Computer, 20(9), 17-41.

Jonassen, D. H. (1988) Desigmng structured hypertext and structuring access to hypertext. Education Technology.

Jonassen, D. H. and Gabringer, R. S. (1990) Problems and Issues in Designing Hypertext/Hypermedia for Learning, In Jonassen, D. H. and Mandl, H. (Eds.) Designing Hypermediafor Learning. NATO ASI Series, Springer-Verlag, Berlin Heidelberg.

Kozrna, R. B. (1992)Constructing knowledge with learning tool. In P.A. Kommers, D. H. Jonassen and J. T. Mayes (editors) Cognitive Tools for Learning, Springer-Verlag

Koshy, T., Gramopadhye, A. K., Kennedy, T. and Ramu, N. V. Application of Hypertext Technolog"y to Assist Maintenance on the Shop Floor (In review Computers and Industrial Engineering)

Leventhal, L. M, Teasley, B. M., Instone, K. Rohlman, D. S. and Farhat, J. (1993) Sleuthing in hyperholmes: an evaluation of using hypertext vs. a book to answer questions. Behavior andlnJbrmaaon Technology, 12(3), 149-164.

Marchionini, G. (1988) Hypermedia and Learning. Freedom and Chaos. Education Technology, 8-12.

Nielsen, J. (1990) The art of navigating through hypertext. Communications of the A CM, 33 (3), 298-310.

Roselli, T. ( 1991) Control of user disorientation in hypertext systems. Education Technology, 42-46.

Valdez, F., Chignell, M. and Glenn, B. (1988) Browsing models for hypermedia databases. In Proceedings of the Human Factors Society - 32nd Annual Meeting, Anaheim, CA. 318-322.

Webb, J. M. and Kramer, F. A. (1987) Learning hierarchical menu systems: A comparative investigation of analogical and pictoral format. Proceedings of the Human Factors Society 31st Annual Meeting.

ZeUweger, P. T. (1989) Scripted documents: A hypermedia path mechanism. In Proceedings of Hypertext.