1

Usability Evaluation on structural navigation and link terms of an interactive web application in eBay E-commerce site

Prof.Francesca Rizzo, Mohammand Jannatul Ferdous, Pintu Chandra Shill, Mohammad Iqbal Mohummud, Lin Liran

Politecnico di Milano

Via Castelnuovo, 7, CO-22100, Como, Italy

rizzo@elect.polimi.it, {ferdous_cs,lpintucskuet,iqbalmahmood2002}@yahoo.com, ullillr@hotmail.com

ABSTRACT Usability is a quality attribute that assesses how easily user interfaces are to be use which is defined by five quality components, learnability, efficiency, memorability, errors and satisfaction. Structural navigation and link terms are the great important role in design of graphical user interface of an interactive application. The eBay site is the ideal example of the best seller eCommerce site world wide. We therefore decided to conduct a Heuristic Evaluation by Jacob Nielsen guideline to find the usability problem of the current site and provide user a new graphical user interface to browse the eBay site with more satisfaction, efficiency and efficient way. This paper presents the usability testing of an interactive application eBay E-commerce site in order to increase the user satisfaction, efficient and efficiency. We identified some problems by conducting Heuristic Evaluation and made hypothesis according to the problems. Then we made a prototype to take the end-user test, in terms of complexity. The results of the experiment represents that our implemented prototype shown less complexity rather than original eBay site in order to performed specific tasks. Moreover, users feel more satisfaction using our implemented prototype rather than using the original eBay site. General terms Design, Structural Navigation, Heuristic Evaluation, Link terms, Experimentation, Hypothesis. Keywords World Wide Web, Usability, Experimental Evaluation, E-Commerce, Electronic Data Interchange.

1. INTRODUCTION The Web application is a navigational system. The basic user interaction is to click on hypertext links in order to move around a huge information space with hundreds of millions of pages. Because the space is so vast, navigation is difficult, and it becomes necessary to provide users with navigational support beyond the simple "go-to" hyperlinks [1].

Figure 1: Tree structure navigation of sample site

Structural links are links that systematically point to other levels of the site structure as well as to siblings or children in a hierarchy. It is important to have the same structural links on all pages so that the user will understand what structural navigation options to expect [1].

2

The structural navigation bar is becoming a ubiquitous element on many websites. This experiment tests whether this type of navigational reference is truly valuable. E-Commerce (electronic commerce) is the buying and selling of goods and services on the Internet, especially in the World Wide Web. In practice, this term and a newer term, E-Business, are often used interchangeably. Electronics Commerce can be between two business transmitting funds, goods, services or data or between a business and a customer.

It is also true to say, however, that eCommerce also includes all inter-company and intra-company functions (such as sales, marketing, accounts, logistics, manufacturing, and negotiation) that enable commerce and use electronic mail, EDI (Electronic Data Interchange), file transfer, digital fax, video conferencing, workflow, or interaction with a remote computer. E-Commerce also includes buying and selling over the World-Wide Web and the Internet, electronic funds transfer, smart cards, digital cash, and all other methods of completing business transactions over digital networks. Our experimental site eBay is the ideal example of an eCommerce site.

The paper is organized as follows: in Section 2 summarizes the functionality. Section 3 describes the rationale and the motivation of our experiment. Section 4 describe and discuss the experimental methodology, conditions, equipment and result obtained from the experiments. Finally, Section 5 & 6 reports research problem and introduces the future works.

2. FUNCTIONALITY We describe about eBay website which is the large online trading company. EBay is the world’s online marketing place which place for buyers and seller to come together and trade almost anything. Its original American website, Now its established in several countries Australia, Austria, Belgium, Canada, China, France, Germany, Hong Kong, India, Ireland, Italy, Netherlands, New Zealand, Poland, Philippines, Singapore, South Korea, Spain, Sweden, Switzerland, Taiwan and United Kingdom. There are some groups of link in the main page of eBay website such that specialty sites, categories, more eBay sites, global site etc.

Figure 2: Screen shot of eBay eCommerce Site

The eBay specialty sites offer you an almost endless choice of items to purchase and many different ways to shop bid and buy. It contains Express, eBay Stores, eBay Motors, eBay Business, Help.com, Apartments on rent.com, StubHub tickets in the left panel menu box with heading specialty sites. With The eBay Express it's faster and easier than ever to breeze through all shopping. You can find the best deals on everything from designer jeans to a complete home theater system by few mouse clicks. Everything is in eBay Express which is offered at a cheapest price ordered by top eBay sellers. You can get what you want now without bidding and waiting. You can go to eBay Stores to visit the online storefronts of experienced, high-volume eBay sellers who provide additional shopping convenience. All important navigation links are present in the home page especially in the top menu and left panel.

The eBay Motors is a special area of eBay that makes it easy to buy and sell vehicles and parts of all types: cars, trucks, SUVs, vans, motorcycles, boats, airplanes, and related accessories by navigating categories. User can buy books, CDs, video games, and much more at set prices. Sellers set prices ahead of time, so you know exactly what you'll pay before you buy.

Half.com sellers are individuals and small businesses who sell directly to you, keeping prices low.Rent.com is the only transaction-based Internet listing website in the apartment and rental housing industry. Its unique business model has made it the number one third-party producer of verified lease transactions in the nation

3

online and offline. The Rent.com service is available to renters and property owners and managers in most American cities, including the nation's top fifty metropolitan markets. So, eBay Motors, Half.com and Rent.com all special sites are inside the left panel group box and only eBay Motors, Express, Stores are in the Top menu with the catagories . Body contains the advertisements in the home page and list of items in the others pages. The following figures is showing the basic structural layout of eBay site.

Figure 3: General site layout of eBay

The main eBay site and most of the special eBay site layout are left align like above figure. Only few special sites (i.e. eBay Motors & Express) are centered align. The eBay site is developed in distributed architecture. It has several web servers with distributed database. For example, www.ebay.it is for the Italian user and www.ebay.com for the American and global user but all sites are using the same database schema from the distributed database. There are lot of database are placed around the world but all are synchronized with each other. User first hit the site by putting the site address in the address bar of explorer and connects to the nearest web server. Web server connects to the distributed database to retrieve the list of possible items or others things such as Italian web server will connect to the Italian DB server or nearest DB server. Here the

following figure is showing the complete conceptual eBay architecture.

Figure 4: Architectural view of eBay

3. THE RATIONALE AND MOTIVATION Our aim was to do usability evaluation on an interactive application. We have choosen the web application as interative application because web is changing our life style and make faster everyday. E-commerce is one of the most important factor in respect to change our life style. E-commerce is continued to be a driving force behind future economic growth worldwide, and important for developing countries. It plays an important role in economic growth for developing countries. It helps countries improve trade efficiency and facilitates the integration of developing countries into the global economy. Electronic commerce only provides all information to the users using E-commerce website. User interaction with the website depends on the flexible, efficient satisfaction use of the website. These usability dimensions depend on the Structural navigation, link consistency, link terms make sense and image have alternate text. So, we motivated to work with link terms and mainly structural navigation of the eBay site application which would be the best for the eBay designer, manager to make their applications more usable for the end-user.

4

4. EBAY USABILITY EVOLUATION THROUGH A CONTROLLED EXPERIMENTATION

4.1 Experiment We have ranked the violation level which violates the usability principles. The table 1 shows the violation ranking. Then we found six problems which violets rule of Jacob Nielsen principles. The table 2 shows the six problem including violation level, principal & suggestions.

Principle Violation Ranking little bit 1 Little-bit more 2 Average 3 More than Average 4 Extreme 5

Table 1: Principle violation ranking

Problem Level Principle Suggestion

P1: In the buying pages, there is no back navigation link when the users want to buy any product.

3 Flexibility and Efficiency of use.

There should be back navigation link in the buying page.

P2: Top structural navigation contains only some links of left structural navigation. Few of them are simple items links of specific domain. i.e. “eBay Motors” is the item link of Special Site Domain of left panel.

2 Consistency and Standards

Top structural navigation should be consistent with the left structural navigation.

P3: Left structural navigation, special sites domain contains different structural design of each navigation link. For example “eBay Express” is centered layout but other sites are not.

3 Flexibility and Efficiency of use

All special sites should be consistent.

P4: There is no difference between Handsets-Wireless and Motorola in the following link though user is visiting Motorola products. Cell Phones & PDAs > Bluetooth Wireless Accessories > Headsets - Wireless > Motorola

4 Visibility of System Status

Cell Phones & PDAs > Bluetooth Wireless Accessories > Headsets - Wireless >

Motorola Motorola link should be little bit highlighted so that user can understand that they are visiting Motorola products.

P5: Each item page of categories does not maintain the left panel layout. Cameras & Photo > Bags, Cases & Straps > For Cameras > Pro, Aluminum, Hard Cases

4 Consistency and standards

Follow the standard of the three panel layout.

P6: The meaning of eBay Motors makes the users confused, since actually it is used to only sell second hand cars.

3 Visibility of System Status

Change the name or make some meaningful text explanation.

Table 2: Problems of the experiment

5

Figure 5: Difference of current site and prototype (P2 & P6)

4.1.1 The Hypothesis

Problem Null Hypothesis Alternative Hypothesis

P1: In the buying pages, there is no back navigation link when the users want to buy any product.

There is no problem to back home page from the each item category of any module for the user.

There is no way to back home with respect to the original eBay site from the each item category of any module for the user. Our implemented prototype improves the user performance. We expect the better performance of the implemented prototype with respect to the original eBay site.

P2: Top structural navigation contains only some links of left structural navigation. Few of them are simple items links of specific domain. i.e. “eBay Motors” is the item link of Special Site Domain of left panel.

Users easily recognize that it is the most important navigation domain of the left structure and there is no significant difference between original eBay sites and our implemented prototype.

Users can not recognize that it is the most important navigation domain of the left structure with respect to the original eBay site and there is a significant difference between original eBay site and our implemented prototype.

6

P3: Left structural navigation, special sites domain contains different structural design of each navigation link. For example “eBay Express” is centered layout but other sites are not.

There is no problem for the user to access different structural layout of different domain of special sites.

It is not feasible & efficient for the user to access different structural layout of different domain of special sites and there is a significant difference between original eBay site and our implemented prototype.

P4: There is no difference between Handsets-Wireless and Motorola in the following link though user is visiting Motorola products. Cell Phones & PDAs > Bluetooth Wireless Accessories > Headsets - Wireless > Motorola

There is no problem to understand the status of the Motorola product page by the user.

Users can not understand easily status of the Motorola product page to see the content. We accept the better performance of our implemented prototype with respect to the original eBay stie.

P5: Each item page of categories does not maintain the left panel layout. Cameras & Photo > Bags, Cases & Straps > For Cameras > Pro, Aluminum, Hard Cases

User do not need left panel to access for the others link.

Users can not access if they want to navigate from left panel of each item and there is a significant difference between original eBay site and our implemented prototype.

P6: The meaning of eBay Motors makes the users confused, since actually it is used to only sell second hand cars.

Users can easily identify that the eBay motor sells only the second hand cars.

Users can not identify that the eBay motors sell only the second hand cars by the eBay Motors navigation.

Table 3: Null and alternative hypothesis of problems

Figure 6: Difference of current site and prototype (P4 & P5)

7

4.1.2 Factor and Block 4.1.2.1 Experimental Factor In our project, Graphical representation (i.e. efficient design of Graphical user interface) can influence the user performance. For this reason Graphical user interface (GUI) is the independent variable of our project and user performance is the dependent variable of our project. 4.1.2.2 Blocks Subject/Education Masters Computer Engineering CM group

Management Engineering MM Group

4.1.3 Subjects Subjects were selected among the Masters students of the computer Engineering and Management Engineering at politecnico di Milano, Italy. 12

subjects were recruited and allotted in to two different groups. Each group contained 12 users, 6 users were from computer Engineering and 6 users were from Management engineering. The two experimental groups did not differ in relation to the expertise on the task domain, they had to perform and their expertise on the use of Web browsers. In our experiment we assigned same subjects in all experimental condition. 4.1.4 Experimental Condition Group Experimental Condition CM group Condition 1: Subjects use

original eBay site. MM Group Condition 2: Subjects use

original eBay site. CM group Condition 3: Subjects use

our implemented prototype. MM group Condition 4: Subjects use

our implemented prototype.

Figure 7: Difference of current site and prototype (P3)

8

4.1.5 Materials One prototype was implemented for the six problems in order to get the user test. 4.1.6 Testing Method We recruited a convenience sample of 12 test participants made up of students at the politecnico di Milano. Twelve participants were female and two were male. All had access to the Internet at university for study purpose, buying and work.

The participants were asked to perform a set of questions at the eBay Web site: www.eBay.com .

In order to verify the significant differences in performance to perform some specific tasks using our implemented prototype and the original eBay site. The CM group was asked to use the original eBay site and MM group was asked to use the original eBay site. CM group was asked to use our implemented prototype and MM group was asked to use our implemented prototype.

Figure 8: Structure of operative stages of the experiment

Questionnaires: Read the following statement carefully. What do you feel about that statement? Give your answer according to your thinking just testing each of them separately: 1. Do you feel any difficulties to back from buying page when you want buy any product items? A. Very difficult B. difficult C. little difficult D. no difficult E. N/A 2. Top menu contains special sites links. Do you feel any problem to navigate other special site from the top menu which is available in the left panel? A. Very difficult B. difficult C. little difficult D. no difficult E. N/A

3. Do you feel any problem to access information from different structural layout according to experimental application (i.e. In the current application “eBay Express” and “eBay Motors” special sites different structural layout but our implemented prototype is same layout for different sites.)? A. Very difficult B. difficult C. little difficult D. no difficult E. N/A 4. Can you easily identify the Motorola product items page to see the link: Cell Phones & PDAs > Bluetooth Wireless Accessories > Headsets - Wireless > Motorola with respect to the link Cell Phones & PDAs > Bluetooth Wireless Accessories > Headsets - Wireless > Motorola A. Very difficult B. difficult C. little difficult D. no difficult E. N/A 5. Do you feel any difficulties to access if you want to navigate from left panel of each item? A. Very difficult B. difficult C. little difficult D. no difficult E. N/A 6. Can you understand easily that eBay motor only sell the second hand car by the respective link text? A. Very difficult B. difficult C. little difficult D. no difficult E. N/A We uploaded our survey questionnaires in online http://www.surveymonkey.com site to take the user test result.

Figure 9: Screen shot of online survey questionnaire

9

We have taken the answer value according to the following table to analyze result. Users Answer Complexity Very difficult 5(A) Difficult 4(B) Little difficult 3(C) No difficult 2(D) N/A 1(E)

Table 4: Users Answer Complexity

We have sent the online questionnaires link (http://www.surveymonkey.com/s.aspx?sm=0TVgqn1Y5RFADvBV5tr_2fgA_3d_3d) to 12 users by email and collect response from the summery section. We also collect it for the individual user result in the table.

Figure 10: Screen shot of survey response summery

4.1.7 The Procedure The experiment was performed on the manipulation of the independent variable (Design criteria i.e. Graphical user interface) and on the measurement of the dependent variable values (user satisfaction). We assigned our implemented prototype and original eBay site to each participant and ask to perform a some specific task and participant performance is evaluated using some questionnaires.

4.1.8 Preparation Our subjects are the 12 students of Politecnico di Milano in the department of Computer Engineering and Engineering Management. We prepared our participants to give the experimental product i.e eBay websites and its wed address is www.eBay.com. At first our participants browse this eBay website if they are not familiar to this website. We gave our implemented prototype and original eBay website and collect data from the participants using online questionnaires. 4.1.9 The Results Our experimental hypothesis was verified using the collected data from the four groups.

User/Problem P1 P2 P3 P4 P5 P6Anup Singh A C D B D A Shah Reza C C C D D A Sun jin B B C D C A Jinagi taw A B B C C A Fuyin liang C C C C D A Liang B A B C D A Total Complexity

24 22 19 17 14 30

Average Complexity

4 3.7 3.1 2.8 2.3 5

Table 5: User test complexity Result of the Original eBay site (MM Group)

User/Problem P1 P2 P3 P4 P5 P6Anup Singh D D D A D D Shah Reza D D D A D D Sun jin D D D B D D Jinagi taw C D C B D D Fuyin liang D D D A C D Liang D D C D D D Total Complexity

13 12 14 25 13 12

Average Complexity

2.2 2 2.3 4.2 2.2 2

Table 6: User test complexity Result of the Our Proposed Prototype (MM group)

10

User/Problem P1 P2 P3 P4 P5 P6 Yucing A C D D D A Nazrul B C D E D A Anna A B E C B A Balajit A B D C C B Mohammad A B B B C A Sumon B B D D C D Total Complexity

28 22 13 15 17 26

Average Complexity

4.7 3.7 2.2 2.5 2.8 4.3

Table 7: User test complexity Result of the Original eBay site (CM group)

User/Problem P1 P2 P3 P4 P5 P6Yucing D D D A D B Nazrul C C D E D B Anna D D D D C D Balajit D C E D D D Mohammad D D D D D D Sumon D B D D C D Total Complexity

13 16 11 14 14 16

Average Complexity

2.2 2.7 1.8 2.3 2.3 2.7

Table 8: User test complexity Result of the our implemented prototype (CM group)

Result Analysis We have analyzed the user test data by using statistical test (t-test). In t-test, our calculated result is tcalc and tabulated result is ttab and we compared the tcalc and ttab in order to decide acceptance and reject the NULL hypothesis. Numerical Analysis: Fig.11 shows the average complexity when subjects browse the original eBay site and our implemented prototype with respect to the specific problem. For example, In problem1 original eBay site browsing complexity is 4.4 and our implemented prototype complexity is 2.2 which shows that our implemented prototype given better performance rather than the original eBay site.

Difference of average difficulties between original and prototype eBay

00.5

11.5

22.5

33.5

44.5

5

Prob-1 Prob-2 Prob-3 Prob-4 Prob-5 Prob-6Problem

Diffi

culty

Original

Prototype

Figure 11: Graphical chart of average complexity

Statistical Analysis: Problem 1: In MM group tcalc(4.5675) > ttab(2.22814). The mean of Group One minus Group Two equals 1.83. 95% confidence interval of this difference: From 0.94 to 2.73. For this reason we reject the NULL hypothesis and accept the alternative hypothesis. In CM group, tcalc(9.3026) > ttab(2.22814). The mean of Group One minus Group Two equals 2.50 95% confidence interval of this difference: From 1.90 to 3.10. For this reason we accept the experimental hypothesis and reject the NULL hypothesis. In the problem1 we actually accept the experimental hypothesis for CM and MM group and there is a significant difference between original eBay site and our implemented prototype. From the users point of view and our hypothesis test that our implemented prototype is superior to the original eBay site. And then we combine the user test of CM group and MM group and test the statistical hypothesis. In the combination of CM and MM group the tcalc(8.6232) > ttab(2.81876). The mean of Group One minus Group Two equals 2.17. 95% confidence interval of this difference: From 1.65 to 2.69. for this reason this difference is considered to be extremely statistically significant. From our statistical Analysis we finalize that we accept the experimental hypothesis and reject the null hypothesis that our implemented prototype shown the better performance rather than the original eBay site.

11

Problem 2: In MM group tcalc(5.0000) > ttab(3.16927). The mean of Group One minus Group Two equals 1.67. 95% confidence interval of this difference: From 0.92 to 2.41. For this reason we reject the NULL hypothesis and accept the alternative hypothesis. In CM group, tcalc(2.5355) > ttab(2.22814) The mean of Group One minus Group Two equals 1.00. 95% confidence interval of this difference: From 0.12 to 1.88. For this reason we accept the experimental hypothesis and reject the NULL hypothesis that there is a significant difference between the original eBay site and our implemented prototype. For problem2, we combine the user test of CM group and MM group and test the statistical hypothesis. In the combination of CM and MM group the tcalc (5.0143) > ttab(3.16927). The mean of Group One minus Group Two equals 1.33. 95% confidence interval of this difference: From 0.78 to 1.88 and the standard error of difference = 0.266 with df = 22. For this reason there is significant difference between the original eBay site and our implemented prototype. Finally for problem2 we accept the experimental hypothesis and reject the NULL hypothesis that our implemented prototype shown the better performance rather than the original eBay site. Problem 3: In MM group tcalc(2.2361) > ttab(1.812461). The mean of Group One minus Group Two equals 0.83 95% confidence interval of this difference: From 0.00 to 1.66. For this reason we reject the null hypothesis and accept the alternative hypothesis. In CM group, tcalc(0.7670) > ttab(0.260185) The mean of Group One minus Group Two equals 0.33 95% confidence interval of this difference: From -0.64 to 1.30. For this reason we accept the experimental hypothesis and reject the NULL hypothesis that there is a significant difference between the original eBay site and our implemented prototype. But user complexity analysis of this problem, there is little bit better performance shown in our implemented prototype with respect to the original eBay site. We combine the user test of CM and MM group and performed statistical analysis that tcalc(1.8184) >

ttab(1.321237). The mean of Group One minus Group Two equals 0.58. 95% confidence interval of this difference: From -0.08 to 1.25. For this reason there is significant difference between the original eBay site and our implemented prototype. Finally, statistical analysis and complexity analysis we shown that our implemented prototype is little bit better than the original eBay site. Problem 4: In MM group tcalc(2.3489) > ttab(1.812461). The mean of Group One minus Group Two equals -1.33 95% confidence interval of this difference: From -2.60 to -0.07 For this reason we reject the NULL hypothesis and accept the alternative hypothesis. In CM group tcalc(0.2370) > ttab(0.130185). The mean of Group One minus Group Two equals 0.17. 95% confidence interval of this difference: From -1.40 to 1.73. For this reason we reject the null hypothesis and accept the alternative hypothesis. We combined the user test of CM group and MM group and test the statistical hypothesis. In the combination of CM and MM group the tcalc(1.1342) > ttab(0.685805). The mean of Group One minus Group Two equals -0.58. 95% confidence interval of this difference: From -1.65 to 0.48 for this reason this difference is considered to be extremely statistically significant. Finally we performed that our implemented prototype shown the better performance rather than the original eBay site. Problem 5: In MM group tcalc(0.6202) > ttab(0.260185). The mean of Group One minus Group Two equals 0.17. 95% confidence interval of this difference: From -0.43 to 0.77 .For this reason we reject the NULL hypothesis and accept the alternative hypothesis. In CM group tcalc(1.3416) > ttab(0.699812). The mean of Group One minus Group Two equals 0.50. 95% confidence interval of this difference: From -0.33 to 1.33. For this reason we reject the NULL hypothesis and accept the alternative hypothesis.

12

We combined the user test of CM group and MM group and test the statistical hypothesis. In the combination of CM and MM group the tcalc(1.4306) > ttab(1.372184). The mean of Group One minus Group Two equals 0.33. 95% confidence interval of this difference: From -0.15 to 0.82. For this reason this difference is considered to be extremely statistically significant. Problem 6: In MM group, we can not analyze the perfect statistical test. Our implemented prototype complexity is 2 and the original prototype complexity is 5. For this reason, we performed that our implemented prototype shown the better performance rather than the original eBay site. In CM group tcalc(2.5649) > ttab(2.22814). The mean of Group One minus Group Two equals 1.67. 95% confidence interval of this difference: From 0.22 to 3.11. For this reason we reject the NULL hypothesis and accept the alternative hypothesis. The result of statistical test of combined CM and MM group: tcalc(6.8461) > ttab(3.16927). The mean of Group One minus Group Two equals 2.33. 95% confidence interval of this difference: From 1.63 to 3.04 for this reason this difference is considered to be extremely statistically significant. Finally, we analyzed that our implemented prototype shown the better performance rather than the original eBay site. 5. RESEARCH PROBLEM We faced the following problems when we take our experiment:

Some users were unknown to use eBay website. Most of the user did not understand the questions

properly to find difficulties. User felt confused about the very difficult and

difficult context of the research methodologies questionnaires.

6. FUTURE STEPS In this report, we have analyzed the heuristic evaluation using Nielsen principles on Structural Navigation mainly. We think that future steps would be

To analyze the heuristic evaluation using the link terms make sense and the images alternate text.

Future step can also have to compare internal site structural navigation for usability testing.

7. CONCLUSION The original eBay website does not provide user satisfaction, efficient user browsing functionalities according to our experiment and result analysis. Our implemented prototype improves the user satisfaction, browsing functionalities by correcting and adding some user friendly functionalities with respect to the original eBay website and our experimental result have shown that our implemented prototype is better than the original eBay website. Finally, our implemented prototype removes some usability problems of the original site and improves user interfaces which would be the example for the eBay designer. 8. REFERENCES [1] http://www.mariosalexandrou.com

/definition/ecommerce.asp [2] http://mashable.com/2006/04/26/ebay-express-ebay-

made-easy [3] http://www.ebay.com [4] http://www.wdvl.com/ Authoring/ Design/ Usability/

use3_1.html [5] http://www.otal.umd.edu/SHORE2000/webnav/

introduction.html [6] Noah Lazar and Michael Eisenbrey °Website

Structural Navigation°, Department of Computer Science, University of Maryland, 2000.

[7] Angela Colter, Kathryn Summers, Cheri Smith,

“Exploring User Mental Models of Breadcrumbs in Web Navigation, University of Baltimore, School of Infrmation Arts and Technologies.

[8] http://www.graphpad.com/quickcalcs /ttest1.cfm