Embed Size (px)
Citation preview
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
What does getting WET(Web Event-Logging Tool)Mean for Web Usability?
Michael P. Etgen & Judy Cantor
User Experience Engineering Division
AT&T Labs
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
Web Data Collection Approaches
Traditional Usability Data Collection:– Hand coding in person or from video– Advantages: ask questions, get subjective responses,
see exactly what happened
– Disadvantages: expensive, time-consuming, can introduce human error, may require extra equipment
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
Web Data Collection Approaches Server Logs
– Servers automatically log user requests and server responses
– Advantages: automatic and inexpensive
– Disadvantages: • no data collection on Browser buttons/menus• can’t log client-only interactions (e.g., anything that doesn’t
send a request to the server) • validity of data - dynamic IP addressing and browser/proxy
caching can provide misleading information• huge amount of data to parse
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
Web Data Collection Approaches Automated Data Collection Software
– Software runs in addition to the browser, or “logging browser” developed
– Advantages: automatically collects wide range of data at Operating System/Application level
– Disadvantages:• cost of buying or developing software• installation of software on each users’ client, and retrieval of
data from each client• platform dependence
• can’t get content-specific data
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
Web Data Collection Approaches WebVIP from NIST
– Copy site, add inline event handling code to each link
– Advantages: client platform independence, data is content-specific, inexpensive
– Disadvantages:• requires copying entire site, and altering code for each link
• no data collection on Browser buttons/menus (use inferred)
• won’t work if Javascript is disabled
• no data collection on HTML elements other than links (e.g.,
checkboxes, radio buttons, selection lists, etc.)
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
Web Data Collection Approaches
Server Side
(1) Browser generates events in response to user interactions with web page: click, load, change, mousedown, etc.
(2) Javascript can temporarily log event time, type, and source info on Client
server responses
client requests
Internet
(3) Javascript can format event log and send it back to server
An Alternative: Web Event-Logging
Client Side
(1)
(2)
(3)
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
Web Data Collection Approaches Web Event-Logging Tool
– Advantages:• content-specific, client-side data collection• client / server platform independence• code is modular, so easy to insert/delete, does not require copied site
or adding code to individual HTML elements• you choose events/pages/features/etc. to log, and only log those• inexpensive
– Disadvantages:• won’t work if Javascript is disabled• no data collection on Browser buttons/menus (use inferred)• Solution for getting data back is specific to site/application
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
Web Data Collection Approaches Web Event-Logging Tool
– Examples of events that can be logged and the sources of the event:• load, unload, move, resize - window• click - link, area, button, radio, checkbox, reset, submit• mousedown, mouseup - link, button, radio, checkbox, reset,
submit• change - text, textarea, select• keypress - text, textarea, password• mouseover, mouseout - link, area• blur, focus - window, text, textarea, password, select
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
Implementation in a Web App Background: Planning and Coordination
tool for Executive Council– Functionality: Meetings, Action Items, Meeting
Minutes & Presentations, Confirm Availability, Contact Members
– Technology: ColdFusion and MSAccess database
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
Implementation in a Web App Configuring WET
– Basic Steps for Any Site/Application• Specify events and temporary storage method• Place a copy of WET.js on web server• Insert call to WET.js in all document headers• Specify method for log retrieval
– Additional Steps for current trial• Code for turning WET on/off based upon user login• Code for Start/Stop links, created as dynamically
positioned layer so won’t scroll off screen
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
Implementation in a Web App
Application Before WET
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
Implementation in a Web App
After WET is configured and running
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
Implementation in a Web App Method
– Study directions, task list, and background information were sent to participants by e-mail
– Testing was remote, no direct contact between testers and participants while performing tasks
– Four participants completed trial– 7 Tasks covered all the basic functionality of the
application
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
Implementation in a Web App Method
– Metrics and Objectives• Three metrics were calculated for each task and
compared to usability objectives– Task Completion - examined page access information in
log– Time to Complete Task - subtracted time for click on stop
link from click on start link – Deviation from Ideal Interaction Model - comparison to
expert path
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
Implementation in a Web App Example from Trial Tasks
– “Contact a specific council member (e.g., Judy Cantor) and send an e-mail to them indicating that you are finished with the trial.”
– Metrics and Objectives• Task Completion - 3 out of 4 successfully complete task• Time to Complete Task - 3 out of 4 complete task within 120
seconds.• Deviation from Ideal Interaction Model - 3 out of 4 deviate from
ideal no more than twice
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
12:46:44.242, load, home.cfm12:46:49.726, click, Link:javascript:task(‘start’), home.cfm12:46:54.867, click, Link:contact.cfm, home.cfm
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
12:46:57.570, load, contact.cfm12:47:04.258, click, checkbox=user=mike, contact.cfm12:47:07.898, click, submit=Compose Mail, contact.cfm
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
12:47:08.103, load, mail_form.cfm12:47:10.346, change, text=subject, mail_form.cfm12:47:17.582, change, text=message, mail_form.cfm12:47:35.957, click, submit=send, mail_form.cfm
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
12:47:39.586, load, contact.cfm12:47:42.961, click, Link:javascript:task(‘stop’), contact.cfm
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
Implementation in a Web App Results for Example E-mail Task
– Task Completion: 4 out of 4 successfully completed the task
– Time to Complete Task: 3 out of 4 successfully completed the task in less than 120 seconds.
– Deviation from Ideal Interaction Model: 4 out of 4 had no more than 2 deviations
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
Implementation in a Web App Lessons Learned
– Showing companion start/stop graphics was not reliable method for indicating task start
• participants couldn’t recall if they’d clicked “start”, so we had to use first click on menu item on home page.
• all remembered to click“stop” when finished though.• Possible solutions:
– Place “start” in dialog window on top of home page so the user cannot proceed without clicking it.
– Change appearance of start after it has been clicked
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
Implementation in a Web App Lessons Learned, continued
– Think about what needs to be logged carefully• Amount of data can be overwhelming
– For Usability Testing: choose the minimum number of events that reflect how site is being used
– For Usability Monitoring: log only certain pages, or certain functionality
– Think about temporary storage method in terms of site/application architecture
• some possibilities are session-only cookies, frame variables, variables in parent window, etc.
• best to use method which least alters site/application
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
Future Directions To Do List:
– Apply the technique to other web-relevant technologies that use events (e.g., Java applets/applications), and integrate that data with the Javascript WET data
– Investigate whether generalized configuration modules are possible for certain “types” of sites/applications, so implementation isn’t as site/application specific
– Make code publicly available for free to anyone who wants to use it
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
Future Directions Wish List:
– If we can create generalized configuration modules, then we could create a wizard for installation and configuration
– For a basic set of metrics, create an analysis wizard for data summarization and analysis
What does getting WET Mean for Web Usability?
HFWEB June 3, 1999
Conclusion The technique of Web Event-Logging (as
embodied in WET) provides the usability engineer with a method of collecting web usability data that is automated, customizable, and comprehensive.
What does getting WET (Web Event-Logging Tool) Mean for Web Usability?– It means you cancan gather usability data on a
site/application within the limited time usually afforded to such activities on web projects.
