NASA Etoys In Space Usability Evaluation

Etoys in SpaceHeuristic Review of Usability and Design Issues

byMike Lee

and theOLPC Learning Club DC

May 2009Draft 2

Etoys in Space

• OLPC Learning Club Etoys in Space JamSession

• Post Jam Meeting and Further Analysis• Comments on Re-packaging for OLPC

Deployments• Commentary on the Individual Activities• Next Steps

• Slow load time on some activities• New Etoys menu bar interferes with activity content objects• Text in some text blocks overflows• Animation sprites sometimes run slowly• Layouts for 1024x768 now feel cramped• 1024x768 background images repeat• Some pages on NASA Connect web site don’t fit on XO screen• The Etoys Activities are located here:

http://www.pcs.cnu.edu/~rcaton/SqIndex/squeakindex.html

Initial Observations of NASA Squeak WebActivities Running on OLPC XO

December 2008, Sugar OS 8.2.1, Etoys 3

OLPC Learning ClubEtoys in Space Jam

Etoys in Space Content Inventory CreatedDecember 2008

Posted to: http://www.visuallee.com/etoys

Insights from the Jan. 9, 2009 Conference Callhosted by Nortel

• Randy Caton says it’s pretty easy to fix the NASA Squeak WebActivities. He’ll do it based on feedback logged by the Learning Club.

• But he would not only like bugs noted, he also wants suggestions forimproving overall usability and the interactive learning experience.

• He also wants ideas on how the lessons can be redesigned for use inthe least developed countries (LDCs) where there is much lessclassroom support.

• He provided a link to a site where the activities had been translated intoSpanish.http://www.pcs.cnu.edu/~rcaton/sp/squeakindexSP.html

• The Powerpoint for the conference call is posted on Slidesharehttp://www.slideshare.net/fotogeek/etoys-in-space-conf-call-presentation

Email comments from Kathleen Harness (via Rita)January 12, 2009

Middle/high school? I don't know how much physics and math are assumed so perhaps all the text in the booksis needed, but perhaps not. The cartoon characters look like they will appeal to younger students than thecontent of lessons. Are the lesson goals tightly aligned with specific national standards and can/will teachersuse them during class or as an extra credit activity? Sadly, many teachers I talk to about Etoys are sopressed for time and so focused on testing requirements that they won't use projects that are not preciselywhat they need.

Some objects in the projects are locked and others are not. I leave almost everything unlocked in the spirit ofexploration and to give visitors as much power as possible; break it and open the project again. This will alsohelp with scale-to-fit/full screen choices if someone can just pick up something that is in the way.

The Playing button in the student section led me expect a demo of the lesson. I would change the name of thatbutton to more closely match what students find there.

Two suggestions to free up more space on the screen. You can see examples of both in use atwww.EtoysIllinois.org

• The K-5 Technology Passport lessons incorporate Etoys Help Quick Guides as references in the word docmaterials and in a project flap as a shortcut for people who like a project and want to try it on their own.Anyone trying the project has Etoys open and can click the question mark in the Navigator Bar. Referring tothe guides by name cuts out a lot of repetition of information in documents and lets people choose thematerial they need to read and skip the material they know.

• Flaps take up less space on the screen than books and leave more room for objects, scripts, and buttons.Create flaps with (alt-shift-w). Flaps have an extensive options menu. Name the flap what ever you wish:Directions, Page 1, Challenge 1. They show the project's organization and content in a way that a bookformat does not. Flaps can have text and scripted objects. My students prefer making a flap for their projectslike puzzles and games when they want to provide rules, advice, or puzzle pieces.

Please let me know if I can be of further help, I am very glad your project is going forward.

Regards,Kathleen [email protected]

Etoys in Space Jam Participants

January 17, 2009Arlington Career Center, VA

Participants:• James Crawley• Edward Dale• Lisa Dale• Sarah Elkins• Jeff Elkner• Mike Lee• Pat Paul• Ton van Overbeek

Etoys in Space Jam Session Notes

• Eight participants from the club of varied backgrounds, two withsome with software testing experience

• Realized three hours was not nearly enough time• An initial 45-minute orientation was not enough; Jam

participants had difficulty understanding the Etoys interface aswell as the rationale of some of the activity content

• Participants logged a few bugs• The group had a useful 30-minute conversation on their initial

impressions of the Etoys activities• Jakob Nielsen’s Ten Usability Heuristics were used as a guide

Etoys in Space Jam Session Participant Observations

• Most commonly observed was the inconsistent content designacross the activities

• Noted the extensive amount of reading required on theintroductory web pages and in the lessons

• A home-school parent said she would not use these particularactivities due to the amount of advance preparation apparentlynecessary and unusual Etoys user interface

• Book page turning camera sound was annoying to several• One participant was put off by typos in the activity book text• Animation sprites left trails in some cases (on the XO laptop)

Jakob Nielsen’s Ten Usability Heuristics - 1

1 Visibility of system status The system should always keep users informed about what is going on, through appropriate feedback within

reasonable time.

2 Match between system and the real world The system should speak the users' language, with words, phrases and concepts familiar to the user, rather

than system-oriented terms. Follow real-world conventions, making information appear in a natural andlogical order.

3 User control and freedom Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the

unwanted state without having to go through an extended dialogue. Support undo and redo.

4 Consistency and standards Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow

platform conventions.

5 Error prevention Even better than good error messages is a careful design which prevents a problem from occurring in the first

place. Either eliminate error-prone conditions or check for them and present users with a confirmationoption before they commit to the action.

Source: http://www.useit.com/papers/heuristic/heuristic_list.html

Jakob Nielsen’s Ten Usability Heuristics - 2

6 Recognition rather than recall Minimize the user's memory load by making objects, actions, and options visible. The user should not have to

remember information from one part of the dialogue to another. Instructions for use of the system should bevisible or easily retrievable whenever appropriate.

7 Flexibility and efficiency of use Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such that

the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.

8 Aesthetic and minimalist design Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information

in a dialogue competes with the relevant units of information and diminishes their relative visibility.

9 Help users recognize, diagnose, and recover from errors Error messages should be expressed in plain language (no codes), precisely indicate the problem, and

constructively suggest a solution.

10 Help and documentation Even though it is better if the system can be used without documentation, it may be necessary to provide help

and documentation. Any such information should be easy to search, focused on the user's task, listconcrete steps to be carried out, and not be too large.

Source: http://www.useit.com/papers/heuristic/heuristic_list.html

Post Jam Meeting andFurther Analysis

Randy Caton’s PresentationMarch 21, 2009

Randy was able to attend aLearning Club meeting to presenton his projects and gained someinsights in conversation withmembers who participated.

Further Analysis

• Quick test of Etoys on small-screen netbooks for layout• Captured a large library of screenshots from web site and the Etoys

activities for reference• Created a user experience flow diagram for the GIS activity• Evaluated for consistency of layout and design• Developed wireframe diagrams to show standardized layouts• Suggested additional authoring and development tools

Etoys in Space in on Netbooks

• Netbooks have become a majornew market category withincreased product development forthe education market.

• Content design and technicaltradeoffs will have to be made infurther work on the Etoys in Spaceactivities.

• Asking the user to move activityobjects around to fit a given screenresolution is not ideal.

Netbook Screen Size Chart

This chart is NOT actual size. Please load the original from here:http://farm3.static.flickr.com/2476/3549525335_a6cf3c4402_o.png

Source: http://www.eeepc.it/en/monitor-dei-netbook-dimensioni-a-confronto/

Also see Wikipedia’s comparison of netbooks:http://en.wikipedia.org/wiki/Comparison_of_netbooks

Student Interaction Flow Diagram

• The Student path from the web site through the GIS Challenge activitywas flow diagramed (next slide)

• Multiple web pages must be traversed by the student to gather thenecessary introductory information yet no back and forth navigation isoffered (besides the back button in the browser)

• Just before the student enters the Etoys activity, it is suggested that theindividual make a detour to the Etoys Quickstart Guide

• Once inside the activity, the student relies heavily on 17 pages of bookcontent to complete the five challenges in the activity as well aslearning the script and how to modify it

GIS Challenge - Student Interaction Flow

Read 17 BookPanels

Web ActivitiesIndex Page

WebSite

EtoysActivity

PostActivity

Web ActivityStart Page

Web ActivityPage

(800x600px)

Web ActivityPage

(Students)

NASA ConnectVideo

Content

EtoysQuickstart

Guide

Etoys Tips

CheckBrowser andResolution

Settings

Study ProbeScript

How to Modifythe ScriptChallenge 2

Challenges1 - 5

Challenges1 - 5

Challenges1 - 5

Challenges1 - 5

Reflect &Discuss

Student Interaction Flow

Web Page Task StepLEGEND

Launch Activity

Quit Activity

Etoys in Space Introductory Web Pages - 1

Click

http://www.pcs.cnu.edu/~rcaton/SqIndex/squeakindex.htm l


Click

http://www.pcs.cnu.edu/~rcaton/ESgps/index.html


Click

http://www.pcs.cnu.edu/~rcaton/ESgps/start68.html


http://www.pcs.cnu.edu/~rcaton/ESgps/students68.html

Click

LaunchesEtoy

Activity

Inconsistent Layout Across the Different Activities

Layout Change Recommendations

• Unify NASA logo, activity title and spot artwork into single dismissible flap panel• Place book in upper left corner of all activities• Standardize to stationary, horizontal and vertical action layouts• Keep the Control and Watcher panels together and always near the action area• Group together function buttons, move others into a flap• Keep the existing flaps along the bottom edge of the screen• Gutter space between block objects and groups of small objects needs to be

consistent• Menu bar will stay visible and clear of activity content• Investigate how layouts’ resolution-dependent animation scripts can support

various netbook screen sizes all the way up to OLPC XO 1200 x 900px andlarger

• The wireframe layout diagrams that follow assume that the Etoys in Spaceactivities will be individually accessed via web pages or as clickable file icons

Book

Controls Watcher

StationaryAction Area

FunctionButtons

Wireframe 1 - Stationary Action Layout

ABOUT

BUTTONS

This wireframe is illustrative and not pixel-precise.

Controls

Horizontal Action Area

Book Watcher FunctionButtons

Wireframe 2 - Horizontal Action Layout

ABOUT

BUTTONS


Controls

VerticalAction Area

BookWatcher

FunctionButtons

Wireframe 3 - Vertical Action Layout

ABOUT

BUTTONS


Controls



Wireframe 4 - About Flap Open

ABOUT

BUTTONS

SundialChallenge

NASA Logo -->

Activity Title -->

Spot art -->

Lorem ipsum dolor sit amet,consectetur adipiscing elit.Nunc interdum laciniamalesuada. Donec vitaeneque a mauris auctordapibus. Nunc interdumcursus sem, non elementumleo euismod sed.

Welcome text -->


Controls


Book Watcher

Wireframe 5 - Buttons Flap Open

ABOUT

BUTTONS

Text if needed. -->


FunctionButtons

Lorem ipsum dolor sit amet,consectetur adipiscing elit.Nunc interdum laciniamalesuada.

Controls



Wireframe 6 - Expanded About Flap Open

ABOUT

BUTTONS

SundialChallenge

NASA Logo -->

Activity Title -->

Spot art -->

Lorem ipsum dolor sit amet,consectetur adipiscing elit.Nunc interdum laciniamalesuada. Donec vitaeneque a mauris auctordapibus. Nunc interdumcursus sem, non elementumleo euismod sed.

Welcome text -->


Book containingtext from the introductory

Web site pages.

Book Buttons Violate Fitts’ Law

The time to acquire a target isa function of the distance toand size of the target.

Explanations of Fitt’s Law:http://particletree.com/features/visualizing-fittss-law/ http://en.wikipedia.org/wiki/Fitts's_law

The buttons should be as tall asthe Mac or Windows mousecursor and generally provideample area for the user to landthe cursor and accurately effect amouse click.

Invoke Display of Bigger Book Buttons As Default

We recommend that the default navigation buttons for the booksbe set to the large size. The green text telling the user how to usethe arrow buttons is unnecessary.

A Good Book Design Example

http://www.etoysillinois.org/library.php?sl=356





Example of Buttons Stored in a Flap


Check Color Contrast

To accommodate individuals with colorblindness or other visual impairments contrastbetween adjacent colors should be maintained.For example, some of the flap colors in theRobot Challenge do not have enough contrast.One way to gauge contrast is to print a screen ingrayscale to an inkjet or laser printer or converta screenshot to grayscale.

Suggestions For Generating Fresh Artwork

• Re-create Norbert, Zot and props so that they can bephotographed in dioramas from any number of angles with adigital camera to create new artwork. Possible materials:– Sculpy clay– LEGO Bricks

• Find a student cartoonist or digital artist to re-create the artwork• Better background textures can be photographed with a digital

camera or downloaded as free Creative Commons licensedmaterial from Flickr

One Essential Reference Book

http://www.amazon.com/About-Face-Essentials-Interaction-Design/dp/0470084111/

Screen Ruler Utilities

For Microsoft Windows:http://www.spadixbd.com/freetools/jruler.htm

http://www.microfox.com/

For Mac:http://www.pascal.com/software/freeruler/

These on-screen pixel rulers are very useful to check for consistentspacing between user interface elements.

Comments on Re-packagingEtoys in Space for

OLPC Country Deployment

Re-Packaging Etoys in Space forOLPC Country Deployment

We first recommend embarking on an extensive redesign of thecurrent individual Etoys in Space activities using some of therecommendations presented by the Learning Club.

• The Etoys in Space activities will need to be bundled as a single.XO file into existing laptop activity collections and/or madeavailable for download online as an .XO file, which is acompressed ZIP file

• .PR files for the separate Etoys in Space activities must all belinked to a launcher page with icons for each activity

• The content that prefaces each activity on the web site must beedited for brevity and clarity and bundled into the .PR files

• We recommend deconstructing OLE Nepal’s excellent E-PaathEtoys Activities to develop content, design and implementationspecifications suitable for country deploymenthttp://www.olenepal.org/activities_download.html

OLE Nepal E-PaathScreenshot 1




Commentary onIndividual Activities

Commentary on Individual Activities

A series of reduced screenshots of the starting state of each of theactivities follows. Each has callouts with comments.

• In general, the Escape Browser button implemented in the RatioChallenge and some of the other activities should be deployedthroughout. The additional text label isn’t necessary.

• It is assumed that a standard workaround for the overlap withthe menu bar at the top can be developed.

• Do not use bold for the body text in the books. Subheads aremissing in some books and should be used consistently.

GIS Challenge - Commentary

Why is this the only activitythat starts with a scriptshowing?

The forest map is too smalland is lost on the page. Thesize seems forced smallunnecessarily because it isdesired to illustrate theconcept of pixels.

The tiny probe dot could bereplace with a crosshair.

This page should useWireframe 1 - StationaryAction Layout. The visibilityof the large script panelshould be managed byshow/hide buttons in thebook.

Sundial Challenge - Commentary

The background artwork issimply too busy and becausethe characters are largelyobscured by the foregroundpanels, does not add value tothe user’s experience.

Wireframe 1 Stationary Actionshould be used.

Park and Mail Ball Gravity - Commentary

Wireframe 3 - Vertical ActionLayout should be used hereleaving much more roomaround Zot the dog.

Subtitles should notlook like clickable

buttons.

Subtitles should notlook like clickable

buttons.

Park and Mail Challenge Mass Spring - Commentary

Wireframe 1 should be used here.

Norbesaur Challenge Dino Run - Commentary

Wireframe 2 for HorizontalAction should be used here.

Norbesaur Challenge Dino Track - Commentary

The background isoverpowering. See

next slide for acalmer sand texture

image.

Wireframe 2 for HorizontalAction should be used here.

Norbesaur Challenge Dino Track - Better Sand Texture

Flickr sand texture CC: http://www.flickr.com/photos/tactus/483287393/

Rocket Challenge - Better Rocket Art

Flickr sand texture CC: http://www.flickr.com/photos/mwboeckmann/2425571382 /

Laser Challenge - Commentary

Wireframe 1 for StationaryAction should be used here.Stack the books. The airplaneanimation is gratuitous.

Robot Challenge Mission 1 - Commentary

Avoid the use ofvertical text.

Wireframe 3 for Vertical Actionshould be used here.

Robot Challenge Mission 2 - Commentary

Low color contrastbetween text and

flap color.

30% larger buttonswould make themeasier to click and

provide some spacearound the button

text for betterreadability.


Rocket Launch Challenge - Commentary


The clouds, rocket, gantry andblue ocean look hastily drawn.It seems like it would be easyto replace these with publicdomain imagery from NASA.

The computer-rendered rocketabove the control center panellooks like it could be part of thesimulation. Place it and the titlein the About Flap.

Exercise Challenge - Commentary

Wireframe 1 for StaionaryAction should be used here.

GPS Challenge - Commentary

Wireframe 1 for StaionaryAction should be used here.

Theorem Challenge - Commentary

Wireframe 1 for StationaryAction should be used here.

Freefall Challenge Elevator - Commentary


Titles should notlook like clickable

buttons.

Freefall Challenge Freefall - Commentary


Titles should notlook like clickable

buttons.

Ratio Challenge - Commentary


Cars Challenge - Commentary

Wireframe 1 for StationaryAction should be used herewith the exception that thebook does fit well in the ovalraceway.

Pyramid Challenge - Commentary


Placing Flap at topbreaks from your

own style here.

Graph Challenge - Commentary

Layout looks OK.

Plotting - Commentary

Layout looks OK.

Demo - Commentary

Layout looks OK.

This demo doesn’t seemcompelling enough.

Next Steps

• Further conversation with Randy Caton• A project roadmap?• Funding?

Contact

• Mike Lee curiouslee [at] gmail.com• http://www.olpclearningclub.org

Education

NASA Etoys In Space Usability Evaluation