Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
1
Some Findings from WorldCat Local Usability Tests Prepared for ALA Annual, July 2009
Test goals 2
Test methods 2
Test partners 2
Combining local, group, and global scope 2
Journal article content 2
Simple search box 2
Advanced search options 2
Search result pages 3
• Orderofsearchresults 3
•What“relevance”means 3
• Theinterplayoflocationandrelevance 3
• Detailsonsearchresultpages 4
• EditionsandFRBR 4
• Facetedbrowsing 5
Item Details pages 5
• Gettingprintitems 6
• Linkstoelectronicresources 6
• Rearrangementofhigh-value 7 bibliographicdata
Collaboration and personal workflow tools 7
• Reviews 7
• Recommendations 7
• Tags 7
• Ratings 7
• Lists 7
Navigation 8
213941
WorldCatLocalprovideslocallybrandedaccessthroughasimplesearchboxandadvancedsearchoptions
tolocal,groupandglobalholdings,print,non-printandelectronicresources(includingfulltextfrom
electronicsubscriptions)foracademicandpubliclibraryusers.Searchresultsincluderelevanceranking,
FRBR-basededitionclustering,andfacetedbrowsing.Itemdetailsincludecollaborativeandworkflow
features:lists,tags,reviews,andratings.
OverthepasttwoyearsOCLChasconductedclosetoadozentestsofWorldCatLocal,mostwithacademic
libraryusers,somewithpubliclibraryusers,andmanyincooperationwithpilotlibraries.Thissummary
assemblessomekeyfindingsfromourongoingtestingprocess.
Contents
2
Test goalsOftenourusabilitytestsaresummative:thegoalistovalidatedesignsorchangeswe’vemade.Sometimesourtestsareformative:thegoalistoprovokeorcorrectourthinkingaboutanemergingapproachordesignatapreliminarystage.
Test methodsWe’vedoneusabilitytestsinanumberofways:oftentask-basedsessionswithprototypeorproductionsystemsfollowedbyquestionnairesandinterviews;sometimescontextualinterviews,wherethetestparticipanttrainstheobserver.Someofthesewererelativelylargetests,withapproximatelyfifteenparticipants,andothersweresmalltestswithonlyfiveparticipants.Oftenweconducttestsatcolleges,universitiesorpubliclibraries.Sometimestheuseristhereorathome,butweconductthetestfromOhioorCalifornia.
AnagiledevelopmentmethodologyisusedforWorldCatLocal:wedesign,test,pilot,test,design.Itisaniterativeandongoingprocess.
Test partnersWedothesetestsincollaborationwithstaffatpilotsiteswhoareoftenveryinvolvedinplanningthetestsandanalyzingthefindingswithus.TestpartnershaveincludedtheUniversityofWashington,theUniversityofCalifornia(Berkeley,DavisandIrvinecampuses),theOhioStateUniversity,thePeninsulaLibrarySysteminSanMateo,California,theFreeLibraryofUrbanaandtheDesPlainesPublicLibrary,bothinIllinois.LaterinJulywe’llbetestingwithNortheasternIllinoisUniversityandtheUniversityofIllinois,Springfield.
Usabilitytestslikethesehavesomeadvantages.Weseewhatpeopleactuallydo(inanartificialsetting)ratherthanhearwhattheysaytheywoulddo.Wecanprobeinteractively,andgetparticipantstoreflect.Wecanobservebehavioracrossdifferentinstitutionsandovertime.
Thesetestshavesomelimitationstoo.Oursamplesareinsomewaysdiverse:theycomefromdifferentinstitutions,buttheyarenotreliablyrepresentative.They’rejustlooselyalignedwithourdevelopingpersonasandprofilesforWorldCatLocalbyrank,discipline,ageandgender.Testparticipantsarepredominantlyfromthesocialsciencesandhumanities;afewhavebeenfromthesciences,engineeringandnursing.
SurveysandfocusgroupsthatcangiveaviewofbroaderandmorereliablyrepresentativesamplesthantheusabilitytestssummarizedherearealsoconductedbyvariouspartsofOCLC.
Combining local, group and global scopeAcademicusershaveastronglyfavorableresponsetosearchinglocal,groupandworldwidecollectionstogether.
Forsomepubliclibrarypatrons,localmeansthisparticularbranch,nottheirwholecitysystem.
Publiclibrarypatronsareinterestedinwhat’s“nearby”thattheycanget,withoutalwaysrecognizingtheinterlendingpartnershipthatsupportsit.
Journal article contentBothundergraduateandgraduatestudenttestparticipantsrecognized,understoodandwelcomedthebasicconceptofcombiningarticlesformdifferentsourcesandcombiningthemwithbooks.Theyrepeatedlymentionedtheinclusionofjournalarticlecontentassomethingtheyvaluedhighly.However,mostacademicparticipantsinonetest(nineoffourteen)wronglyassumedthatjournalarticlecoverageincludesallthelicensedcontentavailableattheircampuses.
TheadditionofmorelicensedcontenttoWorldCatandsupportforfederatedsearchingofdatabasesoutsideWorldCatwillenlargethesetofsourcestosearch.Wehavetestedmetasearchprototypes;however,thoseresultsarenotincludedhere.Metasearchingandmulti-databasesearchinginasingleinterfaceintroduceusabilitychallengesandneedfurthertesting.
Simple search boxFacultypraiseditas“Googly.”Veryfewparticipantsdemonstratedanyconcernaboutwhethersurnamesshouldprecedeforenames.Repeatedly,academicusersexpressedappreciationforasearchboxwheretheycan“justtypeanything.”Mostfaculty,studentandpubliclibrarypatronsuseditoften,searchingfromwhateverpagetheyhappenedtobeoneitherwithinWorldCatLocaloralibrary’shomepage.
Advanced search optionsInatestwith14participants(sevenundergraduatesandsevengraduatestudents)onlyhalfofparticipantseverusedtheadvancedsearchscreen,andinfewerthanhalfofthetotalsearches—ninetimesin66searches.
Whenparticipantschosetouseadvancedsearch,theyoftenmentionedadesiretoavoidatoo-largeresult.Inthisandotherteststheystatedapreferenceforsearchingbyauthor
Some Findings from WorldCat Local Usability TestsPrepared for ALA Annual, July 2009
3
toavoidworksaboutaperson,ortitletoavoidfindingcommontitlewordsassubjects.
Whenjournalarticlecontentwasinvolved,academictestparticipantsoftenmentionedadesireforajournaltitleindex.Wehavenowbuiltthatjournaltitleindex.
Wehavealsomadesomelanguageadjustmentsforacademicusers,saying“journal”insteadof“serial,magazine,newspaper.”
We’vecausedsomeproblemswithpromptedvalues,andsolvedsomeproblems.Earlyonwemadesearchtermspersistonadvancedsearchscreen(becausewebsearchengineshavetheuser’ssearchtermspersistintotheadvancedsearchform),butreceivedmanycomplaints.Nowthosetermsarenotpersistent.Wearealsolookingatalternativesforrevisingasearchorstartinganewsearch.
Search result pagesWehadquestionsandgotfindingsabouttheorderofsearchresults,editionsandFRBR,whatdetailsweshow,andfacets.
Search results, May 2007
Search results, July 2009
Order of search results
Weinvestigatedwhatthedefaultordershouldbeinanacademicsetting,whetheritdiffersbetweenadvancedresearchersandundergraduates,whatthedefaultordershouldbeinapubliclibrarysetting,howwellthedefaultrankingsupportsfindingknownitems,andhowwellitsupportstopicalsearching.
Wefoundthatbothforstudentsandforscholars,inboththeknownitemcaseandthetopicalsearchcase,theexpectedandpreferredorderwasrelevance.Nearlyallparticipantsexpectedit;alargemajoritypreferredit.Asmallminoritypreferreddatedescending,andthoseparticipantswereabletochangethesortorder.MostacademictestparticipantsrecognizedtheorderofsearchresultsinWorldCatLocalas“relevance.”
Forpubliclibraryusers,itisnotyetclearfromourtestingwhattheexpectedorderis.Somepubliclibrarypatronshadahardtimedeterminingwhatordersearchresultswerein.
Whensearchingforknownitems,testparticipantsexpectedtheitemtobeonpageoneofthesearchresults.Whenitwasn’t,themostcommonbehaviorwastosearchagain,usingmorewordsorquotationmarks.
What “relevance” means
Fortopicalsearches,testparticipants—advancedresearchers,facultyandgraduatestudents—statedthatthe“best”itemswouldbeatthetopofthelist.Thisis“relevance”inaspecialsense,onethatincludessomethinglikerenown.
Academictestparticipantsdemonstratedmoretrustthancuriosityabouthowthismightbeaccomplished.Mostfacultyandgraduatestudentscouldsuggestseveralfactors—relevance,popularity,andlocation—thatmightbeinvolved.Butattentiontothatwassomethingtheysaidtheyregardedaslibrarians’expertise,nottheirs.Theytrusttheirlibrariestohavemadetherightchoicesabouthowtodothisforthem.
Someparticipantsremarkedwithsurpriseonthedesireditemnotbeingfirstinsearchresultsorbeingprecededbyanitemwheretheycouldn’tseethematchingtermsinthebriefsearchdisplay(becausetheywereincontentsnotesorelsewhere).Thisranking,however,wasnotusuallyanobstacletoparticipantsgettingthedesireditem.
The interplay of location and relevance
Thedefaultresultrankinginourfirsttestswasasubtleone.Localholdingswerelistedfirst,goingfrommosttoleastrelevant,followedbygroupholdingsgoingfrommosttoleastrelevant,followedbyworldwideholdingsgoingfrommosttoleastrelevant.
Some Findings from WorldCat Local Usability TestsPrepared for ALA Annual, July 2009
4
Undergraduatetestparticipantsgenerallyreportfeelingwellservedbythisrankingthatputsitemsfromthelocalinstitutionatthetopofthelist.Participantsweregenerallynotawarethatitemsfromthatcollectionwerepromotedtothetopofthelist.
Facultyandgraduatestudenttestparticipantsgenerallydidnoticethatlocalmaterialswereatthetopofthelist.Buttheygenerallydidnotanticipatethatrankingbybothlocationandrelevancewouldunpredictablyputhighlyrelevantitemsoutofsight,afterlessrelevantones.Thiswasaconcernforscholarlytestparticipantswhowereoftennotinterestedonlyinlocalmaterials.
Inresponse,wecreatedasortoptionforrelevancewithoutlocation,stillweightedbyholdingsbutwithoutlocal,groupandglobalgrouping.Nowthatoptioncanbesetbyaninstitutionasthedefault.Atthesametime,weaddedamoreovertcontrolforlocation:apull-downmenunearthesearchbox.Insubsequenttests,someundergraduateparticipantsusedthepull-downmenutoscoperesults.Forthosewhodid,itdidwhattheyexpected.Allgraduatestudentsandfacultyusedthiscontroloftenandproficiently.
Details on search result pages
Manyparticipants,frombothacademicandpubliclibraries,statedthattheyexpectedtoseetheirsearchtermsintheitemsatthetopofthesearchresultpage.Matchesintitlesarevisible,butmatchesinsubjects,whichhappenoftenandcanbeequallyimportant,arenotvisiblenow.Testparticipantsexpectedtheevidenceofmatchingtobevisible,andtendedtodisregarditemsinwhichtheirsearchtermswerenotvisible.
Testparticipantsatpubliclibrariesoftenneededtoseeholdingsforthelocalbranch,notthewholecitysystem,onthesearchresultspage.Loadinglocalholdingsrecordsisaremedywhenthisisaproblem.
Anindicationofwhat’savailableisabasisforselection.Severalparticipantsinvarioustestsexpressedapreferencetoseeanindicationofcirculationstatusonthesearchresultspage.However,nearlyalltestparticipantswereabletodeterminewhetherlocallyheldmaterialswereavailablebyexaminingtheitemdetailspage.
Manyparticipantsexpressedadesiretoseeanabstractorsummaryonthesearchresultspage.Thisisachangeweareevaluatingnow.
Authorswereanimportantbasisforselectionforadvancedscholars.Facultyandgraduatestudentsindicatedthatrecognizingareputableauthorintheirfieldwassomethingtheyusedtodecidewhetheraniteminasearchresultshouldbelookedintofurther.Noneoftheundergraduate
studentsinourcontextualinterviewsindicatedthiswasafactorintheirselectionprocess.
Editions and FRBR
Weinvestigatedwhethertestparticipantsweremisledbyseeingonlyonemanifestationonsearchresultsrepresentingallothermanifestationsofthatwork;whetherweshowtherightmanifestationonsearchresultpages;whetherparticipantswereconfusedbyseeingmorethanonemanifestationonsearchresultpageswhenFRBRizationisimperfect;whetherparticipantsgetthelatesteditionofaworkwhentheyknowtheyneedthelatestedition,butdon’tknowtheyearofthatedition.
Wefoundthatingeneralwork-levelgranularityforsearchresultsbasedonFRBRratherthanedition-levelgranularityfittheexpectationsofbothacademicandpubliclibrarytestparticipants.Evenscholarshaddelegatedattentiontowhicheditiontostartwithtous.Puttingthedetailsofoneeditioninsearchresultsdidn’tstopstudentsorscholarsfromfindingothereditionswhentheywantedthem.IncompleteFRBRizationhasanegativeimpactonperformance,notunderstanding.
Undergraduatetestparticipants,primarilyinthehumanitiesandsocialsciences,reportedthattheyseldom(sevenparticipantsoftenparticipants)ornever(threeparticipants)lookedforaspecificeditionofabook.
Werepeatedlyheardeditionssometimesmattertoscholars—thelast,thefirst,thelastduringtheauthor’slifetime.Historiansexpressedapreferenceforthefirsteditionofprimarysourcesandthemostrecenteditionofsecondarysources.
Wefoundthatthedefaulteditionmatters.Evenadvancedresearchersacceptedthedefaultwhennotdirectedtofindaspecificedition,withoutrecognizingthatitwasnottheeditiontheylaterdescribedaspreferring(i.e.,theearliestorlatestedition).Wearenowinvestigatingwhetherthisdefaultshouldbethemostrecenteditionthat’savailablelocallyorthemostwidelyheldeditionthat’savailablelocally.
Whenparticipantsdidhaveaspecificeditioninmindandhadinformation(publisher,date,editor)toidentifyit,theygenerallysearchedwiththatinformationandsawthecorrespondingeditioninthesearchresult,sinceweshowthematchingedition.Nearlyallacademictestparticipants—bothundergraduatesandgraduatestudentsorfaculty—havebeensuccessfulatthissortoftask.
However,inourfirsttestsnoundergraduatesorgraduatestudentsweresuccessfulatfindingthelatesteditionofaworkiftheydidn’talreadyknowthedate.Thistaskisoneinstanceofidentifyingaparticulareditionwithoutknowingin
Some Findings from WorldCat Local Usability TestsPrepared for ALA Annual, July 2009
5
advanceanywordsthatcouldbeusedtofinditbysearching.Performancewasalmostuniformlypoorforfaculty,too.
Weaddeda“viewalleditionsandformats”link,firstatthetopoftheitemdetailspage,andthenalsoonthesearchresultspage.Afterthat,undergraduatesaswellasadvancedresearchersgenerallyfoundthelatesteditionwithoutknowinginadvancetheyearofthatedition(evenwhen,asoftenhappens,thateditionisnottheoneshowninbriefsearchresults).
Mostadvancedresearchersexpressedadesiretohaveallavailableeditionslisted(ontheeditionstab,asitthenwas)inreversechronologicalorderwithoutregardtolocation.Publiclibrarypatronsdidn’texpecttoseenon-Englisheditions,especiallynotatthetopofthatlist.Wecreatedaneweditionspage,whereuserscansortbydateandlocation,refinebyformat,seelanguagesandscopebylocation.Wechangedsortofeditionstoremovegroupingbylocal,thengroup,andthenglobalholdings.Weareconsideringaddingmoreeditioninformationforhighlyused/searchformatsandeditionsontheitemdetailspage.
Faceted browsing
Weinvestigatedwhetherfacetsarenoticedbytestparticipants,whethertheyworkasexpected,andwhetherthereareanyobstaclestouse.
Participantssaidanddemonstratedthattheynoticedfacets.Theysometimesusedfacets,andfacetsgenerallyworkedasexpected.
However,facetsweremoreoftenpraisedthanused.Insubjectsearchinginparticular,facetswerenotoftenusedbyundergraduateorpubliclibrarytestparticipants.Threeoutoftenusersinatestofsubjectsearchingusedfacets.Thefacetstheyusedwerelanguage,formatandyear.Participantsinthistestdidnotusethesubjectfacet(thencalledContent).Whenaskedwhy,severalsaidtheycouldn’tbesurewhattheywouldbeeliminatingiftheydid.
Subjecttermsatfirstappearedinafacetlabeled“content.”Nowthatfacetislabeled“topic.”Weaddedawaytoreversefacetchoicesundereachfacet.Wechangedtheorderoffacetstoputthemore-often-usedonesclosertothetopofthepage.
Item details pages
Item details, May 2007
Item details, July 2007
Item details, July 2009
Some Findings from WorldCat Local Usability TestsPrepared for ALA Annual, July 2009
6
Somethingsworkedwellevenwiththeearliestversionswetested,includinggettingprintitemsfromthelocalcollection,atleastforacademiclibraryusers.Somethingswereproblematic,includinglinkstoelectronicversionsandsupportingacademicworkflowsforcopingandpastingcitations.Weundertookamajorrevisionofthispage,testinghigh-fidelityprototypespartwaythroughthedesignprocess.
Getting print materials from the local collection
Canusersidentifywhat’savailableandwhere?
Foracademiclibraryusers,thisworkedwell.Whileafewtestparticipantsexpressedapreferenceforseeinginformationaboutavailabilityonthesearchresultspage,goingontotheitemdetailspageforthisinformationprovednottobeanobstacle.Locationandavailabilityinformationwasvisibleandrecognized.
Publiclibraryuserswantedtoseeholdingsfortheirlocalbranch,notthewholecitysystem.Thisgranularitycanbeanissue,asitwasinourearlypubliclibrarytests,dependingonhowholdingsarerepresented.Loadinglocalholdingsrecordsisaremedy.
Getting print materials from the group collection
Inourearlytestsparticipantscouldn’ttellwhethercopieswouldbeavailablefromtheconsortialcollection.
Weaddedcapacitytogetavailabilityinformationandsetthefulfillmentpolicyandworkflowforconsortialholdings,soWorldCatLocalcanofferabuttontogetacopyfromtheconsortialcollectionwhenitisavailableandproposeanotherapproachwhennoconsortialcopyisavailable.
Academictestparticipantsingeneralhadahighawarenessof—andrelianceon—thegroupcollectionsupportedbyaconsortiumorstatewidesystemtheirinstitutionmightbeapartof.
Somepubliclibraryuserswereconcernedalmostexclusivelywithwhatwasheldintheirlocalbranch.Othersregardedallmaterialstheycouldhavespeedyandsureaccesstoasequivalent.Notalltheseusersrecognizedtheconsortialarrangementsthatmadethatpossible.
Getting print materials from the worldwide collection
Undergraduatetestparticipantssawthewaytorequestitemsfromoutsidethelocalconsortiumbut,inearlytestsdidn’tseeminterestedindoingit.Thatwasprobablyanissuewiththetestdesign.Therearesomebarriersandsomeformstofillout.Theseparticipantswerenot,infact,deeplyinterestedintheitemswehadthemsearchfor,nordidtheyreallyexpecttogetthem.ActivitylogsshowthatinterlibraryloanisquiteaccessiblethroughWorldCatLocal.Interlibrary
loanwasoflimitedinteresttomostofourpubliclibrarytestparticipants.
We’vemovedthecontrolsforgettinglocal,groupandglobalmaterialstoclarifywherethematerialiscomingfrom.We’retestingtheminthesummerof2009inpublicandacademiclibrariesinIllinois,andwe’lltestthemagainwiththeUniversityofCalifornia.
Links to electronic resources
Inearlytests,accessingelectronicresourcesprovedtobeveryproblematic.Atfirst,halfofourtestparticipantsoverlookedthebigbuttonforviewingonlineorcheckingforelectronicresources.Wemovedthatbutton.
Inthenextroundoftests,participantsdemonstratedpatternsofmisunderstandingaboutthemanylinksandiconsassociatedwithelectronicresources.Thoselinkswerestilldispersedonvariouspartsofthepage.Participantswerenotverysuccessfulinknowingwhenelectronicresourceswereavailable,orretrievingthemwhentheywere.
Weredesignedtheitemdetailspagetogathertogetherandprioritizepreviouslyscatteredlinksforelectronicfulfillmentandinformationaboutavailability.Weremovedsomeconfusingicons.Also,WorldCatLocalnowallowsforcustomizationofthewordsshownontheopenURLresolverbuttonsotheyresonatewithwhatauserseesonotherpagesformtheirlibrary.
Wetestedjpegprototypesofthereviseddesignwithsixgraduatestudentsfromavarietyofdisciplines.Wesawimprovements.Thatdesignisinproductionnow,buttherearestillsomeproblems.Therecanstillbeamixoflinksonthispage:onesthatleadtofulfillmentbecausealibraryhasasubscriptionorbecausethey’reopenaccess,andonesthataredeadends,becauseneitherofthoseisthecase.Werecognizethatthisisausabilityissuethatcanmisleadusersintothinkingtheyhaveaccesstosomethingthatwedon’t,andwe’veinitiatedactiontoresolvetheissue.Wecontinuetolistentofeedbacktomakethedisplayofelectroniclinksaccurateforusers.Thisposesmanychallengesaswetrytoaccommodatedifferentlibrarycatalogingpracticesaswellasmeettheexpectationsoftheusersofthesystem.Inaddition,therehavebeencommentsabouthowweshowandhidelinksontheitemdetailspage.Bydefault,weshowthreelinksandiftherearemoretheyarecollapsedandtheusermustclickon“showmore”toexpandandseethem.We’renowinvestigatingwaystoindicatewhichlinkscarrythelocallibrary’sguaranteeofreliabilityand,whentherearemore,howtoshowmore.
Some Findings from WorldCat Local Usability TestsPrepared for ALA Annual, July 2009
7
Rearrangement of high-value bibliographic data
Inourredesignofitemdetailspageswealsoaimedtopromotehigh-valuebibliographicdetails.
Whenparticipantsexamineditemdetailspages,thesummaries,abstractsandtablesofcontentsonthosepageswerehighlyvalued.Participantsoftenselectedthemwiththemousepointerandsometimesreadthemaloud.Subjectheadingssometimesplayedasimilarrole,supportingevaluationratherthandiscoveryofrelateditems.
Wemovedsummaryinformationtothetopofthepageintheredesignoftheitemdetailspage.Weareevaluatingmovingotherinformation–perhapscontentsandphysicaldescription–tothetopofthepage.Surveydatawillhelpusdeterminewhatinformationismostvaluabletousersinthisarea.
Collaboration and personal workflow toolsWedidsomeformativetestingofcollaborationandpersonalworkflowtoolswithjpegprototypesandgraduatestudents.Ourchallengesinintroducingthesecollaborativefeaturesarenotunique.IdentityandresponsibilityarecriticalpiecesforallWebsitesandnotuniquetoourtestsortolibrariesingeneral.However,academicsareanaudienceespeciallyawareofreputation,andacademicusershavespheresinwhichtheseidentitiesandresponsibilityarecreated,outsideofthelibrarycatalog.
User-contributedreviewsgotmixedreactionsfromgraduatestudents.Responsetoreadingreviewswasmostlypositive.Sixofsevenparticipantswouldreadone.Onlyoneofsevenwouldwriteone.Itmatterswhodoeswritethem.Wesawastrongpreferencewhatparticipantscalled“editorial”reviewsoveruser-contributedreviewsinseveraltests.Amongpubliclibrarypatrons(eightofthemintwoIllinoislibraries),halfsaidtheywouldcontributereviews.
WechangedthedesignoftheItemDetailspagesothatreviewsarecollapsedbydefault.
“Readersrecommend”(based,atthetime,onrecommendationsfromWeRead)gotlessmixed—largelynegative—reactions.Onegraduatestudenthadapositivereactionbecause,hesaid,theserecommendationsmightbebasedonconnectionshecouldnototherwisehavemade.Fivegraduatestudentparticipantshadnegativereactionsthatrangedfromdoubttoasenseofviolation.Negativereactionshadmoretodowithwhotheotherreadersmightbethanwiththeideaofrecommendations(whichwerepraisedinMedLine)orwiththespecificitemsrecommended(whichwereignored).Tobevalued,recommendationsneedtocomefromameaningfulsource.Whatweoftenheard
matteredwasnotjustlevel—facultyrecommendationsoftenmeantmorethanstudentrecommendationstotestparticipants—butalsoinstitutionanddepartment.Andnotonly—orevenmainly—thesearcher’sowninstitution,butratheraninstitutiontheparticipantrecognizesashighlyregardedinhisorherparticulardiscipline.
WeremovedWeReadrecommendations,andinsteadweaddedarecommenderthatlooksatauthor,subjectandseriestitletorecommenditems.Thesefindingspromptedustoconductfocusgroupswithlibrarianstoexplorepossiblesourcesofmoreauthoritativereviewsandrecommendations.
Graduatestudents’reactionstotagsasawayofdiscoveringrelateditemsweremixed,butmostlypositive.Fiveoutofsevenparticipantswoulduseothers’tagsasawaytosearch.(Threeparticipantsbeganwithanegativeviewandmovedtoapositiveone.)Testparticipantssaidtheyvaluedthewaytags“gobeyondofficialsubjects.”Theyhaddifferentideasaboutwhatthiswaymightbe:someparticipantsexpectingtagstobebroader,othersnarrower,ormorecurrent,thanlibraryterms.Twoofsevenparticipantswouldnotusetagsprovidedbyothers;bothquestionedtheexpertiseorviewpointofthoseunknownandtoovariousothers.Oneparticipantsaidshewouldaddtagsofherown,onesaidshewouldnot.Themostcriticalusers—scholars—sawmorepromiseintags.Theonetestparticipantwithastrongpreferenceforthe“official”subjectsprovidedbylibrarianswasanundergraduate.
Wetestedstarratings.Graduatestudentreactionstoratingswerealmostallstronglynegative.Sixparticipantswouldnotrelyonarating.Fiveparticipantswouldnotrateanitem.Themainobjectionherewasnotaboutwhoisspeaking,butoneaboutpurpose.Participantspointedoutthattheywerenotreadingforleisure(acontextinwhichtheymightrelyonratings)butratherconstructinganargumentordevelopingaposition.Thefactthatsomethingisdisagreeableormistakenwouldnotmakeituseless.Publiclibraryusers,whosegoalsincludeleisure,mighthaveverydifferentreactionstoratings.
WeremovedStarratingsfromthebibliographicareaforWorldCatLocal.
Graduatestudentsvaluedlists.Theywouldbevaluedmorehighlyiftheauthorwereknownorauthor’sinstitutionalaffiliationwererecognizedandrespected.Nevertheless,inonetestonlyoneoffouruserscreatedabibliographyinWorldCatLocalwithoutbeingtoldtodoso.Instead,userscutandpastedreferencesintoaMicrosoftWorddocument.Advancedscholarsexpressedthesamepreference.
Thenewdesignoftheitemdetailspagenowinproductionprovidesforahighlight/copy/pasteworkflow.
Some Findings from WorldCat Local Usability TestsPrepared for ALA Annual, July 2009
8
NavigationNavigationpastthesecondorthirdpageofsearchresultsdidnothappenoften,evenforscholars.Althoughwehadparticipantsdotopicalsearchesintheirareasofexpertise,wecouldnotgetthemtolookpastthefirsttwopagesofsearchresults.Completenessofthesesearchresultsandwhatwasgoingonatthetailoftherankingwerenotaconcernforourtestparticipants.
Selectionisoftenmadefrombriefsearchresultpages,withoutgoingfurther.Whenparticipantsdogofurther,summary,abstractandcontentsnotesaremostvalued.
Allparticipantsreliedheavilyonthebrowser’sbackbuttonfornavigation.Someparticipantsexpressedadesireforamoreexplicitwaybacktosearchresults.Weaddeda“returntosearchresults”linktotheitemdetails.
Werecognizethatacademiclibraryusersandpubliclibraryusershavedifferentneeds,goalsandexpectations.Manychangeswehavemadesofarhavebeenbasedontheneedsofacademicusers.Aswedevelopanddifferentiatepersonasandprofilesforacademicusersontheonehandandpubliclibraryusersontheother,weexpecttocreatedifferentdesignthemestoreflectthesedifferences.So,forexample,reviewsandratingscanbemoreprominentforpubliclibrariesusingWorldCatLocalthanforacademiclibraries.
Some Findings from WorldCat Local Usability TestsPrepared for ALA Annual, July 2009