E-Resources Usage Data: Apples to Oranges and Fixing Holes presented by George Boston Electronic Resources and Serials Librarian Western Michigan University

E-Resources Usage Data: E-Resources Usage Data: Apples to OrangesApples to Orangesand Fixing Holes and Fixing Holes

presented by presented by

George BostonGeorge BostonElectronic Resources and Serials LibrarianElectronic Resources and Serials Librarian

Western Michigan University Libraries Western Michigan University Libraries

Michael WhangMichael WhangHead of Web & Internet ServicesHead of Web & Internet Services

Western Michigan University Libraries Western Michigan University Libraries

2008 Electronic Resources & Libraries Conference, Atlanta, 2008

Who we are?Who we are? Western Michigan University is a student centered Western Michigan University is a student centered

research university with an enrollment of 25,000research university with an enrollment of 25,000Undergraduate programs – 141Undergraduate programs – 141Master’s Level programs – 66Master’s Level programs – 66Doctoral and Specialist programs – 30Doctoral and Specialist programs – 30

Library spends about $6 million on resources ($3 Library spends about $6 million on resources ($3

million on electronic resources)million on electronic resources)

29,000 unique ejournals, 66,000 ebooks, 400 A/I and 29,000 unique ejournals, 66,000 ebooks, 400 A/I and full text collection databasesfull text collection databases


What are the problems?What are the problems? Large amount of money is spent on electronic Large amount of money is spent on electronic

resources, there is a need to justify resources, there is a need to justify expenditure in an era of tightening budgetsexpenditure in an era of tightening budgets

We need to discover holes in the collectionWe need to discover holes in the collection

Deliver electronic resources that better Deliver electronic resources that better matches people’s workflowmatches people’s workflow


How do we address them?How do we address them? We need to look at:We need to look at:

What we haveWhat we have How the existing resources are being usedHow the existing resources are being used Where users are discovering our resources and Where users are discovering our resources and

the path they take to access them. Are the users the path they take to access them. Are the users coming from:coming from:

Library Catalog?Library Catalog? Library Web site?Library Web site? Subscribed databases?Subscribed databases?

Electronic Resources use statistics are Electronic Resources use statistics are available in a variety of types, such as:available in a variety of types, such as:


Statistics types available now:Statistics types available now:

COUNTER compliantCOUNTER compliant Non-COUNTERNon-COUNTER Scholarly StatsScholarly Stats SFXSFX Proxy server logsProxy server logs Web server logsWeb server logs

How do we gather these?How do we gather these? And how do we put them all together?And how do we put them all together?


COUNTER compliant statisticsCOUNTER compliant statistics AdvantagesAdvantages

Easy to compare resources between reports of the same Easy to compare resources between reports of the same typetype

Industry standard approach is desirable.Industry standard approach is desirable.

DisadvantagesDisadvantages Implementations vary from publisher to publisherImplementations vary from publisher to publisher Various types available:Various types available:

Journal Report 1: Number of Successful Full-Text Article Requests by Month and JournalJournal Report 1: Number of Successful Full-Text Article Requests by Month and JournalJournal Report 2: Turnaways by Month and JournalJournal Report 2: Turnaways by Month and JournalJournal Report 3: Number of Successful Item Requests and Turnaways by Month, Journal and Journal Report 3: Number of Successful Item Requests and Turnaways by Month, Journal and

TypeTypeJournal Report 4: Total Searches Run by Month and Service Journal Report 4: Total Searches Run by Month and Service Database Report 1: Total Searches and Sessions by Month and DatabaseDatabase Report 1: Total Searches and Sessions by Month and DatabaseDatabase Report 2: Turnaways by Month and DatabaseDatabase Report 2: Turnaways by Month and DatabaseDatabase Report 3: Total Searches and Sessions by Month and ServiceDatabase Report 3: Total Searches and Sessions by Month and Service


COUNTER compliant statistics COUNTER compliant statistics exampleexample


Non-COUNTERNon-COUNTER AdvantagesAdvantages

Driven by provider’s capabilitiesDriven by provider’s capabilities

DisadvantagesDisadvantages What does the data mean? (a non-standard What does the data mean? (a non-standard

approach can give useless results unless we know approach can give useless results unless we know what the data actually represents)what the data actually represents)


Non-COUNTER exampleNon-COUNTER example


Scholarly StatsScholarly Stats

AdvantagesAdvantages Integrated reports provide easily Integrated reports provide easily

comparable datacomparable data

DisadvantagesDisadvantages Mixes COUNTER and non-COUNTERMixes COUNTER and non-COUNTER Only certain types of reports gatheredOnly certain types of reports gathered Interface maintenance problemsInterface maintenance problems


Scholarly Stats exampleScholarly Stats example


SFX generatedSFX generated AdvantagesAdvantages

SFX integration with the Library OPAC can create SFX integration with the Library OPAC can create a centralized statistical repositorya centralized statistical repository

Several useful reports are available from the SFX Several useful reports are available from the SFX administrative moduleadministrative module

DisadvantagesDisadvantages Limited to statistics on materials accessed via the Limited to statistics on materials accessed via the

SFX interfaceSFX interface Non-Counter compliantNon-Counter compliant Difficult to retrieve from SFX server (Easily done, Difficult to retrieve from SFX server (Easily done,

but a non-trivial process)but a non-trivial process)2008 Electronic Resources & Libraries Conference, Atlanta, 2008

SFX reports available:SFX reports available:11 Number of requests and clickthroughs per dayNumber of requests and clickthroughs per day22 Number of requests and clickthroughs per sourceNumber of requests and clickthroughs per source33 Number of requests and clickthroughs per object typeNumber of requests and clickthroughs per object type44 Number of requests and clickthroughs per service typeNumber of requests and clickthroughs per service type55 Number of SFX requests with/without full-text servicesNumber of SFX requests with/without full-text services66 Top target services shown in the SFX menuTop target services shown in the SFX menu77 Number of clickthroughs per targetNumber of clickthroughs per target88 Number of clickthroughs per target serviceNumber of clickthroughs per target service99 Number of clickthroughs for one particular serialNumber of clickthroughs for one particular serial1010 Most popular serials selected by targetMost popular serials selected by target1111 Most popular journals selected by sourceMost popular journals selected by source1212 Journals requested but have no full-textJournals requested but have no full-text1313 Selected document delivery targets by sourceSelected document delivery targets by source1414 Books accessed via SFX ranked by useBooks accessed via SFX ranked by use1515 Services preferred over full-textServices preferred over full-text1616 Unused full-text journalsUnused full-text journals1717 Number of requests which resulted in SFX menu screen without servicesNumber of requests which resulted in SFX menu screen without services1818 Number of requests and clickthroughs by IP addressNumber of requests and clickthroughs by IP address1919 Most popular journalsMost popular journals2020 OpenURLs that resulted in no fulltext services, selected by sourceOpenURLs that resulted in no fulltext services, selected by source


SFX example: Report #2SFX example: Report #2


SFX example: Report #8SFX example: Report #8


ProxyProxy server usage logs server usage logs


•AdvantagesAdvantages–Shows precise information about off-campus Shows precise information about off-campus connections connections

•When the connection occurredWhen the connection occurred

•Where the computer location (IP) connection Where the computer location (IP) connection originatedoriginated

•How long was the user connected?How long was the user connected?

•DisadvantagesDisadvantages–CrypticCryptic–In order to resolve an IP to a named domain an extra In order to resolve an IP to a named domain an extra step is step is

neededneeded

Web server logsWeb server logs AdvantagesAdvantages

Tells us about “Tells us about “connectionsconnections”” Where people come fromWhere people come from What they were looking forWhat they were looking for How long they stayedHow long they stayed Where they leftWhere they left

DisadvantagesDisadvantages Does not tell us about Does not tell us about people’s workflowpeople’s workflow Logs are merely assumptionsLogs are merely assumptions


What do we know about ourWhat do we know about ourvisitors?visitors?

Since January 1, 2008:Since January 1, 2008:

125,922 visitors have come to 125,922 visitors have come to www.wmich.edu/librarywww.wmich.edu/library

76.40% were visitors that returned more 76.40% were visitors that returned more than oncethan once

23.60% were “new” visitors23.60% were “new” visitors2008 Electronic Resources & Libraries Conference, Atlanta, 2008

Where are they coming from?Where are they coming from?


Where are they coming from?Where are they coming from? United States (132,082)United States (132,082)

MichiganMichigan ArizonaArizona New YorkNew York IllinoisIllinois CaliforniaCalifornia

United Kingdom (371)United Kingdom (371) Canada (367)Canada (367) Germany (333)Germany (333) Malaysia (247)Malaysia (247)


Where in the U.S.?Where in the U.S.?


WMU’s Top 10WMU’s Top 101.1. KalamazooKalamazoo2.2. Grand RapidsGrand Rapids3.3. Kent CountyKent County4.4. Byron CenterByron Center5.5. MolineMoline6.6. PortagePortage7.7. GrandvilleGrandville8.8. Battle CreekBattle Creek9.9. LansingLansing10.10. CaledoniaCaledonia


How are they getting here?How are they getting here?


How are they getting here?How are they getting here?

Over 37% either typed in our URL or Over 37% either typed in our URL or clicked bookmark (direct referrals)clicked bookmark (direct referrals)

Over 35% came from a search engine: Over 35% came from a search engine: Google, Yahoo, MSN, AOL, AskGoogle, Yahoo, MSN, AOL, Ask

6% typed in ‘library’ or ‘libraries’6% typed in ‘library’ or ‘libraries’


Search KeywordsSearch Keywords


The WMU Library Top 10The WMU Library Top 10

1.1. Library Homepage (181,510)Library Homepage (181,510)2.2. Articles (37,529)Articles (37,529)3.3. Course Reserves (12,652)Course Reserves (12,652)4.4. ‘‘P’ page of our A-Z list of databases (6,901)P’ page of our A-Z list of databases (6,901)5.5. Journals (6,245)Journals (6,245)6.6. Subject Guides (6,003)Subject Guides (6,003)7.7. Books (5,505)Books (5,505)8.8. ‘‘C’ page of our A-Z list of databases (4,237)C’ page of our A-Z list of databases (4,237)9.9. ‘‘E’ page of our A-Z list of databases (3,876)E’ page of our A-Z list of databases (3,876)10.10. Hours (3,875)Hours (3,875)


What does Web server data What does Web server data mean to us?mean to us?

Provides a dashboard to measureProvides a dashboard to measure Instruction / Outreach activitiesInstruction / Outreach activities Page, layout, and content designPage, layout, and content design

Helps us set prioritiesHelps us set priorities Instruction / Outreach activitiesInstruction / Outreach activities Keywords to inform SEO and site navigationKeywords to inform SEO and site navigation Page, layout, and content designPage, layout, and content design Drive library web development & user researchDrive library web development & user research


Focus on the benefitsFocus on the benefits

Positioning to persuadePositioning to persuade

Show them exactly Show them exactly where to gowhere to go

Optimize page design to drive Optimize page design to drive resource discovery and funnelingresource discovery and funneling

Putting them all togetherPutting them all together

List usage statistics by report typeList usage statistics by report type COUNTERCOUNTER non-COUNTERnon-COUNTER Scholarly StatsScholarly Stats SFXSFX Proxy server logsProxy server logs Web server logsWeb server logs


How is this done?How is this done? Analyze and identify usage trendsAnalyze and identify usage trends

Determine the “usage vector” for a particular resourceDetermine the “usage vector” for a particular resource How the user is getting to the informationHow the user is getting to the information

OPAC?OPAC? Library Web site?Library Web site? Subscribed A/I databases?Subscribed A/I databases? Google and the web?Google and the web?

From where is the user linkingFrom where is the user linking On campusOn campus Off campusOff campus


Once we know where and how Once we know where and how users are linking, then…users are linking, then…

What are the numbers?What are the numbers?

What is the provenance of the numbers?What is the provenance of the numbers?

What activities are driving usage?What activities are driving usage? Normal academic sessionNormal academic session Break periodBreak period Technical considerationsTechnical considerations

What is the usage of a resource over time?What is the usage of a resource over time?


How do we use them?How do we use them?


• No turnaways for Oxford Reference Online, but 64 turnaways for Reader’s Guide Retrospective in 2006, response increased our number of users by one. Decreasing turnaways to three in 2007.

How do we use them?How do we use them?


•In September 2007 we did a major redesign of the web page and the SFX menu (which points users to electronic journal content)

•The SFX Citation linker is the primary clickthrough path to electronic journal content in both April 2007 (before the redesign) and in February 2008 (after the redesign)

The FutureThe Future SUSHISUSHI

Integration into ERM (Verde)Integration into ERM (Verde)

Localized eMetrics database?Localized eMetrics database? Where report data can be easily collated and accessed Where report data can be easily collated and accessed

Right now statistics are gathered and collated manuallyRight now statistics are gathered and collated manually

Viral marketing of Subscribed Electronic ResourcesViral marketing of Subscribed Electronic Resources User-centered access vectorsUser-centered access vectors Identify low usage resources/Why?Identify low usage resources/Why?

Not usefulNot useful Nobody knows about themNobody knows about them

How do we fix thatHow do we fix that Web placementWeb placement Online Catalog?Online Catalog? Metasearch?Metasearch?


References:References:ARL New Measures Initiative .The E-Metrics Project The Importance

of Network Measures and Statistics. http://www.arl.org/bm~doc/emetrics_module_1_importance011402-arlmode.ppt

ARL New Measures Initiative. Preparing YOUR Library to Collect, Report, Analyze, and Use the Statistics and Measures. http://www.arl.org/bm~doc/emetrics_module_2_preparation_011402_arlmode.ppt

ARL New Measures Initiative. Recommended Statistics and Measures for Library Networked Services.http://www.arl.org/bm~doc/emetrics_module_3_statistics.ppt

IFLA General Conference (73rd) Evaluating the library website: Statistics and quality measures. http://www.ifla.org/IV/ifla73/papers/074-Poll-en.pdf


Contact InformationContact InformationGeorge BostonGeorge BostonElectronic Resources and Serials LibrarianElectronic Resources and Serials LibrarianWestern Michigan University LibrariesWestern Michigan University [email protected]@wmich.edu269-387-5147269-387-5147

Michael WhangMichael WhangHead of Web & Internet ServicesHead of Web & Internet ServicesWestern Michigan University LibrariesWestern Michigan University [email protected]@wmich.edu269-387-5184269-387-5184


Documents

E-Resources Usage Data: Apples to Oranges and Fixing Holes presented by George Boston Electronic Resources and Serials Librarian Western Michigan University