Implementation & Usage of bXtm at the
University of Minnesota
Janet ArthUniversity of Minnesota Twin Cities
& Sunshine Carter
University of Minnesota Duluth
U of M Duluth (UMD) and U of M Twin Cities (UMTC) Enrollment, Fall 2011
Enrollment Duluth Twin Cities
Undergraduate Students 9,782 30,610
Graduate Students 765 13,562
Professional Students 355 3,625
Nondegree Students 904 4,760
Totals 11,806 52,557
bX• About bX• UMTC went live 7/15/2010• UMD went into production 9/20/2010• Both sites use the SFX “simplified menu”
Selected bX Configuration Options• Show only recommendations with full text or all
recommendations?• Display 1-5 initial articles in the bX service window• Display 4-40 articles in the extended
recommendation list• Articles with full-text can be linked or un-linked• SFX button can be placed next to article
recommendations• Full text icons can be displayed or removed• Minimal recommendation level can range from 0-
100%. This relates to the percentage of relevancy to an article
bX @University of Minnesota Duluth
UMD Configurations• After 1 ½ months of use, UMD Reference opted to
change four features– Recommendation display: changed from 3 to 5– FindIt Logo: changed from the ‘SFX swish’ to our
UMD FindIt logo– Timeout: changed the bX timeout from 5 to 10
seconds– Minimal Recommendation Level: changed from 0%
to 40%
bX@UMD
Query 2: Number of SFX requests and clickthroughs per source
Requests: The number of times users clicked the SFX button from a particular source (e.g., from with a databases or from a bX recommendation on the SFX Menu) resulting in getting the SFX Menu
Clickthroughs: The number of times a user clicks on one of the options on that SFX Menu (there can be multiple clickthroughs on a single menu)
Note: In the case of bX, the number of requests is the real bX clickthrough count (until we activate bX in Primo or start using APIs in other places).
UMD SFX Trend (Q2)
Query 4: Number of SFX requests and clickthroughs per service type
• Service types include:• getFullTxt• getRecommendation: bX• getReference: capture citation or push to
RefWorks• getHoldings: MNCAT/Aleph catalog look-up• getDocDel: push to ILLiad
• The number of SFX Menu requests that contained a bX recommendation and number of clicks on the bX recommendation
bX Usage at UMD -- OverviewAY2010/11 - AY2011/12
Top Sources (requests and clickthroughs
•SFX A-Z List•Academic OneFile•PsycINFO•bX Menu•Google•Academic Search Premier•Thomson Gale (?)•Catalog•PubMed•RefWorks•Web of Science•Gale (?)•Primo (journals)•Web of Knowledge*Query 2 in SFX
Top Services (requests and clickthroughs
• getFullTxt• getRecommendation• getHolding• getDocumentDelivery*Query 4 in SFX
Top Journals Recommended by bX• Sex roles• Science• Pediatrics• Personality and Individual Differences• Medicine and science in sports and exercise• Nature• Journal of personality and social psychology• Educational leadership• Psychological bulletin• Journal of autism and developmental disorders*Query 11 in SFX
bX Usage at UMD -- DetailedAY2010/11 - AY2011/12
• bX recommendations were provided on 72,916 & 71,785 SFX service windows (Q4R)
• End users clicked on at least one recommendation 11,002 & 10,612 times (Q4CT)
• 17,189 & 17,007 bX recommendations were clicked on by end-users in SFX (Q2R)
• After clicking on a bX recommendation, end users selected 15,293 & 15,260 services. (Q2CT)
Looking at Interlibrary Loan … Why?
There was an Library Journal - Ex Libris Webinar on the bX Scholarly Article Recommender Service, with David Walker (California State University) and Nettie Lagace (then at Ex Libris). David Walker noted that some of the CalState sites experienced a notable increase in ILLs after implementing bX.
Originally broadcast February 17, 2010.
Webinar is still available from this url:www.exlibrisgroup.com/category/LJ-ELbXWebinar
Document Delivery (ILL) & SFX @ UMDJa
n 10
Feb
10
Mar
10
Apr
10
May
10
Jun
10
Jul 1
0
Aug
10
Sep
10
Oct
10
Nov
10
Dec
10
Jan
11
Feb
11
Mar
11
Apr
11
May
11
Jun
11
Jul 1
1
Aug
11
Sep
11
Oct
11
Nov
11
Dec
11
Jan
12
Feb
12
Mar
12
Apr
12
May
12
0
100
200
300
400
500
600
700
Any SFX Menu clickthrough to ILLiad (Q4)
Linear (Any SFX Menu click-through to ILLiad (Q4))
ILLiad Requests Created from Any SFX Menu (ILLiad reporting)
Linear (ILLiad Requests Created from Any SFX Menu (ILLiad reporting))
bx SFX Menu clickthrough to ILLiad (Q13)
ILLiad Requests Created from a bX SFX Menu (ILLiad reporting)
From SFX, patrons follow through with their ILL requests 35.7% of the time.
Document Delivery (ILL) & bX @ UMD
Conclusions about bX @ UMD• Downward trend at UMD for clickthroughs and
requests, in general• bX is appearing on a lot of service windows, but
recommendations are generally life and health sciences related
• ILL requests from SFX menus is on the decline• BUT, more requests to ILL are being made through bX
generated SFX menus• Higher percentage of follow-through for requests to
ILL via bX generated SFX menus
Why?• Are users getting better at determining relevancy of
an article?• Is the decline of ILL requests from SFX due to our
ever increasing access to content. Will the end of the “big deals” increase these requests.
• Is bX recommending a higher percentage of articles we don’t have access to? UMD does not subscribe to most health sciences literature.
• Is bX recommending highly relevant articles so as to entice patrons to actually request them through ILL (if we don’t have access)?
• Your thoughts and comments?
bX @University of Minnesota Twin Cities
U of M Twin Cities: ProcessEvaluation Phase
• A subcommittee of the Current Awareness / Personal Information Management (CAPIM) group led the initial evaluation phrase, during a 30 day free trial
• Trial period was May 15-June 15, 2010• bX was installed on the test server for the trial
– is a bit cumbersome to change database-generated OpenURLs so the link goes to the test server
– did not want to potentially offer it for only a month to the public
U of M Twin Cities: Process
• CAPIM recommended purchase of bX• bX was installed on the production server• The bX portion on the SFX Menu was customized
based on input from the CAPIM group and the TC’s SFX Advisory Council
• UMTC went live with bX on July 15 with no fanfare• Have not yet implemented bX in Primo because of
the lack of configurability (don’t want thumbs up/thumbs down – SI 16384-221290 )
UMTC – bX on the SFX Menu
http://tc.liblink.umn.edu/sfx_local?sid=Entrez:PubMed&id=pmid:18212285Key changes:
• Replaced SFX red graphic with our SFX graphic• Removed the “document” image that was to
the left of the SFX graphic• Modified the “Users interested in …” wording• Added bX information to SFX FAQ• Moved the bX recommendations down the
menu, but later moved them back up
UMTC SFX Menu (before August 2012)
UMTC SFX Menu (since August 2012)
UMTC Query 2: Top Twenty Sources of Menu Requests
0
100000
200000
300000
400000
500000
600000
2010-11 School Year
2011-12 School Year
UMTC Query 2: Overall SFX Menu UsageRequests & Clickthroughs
0
50000
100000
150000
200000
250000
300000
2010-11 Requests2010-11 Clickthroughs2011-12 Requests2011-12 Clickthroughs
UMTC Query 2: bX Menu Requests & Clickthroughs
0
2000
4000
6000
8000
10000
12000
14000
16000
2010-11 Requests2010-11 Clickthroughs2011-12 Requests2011-12 Clickthroughs
UMTC Query 4: Options displayed on SFX Menu
0
500000
1000000
1500000
2000000
2500000
2010-2011 Requests2010-2011 Clickthroughs2011-2012 Requests2011-2012 Clickthroughs
UMTC ILLiad requests from SFX Menufrom Query 13 and ILLiad Statisticsabout 60% are actually requested
0
500
1000
1500
2000
2500
3000
3500
Septe
mber
Octob
erNo
vemb
erDe
cemb
er
Janua
ryFe
bruary
March April
May
2010-11 getDocDelclicks2010-11 Completedin ILLiad2010-11 from bXgenerated menu2011-12 getDocDelclicks2011-12 Completedin ILLiad2011-12 from bXgenerated menu
UMTC ILLiad requests from SFXQuery 13: School Year 2011-2012
Month bX SFX Menu to ILLiad clickthrough
Any SFX Menu to ILLiad clickthrough
SFX to ILLiad clickthrough to ILL completed in ILLiad
September 20 2720 1657
October 15 3136 1871
November 18 2792 1590
December 6 1902 1167
January 3 2398 1511
February 37 3446 2048
March 29 3102 1886
April 22 2783 1810
May 8 2008 1470
Year Totals 158 24287 15010
UMTC Query 11: bX-generated SFX Menu Most Popular Journals 2011-2012 (requests/clickthroughs)
Note the absence of the “hard” sciences
JAMA (560/509) Journal of Adolescent Health (289/322)New England Journal of Medicine (513/523)
Medicine and science in sports and exercise (287/284)
Science (450/463) Nature (279/281)Journal of Personality and Social Psychology (410/401) Developmental Psychology (275/300)
Pediatric (408/593) Journal of Advanced Nursing (270/290)Sex Roles (377/385) American Psychologist (267/286)
Journal of Marriage and Family (365/400)
Journal of the American Academy of Child & Adolescent Psychiatry (260/284)
Child Development (358/425) The American Journal of Psychiatry (252/252)
Social Science & Medicine (320/302) Harvard Business Review (247/230)Lancet (313/305) Personality and Individual Differences (245/238)
UMTC: bX-generated SFX MenusTop Targets 2011-2012 Clickthroughs (mysql query)
bX Recommendations (16,625) All [28] Wiley Online targets (5637)
EBSCO Academic Search Premier (3895) All [28] EBSCOhost targets (9177)
Elsevier Science Direct Elsevier (3306) All [44] Elsevier Science Direct targets (8666)
Miscellaneous Free (2738) All [28] JSTOR collections (3630)
Journals at OVID (2562) All [7] Sage targets (3765)
RefWorks (2347) All [92] Highwire targets (3154)
DOAJ (1124) All [20] Springer targets (2420)
Aleph Catalog getHoldings (881) All [2] APA targets (1947)
Capture Citation (785) All [2] Taylor & Francis targets (1481)
Cambridge University (405) All [10] Gale targets (1097)
MDConsult (404) All [2] PubMed Central targets (586)
Project Muse (392) All [4] Wilson targets (574)
Absence of Hard Sciences??
Likely due in part to the lack of recommendations when the OpenURL is DOI driven; as it is for many sites in the areas of physics, engineering and computer sciences. See Incident #:16384-216084 and the following example.
Compare
• PMID-driven OpenURL gets recommendations:http://tc.liblink.umn.edu/sfx_local?sid=Entrez&id=pmid:20161330
• DOI-driven OpenURL for same article has no recommendations:http://tc.liblink.umn.edu/sfx_local?ctx_enc=info%3Aofi%2Fenc
%3AUTF-8&ctx_id=10_1&ctx_tim=2010-10-07T10%3A22%3A2CDT&ctx_ver=Z39.88-2004&rfr_id=info%3Asid%2Fsfxit.com%3Acitation&rft.genre=article&rft_id=info%3Adoi%2F10.1016%2Fj.foodchem.2009.01.087
PMID-driven OpenURL
DOI-driven OpenURL
bX Menus from other sites
• Kansas State University http://getit.lib.ksu.edu/sfxlcl3/?sid=Entrez:PubMed&id=pmid:18212285
Thanks to Jamene Brooks-Kieffer at KSU for the mysql queries (from her ELUNA presentation in 2011)
• University of Iowa http://infolink.lib.uiowa.edu/sfx_local/?sid=Entrez:PubMed&id=pmid:18212285
A few additional slides
Query 2 Menu Requests Percent of TotalTotal 2,153,530 100.00 %SFX A-Z List 603,883 28.04 %Entrez PubMed 363,667 16.89 %Google Scholar 203,528 9.45 %SFX Citation Linker 144,945 6.73 %MARC It records in Aleph 144,397 6.71 %Web of Knowledge 125,896 5.84 %EBSCO Academic Search Premier (ASP) 121,602 5.65 %OVID MEDLINE 107,031 4.97 %SFX bX Menu 84,936 3.94 %OVID PsycINFO 73,350 3.41 %MARC It records in Primo 63,368 2.94 %SciFinder CAPLUS 61,513 2.86 %
UMTC Query 2: Requests 2010-2011 (Sept-May)Note: database/source counts are platform specific, e.g., there are also SFX menu
requests that come from ASP via MetaLib & Primo that are not included here, the ASP count here is just for menu requests coming from the native EBSCO interface
Query 2 Menu Requests Percent of TotalTotal 2,089,070 100.00 %Entrez PubMed 408,130 19.54 %SFX A-Z List 354,123 16.95 %Google Scholar 250,838 12.01 %SFX Citation Linker 153,189 7.33 %MARC It records in Aleph 124,921 5.98 %WoK Web of Science 95,767 4.58 %EBSCO Academic Search Premier (ASP) 92,773 4.44 %OVID MEDLINE 83,909 4.02 %OVID PsycINFO 78,956 3.78 %SFX bX Menu (bX not activated in Primo) 74,790 3.58 %MARC It records in Primo 69,002 3.30 %SciFinder CAPLUS 46,016 2.20 %
UMTC Query 2: Requests 2011-2012 (Sept-May)Same note applies to this table
UMTC Query 11: bX-generated SFX Menu Most Popular Journals 2010-2011 (requests/clickthroughs)
Again note the absence of the “hard” sciences
New England Journal of Medicine Sex RolesJAMA Child DevelopmentJournal of Marriage and Family Journal of Youth and AdolescenceScience American Psychologist
Journal of Advanced NursingMedicine and science in sports and exercise
PediatricsJournal of the American Academy of Child & Adolescent Psychiatry
Journal of Personality and Social Psychology Harvard Business ReviewNature The American Journal of PsychiatryLancet Journal of CommunicationJournal of Adolescent Health Social Science & Medicine
University of Minnesota Twin CitiesWeisman Art Museum