Implementation & Usage of bXtm at the

University of Minnesota

Janet ArthUniversity of Minnesota Twin Cities

& Sunshine Carter

University of Minnesota Duluth

U of M Duluth (UMD) and U of M Twin Cities (UMTC) Enrollment, Fall 2011

Enrollment Duluth Twin Cities

Undergraduate Students 9,782 30,610

Graduate Students 765 13,562

Professional Students 355 3,625

Nondegree Students 904 4,760

Totals 11,806 52,557

bX• About bX• UMTC went live 7/15/2010• UMD went into production 9/20/2010• Both sites use the SFX “simplified menu”

Selected bX Configuration Options• Show only recommendations with full text or all

recommendations?• Display 1-5 initial articles in the bX service window• Display 4-40 articles in the extended

recommendation list• Articles with full-text can be linked or un-linked• SFX button can be placed next to article

recommendations• Full text icons can be displayed or removed• Minimal recommendation level can range from 0-

100%. This relates to the percentage of relevancy to an article

bX @University of Minnesota Duluth

UMD Configurations• After 1 ½ months of use, UMD Reference opted to

change four features– Recommendation display: changed from 3 to 5– FindIt Logo: changed from the ‘SFX swish’ to our

UMD FindIt logo– Timeout: changed the bX timeout from 5 to 10

seconds– Minimal Recommendation Level: changed from 0%

to 40%

bX@UMD

Query 2: Number of SFX requests and clickthroughs per source

Requests: The number of times users clicked the SFX button from a particular source (e.g., from with a databases or from a bX recommendation on the SFX Menu) resulting in getting the SFX Menu

Clickthroughs: The number of times a user clicks on one of the options on that SFX Menu (there can be multiple clickthroughs on a single menu)

Note: In the case of bX, the number of requests is the real bX clickthrough count (until we activate bX in Primo or start using APIs in other places).

UMD SFX Trend (Q2)

Query 4: Number of SFX requests and clickthroughs per service type

• Service types include:• getFullTxt• getRecommendation: bX• getReference: capture citation or push to

RefWorks• getHoldings: MNCAT/Aleph catalog look-up• getDocDel: push to ILLiad

• The number of SFX Menu requests that contained a bX recommendation and number of clicks on the bX recommendation

bX Usage at UMD -- OverviewAY2010/11 - AY2011/12

Top Sources (requests and clickthroughs

•SFX A-Z List•Academic OneFile•PsycINFO•bX Menu•Google•Academic Search Premier•Thomson Gale (?)•Catalog•PubMed•RefWorks•Web of Science•Gale (?)•Primo (journals)•Web of Knowledge*Query 2 in SFX

Top Services (requests and clickthroughs

• getFullTxt• getRecommendation• getHolding• getDocumentDelivery*Query 4 in SFX

Top Journals Recommended by bX• Sex roles• Science• Pediatrics• Personality and Individual Differences• Medicine and science in sports and exercise• Nature• Journal of personality and social psychology• Educational leadership• Psychological bulletin• Journal of autism and developmental disorders*Query 11 in SFX

bX Usage at UMD -- DetailedAY2010/11 - AY2011/12

• bX recommendations were provided on 72,916 & 71,785 SFX service windows (Q4R)

• End users clicked on at least one recommendation 11,002 & 10,612 times (Q4CT)

• 17,189 & 17,007 bX recommendations were clicked on by end-users in SFX (Q2R)

• After clicking on a bX recommendation, end users selected 15,293 & 15,260 services. (Q2CT)

Looking at Interlibrary Loan … Why?

There was an Library Journal - Ex Libris Webinar on the bX Scholarly Article Recommender Service, with David Walker (California State University) and Nettie Lagace (then at Ex Libris). David Walker noted that some of the CalState sites experienced a notable increase in ILLs after implementing bX.

Originally broadcast February 17, 2010.

Webinar is still available from this url:www.exlibrisgroup.com/category/LJ-ELbXWebinar

Document Delivery (ILL) & SFX @ UMDJa

n 10

Feb

10

Mar

10

Apr

10

May

10

Jun

10

Jul 1

0

Aug

10

Sep

10

Oct

10

Nov

10

Dec

10

Jan

11

Feb

11

Mar

11

Apr

11

May

11

Jun

11

Jul 1

1

Aug

11

Sep

11

Oct

11

Nov

11

Dec

11

Jan

12

Feb

12

Mar

12

Apr

12

May

12

0

100

200

300

400

500

600

700

Any SFX Menu clickthrough to ILLiad (Q4)

Linear (Any SFX Menu click-through to ILLiad (Q4))

ILLiad Requests Created from Any SFX Menu (ILLiad reporting)

Linear (ILLiad Requests Created from Any SFX Menu (ILLiad reporting))

bx SFX Menu clickthrough to ILLiad (Q13)

ILLiad Requests Created from a bX SFX Menu (ILLiad reporting)

From SFX, patrons follow through with their ILL requests 35.7% of the time.

Document Delivery (ILL) & bX @ UMD

Conclusions about bX @ UMD• Downward trend at UMD for clickthroughs and

requests, in general• bX is appearing on a lot of service windows, but

recommendations are generally life and health sciences related

• ILL requests from SFX menus is on the decline• BUT, more requests to ILL are being made through bX

generated SFX menus• Higher percentage of follow-through for requests to

ILL via bX generated SFX menus

Why?• Are users getting better at determining relevancy of

an article?• Is the decline of ILL requests from SFX due to our

ever increasing access to content. Will the end of the “big deals” increase these requests.

• Is bX recommending a higher percentage of articles we don’t have access to? UMD does not subscribe to most health sciences literature.

• Is bX recommending highly relevant articles so as to entice patrons to actually request them through ILL (if we don’t have access)?

• Your thoughts and comments?

bX @University of Minnesota Twin Cities

U of M Twin Cities: ProcessEvaluation Phase

• A subcommittee of the Current Awareness / Personal Information Management (CAPIM) group led the initial evaluation phrase, during a 30 day free trial

• Trial period was May 15-June 15, 2010• bX was installed on the test server for the trial

– is a bit cumbersome to change database-generated OpenURLs so the link goes to the test server

– did not want to potentially offer it for only a month to the public

U of M Twin Cities: Process

• CAPIM recommended purchase of bX• bX was installed on the production server• The bX portion on the SFX Menu was customized

based on input from the CAPIM group and the TC’s SFX Advisory Council

• UMTC went live with bX on July 15 with no fanfare• Have not yet implemented bX in Primo because of

the lack of configurability (don’t want thumbs up/thumbs down – SI 16384-221290 )

UMTC – bX on the SFX Menu

http://tc.liblink.umn.edu/sfx_local?sid=Entrez:PubMed&id=pmid:18212285Key changes:

• Replaced SFX red graphic with our SFX graphic• Removed the “document” image that was to

the left of the SFX graphic• Modified the “Users interested in …” wording• Added bX information to SFX FAQ• Moved the bX recommendations down the

menu, but later moved them back up

UMTC SFX Menu (before August 2012)

UMTC SFX Menu (since August 2012)

UMTC Query 2: Top Twenty Sources of Menu Requests

0

100000

200000

300000

400000

500000

600000

2010-11 School Year

2011-12 School Year

UMTC Query 2: Overall SFX Menu UsageRequests & Clickthroughs

0

50000

100000

150000

200000

250000

300000

2010-11 Requests2010-11 Clickthroughs2011-12 Requests2011-12 Clickthroughs

UMTC Query 2: bX Menu Requests & Clickthroughs

0

2000

4000

6000

8000

10000

12000

14000

16000

2010-11 Requests2010-11 Clickthroughs2011-12 Requests2011-12 Clickthroughs

UMTC Query 4: Options displayed on SFX Menu

0

500000

1000000

1500000

2000000

2500000

2010-2011 Requests2010-2011 Clickthroughs2011-2012 Requests2011-2012 Clickthroughs

UMTC ILLiad requests from SFX Menufrom Query 13 and ILLiad Statisticsabout 60% are actually requested

0

500

1000

1500

2000

2500

3000

3500

Septe

mber

Octob

erNo

vemb

erDe

cemb

er

Janua

ryFe

bruary

March April

May

2010-11 getDocDelclicks2010-11 Completedin ILLiad2010-11 from bXgenerated menu2011-12 getDocDelclicks2011-12 Completedin ILLiad2011-12 from bXgenerated menu

UMTC ILLiad requests from SFXQuery 13: School Year 2011-2012

Month bX SFX Menu to ILLiad clickthrough

Any SFX Menu to ILLiad clickthrough

SFX to ILLiad clickthrough to ILL completed in ILLiad

September 20 2720 1657

October 15 3136 1871

November 18 2792 1590

December 6 1902 1167

January 3 2398 1511

February 37 3446 2048

March 29 3102 1886

April 22 2783 1810

May 8 2008 1470

Year Totals 158 24287 15010

UMTC Query 11: bX-generated SFX Menu Most Popular Journals 2011-2012 (requests/clickthroughs)

Note the absence of the “hard” sciences

JAMA (560/509) Journal of Adolescent Health (289/322)New England Journal of Medicine (513/523)

Medicine and science in sports and exercise (287/284)

Science (450/463) Nature (279/281)Journal of Personality and Social Psychology (410/401) Developmental Psychology (275/300)

Pediatric (408/593) Journal of Advanced Nursing (270/290)Sex Roles (377/385) American Psychologist (267/286)

Journal of Marriage and Family (365/400)

Journal of the American Academy of Child & Adolescent Psychiatry (260/284)

Child Development (358/425) The American Journal of Psychiatry (252/252)

Social Science & Medicine (320/302) Harvard Business Review (247/230)Lancet (313/305) Personality and Individual Differences (245/238)

UMTC: bX-generated SFX MenusTop Targets 2011-2012 Clickthroughs (mysql query)

bX Recommendations (16,625) All [28] Wiley Online targets (5637)

EBSCO Academic Search Premier (3895) All [28] EBSCOhost targets (9177)

Elsevier Science Direct Elsevier (3306) All [44] Elsevier Science Direct targets (8666)

Miscellaneous Free (2738) All [28] JSTOR collections (3630)

Journals at OVID (2562) All [7] Sage targets (3765)

RefWorks (2347) All [92] Highwire targets (3154)

DOAJ (1124) All [20] Springer targets (2420)

Aleph Catalog getHoldings (881) All [2] APA targets (1947)

Capture Citation (785) All [2] Taylor & Francis targets (1481)

Cambridge University (405) All [10] Gale targets (1097)

MDConsult (404) All [2] PubMed Central targets (586)

Project Muse (392) All [4] Wilson targets (574)

Absence of Hard Sciences??

Likely due in part to the lack of recommendations when the OpenURL is DOI driven; as it is for many sites in the areas of physics, engineering and computer sciences. See Incident #:16384-216084 and the following example.

Compare

• PMID-driven OpenURL gets recommendations:http://tc.liblink.umn.edu/sfx_local?sid=Entrez&id=pmid:20161330

• DOI-driven OpenURL for same article has no recommendations:http://tc.liblink.umn.edu/sfx_local?ctx_enc=info%3Aofi%2Fenc

%3AUTF-8&ctx_id=10_1&ctx_tim=2010-10-07T10%3A22%3A2CDT&ctx_ver=Z39.88-2004&rfr_id=info%3Asid%2Fsfxit.com%3Acitation&rft.genre=article&rft_id=info%3Adoi%2F10.1016%2Fj.foodchem.2009.01.087

PMID-driven OpenURL

DOI-driven OpenURL

bX Menus from other sites

• Kansas State University http://getit.lib.ksu.edu/sfxlcl3/?sid=Entrez:PubMed&id=pmid:18212285

Thanks to Jamene Brooks-Kieffer at KSU for the mysql queries (from her ELUNA presentation in 2011)

• University of Iowa http://infolink.lib.uiowa.edu/sfx_local/?sid=Entrez:PubMed&id=pmid:18212285

Thank you!

• Questions?

• Janet Arth: arth@umn.edu• Sunshine Carter: scarter@d.umn.edu

A few additional slides

Query 2 Menu Requests Percent of TotalTotal 2,153,530 100.00 %SFX A-Z List 603,883 28.04 %Entrez PubMed 363,667 16.89 %Google Scholar 203,528 9.45 %SFX Citation Linker 144,945 6.73 %MARC It records in Aleph 144,397 6.71 %Web of Knowledge 125,896 5.84 %EBSCO Academic Search Premier (ASP) 121,602 5.65 %OVID MEDLINE 107,031 4.97 %SFX bX Menu 84,936 3.94 %OVID PsycINFO 73,350 3.41 %MARC It records in Primo 63,368 2.94 %SciFinder CAPLUS 61,513 2.86 %

UMTC Query 2: Requests 2010-2011 (Sept-May)Note: database/source counts are platform specific, e.g., there are also SFX menu

requests that come from ASP via MetaLib & Primo that are not included here, the ASP count here is just for menu requests coming from the native EBSCO interface

Query 2 Menu Requests Percent of TotalTotal 2,089,070 100.00 %Entrez PubMed 408,130 19.54 %SFX A-Z List 354,123 16.95 %Google Scholar 250,838 12.01 %SFX Citation Linker 153,189 7.33 %MARC It records in Aleph 124,921 5.98 %WoK Web of Science 95,767 4.58 %EBSCO Academic Search Premier (ASP) 92,773 4.44 %OVID MEDLINE 83,909 4.02 %OVID PsycINFO 78,956 3.78 %SFX bX Menu (bX not activated in Primo) 74,790 3.58 %MARC It records in Primo 69,002 3.30 %SciFinder CAPLUS 46,016 2.20 %

UMTC Query 2: Requests 2011-2012 (Sept-May)Same note applies to this table

UMTC Query 11: bX-generated SFX Menu Most Popular Journals 2010-2011 (requests/clickthroughs)

Again note the absence of the “hard” sciences

New England Journal of Medicine Sex RolesJAMA Child DevelopmentJournal of Marriage and Family Journal of Youth and AdolescenceScience American Psychologist

Journal of Advanced NursingMedicine and science in sports and exercise

PediatricsJournal of the American Academy of Child & Adolescent Psychiatry

Journal of Personality and Social Psychology Harvard Business ReviewNature The American Journal of PsychiatryLancet Journal of CommunicationJournal of Adolescent Health Social Science & Medicine

University of Minnesota Twin CitiesWeisman Art Museum