Embed Size (px)
DESCRIPTION
A systematic search and synthesis of literature of point-of-care tool, UpToDate
Citation preview
UpToDate Systematic Narrative Review – Thematic TablesGiustini, Wang, Read – October 2011 Update
Statement of Problem: In the provision of point-of-care (POC) tools, few products seem to cause the level of debate and disagreement than the Wolters Kluwer product, UpToDate (UTD). On the one hand, there are health librarians who view the tool as overpriced – and simply not as current or evidence-based as its popularity might suggest. On the other, there are the medical students, residents and physicians (especially those in internal medicine, primary care; and subspecialities, cardiovascular medicine and endocrinology to oncology, rheumatology and surgery) who frequently ask for it and rate it among the best POC tools available. As a group of health librarians and clinicians, we wondered whether the apparent differences of opinion and viewpoint about UTD’s value in the clinical setting would be borne out in the literature. We sought to examine the evidence by performing a systematic search and narrative review/analysis of all papers about UTD in head-to-head comparisons, evaluations and other empirical studies.
Research Questions:1. What is the consensus in the literature about UpToDate and its value to clinical decision-making?2. What are the differences between how UpToDate is perceived by medical and library professionals? For example, currency; ease of use; evidence-based methods; number of references. What are the most obvious differences? 3. What can medical and library professionals learn from each other about UpToDate? 4. In the absence of UpToDate in hospitals and health organizations, what other POC sources might be viewed as suitable more cost-effective replacements (or complements)?
Theme 1: Head to head comparisons of point-of-care tools – 10 papers (1 paper added Oct 1st )Theme 2: Physicians’ use of UpToDate vs. other resources – 18 papers (2 papers added Oct 1st )Theme 3: Quality of Hospital and / or Library with access to UpToDate – 2 papersTheme 4: Survey of Physician/Librarian Usage of UpToDate – 20 papersAppendix: Additional papers that mention UpToDate
n=50 papers (as of October 1st 2011)
Theme 1: Head-to-head comparisons of point-of-care toolsBanzi (2), Burkiewicz (Ketchum, Ketterman, Shurtz, Taylor, Trumble – Librarians,
Sayyah, Turan = 10 papers
Study authors / journal Date Tools used in Comparison Methods Results
Banzi R, Cinquini M, Liberati a, et al. Speed of updating online evidence based point of care summaries: prospective cohort analysis. BMJ. 2011;343(sep22 2):d5856-d5856
2011 Clinical Evidence, Dynamed, EBM Guidelines, eMedicine, and UpToDate
Evaluated updating speed by measuring incidence of research findings in POC summaries on potentially eligible newsworthy pieces of information
Selected all systematic reviews by American College of Physicians (ACP) Journal Club and Evidence-Based Medicine Primary Care and Internal Medicine
How fast point of care summaries are updated - used a prospective cohort1 design over one year
EBM Guidelines, UpToDate, and Dynamed refer to “a continuous update,” meaning that new research findings are incorporated into the summaries every time they are published.
UpToDate is the only product that clearly reports quantitative data on the topic updated (35% of all contents during a four month cycle).
Cochrane systematic reviews were more likely to be cited by EBM Guidelines than by UpToDate
Dynamed clearly dominates the other products in terms of updating
Banzi R, Liberati A, Moschetti I, Tagliabue L, Moja L. A review of online evidence-based practice point-of-care information summary providers. J Med Internet Res. 2010;12(3):1-16.http://www.jmir.org/2010/3/e26/
2010 Identifies 18 EBP POC summary providers, inc. UpToDate
Selected EBP point of care tools through Medline searches, Google, library websites, conference proceedings in 2008Assessed coverage, editorial quality / EBM methods in tools
UpToDate ranked #2 on editorial quality; #4 on evidence based methodology scores
Not about UpToDate specifically but ranked in top quartile for ebm scores / editorial quality
Burkiewicz JS , Vesta KS, Hume 2005 FIRSTConsult, Review of handheld Describes features/
1 Two reviewers independently checked whether each sampled systematic review was cited in at least one chapter of the five point of care information summaries. This was done monthly at the same time for each product.
AL. Update in handheld electronic resources for evidence-based practice in the community setting. Ann Pharmacother. Dec;39(12):2100-4. http://www.ncbi.nlm.nih.gov/pubmed/16288067?dopt=Abstract
InfoRetriever, and UpToDate
e-resources for EBP utility in community pharmacy setting
Searched Medline/Internet
requirements of each; UpToDate requires large system memory
(Nothing else provided in this summary?)
Ketchum AM, Saleh AA, Jeong K. Type of evidence behind point-of-care clinical information products: a bibliometric analysis. J Med Internet Res. Feb 18;13(1):e21.http://www.jmir.org/2011/1/e21/
2011 ACP PIER, Clinical Evidence, DynaMed, FirstCONSULT, and UpToDate
6 tools & four topics: asthma/hypertensionhyperlipidemia/ carbon monoxide poisoning; comparative analysis
4 indicators: distribution of citations, type of evidence, product currency & citation overlap.
Clinical Evidence & DynaMed show higher % of citations for asthma while ACPPIER and UpToDate have greater % of citations for hypertension. Conclusion: no standards for guidance on developing content for these products; differences in POC tools is seen in references
Ketterman, E. & Besaw, M. E. An Evaluation of Citation Counts, Search Results, and Frequency of Updates in DynaMed® and UpToDate®. Journal of Electronic Resources in Medical Libraries, 7(4), 273-280.http://www.informaworld.com/smpp/content~db=all~content=a930553472
2010 DynaMed and UpToDate
Head to head comparison based on 4 criteria: search result counts, search result answers, reference counts, and currency of updatesTotal of 141 of most recent topics generated
UpToDate had greater number of instances (63.8%) where initial result count was higher than DynaMed (36.2%).
Percentage of hits slightly higher for UpToDate (89.4%) compared with DynaMed (87.2%)
UpToDate had more instances (74) where it had more references compared to DynaMed (55)
DynaMed returned with more current topic 123 times, where UpToDate was more current 13x
Sayyah Ensan L, Faghankhani M, Javanbakht A, Ahmadi S-F, Baradaran HR. To Compare PubMed Clinical Queries and UpToDate in Teaching Information Mastery to Clinical Residents: A Crossover Randomized Controlled Trial. PloS One. 2011;6(8):e23487
2011 PubMed Clinical Queries and UpTodate
Participants in the study were randomly assigned to answer 2 clinical scenarios using either UpToDate or PubMed Clinical Queries. They then crossed to use the
First time users using UpToDate answer higher proportion of questions in shorter time than Pub Med Clinical Queries; UTD is efficient because: information is organized in entries rather than articles; each discusses a
other database to answer 2 different clinical scenarios. A proportion of relevantly answered clinical scenarios, time to answer retrieval, and users' satisfaction were measured in each database
complaint, disease, or a category of a disease; provided by integrating best evidence by experts to address management options for health problems and recommendations are graded on basis of level of evidence.
Shurtz S, Foster MJ. Developing and using a rubric for evaluating evidence-based medicine point-of-care tools. J Med Libr Assoc. 2011Jul;99(3):247-54
2011 ACP PIER, DynaMed, Cochrane Library, Essential Evidence Plus, and MD Consult's First Consult
Previous studies were designed to evaluate EBM tools to answer clinical questions. The goal here was to develop rubrics for assessing tools based on criteria (reviewing content, search options, quality control, and grading; searching for treatments of common diagnoses and evaluating summaries
Medical librarians play an important role in evaluating quality of POC tools; assessment tools, such as rubrics, aid in evaluation; some articles in lit review favoured one EBM tool BUT major finding of this review is current POC tools were quite similar relative to their content coverage and search options
Taylor B . What -- no UptoDate ? Regina Qu ' Appelle Health Region Library replaces UptoDate with BMJ ' s PointofCare . JCHLA. 2009 Dec; 30 (4): 147-8
2009 (UpToDate, Clinical Evidence (BMJ), ACP PIER, DynaMed, and FIRSTConsult)
Compared results for four conditions: hypertension, hyperlipidemia, carbon monoxide poisoning, and asthma, then determined the level of evidence of each reference
FIRSTConsult yields higher levels of evidence in its monographs followed by Clinical Evidence, ACP PIER, DynaMed and UpToDateStudies show UpToDate has some competitors that challenge its spot as point-of-care(POC) tool favoured by physicians, medical students
Trumble J , Anderson M , Caldwell M , et al . A systematic evaluation of evidence based medicine tools for point - of - care . Texas Health Science Libraries Consortium . In : Proceedings of the 34 th SCC / MLA Annual Meeting ; Oct 20–24. College Station , ( TX ). http://www.thslc . org / papers . ht ml
2006 3 products self-identified as POC tools ranked in 18 categories, levels of evidence, ease of navigation, and link to PubMed
Study by 7 health and medical librarians. Texas Health Science Libraries Consortium
ACP Pier and Clinical Evidence rated as top two POC tools, UpToDate fell in middle somewhere
Turan M, Serdar MA, Cihan M. Rapid access to information resources in clinicalbiochemistry: medical applications of Personal Digital Assistants (PDA). Clinicaland Experimental Medicine.Jun 30;8(2):117-122.
2008 ePocrates, Inforetrieve, Pepid, eMedicine,FIRST Consult, 5 laboratory ebooks released by Skyscape and/or Isilo)
Assessment based on # of tests included, softwarecontent of detailed information for each test-like process, interpretation ofresults, reference ranges, critical values, interferences, equations, pathophysiology,supplementary technical details such as sample collection principles and additional info such as linked references, evidence-based data, test cost, etc
“…it is necessary to keep several software products in our PDAs to meet our demands. For example, combining UpToDate with ePocrates or combining Pepid with a Skyscape ebook would be appropriate…”Regarding evaluation results, unique software is not available to combine evidence-based clinical & laboratory data
Theme 2: Physicians’ use of UpToDate vs. other resources to answer questions Ahmadi, Blackman, Campbell (librarian), Cao, Collier, Fenton (librarian), Frijns, Goodyear-Smith,
Graber, Hoogendam, Jaeschke, Kelson, Kim, Koonce, Lucas, Patel, Schilling, Shariff, Thiele n= 18 papers
Study authors / journal Date Resources used in comparison
Methods Results
Ahmadi SF et al. A comparison of answer retrieval through four evidence-based textbooks (ACP PIER, Essential Evidence Plus, First Consult, andUpToDate): a randomized controlled trial. Med Teach. 2011;33(9):724-30.
2011 ACP PIER, Essential Evidence Plus (formerly InfoRetriever),First Consult, and UpToDate.
112 residents were randomly allocated to four groups to use: (1) ACP PIER, (2) Essential Evidence Plus (formerly InfoRetriever), (3) First Consult, and (4) UpToDate. Participants received 3 of 24 questions randomly to retrieve the answers from the assigned textbook.
Attending and resident physicians have in-hospital access to computers capable of searching Best Evidence, Medline, UpToDate, and other evidence-based resources
Blackman D, Cifu A, Levinson W. Can an electronic database help busy physicians answer clinical questions? J Gen Intern Med.; 17Suppl1:220.4.
2002Textbooks, Medline, PDR, colleagues, web sites
2 study sites, 4 primary care doctors randomized to UpToDate; 6 physicians to controls; interviewer onsite 4-5 weeks spoke to MDs after visits / resources used to answer Qs / did answers changes patient care? Answers recorded
678 patient visits; most Qs re: therapy, diagnosis
UpToDate MDs answered more questions 34.3 vs. 18.7 p=0.17, change in pt plan 35% vs. 19.5% p=0.23
Intervention-almost entirely UpToDate 50%- medical textbooks 13.8% literature search 12.5 colleagues 6.7
Conclusions- access/training to UpToDate increases use; trend toward answering more Qs & change in Rx plans
Campbell R, Ash J. An evaluation of five bedside information products using a user-centered, task-oriented approach. J Med Libr Assoc. ;94(4): 435–41, e206–7.
2006 ACPPier, DISEASEDEX, FIRSTConsult, InfoRetriever
Randomized trial- physicians, residents, medical students, physician assistants, physician assistant students, nurses, nursing students, pharmacists, and informatics students with current or
Answered more Qs with UpToDate: 2.5 questions using UpToDate, p<0.001, 1.9 questions in DISEASEDEX, 1.7 questions in both FIRSTConsult and InfoRetriever & 1.6 Qs in ACP PIER.
previous clinical experience. No additional training was provided
Better user experience p <0.001 Higher satisfaction with UTD p<0.006
13 (73%) ranked UpToDate the best; no one ranked it the worst
Cao Y, Liu F, Simpson P, Antieau L, Bennett A, Cimino JJ, Ely J, Yu H. AskHERMES: Anonline question answering system for complex clinical questions. J Biomed Inform.Apr;44(2):277-88.
2011 AskHERMES AskHERMES allows physicians to enter a question with minimal formulation to navigateamong answer sentences to quickly meet their information needs
AskHERMES’ performance is comparable to other systems; when answering complex clinical Qs, has potential to outperformGoogle and UpToDate
Collier AP, Heilig LF, Schilling LM, Dellavalle RP. Clinical questions asked by medical students: a learning tool for dermatology rotations. Dermatology;214:108-111
2007 Journal article/abstracts, websites and textbooks
Medical students in dermatology clinics at Denver VA had to formulate and answer a clinical question during patient encounters, then complete a survey about experience.
49% (44/89) of medical students finished study
Most frequently used source of clinical information: journal abstracts/ articles (55%, 24/44), UpToDate (50%, 22/44), websites (27%, 12/44) printed textbooks (25%, 11/44).
Increased knowledge, confidence and patient care skills after completing exercises
Fenton SH , Badgett RG . A comparison of primary care information content in UpToDate and the National Guideline Clearinghouse . J Med Libr Assoc Jul ;95(3):255-9. Erratum in : J Med Libr Assoc . 2007 Oct ;95(4):473.
2007 Head to head comparison of UpToDate and NGC
752 ?Qs selected from clinical Qs collection of NLM
UTD and National Guidelines Clearinghouse (NGC) searched utilizing keywords from ?s
UpToDate returned 1 or more documents for 580 questions, NGC returned at least 1 document for 493 questions (77.1% versus 65.5% of question sampled, P = 0.001)
Together they returned content for 91% of searches (n = 685).
NGC retrieved a mean of 16.3 documents per Q vs 8.7 documents from UpToDate
Frijns J, Dyserinck H, Gu P, Loep M. Finding the evidence for therapeutic PICO questions on four electronic resources. Library Review. Jan 1;53(3):142-149.
2004 TRIP,Cochrane Library, SUMSearch.
Chose ten therapeutic questions from PDQ Evidence-Based Principles and Practice and some EBM related Web sites and formulated them into the structured clinical questions by using a PICO process
All search examples about therapeutic interventions. For each, a different search strategy was built to be performed on four electronic resources, using MeSH, text words or both if applicable. All MeSH were exploded
Topic review is the exclusive type of study in UpToDate
Users need to use one or two common medical terms to obtain hits. Search terms include name of disease, symptoms, trial or drug name but direct Boolean search, author’s name, journal or year not yet permitted
# of search results in UpToDate or TRIP is smaller than in Cochrane or SUMSearch
Shortest time on UpToDate to get answers compared to other tools
Goodyear-Smith F, Kerse N, Warren J, Arroll B. Evaluation of e-textbooks. DynaMed, MD Consult and UpToDate. Aust Fam Physician. Oct;37(10):878-82.
2008 DynaMed,MD Consult (including FirstConsult)
200 general practitioners accessed 3 e-textbooks. Completed survey and used random selection to answer 4 clinical questions: screening, diagnosis, treatment & prognosis.
122 GPs had at least one hit through study site. 84 GPs completed questionnaire; 77 completed interview (36% of enrolled, 61% of users).
Favourite e-text in questionnaire, 35 (43%, 95% CI: 32, 54) is UpToDate
Users more likely to prefer UpToDate; as a whole - no winner; most-preferred text in same order as before w 31 (43%) rating UpToDate as most preferred
UpToDate often gave too much info; more GPs prefer UTD as textbook of choice (43%); 75% of med
students used UTD when offered access
Graber MA, Randles BD, Ely JW, Monnahan J. Answering clinical questions in the ED. Am J Emergency Medicine. Feb;26(2):144-7.
2008 Print, e/textbooks, Google
Observational study of 26 physicians at 2 institutions; physicians followed for 2 shifts; number / type of questions recorded. % answered, resources used, barriers to answering recordedN= 235 questions were recorded or about 5 questions per 8-hour shift
Most common source paper or computer drug reference (47 [37%] of 126 questions). Online resources comprise 2nd most common resource, (36 of 126 or 29%) with Google and UpToDate dominating
Hoogendam A, Stalenhoef AF, Robbé PF, Overbeke AJ. Answers to questions posed during daily patient care are more likely to be answered by UpToDate than PubMed. J Med Internet Res. Oct 3;10(4):e29.
2008 PubMed & UpToDate
40 residents & 30 internists working in academic medical centers use PubMed and UpToDate according to observation portal in daily medical careAnalyzed 1305 patient-related questions sent to PubMed and/or UpToDate between October 1, 2005 and March 31, 2007 using portal. Complete answer was found in 594/1125 (53%) questions sent to PubMed or UpToDate
UpToDate answered more questions than PubMed on all major topics but difference detected when question was related to etiology (P < .001) or therapy (P = .002).
Time to answer was 241 seconds (SD 24) for UpToDate and 291 seconds (SD 7) for PubMed.
Participants preferred UpToDate & answered more patient-related questions in daily medical care with UpToDate than PubMed
Comparable to previous research: UpToDate preferred over PubMed; perceived as equal or better for patient Qs.Findings show starting search in UpToDate followed by PubMed (when answer is not
satisfactory) is sensible strategy
Jaeschke R. Up To Date [review] Evid Based Med 5: 40.
2000 Cochrane reviews Short review where colleague unfamiliar with UpToDate identified 10 clinical questions for which good quality evidence is available10 clinical trainees on clinical teaching unit in Hamilton, Ontario, were asked to grade user friendliness of variety of evidence resources
On scale from 1 (easy to use), to 3 (soso), to 5 (difficult to use), UpToDate assigned “1” by all ten trainees (best mean score 1.8, Cochrane 3.8).
UpToDate most com-monly used information source among trainees
UpToDate used for general orientation to a topic - but people trying to learn more about basic biology, physiology or anatomy will want to look elsewhere. (2000 – older study)
Kelson J. UpToDate rated highest in a combined task assessment/user-satisfaction study of 5 clinical information resources. Evidence Based Library and Information Practice.;2(3): 91-3.
2007 ACP’s PIER, DISEASEDEX, FIRSTConsult, InfoRetriever
Participants randomly allocated to 3 test questions which they tried to answer using each of 5 commercially available clinical information resources
Participants were physicians (44%), medical informatics students with clinical experience (28%), pharmacists (17%), nurse (6%) and MRI technologists (6%).
Participants answered more questions with UpToDate (average 2.5 questions) compared to other resources; ranged from average of 1.6 (ACP’s PIER) to 1.9 (DISEASEDEX) Qs answered
UpToDate scored higher (Friedman test) on ease of use, clarity of screen layout and how well participants’ needs were satisfied
13 participants (73%) rated UpToDate as best
** Caution advised with respect to conclusions as 72% of
participants were familiar with UTD already **
Kim S, Willett L, Murphy D, O'Rourke K, Sharma R, Shea J. Impact of an evidence-based medicine curriculum on resident use of electronic resources: a randomized controlled study. JGIM: Journal of General Internal Medicine. November;23(11):1804-1808.
2008 Ovid, InfoPOEMs (MDConsult Non-medical resources (Google)
All second and third-year med residents (N=50) randomized to EBM teaching group or control using computer-generated randomization stratified by level of training.
Residents assigned to EBM teaching participated in six 2-hour workshop sessions (12 hours total) during an elective month
Twenty-five residents were randomized to EBM teaching group & 25 to control group
Most commonly accessed resources for EBM teaching group were Ovid (71% of residents accessed) and InfoPOEMs (62%).
Majority of control group accessed UpToDate (67%) & MDConsult (58%).
UpToDate and eMedicine do not consistently use explicit evidence-based processes but provide similar recommendations for patient management as evidence-based resources
Koonce TY, Giuse NB, Todd P. Evidence-based databases versus primary medical literature: an in-house investigation on their optimal use. J Med Libr Assoc. Oct;92(4):407-11.**Librarian study**
2004 Cochrane Database of Systematic Reviews
HealthGate Clinical Guidelines
40 complex clinical questions randomly selected from in-house archival database received by librarians during clinical roundsLibrarian team asked to search UpToDate, Cochrane Database of Systematic Reviews, and HealthGate Clinical Guidelines (formerly, EBM Solutions)Librarians established consensus on whether a question was fully or partially answered by one of the resources and instructed to record instances where primary literature was needed to answer question completely.
EBM resources provide answers for general patient care management questions more frequently than for complex clinical questions. Results showed that EBM resources provide more partial answers for complex CICS questions than for the Pathways questionsNo direct mention of UpToDate
Lucas BP, Evans AT, Reilly 2004 Best Evidence, Pre-post evaluation Attending and
BM, Khodakov YV, Perumal K, Rohr LG, Akamah JA, Alausa TM,Smith CA, Smith JP. The impact of evidence on physicians' inpatient treatment decisions. JGen Intern Med. May;19(5 Pt 1):402-9.
Medline,“Searching the literature could improve the treatment of many medical inpatients, including those already receiving evidence-based treatment…”
resident physicians have in-hospital access to computers capable of searching Best Evidence, Medline, UpToDate, and other evidence-based resources
Patel MR, Schardt CM, Sanders LL, Keitz SA. Randomized trial for answers to clinical questions: evaluating a pre-appraised versus a MEDLINE search protocol. J Med Libr Assoc.;94(4):382-7.
2006 MEDLINE first and last search protocol
Residents randomized to MEDLINE-first (protocol A) group searched 120 questions, and residents randomized to the MEDLINE-last (protocol B) searched 133 questions.
“...UpToDate can be costly, ranging from over $200 per individual to over $40,000 for an institution”
In protocol A, 104 answers (86.7%) and, in protocol B, 117 answers (88%) were found to clinical questions. In protocol A, residents reported that 26 (25.2%) of the answers were obtained quickly or rated as "fast" (<5 minutes) as opposed to 55 (51.9%) in protocol B, (P = 0.0004).
Subset of questions and articles (n = 79) reviewed by faculty who found both protocols identified similar numbers of answer articles that addressed questions and were felt to be valid using critical appraisal criteria
Schilling LM, Steiner JF, Lundahl K, Anderson RJ. Residents' patient-specificclinical questions: opportunities for evidence-based learning.
2005 MEDLINE In 2002, 43 internal medicine residents at University of Colorado outpatient clinic answered specific clinical questions based
Most frequently used methods to retrieve information were Medline 73% (115/158) and UpToDate 70%
Acad Med.;80(1):51-6.
on patients seen as out-patients
(110/158).
UpToDate (45%, 71/158) & journals (42%, 66/158) were most helpful;
Residents rated impact of information on five point scale (1 = strongly disagree; 5 = strongly agree) for the following: used to assist patient care (mean 4.1), improved care (mean 4.0), improved communication (mean 4.3), improved confidence in care (mean 4.3), improved knowledge (mean 4.6), and improve care for future patients (mean 4.4)
Shariff SZ, Bejaimal SA, Sontrop JM, Iansavichus AV, Weir MA, Haynes RB, Speechley MR, Thind A, Garg AX. Searching for medical information online: asurvey of Canadian nephrologists. J Nephrol. 2011 Feb 23:0.
2011 UpToDate (92%), PubMed (89%), Google (76%) and Ovid MEDLINE (55%)
Survey looked at how nephrologists use online information sources; cross-sectional study (2008 to 2010); random sample of Canadian nephrologists’ survey of online search practices; respondents revealed search preferences, use of 9 online information sources
Nephrologists routinely use a variety of online resources to search for information for patient care. These include bibliographic databases, general search engines and specialized medical resources.
Thiele RH, Poiro NC, Scalzo DC, Nemergut EC. Speed, accuracy, and confidence in Google, Ovid, PubMed and UpToDate: results of a randomised trial. Postgrad Med J;86:459-465.
2010 Google, Ovid, PubMed
Medical students, residents & attending physicians answered four anaesthesia and/or critical care questions in 5 min, using Google, Ovid, PubMed, or UpToDate (only one search tool per Q). After each search, participants rated results on a four
Users of Google and UpToDate were more likely than users of PubMed to answer questions correctlySubjects had most confidence in UpToDate; searches with Google and UpToDate were faster than with PubMed or Ovid
point scale; one to 3 weeks later, users were randomized to one of 4 search tools to answer eight questions, four of which were repeated
Theme 3: Quality of Hospital and / or Library with access to UpToDate Bonis, Jones n=2 papers
Study authors / journal Date Methods Results
Bonis PA, Pickens GT, Rind DM, Foster DA. Association of a clinical knowledge support system with improved patient safety, reduced complications and shorter length of stay among Medicare beneficiaries in acute care hospitals in the United States. Int J Med Inform. Nov;77(11):745-53. Epub 2008 Jun 19.
2008 Compared US hospitals with online access to UpToDate with hospitals in the top 100 Thomson database
“…Hospitals with access to UpToDate (n=424) were associated with significantly better performance than other hospitals in the Thomson database (n=3091) on risk-adjusted measures of patient safety (P=0.0163) and complications (P=0.0012) and had significantly shorter length of stay(by on average 0.167 days per discharge, 95% confidence interval 0.081-0.252 days, P<0.0001). All ofthese associations correlated significantly with how much UpToDate was used at each hospital. Mortality was not significantly different between UpToDate and non-UpToDate hospitals….”
Hospitals with UpToDate n=424 had significantly better performance than other hospitals in the Thomson database n=3-01 on risk adjusted measures of pt safety, complications, shorter length of stay
“Whether UpToDate was a marker of quality, an independent cause of it or one of several features of hospital quality that led to better performance remains unclear.”
Jones DA, Poletti EJ, Stephenson P. Demonstrating the Value of Library Services in South Central VA Health Care Network Medical Centers. Journal of Hospital Librarianship. 2010; 10(3): 211-23.
2010 A regional network studied value of library services by focusing on I. value provided by literature searches II. interlibrary loan through statistics and comparisons of commercial costs and III. efficiency of electronic resources through cost per use
Results: mediated searching impacts decision making and cost; library cost benefits, lower cost per use of regional resources. Shows value of library services & role of provision of health information in quality care
See also: Lucas, 2004See also: Schilling LM, Steiner JF, Lundahl K, Anderson RJ. Residents' patient-specific clinical questions: opportunities for evidence-based learning. Acad Med. ;80(1):51-6. where Residents rated impact of information on five point scale (1 = strongly disagree; 5 = strongly agree) for the following: used to assist patient care (mean 4.1), improved care (mean 4.0), improved communication (mean 4.3), improved confidence in care (mean 4.3), improved knowledge (mean 4.6), and improve care for future patients (mean 4.4)
Theme 4: Survey of Physician/Librarian Usage of UpToDate Chiu, Cooper, Davies (librarian), Dee, Dezee, Edson, Farrell (librarian), Johnson, Lai, Leff, McCord, Peterson,
Phua, Peterson, Shariff, Somal, Thun, Wallace, Wandersee, Wheeler (n=20 papers)
Study authors / journal
DateType of Physician OR Librarian Led study?
Methods Results
Chiu Y , Weng YH , Lo HL , Ting HW , Hsu CC , Shih YH , Kuo KN . Physicians ' characteristics in the usage of online database : a representative nationwide survey of regional hospitals in Taiwan. Informatics for Health and Social Care. ;34(3):127-135.
2009Physician survey
Structured physician survey in Taiwan; 457 complete responses. Internet-based resources accessed more than personal or paper ones for clinical information
Asked if they had access to Index to Chinese Periodical Literature (ICPL), Chinese Electronic Periodical Services (CEPS), Cochrane Library, MD Consult, MEDLINE, ProQuest & UpToDate
Best -known databases- MEDLINE (80.1%) and ICPL (76.4%), Cochrane Library (49.9%) CEPS (43.8%) ProQuest (37.9%), MD Consult (38.1%), and UpToDate (36.1%)
Most accessed- (1) MEDLINE (86.2%), (2) Cochrane Library (51.2%), (3) ICPL (48.6%), (4) UpToDate (45.7%), (5) ProQuest (43.8%), (6) MD Consult (43.3%) and (7) CEPS (33.1%).
Cooper AL, Elnicki DM. Resource utilisation patterns of third-year medical students.Clin Teach. Mar;8(1):43-7.
2011 Librarian Led study
144 students surveyed Most medical students use UpToDate for physician rounds & admitting (64 and 67%); exam prep; question books associated with significant improvement in scores (77 vs 72%, p < 0.01).
Davies KS. Physicians and their use of information: a survey comparison between the United States, Canada and the United Kingdom. J Med Libr Assoc. ;99(1):88-91.
2011 Librarian Led study
498 e-mails were delivered to US physicians and 416 to Canadian physicians.80 responses were received from both US and Canadian physicians, which was aresponse rate of 16% and 19%, respectively.
In UK, 2351 emails were delivered with 636 questionnaires completed representing a response rate of 27%.
Physicians in US (43.8%) and Canada (40.0%) more than twice as likely to use wireless laptops compared to UK (18.8%). Canadian (51.3%) and US physicians (41.3%) more than twice likely to use personal digital assistants (PDAs) compared to UK physicians (16.3%)
Most-frequently used subscription-based resources were UpToDate & MD Consult; used more often by physicians from US and Canada compared to UK.
Previous research compares physicians' use of electronic resources, MEDLINEfeatures, preferred sources for clinical information, perceived barriers to accessinginformation for patient care in specific countries, attitudes toward evidence-basedmedicine (EBM) and understanding of EBM
Study compares aspects focusing on EBM resources in Canada, UK & US.
MEDLINE/PubMed most frequently used resource for all physicians.
Dee KJ, Durning S, Denton GD. Effect of electronic versus print format and differentreading resources on knowledge acquisition in the third-year medicine clerkship.Teaching & Learning in Medicine. Fall; 17 (4): 349-54.
2005 Medical students at the USU in Bethesda (uniform services university)
Determines if format (electronic or print) or choice of reading materials is associated with knowledge acquisition during 3-year internal medicine clerkship
114 -3rd-year internal medicine clerkship students participated
Most commonly reported resources were UpToDate (99%), review or question book (93%), and Harrison's Principles of Internal Medicine (82%)
DeZee KJ , Durning S , Denton GD . Effect of electronic versus print format and different reading resources on know . ledge acquisition in the third - year medicine clerkship. Teaching & Learning in Medicine. Fall; 17 (4): 349-54.
2005 Medical students
Prospective cohort study- survey of reading materials used for NBME exam med students
114 3rd year int med students surveyed
UpToDate (99% some use (58% most important resource), review or question book (93%, most important 27%), and Harrison’s Principles of Internal Medicine (82%, most important 7%).
UpToDate was most useful, then MDConsult then Harrison’s
Edson RS , Beckman TJ, West CP, Aronowitz PB, Badgett RG, Feldstein DA, Henderson MC, Kolars JC, McDonald FS. A
2010 Residents Residents at 5 IM residencies surveyed on reading and learning habits and preferences
189/413 responses- 77.7% of residents reported reading less than 7 h a week; 81.4% read in response to patient care encounters.
multi-institutional survey of internal medicine residents' learning habits. Med Teach. 2010; 32(9): 773-5.
Preferred electronic; 94.6% UpToDate most effective for knowledge acquisition; 88.9% UpToDate first choice for answering clinical questions
Farrell A . An evaluation of the five most used evidence based bedside information tools in Canadian health libraries . Evid Based Libr Inf Pract .;3(2):3– 17.
2008 Librarian led study
Survey to CANMEDLIB electronic mail list to identify most frequently used evidence based bedside information tools; 52 responses out of possible 450-500.
Clinical questions were used to measure the comprehensiveness of each resource and the levels of evidence they provided to each question
5 most used evidence based tools were BMJ Clinical Evidence, UpToDate, First Consult, Bandolier & ACPPier
Librarians satisfied with ease of use, efficiency & informative nature of resources
UpToDate was easiest to use & most comprehensive- provided information for highest # of clinical questions, level of evidence only 7% of the time
Johnson PT, Makary MA, Chen JK, Eng J, Fishman EK. A Comparison of World Wide Web Resources for Identifying Medical Information. Academic Radiology. Sep 1;15(9):1165-1172.
2008 Medical students at Johns Hopkins
Case cross-over study randomly assigned 89 medical student volunteers to use either Google or any other web-based resource (excluding Google) to research 10 advanced medical questions in a multiple choice exam.
A total of 135 students enrolled, for an initial response rate of 28% (135/480)
Few students said they used UpToDate for detailed, complicated or management information, relying on other sites (Wikipedia, eMedicine) for basic information
“UpToDate is recommended by the American Academy of Family Practitioners.”
Despite preferences for UpToDate, when instructed to use resources other than Google to identify medical information, most common tool selected was a search engine (Yahoo, AltaVista) followed by Wikipedia
Lai CJ, Aagaard E, Brandenburg S, Nadkarni M, Wei HG, Baron R. Brief report: multiprogram evaluation of reading habits of primary care
2006 Medical students
Participants asked open-ended /5-point Likert questions about reading habits: time spent reading, preferred resources, motivating and inhibiting factors
98% of residents reported using UTD regularly, 44% literature search, 35% Google or other search engines
Motivating factors were medical problems of
internal medicine residents on ambulatory rotations. J Gen Intern Med. 2006;21(5):486–9.
124 second- or third-year PCIM residents on ambulatory care rotation for at least 7 days without overnight call responsibilities
individual patients and preparation for presentations
Inhibiting factors (including family / personal responsibilities & lack of motivation)
Online-only sources used most often; UpToDate used by almost all respondents; in a shift from early 1990s, when textbooks, journals, and pocket manuals were primary medical resources
Leff B, Harper GM. The reading habits of medicine clerks at one medical school:frequency, usefulness, and difficulties. Acad Med.;81(5):489–94.
2006 Internal medicine clerkships (medical students)
18-item survey tool to 120 students on their first inpatient internal medicine clerkships
One hundred twelve of 120 (93%) medicine clerks completed survey. Said theyread an average of 10.8 (SD 5.6) hours per week (median ten hours per week, range one to 30 hours per week)
Most commonly used and useful reading sources were UpToDate & test prep texts
Students spend significant time reading online sources during medicine clerkship, especially UpToDate
Medical educators should be familiar with sources and contribute to maximizing effectiveness for students
McCord G, Smucker WD, Selius BA, Hannan S, Davidson E, Schrop SL, Rao V,Albrecht P. Answering questions at the point of care: do residents practice EBM ormanage information sources? Acad Med.;82(3):298-303. **good study**
2007 Family residents – medical students observed
Questionnaire asked 37 full-time faculty and residents about best sources, subscriptions,why they use PDAs; experience preventing medical errors
44% of questions were found by asking attending physicians, 23% by consulting PDAs; 20% books; 72% answered in 2 mins; residents rated UpToDate as best source but used it only 5x
PDAs were used for ease, time, accessibility; errors discovered or prevented with PDAs were medication related; no participants required use of specificmedical info resources
Phua J, Lim TK. How residents and interns utilise and perceive the personaldigital assistant and UpToDate. BMC Med
2008 Residents and interns
Study evaluated how residents and interns use personal digital assistants & UpToDate
Questionnaire survey of
Only 76 doctors (56.7%) used UpToDate; even though hospital had institutionalsubscription; 93.4% of users recommend UpToDate; only 57.9% say UpToDate led to
Educ. Jul 14;8:39. **good study**
residents and interns in tertiary teaching hospitals; out of168, 134 (79.8%) responded; 54 doctors (40.3%) owned PDA; PDA seen as most useful for drug information, followed by med references, medcalcs
change in care
“.... 93.4% of users recommend UpToDate to colleague, [but] only 57.9% stated use of UpToDate had led to change in management of patients...”
Peterson Phua MW, Rowat J, Kreiter C, Mandel J. Medical students' use of information resources: is the digital age dawning? Acad Med. 79(1):89-95.
2004 Medical students on clerkships
Med students adopted UpToDate as clinical resource during clerkships
116 of 154 students (75%) responded
85% identified primaryresources (UpToDate 53%, MDConsult 33%; p <.001 compared to paper; reported using info resources daily and required less than 15minsto answer most questions
Shariff SZ, Bejaimal SA, Sontrop JM, Iansavichus AV, Weir MA, Haynes RB,Speechley MR, Thind A, Garg AX. Searching for medical information online: asurvey of Canadian nephrologists. J Nephrol. Feb 23.
2011 Canadian nephrologists
Surveyed search preferences, practices and use of 9 online information sources
Respondents (n=115; 75% response rate) comprised academic (59%) and community-based (41%) nephrologists; average of 48 years old and in practice for average of 15 years
Nephrologists used online sources for patient treatment info such as UpToDate (92%), PubMed (89%), Google (76%) and Ovid MEDLINE (55%)
Community-based nephrologists more likely to use UpToDate (91%),while academic nephrologists divided between UpToDate (58%) & PubMed (41%)
Somal K, Lam WC, Tam E. Computer and internet use by ophthalmologists and trainees in anacademic centre. Can J Ophthalmol. Jun;44(3):265-8.
2009 Residents and trainees
Online survey eMedicine used more than UpToDate
Thun AMV, Stephens MB. Military Medical Informatics: Accessing Informationin the Deployed Environment. Military Medicine. Mar 1;174(3):259-264.
2009 Military physicians
What sources are commonly used by staff military physicians, survey asked todescribe sources of medical information used
Military physicians use Internet to access medical info every day; frequency and pattern of use differs if on deployment; common sources are general search engines (Google)
PubMed & MEDLINE are more commonly used by
military surgeons, while filtered secondary sources(UpToDate, MD Consult) used by military primary care physicians
Wallace RL. ETSU medical residents' clinical information behaviors,skills, training, and resource use. East Tennessee State University.
2007 Residents & clinical faculty
Residents surveyed about use of info ; 217 residents surveyed,105 returned survey for return rate of 48% ; clinical faculty surveyed in order to measure responses of residents against their instructors
Residents used Google and Web frequently. PubMed was rated as valuable resource; journals and UpToDate were important resources for residents
Wandersee JR. Library check-up; medical residents and fellows' libraryquestionnaire responses. Journal of Hospital Librarianship, 2008 8(3), 323-331.
2008 Library investigator
Residents didn’t know about library services; many said they could improve searching skills & wanted training but did not attend 1-hourorientation; 65% said they wanted more training
Reported use of UpToDate was 90% in 2005 and 2006
UpToDate and OVID were most popular
Wheeler TR. Implementation of a medical knowledge resource in the electronic medicalrecord -- can more clinical questions be answered at the point of care? J Hosp Librarianship.2007;7(4), 15-29.
2007 Primary care providers
Information seeking behaviors by providers; studied to see needs while seeing patients; based on patterns established in literature
119 respondedWhy UpToDate?1. Users feel answers to questions are in UTD2. Access and proximity (only 1 click away)3. Physician champions4. Intuitive interface (easy navigation)5. Timeliness: updated regularly6. Improved speed of network, decreasing wait time
45% relied on textbooks; 13% on journals; 28% on electronic resources as primary sources; questionnaire asked clinicians what e-products used most based on resources available
26 % preferred UpToDate, 16% PubMed, 13%Micromedex; 12% UpToDate preferred OVID Medline with links to full text
Appendix: Other papers that mention UpToDate Jones DA, Esparza J, Duggar DC. Reflecting on the concept of connecting clinicians with library resources and librarians through the electronic health record system [abstract]. Medical Library Association Conference, Washington, DC, 2010.“Objective: To evaluate usefulness of links to selected library resources placed in the electronic health record (EHR) system as perceived by internal medicine residents and to collect statistical data on usage of the AskALibrarian link in the EHR. Methods: Through a collaborative effort with computer services, the library now offers links in the EHR to: (1) library resources to aid clinical decision making, (2) the library’s consumer health information (CHI) site to provide educational materials for patients, and (3) the AskALibrarian service. To build awareness, these resources and services have been promoted through announcements on the library web page, through email messages from computer services, in the institutional and library newsletters, and during morning report and hospital rounds. Focus groups with internal medicine residents were held to determine their perceptions about the usefulness of these electronic library resources linked from within the EHR. Participants were consulted on which resources they find most useful in clinical decision making. Additionally, statistics are being collected on the number of questions received through the AskALibrarian link in the EHR. Results: Most of the residents who participated in the focus groups were aware of at least some library resources in the EHR, although none of them reported being aware of healthelinks, the …library’s CHI website. When asked about the most useful resource in the EHR for clinical decision making, UpToDate was mentioned most often. Some indicated that they like DynaMed because they can access it on their smart phones. None of them reported using the AskALibrarian form in the EHR, as supported by usage statistics. Participants’ responses indicated they preferred personal contact with librarians. Conclusion: The library staff must find additional ways to promote awareness of resources in the EHR and to educate users regarding CHI resources for patients. The library and computer services should replace nonused resources in the EHR with those which clinicians indicate they need…”
McDonald FS, Zeger SL, Kolars JC. Factors associated with medical knowledge acquisition during internal medicine residency. J Gen Intern Med. 2007 Jul;22(7):962-8. Epub 2007 Apr 28. “…UpToDate (Waltham, MA) is a proprietary electronic knowledge resource widely used by trainees and clinicians because of ease of use, broad coverage of multiple topics, and frequent updates.21–24 This learning resource was selected as a study parameter for 2 reasons. First, UpToDate is a common knowledge compendium that is used by many internal medicine residents. A survey of nearly 18,000 residents accompanying the IM-ITE in 2003 noted that UpToDate was the most frequently used electronic resource for clinical information (Schultz, October 4, 2005, personal communication). Second, usage of this learning resource is objectively and quantitatively measurable. To facilitate provision of CME credits, UpToDate tracks usage for each subscriber. Usage data are measured by the amount of time a user is logged on to UpToDate, with a maximum of 10 minutes recorded for any topic per accession. For example, if a topic is accessed for 3 minutes and then the user logs off or accesses another topic within the program, then, measured time for the topic is 3 minutes. If a topic is accessed and then left open for more than 10 minutes, the measured time is 10 minutes. Printing a topic review results in 10 minutes of credit. Hence, 1 hour of credit represents reading or printing at least 6 topics.Beginning in May 2002, the Mayo Clinic Internal Medicine Residency, Rochester, MN purchased individual subscriptions to UpToDate for all of its residents for internet use (see Table 1). Residents were provided with UpToDate without any expectations of when it should be used or what should be read. Thus, each individual resident’s use of the resource was self-directed. Neither the residents nor the investigators knew that UpToDate use was being electronically tracked before the 2003 IM-ITE. Residents had access to UpToDate for at most 6 months before the 2002 IM-ITE exam and 18 months before the 2003 exam. None of the authors have any affiliation with UpToDate nor have they received any financial support from UpToDate. Other than providing the requested data, UpToDate was not involved in the design of the study nor the analysis of the data.”
