Effectiveness of training health professionals in literature search skills using electronic health databases—a critical appraisal

© Health Libraries Group 2003

Health Information and Libraries Journal

,

20

, pp.33–41

33

Blackwell Publishing Ltd.

Effectiveness of training health professionals in literature search skills using electronic health databases—a critical appraisal

Anupama Garg* & Kathleen M. Turtle†, *Library Database Training Facilitator, †Librarian, Postgraduate Education Centre, Chorley District General Hospital, Preston Road, Chorley, UK.

Abstract

The objective was to assess the effect on health professionals’ skills of one toeight hours literature search and retrieval training from electronic health data-bases. We searched: Cochrane library (2002; Issue 3),

(1977–2002/5),

(1980–2002/7);

(1982–2002/5);

(1982–2002/7),

(1994–2002/5),

(1985–2002/6);

(1969—current), NRR (2002, Issue 2), theworld-wide-web and references. The selection criteria consisted of randomisedcontrolled trials, controlled before and after, and controlled cohort studies incomparison with no training. The intervention had to be one to eight hourstraining in literature search and retrieval skills for health professionals. Theoutcome was the effect on health professionals’ literature search and retrievalskill levels measured through reliable instruments. For data collection and ana-lysis, one reviewer extracted data and assessed the quality of the studies and thesecond reviewer checked it. The results indicate that there is some evidence ofpositive impact on health professionals’ skill levels in literature searching andthey find the training useful. In conclusion, the size of the positive effect isdebatable as only three small and methodologically weak studies met the inclu-sion criteria and out of those only two showed the positive effect.

Background

Clinical governance is a UK government initi-ative to improve the quality of service within theNational Health Service.

1

Evidence-based prac-tice, continuous professional development (CPD)and lifelong learning have all been identifiedas instruments of clinical governance or qualityimprovement.

2

Accessing and retrieving relevant and up-to-date literature is one of the major parts of

evidence-based practice. It is also crucial for life-long learning and CPD of health professionals.The provision of online access to world-wideknowledge bases, to support decision making andprofessional development, has been recognizedas an information strategy objective by the UKgovernment.

3

National Health Services’ research anddevelopment strategy also places a great emphasison electronic means of dissemination of researchfindings.

4

The national policy to disseminate inform-ation electronically assumes that health profes-sionals have access to and the skills to searchappropriate databases in order to access andretrieve information.

Correspondence: Kathleen M. Turtle Postgraduate Education Centre,Lancashire Teaching Hospitals NHS Trust, Chorley Hospital, PrestonRoad, Chorley PR7 1PP, UK. E-mail: [email protected]

Effectiveness of training in literature search skills,

Anupama Garg and Kathleen M. Turtle



,

20

, pp.33–41

34

This paper focuses on the local level wherehealth professionals working for Chorley andSouth Ribble NHS Trust, Preston Acute Hospi-tals NHS Trust, Preston Primary Care Group(PCG) and Chorley Primary Care Trust use thelibrary services situated at Preston Acute Hospi-tal, Chorely and South Ribble District GeneralHospital and Sharoe Green Hospital. (Since thissystematic review was written, Chorley and SouthRibble NHS Trust and Preston Acute HospitalsNHS Trust, mentioned above) have unified tobecome Lancashire Teaching Hospitals NHSTrust.)

Training in literature search skills using onlinedatabases is offered either as a scheduled one-hoursession or on demand. During these sessions,librarians are frequently asked questions regard-ing searching and computer literacy.

The top eight questions that are asked and the inferred training needs of the local health professionals

1

I am not very good with computers. What do Ido? (Computer literacy.)

2

Can you show me how to use the computer forfinding research? (Sources of information andexperience with the search interface.)

3

How can I get the research on this topic?(Sources of information, knowledge about thedatabases and principles of searching.)

4

There are thousands of articles on this topic.How do I get fewer? (Use of Boolean and Limitsfunction.)

5

What do these stand for? (Understanding offield labels.)

6

Can I save this search? (Experience with thedatabase search interface and computer literacy.)

7

How do I print out these references? (Com-puter literacy and experience with the searchinterface.)

8

Why can’t I see the full article? (Assumed avail-ability of full text of articles.)These questions demonstrate a clear need for

training. The one-hour sessions currently pro-vided are inadequate in terms of content coveredand trainee skills developed and the librarians feeldissatisfied with the quality of training they areable to provide in an hour.

The need for training in electronic literaturesearch and retrieval among Chorley and Prestonhealth professionals is not unique. Pyne

et al.

’s

5

survey of London acute and community servicehealth professionals and Urquhart

et al.

’s

6

studyof trainee general practitioners and hospitaldoctors from 13 hospital sites in England andWales reveal similar needs and it would be logicalto assume that there is a UK-wide need fortraining in literature search skills among healthprofessionals.

The librarians would like to provide a moreco-ordinated and comprehensive literature searchand retrieval training programme with a built-inelement of computer literacy for the local healthprofessionals. Before starting such a trainingprogramme, a review of the literature was carriedout in order to establish what the optimum periodof training was. The timescales considered werebetween one hour and eight hours.

Eight hours is an arbitrary upper time limit,chosen because we believe it is the maximum thelibrary professionals can provide due to timeconstraints.

Objective of the review

To assess what effect training (one to eight hours)has on health professionals’ search and retrievalskills, when they are using electronic healthdatabases.

Criteria for considering studies for the review

Type of intervention.

Educational interventionswith a co-ordinated online literature search andretrieval education programme (1–8 h) offered,either as a single course, or as part of anothercourse, with the proviso that there is data includedwhich demonstrates improvement in literaturesearch and retrieval skills.

Types of outcome measures.

Health professionals’skills in literature search and retrieval fromelectronic databases were assessed by measuringpre- and post-training competence.

Types of studies.

Randomised controlled trials(RCTs), controlled before and after studies





,

20

, pp.33–41

35

(CBAs) and controlled cohort studies. Theminimum requirement is that there has to be acomparison with no training.

Types of participants.

Qualified health professio-nals and student health professionals in anysetting.

Search strategy for the identification of studies.

Subject-specific search strategies were formulatedusing free text words (ft) and thesaurus or indexterms (ME) for each database. Both print andelectronic sources were used.

Sources

•

Cochrane Library (2002, Issue 3)

using: Health-Personnel* (ME), PAM* (ft), Students-Health-Occupations* (ME), Evidence-Based-Medicine*(ME), evidence-based practice (ft), literaturesearch* (ft), evidence search* (ft), databasesearch* (ft), Education* (ME).

•

MEDLINE

(1977–2002/5)

using: Health-Personnel(ME explode), PAM* (ft), Students-Health-Occupations (ME explode), Databases-Bibliographic (ME explode), Information-Storage-and-Retrieval (ME explode), literaturesearch* (ft), database search* (ft), evidencesearch* (ft), evidence retriev* (ft), Evidence-Based-Medicine (ME explode), evidence-basedpractice (ft), Education (ME explode).

•

EMBASE

(1980–2002/7)

using: health-care-personnel (ME explode), student (ME explode),PAM* (ft), bibliographic-database (ME explode),information-retrieval (ME explode), literaturesearch* (ft), evidence search* (ft), evidenceretriev* (ft), database search* (ft), evidence-based-medicine (ME explode), evidence-basedpractice (ft), education (ME explode).

•

CINAHL

(1982–2002/5)

using: Health-Personnel(ME explode), PAM* (ft), Students-Health-Occupations (ME explode), Reference-Databases-Health (ME explode), Computerized-Literature-Searching (ME explode), Professional-Practice-Evidence-Based (ME explode), database search*(ft), Education (ME explode).

•

ASSIA

(1982–2002/7)

using: Personnel or Person-nels (ME), Students (ME), PAM* (ft), databasesearch* (ft), literature search* (ft), Evidence-Based (ME), Teaching (ME), Training (ME).

•

BNI

(1994–2002/5)

using: Paramedical-Professions(ME explode), Medical-Profession (ME), nurse*(ft), student* (ft), Literature-Searching (ME),Evidence-Based-Practice (ME), education (ft),train* (ft), teach* (ft).

•

ERIC

(1985–2002/6)

using: health-personnel(ME explode), PAM* (ft), allied health occupa-tions education (ME explode), nursing educa-tion (ME explode), medical students (MEexplode), Literature search* (ft), informationretrieval (ME explode), information seeking(ME explode), Evidence-Based Practice (ft),training (ME explode), education (MEexplode).

•

LISA

(1969—current)

using: health profession-als (ME), students (ME), allied health profes-sion* (ft), PAM* (ft), information seekingbehaviour (ME), evidence-based practice (ft),evidence-based medicine (ft), training (ME),education (ME).

•

National Research Register

(

2002, Issue 2

) usingterms as for Cochrane Library.

•

Sheffield School of Health and Related Research(UK)

site was also searched for conferenceproceedings as the school has a special interestin this area.

•

Internet

. AltaVista search engine using searchterms: ‘literature search training’ and (healthprofessional* or student*).Citations were also tracked from the available

articles and books.

Methods of the review

Results of the literature search were initiallyscreened by one reviewer (AG).

The following were excluded in the firstinstance: studies that described teaching methodsand curriculum; trainee needs analyses and evalu-ation studies that only evaluated the trainingprovision in terms of trainee satisfaction; pre- andpost-training database usage levels and/or just anincrease in the knowledge about literature searchand retrieval terminology.

Only studies that evaluated the impact of 1–8 hliterature search and retrieval training on healthprofessionals’ literature search skills were consid-ered. If the information regarding the number ofhours of training provided was unavailable or





,

20

, pp.33–41

36

could not be obtained, the study was excluded.Studies were included if a full article could beobtained. No disagreement about the inclusion ofstudies arose.

Three studies met the inclusion criteria. Out ofthese, two, Rosenberg

et al

. 1998

7

and Ericksonand Warner, 1998

8

were randomised controlledtrials (RCTs). These were appraised using thequestions suggested by the Critical AppraisalSkills Programme, Oxford

9

and NHS Centrefor Reviews and Dissemination, York.

10

Thesequestions help to judge the methodological qual-ity of the trials and relate to the randomizationprocess, level of blinding to the intervention,baseline equality of the control and interventiongroups, validity and reliability of the measur-ing instrument used and the reliability of the

results. The third study, Ghali

et al

. 2000

11

wasa controlled-before-and-after (CBA) study. Itsappraisal was based on the questions produced byCrombie.

12

Assessments of methodological quality wereundertaken by one reviewer (AG) and checked bythe second reviewer (KT). All disagreements wereresolved by discussion. The studies (nature, resultsand methodological quality assessments) havebeen summarized in the Table 1.

Quantitative synthesis can only be applied tosystematic reviews where the interventions, partic-ipants, outcomes and study designs are similarenough to suggest the results can be pooled. In thisreview all the studies had different pre- and post-teaching measurement instruments and methods,so quantitative synthesis was not applicable.

Table 1 Summary of studies: nature, results and reviewers’ comments on the methodological quality and results of the studies.

Study referenceType and sizeof the study

Reviewers’comments on themethodological quality of the study

Study results: impact on skills

Reviewers’ comments on the reliability and validity of results

Rosenberg et al. 19987 RCT Satisfactory Effective, P = 0.0001 Only ulcer problem resultsnc = 54 (training effect for are reliable and validni = 54 increase in median search

score in comparison with control). Increase in yield score P = < 0.0001(ulcer problem) and P = 0.5 (cardiac problem)in comparison with control

Erickson and Warner, 19988 RCT Weak Not effective: no Unreliable and invalidnc = 8 difference in search recall results due to majorni = 23 and precision between systems change during the(11 + 12) control and intervention

groups trial and a large dropoutrate from the trial

Ghali et al. 200011 CBA (quasi- Weak Effective, P = 0.002 Unreliable resultsexperimental (significance of betweendesign) group differences innc = 26 change from baseline forni = 34 increase in MEDLINE

search skills) and P = 0.002 for tendency to use computerized searches

nc, number of participants in the control group; ni, number of participants in the intervention group.





,

20

, pp.33–41

37

Description, results and reviewers’ comments on the methodological quality of the included studies

Eighteen potentially useful studies were foundbut only three met the inclusion criteria and onlythese were formally assessed. Brief description,results and methodological appraisal of each ofthe included study is as follows.

Rosenberg

et al

. 1998

7

In this single-blind study, one hundred and eightfirst clinical year students at Oxford UniversityMedical School, UK were randomised to equalsize control and intervention groups. The inter-vention group received three hours inter-active training, in small groups, on

viaWinSpirs (Silver Platter’s Windows

searching software). Their pre- and post-trainingsearches (conducted within three months of thecompletion of training) were assigned a searchscore (maximum 18) based on the technicalaspects of the search quality, like use of free-textsearching, truncation, MeSH searching, Booleanoperators and limits, etc. The quality of evidenceretrieved was assessed and scored (0, worst; 4,best) by clinicians. Students also rated theirsatisfaction with training on a 1 (not very useful)to 6 (extremely useful) scale.

The control group did not receive any trainingand they were assessed once, simultaneouslywith the intervention group’s post-trainingevaluation.

Results.

Median, interquartile ranges and therelated

P

-values for ulcer and cardiac problems areshown in Tables 2 and 3, respectively. Out of the67% of the trainees who responded, 96% found itvery useful.

Methodological quality Randomization.

Standard Random Numberschart was used for randomization and it wasblocked to ensure equal number of participants inthe intervention and control groups.

Level of blinding.

It is a single blind study wheremarkers were blind to the identity of the studentsand whether they were marking pre- or post-training search strategies.

Baseline equality of control and interventiongroups.

Students in both the groups had pre-vious search experience and one student in eachgroup had received formal training in literaturesearch and retrieval before. No other informa-tion on the baseline equality of the groups wasavailable.

Equality of treatment for control and interventiongroups.

The groups were treated in the same way

Ulcer problem Search score (0–18) Yield score (0–4)

Before training intervention group 4 (3–6) 1 (0–2.5)After training intervention group 9.5 (9–11) 4 (3–4)Control group 4 (3–7) 0 (0–2)Training effect in comparison to control 5 (4–6) 3 (1–4)

For training effect on median search score in comparison to control, P = < 0.0001; for training effect on median yield score in comparison to control, P = < 0.0001.

Cardiac problem Search score (0–18) Yield score (0–4)

Before training intervention group 5 (3–6) 4 (4–4)After training intervention group 6 (7–9.5) 4 (4–4)Control group 4 (4–6) 4 (3–4)Training effect in comparison to control 3 (2–3) 0 (0–0)

For training effect on median search score in comparison to control, P = < 0.0001; for training effect on median yield score in comparison to control, P = 0.5.

Table 2 Median, interquartile ranges and the related P-values for ulcer problem.

Table 3 Median, interquartile ranges and the related P-values for cardiac problem.





,

20

, pp.33–41

38

during the training intervention, apart from thetraining provided to the intervention group.Were all the participants who entered the trialproperly accounted for at its conclusion? In theintervention group, only 45 out of the allocated 54participated in the training and out of this only 38were given the post-training test. The reasons forthe nine participants withdrawing from the trainingand seven not being given the post-training test arenot discussed. All pre-test results (45 participants)were included in the statistical analysis.Reliability and validity of the measuring instrumentused. The search score assessment criteria used inthe study has face validity but its reliability islimited by the fact that the search scores achievedby the trainees partially depend on the complexityof the search problem on which they are tested. Ifthe search problem requires the formulation of avery detailed search strategy, i.e. using all thetechniques taught to retrieve relevant articles,student’s search score would be high, but if itdoesn’t the search score would be low. This isobvious from the results of the students’ thatsearched on the ‘Cardiac problem’. Their searchscores showed less improvement after trainingbecause the assigned problem did not require theformulation of a detailed search strategy toretrieve relevant articles in the first instance. Thisproblem transcends to yield score measurement aswell. Contrary to the expectations, the directrelationship between the search score and the yieldscore doesn’t exist, i.e. for a particular searchstrategy the search score given may be low but theyield score could be high, as a simple searchstrategy could sometimes lead to the retrieval ofrelevant articles on which the yield score depends.How precise are the results? Results related to the‘Ulcer problem’ are valid and reliable.

Erickson and Warner, 19988

This single-blind, small, randomised controlledtrial was conducted in Thomas JeffersonUniversity Hospital, Philadelphia, USA. Theparticipants (31) were obstetrics and gynaecology,trainee residents. They were assigned to threegroups using opaque sealed envelopes. The firstintervention group (11 students) received one-hour individual tutorials and performed searches,

the second intervention group (12) attended thesession where all searching was conducted by theinstructor and the control group (8) did not receiveany training. Both the intervention groups weretrained in searching through CD-plus.Two searches were conducted prior to teachingand two after it. They were rated for relevance bythe faculty members who were blinded to thestudy. The seven-point relevance scale used wasdeveloped by Haynes et al. 199013 at McMasterUniversity. The relative search recall (number ofrelevant articles retrieved by one searcher dividedby the number of relevant articles retrieved by thegroup) and precision rates (number of relevantarticles retrieved by one searcher divided by thetotal number of articles retrieved by that searcher)were calculated by the students.

The residents also rated their search satisfactionon a 1 (very dissatisfied) to 5 (very satisfied) scale.Midway through the study the university’s searchsystem was changed to Ovid search interface sothe students were trained and conducted pre-training searches using CD-plus but did the post-training searches through Ovid interface.

Results. No difference in search recall, precisionand satisfaction were found between control andintervention groups.

Methodological qualityRandomization. Students were first stratifiedaccording to their postgraduate year. Then opaquesealed envelopes were distributed by the researchersto randomise the students into a control and twointervention groups. It is not a standard method ofrandomization because there is a possibility ofresearchers introducing bias into the study.Level of blinding. Markers were unaware of theidentity of the students.Baseline equality of control and interventiongroups. Prior to the intervention, the studyparticipants filled in a questionnaire indicatingtheir computer experience and usage andproficiency. The tutorial group students weredisproportionately female (75%) and 33% had at home so the groups were not equal atthe baseline in important ways.Equality of treatment for control and interventiongroups. No conclusions about the equality of

Effectiveness of training in literature search skills, Anupama Garg and Kathleen M. Turtle

© Health Libraries Group 2003 Health Information and Libraries Journal, 20, pp.33–41

39

treatment for control and intervention groupparticipants can be made on the basis of thewritten report of the trial.Were all the participants who entered the trial prop-erly accounted for at its conclusion? Ninety and84% of the participants did the first and secondpre-teaching searches, respectively, while only 71%did the third and 58% did the fourth post-teachingsearch. Only 29% of the participants submittedtheir citation lists for the fourth search. This declin-ing participation has been noted in the study.Reliability and validity of the measuring instrumentused. The study used a pre-validated instrumentdeveloped by Haynes et al. 199013 for determiningthe relevance of searches. They found its intra-rater (weighted kappa = 0.79, 95% CI (0.74–0.84),an excellent level of agreement) and inter-raterreliability (weighted kappa = 0.49, 95% CI (0.41–0.49), a fair level of agreement).How precise are the results? The results of thisstudy are totally unreliable due to the majorsystems change during the trial—from CD plusto Ovid interface, and because of the largeparticipant dropout rate of the trial.

Ghali et al. 200011

This controlled before-and-after trial was conductedat two mobile sites of Boston University MedicalSchool with 60 (ni = 34, nc = 26), third-year medicalstudents. The intervention group received trainingin how to harness as a part of an Evidence-based Medicine (EBM) training programme.They participated in four, 90 min sessions over aperiod of 4 weeks. The instructions on harnessing were only given during the first session,while during the rest of the sessions studentsconducted the searches related to the clinicalquestions given. The control group received nodatabase search and retrieval training.

To determine the efficacy of the intervention,students in control and intervention groups weresurveyed immediately before and after the courseusing a self-reporting questionnaire. One of thequestions related to the use of where stud-ents rated their own skills on a six point ordinal scale:1 (extremely effective) to 6 (extremely ineffective).

Mean change in scores from the baseline wascalculated for both the groups. The P-values werecalculated for the differences between groupsusing Wilcoxan rank sum test.

Results relevant to this review. The results areshown in Table 4.

Methodological qualityLevel of blinding. It is an unblinded study asthe students reported on their own performancethrough a self-reporting questionnaire.Baseline equality of control and interventiongroups. Students were not checked for anyimportant baseline characteristics such as theirlevels of computer experience.Equality of treatment for control and interventiongroups. They were treated equally.Reliability and validity of the measuring instrumentused. The study details do not tell whether theinstrument’s reliability and validity was checked.Were all the participants who entered the trialproperly accounted for at its conclusion? All part-icipants undertook the training and filled in thepre- and post-training questionnaires.How precise are the results? The post-coursereduction in control group’s skill levelcould purely be attributed to psychologicalreasons. It could be argued that the control groupstudents felt that they have become worse or areworse than before because the other group hadreceived training while they did not. Similarargument could be made for the post-training

Table 4 Results relevant to this review.

Intervention group change in score from the baseline

Control group change in score from the baseline P

Skill in using +0.7 −0.3 P = 0.002MEDLINE



40

increase in -skill level reported by theintervention group. This has negative implicationsfor the validity of the results reported for theincrease in skill level by the researchers.

Results of the review

The combined results of primary studies (Table 1)conducted in teaching hospitals, mainly with studentdoctors, show that there is some improvement intheir searching skills and they find the traininguseful. These results are similar to the conclusionsreached by a rapid review on the informationskills training presented in the Evidence-basedLibrarianship Conference in Sheffield, 2001.14

Discussion

Although the studies show a positive impact oftraining on the searching skills of the trainees, noconsensus on the quantification of the level ofskills improvement can be reached due to the smallnumber of studies and the poor quality of most ofthe existing studies.

There is also no set definition of the optimal levelof skill in literature searching which must be reachedto be efficient in practice. The generalisability andconclusions that can be drawn from the small num-ber of trials (of mixed quality) found are limited.

One explanation for the shortage of research inthis area could be that the educational researchneither attracts high level of funding nor has theprestige associated with the research effort inmedical sciences.15

None of the studies assesses what the optimumperiod of training might be.

The scope of this review was limited only todetermining the impact of training on healthprofessionals’ literature search and retrieval skillsfrom electronic health databases due to the dif-ficulties of measuring the effect of such interven-tions on their clinical practice.6,16 No effort wasmade to measure the effect of such interventionson clinical practice.

It is also based on the analysis of the publishedstudies only. One unpublished study17 was foundand efforts were made to procure it without suc-cess. This has negative implications for the overallprecision of the review’s findings.18 At this stage,

we accept that the review’s results may not beprecise but it still gives us a clear indication of theneed for good-quality further research in this area.

The scope of the review could be widened byincluding studies whose evaluation was limited tomeasuring more basic learning outcomes, such asincrease in knowledge about the informationsources, literature search and retrieval process andits associated terminology, and increase in theusage of electronic databases.19,20 We believe theselearning outcomes are the basis for developing agood level of skill in literature search and retrieval.The reviewers believe that a good level of skill inthis area could be defined as the ability to executesearches and retrieve information independently.

Reviewers’ conclusions

Implications for practice

Evidence from research, mainly conducted in theteaching hospitals and medical schools withmedical students, shows some effectiveness inimproving health professionals’ searching skills.There is no clear evidence about the level of effect,whether the effect lasts and what the effects ofteaching are on fully qualified doctors, nurses andon the members of allied medical professions.

Because there is a lack of good-quality researchevidence, the development and implementation ofany new literature-search training programmeshould be considered on an experimental basisonly with in-built vigorous evaluation strategies toprove local effectiveness.

Implications for research

There is an urgent need for good-quality researchthat is methodologically rigorous, especially thoseusing controlled before-and-after studies, whichmeasure impact of training on skills throughobjective, valid and reliable instruments, conductedin non-teaching hospitals and in primary caresettings with a variety of health professionals.

Acknowledgements

The reviewers would like to thank Ann Green andAnn Chadwick, Assistant Librarians, at Chorley



41

and South Ribble District General Hospitallibrary for their prompt and efficient service inproviding the requested articles.

References

1 Department of Health. The new NHS. Modern. Dependable. London: The Stationery Office Limited, 1997.

2 Department of Health. A First Class Service. Quality in the New NHS. London: Department of Health, 1998.

3 Department of Health. Information for Health. An Information Strategy for the Modern NHS 1998–2005. London: Department of Health, 1998.

4 Department of Health. Research and Development. Towards an Evidence-Base for Health Services. Public Health and Social Care. Information Pack. London: Department of Health, 1998.

5 Pyne, T., Newman, K., Leigh, S., Cowling, A. & Rounce, K. Meeting the information needs of clinicians for the practice of evidence-based healthcare. Health Libraries Review 1999, 16, 3–14.

6 Urquhart, C., Massiter, C., Thomas, R., Sharp, S. & Smith, J. Getting information to vocational trainees: report of the GIVTS project. Library and Information Commission Research Report 26. London: Library and Information Commission, 1999.

7 Rosenberg, W. M. C., Deeks, J., Lusher, A., Snowball, R., Dooley, G. & Sackett, D. Searching skills and evidence retrieval. Journal of the Royal College of Physicians of London 1998, 32(6), 557–63.

8 Erickson, S. & Warner, E. R. The impact of an individual tutorial session on use among obstetrics and gynaecology residents in a academic training programme: a randomised trial. Medical Education 1998, 32, 269–73.

9 Learning and Development Division, Public Health Resource Unit. Headington, Oxford: Institute of Health Sciences. Available at: http://www.phru.org.uk/∼ casp/resources/rct∼ tool.htm.

10 NHS Centre for Reviews and Dissemination. Undertaking

systematic reviews of research on effectiveness. CRD Guidelines for Those Carrying Out or Commissioning Reviews. CRD Report 4. York: NHS Centre for Reviews and Dissemination, University of York, 1996: 75.

11 Ghali, W. A., Saitz, R., Eskew, A. H., Gupta, M., Quan, H. & Hershman, W. Y. Successful teaching in evidence-based medicine. Medical Education 2000, 34, 18–22.

12 Combie, I. K. The Pocket Guide to Critical Appraisal. London: BMJ Publishing, 1996: 49.

13 Haynes, R. B., McKibbon, K. A., Walker, C. J., Ryan, N., Fitzgerald, D. & Ramsden, M. F. Online access to in clinical settings. A study of use and usefulness. Annals of Internal Medicine 1990, 112, 78–84.

14 Brettle, A. Information skills training: a rapid review of the literature. Evidence Based Librarianship Conference, 3–4 September 2001, Sheffield. UK.

15 Hutchinson, L. Evaluating and researching the effectiveness of educational interventions. British Medical Journal 1999, 318(7193), 1267–9.

16 Booth, A. & Falzon, L. Evaluating information service innovations in the health service: ‘If I was planning on going there I wouldn’t start from here’. Health Informatics Journal 2001, 7, 13–9.

17 Clancy, M. Evaluation and appraisal of teaching evidence-based practice to PAMs. A completed (1998) but unpublished study. Publication ID: N0231084751. Source: The National Research Register at http://www.update-software.com/nrr/CLIBINET.EXE?A=1&U=1001&P=10001.

18 NHS Centre for Reviews and Dissemination. Literature searching and study retrieval. In: Undertaking Systematic Reviews of Research on Effectiveness. CRD guidelines for those carrying out or commissioning reviews. CRD Report 4. York: NHS Centre for Reviews and Dissemination, 1996, 19–25.

19 Griffin, N. L. & Schumm, R. W. Instructing occupational therapy students in information retrieval. The American Journal of Occupational Therapy 1992, 46(2), 158–61.

20 Wallace, M. C., Shorten, A. & Crookes, P. A. Teaching information literacy skills: an evaluation. Nurse Education Today 2000, 20, 485–9.

http://www.phru.org.uk/~casp/

http://www.update-software.com/nrr/

Documents

Effectiveness of training health professionals in literature search skills using electronic health databases—a critical appraisal