1/12

REVIEW PROTOCOL TEMPLATE

Recommendation 1

The effectiveness of health professionals’ education and training institutions designing and implementing continuous development

programmes for faculty and teaching staff relevant to the evolving health-care needs of their communities.

A systematic review

Review protocol

Organization, City, Country: University of Cambridge, Cambridge, United Kingdom Prepared by: Date:

Alison Andrew July 2011

Senior supervisor Francesca Celletti Project lead Rebecca Bailey Research team members Rachel Marie Amiya, Dolea Carmen Mihaela, Annette

Mwansa Nkowane, Erica Lynette Wheeler, Rodrigo Rodriguez Fernandez.

2/12

Table of Contents

................................................................................................................................................................ 1

TO INVESTIGATE THE EFFECTIVENESS OF CONTINUOUS DEVELOPMENT PROGRAMMES FOR FACULTY AND TEACHING STAFF, INCLUDING CURRENT HEALTH WORKERS, WHICH UPDATE AND DEVELOP TEACHING SKILLS (E.G. CURRICULUM DEVELOPMENT AND INSTRUCTIONAL DESIGN), IN BOTH UNDERGRADUATE AND POSTGRADUATE PROGRAMS, ON THE QUANTITY, QUALITY AND RELEVANCE OF HEALTH PROFESSIONALS.......................................................................................................................... 1

A SYSTEMATIC REVIEW ............................................................................................................................ 1

1.0Background .................................................................................................................4

2.0Objective .....................................................................................................................4

3.0Review Question .........................................................................................................4

4.0Evidence gathering and study selection ......................................................................5

5.0Assessment of risk of bias and data extraction ...........................................................7

6.0Data synthesis ............................................................................................................8

7.0Dissemination .............................................................................................................8

8.0Resource implications .................................................................................................8

9.0References ...................................................................................................................8

Appendix A: database search strategy ...........................................................................10

Appendix B: data extraction forms (see attached excel workbook) ..............................12

Appendix C: Example of databases, websites and journals which could be searched ..12

3/12

1.0 Background

Include definitions, answers to background questions, description of the intervention and how it might work to help better understand the question.

This question refers to the effectiveness of continuous development programmes for faculty and teaching staff which are focused on which updating and developing teaching skills. Such interventions could be in the form of one off workshops, seminars or short courses or continuing programmes such as fellowship schemes; all aimed at improving teaching ability. There have been systematic literature reviews that have analysed the effectiveness of such interventions (most notably Steinert et al. 2006 reviewed the effectiveness of such interventions in medical education) yet the primary issue lies in the outcomes measured. Few studies have measured the effect of such programmes past the level of participants’ reaction to the programme. Studies that measured further outcomes (learning in the form of a change in attitudes, knowledge or skills, a change in behavior or results in the form of a change in the system/ organizational practice or a changes in learning or performance of health students) were those included in Steinert's systematic review. However, since the outcomes of interest in this systematic review are the quantity, quality and relevance of health graduates and health practitioners the only outcomes relevant to this literature review are those measured are changes in the quantity, quality and relevance of health graduates or professionals that can be specifically attributed to such continuous development programmes for faculty and teaching staff. Other process outcomes will be recorded but are not relevant for the purpose of evaluation of evidence.

2.0 Objective

To investigate the effectiveness of Continuous Development Programmes for faculty and teaching staff, including current health workers, which update and develop teaching skills (e.g. curriculum development and instructional design), in both undergraduate and postgraduate programs, on the quantity, quality and relevance of health professionals.

3.0 Review Question

For the purposes of this literature review, the population, intervention, comparators and outcomes (PICO) framework to inform the review objectives are presented below.

Population Intervention Comparison Outcome

Faculty and teaching staff programs

Continuous Development Programmes for faculty and teaching staff which update and develop teaching skills

No Continuous Development Programmes for faculty and teaching staff which update and develop teaching skills

Quantity, quality and relevance of health professionals

4/12

4.0 Evidence gathering and study selection

See Appendix C for examples of databases, websites and journals which may be searched.

4.1 Evidence gathering

The evidence gathering approach will have four components:

4.1.1 Searching databases

The databases in the table below will be searched with a pre-determined strategy as detailed in Appendix A. In cases where the search results are small in number, search terms will be reduced to maximize the search sensitivity.

Topic/Field Database

Ex: Education Ex: ERIC, Campbell, Education Research Complete…

Health PUBMED

4.1.2 Hand searching

The following journals and websites will be hand-searched for relevant articles:

Resources that will be searched by hand

• Teaching and Teacher Education • Academic Medicine • Medical Teacher • International Journal for Academic Development • Biomed (http://www.biomedcentral.com/)

WHO's Library Database (WHOLIS)

Website of the Foundation of Advancement of International Medical Education and Research (FAIMER)

International Council of Nurses

Council on Graduate Medical Education

American Academy of Family Physicians

American College of Physicians

5/12

4.1.3 Expert network consultations

A network of health professional education experts will be consulted by email to identify additional grey literature or research that has not been found through the above processes. The following experts and networks will be contacted.

Experts and networks will be contacted regarding additional articles

Consultation with relevant members of the Core Guidelines Development Group

4.1.4 Reference searches

Bibliographies of those papers that match the eligibility criteria below will be searched by hand to identify any further, relevant references, which will be subject to the same screening and selection process.

4.2 Eligibility criteria

After gathering the evidence, the following eligibility criteria will be applied to the results and all identified references screened independently by two reviewers (Aaron Stoertz and Francesca Celletti) using a three-stage approach to reviewing the title, abstract and full text.

4.2.1 Types of studies:

All types of evaluative study designs are eligible for inclusion, including grey literature. Studies will not be selected on methodological quality.

4.2.2 Types of participants: This is a succinct description of the types of participants that you will include. For example, you might copy the health professionals definition here.

The participants this literature review will include are faculty and teaching staff including current health workers and/or residents or interns who also teach students from any of the health professions listed below. To facilitate the development of the WHO guidelines for transforming and scaling-up health professional education and training, the literature review will include the following lists of health professionals, as defined by the International Standard Classification of Occupations (ISCO) (WHO, 2010a; ILO, 2008): Core List Medical Doctors - both Generalist and Specialist Practitioners, including Public

Health Doctors (ISCO-08 minor group 221) Nursing Professionals, including Public Health Nurses (ISCO-08 minor group 222, unit

2221)

6/12

Midwifery Professionals, including Public Health Midwives (ISCO-08 minor group 222, unit 2222)

Dentists (ISCO-08 minor group 226, unit 2261) Pharmacists (ISCO-08 minor group 226, unit 2262)

4.2.3 Types of intervention: This is a succinct description of the types of intervention you will include.

Continuous Development Programmes for faculty and teaching staff which update and develop teaching skills. There is a wide variety of programmes that qualify as such e.g. workshops, seminars, short courses, fellowship schemes.

Articles that focus on programmes that develop and enhance teaching skills and educational methods (including PBL) not programmes that develop teaching content.

4.2.4 Types of outcome measures:

The primary outcomes of interest are the quantity, quality and relevance of practicing health professionals. These are defined by a number of measurable outcomes found in the Outcomes Framework document. Other important outcomes include values and preferences, resource use/costs, benefits and harms, as well as all other unintended effects of the intervention. Studies that include other outcomes should not be excluded at this stage in the evidence retrieval. Other secondary outcomes can also be defined, as needed.

4.3 Exclusion criteria

Editorials, newspaper articles and other forms of popular media will be excluded. Failure to meet any one of the above eligibility criteria (section 3.2) will result in exclusion from the review and any apparent discrepancies during the selection process will be resolved by a third, independent reviewer. The number of excluded studies (including reasons for exclusion for those excluded following review of the full text) will be recorded at each stage.

5.0 Assessment of risk of bias and data extraction

Following the initial selection of literature, the reviewers will consult with the WHO Secretariat regarding the process to be followed for summarizing the studies. Data will be extracted from relevant papers using predefined evidence summary templates attached in Appendix B. Data will be collected regarding the reasons for exclusion,

7/12

characteristics of included studies, participants, interventions (including comparators) and outcomes. The final decision for inclusion or exclusion will be made by a team consisting of the WHO Secretariat, methodologist and researchers conducting the review. Any potential disagreement will be recorded and resolved by further discussion.

Risk of bias across studies will be assessed using the approach outlined by the Grading of Recommendations Assessment Development and Evaluation (GRADE) working group. Any disagreements will be recorded and resolved by involvement of an additional reviewer.

6.0 Data synthesis

The availability of appropriate data and resources to conduct a meta-analysis will be considered, where feasible.

7.0 Dissemination

A final set of tables including a GRADE Evidence Table and Descriptive Evidence Table will be produced and submitted to the WHO Secretariat as stipulated in the Procedures for the Retrieval of Evidence and Summary of Evidence. In addition, a manuscript will be prepared for submission to a peer-reviewed journal (a more specific dissemination proposal can

go here as well).

8.0 Resource implications

The project lead will work closely with the WHO Secretariat to define the scope and methods of the review and facilitate access to unpublished literature, supporting translation of foreign language literature where necessary. Proposed milestones and timescales are outlined below:

MILESTONES

Identification and retrieval of the evidence

Conduct literature review, hand searching, and contacts with experts and networks

Summarizing evidence in Descriptive

Evidence Tables (Table 1. Summary of systematic literature reviews, Table 2. Summary of relevant literature)

Development of GRADE Evidence Tables in collaboration with WHO, Geneva, and the GRADE network methodologist

8/12

9.0 References

When citing articles, please use the WHO standard citation format, called the Harvard system. The Harvard citation system shows the author and date in the body of the text. This may be done in one of two ways: Ballance, Ewart & Fitzsimmons (2001) have reported ... It has been reported (Ballance, Ewart & Fitzsimmons, 2001) that ... Some examples of formatting reference lists: Article in a Journal Burt BA, Pai S (2001). Sugar consumption and caries risk: a systematic review. Journal of Dental Education, 65:1017–1023.

Chapter in a book Melton L J III (1995). Epidemiology of fractures. In: Riggs BL, Melton L J III, eds. Osteoporosis: etiology, diagnosis, and management, 2nd ed. Philadelphia, PA, Lippincott-Raven, 225–247.

Corporate author Heart Protection Study Collaborative Group (2002). MRC/BHF Heart Protection Study of antioxidant vitamin supplementation in 20 536 high-risk individuals: a randomised placebo-controlled trial. Lancet, 360:23–33.

WHO Publication with no named author World Health Organization (2003). The international pharmacopoeia, 3rd ed. Vol. 5. Tests and general requirements for dosage forms; quality specifications for pharmaceutical substances and tablets. Geneva, World Health Organization.

More on the WHO-style can be found here: http://www.nlm.nih.gov/bsd/uniform_requirements.html

9/12

Appendix A: database search strategy

PUBMED Search strategy

Population (Faculty[Mesh] OR Preceptorship[Mesh] OR Mentors[Mesh] OR Faculty[tw] OR Faculties[tw] OR "Academic physician"[tw] OR "Academic Physicians"[tw] OR teachers[tw] OR teacher[tw] OR mentors[tw] OR mentor[tw] OR preceptor[tw] OR preceptors[tw] OR tutor[tw] OR tutors[tw] OR educator[tw] OR educators[tw] OR "teaching staff"[tw] OR lecturer[tw] OR lecturers[tw] OR instructor[tw] OR instructors[tw] OR trainer[tw] OR trainers[tw] Intervention "Staff development"[Mesh] OR "Faculty development"[tw] OR "Faculty-development"[tw] OR "Staff-development"[tw] OR "Staff development"[tw] OR "Faculty training"[tw] OR "Faculty-training"[tw] OR "teacher training"[tw] OR "teacher-training"[tw] OR "Continuing development"[tw] OR "Continuous development"[tw] OR "Professional development"[tw] OR "professional continuing education"[tw] OR "Professional development"[tw] OR "Human resources development"[tw] OR "Professional training"[tw] OR "teacher improvement"[tw] OR "teaching improvement"[tw] OR "teaching skills improvement"[tw] OR "teaching-improvement"[tw] OR "teacher development"[tw] OR "teaching development"[tw] OR "teaching to teach"[tw] OR "teach the teacher"[tw]

Search Most Recent Queries Time Result

#3 Search (#1) AND #2 Limits: English, Publication Date from 1990 to 2011 08:38:26 3505

#2 Search "Staff development"[Mesh] OR "Faculty development"[tw] OR

"Faculty-development"[tw] OR "Staff-development"[tw] OR "Staff

development"[tw] OR "Faculty training"[tw] OR "Faculty-training"[tw]

OR "teacher training"[tw] OR "teacher-training"[tw] OR "Continuing

development"[tw] OR "Continuous development"[tw] OR

"Professional development"[tw] OR "professional continuing

education"[tw] OR "Professional development"[tw] OR "Human

resources development"[tw] OR "Professional training"[tw] OR

08:38:07 11026

10/12

"teacher improvement"[tw] OR "teaching improvement"[tw] OR

"teaching skills improvement"[tw] OR "teaching-improvement"[tw]

OR "teacher development"[tw] OR "teaching development"[tw] OR

"teaching to teach"[tw] OR "teach the teacher"[tw] Limits: English,

Publication Date from 1990 to 2011

#1 Search Faculty[Mesh] OR Preceptorship[Mesh] OR Mentors[Mesh] OR

Faculty[tw] OR Faculties[tw] OR "Academic physician"[tw] OR

"Academic Physicians"[tw] OR teachers[tw] OR teacher[tw] OR

mentors[tw] OR mentor[tw] OR preceptor[tw] OR preceptors[tw] OR

tutor[tw] OR tutors[tw] OR educator[tw] OR educators[tw] OR

"teaching staff"[tw] OR lecturer[tw] OR lecturers[tw] OR

instructor[tw] OR instructors[tw] OR trainer[tw] OR trainers[tw] Limits:

English, Publication Date from 1990 to 2011

08:37:55 62612

ERIC Search Strategy

((((Thesaurus Descriptors:"Faculty" or Thesaurus Descriptors:"Teachers" or Thesaurus

Descriptors:"Practicum Supervision") and (Thesaurus Descriptors:"medical

education" or Thesaurus Descriptors:"medical schools" or Thesaurus

Descriptors:"medicine" or Thesaurus Descriptors:"dentistry"or Thesaurus

Descriptors:"dental schools" or Thesaurus Descriptors:"nursing" or Thesaurus

Descriptors:"nursing education" or Thesaurus Descriptors:"pharmacy" or Thesaurus

Descriptors:"pharmaceutical education")) or (Thesaurus Descriptors:"medical school

faculty" or Thesaurus Descriptors:"dental school faculty")) and (Thesaurus

Descriptors:"Faculty Development" or Thesaurus Descriptors:"faculty

improvement" or Thesaurus Descriptors:"Teacher Improvement" or Thesaurus

Descriptors:"Staff Development" or Thesaurus Descriptors:"Inservice Teacher

Education" or Thesaurus Descriptors:"Teacher workshops" or Thesaurus

Descriptors:"Teacher Education Programs"))Publication Date:1990-2011

Appendix B: data extraction forms (see attached excel workbook)

1) GRADE Evidence Table Template.xls 2) Descriptive Evidence Table.xls

11/12

Appendix C: Example of databases, websites and journals which could be searched

Databases

Databases for biomedicine and health sciences:

Pubmed/MEDLINE

CINAHL

Cochrane Library (CDSR)

Health Systems Evidence (PPD/CCNC)

UK Pubmed Central

Global Health Library (Global Index Medicus)

UK Pubmed Central Databases for social sciences:

ISI Web of knowledge

Campbell Library

Social Science Research Network

Databases for education:

ERIC

BEME database

Regional databases for biomedicine and health sciences:

African Index Medicus (www.indexmedicus.afro.who.int)

Australasian Medical Index (www.nla.gov.au/ami)

Index Medicus for Eastern Mediterranean Region (www.emro.who.int/his/vhsl)

Chinese Biomedical Literature Database (CBM) (www.imicams.ac.cn/cbm/index.asp)

IndMED (www.indmed.nic.in)

KoreaMed (www.koreamed.org/searchbasic.php)

LILACS (www.bases.bireme.br/cgi-bin/wxislind.exe/iah/online/?IsisScript=iah/iah.xis&base=LILACS&lang=i)

Index Medicus for the South-East Asia Region (IMSEAR)

(www.library.searo.who.int/modules.php?op=modload&name=websis&file=imse

ar)

Panteleimon for Russia and Ukraine (www.panteleimon.org/maine.php3)

Western Pacific Region Index Medicus (WPRIM) (wprim.wpro.who.int/SearchBasic.php) PASCAL (www.international.inist.fr/article21.html)

General databases:

Google Scholar

12/12

4.1.2 Hand searching

The following journals and websites will be hand-searched for relevant articles:

Resources that will be searched by hand, for peer-reviewed articles and grey literature

Assessment & Evaluation in Higher Education

American Journal of Health Studies

Teaching and Teacher Education

Academic Medicine

Journal of Interprofessional Care

Academic Medicine

Medical Teacher

International Journal for Academic Development

WHO's Library Database (WHOLIS)

SAMMS

HRH Global Resource Center

World Federation of Medical Education

International Council of Nurses

Association of American Medical Colleges

Council on Graduate Medical Education

American Academy of Family Physicians

American College of Physicians

World Bank

HMIC database

Grey Literature Network Service http://www.greynet.org

OPENSIGLE System for Information on Grey Literature in Europe http://opensigle.inist.fr/

NewYork Academy of Medicine Grey Literature site http://www.nyam.org/library/online-resources/grey-literature-report/

FADE Library http://www.fade.nhs.uk/

HMIC: Health Management Information Consortium http://library.nhs.uk/help/resource/hmic

Biomed http://www.biomedcentral.com/

INTUTE http://www.intute.ac.uk/