Elizabeth A. Lindsay MS, PhD

Program Planning Committees: Tools for Building Capacity

4th National Accreditation Conference

Toronto, Ontario

September 21, 2012

Disclosure

I have no actual or potential conflict of interest in relation to this workshop.

Learning Objectives

After active engagement in this workshop, participants will be better able to:

• Identify recommended competencies for planning committee members

• Discuss planning committee roles and responsibilities with a focus on building capacity and meeting standards

• Identify and develop practice planning committee solutions for use in faculty development

Agenda

A. Background for the workshop – 20 min • Why are planning committees important? • Is there currently a problem with planning processes? • What do committee members need to be able to do – what is the scope of their

roles? • Overview of the “Gatekeeper Study” results B. Small group discussion – 20 min Objectives - to discuss specific competencies and tools directed at selected aspects

of the planning process for large or small group learning C. Full group discussion - 35 min • Presentation of examples of tools to support planning processes • Examples of tools recommended in small group discussion D. Evaluation/Feedback – 5 min.

What does a high functioning program planning committee look like?

Is our present system working well?

What do they presently do?

Usual Practice for Planning One-day Programs

• 2-3 face-to- face meetings before event

• E-mail communication to finalize faculty, titles & objectives

• Occasionally, 1 debriefing meeting

Impact of Group Learning on Clinical Practices

• Multiple reviews show us the challenges of demonstrating impact of educational programs on competency and performance and there are pressures to improve the impact of our programs as well as to expand the type of activities we offer.

• However, group learning continues to be the predominant CME/CPD

activity for most physicians, as reported by the MainCert and MainPro programs in Canada

• Recent study of family physicians in the Ottawa region demonstrated that they are quite satisfied with the CME/CPD options available to them and believe that group learning does impact their clinical practices….little pressure to change.

• Expectations for the planning processes for accredited group learning and on-line learning are clearly defined – how well are we doing them? Those who evaluate applications for accreditation can advise us.

Seems appropriate that we try to improve impact

How?

Identify ingredients that can increase impact

Transfer this knowledge and skill to those who build or lead the programs

Step 1 Clarify target

audience and carry out

needs assessment process

Step 2 Identify subject area for

content and write learning

objectives

Step 3 Outline content

area to reflect learning

objectives and choose

format/methods/faculty

Step 4 Invite faculty and share

learning objectives,

recommendations for learning

formats

Step 5 Execute sessions

according to plan and evaluate

sessions, report to planning

committee

Step 6 Feedback and

other learning from past

meetings Synthesize

results

Steps for

Building

an

Accreditable

Program

What do we know about impact of group learning?

Group learning ( lectures, workshops, small groups, rounds) Effect size average - 6-10% * Key factors that determine impact - amount and type of interaction;

– less complex behaviours and more serious outcomes* Previously noted factors that enhance impact: needs based; relevance to practice reflection before, during and after meetings commitment to change at educational events combined with other interventions sequential repetition

*Forsetlund et al 2009

What do planning committees need to be able to do? From the AAMC/SACME Harrison survey*….

Toward a more effective product: Enhancing didactic activities • Practice enablers (patient material, flow-sheets, algorithms etc at or after course) • Follow-up method post-course to reinforce learning

Examples used in the survey • Pre-course planning ( needs assessment, objectives, choose methods, evaluate) • In-course enhancement

– Meaningful interactivity – Simulations – Tools for patient management

• Follow-up methods post-course to reinforce learning • Sequential learning

* Association of American Medical Colleges. Academic CME in the US and Canada: The 2010 AAMC/SACME Harrison survey p. 13

Basic Assumptions for the Gatekeeper Study

Investigators: Elizabeth Lindsay, Paul Hendry, Michael Allen, Heather Armson , Jatinder Takhar

• Health care providers who serve on planning committees and as presenters are the “gatekeepers” to improving effectiveness of group learning programs

• There is a relationship between program design, implementation processes and potential impact of group learning programs on participants

We Applied Behavior Change Frameworks* to Guide our Study Questions

Are physicians who serve on planning committees

Aware of best practices in program design?

Willing to apply new planning processes?

Do physicians who serve on planning committees believe that selected planning processes are:

Important?

Feasible?

Will make affect impact of the program?

*PRECEDE (Green) and Theory of Planned Behavior (Azjen)

Study Participants

• 52 physicians who had participated in planning committees at the University of Ottawa over an 2.5 yr period.

Respondents

• Family physicians - 17

• Other specialists - 15

Total 32/62%

Competency areas selected for study**

• Assessed information that describes the learning needs of potential participants in the program eg. gaps between current and recommended clinical practices

• Created learning objectives for the program that state what the learner will be able to do as a result of participating in the specific sessions.

• Recommended types of learning formats that work best for different types of objectives eg for knowledge based or skills based outcomes

• Discuss objectives and recommended learning formats with faculty/teachers to ensure mutual understanding

• Encourage faculty/teachers to present the level of evidence supporting educational messages (eg. Meta-analysis, RCT, expert opinion)

• Identify strategies that will facilitate practice integration of recommended best practices or address barriers to implementation

• Audit programs to assess existence of bias or compliance with accreditation standards

**Selected from Alliance for CME list of competencies for CME/CPD professionals

Study participants

Frequency

Scale 1-5

Willingness

Scale 1-5

Importance

Scale 1-5

Feasibility

Scale 1-5

Impact

Scale 1-5

Mean Responses

4 or 5

Mean Responses

4 or 5

Mean Responses

4 or 5

Mean Responses

4 or 5

Mean Responses

4 or 5

Assess needs

4.0 78% 4.2 81% 4.9 100% 4.1 77% 4.6 90%

Create learning objectives

4.2 78% 4.4 81% 4.6 100% 4.4 90% 4.2 87%

Recommend

learning formats

4.2 81% 4.3 81% 4.8 99% 4.4 90% 4.5 93%

Discuss obj &form with presenters

3.4 56% 4.1 78% 4.6 97% 4.1 80% 4.3 87%

Encourage careful data presentation

2.9 31% 4.1 75% 4.5 87% 4.1 70% 3.8 58%

Identify integration strategies

3.3 41% 4.1 78% 4.5 90% 3.9 69% 4.1 76%

Audit for bias and standards

2.0 16% 3.5 53% 4.3 90% 3.6 54% 3.4 43%

Conclusions

• High levels of willingness to improve

• Strong belief in the importance of all factors but

• Not as strong regarding belief of potential impact

• Lots of variability regarding feasibility

Purpose and process of small group discussion

Focus on one element of the planning process

Considering our expectations regarding planning committees – what can we do to support them?

Describe and create a list of specific processes and tools you use or think we should use to support planning committees

Topics for small group discussion

Composition of the Planning Committee – The Chair and Preparation

Composition of the Planning Committee – The Committee Members

Competencies and Pre-work

Structure and Process of Planning Meetings – During meetings

Structure and Process of Planning Meetings – Between Meetings

Choose effective presenters and helping them prepare

Choose educational strategies that go beyond didactic presentations with

Q&A

Full Group Discussion

With the package of resources as a basis --- add the ideas raised by each of the small groups

Topics for small group discussion

Composition of the Planning Committee – The Chair and Preparation

Qualities of a high functioning chair

Preparation Ideas – evidence for importance of improving impact

Composition of the Planning Committee – The Committee Members

Competencies and Pre-work Content Experts

Audience Representatives

Education Designer

Event Planner

Topics for small group discussion

Structure and Process of Planning Meetings – During meetings

Agenda should reflect time for different steps in the process in order to set up expectations for the meeting process eg. Time for looking at needs assessment information

Use a planning template that can be completed during discussion at the meeting, columns for Gap and Objective, Title, Presenters – event planner can do some of this with help from the chair or another committee member

Structure and Process of Planning Meetings – Between Meetings

The event planner, education designer and chair can work on planning template between meetings to ensure the process moves forward in a timely manner

Individual members of the committee may have tasks that they have agreed to complete eg. Check on availability of presenter or find a particular resource – ideally meeting planner should follow-up to gather this information and apply it to the planning template.

Topics for small group discussion

Choose effective presenters and helping them prepare

Presenters strengths – ideas for how to find them – Knowledgeable and creditable regarding content

– Demonstrates good understanding of the clinical setting in which learner works and the challenges faced there

– Clarity regarding content and presentation format

– Sufficient interaction to ensure understanding eg. Able to answer questions appropriately

– Create template letters of invitation to potential presenters to ensure draft objectives and title are included and invite feedback

Choose educational strategies that go beyond didactic presentations with

Q&A

Match objective with learning format – see Michie

What makes a workshop, a workshop?

Improving interaction – Tip sheets

Resources

• Strauss S, Tetroe, J, Graham ID. Knowledge Translation in Health Care: Moving from evidence to practice. 2009. Wiley-Blackwell. Oxford.

• Green LW, & Kreuter MW. Health Program Planning: An educational and ecological.

4th edition 2005. McGraw Hill. Toronto.

• Ajzen I. The theory of planned behavior. Organizational Behavior and Human Decision Processes, 1991;50:179-211

• Michie S, Johnson M, Francis J, Hardeman W, Eccles M. From theory to intervention: Mapping theoretically derived behavioural determinants to behaviour change techniques. Applied Psychology:An International Review, 2008, 57(4):660-680.

• Michie S, Johnson M, Abrahm C, Lawton R. Parker D, Walker A, on behalf of the “Psychological Theory” Group. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care 2005;14:26-33.

• Moore, DE. How physicians learn and how to design learning experiences for them: An approach based on an interpretive review of evidence. In:Hager, M, Russell S & Fletcher SW (Eds.). Continuing education in the health professions: Proceedings of a conference. A Report for the Josiah Macy Foundation, 2008; http://www.josiahmacyfoundation.org/index.

Contents of Handout

1. Title page - University of Ottawa Office of Continuing Medical Education (OCME)

Handbook for Planning Committees A How‐to‐Guide for the development of an education program that may be accredited for Family Physicians (MainPro 1) and Specialists (MOC)

2. Examples of slides for discussion of KT, evidence of impact of group learning.

3. Excel planning template for listing of topics, gaps and objectives

4. Tip Sheets – Example – Interactivity

5. Invitation template to participate on a planning committee, including roles, responsibilities, time commitment, timelines and expectations regarding response to communication.

6. K to A framework

7. Michie matching of objectives and methods