AMBULATORY PEDIATRICS Volume 4, Number 1 Supplement117Copyright q 2004 by Ambulatory Pediatric Association January–February 2004

Effect of a Teaching Skills Program on Faculty Skills and Confidence

Michelle S. Barratt, MD, MPH; Virginia A. Moyer, MD, MPH

Background.—Although education is a central mission of medical schools, effectiveness of faculty as teachers isvariable, with many faculty lacking formal training. Therefore, a series of sessions around teaching skills was developed.

Objective.—To improve the skill, confidence, and comfort levels of faculty who teach general pediatrics.Method.—A faculty development program consisting of nine 1-hour sessions on a variety of topics and a 3-hour

workshop on Feedback and Evaluation was provided to all faculty members in a single academic division. The programwas evaluated with a needs assessment and a quantitative postprogram self-assessment. A qualitative e-mail survey wasperformed 3 years after completion to assess durability.

Results.—The 13 full-time faculty members (10 women) in the Community and General Pediatrics Division attendeda mean of 6.5 sessions each (range 4–10). All 13 participants completed the 22-item postprogram survey regardingcomfort with and knowledge of teaching techniques. Statistically significant change was seen for all items. Commentsfrom the open-response section reflected specific improvement in the area of feedback and the need for increased timeto focus on teaching. Responses to the follow-up survey (46% response rate) were positive about the program in generaland about the continued use of the skills that were learned in the program.

Conclusions.—Busy general pediatric faculty attended a majority of sessions in a faculty development program onteaching. Skill and comfort levels significantly improved in many of the areas covered, and the skills were still beingused 3 years later.

KEY WORDS: ambulatory pediatrics; faculty development; medical education

Ambulatory Pediatrics 2004;4:117 120

For most medical schools, including the Universityof Texas-Houston Medical School, education is acentral mission of the institution.1 In response to a

1998 Liaison Committee on Medical Education (LCME)site visit report, the University of Texas medical schooldean established a task force to improve teaching by res-idents, which was expanded to faculty to help maintainconsistency in the experiences of learners. The resulting‘‘Teaching to Teach’’ program was presented to incomingchief residents for the core third-year medical student ro-tations. In pediatrics, the faculty of the community andgeneral pediatrics division are responsible for the bulk ofteaching in pediatrics for third-year medical students aswell as supervision of in-patient and out-patient generalpediatric patient care by students and residents. Few ofthe faculty in the division in 1998 had formal training inteaching skills: only 1 had completed a fellowship in am-bulatory pediatrics, 5 had served as chief residents, andothers had limited exposure to educational training ses-sions at professional meetings. The division director sup-ported the formal training program by allocating divisionmeeting time, mandating attendance, and supporting timefor the presenter (M.S.B.) to prepare. We describe the fac-ulty development program created to improve faculty

From the University of Texas-Houston, Health Science Center,Houston, Tex.

Address correspondence to Michelle S. Barratt, MD, MPH, Uni-versity of Texas-Houston, Health Science Center, 6431 Fannin, MSB3.150, Houston, TX 77030 (e-mail: michelle.s.barratt@uth.tmc.edu).

Received for publication April 8, 2003; accepted June 30, 2003.

skills, confidence, and comfort levels with various teach-ing skills, and its evaluation.

METHODS

Needs Assessment

A 2-page survey to assess needs in the area of teachingto teach was distributed to division members. This as-sessment included 21 statements regarding comfort andfamiliarity with common teaching areas; responses wereon a 5-item Likert scale from strongly agree to stronglydisagree and an additional option of no opinion. An ad-ditional 8 questions addressed current teaching assign-ments and focused on 10 possible teaching approaches(eg, mini-lectures, work rounds, formal lectures, obser-vation, demonstration) currently used by respondents.Nine of 13 (69%) eligible faculty completed the needsassessment. Self-reported areas of strength (strongly agreeor agree somewhat selected by all respondents) were‘‘giving positive feedback,’’ ‘‘enjoy teaching residents,’’‘‘teaching helps me be a better clinician,’’ and ‘‘teachingrounds promote clinical problem solving.’’ Areas of need(disagree or neutral selected by $2/3 of respondents) in-cluded ‘‘giving critical feedback,’’ ‘‘facilitating the learnerto give me feedback,’’ ‘‘capable of assessing learningneeds of residents and students,’’ ‘‘setting objectives forlearners before starting rounds,’’ and ‘‘awareness thatlearners have different learning styles.’’ Most respondentsreported teaching on work rounds and mini-lectures astheir primary teaching modes, with a minority using for-mal lectures, clinical skill demonstration, or direct obser-vation of trainees.

AMBULATORY PEDIATRICS118 Barratt & Moyer

Intervention

We set out to plan a curriculum that would improveteaching skills for the members of the division and wouldmirror the skills recommended by the dean for inclusionin the program for residents. The program consisted ofeight 1-hour interactive sessions provided at monthly in-tervals in 1998 and 1999 and a 3-hour workshop in 1999.Topics mirroring those of the dean’s task force were bed-side teaching,2,3 teaching during rounds,4,5 and feedbackand evaluation.6 Topics added from the needs assessmentwere the following: effective teaching and adult learning,3

setting goals and defining objectives for teaching,7 givingcritical feedback, lecturing,8 and modeling as a teachingtechnique. A grand rounds on feedback and evaluationwas provided in early 2000 for a total of 10 sessions. Allfaculty in the division were invited to attend all the ses-sions. The training took place over a period of 20 months;the initial sessions were completed over 13 months, andthe grand rounds was held 7 months after the end of themonthly sessions as a boost for the division and an intro-duction for other pediatric faculty.

The eight 1-hour sessions were held in an open-groupinteractive format, starting with a background review byM.S.B. or an expert guest presenter and including groupdiscussion of the topic. Each 1-hour session took the placeof a regularly scheduled division meeting with attendanceby all members expected by the division director. The 3-hour workshop on feedback and evaluation was developedin collaboration with faculty from another institution. Thegrand rounds was presented in an interactive format butwas limited by the large lecture hall typical of these for-mal presentations.

Evaluation

At the end of the program, participants completed aquantitative self-assessment questionnaire (with open re-sponses as well) rating their perceptions of their skills andconfidence both before and after the program.9 Only full-time faculty who were members of the division during theentire time period (n 5 13) were included in the finalanalysis. Responses were entered into NCSS, and analysiswas performed with descriptive statistics, frequency ta-bles, and the Wilcoxon Signed-Rank Test for Differencein Medians with continuity correction, a nonparametrictest comparing the before and after measurements of char-acteristics of teaching.10,11 We chose this test because thedistributions were found to be nonnormal. Three yearsafter the program, a confidential e-mail request for com-ments on any long-lasting effects of the program was sentto participants. These comments were collated by the sec-retary and submitted without identifiers for qualitative re-view.

RESULTS

Thirteen (10 [77%] women) faculty participants werefull-time members of the division during the entire pro-gram. At that time, the department had 71 clinical faculty(62% women). All division members were included ex-cept for the first author (M.S.B.), 2 faculty members who

left during the 20-month period for other employment,and a faculty member who was not full-time in the divi-sion. Five (38%) had been on the faculty more than 10years, and 8 (62%) had been on the faculty 5 years or lesswhen the intervention began. Participants attended an av-erage of 6.5 sessions (range, 4–10; median 7). Attendanceat the individual sessions ranged from 5 (at the 3-hourworkshop) to 11, with a median of 9 at the 1-hour ses-sions.

All 13 participating full-time faculty members com-pleted the follow-up quantitative self-assessment question-naire with retrospective assessment of preintervention sta-tus. Faculty were asked to ‘‘rate the degree to which eachvariable was characteristic of your teaching before April’98 (PRE) compared to NOW’’ on a Likert-like scale from1 (minimally) to 5 (highly). Statistically significant changewas seen in all items (Table).

In the open-response section, 7 (54%) respondents spe-cifically commented on improved ability to give feedback,and 2 (15%) commented on improved lecturing skills.Other comments included the need for more time to focuson teaching, the need for refresher sessions, and the use-fulness of hearing how other faculty members handle spe-cific situations. Frustrations identified included settinggoals and objectives for daily teaching as burdensome,wishing all sessions could have been attended, creatingmore time for teaching, wishing for ways to diplomati-cally unteach bad ideas (trying to encourage the use ofevidence based medicine, critical appraisal, and commonsense) in the face of dogmatic anecdotal teaching by oth-ers, needing tips to motivate residents, and needing ateaching syllabus.

In addition to faculty self-report, an indirect measuresuggests that the program improved faculty teaching. Allfaculty are eligible to receive a Dean’s Excellence inTeaching Award each year. The department chair uses im-plicit criteria to choose recipients. In the 3 years beforethe intervention, 26% to 31% of pediatric award winnerswere in our division (the division represents approximate-ly 18% of departmental clinical faculty). During the in-tervention, 11 award winners (48% of pediatric awardwinners) were in our division. The following year, 6 ofpediatric award winners (35%) were in our division.

The 6 of 13 participants (46%) who responded to thefollow-up survey noted long-lasting effects in the areas ofgeneral teaching skills (‘‘paying more attention to how Iteach,’’ ‘‘focused, direct topic-based teaching,’’ ‘‘settingindividual goals and objectives for each trainee,’’ ‘‘I getthem [students] to write down and commit themselves’’),feedback and evaluation (‘‘spending more time thinkingabout how what I tell the student/resident can help himor her change,’’ ‘‘giving more specific feedback,’’‘‘worked more on providing feedback’’), and overall use-fulness (‘‘the program . . . had a significant influence,’’ ‘‘Istill use the ideas I learned in these sessions,’’ ‘‘I havepaid more attention to self-assessing my teaching,’’ ‘‘veryhelpful,’’ ‘‘durable’’).

AMBULATORY PEDIATRICS Effect of a Teaching Skills Program 119

Pre- and Post-training Self-Assessment of Teaching Knowledge and Skill*

Area of TrainingPretraining Mean

(SD)Posttraining Mean

(SD) P Value

Teacher knowledge

Familiarity with Adult Learning TheoryAware of learners’ needsSet goals for teachingDefine objectives for learnersIncorporate suggestions into my teachingRead about teaching techniquesAware of resources to assist growth as teacher

2.07 (1.15)3.00 (0.81)2.61 (0.96)2.46 (1.05)3.18 (0.98)2.46 (1.33)2.08 (1.12)

3.53 (1.05)3.69 (0.75)3.69 (0.85)3.61 (1.04)3.91 (0.70)3.38 (1.04)3.08 (0.95)

.001,.01,.001,.001,.01

.001

.001

Teacher skills

Comfort with outpatient teaching skillsUtilize outpatient microskills modelComfort with inpatient teaching skillsComfort with lecture skillsAttitude supportive of learning skills in teaching to teachRegularly self-assess teaching

3.46 (1.05)2.00 (1.15)3.38 (1.04)2.84 (0.80)3.00 (1.00)2.66 (1.23)

3.85 (0.69)2.80 (1.13)3.92 (0.49)3.77 (0.83)3.85 (0.99)3.41 (0.99)

,.09,.01,.03

.002

.005

.005

Feedback and evaluation

Differentiate evaluation from feedbackGive positive feedbackGive critical feedbackFacilitate receipt of feedback from learnersTeach with feedback

2.67 (1.23)4.31 (0.94)2.38 (1.12)2.53 (0.97)2.91 (1.00)

4.00 (0.85)4.69 (0.63)3.46 (0.88)3.38 (0.87)3.58 (0.79)

.001

.01

.001

.002

.005

Other

Reflect cultural issues in teachingComfortable with problem teacher-learner interactionsAware of ways to promote my teaching for faculty promotion

3.08 (1.12)2.23 (0.83)1.92 (1.04)

3.69 (0.75)2.92 (1.04)2.38 (0.96)

,.02.003

,.02

*Degree to which each variable was characteristic of teaching preintervention compared with time of survey; 1 5 minimally, 5 5 highly.

DISCUSSION

Personal communication and responses to the initialneeds assessment indicate that our faculty are invested inteaching. The training focused on areas in which the par-ticipants expressed a specific need (such as giving criticalfeedback and facilitating receiving feedback from train-ees) or on areas that were common needs for pediatricacademic faculty (such as lecturing skills). Time manage-ment was woven into the sessions because many partici-pants expressed difficulty making teaching a priority withlittle to no dedicated time to teach. Faculty seemed moreable to differentiate feedback from evaluation after theirtraining, which included role-playing experiences duringthe sessions. Most of the participants in the program al-ready were comfortable with giving positive feedback, butmost wanted assistance with giving critical feedback andfelt that they improved in their comfort in this area.

The curriculum centered on a single division of generalpediatrics within a department of pediatrics, in contrast tomuch of the literature on generalist faculty develop-ment,12,13 in which interventions were department- orschoolwide. One clear advantage of this approach is thatour cohesive group will be able to continue working to-gether on these areas. Team activities have been identifiedby generalist faculty as a way to stimulate individual fac-ulty development.14 Our participation level of 100% offaculty attending 4 or more sessions contrasts with theparticipation level of 40% attending 2 or more conferenc-es in a series of 7 over the course of 1 year, as obtainedby Lye et al.15 The utility of the program is based on its

reliance on local faculty to teach in areas of personal ex-pertise, its dependence on faculty discussion and interac-tion to supplement the didactics, and its provision of amanageable time commitment by attendees. We believethat our program was successful because the division di-rector valued it and facilitated attendance for the faculty.The session with the lowest attendance was the one re-quiring a 3-hour commitment.

Evaluation of the program was limited in that we didnot formally evaluate change in style, content, or qualityof teaching before and after the intervention16 but usedonly self-reported skill, comfort levels, and long-term use.Our training focused on a single division of general pe-diatrics, limiting the number of participants; however, thepositive effects were encouraging, and the program wehave described would be easy to replicate in other pedi-atric academic divisions.

CONCLUSION

Local training is feasible, well attended, and appears toimprove teaching skills. Frustrations were common, andthere was interest in future sessions. Follow-up sessionshave included how to use educational activities for pro-motion, which was identified as a need on the programevaluation.

ACKNOWLEDGEMENTSThe Ambulatory Pediatric Association Faculty Development

Scholars Program directly influenced this work and provided theneeded encouragement to seek publication of our experiences. Spe-cial thanks go to Dr Paula O’Neill (The University of Texas-Houston

AMBULATORY PEDIATRICS120 Barratt & Moyer

Dental Branch) and Dr Virginia Niebuhr (University of Texas Med-ical Branch) for their assistance in teaching our faculty.

REFERENCES1. University of Texas-Houston. Mission Statement. Available at:

http://www.med.uth.tmc.edu/. Accessed April 2002.2. El-Bagir M, Ahmed K. What is happening to bedside clinical

teaching? Med Educ. 2002;36:1185–1188.3. Whitman N, Schwenk TL. The Physician as Teacher. 2nd ed.

Salt Lake City, Utah: Whitman Associates; 1997.4. Bellet PS. How to improve teaching on the hospital wards. Arch

Pediatr Adolesc Med. 1994;148:652–654.5. Osborn L, Whitman N. Ward Attending: The Forty Day Month.

Salt Lake City, Utah: Dept of Family and Preventive Medicine;1991.

6. Ende J. Feedback in clinical medical education. JAMA. 1983;250:777–781.

7. Roberts KB, DeWitt TG. Faculty development of pediatric prac-titioners: complexities in teaching clinical precepting. Pediat-rics. 1995;97:389–393.

8. Vella J. Lose the Podium. Convene. November 1998:28–33.

9. Levinson W, Gordon G, Skeff KM. Retrospective versus actualpre-course self-assessments. Eval Health Professions. 1990;13:445–452.

10. NCSS 2000 Statistical System, for Windows [computer pro-gram]. Number Crunchers Statistical Systems. Kaysville, Utah:NCSSS; 1998.

11. Snedecor GW, Cochran WG. Statistical Methods. 8th ed. Ames,Iowa: Iowa State University Press; 1996.

12. Walling AD. Medical Education 101—a faculty developmentcourse. Acad Med. 1999;74:609.

13. Hewson MG. A theory-based faculty development program forclinician-educators. Acad Med. 2000;75:498–501.

14. Baldwin CD, Levine HG, McCormick DP. Meeting the facultydevelopment needs of generalist physicians in academia. AcadMed. 1995;70(suppl):S97–S103.

15. Lye PS, Simpson DE, Wendelberger KJ, Bragg DS. Clinicalteaching rounds. Arch Pediatr Adolesc Med. 1998;152:293–295.

16. Hewson MG, Copeland HL, Fishleder AJ. What’s the use offaculty development? Program evaluation using retrospectiveself-assessments and independent performance ratings. TeachLearn Med. 2001;13:153–160.