Embed Size (px)
Citation preview
April 2014
UT-COM Chattanooga 8 categorical/year ◦ Janara Huff PD 12 years ◦ Marielisa Rincon APD 3 years ◦ Melissa Hamp ex-PD 4 years (director adolescent med) ◦ Annamaria Church ex-PD (director general pediatrics)
Kaiser Permanente Northern California 8 categorical/year ◦ Abhay Dandekar PD 5 years (after APD for 4 years)
University of Missouri—Columbia 6 categorical/yr ◦ Aneesh Tosh APD 3 year
University of Oklahoma-Tulsa 6 categorical and 3 med-
peds/year ◦ Keith Mather PD __ years
We have nothing to disclose
Milestone integration 63% EPAs 53% Individualized curriculum 43% Faculty development 43% CCC 33% Quality Improvement 23% Professionalism 13% Scholarship 10% Global health curriculum 10%
Smaller program LISTSERV-- 80% thought this would be helpful ◦ Improve opportunities to collaborate with each other or with
APPD LEARN ◦ Ask questions ◦ Share resources ◦ APPD office will help us build the LISTSERV ◦ Need contact information if you are interested
PEC and what is required for the Annual Program Report
–Is this WebADs or something else CLER visit What do you do with pediatrician needing another year
of training to sit for the boards?
Better tools to train faculty ◦ Videos to assess different subcompetencies -beyond patient care and medical knowledge
Need time to make faculty better assessors Feedback on how they do with their evaluations Unlikely to happen in current environment focused
on clinical productivity
Concept of developmental progression is clear Many evaluations in different settings by different observers are
required Milestone assessment done really well takes time ◦ Most faculty do not analyze the milestone descriptors when completing
evaluations and are really using “gut feeling” about how this resident compares to others
◦ Novice-expert = 1-5 or 1-9 ◦ Typical ratings by academic year: PGY1 mostly 1-2 PGY2 mostly 2-3 PGY3 3-4
◦ Likability factor still plays a role ◦ In small programs knowing about “problem” residents may impact the
evaluation Are the current evaluations more meaningful than “pre-
milestone”?
Same members as our old curriculum committee --experienced about resident education/evaluation ◦ 4 subspecialists (PD included as observer) and 3
Gen Peds ◦ APD is chair 1 member reviews resident file and “presents” Use everything online and in folder (emails good and bad,
mentor, 360s, handover, colleague, completion of non-patient care requirements)
Shared knowledge/opinion of committee modifies Final report written Discussed with resident
◦ Milestone placement works well when we use all information available (mentor meetings, self-evaluations, emails, participation in non-clinical activities, colleague evals, 360 evals, morning report and handover assessment) in addition to the global rotation evals. ◦ Easily identifies residents who are underperforming ◦ Comments are extremely helpful in deciding exactly
where a resident is functioning A comment only evaluation form may be more useful
on some rotations than evaluations of sub-competencies
Our old evaluation questionnaires are now “mapped”to Milestones using New Innovations ◦ Faculty all have Milestone descriptors, short and
long versions, but the evaluations themselves offer only novice-expert buttons ◦ Only IP chose to use a full Milestone Evaluation
with all descriptors on the questionnaire to facilitate their assessment
Striking differences in resident self assessment from Milestone evaluation based on CCC assessment
Summary of Findings in addition to the Milestone
Scheduled 2 days of faculty time to evaluate all 24 residents in May
Each member will have 4 residents assigned to review ahead of time and 4 reports to write at the end ◦ About 2 hours of preparation for each resident
We haven’t entered data into the ACGME yet
Where is everyone with this? What problems have you encountered? How much faculty development have you done in
regard to Milestone assessments? How comfortable are you with your faculty
expertise in evaluating residents? Has anyone entered data into ACGME database and
how hard was it?
Introduction Initial experiences Implementation trial in our program Future challenges
Allow a framework for faculty to make competency-based decisions on the level of supervision required by trainees.
Units of professional practice EPAs are executable, observable, and
measurable in their process and outcome Translation of competency into practice EPA’s- Descriptors of the work that
physicians do ◦ Competencies are descriptors of physicians
EPAs usually require multiple competencies in an integrative, holistic nature.
EPAs need to be described ◦ “Tomorrow, you will be able to…..” ◦ Use of EPA worksheet
EPAs need to operationalized ◦ Need to go from “small” EPA’s (1) Performing a procedure ◦ To (2) more refined collections of these that
determine the activities of our profession Caring for the well newborn
Can we trust this trainee to execute this EPA? Observation but no execution, even with direct supervision ◦ Execution with direct, proactive supervision ◦ Execution with reactive supervision, ie, on request
and quickly available ◦ Supervision at a distance and/or post hoc ◦ Supervision provided by the trainee to more junior
colleagues
Can we trust this trainee to execute this EPA? ◦ Faculty training/consensus to review criteria for
entrustment How many supervisors? How many observed
procedures? How many reviewed documentation encounters? Expectations for preparation?
◦ Engage learners in process Self-assessment ◦ Build strength in numbers PD + rotation director + independent supervisor ◦ Document your progress and review Learners and supervisors review on ongoing basis Paper vs. electronic
Can we trust this trainee to execute a larger unit of professional activity? ◦ Linking the EPA to competencies and milestones
enables us to focus on the translation into practice ◦ Making decisions for supervising at a distance
requires observed proficiency, and usually multiple times ◦ Factor in variability in making entrustment
decisions: trainee, supervisors, circumstances, and the EPA itself
Can we trust this trainee to execute a larger unit of professional activity? Using the framework of the CEPAER (Core EPAs for entering residency) draft from AAMC to inform our work
CCC activity to further inform decisions on EPA’s – small and large ◦ CLER visit experience helped accelerate this by
helping to update and allow access to guidelines for supervision
Make milestone based assessment tool a granular approach through detailed roadmaps at the competency level
Make EPA’s a more holistic approach to assessing the integration of competencies required to care for patients
Elements of success that you’ve had at your program? ◦ Faculty development and learner understanding? ◦ Implementation ideas?
Struggles and challenges? ◦ Time? ◦ Comfort?
Resources- Each other(!); APPD/ABP; work of leaders and leadership
Had one faculty development session on how to mentor resident selection of individualized months. ◦ This is not adequate to prepare most mentors for this role works only for those on your inner circle of faculty many faculty are just signing off on resident plans
Adherence to the idea of 6 individual months for every resident without double counting ◦ We started this way but have had to rethink ◦ Initially divided them 2 individual months/year Individual months are better utilized later in curriculum
Probably an error--Allow one individualized month be a creative supervisory month ◦ i.e. supervise in non-standard location for your program
such as ED, PICU, NICU ◦ Popular idea with the residents
You have to have enough residents to cover key areas of the hospital unless you have resources for alternative providers ◦ Inpatient Ward: Currently 4/2-3/2-3
Tried 3/2/3 months but spread too thin 2 PICU and 2 NICU --third NICU/PICU recommended for
fellowship/hospitalist tracks are now counted as individualized months 1 nursery month plus time as supervisor Total 14-15 IP units, down from 17
◦ Outpatient Clinic: we require 2/2/2 2 ED and 1 evening acute care clinic 1 community/advocacy Total 10 OP units
Have had to “double count” 2-3 of the individualized months with subspecialty rotations or run out of months
Remediation months also count as individualized months Resident selection issues: few get truly creative rotations
We have 3 basic tracks with considerable overlap ◦ Community pediatrics-supervision can be 3 OP/2 IP
suggestions for private office, extra advocacy, optho/ent, dermatology, psychiatry, extra D/B, 1-2 weeks gyn
◦ Subspecialty fellowship-list of recommended rotations for each but many similarities extra PICU, NICU, IP, anesthesia, consider surgery and surgical
subspecialty rotations, extra rotation in subspecialty of interest ◦ Hospitalist—not significantly different from procedural fellowship
recommendations
No research, global health, advocacy, rural tracks
How many really get 6 individualized vs 3-4 after “double counting”?
How are they built in to your PGY1, 2, 3 years? ◦ i.e. 2/2/2 or 0/3/3, etc
Have your residents been creative and thoughtful in their choices?
Do you have resources to allow creative or non-standard rotations?
How have faculty mentors been prepared for advising about the choices?
How many have resources for “away rotations”?
A successful experience
The Columbia Connection
Aneesh Tosh, MD, APD, University of Missouri-Columbia
Keith Mather, MD, PD, University of Oklahoma-Tulsa
Renda Chubb, PC, University of Oklahoma-Tulsa
We have no relevant personal financial relationships or affiliations.
We do not intend to discuss or reference any off-label or unapproved drugs or devices.
Identify ACGME and Pediatric RRC requirements that may present hurdles for small pediatric residency programs.
Identify strategies that small programs can
incorporate that satisfy the requirements and can be cost-saving to the institution.
How many small programs have to send their residents to an away rotation to complete their requirements?
How many small programs do not have: ◦ A board-certified DBP pediatrician? ◦ A board-certified Adolescent Medicine physician? ◦ A board-certified Peds EM physician?
Keith was named as the new PD of the Pediatric Residency in Tulsa, OK, after the program had recently completed a site review
Three months later, Keith was notified by his DIO that the University had received notice of probationary status due to multiple citations.
Additional problem: Keith has no program director experience.
The OU-Tulsa Pediatric Residency Program does not have a board-certified Adolescent Medicine Specialist. The program utilizes a pediatrician with special interest in this area, but she does not fulfill the alternative qualifications.
“In the middle of difficulty lies opportunity.”
Aneesh and Keith share a shuttle from the airport to the hotel. Small talk about Missouri and the St Louis Cardinals
They share a table at the meeting the next morning, and Keith finds out that Aneesh as the APD is also one of two Board-Certified Adolescent Specialists at Missouri-Columbia (same size program)
Unfair! But………..
The two programs could develop a mutual win-win situation?
The Chairs of both departments could come to an agreed-upon financial arrangement?
The Pediatric RRC could actually approve the sharing of the Adolescent Specialists?
The faculty members at OU-Tulsa did not want to accept the arrangement?
350 miles
Specialists from Missouri-Columbia could not actually have liability of direct patient care
OU-Tulsa Educational Material and the Rotational Goals and Objectives were reviewed and adjusted by the Specialists
Resident performance evaluation must be by faculty from both programs
OU-Tulsa must have active educational involvement of the Adolescent Specialists from Missouri-Columbia
All RRC/ACGME requirements had to be met
OU-Tulsa Pediatric Program had a site review in January of 2013.
10 year accreditation Special commendation cited OU-Tulsa has recently hired an Adolescent
Specialist with a smooth transition this academic year from the Missouri-Columbia oversight process.
The city of Tulsa has a beautiful Pediatric Emergency Center with no Board-Certified Pediatric Emergency Medicine Physician.
Help?
Non-university settings or peripheral campus Clinician- educator faculty No basic science research ongoing Limited funding for “academic time” for faculty (3-
10%) Limited funding to send residents for presentation of
their work (resident salary $48,000) For residents seeking fellowships from small
programs some demonstrated scholarship is key to getting an interview ◦ Quick case report as PGY-1 to add to CV before fellowship
application plus ◦ Some larger project identified and ongoing
University affiliated program with clinical research options only Research support is organized through the GME department ◦ “How to” conference for interns ◦ Research resources on-line ◦ Project review committees pre- IRB ◦ Faculty mentoring ◦ Graphics assistance ◦ Biostatistician service
Research Week on campus with selection of best resident work for poster or podium presentation ◦ Judges from local and outside ◦ Monetary prizes awarded at a dinner
Dean had $ to send residents to present posters elsewhere ($1250/resident) but this may be taken away ◦ SSPR, PAS, AAP or specialty societies
Other scholarly paths not as well supported as research Minimal (or even negative) faculty reward for helping residents
(bonus based on wRVUs) with loss of previous bonus if not met next 6 months
All residents must do a scholarly project- did not seem fair in small program to expect some to do and some not. Pairs allowed if a lot of data extraction needed ◦ Research ◦ Advocacy ◦ Quality improvement ◦ Education
Excel file of faculty ideas kept by coordinator Each scholarly path with written descriptors of
requirements and timeline. Similar work load except some don’t require IRB
Physician Champion for each area
Chair is QI champion ◦ Committee to review QI ideas and assist with project development
both for small QI and larger scholarly projects ◦ New QI presentation day campus-wide with awards similar to
research day No formal setting for advocacy or education projects
presentations except at noon for our own department Since implementing these routes to scholarly activity 6
years ago only 2 residents have “escaped” without final presentation of a project ◦ Need faculty to drive this effort. Few residents do it
independently. Reminders to question the residents on progress with scholarly activity added to the mentor review and semi-annual PD review Make it a professionalism assessment point
◦ Having one faculty with dedicated time for pushing this requirement at resident meetings and willing to send email timeline reminders and prompts is very helpful. Last year we did not have this person
Other approaches to getting resident scholarly work done?
Problems getting faculty to help? How many programs require it of every resident vs
just those interested or those trying to get fellowships?
How is the faculty rewarded for time/effort? ◦ Is your administration encouraging scholarly activity?
Chattanooga fortunate to have ~40 paid faculty: ◦ A few of them do most of the scholarly work as far as
publishing or editing Make Grand Rounds a CME activity: ◦ Lots of paperwork! ◦ Many faculty benefit because it counts as scholarly activity
Other opportunities: ◦ Our faculty also give talks to the FP annual CME conference
and outlying area hospitals Helping with resident scholarly activity—can result
in publications
Listserv for resource, collaboration, support PEC CLER visit Practicing pediatricians needing 1y of training to
satisfy ABP requirement with lapse of eligibility Open Forum
https://www.appd.org/amsurvey/
