PSYCHOMOTOR SKILLS IN RESIDENCY TRAINING

PSYCHOMOTOR SKILLS IN RESIDENCY TRAININGJose Y. Cueto Jr., MD, MHPEdChairman, 2012 Professional Regulatory Board of MedicineProfessional Regulation CommissionObjectives for the session:1. Discuss the educational principles behind the acquisition and development of psychomotor skills2. Examine required number of operations for OB-GYN residents3. Discuss research studies relevant to training and how to utilize research data4. Formulate a system of determining and validating requirements in number of operations in residency training (both for graduation and for certification)5. Formulate a system of documentation and reporting

Objectives for the session:6. Determine how the number of operations performed during residency will affect credentialing and privileging in hospitals7. Discuss performance-based evaluation, specifically the Objective Structured Clinical Exam (OSCE)

Main Objective of Training NOVICE COMPETENT EXPERT SURGEON I. Educational PrinciplesFitt and Posner (1967): 3 phases of skills acquisition

Millers PyramidFitt and Posner: 3 Phases1. Cognitive phase

2. Associative phase

3. Autonomous or Fixation phaseFitt and Posner: 3 Phases PHASES

COMPONENTS / FEATURESCOGNITIVE PHASE (the mental part)Understanding the principle behind the procedureKnowing the indications and contraindicationsRecognizing the risks and complicationsIdentifying and analyzing the steps and their proper sequenceIdentifying the critical parts of the procedure Internalizing what are observed during assists and what are discussed during rounds and conferences

Fitt and Posner: 3 Phases PHASES

COMPONENTS / FEATURES2. ASSOCIATIVE PHASE (the action or activity part)

Converting the mental picture into actual actionPerforming the procedure according to determined sequenceEnsuring guidance and supervision by ConsultantObtaining feedback on what was done right and what needs to be correctedAllowing adequate practice to polish rough movementsDeveloping ability to concentrate on the procedure

Fitt and Posner: 3 Phases PHASES

COMPONENTS / FEATURES3. AUTONOMOUS or FIXATION PHASE (the refined, smooth part)Mastering the steps and correct sequenceDeveloping smooth movements with minimal wasted moves Making the skill become automaticModifying the procedure when conditions require itPrecaution: If uncorrected during the 2nd phase, there is danger of incorporating erroneous habits which will be difficult to unlearn

Millers PyramidProgression of Skills Acquisition FITT and POSNER: 3 PHASES

MILLERS PYRAMIDCognitive PhaseKnowsKnows howAssociative PhaseShows howAutonomous or Fixation PhaseDoesProgression of Skills Acquisition and Development 2nd Assist (mainly exposure)

1st Assist (limited participation in procedure)

Surgeon (under direct supervision)

Surgeon (independently performing) Tracking Progression and Scheme of Reporting: 3 columns Operation / ProcedureNumber of First Assists Number of supervised operations / proceduresNumber of independently performed operations / procedures1. Total abdominal hysterectomy2. Cesarian hysterectomy3. Adnexal surgery4. Cesarian section5. Vaginal HysterectomyII. Handbook of PBOGS, 2006 PROCEDURE 1st 2nd 3rd 4th TOTALNSD 50 20 70Episiorrhaphy 25 10 35Outlet forceps / vacuum extraction 15or 15 15Dilatation and Curettage 20 20 40Manual extraction of placenta 3 3Partial breech extraction 7 or 8 8Cesarian section 10 20 30Abdominal hysterectomy 5 5 8Adnexal surgery 5 5 10Vaginal hysterectomy 1 or 1 1Cesarian or postpartum hysterectomy 1 or 1 1Evacuation of H. mole 1 or 1 1My ObservationDescription of The Residency Training Programs should include a section on Psychomotor SkillsInclude an explanation on the required number of operations using statisticsClarify the progression of operative cases handled by residents and provide the basis Documentation and reporting found in Basic Requirements for Examination, page 42; but not discussed in the curriculum (same requirements for graduation and certification)General PrinciplesProgression: Trainees first learn simple tasks, before they progress to procedures of medium difficulty, and finally to complicated proceduresTransfer of learning / transfer of skills: what trainees learn from simple skills are utilized in the performance of medium and complicated skillsTrainees need to learn to pay attention to details, follow proper sequence, observe meticulous technique

Application of Principles1. The simpler the procedure, the shorter it takes to learn, and the less number of cases needed to master it 2. The more complicated the procedure, the longer it takes to learn, and the more number of cases needed to master it3. The more complicated the procedure, the higher the need for supervision and guidanceImplementation Residency Training Committee1st yr1st yr2nd yr3rd yr4th yrRoles according to year level Operation 1st yr 2nd yr 3rd yr 4th yr

Requirement for Abdominal hysterectomy 5 5Role 1 2nd assist 1st assist surgeon surgeon

Role 2 2nd assist 1st assist (12)Surgeon, supervised (3)Surgeon, independent (7)Evaluating Phases of LearningFitts 3 Phases Millers PyramidMethod of Evaluation/Setting

1. Cognitive PhaseKnowsKnows howDiscussion Pre-op conferenceQ and A ORRecord review Ward / office

2. Associative PhaseShows howDirect observation Skills lab (Supervised by Simulations Consultant) OR

3. Autonomous PhaseDoesDirect observation ORRecord review WardOutcome evaluation Rounds ConferenceIII. Researches1.Does Residency Training Improve Cognitive Competence in Obstetric and Gynaecologic Surgery?, Balayla, Abenhaim and Martin, McGill University, Montreal, Can, J Obstet Gynaecol Can 2012;34(2):1901962. Competency-based Residency Training: The Next Advance in Graduate Medical Education, Donlin Long, MD, PhD, Johns Hopkins University School of Medicine, Academic Medicine, Dec 2000

Researches3. Factors Associated with a Successful Outcome in the PBS Certifying Examinations, Crisostomo and Marfori, Philippine Journal of Surgical Specialties, Oct-Dec, 2010

Does Residency Training Improve Cognitive Competence in Obstetric and Gynaecologic Surgery? (JOGC,2012)Objectives:1.To develop an operative knowledge assessment tool to evaluate the cognitive competence of trainees in obstetric and gynaecologic surgery 2.To determine the rate of change in competence during a five-year residency program .

J Obstet Gynaecol CanMethods: Twenty-eight participants in five training groups (PGY-1 to PGY-5) in McGill Universitys residency program in obstetrics and gynaecology Evaluation based on surgical cognitive competence (SCC) assessment tools Three different obstetric and gynaecologic operations: open total abdominal hysterectomy (TAH), Caesarean section, and laparoscopic bilateral tubal sterilization (BTL) J Obstet Gynaecol Can Performance of an operation/procedure Three fundamental components: 1. cognitive factor 2. technical element 3. judgment componentJ Obstet Gynaecol Can COMPONENT

FEATURE1. cognitive factor

the knowledge of the theoretical steps of the procedure 2. technical element

takes the theoretical steps into account and translates them into the performance of the operation

3. judgement component

comes from surgical experience and allows a surgeon to rely on his or her own intuition to determine the appropriate operative course of action on a case by case basisJ Obstet Gynaecol Can Summary of Findings:

OPERATIONPERCENTAGE INCREASE IN COGNITIVE COMPETENCE PER YEAR1. Total Abdominal Hysterectomy 15.73%2. Cesarian Section

8.06%3. Laparoscopic BTL

16.31%J Obstet Gynaecol Can Findings:At level of PGY-5, residents had 100% surgical cognitive competenceThis type of information may be helpful in ascertaining how long a residency program should be

Competency-based Residency Training: The Next Advance in Graduate Medical Education (AM, 2000)Donlin Long, MD, PhDProfessor of Neurosurgery, Johns Hopkins UniversityStudied NSS residentsIntroduced competency-based program

Academic MedicineTraditional ProgramCompetency-based Program1. Fixed number of years2. Residents have to learn all specified knowledge and skills in the allotted time3. Problem of evaluating competence of every resident4. Graduate may not be competent to perform required procedure or manage particular patients.

1. Specifies maximum duration2. Time taken to acquire knowledge and skills is based on the abilities of individual trainees 3. Evaluation of every resident in every procedure4. Resident is evaluated and certified to have acquired competence and confidence to practice independently

Factors Associated with a Successful Outcome in the PBS Certifying Examinations (PJSS, 2010) Objective: To determine the factors associated with a successful outcome in the PBS certifying exams (written and oral)

Method: Retrospective, cross-sectional study utilizing 370 candidates from 2006-2009, with 137 (37.0%) successful outcomesPJSS Significant Factors:Younger age of examineesPrevious performance in the RITETaking the exam within a year of completion of residencyTraining in a university-based programUndertaking subspecialty fellowship during the examination year

PJSSOther Factors (Not Significant)Sex (M-F)Marital StatusCase volumeContinuous/interrupted programLocation of training program (MM vs. outside)PJSSCASE VOLUME performed during residency (major operative procedures):1. High volume: 299 0r more cases (upper 3rd)2. Medium volume: 171-298 cases (middle 3rd)3. Low volume: less than 171 cases (lower 3rd)

NOTE: did NOT influence performance in written and oral examsOpen for further researchRelationship of case volume to: 1. surgical cognitive competence 2. technical element 3. judgment componentUtilizing Research Data (PJSS) FAVORABLE FACTORS (passing the PBS Certifying Exam)

Accreditation CommitteeResidency Training Committee

DECISIONS / ACTIONS1. Graduation from a university-based training programClosely monitor the residents and graduates from govt and private institutions2. Satisfactory performance in the Residency In-training ExamInstitute remedial measures for residents with low scores in the RITE(identify topic areas)3. Taking the exam within one year after end of residencyEncourage / require graduates to take the certifying exam within one year after end of trainingUtilizing Research Data (JOGC) FACTORS IDENTIFIED

DECISIONS / ACTIONS1. Percentage increase in surgical cognitive competence per year; reaches 100% at level PGY5Policy: convert all 4-year programs to 5-year programs2. Ability of the resident to identify the most critical steps (given 24-32 total steps in certain procedures)Provide in-depth discussions and adequate exposure and practice prior to allowing resident to perform actual operation3. The cognitive factor and judgment component are more important than the technical factor in the performance of proceduresEvaluate the development of cognitive factor and judgment component of operative skillsUtilizing Research Data (AM) FACTOR IDENTIFIED

DECISION / ACTIONAbilities of residents and pace of acquiring knowledge and learning operative skills differ.Provide flexibility in the duration of a training program but place a limit or maximum duration.

Evaluate competencies in each procedure before certifying for promotion to higher level or for graduation.

When in doubt, provide extension for additional operative casesThe essential messages1. The process of acquiring and developing operative skills is more important than the output (number of operations performed).

2. Quality is more important than quantity.

3. Supervision and feedback are critical. There may be institutions where residents get to perform so many operations by themselves, but they never get to know which steps are done correctly or incorrectly.The Essential Messages4. The cognitive factor and the judgment component are very critical in the performance of procedures

5. The fixed duration of residency training may not be appropriate for a number of residents

Determining RequirementsFactors to consider:1. Degree of difficulty: simplicity / complexity of procedure /low-risk / high-risk *The simpler the procedure, the lower the number required to acquire competence *The more complex / complicated the procedure, the higher the number required to acquire competenceDetermining RequirementsFactors to consider:2. Trainee factor: fast / slow learner dexterity with procedures *The fast learner and the trainee with dexterity / adeptness at performing procedures will require lower number of cases *The slow learner and the trainee with clumsiness in performing procedures will need a higher number of cases Determining RequirementsFactors to consider:3. Institutional factor: high volume vs. low volume private vs. charity/service patients

*The resident belonging to a high-volume hospital will require a lower number of cases (reinforcement) *The resident belonging to a low-volume hospital will require a higher number of cases (too few and far in-between)Determining required number of operations1. Use of the Delphi technique12-30 ExpertsList of operations3 roundsQuestions to answer QuestionsBased on your experience and expert opinion:How many times should a resident assist in the following procedure before he can be given his first case?How many times should a resident perform this procedure under direct supervision before he is allowed to perform it independently?How many times should a resident perform this procedure before he can acquire the competence and the confidence to perform it safely on his own?

Three RoundsRound 1: 12-30 experts give their proposed number of operations, based on the questions; without them communicating with each otherRound 2: the experts are given feedback on how their colleagues answered the questions (tabulation of results); afterwards they are asked for their modified list of proposed number of operationsRound 3: the experts are gathered and they are asked to arrive at a consensus regarding required number of operationsValidation Stage1. Identify institutions:GovernmentUniversity-basedPrivate2. Conduct a parallel research study: longitudinal tracking of residents until they get to the certifying exams3. Based on results, modify/maintain the requirementsDetailed Documentation of Operative Experience (35 cases) List of OperationsNumber of Assists Number of supervisedoperationsNumber of independently performed operationsTotal abdominal hysterectomy 15 Cesarian/postpartum hysterectomy 1Vaginal hysterectomy 1Adnexal procedures 10 Vaginal extraction of H. mole 1 Indicated manual extraction of placenta 1Breech deliveries 1Outlet forceps or vacuum 5My ObservationNotation: Starting 2006, only 70% of cases with complete transfer of technical responsibility is allowedWhat is the basis for the policy?What problem does it solve? Lack of residents cases?Consultants cases given to residents: fall under 2nd column (supervised cases)Private cases: no complete transfer of responsibility demo cases to show residents how procedures should be doneResidents Responsibilities1. provide preoperative evaluation, assessment of risks2.identify indications/contraindications to planned procedure, possible complications3. perform the procedure, modifying it in presence of unforeseen conditions4. providing immediate and long-term care.

Note: For private patients, decisions will always be made by the Attending Consultant (The judgment component is lacking)Deficiencies and ActionsWhat happens if residents do not meet requirements? Extension of rotation in a particular service where deficiencies occurred.Reduce the number of residents or admit residents every other year.Terminate program after adequate opportunities for correcting deficiencies have been given

Implications for Future Credentialing and Privileging Operations / Number Performed / Required Allowed Not Allowed1. Dilatation and curettage 50 (40) / 2. Cesarian section 50 (30) /3. Adnexal surgery 15 (10) /4. Abdominal hysterectomy 10 (10) /5. Vaginal hysterectomy 1 (1) x6. Cesarian hysterectomy 1 (1) xQuestionWill the departments accept deficiencies, and still allow residents to graduate?

Will the board accept deficiencies and still allow graduates to take the certifying exams?Recommendations1. Shift FOCUS to process of acquiring and developing operative skills rather than the number performedRecommendations1. Shift FOCUS to process of acquiring and developing operative skills rather than the number performed

2. Aim for qualitative improvement by providing guidance, supervision and feedback.Recommendations1. Shift FOCUS to process of acquiring and developing operative skills rather than the number performed

2. Aim for qualitative improvement by providing guidance, supervision and feedback

3. Pay attention to the cognitive factor and the judgment component of performance.Recommendations1. Shift FOCUS to process of acquiring and developing operative skills rather than the number performed

2. Aim for qualitative improvement by providing guidance, supervision and feedbac

3. Pay attention to the cognitive factor and the judgment component of performance

4. Conduct researches on different components or aspects of residency trainingPerformance-based EvaluationThe Objective Structured Clinical Exam (OSCE)Multiple stationsTime allotted: 5-25 minsWell-defined clinical taskUse of real or standardized patient (SP)Use of ratersRating scales and checklistsOSCE MILLERS PYRAMID

LEVEL OF PERFORMANCE MOST EFFECTIVELY MEASURED BY OSCE KNOWS KNOWS HOW +++ SHOWS HOW +++++ DOESOSCE COMPETENCIES LEVEL OF PERFORMANCE EFFECTIVELY MEASURED BY OSCEKNOWLEDGE ++PROBLEM-SOLVING +++++ CLINICAL DECISION-MAKING +++++SKILLS, HISTORY-TAKING +++++SKILLS, PHYSICAL EXAM +++++SKILLS, PROCEDURES +++++SKILLS, INTERPERSONAL +++ATTITUDES +++OSCE DevelopmentPlanning/preparing needed resourcesIdentification of competencies (test blueprint)Identification/recruitment of ratersTraining of standardized patients and ratersConducting workshops on standard-settingConstructing rating scales/checklistsGathering diagnostic materialsSelecting venuePilot-testing

SummaryEducational basis for the acquisition of skillsRe-examined PBOGS requirementsResearch studies on trainingSystem of determining requirements and validationSystem of documentation and reportingDiscussion on policiesFuture implicationsOSCE ENDWorkshop Activity1.Determining requirements in 3 columns (Delphi method)

Operative ProcedureNumber of1st AssistsNumber ofSupervised operationsNumber of independently performed operationsRound 1Round 2Round 3Workshop Output1.Determining requirements in 3 columns (Delphi method)

Caesarian sectionNumber of1st AssistsNumber ofSupervised operationsNumber of independently performed operationsGroup 18715Group 2555Group 310312Workshop Output1.Determining requirements in 3 columns (Delphi method)

TAHBSONumber of1st AssistsNumber ofSupervised operationsNumber of independently performed operationsGroup 112818Group 21055Group 31058Workshop Output1.Determining requirements in 3 columns (Delphi method)

TAHBSONumber of1st AssistsNumber ofSupervised operationsNumber of independently performed operationsGroup 1Group 2Group 3Workshop Activity2. Discussion: graduates who cannot pass the certifying exams: reasons / courses of action PBOGS Passing: 65-67% (2001-2006) Philhealth: CS done by Diplomates/Fellows (60%) CS done by GPs with training (30%) CS done by MDs w/o training (7.4%) Physician Act of 2012: Art. V. Sec.28 (k): Performing an area of specialization without fulfilling specialization requirements prescribed by the AIPO and the Board of Medicine 68