Upload
sanam
View
22
Download
0
Embed Size (px)
DESCRIPTION
IIME Workshop on Assessment. Meeting Summary Chengdu, China February 27, 2003. MCQ Summary. Hardest working group!! 150 Item test translated into Chinese Translation of each question reviewed by the entire group for clarity Examination details reviewed Location - PowerPoint PPT Presentation
Citation preview
IIME Workshop on IIME Workshop on AssessmentAssessmentMeeting SummaryMeeting Summary
Chengdu, ChinaChengdu, China
February 27, 2003February 27, 2003
MCQ SummaryMCQ Summary
• Hardest working group!!• 150 Item test translated into Chinese• Translation of each question reviewed by the
entire group for clarity• Examination details reviewed
– Location– Receiving and returning examination materials– Rules for how to proctor the examination
• Leader of MCQ team to recruit assistant proctor
OSCE SummaryOSCE Summary
• 15 Station (10 room) OSCE– 5 Long SP, 5 Short SP, 5 Non-SP Cases
• Blueprint of exam finalized• 1-day examination (3 sessions of 2 ½ hours)• Schools with 60 or 90 students can share
patients and/or repeat day 1 examination• CMB support provided through Association for
Chinese University Medical Colleges
Med 0201: Coronary heart
Med 0701:Asthma
Surg 0401:Acute choly
ER 0101: Low abdo pain
Psych 0601: Depression
Psych 0604:Panic disorder
Peds 0801: Malnutrition
Peds 0301: Fever + cough
OBG 0502:High risk preg
OBG 0501: Ectopic preg
OBG paper case, pelvic model, mcq
Surgery gowning or sterile technique,
tube, or CPR
Literature Search: computer
Medicine paper case, lab report, mcq,
auscultion
Medicine paper cases, Xray, ECG,mcq
Long SP cases Short SP cases Non-SP cases
2003 OSCE Day 1
Observation/LogbookObservation/Logbook Summary Summary• Scales finalized for all 17 GMER • Faculty at this meeting trained to use the
observation form• IIME to create evaluation form and send final
version before April 30, 2003• Faculty development plan workshops at your home
schools• Observations before the examination• Submit forms back to IIME for scoring and analysis
Tasks for March 2003Tasks for March 2003
• Leader for IIME project from each school will be primary contact for all information
• IIME will create spreadsheet for student registration process
• Leader registers students for examination– Spreadsheet returned to IIME with student
identification numbers– Additional data on students for comparison to your
school-based examinations and grades
• IIME creates draft of faculty observation forms
Tasks for April-June 2003:Tasks for April-June 2003:Faculty ObservationsFaculty Observations
• IIME creates final evaluation forms and sends to each school
• Schools begin faculty development workshops to teach how to use the evaluation forms
• Begin faculty observations in June or July for 3 months
• At least 3 evaluations for each student
Tasks for April – June 2003:Tasks for April – June 2003:Information for Students and TeachersInformation for Students and Teachers
• IIME will develop a brochure in Mandarin to educate students about the IIME project:– MCQ– OSCE– Faculty Observations
• Also provide brochure to teachers
• Draft score report to deans and presidents
Tasks for July:Tasks for July:Standardized Patient DevelopmentStandardized Patient Development
• Possible meeting in July (Shenyang and/or Chengdu)
• 1-day workshop
• OSCE Coordinator, Standardized Patient Trainer, OSCE team (1-2 people/school)
• Details of how to hire, train, and work with standardized patients.
Tasks for August – September Tasks for August – September 20032003
• Final room arrangements for MCQ, receive test books
• Final preparations for OSCE, complete training of SPs, recruit faculty
• Complete faculty observations in clerkships and prepare forms to return to IIME
October 2003!!!October 2003!!!
• MCQ Administration– Secure receipt, administration, and return of
test materials
• OSCE Administration– Score reports created on-site and email to
IIME
• Observations– Forms sent to IIME by secure mail
Timeline: 2003Timeline: 2003
MCQ OSCE OBSERVATION
March Student Lists
Final Stations
Draft Evaluation Forms
April Faculty Workshops
May Faculty Workshops
June Recruit Proctors
Recruit Faculty
Begin Assessments
Timeline: 2003Timeline: 2003
MCQ OSCE OBSERVATION
July Reserve Exam Room
Additional Meeting
Student Assessments
August Sample Material
Training of SPs
Student Assessments
September Receipt of Test Books
Test Center Preparation
Student Assessments
October Examination Examination Forms submitted
Quality Assurance ProgramQuality Assurance Program• MCQ examination
– Statistical review for quality of questions– Comparison with your home school examinations
• OSCE examination– Anonymous videotapes sent to review and score at
other schools
• Observations– Statistical review for quality of scores reported by faculty
• Student and faculty assessment of examination– How realistic?– Fair coverage of domains?
Score ReportsScore Reports• Data security in the IIME office• Score reports created by GMER item, by
discipline, and by student to return to IIME leader at your school.
• Comparison of scores with other Chinese students WILL BE anonymous and combined ONLY.
• Areas of success and improvement will be identified in this report
• Comparison of these tests with your school examinations can be provided
Score Report: Major DomainsScore Report: Major Domains
• Professional Values, Attitudes, Behaviors and Ethics
• Scientific Foundation of Medicine• Clinical Skills• Communication Skills• Population Health and Health Systems• Management of Information• Critical Thinking and Research
Score Report: ProfessionalismScore Report: ProfessionalismProfessional values which include excellence,
altruism, responsibility, compassion, empathy, accountability, honesty and integrity, and a commitment to scientific methods;
Self-regulation and a recognition of the need for continuous self-improvement with an awareness of personal limitations including limitations of one’s medical knowledge;
Recognition of the moral obligation to provide end of life care, including palliation of symptoms;
Ability to effectively plan and efficiently manage one’s own time and activities to cope with uncertainty, and the ability to adapt to change;
Personal responsibility for the care of individual patient.
Score Report: Scientific Score Report: Scientific FoundationsFoundations
01020304050
60708090
100
My School All Schools
My School
All Schools
Mean 74 72
5th Percentile 66 68
95th Percentile 82 78
Students Below Standard
2% 10%
Score Report: PediatricsScore Report: Pediatrics
01020304050
60708090
100
My School All Schools0
1020304050
60708090
100
My School All Schools
MCQ OSCE
Score Reports: Correlation of Score Reports: Correlation of Overall MCQ ExaminationOverall MCQ Examination
Correlation Significance
Entrance Examination r=0.45 p<0.001
Year 2 Examination r=0.52 p<0.001
Internal Medicine Clerkship r=0.31 p<0.01
Surgery Clerkship r=0.22 p<0.05
Pediatrics Clerkship r=0.28 p<0.05
Psychiatry Clerkship r=0.15 p<0.08
IIME 2003IIME 2003
• Security and confidentiality
• Pilot Study
Why We Will SucceedWhy We Will Succeed
• International Standard Outcomes
• International Standard Assessment Tools
• International Quality Assessment Process
• International Quality Leaders, Faculty and Assistants working on the IIME project at EVERY school!