Embed Size (px)
Citation preview
© 2012 NBOME
www.nbome.org
© 2012 NBOME
www.nbome.org
OMEL-NBOME UPDATE
January 4-7, 2013 Phoenix, AZ John R. Gimpel, DO, MEd President & CEO
“To protect the public by providing the means to assess competencies for osteopathic
medicine and related health care professions”
© 2012 NBOME
www.nbome.org
OMEL 2013
To protect the public by providing the means to assess competencies for osteopathic medicine
and related healthcare professions
Mission
© 2012 NBOME
www.nbome.org
OMEL 2013
To be the global leader in assessment for osteopathic medicine and related health care
professions
Vision
© 2012 NBOME
www.nbome.org
NBOME Board of Directors 2012
© 2012 NBOME
www.nbome.org
NBOME Update
NBOME Update – March 2012 2013 NBOME Board Member locations in relation to COM locations
NBOME Board Member
COM Location
NBOME Offices
© 2012 NBOME
www.nbome.org
© 2012 NBOME
www.nbome.org
Strategic Plan 2012-2016: Strategic Objectives
Acting within our available resources and consistent with our mission, vision and values, the NBOME will focus on:
Continuum of Competency Assessment
Expand competency assessment across the education, training and
practice continuum
Performance Excellence
Integrate performance excellence in our
products, processes and services
Engaged Workforce
Cultivate the most capable, most qualified and engaged national
faculty and staff
International Growth &
Development
Expand our presence in the international arena
Collaboration
Strengthen collaborative efforts in assessments, research and patient
safety across the continuum of
osteopathic education and practice
OMEL 2013
© 2012 NBOME
www.nbome.org
COMLEX-USA Level 1 (*Ongoing cohort)
Year N
(1st Timer)
Passing Rate
(1st Timer)
Level 1
2006-2007 3,434 88.50%
2007-2008 3,655 90.00%
2008-2009 3,773 89.70%
2009-2010 4,237 89.60%
2010-2011 4,611 87.60%
2011-2012 4,853 91.90%
2012-2013* 4,984 90.93%
© 2012 NBOME
www.nbome.org
COMLEX-USA Level 2-CE (*Ongoing cohort)
Year N
(1st Timer)
Passing Rate
(1st Timer)
Level 2-CE
2005-2006 2,785 87.00%
2006-2007 3,114 88.10%
2007-2008 3,587 90.60%
2008-2009 3,369 92.00%
2009-2010 3,807 82.60%
2010-2011 4,195 86.50%
2011-2012 4,569 86.80%
2012-2013* 4,367 90.38%
© 2012 NBOME
www.nbome.org
COMLEX-USA Level 2-PE (*Ongoing cohort)
Examination Outcome
2004-
05
2005-
06
2006-
07
2007-
08
2008-
09
2009-
10
2010-
11
2011-
12
2012-
13*
N =
2,720
N =
2,856
N =
3,099
N =
3,476
N =
4,353
N =
3,843
N =
4564
N =
4749
N =
909
Pass 96.1 95.3 93.4 93.5 94.9 94.8 96.1 95.0 94.3
Fail 3.9 4.7 6.6 6.5 5.1 5.2 3.9 5.0 5.7
Both Domains 0.1 0.2 0.7 0.7 0.6 0.5 0.3 0.4 0.2
Humanistic Domain 0.7 0.5 1.8 3.5 2.6 2.7 2.3 3.3 4.0
Biomedical/
Biomechanical Domain 3.1 4 4.1 2.4 1.9 2 1.3 1.3 1.5
© 2012 NBOME
www.nbome.org
COMLEX-USA Level 3 (*Ongoing cohort)
Year N
(1st Timer)
Passing Rate
(1st Timer)
Level 3
2005-2006 2,434 88.70%
2006-2007 3,173 87.70%
2007-2009 3,798 91.10%
2009-2010 3,503 91.90%
2010-2011 3,660 91.40%
2011-2012 4,357 95.00%
2012-2013* 2,511 94.19%
© 2012 NBOME
www.nbome.org
Blue Ribbon Panel on Enhancing COMLEX-USA
OMEL 2013
• Research and expert consensus on enhancement of COMLEX-USA to assure that it continues to reflect the evolving practice of osteopathic medicine
• Evolving towards a competency-based schema (published Fundamental Osteopathic Medical Competency Domains document-July 2011); extensive blueprint work being completed in 2012-2013
• Organizations with representatives on the panel: FSMB, AOA, AAOE, OPDA/CMSS, ACGME, AODME, AACOM, ECOP, NBOME
• Blue Ribbon Panel Update and 10 Recommendations posted to NBOME Website March 21, 2012
© 2012 NBOME
www.nbome.org
10 BLUE RIBBON PANEL RECOMMENDATIONS DECEMBER 2011
www.nbome.org
© 2012 NBOME
www.nbome.org
• RECOMMENDATION #1
– The NBOME will adopt a two-decision point model for the COMLEX-USA examination whereby:
• A series of assessments will lead to a decision for entry into supervised practice, i.e., in residency/GME training (Decision Point 1).
• A series of assessments will lead to a decision for entry into unsupervised practice, i.e., unrestricted licensure (Decision Point 2).
– Implementation will begin with assessments for Decision Point 2.
• RECOMMENDATION #2
– All osteopathic competency domains will be assessed at each of the two decision points.
• RECOMMENDATION #3
– At a minimum, cognitive (predominantly multiple choice question-based) and clinical skills examinations will continue to comprise the COMLEX-USA examination at Decision Point 1; cognitive examinations will comprise assessments used at Decision Point 2.
COMLEX-USA Blue Ribbon Panel Update
© 2012 NBOME
www.nbome.org
• RECOMMENDATION #4
– The NBOME will adopt a new COMLEX-USA examination blueprint, to include the following dimensions:
• Patient Presentations (Replaces current Dimension 1 – Clinical Presentation)
• Competency Domains (Replaces current Dimension 2 – Physician Tasks)
• Lifecycle (New Dimension 3)
• RECOMMENDATION #5
– The NBOME will create a new item bank (repository for examination items) to support the enhanced COMLEX-USA examination. Concurrently, item coding procedures will be reviewed to reflect this new blueprint for the series.
• RECOMMENDATION #6
– COMLEX-USA cognitive examinations will continue to provide pass-fail as well as numeric scores, while the clinical skills assessments will provide pass-fail determinations only.
COMLEX-USA Blue Ribbon Panel Update
© 2012 NBOME
www.nbome.org
• RECOMMENDATION #7
– The NBOME will continue to develop and pilot test novel test items and formats to assess an expanded competency subset. These novel item types include: clinical decision-making (CDM)/key features clinical case formats, authentic patient presentations with audiovisual clips, heart and lung sounds, evidenced-based medicine items such as pharmaceutical advertisement items and abstract interpretation items, etc.
• RECOMMENDATION #8
– The NBOME shall provide a display-worthy document (e.g. a certificate) to all candidates who successfully pass all components of COMLEX-USA.
• RECOMMENDATION #9
– The NBOME will explore expanded eligibility criteria and/or a portfolio model for certain competency domains.
• RECOMMENDATION #10
– Documentation of SOAP notes on the COMLEX-USA Level-2 Performance Evaluation will transition from paper-and-pencil written notes to computer keyboard entry by July 2014
COMLEX-USA Blue Ribbon Panel Update
© 2012 NBOME
www.nbome.org
OMEL 2013
• National Faculty Transformation 2013
• Level 2-PE implementing computer keyboard entry 2014
• Novel test items live in Level 1-3
• Clinical Decision-Making (Key Features) Test Items showing promise for Level 3/Decision Point 2
• Structural changes to Level 3 (Decision Point II) likely 2015-2020; yearly updates to AOA, AACOM, AODME, COCA, AAOE, FSMB
• Score reporting enhancements in 2012; further anticipated enhancements to area-level diagnostics for examinees as well as COMs and PDs (outcomes-based: direct evidence of student learning)
Process of Gradual Evolution to Enhanced COMLEX-USA
© 2012 NBOME
www.nbome.org
NBOME Update
• Focus on training, feedback, and item quality for COMLEX-USA and other NBOME Examination National Faculty Members
• AACOM, AODME, AOA leadership and members well represented on the NBOME Board, Committees, and across the workforce (item writers, exam review, standard-setting panels, etc.)
• Pilot studies show enhanced test item quality and performance with approaches that include item-writing workshops held at the NBOME Offices
• Further applies to critical CQI processes such as key validation and review committees
• Recruitment and competitive application processes for limited number of contracted DO/PhD Chairs and Contracted Item-Writers
Transformation of NBOME National Faculty
OMEL 2013
© 2012 NBOME
www.nbome.org
• COMLEX-USA uses “criterion-referenced” methodology
• COMLEX-USA exams are “Pass/Fail” – “Pass” given if you meet or exceed the established passing score
– Passing means having met or exceeded the standard for competency (minimum competency required)
– You cannot retake the test for a higher score if you pass
• Score Reporting (2-digit, 3-digit and percentiles and pass/fail)
• Scores for COMLEX-USA – Mean score – 500
– COMLEX-USA Level 1 passing score – 400
– COMLEX-USA Level 2-CE passing score – 400
– COMLEX-USA Level 2-PE- provides “pass/fail” only
– COMLEX-USA Level 3 passing score – 350
• When are COMLEX-USA results available? – Cognitive Examinations (Level 1, 2-CE, 3): 4-6 weeks after exam
– Level 2-Performance Evaluation: 8-10 weeks after exam
COMLEX-USA Scoring and Reporting
NBOME Update
© 2012 NBOME
www.nbome.org
COMLEX-USA Level 1 (April 2008) 2 digit 3 digit Percentile • 75 400 10 • 78 450 28 • 82 500 51 • 85 550 73 • 89 600 89 • 92 650 96 • 96 700 99 *vary by testing cycle
Sample Conversions* (Calculator on NBOME website)
NBOME Update
© 2012 NBOME
www.nbome.org
Phase 1, Phase 2 and Phase 3 have distinct test forms
(Faculty Edition available)
Visit: www.nbome.org and click on COMSAE in the left navigation
COMSAE: Comprehensive Osteopathic Medical Self-Assessment Examination
NBOME Update – March 2012
© 2012 NBOME
www.nbome.org
NBOME Update
OMEL 2013
COMSAE and COMLEX Score Relationship
© 2012 NBOME
www.nbome.org
COMSAE Phase 1 / COMLEX-USA Level 1 Performance
0
100
200
300
400
500
600
700
800
0 100 200 300 400 500 600 700 800
CO
ML
EX
Sta
nd
ard
Sc
ore
COMSAE Standard Score
N = 2547 r = 0.7
OMEL 2013
© 2012 NBOME
www.nbome.org
NBOME Subject Examinations
OMEL 2013
© 2012 NBOME
www.nbome.org
• 22/26 COMs currently using since launched July 2011
• Seven (7) Core Clinical Disciplines – designed for end of
clerkship/clinical rotation or course evaluations (OPP, Surgery,
Pediatrics, OB-GYN, Psychiatry, Family Medicine, Internal
Medicine): NEW- Emergency Medicine (2014/2015)
• Osteopathically distinctive assessments; Content reflects the
latest development of the subject and consensus “best-practice”
guidelines
• Feature learner-centered objectives and teaching and learning
resources
• On-line adaptability and flexibility; proctored and secure
COMAT Subject Examinations
OMEL2013
© 2012 NBOME
www.nbome.org
NBOME Update
OMEL 2013
• Comprehensive Osteopathic Medical Variable Examination
• A computer-delivered 400 multiple-choice item examination available year round in all states
• To assist state medical board to reassess competency of osteopathic physicians for licensure purposes
• The COMVEX Committee is currently performing a comprehensive review and update for COMVEX
COMVEX
© 2012 NBOME
www.nbome.org
Department for Continuous Professional Development and Innovation
• Erik Langenau, DO, VP
• Dot Horber, PhD, Associate Director
• Luciana Goldstein, MLIS, MSEd, Learning Technologies Specialist
• Melissa Turner, MS, Senior Quality and Patient Safety Officer
• Supported by 85 FTEs and 600 member NBOME National Faculty
© 2012 NBOME
www.nbome.org
Continuous Professional Development (OCC, MOL)
• Osteopathic Performance Assessment and Improvement Module (OPAIM®) – Web-based asynchronous platform
– Formative assessment
– Practice assessment
– Educational components: journal club, video demonstrations, screencasts, etc
– Self reflection
– Individualized Learning Plan
– Pre and post knowledge assessment
– Pre and post performance assessment
– Osteopathically distinct
© 2012 NBOME
www.nbome.org
OPAIMTM
• NBOME’s Goal – To assist specialty boards, colleges and stakeholders to
develop OPAIMTM programs that are convenient, user-friendly, practical and applicable.
– Help to create OPAIMTM programs which can be tailored to individual needs.
• Two OPAIMTM prototypes are in development – Medication Safety Communication
– OMT For Low Back Pain in Pregnancy
– These will be used for demonstration purposes to illustrate how NBOME can assist specialty boards, colleges and other stakeholders to develop OPAIMTM programs suitable to their needs.
© 2012 NBOME
www.nbome.org
Department of Continuous Professional Development and Innovation
© 2012 NBOME
www.nbome.org
Department of Continuous Professional Development and Innovation
© 2012 NBOME
www.nbome.org
OMEL 2013
• OCC
• FSMB’s MOL
• FSMB’s MOL Pilots
• In-Service Examinations
• COMVEX
• Possible Practice-Tailored MCQ Examinations for Post Licensure Evaluation (e.g., Physician Reentry, Nonclinical Physician OCC/MOL)
• COLAs- AOBEM
• Innovative CME Pilots
• Faculty Development
Other NBOME Involvement in CPD for DOs
© 2012 NBOME
www.nbome.org
1. American Academy of Pediatric Dentistry in-service examinations
2. American Osteopathic Board of Emergency Medicine (AOBEM) computer-delivered certification examinations 3. AOBEM computer-delivered recertification examinations 4. American Osteopathic Board of Pediatrics (AOBP) computer-delivered
certification examinations 5. American College of Osteopathic Pediatrics (ACOP) web-delivered in-service examinations
THE CURRENT NBOME CLIENT EXAMINATIONS INCLUDE:
OMEL 2013
© 2012 NBOME
www.nbome.org
THE CURRENT NBOME CLIENT EXAMINATIONS INCLUDE:
6. AOBP computer-delivered recertification examinations 7. American College of Osteopathic Emergency Physicians in-service examinations 8. American College of Osteopathic Family Physicians in-service examinations 9. American College of Osteopathic Internal Medicine (ACOI) web-delivered in-service examinations. 10. American College of Osteopathic Surgeons web-delivered in-service examinations
OMEL 2013
© 2012 NBOME
www.nbome.org
Director of Test Development, Cognitive Testing (CHI)
Administrative Director-Cognitive Testing (CHI)
Consultants- OPAIMS (PHL)
Other National Faculty Positions (Chairs, Item Writers)
www.nbome.org
CURRENT OPEN NBOME POSITIONS
OMEL 2013
© 2012 NBOME
www.nbome.org
Bruce Bates, DO
Senior VP for Cognitive Testing (3/1/13)
Hao Song, PhD
Senior Director for Psychometrics and Research
Department of Cognitive Testing
OMEL 2013
New NBOME Leadership Staff
© 2012 NBOME
www.nbome.org
US Osteopathic Medical Regulatory Summit 2012
• Opportunities
• Threats
• Hope
• Concerns
Era of Unprecedented Change
© 2012 NBOME
www.nbome.org
US Osteopathic Medical Regulatory Summit
Our Case, Our Value, Our Choice, Our Strength… For Our Patients
© 2012 NBOME
www.nbome.org
© 2012 NBOME
www.nbome.org
• Website Handout- “What PDs Should Know About COMLEX-USA…”
• Organization of Program Director Associations/Council of Medical Specialty Societies (ACGME program reps) and AODME added to Blue Ribbon Panel and NBOME Liaison Committee
• NBOME Annual Report Sent to All Program Directors (AOA and ACGME programs)
• Presentations and publications with CORD (EM), ACP (IM), ACGME New PD Workshop (2012), etc.
• Exhibit booth at AAMC Annual meeting
• ERAS score reporting features interpretation of COMLEX-USA scores
• Numerous other ongoing initiatives
Initiatives to Reach Residency Program Directors
OMEL 2013
© 2012 NBOME
www.nbome.org
• NRMP Program Director Survey 2012 Shows 77 percent of ACGME Residency Program Directors Use COMLEX-USA for DOs…
Misconceptions Common!
© 2012 NBOME
www.nbome.org
AUDIENCE RESPONSE POLLING- FROM USA OSTEOPATHIC MEDICAL REGULATION SUMMIT 2012
© 2012 NBOME
www.nbome.org
Audience Response Polling Statements
1 2 3 4 5
83%
10%
0%3%3%
Statement 1 Because of the substantial contributions of osteopathic medicine to healthcare and the entire house of medicine in America over the last 138 years, a distinctive osteopathic medical profession is valuable to the health of the American public.
1. Strongly Agree
2. Agree
3. Neutral/No Opinion
4. Disagree
5. Strongly Disagree
© 2012 NBOME
www.nbome.org
Audience Response Polling Statements
1 2 3 4 5
90%
3%0%
7%
0%
Statement 2 Universally recognized and distinctive
osteopathic medical school education and
accreditation (COMs and COCA),
osteopathic specialty board
certification/continuous certification
(AOA-BOS/OCC), and osteopathic
competency assessment and testing for
licensure (COMLEX-USA) are critical to
maintaining a separate and distinctive
osteopathic medical profession.
1. Strongly Agree
2. Agree
3. Neutral/No Opinion
4. Disagree
5. Strongly Disagree
© 2012 NBOME
www.nbome.org
Audience Response Polling Statements
1 2 3 4 5
67%
20%
0%
10%
3%
Statement 4 Patients are best served by seeking care from osteopathic physicians who have qualified by virtue of passing graduation requirements and competency assessments designed for the practice of osteopathic medicine (e.g., COMLEX-USA, osteopathic specialty board examinations and training requirements, OCC) and validated for that distinct purpose.
1. Strongly Agree
2. Agree
3. Neutral/No Opinion
4. Disagree
5. Strongly Disagree
© 2012 NBOME
www.nbome.org
THANK YOU!
The NBOME’s mission is to protect the public
by providing the means to assess
competencies for osteopathic medicine and
related health care professions.
