Certified AviationManagerBehind the Credential

Las Vegas, NV October 10, 2011

Schroeder Measurement Technologies

• Our roots go back to 1985 with the founding of AppliedMeasurement Services

• Presently work with nearly 200 licensing boards and professionalassociations

• Through IQT we have over 300 computer-based testing centersworld wide

• ESS – the only licensed private investigation firm that isexclusively focused on examination-related intellectual propertyloss prevention

With Iso Quality Testing and Examination Security Solutions

And, aviation supports our business

3

Certified Aviation Managers

• Certified Retinal Angiographers - Certified Neuroscience Nurse

• Certified Fire Protection Specialist - Certified Anesthesiologist

• Certified Orthopedic Technologist - Certified Professional Trainer

• Certified Alcohol & Drug Counselor - Certified Emergency Physician

• Certified Pain Management Educator – Certified Hemiodialysis Nurse

• Certified Forester - Certified Orthopedic Surgeon

• Certified Retirement Financial Advisor - Certified Professional Organizer

• Certified Optical Coherence Tomography Specialist

• Certified Nuclear Cardiology Technician - Certified Lymphedema Therapist

• Certified Personal Trainer - Certified Fire Inspector

• Certified Surgical Assistant - Diplomate in Veterinary Ophthalmology

In Good Company

4

Standards for Credentialing Programs

• American National Standards Institute (ANSI)

• National Commission for Certifying Agencies (NCCA)

5

CAM FACT: NBAA is presentlyin the process of seekingcertification through NCCA forthe CAM program.

Process of Developing a CredentialingProgram

• Role Delineation Study

• Examination Design

• Question Development

• Examination Assembly

• Establish Minimum Passing Score

• Determine Reporting Scale

Role Delineation Study• Initial Research into Role

– How are people trained for the role?

– Review of relevant textual materials

– Interviews with role incumbents

• Focus Group or Survey

– Define important elements of knowledge, skill and ability (KSAs)

– Organize KSAs into content areas

– Obtain Subject Matter Expert (SME) input

CAM FACT: The most recent roledelineation study was done in 2008 andused a focus group approach

Examination Design

• Determine How the KSAs will be tested

– Multiple-choice questions

– Performance-oriented scenarios

– Simulation-related

• Associate important KSAs with examination types

• Weight content areas

• All decisions made by SMEs

CAM FACT: The examination has 150 itemsthat are scored spread across 5 domains.The examination also contains 25 un-scoredquestions on which statistics are gathered.

Question Development

• SMEs trained in how to write questions

• Questions written by SMEs

• Questions critiqued by editors knowledgeable in testing

• Questions reviewed and revised by SMEs

• Questions approved for use by SMEs

• Questions associated with test design outline by SMEs

CAM FACT: Two item writing and reviewworkshops are held each year in June andOctober. Prior to the workshops, questionsare written on-line and edited. Items receivefinal approval for use at the workshops.

Examination Assembly

• Questions selected from bank by test developer using thefollowing criteria

– Conformance with content area weighting

– No question contains information that helps answer another question

• Assembled examination reviewed, revised and approved bySMEs

• After approval, the examination goes through a 4 to 6 weekprocess of editing, quality assurance checking and publishing tothe computer-based testing network.

CAM FACT: At least two SMEs checkeach test form for duplicative, out ofdate, and inaccurate content

Establish Minimum Passing Score

• Recognized procedures meeting testing industry standards usedto set minimum passing scores

• SMEs trained in the application of the procedure

• Procedures depend in some way on the judgment of SMEs inconnection with the definition of minimum required competenceand the relationship of test questions to that definition

Determine Reporting Scale

• Scores are reported on a scale

• The scale is a mathematical function of the number correct

• The use of a scale supports equating which accounts fordifficulty differences from examination form to form

CAM FACT: The minimum passingscaled score on the exam is 500.Scores are reported in the range from200 to 800.

Maintaining a Credentialing Program

• Continuing Question Development

• Continuing Form Assembly

• Statistical Equating and Psychometric Assessment

• Periodic Role Delineation Updating

• Periodic Test Design Revision

Continuing Question Development

• SMEs meet from time to time to write questions

• On-line item writing is used to reduce meeting costs

• Questions continue to be reviewed and revised by SMEs

• Questions continue to be approved for use by SMEs

CAM FACT: To be approved for pre-testing, at least 4 SMEs must bepresent.

Continuing Form Assembly

• On a regular basis, new forms are assembled

• The basis may depend upon the passage of time or the numberof candidates to which the form is exposed

CAM FACT: A new form of theexam is release every other year

Psychometric Assessment and StatisticalEquating

• Psychometric properties such as test reliability are assessed

• Statistical analyses of questions is conducted to determinequestion difficulty and discrimination

• Statistical form equating creates a new mathematical functionbetween raw scores and reported scores to eliminate theunfairness of form difficulty differences

CAM FACT: Statistical item analysis isperformed on all of the data availableimmediately prior to the assembly of anew form and immediately after the firstadministration of a new form.

Periodic Role Delineation Updating

• As technology, law and other factors change so do roles

• Every 3 to 5 years, the role delineation study is updated

• New KSAs are identified and added to the role while others areeliminated

• The relative importance of KSAs to competence in the role mayalso change

Periodic Test Design Revision

• As changes occur in the role study, changes in test design areneeded

• Such change may result in alteration of the content and/orcomposition of a multiple choice exam

• Such change may also result in adding or eliminating modes oftesting – such as performance tests or simulation

The NBAA CAM Credential

• NBAA has invested heavily in the CAM program

• NBAA has also ensured that the program meets testing industrystandards for credentialing programs

• NBAA is seeking recognition for the program through NCCA

• CAM credential holders should be very proud of their credential

CAM FACT: Over 86 percent of CAMcertificate holders renew their certification

Help Make CAM Even Better

• Credentials assist employers with hiring decisions

• Credentials provide practitioners with a means of havingcompetence recognized

• CAM credential holders can help by emphasizing the importanceof the credential to decision makers in their companies and totheir peers

Get Involved!

• Become a Subject Matter Expert (SME)

• Write examination questions on-line

• Help make the examination more authentic by reflecting yourunique knowledge and experience

21

CAM FACT: SMEs who author sixacceptable questions receive one of fourpoints required for recertification