1

Development and evaluation of a competence based exam for prospective driving instructors Erik Roelofs, Marieke van Onna, Jan Vissers Introduction A growing consensus among driver training and road safety researchers is that driver training should place greater emphasis on higher-order, cognitive and motivational functions underlying driving behavior (Mayhew and Simpson, 2002; Hatakka et al., 2002). This changed conception of driver training has been laid down in the Goals for Driver Education matrix (Hatakka et al., 2002). Recent research seems to support this consensus (Isner, Starkey, & Shepard, 2011; Beanland et al., 2013). Innovative training initiatives appear to counteract overconfidence and address motivational factors such as driving anger, sensation seeking, and boredom (e.g. Isner et al., 2009). Parallel to the doubts raised about the quality of driver training, the quality of driver instructor preparation programs is criticized. The MERIT review study (Bartl et al., 2005) showed that huge variations existed in quality of driver instructor education throughout Europe. The contents did not cover higher order skills. Most programs relied on teacher-focused approaches, which seem to fall short in developing higher order skills. In many European countries the quality of the education of driving instructors is regulated by means of the instructor exam. One may view this as a problem, but on the other side, this also offers opportunities for improvement. A valid and reliable exam that only allows proficient prospective instructors to enter the profession, may have a positive backwash effect on driver instructor education programs, as in other fields of education: teachers teach and students learn what will be tested (Madaus, 1988). In the Netherlands, as part of a new law on driving education in 2003, outcome standards for prospective driving instructors have been formulated. A new two stage competence-based examination (CBE) was put into practice in 2009. This paper focuses on the reliability and validity of this new exam. Material & Methods The various parts of the CBE were designed using Mislevy’s framework for Evidence Based assessment (Mislevy, & Haertel, 2006). Stage 1 of the CBE consists of three theoretical tests which address cognitive decision making skills during driving, lesson preparation, and instruction and coaching. Stage 2 comprises a half a year internship in a certified driving school. This internship is concluded with an in-car performance assessment for instructional proficiency. Assessment data from 4741 prospective learner drivers were used, collected between January 1st 2010 and October 1st 2012. IRT models are used for the scaling of all assessment elements and for equating test versions across the data collection period. Correlational studies are employed to estimate relations among the cognitive decision making skills. The predictive validity of these skills for in-car instructional performance is evaluated. Results and conclusions Results on reliability and validity will be reported during the Driver Behaviour and Training conference. Theoretical and practical implications will be discussed.