WZARD Final Report Version 4 - Minnesota Department of ... · WZARD Evaluation Final Report Page | 4 o Test 3.4 – Compare crash data along I‐94 to a similar corridor without the

Work Zone Accident Reduction Deployment (WZARD) Evaluation

MnDOT Contract #1025980 Work Order No. 1

FINAL DRAFT

WZARD Evaluation Report

June 30, 2017

Prepared For: MnDOT

Prepared by: Kimley‐Horn and Associates

Downtown Detroit Future Transportation System Enhancements

WZARD Evaluation Final Report P a g e | 1

Work Zone Accident Reduction Deployment (WZARD) System Background Minnesota Department of Transportation (MnDOT) led the design and deployment of the Work Zone Accident Reduction Deployment (WZARD) system. It was deployed in January 2012 along eastbound I‐94 between TH 15 in St. Cloud to TH 101 in Rogers; totaling 34 miles. The system was designed such that when a snow plow or maintenance vehicle equipped with Automatic Vehicle Location (AVL) came within a certain pre‐defined geofenced area, a signal would be sent to the central system to determine if a pre‐programmed message was appropriate for dissemination. For example, if an AVL equipped snow plow was active within a certain area, the system would post ‘SNOW PLOW AHEAD USE CAUTION’ on the dynamic message signs (DMS) within that area.

The WZARD System was designed in response to expressed concern from snow plow operators and maintenance personnel for the safety of themselves and other road users along the corridor when plowing or maintenance activities were underway. The primary purpose of this system was to improve safety for snow/ice and other maintenance operations. Goals and objectives for the WZARD System were established by MnDOT when the system was designed and deployed. They were categorized as follows:

Traffic Incident Management

Transportation System Efficiency

Public Communications/Traveler Information Figure 1 provides a map of WZARD System deployment area.

The primary purpose of this system is to improve safety for snow/ice and other maintenance operations. The following goals and objectives were established for the WZARD System by MnDOT:

Traffic Incident Management Improve I‐94 corridor safety during work zone operations Improve safety for traffic incidents and/or traffic enforcement activities Reduce the occurrence of snow plow/vehicle crashes Reduce the occurrence of secondary incidents Transportation System Efficiency Improve traffic safety and mobility Improve travel times along the I‐94 corridor Reduce I‐94 corridor congestion Manage recurrent peak period congestion, including weekend seasonal traffic Reduce vehicle emissions Public Communications/Traveler Information Provide real‐time traveler information along the I‐94 corridor Provide travelers with advance warning of maintenance operations upstream Provide CCTV images to RTMC, District 3 Operations and State Patrol

The goals and objectives for the system largely stayed the same throughout the development process, however, the scope of the project was reduced and cameras were not installed as part of the system. Therefore, the objectives reliant upon CCTV images no longer apply.


WZARD System Evaluation Evaluating the success of a particular ITS deployment is critical to verifying the appropriate use of resources. With the use of benefit‐cost analysis, evaluations, and performance measures, states can best determine the most prudent way to spend limited resources to achieve their current and future goals.

The WZARD System Evaluation was prepared by Kimley‐Horn and Associates with the support from key MnDOT stakeholders. The following Team members provided insight, data, guidance, quality assurance, and support during the WZARD System Evaluation:

Dan Rowe – WZARD System Evaluation Project Manager and Team Leader

Derek Leuer – MnDOT Traffic Safety Engineer

Jakin Koll – MnDOT Meteorologist

Matthew Schleusner – MnDOT Graduate Engineer ‐ Trainee

Michael Schweyen – MnDOT District 6 Traffic Engineer

The Evaluation Team considered existing data sources and developed a general strategy of how the system would be evaluated based on the available resources. The data was obtained and reviewed at a high level and the Evaluation Plan was developed. The Evaluation Plan communicates the intended methodology for assessing the system. The WZARD System was evaluated using available information/data resources to consider if the system effectively met the initial needs of the deployment. Specifically, the WZARD evaluation project team sought to test the following hypotheses:

Hypothesis 1 – The WZARD System activated signs when intended and was used during non‐

winter months.

o Test 1.1 – Cross‐reference system activation logs and weather data to determine if there

appears to be any large snow events in which the system was not activated.

o Test 1.2 – Identify when the WZARD System was used to initiate maintenance messages

(MAINTENANCE VEHICLE AHEAD) during non‐winter months.

Hypothesis 2 – MnDOT snow plow operators and/or maintenance personnel perceived a greater

sense of safety when the WZARD System was activated.

o Test 2 – Interview snow plow operators, maintenance personnel, and engineering staff

that have worked with and without the WZARD System being activated.

Hypothesis 3 – The WZARD System provided measurable safety benefits.

o Test 3.1 – Compare snow plow hit data to determine if a reduction in snow plow hits can

be attributed to the WZARD System.

o Test 3.2 – Compare crash data along the eastbound I‐94 corridor with or without system

deployment to determine if a clear reduction in crashes along the I‐94 eastbound

corridor can be attributed to the WZARD System. It is anticipated that crash data from

2008 through 2011 will represent the ‘without system deployment’ data set and that

crash data from 2012 through 2015 will represent the ‘with system deployment’ data

set.

o Test 3.3 – Compare eastbound and westbound crash data along the I‐94 corridor to

determine if a clear reduction in crashes along the I‐94 eastbound corridor can be

attributed to the WZARD System. It is anticipated that this comparison will consider

crash data during 2012 through 2015 when the WZARD System was deployed.


o Test 3.4 – Compare crash data along I‐94 to a similar corridor without the WZARD System along I‐35 from Forest Lake to Pine City during the same timeframe to determine if the changes in crash rates along the WZARD corridor can be attributed to the WZARD System, or if they are similar to what happened elsewhere. The data is grouped into two timeframes: crash data from 2008 through 2011 will be referred to as the ‘without system deployment’ data set and that crash data from 2012 through 2015 will be referred to as the ‘with system deployment’ data set.

o Test 3.5 – Compare traffic speed data during weather events along the eastbound I‐94

corridor with or without system deployment to determine if a clear reduction in speeds

along the I‐94 eastbound corridor can be attributed to the WZARD System. It is

anticipated that speed data from July 2009 through 2011 will represent the ‘without

system deployment’ data set and that speed data from 2012 through June 2015 will

represent the ‘with system deployment’ data set.

Three hypotheses were tested using available quantitative and/or qualitative data. The precise methodology

and approach for testing each hypothesis depends on the extents, content, and format of the data. The figure

below diagrams the intended evaluation strategy and data that was available for use.

Figure 2: WZARD Evaluation Strategy Diagram


Quantitative and Qualitative Data Quantitative and qualitative data was analyzed to evaluate the effectiveness of the WZARD System. The

following data types were used.

WZARD System Logs The WZARD System was designed to be integrated with MnDOT’s Advanced Traffic Management System (ATMS)

software, Intelligent Roadway Information System (IRIS). The IRIS system logs and archives when each DMS

posts a message. These system logs can be filtered by message, sign, or both. These system logs provide

documentation of when the WZARD System was in use for snow plow and maintenance operations.

Crash Data MnDOT provides public access to crash data through their online Minnesota

Crash Mapping Analysis Tool (MnCMAT). Crash data was retrieved and analyzed

from MnCMAT for the period between 2008‐2015. In addition, MnDOT

provided snow plow crash information, crash reports, and the statewide crash

toolkit information as needed. Snow plow crash information was provided for

2011‐2017. The crash data resources were analyzed and any trends or

measurable results were used to evaluate the effectiveness of the WZARD

System.

Mobility Data

MnDOT collects traffic volume and speed data with Automatic Traffic Recorder (ATR) and Weight‐In‐Motion (WIM) stations that are installed along I‐94 within the project area. MnDOT provided volume and speed data from ATR Station #200 and WIM Station #37 for this analysis. ATR Station #200 is located on the south side of eastbound I‐94 approximately 1.2 miles west of County Road 8 NW in Clearwater.

WIM Station #37 is located north of westbound I‐94 approximately 1.2 miles west of County Road 19/Labeaux Ave NE in Albertville. Volume data in each direction was provided at each recorder location for 2008‐2015. Speed data was provided for the eastbound direction at WIM #37 for 7/15/2009‐6/9/2015. Speed data in both directions was provided for 2013‐2015 at ATR #200 and WIM #37.

Weather Data

Several sources of weather data exist and were considered for use, including: Road Weather Information Systems (RWIS) in Clearwater, MN ROADs data, surrounding airports, and data that is archived for maintenance information. RWIS information is available through the third party, Vaisala website. Archived snow and ice event data for 2011‐2015 was obtained from MnDOT. Weather data from the St. Cloud Regional Airport was obtained via the National Oceanic and Atmospheric Administration (NOAA) Data Online tool. The snow and ice event data and airport weather data were relied upon primarily.

System User Interviews

The WZARD System was designed to address concerns of snow plow operators and maintenance personnel. The system users and stakeholders were interviewed to evaluate the perceived effectiveness of the system as stated in Test 2. System users, i.e. snow plow operators and maintenance personnel, were interviewed as well as system developers to gain a full understanding of how the WZARD System was intended for use versus how it


was ultimately used; their perception of the effectiveness of the system; any lessons learned from development and deployment; and any information that would be helpful to improve the current WZARD System or other deployments was discussed.

The Evaluation Team reached out to system users and stakeholders and asked for their participation. A one page overview of the evaluation consisting of a brief system summary and potential types of questions that would be asked was provided in advance of the interviews (Appendix).

The following stakeholders were interviewed for this evaluation:

May 15, 2017

Domingo Aguilar – MnDOT Transportation Operations Supervisor

Tom Dumont – MnDOT District 3 Traffic Engineer

Mike Kiley – MnDOT Maintenance Supervisor

Dan Muyres – MnDOT Maintenance Vehicle / Snow Plow Operator

May 19, 2017 – Sue Lodahl – MnDOT Maintenance Operations Engineer

May 22, 2017 – Brian Kary – MnDOT Regional Traffic Management Center (RTMC) Director of Operations

May 24, 2017 – Derek Nieveen – Project Engineer at Alliant Engineering, Inc.

June 6, 2017 ‐ Arnie Michalicek (MnDOT Traffic Management System Integrator Specialist)

The stakeholders were asked the following questions:

1. Are you familiar with the WZARD System and the reason it was originally deployed? 2. When did you work within the area of the WZARD System? 3. Did you notice when the WZARD System was activated? 4. Did you notice a difference in driver behavior when the WZARD System was activated? If so, what were

your observations? In general, was driver behavior consistent? 5. Would you recommend the WZARD System in other areas? 6. What would you recommend being done differently with a future WZARD Deployment?

Key information that was obtained through the various interviews. A full summary of each interview can be found in the Appendix.


WZARD System Evaluation Results The WZARD System Evaluation Plan identified the specific hypotheses to be evaluated and tests were defined to communicate the methodology of analysis. The following section provides a summary of data analysis results for each hypothesis and test.

Hypothesis 1) The WZARD System activated signs when intended and was used during non-winter months.

The WZARD System was developed such that signs would be activated when an AVL equipped snow plow or maintenance vehicle enters a pre‐designated geofenced area at a speed less than 45 mph or indicates the snow plow is down or spreader is operating. The AVL sends a message to the system which then posts the appropriate message on the applicable WZARD sign. Hypothesis 1 is focused on determining if the system operated as intended and posted messages during plowing operations. In addition, a review of non‐winter months will provide information regarding whether the WZARD System was utilized beyond snow plow operation times.

Test 1.1) Cross-reference system activation logs and weather data to determine if there appears to be any large snow events in which the system was not activated.

Test 1.1 considers if the WZARD System activated when intended. The following data sources were analyzed to consider system use:

MnDOT Snow and Ice Event Logs – These logs provide documentation of snow and ice event

dates, times, and durations. In addition, when a lane is covered with snow and/or ice and is in

need of plow or maintenance activity (termed ‘lane lost’), it is recorded as well. The snow plow

operators are responsible for completing the logs, and enter the information into the electronic

system manually. There is not currently a set standard for documenting comments or qualifying

events, therefore it is possible that there is a variance in data quality between individual

operators.

NOAA Daily Precipitation and Snowfall Report

WZARD Activation Logs (winter and non‐winter months)

The initial results of this comparison yielded less than expected confidence that the WZARD System was activated when intended. Therefore, further consideration was given to determine if the system was really not activated when intended or if the data sources that were being used for analysis were not as reliable as expected. The following methodology was implemented to analyze system activation:

Compared days with a snow or ice event reported from Snow and Ice Event Logs to days with

WZARD activation reported. Table 1.1.1 provides a summary of when the WZARD System was

activated as expected (EXPECTED FUNCTION) and when the signs were activated but no weather

event was recorded (UNDEFINED FUNCTION). An example of an undefined function could be a

system activation due to a snow plow that was salting on a day when no snow or ice event was

recorded.


Table 1.1.1: Snow and Ice Event Data vs WZARD System Activation

Snow/Ice Event Comparison Number of Days

Snow/Ice Event 179

WZARD System Activated (Winter Only)

184

WZARD Activated During Snow/Ice Event (EXPECTED FUNCTION)

89

WZARD System Activated but no Snow/Ice Event (UNDEFINED FUNCTION)

95

Percent of Time WZARD Activated During a Snow/Ice Event

50%

Compared days with lane lost reported from Snow and Ice Event Logs to days with WZARD

activation reported. Table 1.1.2 provides a summary of when the WZARD System was activated

as expected (EXPECTED FUNCTION) and when the signs were activated but no weather event

was recorded (UNDEFINED FUNCTION). An example of an undefined function could be a system

activation due to a snow plow that was salting on a day when no snow or ice event was

recorded.

Table 1.1.2: Lane Lost vs WZARD System Activation

Lane Lost Event Comparison Number of Days

Lane Lost 118

WZARD System Activated (Winter Only)

184

WZARD Activated During Lane Loss (EXPECTED FUNCTION)

55

WZARD System Activated but no Lane Lost (UNDEFINED FUNCTION)

129

Percent of Time WZARD Activated During a Lane Loss

47%

Compared days with precipitation or snowfall reported from the NOAA data to days with

WZARD activation reported. Evaluated concurrency percentage between snowfall occurrence

and WZARD activation at the following minimum daily snowfall amounts:

o Snowfall > 0.5”

o Snowfall > 1”

o Snowfall > 2”


Table 1.1.3: NOAA Precipitation vs WZARD System Activation

NOAA Snowfall Event Comparison Snowfall Amount Reported

> 0.5" > 1" > 2"

Number of Days Snowfall was Reported

75 52 29

Number of Days Snowfall was Reported and WZARD was Active

41 32 17

Percent of Time WZARD Activated During Snowfall

55% 62% 59%

The above data analysis does not provide confidence that the system was activated when intended based on the data provided. However, when reviewing the results of this analysis, the source of the data should be considered. It is possible that the snow and ice event log data was not thorough or complete. It is also possible that the snow plows within the area were not AVL equipped or did not have AVL turned on. Through discussions with various stakeholders it was mentioned that most of the snow plows were AVL equipped at the time of WZARD System activation. In addition, it is possible that the signs were posting higher priority messages when they would otherwise be posting a WZARD message. Furthermore, it should be noted that several sources have indicated that the WZARD System was turned off on occasion inadvertently during other system maintenance or integration activities. Therefore, it is possible that the WZARD System functioned as intended when all key elements (i.e. system turned on, AVL equipped, no higher priority message posted, etc.) were in place. It was stated in the Stakeholder interviews that the system users perceived that the system was activating as intended.

Test 1.2) Identify when the WZARD System was used to initiate maintenance messages (MAINTENANCE VEHICLE AHEAD) during non-winter months (May through October).

Test 1.2 considers if the WZARD System activated during non‐winter months. WZARD Activation Logs were used to consider system use during non‐winter months. The activation logs were filtered to only include message occurrences with “MAINTENANCE AHEAD USE CAUTION”, “MAINTENANCE VEHICLE AHEAD”, and “MAINTENECE VEHICLE AHEAD USE CAUTION”. A summary of the number of activations during non‐winter and winter months is shown below; this information is provided for the entire length of WZARD System activation (2012 – 2015). The table documents each time an individual sign was activated, and as such, a single snow plow or maintenance vehicle could account for multiple activations as it passes different signs along the corridor.


Table 1.2.1: WZARD System Activation during Non‐Winter Months (May – October)

DMS Message Number of Winter Activations

(Nov ‐ April) Number of Non‐Winter Activations

(May ‐ October)

MAINTENANCE VEHICLE AHEAD USE CAUTION

6,865 1,329

MAINTENANCE VEHICLE AHEAD

0 2

MAINTENANCE AHEAD

USE CAUTION 0 1

Total Maintenance Messages

6,865 1,332

Based on the table above, it was determined that the WZARD System was active during non‐winter months.

Hypothesis 2) MnDOT snow plow operators and/or maintenance personnel perceived a greater sense of safety when the WZARD System was activated.

As stated previously, the impetus of the WZARD System was to address concerns that snow plow operators and maintenance personnel had for their personal and surrounding traveler’s safety while plow or maintenance activity was occurring. The system users and stakeholders were interviewed to evaluate the perceived effectiveness of the system as stated in Test 2.

Test 2) Interview snow plow operators, maintenance personnel, and engineering staff that have worked with and without the WZARD System being activated.

System users, i.e. snow plow operators and maintenance personnel, were interviewed as well as engineering staff to determine if they felt a greater sense of safety with the WZARD System.

Mike Kiley (MnDOT Maintenance Supervisor) and Dan Muyres (MnDOT Maintenance Vehicle / Snow

Plow Operator) provided the most insight into the day‐to‐day function of the system. They both

indicated that they felt that the WZARD System functioned as intended and that the presence of the

signs provided some sense of increased safety. They both stated that they liked the deployment of the

signs for the WZARD System for the system itself and for non‐system uses. However, neither Mr. Kiley

nor Mr. Muyres felt that driver behavior was noticeably affected by the WZARD System.

A full summary of each interview can be found in the Appendix.


Hypothesis 3) The WZARD System provided measurable safety benefits.

The main purpose of the WZARD System is to provide safety benefits for snow plow operators, maintenance personnel, and travelers. The following tests seek to determine if there are any measurable safety benefits that can be attributed to the WZARD System.

Test 3.1 – Compare snow plow hit data to determine if a reduction in snow plow hits can be attributed to the WZARD System.

Data was obtained from MnDOT that documents the date, time, direction, and location of any snow plow and vehicle crash. The snow plow hit data was cross‐referenced with the WZARD System activation logs to determine if any snow plow hits occurred during a WZARD System activation. The NOAA weather data was also cross‐referenced to identify if snowfall was reported on the day of each plow hit to determine if the WZARD System should have been activated.

Table 3.1.1 provides a summary of the snow plow hits that occurred on the I‐94 eastbound lanes within the WZARD System activation area between 2011 and 2017. Snow plow crash data was not available prior to this time.

Table 3.1.1: Plow Hit vs WZARD System Activation Summary

Date Time Direction Location WZARD System Sign

Activated?

1/11/2011 8:30 AM EB I‐94 EB MP 202 No

3/24/2014 8:50 AM EB I‐94 EB / CSAH 75 No

1/8/2015 4:20 PM EB I‐94 MP 178.5 No

1/25/2017 9:10 AM ‐ I‐94 MP 181 No

No snow plow hits occurred when the WZARD System was actively posting messages. Two plow hits occurred during WZARD System deployment (2012‐2015). Of the two plow hits (3/24/2014 and 1/8/2015), snowfall was reported during those days. The WZARD System did not activate on either of those days. The sign activation logs indicate no posted messages on 3/24/2014. The sign activation logs indicate that ‘SLIPPERY CONDITIONS REDUCE YOUR SPEED’ (not a WZARD activated message) was posted on one sign within the WZARD System corridor at the time the plow was hit on 1/8/2015. It is possible that the WZARD System should have activated during two of the plow hits indicated above, but did not. However, it is also possible that the snow plows were not traveling at slow enough speeds to activate the system.

Test 3.2 and 3.3 – Compare crash data along the eastbound I-94 corridor with or without system deployment. Compare eastbound and westbound crash data along the I-94 corridor.

Crash data was obtained from the Minnesota Crash Mapping Analysis Tool (MnCMAT) and analyzed to evaluate if there were any noticeable reductions in crashes during the WZARD System deployment. MnDOT Road Traffic Volume Data (ATR 200) was used to calculate crash rates so that a reasonable comparison could be made.


The crash rate was calculated based on the following formula:

The crash rates were calculated and summarized at different granularities to determine if a noticeable trend was present in either before/after system activation data or eastbound/westbound crashes. Crash rate units are represented in crashes per 100 million vehicle‐miles traveled; 100 million vehicle‐miles traveled were used consistently, such that rates were presented in greater numbers, allowing an easier comparison. Levels of crash analysis included 4‐year ranges (before and after deployment), yearly, winter months only (November‐April), and just days with snowfall reported. Traffic volumes used in the V variable of the crash rate equation were based on the days that were analyzed. The WZARD System was deployed in early 2012. Thus, the data collected in 2008 through 2011 represents the ‘before’ scenario, whereas the data collected in 2012 through 2015 represents during system activation or ‘after.’

The following tables provide a summary of the crash data within the WZARD System activation area before the system was activated (2008 – 2011) and after the system was deployed (2012 – 2015).

Table 3.2.1: Crash Summary Before and During System Deployment

Date Range

Direction Severity Total

Crashes Crash Rate* K A B C N

2008‐2011 Eastbound 2 6 47 108 518 681 66.1

Westbound 4 2 40 96 508 650 63.0

2012‐2015 Eastbound 2 5 41 99 562 709 64.6

Westbound 2 9 42 84 506 643 62.7

*Crash rate is shown in units of crashes per 100 million vehicle‐miles traveled.

The average crash rate for I‐94 eastbound decreased by 1.5 crashes per 100 million vehicle‐miles traveled, a 2.3% reduction. The average crash rate for I‐94 westbound decreased by 0.3 crashes per 100 million vehicle‐miles traveled, a 0.5% reduction. The greater reduction in crash rate in the I‐94 eastbound direction could be attributed to the WZARD System deployment. However, it is difficult to determine that conclusively.

The following tables and charts provide a summary of the crash data within the WZARD System activation area before the system was activated (2008 – 2011) and after the system was deployed (2012 – 2015) in the eastbound and westbound directions. Crash rates are calculated by calendar year (Years) and by winter (Winters), shown in tables and corresponding charts a and b, respectively.


Table 3.2.2a: Crash Summary by Year

Date Range


Crashes Crash Rate K A B C N

2008 Eastbound 0 0 13 29 169 211 83.3

Westbound 1 0 10 35 166 212 85.8

2009 Eastbound 2 2 15 28 123 170 66.5

Westbound 2 0 12 23 118 155 60.0

2010 Eastbound 0 1 11 27 122 161 61.8

Westbound 1 0 8 20 110 139 52.6

2011 Eastbound 0 3 8 24 104 139 53.3

Westbound 0 2 10 18 114 144 54.7

2012 Eastbound 0 0 5 20 113 138 50.1

Westbound 0 2 9 21 118 150 56.7

2013 Eastbound 0 1 9 24 175 209 78.0

Westbound 0 1 15 22 150 188 84.6

2014 Eastbound 0 3 16 30 149 198 71.2

Westbound 1 3 9 23 128 164 63.8

2015 Eastbound 2 1 11 25 125 164 59.5

Westbound 1 3 9 18 110 141 50.1

Each winter is represented by the beginning year, for example, winter 2015 represents November 2015 through April 2016. This was done to clearly show those winters that were more severe. Half years are presented at the beginning and end of each system activation/deactivation period.

0.0

20.0

40.0

60.0

80.0

100.0

2008 2009 2010 2011 2012 2013 2014 2015

CRASH

RATE

(CRASH

ES PER

100 M

ILL VEH

‐MILES TRAVELED

)

CRASH RATE SUMMARY BY YEAR

Eastbound Westbound


Table 3.2.2b: Crash Summary by Winter/Fiscal Year

Date Range



Jan‐Jun 2008

Eastbound 0 0 3 13 64 80 63.4

Westbound 1 0 4 13 66 84 68.2

Jul 2008‐ Jun 2009

Eastbound 2 1 18 30 176 227 88.8

Westbound 1 0 14 35 173 223 86.4

Jul 2009‐ Jun 2010

Eastbound 0 1 10 29 95 135 51.8

Westbound 1 0 7 21 94 123 46.5

Jul 2010‐ Jun 2011

Eastbound 0 3 11 24 134 172 66.0

Westbound 1 0 9 21 136 167 63.5

Jul‐Dec 2011

Eastbound 0 1 5 12 49 67 51.0

Westbound 0 2 6 6 39 53 40.0

Jan‐Jun 2012

Eastbound 0 0 1 11 54 66 48.0

Westbound 0 2 1 7 61 71 53.9

Jul 2012‐ Jun 2013

Eastbound 0 0 8 18 136 162 60.4

Westbound 0 1 16 26 140 183 82.4

Jul 2013‐ Jun 2014

Eastbound 0 1 16 29 187 233 83.8

Westbound 0 1 8 22 127 158 61.5

Jul 2014‐ Jun 2015

Eastbound 0 3 10 32 127 172 62.4

Westbound 2 3 12 23 119 159 56.4

Jul‐Dec 2015

Eastbound 2 1 6 9 58 76 54.7

Westbound 0 2 5 6 59 72 50.7

Although the average crash rate for I‐94 eastbound decreased more than westbound I‐94 during WZARD System activation, a steady trend is not exhibited if the data is considered on an annual basis as shown above in Tables 3.2.2a and 3.2.2b and the associated charts.

0.0

20.0

40.0

60.0

80.0

100.0

J AN ‐ J UN2008

JU L 2008 ‐J UN 2009

JU L 2009 ‐J UN 2010

JU L 2010 ‐J UN 2011

JU L ‐DEC2011

J AN ‐ J UN2012

JU L 2012 ‐J UN 2013

JU L 2013 ‐J UN 2014

JU L 2014 ‐J UN 2015

JU L ‐DEC2015

CRASH

RATE

(CRASH

ES PER

100 M

ILL VEH

‐MILES TRAVELED

)

CRASH RATE SUMMARY BY WINTER

Eastbound Westbound


Table 3.2.3a: Crash Summary During Winter Months (Nov‐Apr) Before and During Deployment (Years)

Date Range



2008 Eastbound 0 0 8 21 132 161 127.5

Westbound 1 0 7 25 122 155 125.9

2009 Eastbound 1 0 10 9 81 101 79.7

Westbound 0 0 9 17 74 100 78.1

2010 Eastbound 0 0 7 11 72 90 69.6

Westbound 0 0 3 12 63 78 59.5

2011 Eastbound 0 2 5 12 70 89 68.9

Westbound 0 1 3 11 72 87 66.7

2012 Eastbound 0 0 3 6 61 70 51.0

Westbound 0 1 3 11 66 81 61.5

2013 Eastbound 0 0 6 16 116 138 103.8

Westbound 0 1 10 16 103 130 118.0

2014 Eastbound 0 2 12 14 95 123 89.2

Westbound 1 1 2 13 89 106 83.2

2015 Eastbound 0 0 3 11 64 78 57.1

Westbound 1 1 4 7 73 86 61.6

0.0

20.0

40.0

60.0

80.0

100.0

120.0

140.0

2008 2009 2010 2011 2012 2013 2014 2015

CRASH

RATE

(CRASH

ES PER

100 M

ILL VEH

‐MILES TRAVELED

)

CRASH SUMMARY DURING WINTER MONTHS (YEARS: NOV‐APR)

Eastbound Westbound


Table 3.2.3b: Crash Summary During Winter Months (Nov‐Apr) Before and During Deployment (Winters)

Date Range



Jan‐Apr 2008

Eastbound 0 0 2 10 56 68 81.0

Westbound 1 0 2 10 52 65 79.4

Nov 2008‐ Apr 2009

Eastbound 1 0 12 18 132 163 128.6

Westbound 0 0 12 25 130 167 130.5

Nov 2009‐ Apr 2010

Eastbound 0 0 6 13 60 79 61.1

Westbound 0 0 4 17 47 68 51.9

Nov 2010‐ Apr 2011

Eastbound 0 2 8 8 85 103 79.7

Westbound 0 0 4 11 90 105 80.5

Nov‐Dec 2011

Eastbound 0 0 2 4 22 28 64.3

Westbound 0 1 0 2 12 15 34.1

Jan‐Apr 2012

Eastbound 0 0 1 5 35 41 44.9

Westbound 0 1 1 5 38 45 51.4

Nov 2012‐ Apr 2013

Eastbound 0 0 5 8 85 98 73.7

Westbound 0 1 7 16 91 115 104.4

Nov 2013‐ Apr 2014

Eastbound 0 0 12 18 130 160 116.1

Westbound 0 0 6 17 94 117 91.8

Nov 2014‐ Apr 2015

Eastbound 0 2 5 14 72 93 68.1

Westbound 2 2 4 9 78 95 68.0

Nov‐Dec 2015

Eastbound 0 0 1 2 14 17 36.9

Westbound 0 0 1 0 30 31 65.8

0.020.040.060.080.0100.0120.0140.0

J AN ‐APR2008

NOV 2008 ‐APR 2009

NOV 2009 ‐APR 2010

NOV 2010 ‐APR 2011

NOV ‐DEC2011

J AN ‐APR2012

NOV 2012 ‐APR 2013

NOV 2013 ‐APR 2014

NOV 2014 ‐APR 2015

NOV ‐DEC2015

CRASH

RATE

(CRASH

ES PER

100 M

ILL VEH

‐MILES TRAVELED

)

CRASH SUMMARY DURING WINTER MONTHS (WINTERS: NOV‐APR)

Eastbound Westbound


It is difficult to determine a clear trend in the crash data within the WZARD System deployment area. However, it is important to note that the following average crash rate decrease in the I‐94 eastbound versus westbound direction before and during system activation:

Table 3.2.4: Average Crash Summary During Winter Months (Nov‐Apr) Before and During Deployment

Date Range 2008‐2011 2012‐2015 % Change

Eastbound 86.4 75.3 ‐13%

Westbound 82.5 81.1 ‐2%

The data was further analyzed to consider only those days during the winter months that had a snow event. Tables 3.2.5a and 3.2.5b and the corresponding charts provide a summary of the results. The crash rates in these tables are higher due to a greater concentration of crashes that occur during snow events.

Table 3.2.5a: Crash Summary During Days with Snow Before and During Deployment (Years)

Date Range



2008 Eastbound 0 0 4 11 95 110 484.3

Westbound 0 0 3 17 77 97 429.8

2009 Eastbound 0 0 5 3 55 63 330.6

Westbound 0 0 7 6 55 68 347.4

2010 Eastbound 0 0 5 4 44 53 373.2

Westbound 0 0 2 8 42 52 349.9

2011 Eastbound 0 1 4 6 40 51 242.3

Westbound 0 0 0 6 50 56 258.4

2012 Eastbound 0 0 0 3 39 42 217.4

Westbound 0 0 3 4 41 48 265.0

2013 Eastbound 0 0 4 12 89 105 323.4

Westbound 0 1 5 12 78 96 344.9

2014 Eastbound 0 0 5 8 53 66 257.7

Westbound 1 0 1 5 49 56 227.9

2015 Eastbound 0 0 2 6 35 43 300.7

Westbound 1 0 2 4 25 32 217.9


0.0

100.0

200.0

300.0

400.0

500.0

600.0

2008 2009 2010 2011 2012 2013 2014 2015

CRASH

RATE

(CRASH

ES PER

100 M

ILL VEH

‐MILES TRAVELED

)CRASH SUMMARY DURING

SNOW DAYS (YEARS)

Eastbound Westbound


Table 3.2.5b: Crash Summary During Days with Snow Before and During Deployment (Winters)

Date Range



Jan‐Jun 2008

Eastbound 0 0 0 6 38 44 303.5

Westbound 0 0 1 8 38 47 325.5

Jul 2008‐ Jun 2009

Eastbound 0 0 8 8 97 113 600.1

Westbound 0 0 8 14 85 107 565.3

Jul 2009‐ Jun 2010

Eastbound 0 0 2 4 34 40 288.1

Westbound 0 0 3 7 29 39 273.1

Jul 2010‐ Jun 2011

Eastbound 0 1 7 5 59 72 277.0

Westbound 0 0 0 7 66 73 267.3

Jul‐Dec 2011

Eastbound 0 0 1 1 6 8 210.5

Westbound 0 0 0 1 6 7 188.7

Jan‐Jun 2012

Eastbound 0 0 0 2 23 25 241.7

Westbound 0 0 1 1 22 24 252.2

Jul 2012‐ Jun 2013

Eastbound 0 0 2 8 62 72 251.9

Westbound 0 1 4 12 71 88 339.0

Jul 2013‐ Jun 2014

Eastbound 0 0 6 11 84 101 370.0

Westbound 0 0 3 8 56 67 281.6

Jul 2014‐ Jun 2015

Eastbound 0 0 2 8 39 49 230.5

Westbound 2 0 2 4 35 43 197.9

Jul‐Dec 2015

Eastbound 0 0 1 0 8 9 213.1

Westbound 0 0 1 0 9 10 237.6

0.0

100.0

200.0

300.0

400.0

500.0

600.0

700.0

J AN ‐ J UN2008

JU L 2008 ‐J UN 2009

JU L 2009 ‐J UN 2010

JU L 2010 ‐J UN 2011

JU L ‐DEC2011

J AN ‐ J UN2012

JU L 2012 ‐J UN 2013

JU L 2013 ‐J UN 2014

JU L 2014 ‐J UN 2015

JU L ‐DEC2015

CRASH

RATE

(CRASH

ES PER

100 M

ILL VEH

‐MILES TRAVELED

)

CRASH SUMMARY DURINGSNOW DAYS (WINTERS)

Eastbound Westbound


Both I‐94 eastbound and westbound exhibit a decrease in average crash rates during snow days. There does not appear to be a correlation in decreased crash rates during snow days with the WZARD System because westbound crash rates decrease more than eastbound. The presence of more mild winters may have contributed to the crash rate reduction. In addition, the previous reduction trend is not seen in the average summary crash rates during only snow days:

Table 3.2.6: Average Crash Summary During Days with Snow Before and During Deployment


Eastbound 357.6 274.8 ‐23%

Westbound 346.4 263.9 ‐24%

Test 3.4 – Compare crash data along a similar corridor along I-35 from Forest Lake to Pine City during the timeframes with and without the WZARD System.

The following tables and charts provide a summary of the crash data within the I‐35 corridor between Forest Lake and Pine City, northbound versus southbound before the system was activated on the I‐94 (2008 – 2011) and after the system was deployed along the I‐94 (2012 – 2015). To be clear, there is not a WZARD System deployment along I‐35 which we are using as a type of control. Interstate 35 between Forest Lake and Pine City was chosen because it had similar characteristics to eastbound I‐94, i.e. interstate, similar AADT, similar proximity to the metropolitan area, and similar weather patterns. Southbound I‐35 was determined to be most comparable with I‐94 eastbound and has been compared more closely in the following tables. Crash rates are calculated by calendar year and by winter. Each winter is represented by the beginning year, for example, winter 2015 represents November 2015 through April 2016. This was done to clearly show those winters that were more severe.

The following tables and charts provide a summary of crash rates along I‐35 during the consistent timeframe of before and during the WZARD System Deployment.

Table 3.4.1: Crash Summary Before and During Deployment along I‐35

Date Range



2008‐2011

Northbound 1 2 36 85 175 299 26.5

Southbound 3 5 57 88 197 350 30.7

2012‐2015

Northbound 1 2 36 85 213 337 29.1

Southbound 6 2 41 100 301 450 38.9

The average crash rate for I‐35 southbound increased by 8.2 crashes per 100 million vehicle‐miles traveled—a 26.7% increase. The average crash rate for I‐35 northbound increased by 2.6 crashes per 100 million vehicle‐miles traveled—a 9.8% increase. This is a striking contrast to the reductions that were calculated along I‐94 eastbound and westbound, which were ‐2.3% and ‐0.5%, respectively. It is difficult to determine the cause of the changes in crash rates along the two facilities. Southbound I‐35 saw a large spike in crashes in 2013 and 2014, and there is no corresponding spike in the other datasets analyzed. However, statistical testing shows no statistically significant changes between the control and the treatment sites.


Table 3.4.2a: Crash Summary by Year along I‐35 by Year

Date Range



2008 Northbound 1 2 12 22 56 93 33.1

Southbound 0 1 16 14 49 80 28.0

2009 Northbound 0 0 9 25 42 76 27.3

Southbound 1 0 10 22 54 87 30.7

2010 Northbound 0 0 8 14 32 54 19.1

Southbound 1 2 13 34 46 96 33.8

2011 Northbound 0 0 7 24 45 76 26.8

Southbound 1 2 18 18 48 87 30.4

2012 Northbound 0 2 14 20 29 65 23.0

Southbound 0 0 8 16 32 56 21.1

2013 Northbound 1 0 8 30 58 97 34.2

Southbound 2 1 10 22 79 114 39.6

2014 Northbound 0 0 7 14 69 90 31.4

Southbound 2 0 15 41 110 168 57.2

2015 Northbound 0 0 7 21 57 85 27.9

Southbound 2 1 8 21 80 112 36.5

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

2008 2009 2010 2011 2012 2013 2014 2015

CRASH

RATE

(CRASH

ES PER

100 M

ILL VEH

‐MILES TRAVELED

)


Northbound Southbound


Table 3.4.2b: Crash Summary along I‐35 by Winter/Fiscal Year

Date Range



Jan‐Jun 2008

Northbound 1 1 4 6 26 38 27.1

Southbound 0 0 9 3 18 30 21.1

Jul 2008‐ Jun 2009

Northbound 0 1 9 27 42 79 28.4

Southbound 0 1 11 17 54 83 29.2

Jul 2009‐ Jun 2010

Northbound 0 0 10 19 45 74 26.2

Southbound 2 1 15 32 50 100 35.2

Jul 2010‐ Jun 2011

Northbound 0 0 8 22 42 72 25.4

Southbound 0 2 16 25 52 95 33.1

Jul‐Dec 2011

Northbound 0 0 5 11 20 36 25.2

Southbound 1 1 6 11 23 42 29.1

Jan‐Jun 2012

Northbound 0 1 6 12 14 33 23.4

Southbound 0 0 4 7 20 31 23.4

Jul 2012‐ Jun 2013

Northbound 1 1 13 22 30 67 23.6

Southbound 1 1 11 16 39 68 23.6

Jul 2013‐ Jun 2014

Northbound 0 0 6 24 78 108 37.7

Southbound 1 0 8 36 118 163 55.5

Jul 2014‐ Jun 2015

Northbound 0 0 5 18 65 88 28.9

Southbound 3 1 16 35 87 142 46.2

Jul‐Dec 2015

Northbound 0 0 6 9 26 41 26.7

Southbound 1 0 2 6 37 46 29.7

0.0

10.0

20.0

30.0

40.0

50.0

60.0

J AN ‐ J UN2008

JU L 2008 ‐J UN 2009

JU L 2009 ‐J UN 2010

JU L 2010 ‐J UN 2011

JU L ‐DEC2011

J AN ‐ J UN2012

JU L 2012 ‐J UN 2013

JU L 2013 ‐J UN 2014

JU L 2014 ‐J UN 2015

JU L ‐DEC2015

CRASH

RATE

(CRASH

ES PER

100 M

ILL VEH

‐MILES TRAVELED

)




The average yearly crash rate for I‐35 southbound is consistently greater than northbound I‐35. There does not appear to be a specific trend exhibited in the southbound direction, however, I‐35 northbound is relatively steady over the entire timeframe. It should be noted that the winter of 2013/2014 was long and severe; thus, the increased crash rate for southbound and northbound directions during this time can be attributed primarily to the weather.

Table 3.4.3a: Crash Summary Before and During Deployment During Winter Months along I‐35 (Years)

Date Range



2008 Northbound 1 2 8 16 36 63 44.9

Southbound 0 0 9 11 32 52 36.6

2009 Northbound 0 0 2 19 17 38 27.5

Southbound 1 0 5 8 30 44 31.3

2010 Northbound 0 0 3 8 22 33 23.6

Southbound 0 1 6 21 31 59 41.9

2011 Northbound 0 0 3 16 27 46 32.7

Southbound 0 1 10 8 27 46 32.4

2012 Northbound 0 0 4 11 20 35 24.8

Southbound 0 0 2 8 19 29 21.9

2013 Northbound 1 0 6 26 41 74 52.6

Southbound 1 0 8 18 59 86 60.2

2014 Northbound 0 0 4 11 43 58 40.8

Southbound 0 0 3 25 81 109 74.9

2015 Northbound 0 0 1 9 32 42 27.8

Southbound 1 0 2 10 36 49 32.2

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

2008 2009 2010 2011 2012 2013 2014 2015

CRASH

RATE

(CRASH

ES PER

100 M

ILL VEH

‐MILES TRAVELED

)

CRASH SUMMARY DURING WINTER MONTHS (CALENDAR YEAR: NOV‐APR)



Table 3.4.3b: Crash Summary Before and During Deployment During Winter Months along I‐35

(Winters)

Date Range



Jan‐Apr 2008

Northbound 1 1 4 6 18 30 32.2

Southbound 0 0 6 3 12 21 22.2

Nov 2008‐ Apr 2009

Northbound 0 1 5 20 25 51 37.0

Southbound 0 0 5 12 36 53 37.7

Nov 2009‐ Apr 2010

Northbound 0 0 3 13 20 36 25.7

Southbound 1 0 9 15 28 53 37.7

Nov 2010‐ Apr 2011

Northbound 0 0 3 15 33 51 36.2

Southbound 0 2 7 16 39 64 45.0

Nov‐Dec 2011

Northbound 0 0 1 5 6 12 25.3

Southbound 0 0 3 2 5 10 20.9

Jan‐Apr 2012

Northbound 0 0 3 9 11 23 24.6

Southbound 0 0 2 5 17 24 27.2

Nov 2012‐ Apr 2013

Northbound 1 0 5 14 18 38 27.0

Southbound 0 0 6 8 23 37 25.9

Nov 2013‐ Apr 2014

Northbound 0 0 5 21 57 83 58.4

Southbound 1 0 5 32 97 135 92.8

Nov 2014‐ Apr 2015

Northbound 0 0 1 11 39 51 33.8

Southbound 1 0 2 13 46 62 40.7

Nov‐Dec 2015

Northbound 0 0 1 2 11 14 27.5

Southbound 0 0 0 3 12 15 29.2

0.0

20.0

40.0

60.0

80.0

100.0

CRASH

RATE

(CRASH

ES PER

100 M

ILL VEH

‐MILES

TRAVELED

)

CRASH SUMMARY DURING WINTER MONTHS (WINTERS: NOV‐APR)



The average winter months crash rates appear to be fairly consistent. The following table presents the average winter months crash rates for the timeframe of before the WZARD System was activated and during deployment.

Table 3.4.4: I‐35 Crash Summary During Winter Months (Nov‐Apr)


Northbound 32.2 36.5 13%

Southbound 35.5 47.3 33%

Similar to what was shown above, I‐35 exhibited an increase in crash rate for southbound and northbound versus the eastbound and westbound decrease that was exhibited along I‐94. I‐35 northbound and southbound increased by 13% and 33%, respectively, whereas, I‐94 eastbound and westbound decreased by ‐13% and ‐2%, respectively.

The data was further analyzed to consider only those days during the winter months that had a snow event. Tables 3.4.5a and 3.4.5b and the following associated charts provide a summary of the results.

Table 3.4.5a: I‐35 Crash Summary During Winter Months with Snow Days (Years)

Date Range



2008 Northbound 1 0 2 6 16 25 92.8

Southbound 0 0 3 6 13 22 78.8

2009 Northbound 0 0 1 5 7 13 60.9

Southbound 0 0 1 2 14 17 76.4

2010 Northbound 0 0 1 4 8 13 70.1

Southbound 0 0 3 12 13 28 148.0

2011 Northbound 0 0 1 7 14 22 76.6

Southbound 0 0 5 3 15 23 77.4

2012 Northbound 0 0 0 2 7 9 45.7

Southbound 0 0 0 2 8 10 50.2

2013 Northbound 0 0 6 14 29 49 157.5

Southbound 1 0 1 7 32 41 130.9

2014 Northbound 0 0 2 5 17 24 95.1

Southbound 0 0 0 14 44 58 223.8

2015 Northbound 0 0 0 2 4 6 31.9

Southbound 0 0 0 2 10 12 67.3


0.0

50.0

100.0

150.0

200.0

250.0

2008 2009 2010 2011 2012 2013 2014 2015

CRASH

RATE

(CRASH

ES PER

100 M

ILL VEH

‐MILES TRAVELED

)CRASH SUMMARY DURING

SNOW DAYS (CALENDAR YEAR)



Table 3.4.5b: I‐35 Crash Summary During Winter Months with Snow Days (Winters)

Date Range



Jan‐Jun 2008

Northbound 1 0 1 0 5 7 39.6

Southbound 0 0 2 1 2 5 27.0

Jul 2008‐ Jun 2009

Northbound 0 0 2 7 12 21 105.7

Southbound 0 0 2 5 18 25 120.2

Jul 2009‐ Jun 2010

Northbound 0 0 1 6 9 16 76.9

Southbound 0 0 3 10 15 28 136.0

Jul 2010‐ Jun 2011

Northbound 0 0 1 9 19 29 94.2

Southbound 0 0 4 7 20 31 95.9

Jul‐Dec 2011

Northbound 0 0 0 0 0 0 0.0

Southbound 0 0 1 0 0 1 15.1

Jan‐Jun 2012

Northbound 0 0 0 1 3 4 30.1

Southbound 0 0 0 1 6 7 52.3

Jul 2012‐ Jun 2013

Northbound 0 0 4 9 8 21 76.6

Southbound 0 0 1 4 18 23 83.9

Jul 2013‐ Jun 2014

Northbound 0 0 3 10 36 49 184.2

Southbound 1 0 0 14 47 62 228.4

Jul 2014‐ Jun 2015

Northbound 0 0 1 2 8 11 51.1

Southbound 0 0 0 4 19 23 107.9

Jul‐Dec 2015

Northbound 0 0 0 1 2 3 49.7

Southbound 0 0 0 2 4 6 104.7

0.0

50.0

100.0

150.0

200.0

250.0

J AN ‐ J UN2008

JU L 2008 ‐J UN 2009

JU L 2009 ‐J UN 2010

JU L 2010 ‐J UN 2011

JU L ‐DEC2011

J AN ‐ J UN2012

JU L 2012 ‐J UN 2013

JU L 2013 ‐J UN 2014

JU L 2014 ‐J UN 2015

JU L ‐DEC2015

CRASH

RATE

(CRASH

ES PER

100 M

ILL VEH

‐MILES TRAVELED

)

CRASH SUMMARY DURINGSNOW DAYS (WINTERS)



Table 3.4.6: I‐35 Crash Rates During Winter Months with Snow Days


Northbound 75.1 82.6 10%

Southbound 95.1 118.1 24%

Similar to what was found above, I‐35 exhibited an increase in crash rate for southbound and northbound versus the eastbound and westbound decrease that was exhibited along I‐94. This trend was further exhibited when considering just the snow days. I‐35 northbound and southbound increased by 10% and 24%, respectively, whereas, I‐94 eastbound and westbound decreased by ‐23% and ‐24%, respectively.

Considering the I‐35 corridor between Forest Lake and Pine City provided a comparison corridor to help determine if the trends exhibited in the crash analysis for I‐94 were consistent elsewhere or if it was possible that the WZARD System was effecting the trends. Because there was not a consistent trend found between the control sites (I‐35 northbound and southbound and I‐94 westbound), a conclusion that the I‐94 eastbound crash rate reduction could be due to the WZARD System cannot be drawn. A statistical analysis is provided in the following section.

Crash Rate Statistical Comparison

Statistical analysis was performed on the I‐94 / I‐35 comparison test to determine the statistical significance of the crash rate reduction that was found along eastbound I‐94 after the WZARD System was deployed. A two‐by‐two Chi‐Square analysis was completed. This analysis compared the crash rates along eastbound I‐94 before and after deployment of the WZARD System with the crash rates along southbound I‐35 during the same time periods. The crash rates used in the Chi‐Square analysis are provided in Table 3.4.7.

Hypothesis: A reduction in the total crash rate is associated with the implementation of the WZARD System from the pre‐deployment to post‐deployment period.

Null‐Hypothesis: There is no reduction in total crash rate associated with the implementation of the WZARD System from the pre‐deployment to post‐deployment period.

Table 3.4.7: Chi‐Square Analysis Matrix

Observed Crash Rate Pre‐Deployment Post‐Deployment

Treated (I‐94 EB) 66.1 64.6

Non‐Treated (I‐35 SB) 34.1 43.2

The reduction is not statistically significant based on a significance level of 0.10. χ2=0.812, ρ=0.3675, degrees of freedom=1. The obtained value of 0.812 exceeds the cutoff of 2.71 corresponding to the 0.10 level. Therefore, ρ>0.10.


This analysis does not support the hypothesis that a reduction in total crash rate is associated with the implementation of the WZARD System from the pre‐deployment to post‐deployment period. Based on a significance level of 0.10, the null hypothesis is not rejected.

Test 3.5 – Compare speed data along the eastbound I-94 corridor with or without system deployment.

Speed data was limited within the WZARD System activation area, therefore the analysis was limited.

Westbound speed data was not available within this section of I‐94, thus a similar analysis to the crash

data analysis was not possible. The following tables provide a summary of the average daily speeds

within the WZARD System activation area along eastbound I‐94 before the system was activated (July

2009 – 2011) and after the system was deployed (2012 – June 2015).

Table 3.5.1: Average Speed Summary Before and During Deployment along Eastbound I‐94 During Snow Days

Date Range Direction Number of Snow Events

Average Volume per Snow Day

Average Speed on Days with Snow

(mph)

Jul 2009 ‐ Dec 2009

Eastbound

15 22,852 64.5

2010 28 19,236 63.2

2011 38 20,106 65.9

2012 32 23,597 67.2

2013 55 21,707 67.7

2014 42 21,604 56.3

Dec 2015 ‐ Jun 2015 17 23,189 44.9

Table 3.5.2: Average Speed Summary Before and During Deployment along Eastbound I‐94 During

Snow Days Date Range 2009‐2011 2012‐2015 % Change

Eastbound 64.5 59.0 ‐9%

The summary above indicates a 9% reduction in average daily speed after the deployment of the WZARD

System. It is difficult to determine the contributing factors of the speed reduction; especially without the

comparison of the westbound direction or other comparative data. It is possible that the reduction of

speeds is at least partially attributed to the WZARD System.


WZARD System Evaluation Summary The WZARD System evaluation consisted of testing three hypotheses:

Hypothesis 1

The WZARD System activated signs when intended and was used during non‐winter months.

The quantitative data analysis does not provide confidence that the system activated when intended.

There was not a strong correlation found with the sign activation logs and either weather data or snow

and ice event log data. However, it should be noted that the data was entered manually and could be

inaccurate. In addition, it is possible that the signs were posting higher priority messages when they

would otherwise be posting a WZARD message. The Stakeholders who were interviewed consistently

expressed confidence in the system and generally stated that the system was activating as intended.

It is possible that the WZARD System functioned as intended when all key elements (i.e. system turned

on, AVL equipped, no higher priority message posted, etc.) were in place. In addition, it was proven that

the system was used during non‐winter months.

Hypothesis 2

MnDOT snow plow operators and/or maintenance personnel perceived a greater sense of safety

when the WZARD System was activated.

MnDOT snow plow operators and maintenance personnel were interviewed as well as system developers to determine if they felt a greater sense of safety with the WZARD System. No system users indicated that they perceived a direct change in driver behavior.

However, those system users who were the most familiar with the system indicated that they felt that

the presence of the signs and ability to post messages (WZARD or non‐WZARD) provided some sense of

increased safety.

Hypothesis 3

The WZARD System provided measurable safety benefits.

The quantitative crash and speed data analysis considered a number of various tests to consider if the WZARD System provided measurable safety benefits. The analysis was completed based on available data sources which were not ideal and included manual entry in some cases.

Even with the limitations, the following key trends were identified:

Eastbound and westbound I‐94 exhibited decreased crash rates after the WZARD System was deployed.

The eastbound direction exhibited greater reductions than the westbound direction when considering the annual rates and winter months (but not statistically significant).

Westbound I‐94 was found to have a slightly greater reduction in crash rate when considering snow days only which could be because of the sample size.

Interstate 35 exhibited increased crash rates during the study timeframe which is not consistent with I‐94.

There was a 9% reduction in average daily speed after the deployment of the WZARD System.


It is possible that a crash rate and speed reduction could be attributed to the WZARD System deployment. However, due to data limitations and small sample size, it is difficult to show that the WZARD System offered measurable safety benefits.

WZARD System Deployment Lessons Learned The WZARD System was developed via the MnDOT Innovation Project Program. It was the first deployment of this system and was developed based on a systems engineering process where the concept of operations and functional requirements were defined prior to system development. As with any new system deployment, challenges are presented and should be learned from. Part of this evaluation process was to identify those lessons learned and document them for future consideration. The history of the WZARD System development and use was communicated throughout Evaluation Team meetings and stakeholder interviews. Several key themes for future consideration were established.

WZARD System Ownership – It was suggested that better roles and responsibilities need to be defined for this deployment. As the system is currently, no specific person is identified to manage and maintain the system. Therefore, since the system was disconnected due to bringing the system in‐house and a server licensing issue in 2015, no one in particular has the ultimate authority to get the system reintegrated and operational.

WZARD System Education – Through discussions with Evaluation Team members and stakeholders, there seemed to be a lack of system education to system users, i.e. snow plow operators were not aware or did not understand how the system worked to be cognizant of improper function or how their actions affected the WZARD System function. Additionally, it was suggested that marketing/education of the system to the travelling public could be beneficial; meaning, that if travelers understood how the system operated, they would be more likely to heed the warnings when the system is activated.

WZARD System Configuration – Due to this system being the first deployment of its kind, system configurations had to be modified to meet the functional requirements of the system. Specifically, the geofenced area surrounding the signs was modified to be able to capture the snow plow at the intended point. Related to this, the latency of the system was a challenge for configuration and geofence definition. There was inherent latency of the WZARD System due to when the polls of the AVL system and Intelligent Roadway Information System (IRIS) were configured. This caused delay with WZARD System message posting, reducing the effectiveness of the system. Therefore, the geofences were broadened to capture the snow plow much in advance of the sign and account for the latency.

WZARD System Support – Although most snow plows and maintenance vehicles within the area of the WZARD System deployment were equipped with AVL, not all were. This situation has improved significantly over time, but should be a consideration in any future deployments.

If the WZARD System is deployed again, the frequency (number of signs per mile) of signs should be reviewed since the number of signs directly influences the capital and operational costs of the system.

Design, test, and develop the evaluation plan, consistent with the system goals, before the deployment of the system and before the system is turned on. As shown in this report, many data items were incomplete and did not give a clear idea of the effectiveness of the system, or if the goals were clearly reached.

A full summary of each interview can be found in the Appendix. Evaluation Team meeting minutes can also be found in the Appendix.


APPENDIX

04/17/2017

For more information visit: mndot.gov

Or contact: Dan Rowe, MnDOT, 651‐234‐7059

[email protected] 1

Work Zone Accident Reduction Deployment (WZARD) Evaluation Background Minnesota Department of Transportation (MnDOT) led the

design and deployment of the Work Zone Accident

Reduction Deployment (WZARD) system. It was deployed in

January 2012 along eastbound I‐94 between TH 15 in St.

Cloud to TH 101 in Rogers; totaling 34 miles. The system

was designed such that when a snow plow or maintenance

vehicle equipped with Automatic Vehicle Location (AVL)

came within a certain pre‐defined geofenced area, a signal

would be sent to the central system to determine if a pre‐

programmed message was appropriate for dissemination.

The WZARD Evaluation will assess the system using available

information/data resources to consider if the system effectively met the initial needs of the deployment.

Specifically, the WZARD evaluation project team will seek to test the following hypotheses:

Hypothesis 1 – The WZARD system activated signs when intended.

Hypothesis 2 – MnDOT snow plow operators and/or maintenance personnel perceived a

greater sense of safety when the WZARD system was activated.

Hypothesis 3 – The WZARD system provided measurable safety benefits.

The tests will utilize available system logs, crash, mobility, weather, and stakeholder interview data to assess the

effectiveness of the WZARD system.

Stakeholder Interview

Thank you for being willing to participate in a Stakeholder interview for the purposes of evaluating the WZARD

system. We anticipate asking you the following type of questions during your Stakeholder Interview:

Are you familiar with the WZARD system and the reason it was originally deployed?

When did you work within the area of the WZARD system?

Did you notice when the WZARD system was activated?

Do you think the WZARD system is activated when intended?

Did you notice a difference in driver behavior when the WZARD system was activated?o If so, what were your observations?o In general, was driver behavior consistent?

Would you recommend the WZARD system in other areas?

What would you recommend to do differently with a future WZARD deployment?

The WZARD system was in use between January 2012 and December 2015. This evaluation will help guide

MnDOT in future similar deployments and provide an understanding of system benefits. Thank you for your

participation in this process.

WZARD System Deployment Area

Deployment Area

Group Discussion

Domingo Aguilar: No, Mr. Aguilar stated that he started his position in 2014 and is not familiar with the system.

Tom Dumont:

Mr. Dumont stated that he was very familiar with the system and explained that the intent of the system was

to create a safer environment for snow plow operators and travelers. He stated that part of the concern

included aggressive driving by truckers and other travelers. He recalled that there were concerns because

drivers were passing snow plows on the shoulder.

Mike Kiley:

Mr. Kiley stated that he is very familiar with the system and has been the maintenance supervisor within the

area since the original deployment. He also shared that he was on the steering committee that guided the

development of the system. Three of the signs within the system are inside of his jurisdiction.

Dan Muyres: Mr. Muyres stated that he was not familiar with the original intent of the system.

Domingo Aguilar: Mr. Aguilar stated that he started his position in 2014.

Tom Dumont: Mr. Dumont stated that he was part of the original development team.

Mike Kiley: Mr. Kiley stated that he was part of the original development team.

Dan Muyres:Mr. Muyres started he began performing maintenance and snow plow driving from St. Cloud to Clearwater 5

years ago, after the system was active.

Domingo Aguilar: Mr. Aguilar was not familiar with the system and is not in a position to regularly notice when it is active.

Tom Dumont: Mr. Dumont does not regularly travel within the area of the system.

Mike Kiley: Mr. Kiley stated that he would estimate that the system worked 90% to 95% of the time.

Dan Muyres: Mr. Muyres stated that he would estimate that the system worked 80% of the time.

Domingo Aguilar: N/A

Tom Dumont:Mr. Dumont noted that the summers have been so light for the past several years that it is difficult to

recall any changes in driver behavior.

Mike Kiley: Mr. Kiley stated that he could not sense any significant change in driver behavior.

Dan Muyres:Mr. Muyres also stated that he could not sense a change in driver behavior and suggested that if a

more specific message was posted may be useful.


Stakeholder Interviews (May 15, 2017)

3. Did you notice when the WZARD system was activated?

The group discussed the function of the system briefly. Mr. Koll who was also in attendance stated that he

would estimate that the system was working approximately 80% of the time.

4. Did you notice a difference in driver behavior when the WZARD system was activated? If so, what were your

observations? In general, was driver behavior consistent?

The stakeholders discussed driver behavior and the use of more specific messages. Mr. Kiley suggested that

more specific messages would not be able to be modified quick enough for movement of the plows, thus

giving potentially false information.

There was some open discussion about the original intent of the system and the aggressive driver behavior.

Driver behavior was a particular problem in the eastbound morning commuting traffic which is why the

system was deployed in the eastbound direction.

Interview Question

1. Are you familiar with the WZARD system and the reason it was originally deployed?

2. When did you work within the area of the WZARD system?

Domingo Aguilar:Mr. Aguilar stated that he liked the system because the system signs could be used to post messages

outside of system messages and he considered that to be useful.

Tom Dumont:

Mr. Dumont stated that he would like to see the results of the evaluation prior to recommending the

WZARD System in other areas. He was concerned that the maintenance of the signs and system would

not out‐weigh the benefit.

Mike Kiley:Mr. Kiley said that he and the snow plow drivers are always in support of more signs and he would

recommend the system for use of the system generated messages and others.

Dan Muyres:Mr. Muyres stated that he would recommend the system because it did provide the snow plow

drivers some piece of mind. However, he would not recommend if it would require large capital cost.

Domingo Aguilar: No specific suggestions.

Tom Dumont: No specific suggestions.

Mike Kiley:Mr. Kiley stated that he preferred the overhead signs and thought future deployments should

consider the use of more overhead signs.

Dan Muyres:Mr. Muyres stated that he would suggest that more specific information be provided to the driver

about the location of the plow ahead.


5. Would you recommend the WZARD system in other areas?

5. What would you recommend to do differently with a future WZARD Deployment?

Sue Lodahl:

Yes, Mrs. Lodahl is generally familiar with the WZARD system and its purpose. She worked in the maintenance division beginning in 2009. She was aware that the system had been shut off

(communications shut down) due to potential legality issues. She informed us that the company that held the patent began charging $40 per month per truck to use the system, so MnDOT

shut the system off due to this new cost. Also, MDSS was moved back to MnDOT to keep it in‐house.

Sue Lodahl: March‐July 2013

Sue Lodahl:Mrs. Lodahl checked the sign when she commuted to/from St. Cloud. She state that the winter was mild and there wasn't much snowfall, so she did not witness activity on the sign, and

therefore could not comment on this question. She did note that she had seen a maintenance message on the signs.

Sue Lodahl: N/A

Sue Lodahl: Due to Mrs. Lodahl's very limited exposure to the system during sign activation, she was not able to comment on this.

Sue Lodahl:Mrs. Lodahl's opinion is that she likes the system, and it should be operational. She believes it should be utilized and that its capabilities could be expanded. She believes the signs should be

utilized for more message functions than they currently are. She thinks it is beneficial that the plow/maintenance vehicle warning signs are beneficial to be shown to the public.

Sue Lodahl:Mrs. Lodahl believes that the system could be more beneficial if it were deployed consistently. She wonders if we can easily implement it to other corridors with signs. She thinks it is

beneficial to display this information to the public.


Stakeholder Interviews (May 19, 2017)



4. Do you think the WZARD system is activated when intended?

Interview Question


5. Did you notice a difference in driver behavior when the WZARD system was activated? If so, what were your observations? In general, was driver behavior consistent?




Brian Kary:Mr. Kary state that he is familiar with the WZARD system and the reason it was originally deployed. He was involved early on in the project. He was also

involved in the project when IRIS became involved with the project.

Brian Kary:Mr. Kary worked with the WZARD system early on. He became more involved in the system once it was deployed. He noted that deployment of the system

involved the RTMC. Mr. Kary was involved with validating the system on the IRIS end.

Brian Kary: Mr. Kary did notice when the system was activated.

Brian Kary: Mr. Kary stated that his role in the project did not involve observing changes in driver behavior. He was involved with overseeing the signs.

Brian Kary: Mr. Kary couldn't recommend one way or another due to his limited role in evaluating the effectiveness of the system.

Brian Kary: Mr. Kary stated that it he would want to see the benefit of the system. He stated that the setup is easy on MnDOT's part.



4. Did you notice a difference in driver behavior when the WZARD system was activated? If so, what were your observations? In general, was driver behavior

consistent?





Stakeholder Interview (May 22, 2017)

Interview Question


Derek Nieveen: Derek understood that the WZARD system was an automated system that warned vehicles of slow‐moving maintenance vehicles on the roadway.

Derek Nieveen:

Derek was involved in the project right away from the beginning, was involved in stakeholder meetings, through con‐ops, and forward. He was involved

through the system engineering process. There was a problem with a latency in messages getting to the signs up to 3 minutes late due to latency with the

information grabbing system.

Derek Nieveen: Wanted snowplow drivers to note if the sign had been activated and they were able to see the messages.

Derek Nieveen: No, testing was done on a clear day. Just tested to ensure the system worked. Did not observe any change in behavior.

Derek Nieveen:Yes, at Iteris they were hoping to deploy this in other areas such as Duluth. The Metro didn't see as much benefit for it. They has spoken with a couple of

people in the Duluth area, but nothing ever came from it.

Derek Nieveen:Removing the latency would be the biggest benefit to the system. It the snowplow could trigger the system directly without needing to go through IRIS it

could function better, but RTMC doesn't like that option since they cannot log the activations.



Stakeholder Interview (May 24, 2017)

Interview Question




consistent?




Arnie Michalicek: Mr. Michalicek stated that he is familiar with the WZARD system and the reason it was originally deployed.

Arnie Michalicek:Mr. Michalicek was involved early on in the project deployment. He was responsible for integrating the system during deployment. He mentioned that

modems were originally deployed to manage the signs and then fiber was installed in the summer of 2011 prior to WZARD System integration.

Arnie Michalicek:Mr. Michalicek said that he completed the operations testing and determined that the signs were activating as intended. Additionally, he stated that he

regularly drives this route and has noticed when the system was active and heads the warning.

Arnie Michalicek:Mr. Michalicek stated that he had not really observe a difference in driver behavior during system activation. However, he said that he personally, drives

more cautiously when the system is activated because he understands how it works.

Arnie Michalicek:Yes, Mr. Michalicek said he would recommend the system elsewhere. He feels the system works as intended and expects that there is benefit in warning

drivers.

Arnie Michalicek:

Mr. Michalicek stated that he would recommend bigger signs that would allow 3 lines of text. Additionally, he would expect the next deployment would go

smoother because the geofencing latency issue has been resolved (or understood). He stated that there was minimal maintenance required over the past 5

years with on 2 repairs needed. Mr. Michalicek also suggested that educating the public about the system could lead to more cautious driving within the

deployment area.



consistent?






Stakeholder Interview (June 6, 2017)

Interview Question


P a g e | 1


Kickoff Meeting Summary

Thursday, March 23, 2017 8:30 AM – 10:00 AM

1. Introductions Mr. Rowe opened the meeting with introductions and participant backgrounds.

2. General Project Overview

a. WZARD System Overview Mr. Rowe continued the discussion and provided a general overview of the WZARD System. He explained the purpose of the system and the current functional challenges. Mr. Koll explained to the project team that it was recently discovered that the reason the system was not currently working was due to the system server having been re‐located. Without access to this server, the appropriate data feeds have not been able to reach the IRIS server to initiate CMS messages.

b. WZARD Evaluation Goals and Objectives Mr. Bourdon briefly discussed the WZARD Evaluation goals and objectives; and provided a high‐level overview of the evaluation project under the current contract. Mr. Bourdon communicated the that the goal of this project is to evaluate the effectiveness of the WZARD deployment. Specifically, the evaluation will seek to determine the following:

Reliability, did the signs activate when intended

Did the signs affect driver behavior

Did the MnDOT snow plow operator(s) sense any change in driver behavior

Name Agency/Department Email

Dan Rowe MnDOT [email protected]

Michael Schweyan MnDOT [email protected]

Matt Schleusner MnDOT [email protected]

Jakin Koll MnDOT [email protected]

Derek Leuer MnDOT [email protected]

Brandon Bourdon Kimley-Horn and Associates [email protected]

Natalie Smusz-Mengelkoch Kimley-Horn and Associates [email protected]

P a g e | 2

Were there any measurable benefits

3. Scope of Work

Mr. Bourdon and Mrs. Smusz‐Mengelkoch discussed the project scope by task.

Task 1 – Project Management: This task includes regular project coordination, progress reporting, and invoicing; PowerPoint Presentation and Project One‐Pager describing evaluation and findings; project meetings:

- Kick‐Off - Evaluation Strategy - Stakeholder Survey/Interviews - Draft Report Review Meeting

Task 2 – Evaluation Plan: The Evaluation Plan memo will define the evaluation approach by identifying the hypothesis to be tested and identifying the measures that will be used to test against the hypothesis. The evaluation plan will also identify other non‐quantitative analysis to be performed to evaluate qualitative issues such as snow plow operator perception. It is envisioned that the Evaluation Plan Memo will be prepared prior to fully discovering all of the available data and will be used to guide how the data will be analyzed.

Task 3 – Data Collection and Analysis: The project team discussed available data at length. The following data components were discussed:

- System Logs – Mr. Rowe stated that Doug Lau would be able to provide system log information. Mr. Rowe suggested that Kimley‐Horn contact Garrett Schreiner with MnDOT who can work with Mr. Lau to provide this information.

- Crash Data – Mr. Leuer stated that he was available to help provide access to crash information through MnCMAT. Mr. Leuer stated that MnCMAT would be the most current information and that if any specific crash information was needed, he would potentially be able to provide crash reports. In addition, Mr. Leuer suggested that the MnDOT claims department could provide information about plow hits. Mr. Leuer stated that he would work with Mr. Rowe to request this information. In addition, Mr. Leuer stated that he could provide statewide crash toolkit information.

- Mobility Data – The project team suggested that information was available via the online traffic mapping application and Automatic Traffic Recorder (ATR) data. Mr. Bourdon stated that he was very familiar with the online data that was available and confident that he would be able to obtain the data we would need. Mr. Rowe suggested contacting Gene Hicks with any mobility data questions.

P a g e | 3

- Weather Data – Mr. Koll stated that potential weather data sources include:

Road Weather Information Systems (RWIS) in Clearwater, MN ROADs data, surrounding airports, or data that is archived for maintenance information. He suggested that the RWIS information would likely be the best source of information and that he would provide a link and directions on how to access the Clearwater RWIS data specifically.

- Stakeholder Interviews – The project team discussed the process of reaching out

to obtain qualitative information from WZARD Stakeholders. Mr. Schweyan offered that he would be able to reach out to Derek Nieveen who was involved with the initial deployment. Mr. Koll stated that he could provide a list of recommended Stakeholders and help with reaching out to those persons when the time was appropriate.

- Miscellaneous Information – The project team discussed other potential sources

of data. Mr. Schweyan stated that the State‐owned software, Business Intelligence (BI), tracks information about lane clearing that may be helpful. He stated that information must be requested by an MnDOT employee. Mr. Koll stated that the AVL information from the snow plows are archived as well. However, he stated that the data is extensive and likely not very helpful. Mr. Rowe offered to help wherever needed.

Task 4 – WZARD Evaluation Report: Mrs. Smusz‐Mengelkoch briefly explained that an Evaluation Report would be prepared which will summarize the evaluation strategy, data analysis, findings, and any potential conclusions. Mr. Schweyan also suggested that a ‘Lessons Learned’ section would be helpful for other districts that may want to implement similar systems. Mrs. Smusz‐Mengelkoch agreed that this information could be helpful and she stated that it would not be difficult to include a brief section explaining observations and findings from the analysis.

4. Schedule

Mr. Bourdon briefly went over the draft schedule. He stated that several of the earliest deadlines are given as a guide and may need to shift slightly but that the project end dates were firm and not expected to be a problem. Mr. Rowe again offered any help obtaining data that MnDOT could provide to help with Kimley‐Horn’s analysis. Mr. Leuer stated that he will be out April 10th through May 12th. He stated that he would share any pertinent information with a co‐worker prior to leaving.

P a g e | 4

5. Next Steps / Other The following items were identified for action:

Kimley‐Horn to request system log information from Garrett Shriner with MnDOT. Complete

Mr. Leuer and/or Mr. Rowe to request snow plow hit information from MnDOT claims department. Complete

Mr. Leuer to provide statewide crash toolkit information. Complete

Mr. Koll to provide a link and directions on how to access the Clearwater RWIS data. Complete

Mr. Schweyan to reach out to Derek Nieveen who was very involved with the initial deployment. Complete

Mr. Koll to provide a list of recommended Stakeholders. Complete

Mr. Koll to help with reaching out to Stakeholders when the time was appropriate.

Kimley‐Horn to provide Draft Evaluation Plan Memorandum March 31, 2017. Complete

Kimley‐Horn to begin data analysis.

Kimley‐Horn to schedule Evaluation Strategy Meeting.

P a g e | 1


Evaluation Plan Meeting Summary

Wednesday, April 12, 2017 10:30 AM – 12:00 PM



Michael Schweyen MnDOT [email protected]


Matthew Schleusner MnDOT [email protected]

Brandon Bourdon Kimley‐Horn brandon.bourdon@kimley‐horn.com

Natalie Smusz‐Mengelkoch Kimley‐Horn natalie.mengelkoch@kimley‐horn.com

Michael Kondziolka Kimley‐Horn michael.kondziolka@kimley‐horn.com

1. Evaluation Plan Overview Mr. Bourdon explained the high‐level purpose of the Evaluation Plan and asked if there were any general comments on the plan from the group. The following comments were discussed:

Project Map – Mr. Rowe stated that Derek Leuer asked that Kimley‐Horn update the project extent figure to provide project termini and make more presentable.

WZARD System use during maintenance activities: - Mr. Rowe asked that the evaluation consider use of the WZARD system for

maintenance operations during the non‐winter months. - It was noted that Mike, Kylie, and Jed used the DMS to communicate

maintenance related messages both manually and using WZARD. - Discussion of whether non‐winter messages were activated by AVL or manually

resulted in conclusion that the message “MAINTENANCE VEHICLE AHEAD” likely was triggered by an AVL vehicle, and other messages likely were manually triggered. Thus, making it possible to determine how the system was activated.

- Mr. Koll said that AVL related triggers may vary seasonally because Air Cards for some vehicle are suspended during non‐winter months.

2. Draft Hypotheses and Tests

Mrs. Smusz‐Mengelkoch began discussion of each hypothesis and associated tests proposed to evaluate the WZARD system.

P a g e | 2

a. Hypothesis 1 – The WZARD system activated signs when intended.

i. Test 1 – Cross‐reference system activation logs and weather data.

Mrs. Smusz‐Mengelkoch noted that this test will be modified to include documenting when the system activated signs during maintenance activities. The study will focus on snow removal operations, so system use for maintenance activities will be to document frequency and utilization only.

Mr. Schweyen mentioned that snow and ice event reports are available from MnDOT. MnDOT keeps detailed records of when maintenance vehicles were deployed during winter months. The group concluded that this would be a better resource to cross‐reference activation logs with to test Hypothesis 1. Mr. Schweyen noted that Dan Flatgard is the MnDOT contact for this data and Mr. Rowe stated that he would contacted him to obtain the data.

b. Hypothesis 2 – MnDOT snow plow operators and/or maintenance personnel

perceived a greater sense of safety when the WZARD system was deployed. i. Test 2 – Interview snow plow operators and maintenance personnel.

No additional input.

c. Hypothesis 3 – The WZARD system provided measurable safety benefits. i. Test 3.1 – Compare snow plow hit data to determine if a reduction in snow plow

hits can be attributed to the WZARD system.


ii. Test 3.2 – Compare crash data along the eastbound I‐94 corridor with or without system deployment.

Mr. Schweyen asked Kimley‐Horn to specify date ranges for “before” and “during” periods in the Evaluation Plan.

iii. Test 3.3 – Compare eastbound and westbound crash data along the I‐94 corridor.

Mr. Schweyen asked if we will be able to provide any statistical analysis to support whether a “clear” reduction in crashes occurred. Mrs. Smusz‐Mengelkoch stated that we will likely not have enough data to show statistical significance, but we will know more and may consider later.

Mr. Schweyen noted that crash frequency varies by year, so crash data should be analyzed only when snow/ice events were recorded.

iv. Test 3.4 – Compare traffic speed data during weather events along the

eastbound I‐94 corridor with or without system deployment to determine if a clear reduction in speeds along the I‐94 eastbound corridor can be attributed to the WZARD system.

P a g e | 3


v. Test 3.5 – Compare eastbound and westbound speed data during weather events along the I‐94 corridor to determine if a clear reduction in speeds along the I‐94 eastbound corridor can be attributed to the WZARD system.


2. Initial Data Review Mr. Kondziolka provided a summary of the initial data review, indicating the source and general availability of the data. a) WZARD System Logs

Mr. Kondziolka stated that Kimley‐Horn has received system activation logs for all DMS events, and specifically for “snow plow ahead” events.

b) Crash Data Mr. Kondziolka stated that Kimley‐Horn has pulled eastbound and westbound crash

data from the MnCMAT from 2008‐2015.

Mr. Kondziolka stated that Kimley‐Horn has received plow hit data from 2010‐2016. Kimley‐Horn will follow up with MnDOT to ask for 2008‐2010 plow hit data.

c) Mobility Data

Mr. Kondziolka stated that Kimley‐Horn is currently in the process of requesting ATR Station #200 data from MnDOT and has placed a call with Mike Merrill.

Mr. Schweyen stated that he may have the data available and will send it to the group if available.

d) Weather Data

Mr. Kondziolka stated that hourly weather data has been retrieved from the National Climate Data Center from 2008‐2015.

As mentioned previously, the group discussed that MnDOT maintenance event data would be better to use rather than interpreting weather data as event data would provide a link to actual snow removal operations.

Kimley‐Horn will use the snow and ice event data rather than hourly weather data once it has been obtained from Dan Flatgard.

3. Stakeholder Outreach Mr. Bourdon provided a summary of the draft interviewee one‐pager that was developed for distribution to Stakeholders to be interviewed. He suggested that Mr. Koll send it out next week and then Kimley‐Horn follow‐up to coordinate interview logistics.

Mr. Koll noted that several of the Stakeholders have already been contacted and informed of upcoming coordination.

Mr. Koll noted that there are 1 or 2 more snow plow operators that will be added to the list (Mr. Knoll to provide contact information).

P a g e | 4

Mr. Schweyen recommended that state patrol also be included in the interviews, and he will provide contact information.

Mr. Rowe noted that snow plow operators initiated the project, so their input will be important.

Mr. Rowe noted that it will be important to inform the plow operators that are participating in interviews that the WZARD system has not been operational since December 2015. Mr. Rowe noted that it would be preferred to interview snow plow operators that worked on this corridor before and during WZARD deployment.

Mr. Schweyen will participate on the call with Derek Nieveen.

It was requested that several additional questions be added to the Stakeholder interview (i.e. how long have you been working on this route and does the system increase how safe operators feel?). A description of WZARD that better clarified the project for those interviewed is to be added. Kimley‐Horn will update and provide to Mr. Koll.

4. Next Steps / Other

The following items were identified for action:

Mr. Rowe to contact Dan Flatgard (MnDOT) to retrieve snow and ice event reports.

Mr. Schweyen to contact state patrol and determine who might be appropriate to interview.

Mr. Schweyen to look into retrieving ATR location #200 information. - Completed.

Mrs. Smusz‐Mengelkoch to send Mr. Koll an updated interviewee one‐pager to be sent with initial email coordination. - Completed.

Mr. Koll to initiate communication with identified interviewees by Friday, April 21st and assist with adding some additional interviewees so Kimley‐Horn can expand the stakeholder list.

Kimley‐Horn to update Draft Evaluation Plan per comments discussed.

Kimley‐Horn to schedule interviews with Stakeholders after Mr. Koll has initiated communication.

Kimley‐Horn to request additional plow crash data.

P a g e | 1


Evaluation Plan Meeting Summary

Wednesday, May 23, 2017 10:00 AM –11:00 AM



Derek Leuer MnDOT [email protected]

Michael Schweyen MnDOT [email protected]


Matthew Schleusner MnDOT [email protected]

Brandon Bourdon Kimley‐Horn brandon.bourdon@kimley‐horn.com

Natalie Smusz‐Mengelkoch Kimley‐Horn natalie.mengelkoch@kimley‐horn.com

Michael Kondziolka Kimley‐Horn michael.kondziolka@kimley‐horn.com

Attachments:

1. Meeting Agenda 2. WZARD Data Analysis Summary 3. Stakeholder Interview Summary

1. Stakeholder Interview Update Mr. Bourdon stated that the following people have been interviewed so far:

Domingo Aguilar

Tom Dumont

Mike Kiley

Dan Meyres

Sue Lodahl

Brian Kary Mr. Bourdon noted that during the interview with Brian Kary, Brian recommended that Susan Porter or Arnie Michalicek would also be good candidates to interview. Mr. Rowe recommended that Arnie should be interviewed, and that Susan should be contacted by phone to determine if she should or had interest in being interviewed. Mr. Bourdon stated that the following people will be contacted and/or interviewed:

Jamie Hukriede

P a g e | 2

Derek Nieveen

Arnie Michalicek

Susan Porter (phone call only) Mrs. Smusz‐Mengelkoch noted that the interviewees generally felt better with the WZARD system in place, but didn’t feel that it was affecting change in driving behavior. Mr. Rowe raised the question of whether the interviewees just like having DMS signs, or if they like the WZARD system? Mrs. Smusz‐Mengelkoch stated that the interviewees generally like the DMS signs, and not specifically the WZARD system. Mr. Schweyen asked if the signs could be used with an expanded geofence to cover a larger area with fewer DMS signs and infrastructure. He also noted that when sign locations are being chosen, locations should be selected to maximize the effectiveness of the sign. Mr. Koll noted that ~650/800 plows are equipped with AVL currently, and therefore in order to get other in place DMS signs to work with the WZARD system should only require programming logic changes.

2. Hypotheses, Tests, and Results Mrs. Smusz‐Mengelkoch began the discussion by noting that Kimley‐Horn has not been able to obtain directional speed data from MnDOT. Mr. Leuer suggested Kimley‐Horn should check if they have access to INRIX or Google historical speed data. Mr. Shweyen added that Streetlight Insight is another source of traffic speed data although it is not likely available for some time.

a. Hypothesis 1 – The WZARD system activated signs when intended. i. Test 1 – Cross‐reference system activation logs and weather data.

Mr. Kondziolka explained to the group the three types of data that were explored/used to perform Test 1.1:

o RWIS Data (insufficient data available—not used) o MnDOT Snow & Ice Event Data

Less confidence in this dataset because it is all entered manually by plow operators (sometime by memory), and therefor may not be 100% accurate

o NOAA Daily Precipitation and Snowfall Data o WZARD Activation Logs

Mr. Kondziolka reviewed the results of the analysis using the MnDOT Snow & Ice Event Data and the NOAA Daily Precipitation and Snowfall Data

Mr. Koll provided some insight as to why the percentages of times that WZARD was activated during an event may be so low:

o Not all plows were equipped with AVL, and the number of plows with it was less during the 2011‐2014 timeframe

P a g e | 3

o The WZARD system was turned off by the RTMC a couple of times

Mr. Koll asked what station the weather data was from o Mr. Kondziolka stated that it was from the St. Cloud Regional Airport

Mr. Koll stated that he would also like to see if we could get weather data from Buffalo or Maple Lake to compare to

b. Hypothesis 2 – MnDOT snow plow operators and/or maintenance personnel

perceived a greater sense of safety when the WZARD system was deployed.

Discussed at beginning of meeting

c. Hypothesis 3 – The WZARD system provided measurable safety benefits. i. Test 3.1 – Compare snow plow hit data to determine if a reduction in snow plow

hits can be attributed to the WZARD system.

Mrs. Smusz‐Mengelkoch discussed Table 3 on page 5 of Attachment 2. She stated that there were only two eastbound plow crashes during the period when WZARD was operational, and the WZARD system was not on during either of those crashes.

o Follow‐up: snowfall was reported on the two days when eastbound crashes occurred while the WZARD system was deployed, so it is possible that the WZARD system should have been active but was not. The WZARD system may not have been activated if the plows were going faster than 45 mph, or if they did not have their plows/spreaders active.

ii. Tests 3.2 & 3.2 – Compare crash data along the eastbound I‐94 corridor with or

without system deployment. Compare eastbound and westbound crash data along the I ‐94 corridor.

Mrs. Smusz‐Mengelkoch reviewed the tables on pages 6‐10 of Attachment 2. She pointed out the following reductions in crash rates for the four years while WZARD was active versus the four years before it was deployed:

o eastbound crash rate decreased by 1.5 crashes per 100 million vehicle‐miles traveled

o westbound crash rate decreased by 0.5 crashes per 100 million vehicle‐miles traveled

Mrs. Smusz‐Mengelkoch pointed out the considerable reduction in eastbound crashes during the winter months in Table 7 of Attachment 2.

Mr. Koll noted that the winter of 2013‐2014 was particularly long and intense, so the spikes in crash activity for that timeframe is to be expected.

It was noted that the crash data is reported by calendar year, and it was requested that the timeframes be shifted to cover full winters rather than calendar years.

P a g e | 4

Mr. Leuer requested that a statistical significance test be performed on the crash data after the timeframes are shifted. He also asked if the analysis should consider different crash types.

Mr. Schweyen noted that one variable that is not represented is the fact that snow plow drivers and snow plows themselves get better each year, which would hopefully lead to improved metrics.

Next Steps / Other

The following additional analysis/requests were identified and Kimley‐Horn said they would review and report back to Dan Rowe which requests they recommended be accommodated after further considering the potential benefit, budget impacts, and schedule impacts:

Mr. Leuer to reach out internal to MnDOT to determine if we can get direction speed data from Jan 2008 – Dec 2016

Kimley‐Horn to look into retrieving weather data for Buffalo or Maple Lake as another basis of comparison for Test 1.1

Kimley‐Horn to look into comparing crash data to I‐35W (between Forest Lake and Pine City)

Kimley‐Horn to look into statistical significance of crash data

Kimley‐Horn to shift reported crash data timeframes from calendar year to winter, such that full, uninterrupted winters are reported rather than calendar years

Kimley‐Horn to complete a draft report and submit it to MnDOT by Friday, June 2nd. o Mr. Bourdon will contact MnDOT if an extension is required

Documents

WZARD Final Report Version 4 - Minnesota Department of ... · WZARD Evaluation Final Report Page | 4 o Test 3.4 – Compare crash data along I‐94 to a similar corridor without the