Development, Implementation, and Tracking of Preventative

Report Number: KTC-21-04/SPR19-568-1F

DOI: https://doi.org/10.13023/ktc.rr.2021.04

Development, Implementation, andTracking of Preventative Safety Metrics

The Kentucky Transportation Center is committed to a policy of providing equal opportunities for al persons in recruitment, appointment, promotion, payment, training, and other employment and education practices without regard for economic, or social status and will not discriminate on the basis of race, color, ethnic origin, national origin, creed, religion, political belief, sex, sexual orientation, marital status or age.

Kentucky Transportation Center College of Engineering, University of Kentucky, Lexington, Kentucky

in cooperation withKentucky Transportation Cabinet

Commonwealth of Kentucky




© 2021 University of Kentucky, Kentucky Transportation Center Information may no tbe used, reproduced, or republished without KTC’s written consent.

Kentucky Transportation Center • University of Kentucky176 Raymond Building • Lexington, KY 40506 • 859.257.6898 • www.ktc.uky.edu










© 2018 University of Kentucky, Kentucky Transportation Center

Information may not be used, reproduced, or republished without KTC’s written consent.

Kentucky Transportation Center • University of Kentucky

176 Raymond Building • Lexington KY 40506 • 859.257.6898 • www.ktc.uky.eduKentuckyKENTUCKY

Transporation Center

Kentucky Transportation CenterCollege of Engineering, University of Kentucky Lexington, Kentucky



The Kentucky Transportation Center is committed to a policy of providing equal opportunities for all persons in recruitment, appointment, promotion, payment, training, and other employment and education practices without regard for economic or social status and will not discriminate on the basis of race, color, ethnic origin, national origin, creed, religion, political belief, sex, sexual orientation, marital status, or age.





































Research Report KTC-21-04/SPR19-568-1F

Development, Implementation, and Tracking of Preventative Safety Metrics

Zamaan Al-Shabbani, Ph.D.

Research Engineer

Ashtarout Ammar Research Associate

Hala Nassereddine, Ph.D.

Assistant Professor

and

Gabriel B. Dadi, Ph.D, P.E. Program Manager and Associate Professor

Kentucky Transportation Center

College of Engineering University of Kentucky Lexington, Kentucky

In Cooperation With Kentucky Transportation Cabinet


The contents of this report reflect the views of the authors, who are responsible for the facts and accuracy of the data presented herein. The contents do not necessarily reflect the official views or policies of the University of Kentucky, the Kentucky Transportation Center, the Kentucky Transportation Cabinet, the United States Department of Transportation, or the Federal Highway Administration. This report does not constitute a standard, specification, or regulation. The inclusion of manufacturer names or trade names is for identification purposes and should not be considered an endorsement.

February 2021

1. Report No. KTC-21-04/SPR19-568-1F

2. Government Accession No.

3. Recipient’s Catalog No

4. Title and Subtitle Development, Implementation, and Tracking of Preventative Safety Metrics

5. Report Date February 2021 6. Performing Organization Code

7. Author(s): Zamaan Al-Shabbani, Ashtarout Ammar, Hala Nasssereddine, Gabriel B. Dadi

8. Performing Organization Report No. KTC-21-04/SPR19-568-1F

9. Performing Organization Name and Address Kentucky Transportation Center College of Engineering University of Kentucky Lexington, KY 40506-0281

10. Work Unit No. (TRAIS)

11. Contract or Grant No. SPR 19-568

12. Sponsoring Agency Name and Address Kentucky Transportation Cabinet State Office Building Frankfort, KY 40622

13. Type of Report and Period Covered

14. Sponsoring Agency Code

15. Supplementary Notes Prepared in cooperation with the Kentucky Transportation Cabinet

16. Abstract What gets measured, gets improved. With respect to the safety and health of Kentucky Transportation Cabinet (KYTC) employees, the primary metric used has been the OSHA recordable incident rate. This incident rate measures how often a Cabinet employee sustains an injury that demands more than basic first aid. This metric is important for understanding injury frequencies, but it does not assist with management of the safety, health, and overall well-being of KYTC personnel. Based on a review of leading safety indicators adopted by various industries, this study devised a comprehensive list of safety metrics the Cabinet will benefit from tracking. Metrics were evaluated, organized, weighted, and compiled into a three-tier scorecard that is used to assess performance at KYTC’s district, area, and executive levels. Five major dimensions of an effective safety program were identified: (1) management leadership and commitment, (2) employee engagement, (3) training and competence, (4) hazard identification and control, and (5) evaluation and improvement. Surveys of KYTC districts found that all metrics performed robustly, while stakeholders at executive levels usually assigned lower scores to the five dimensions. Employee engagement had the lowest score. The Cabinet will benefit from seeking out more opportunities to involve employees in the agency’s safety program. Equally, the study reiterates the value of gaining management buy-in, support, and leadership when working to eliminate incidents and injuries.

17. Key Words employee safety, safety metrics, leading indicators, maintenance worker safety

18. Distribution Statement Unlimited with approval of the Kentucky Transportation Cabinet

19. Security Classification (report) Unclassified

20. Security Classification (this page) Unclassified

21. No. of Pages 29

19. Security Classification (report)

KTC Research Report Development, Implementation, and Tracking of Preventative Safety Metrics

Table of Contents

Executive Summary ....................................................................................................................................................... 1 Chapter 1 Background and Scope of Work ................................................................................................................... 2

1.1 Introduction ........................................................................................................................................................ 2 1.2 Problem Statement ............................................................................................................................................. 2 1.3 Objectives ........................................................................................................................................................... 3

Chapter 2 Literature Review ......................................................................................................................................... 4 2.1 Traditional Safety Performance Indicators ......................................................................................................... 4 2.2 Definition of Leading Indicators .......................................................................................................................... 5 2.3 Categorization and Selection of Leading Indicators ............................................................................................ 5

Chapter 3 Methodology ................................................................................................................................................ 7 3.1 Phase I: Identification and Selection of Main Dimensions .................................................................................. 7 3.3 Phase II: Selection of Leading Indicators ........................................................................................................... 11 3.4 Phase III: Design and Development of the Scorecard ....................................................................................... 12 3.5 Scoring Indicator Achievement ......................................................................................................................... 12 3.6 Scorecard Levels ................................................................................................................................................ 13

Chapter 4 Results and Discussion ............................................................................................................................... 14 4.1 Weighting Results of the Main Dimensions ...................................................................................................... 14 4.2 Final Scorecard Design ...................................................................................................................................... 14 4.3 Results of Pilot Study ........................................................................................................................................ 16

Chapter 5 Conclusion .................................................................................................................................................. 19 Chapter 6 References .................................................................................................................................................. 20 Appendix A: District Scorecard Questions .................................................................................................................. 23

KTC Research Report Development, Implementation, and Tracking of Preventative Safety Metrics

List of Figures

Figure 3.1 Structure of Pair Wise comparison (Shaded boxes were Completed by Survey Respondents) ................. 10 Figure 4.1 Main User Interface of Scorecard .............................................................................................................. 15 Figure 4.2 A Sample of the District Results Sheet ....................................................................................................... 16 Figure 4.3 Pilot Implementation Results for District, Area, and Program Level ......................................................... 18 Figure 4.4 Comparison of the Pilot Trial Results with the 2019 TRIR of Districts ....................................................... 18

List of Tables

Table 3.1 Dimensions of Safety Program ...................................................................................................................... 7 Table 3.2 Relative Importance Comparison Scale Used in Survey .............................................................................. 10 Table 3.3 Sample of Survey Questions ....................................................................................................................... 10 Table 3.4 Sample of Selected Leading Indicators ...................................................................................................... 12 Table 4.1 Relative Importance / Weight of Main Dimensions .................................................................................... 14 Table 4.2 Number of Participants in the Pilot Implementation .................................................................................. 16 Table 4.3 Average Score of Main Metrics on a District Level ..................................................................................... 17

KTC Research Report Development, Implementation, and Tracking of Preventative Safety Metrics 1

Executive Summary

Professional codes of ethics often share the same first principle — do no harm. The commitment to do no harm is firmly embedded within the Kentucky Transportation Cabinet’s (KYTC) mission. KYTC’s Employee Safety and Health branch plays an integral role in harm prevention and works to understand issues in a dynamic work environment that can lead to incidents and injuries. Without tracking and measuring safety practices, efforts to improve safety can be neither targeted nor effective. Based on leading safety indicators adopted by various industries, this study devised a comprehensive list of safety metrics the Cabinet could benefit from tracking. Metrics were evaluated, organized, weighted, and compiled into a three-tier scorecard that assesses performance at KYTC’s district, area, and executive levels. A Cabinet-wide pilot study gathered data on metric performance.

Five major dimensions of an effective safety program were identified during efforts to fine tune the safety metrics scorecard — (1) management leadership and commitment, (2) employee engagement, (3) training and competence, (4) hazard identification and control, and (5) evaluation and improvement. Conversations with safety experts internal and external to KYTC revealed that management leadership and commitment is the most impactful dimension for improving safety, closely followed by employee engagement. Evaluation and improvement was far less impactful than the other four dimensions. Findings reiterate the value of gaining management buy-in, support, and leadership when striving to eliminate incidents and injuries. The safety metrics scorecard pilot study yielded data from eight KYTC districts, three area administrators, and two executive level responses (Figure E1). In district surveys, all dimensions garnered high scores, indicating the metrics performed strongly. However, stakeholders at executive levels usually assigned lower scores to the five dimensions. Employee engagement had the lowest score. These findings highlight the need to seek out more opportunities to involve employees in the agency’s safety program. Examples of engagement include formal, active, and empowered safety committees; inviting employees to help write safety policies and procedures; and involving staff in hazard control practices like delivering toolbox talks, job hazard analyses (JHAs), or safety incident reviews.

Figure E1 Pilot Implementation Results for District, Area, and Program Level

87.63%

64.75%

83.00%77.00%

81.00%

80.00%

71.33%

53.33%58.67%

30.00% 43.00% 62.50%

89.50%

71.00%

50.00%

ManagementLeadership

EmployeesEngagement

Training &Competence

Hazards Identification& Controls

Evaluation &Improvement

DistrictAverage (%)

AreaAverage(%)

CabinetAverage (%)

Total


Chapter 1 Background and Scope of Work 1.1 Introduction Significant efforts have been dedicated to improving the safety performance in the construction industry. Yet, the industry ranks at the top of the list when compared to other industries with respect to safety performance. Multiple reasons are driving the issue including high exposure to hazards, low investment in safety, poor safety culture, and many other causes. One of the main issues that has been discussed in the literature is the reactive nature of safety management in the industry. The traditional methods of measuring safety performance and responding to safety incidents have been criticized for being reactive and less predictive of the safety performance. Metrics such as the Occupational Safety and Health Administration (OSHA) recordable injury rate (RIR) measure of how often a worker is harmed beyond basic first aid including days away, restricted work. Such metrics have been widely used by OSHA, insurance companies, and construction contractors. Typically, those measures are examined over an extended period to identify trends and observe any improvement in safety performance. While they provide excellent insight about the history of safety performance, such measures have been criticized for reflecting the failure of safety systems rather than providing an indication of the failure prior to its occurrence. The highway construction and maintenance sector is no exception. Traditional safety performance measures are still dominant performance measures used by State DOTs. The nature of work hazards, the close proximity to speeding traffic, the movement of heavy equipment and significant amounts of materials, and the extreme working conditions of highway maintenance operations expose DOTs highway maintenance employees to significant risk (Al-Shabbani et al., 2018). Yet, the state DOTs use reactive safety metrics incapable of predicting safety potential incidents in advance. Although significant resources are dedicated every year to improving worker safety in the highway construction and maintenance sector, minimal resources have been invested in adopting proactive measures. As the demand for maintenance work and maintenance workers increases, the need for predictive safety measures becomes critical to improve the safety performance in this sector. While proactive performance measures have been extensively investigated and proven to be effective in the construction industry, minimal efforts have made to examine the use of such measures in the highway maintenance sector. As part of the Kentucky Transportation Cabinet’s (KYTC) efforts to improve employee safety, researchers at the Kentucky Transportation Center (KTC) were asked to develop a proactive safety scorecard to evaluate safety performance of KYTC highway maintenance crews. The scorecard was designed to evaluate safety performance based on five main metrics including management commitment, employee engagement, training and competence, hazards identification and prevention, and evaluation and improvement. The intention is to predict safety performance of maintenance crews at three different level including district level, area level and a program level. The scorecard incorporates multiple leading indicators to assess the performance within each of the five main metrics at every level with the ultimate goal of predicting safety performance of highway maintenance crews and indicating any improvement areas.

1.2 Problem Statement What gets measured, gets improved. In the example of the safety and health for KYTC employees, the primary measure has been the OSHA recordable incident rate. This is a measure of how often a KYTC employee is harmed beyond basic first aid. While this metric is important to understand the frequency of injuries, it does not assist in managing the safety, health, and overall well-being of KYTC personnel. Metrics that assist in indicating a higher likelihood of an injury occurring would be more beneficial to improving safety and health. For example, safety leading indicators answer questions such as “what percentage of employees participate in Near Miss/Good Catch programs?”, “how frequently are job safety analyses performed?”, or “what percentage of new to the site or new to the trade workers are paired with experienced workers?”. Such leading indicators of safety are prevalent in the industrial and commercial sectors of construction and maintenance; however, few exist for KYTC personnel. In addition, the few that are available are not well implemented and tracked.


1.3 Objectives 1. Identify leading safety indicators that are applicable to KYTC. 2. Implement measurement protocols to assess the performance of leading safety indicators 3. Prepare guidance on tracking and interpreting leading indicators as well as suggest remedial actions.


Chapter 2 Literature Review

2.1 Traditional Safety Performance Indicators Traditionally, safety performance in the Kentucky Transportation Cabinet has been measured by metrics such as the Occupational Safety and Health Administration (OSHA) recordable injury rate (RIR); This is a measure of how often a KYTC employee is harmed beyond basic first aid including days away, restricted work, or transfer (DART) injury rate. Such metrics have been widely used by OSHA, insurance companies, and contractors involved in the construction industry. Typically, those measures are examined over an extended period to identify trends and observe any improvement in safety performance. While they are widely used in the construction industry, such measures have been criticized by researchers and practitioners for reflecting the failure of safety systems rather than providing an indication of the failure prior to its occurrence. Therefore, they are known and classified as lagging indicator. Toellner (2001) defines lagging indicators as measurements that are associated with the consequences of an accident. Lagging indicators or lagging measurements provide data after the occurrence of an accident but they do not provide any prediction of safety performance in the future. Several researchers argued that the limitations in using lagging indicators as a measurement of safety performance lean on their deficiency in providing enough data about the origins of accidents (Hinze et al., 2013). Even though they can reflect the level of safety performance and provide a perception of the safety objectives that should be established, they fail in guiding how to achieve those objectives and how to effectively avoid future accidents (Grabowski et al., 2007; Sgourou et al., 2010). In addition, focusing on after the fact-based safety measurement may communicate an unintended message that safety is less important (Mengolini & Debarberis, 2008). Moreover, lagging indicators emphasize the negative side of safety by measuring the presence or absence of accidents instead of emphasizing how safety can be achieved (Guo & Yiu, 2013). Due to the limitations of the lagging indicators, a new approach was required to measure safety performance. Safety management systems (SMS) were developed and implemented as a proactive alternative to measure safety performance. A safety management system is a system consisting of safety policies and objectives, standard targets, planning and organization of work, monitoring and feedback, corrective actions, review and continual improvement (Choudhry et al., 2007). As such, auditing also appeared as a tool for measuring safety performance (Guo & Yiu, 2013). Audits typically determine whether the organization is compliant with safety standards, such as its policies and procedures, applicable legislation and regulations, or other external standards. The quantitative results of audits are often used by organizations as measures to evaluate safety performance against standardized safety management systems (Robson et al., 2012). SMS and auditing have some shortcomings when used as safety performance measures. In auditing, there exist variations and inconsistencies in practice between actual auditing and the international standards on management system auditing guidance (ISO 19011) (Robson et al., 2012). Such discrepancies could have direct implications on the reliability and validity of audit results used in decision-making. For SMS, the concern is similar but not associated with validity or reliability of results. It has more to do with the completeness of the systems in practice. Although safety policies and processes are highly emphasized in SMS, human elements and cultural factors are under accentuated (Choudhry et al., 2007; Wachter & Yorio, 2014). The structural framework of an SMS is robust and rests on the following beliefs (Howell et al., 2002):

1. Developed rules and procedures, which if followed, should keep workers safe. 2. Workers failure follow the developed rules results in incidents. 3. Motivating workers and training them to follow the rules reduce incidents.

However, SMS that are implemented in the construction industry are incomplete (Costella et al., 2009). In addition, the above-mentioned beliefs lead to two essential problems (1) incomplete risk profile and (2) oversimplification of safety phenomena. With such existing problems, safety indicators are unable to provide a true state of safety, which may result in taking ineffective decisions and actions (Guo & Yiu, 2013). In addition, a change in the safety program


will not be realized and identified until at least one or more injuries occur (Hinze et al., 2013). Therefore, an alternative approach in measuring safety should be considered. The concern about the limitations of the traditionally used safety performance measures promoted for extensive research of alternative measures that can be used as predictors of safety performance. These measures questioned why accidents happen? and how safety can be achieved? Therefore, a definition of safety leading indicators was established. 2.2 Definition of Leading Indicators Multiple definitions of safety leading indicators have been reported in the literature. Toellner (2001) defined leading indicators as “metrics associated with measurable system or individual behaviors linked to accident prevention. ”Grabowski et al. (2007) described leading indicators as the precursor of accidents, or the conditions, events, or measures that precede an incident and have a value in predicting the occurrence of an accident/incident/ or the appearance of an unsafe condition. They are associated with proactive activities that hold the potential of predicting hazards and eliminating or controlling the associated risk. Hinze et al. (2013) defined leading indicators as the “measures of the safety process as it applies to the construction work”. When one of those measures indicates a weakness, it signals the need to implementing interventions to improve safety and therefore eliminating the possibility of any negative occurrence. Sinelnikov et al. (2015) argues that defining leading indicators can be achieved by describing their relationship with lagging indicators, and therefore there is a lack of agreement in literature on the basic definition of leading indicators. Regardless of their definition, leading metrics have the potential to predict and prevent accidents and injuries by providing a set of objectives and transforming organizational safety culture from being passive and reactive to proactive and solution driven. However, the success of organizations in reducing the environmental, safety and health concerns relies on how well OHS practitioners track, manage and use the information provided by leading metrics (Sinelnikov et al., 2015). 2.3 Categorization and Selection of Leading Indicators Researchers defined multiple criteria to categorize and select leading indicators. Selection criteria varied widely between studies. Some selection criteria include the indicator’s reliability, validity, representation, proneness to bias, and economic effectiveness (Hale, 2009). Other criteria permit the selection of a leading indicator if it is measurable/quantifiable, consistent, complete, and significantly correlated with reduction in incidents numbers (Akroush & El-Adaway, 2017). While some researchers categorized leading indicators based on their attributes Guo et al. (2016), others classify indicators to passive and active indicators based on their function in predicting safety performance (Hinze et al., 2013). Due to the variation in selection and categorization criteria, more than 300 leading indicators were identified and classified in the literature. Whether labeling them as leading indicators or other terms, different studies suggested different leading indicators. The Construction Industry Institute (CII) funded an extensive study to identify best practices implemented in the construction industry that would cause a difference in safety performance and would drive the industry toward achieving a goal of zero injuries (Hallowell et al., 2013). The study identified the essential components of an effective construction safety program as listed below:

• Demonstrated management commitment • Staffing for safety • Pre-project and pre-task planning • Safety education and training • Employee involvement

• Safety recognition and rewards • Accident/incident investigations • Substance abuse programs • Subcontractor management

The study identified more than 50 leading indicators based on case studies, safety descriptions of award-winning projects, and group discussion with construction safety experts. The following thirteen major leading indicators


were prioritized for being measurable, tracked, representative of diverse group strategies, and being strong indicators of future safety performance:

• Near Miss Reporting • Project management team safety process

involvement • Worker observation process • Stop work authority • Auditing program • Pre-task planning • Housekeeping program

• Owner’s participation in worker orientation sessions

• Foremen discussions and feedback meetings with the Owner’s PM

• Owner safety walkthroughs • Pre-task planning for vendor activities • Vendor safety audits • Vendor exit debrief

Shea et al. (2016) conducted an extended research to include measures that are linked with organizational safety practices and could be identified as leading indicators of occupational health and safety (OHS). They identified the following indicators:

• Accountability for OHS • Consultation and communication about

OHS • Empowerment and employee involvement

in decision making about OHS • Management commitment and leadership • Positive feedback and recognition for OHS

• Prioritization of OHS • Risk management • Systems for OHS (policies, procedures,

practices) • Training, interventions, information, tools

and resources for OHS • Workplace OHS inspections and audits.

In addition, Awolusi et al. (2017) also stated that the most common used leading indicators used in the construction industry are:

• Project management team • Safety process involvement • Worker observation process • Job site audits • Near-miss reporting

• Housekeeping program • Stop work authority • Safety orientation • Safety training

Although there is no consensus among researchers over specific leading indicators, selection criteria provide some common ground that can be utilized in selecting the appropriate indicators. It is also worth mentioning that the available literature has discussed leading indicators within the general context of construction industry. However, no study has investigated the use of leading indicators in the highway construction and maintenance sector. This research project utilized leading indicators to develop a proactive safety scorecard to evaluate safety performance of KYTC highway maintenance crews.


Chapter 3 Methodology

The development of the scorecard was divided into three phases. In the first phase, the main areas/dimensions to be evaluated in the safety program were identified and weighed utilizing the analytic hierarchy process analysis (AHP) and expertise of safety researchers and KYTC safety practitioners. The second phase included selecting the safety leading indicators tailored to KYTC highway maintenance work within each main dimension. The third phase included the design and development of the final product, which is a macro-based excel scorecard that proactively evaluate safety performance on a district level, area level, and program level. The following subsections describe the details of each phase.

3.1 Phase I: Identification and Selection of Main Dimensions

Part I: Identifying the Main Dimensions Since the primary objective of leading indicators is to serve as a measure to evaluate the effectiveness of a safety program (Hinze et al., 2013), it was important to identify the main dimensions/areas of the safety program to be evaluated. Prior to identifying the leading indicators to be utilized in evaluating safety performance, five dimensions/primary leading indicators were identified from the literature. The dimensions are presented in Table 3.1. Table 3.1 Dimensions of Safety Program

Program Dimension Constructs as Reported in Selected Resources

Management leadership and commitment

• Importance of management commitment (Hinze, 2002) • Project management team safety process involvement (Hallowell et al., 2013) • Management leadership (OSHA, 2016) • Management commitment and leadership (Shea et al., 2016) • Project management team (Awolusi & Marks, 2017)

Employee empowerment and worker participation

• Employee involvement (Hinze, 2002) • Worker observation process; Stop work authority; Foremen discussions and

feedback meetings with the Owner’s PM (Hallowell et al., 2013) • Worker participation (OSHA, 2016) • Empowerment and employee involvement in decision making about OHS

(Shea et al., 2016) • Worker observation process; Stop work authority (Awolusi & Marks, 2017)

Training and Competence

• Safety education and training (Hinze, 2002) • Education and training (OSHA, 2016) • Training, interventions, information, tools and resources for OHS (Shea et al.,

2016) • Safety training (Awolusi & Marks, 2017)

Hazard identification and control

• Staffing for safety; Pre-project and pre-task planning; Accident/incident investigations (Hinze, 2002)

• Near Miss Reporting re-task planning; Housekeeping program; Pre-task planning for vendor activities (Hallowell et al., 2013)

• Hazard identification and assessment; Hazard prevention and control (OSHA, 2016)

• Risk management (Shea et al., 2016) • Near-miss reporting; Housekeeping program (Awolusi & Marks, 2017)


Evaluation and improvement

• Safety recognition and rewards (Hinze, 2002) • Auditing program (Hallowell et al., 2013) • Program evaluation and improvement (OSHA, 2016) • Positive feedback and recognition for OHS; Workplace OHS inspections and

audits (Shea et al., 2016) • Job site audits (Awolusi & Marks, 2017)

Management Leadership and Commitment There is a consensus among researchers and practitioners on the importance of management leadership and commitment in driving safety as management lead by example and provide resources and the visionary required for implementing an effective safety and health program (OSHA, 2016). Management commitment plays a major role in the safety culture of the organization (Hinze, 2002). Management engagement, behavior, and commitment toward safety provide a role model, help in building trust with employees, and emphasize the priority of OHS in the organization (Shea et al., 2016). Employee Empowerment and Worker Participation Workers’ participation incorporates their contribution in building, implementing, evaluating, and improving the safety program (OSHA, 2016). Utilizing safety observers to assist in managing behavior-based safety programs reinforce better safety practices and corrects unsafe behaviors (Hinze, 2002). Shea et al. (2016) argued that “employee involvement in decision making will lead to ownership of their behavior and positive outcomes, such as safety behavior”. Training and Competence Worker's knowledge about workplace hazards and control measures allows workers to accomplish their jobs safely and enhances their productivity (OSHA, 2015). Training is a critical way to convey explicit safety knowledge to workers. Training workers on performing tasks safely positively influences their safety performance in the workplace (Hinze, 2002). “The establishment of OHS training, information, tools, and resources are key leading indicators of OHS performance” (Shea et al., 2016). Hazard Identification and Control Identifying and controlling hazards is a proactive process and a core element of any effective OHS program. Identification of workplace hazards is essential element to develop effective risk controls. Hazard assessment and control can increase the chances of improving safety performance (OSHA, 2016) as failing to identify workplace hazards is a root cause of workplace accidents, injuries, and fatalities. Evaluation and Improvement Program evaluation should be conducted: initially to investigate if the program was established and implemented as intended, and periodically to make sure that the program is on the right track. Evaluation uncovers opportunities for improving, adjusting, monitoring, and revaluating the results (OSHA, 2016). Moreover, positive feedback and recognition would help in achieving a high-performance OHS (Shea et al., 2016). It was important to ensure that the selected dimensions are consistent with OSHA Recommended Practices for Safety & Health Programs in Construction. In addition, it was also critical to determine the extent to which each dimension drives safety performance. Therefore, after selecting the dimensions to be evaluated, the research team utilized KYTC safety expertise to calculate the weight of contribution each dimension makes to safety performance. The weight of each dimension was calculated using the AHP analysis as shown in the following subsection. 3.2 Part II: Weighting the Main Dimensions Although the selected dimensions satisfy OSHA recommended practices and guidelines and suit the KYTC program, it is important to quantify the magnitude or weight each dimension contributes by to the program safety


performance. To achieve this objective, expertise of the research team and KYTC safety personnel were utilized to calculate the weight of each dimension using the Analytic Hierarchy Process (AHP) analysis method. AHP was established in 1971 by T. L. Saaty (1971) and has been applied to many decision making problems in manufacturing industries such as selecting a suitable machine in the manufacturing factory (Skibniewski & Chao, 1992). Its popularity eventually led to the creation of an ASTM standard (ASTM E 1765-95) for using AHP in multi-attribute decision analysis (ASTM, 1995). The method divides a complex system into hierarchical elements. The elements are evaluated for their importance against one another through pair wise comparisons. The results of the comparisons become measurable in a comparison matrix. The eigenvector of the matrix is calculated which shows the comparative weight among the elements of the specific hierarchy (Lin & Yang, 1996). Numerous previous studies applied the AHP methodology when evaluating significance of multiple options. The availability, low complexity, and possibility of being used in many fields made AHP a popular method (Podgorski, 2015). The compatibility of AHP with any decision-making research makes it popular in a variety of fields. As an example, Guo et al. (2014) applied AHP to designing electromechanical products in environmental engineering. Shapira and Simcha (2009) state the most important feature of AHP is the capability of assigning weights to qualitative and quantitative factors in order to have a numeric basis for making decisions. One of the first applications of AHP in operational health and safety was in research conducted by Jervis and Collins (2001). The aim of their research was to show managers which field they should invest in to get a return on their investment. According to Aminbaksh et al. (2013), suitable prioritization through AHP is necessary for management, planning, and budgeting of safety related risks. Al-Harbi (2001) presented this method to prequalify contractors in project management by prioritizing criteria in prequalifying decision. Teo and Ling (2006) conducted a study that applied AHP to achieve a high level of safety on construction projects. In a recent study in operational safety and health, Podgórski (2015) demonstrated selection of leading key performance indicators in an operational safety and health management system by applying AHP method for the selection of leading indicators. The process conducted in the AHP comprises the following steps. Step 1: Establishing the Factors of Pair-Wise Comparisons in the Matrix In this step, all the factors in the matrix should be formed. Each of the factors in the matrix is the mean of the respondent’s judgement about that specific factor. The Structure of pair wise comparison (ASTM, 1995) is shown in Figure 3.1. The judgement matrix has cells filled with the mean of respondent’s answers to each question based on the proposed scale in the survey, which will be discussed later. According to Chang et al. (2007), there are rules related to each cell that should be followed. Each cell should have a value greater than zero, cells comparing the same alternative should have a value of 1, and opposite cells should have a value inversely proportional. These rules are written as follows:

𝑎!" > 0,𝑎!! = 1,𝑎!" = 1 𝑎"!( The total number of pairwise comparisons needed for the AHP can be written as 𝑛(𝑛 − 1) ⁄ 2. Step 2: Calculating the priority vector

To calculate priority vectors for weights for the drivers, first synthesizing the pair-wise comparison should be performed. Synthesizing can be calculated by dividing each cell by the sum of its column. As an example, for the cell𝑎!!, it would be:

𝐴 = 0𝑎!! ⋯ 𝑎!"⋮ ⋱ ⋮𝑎"! ⋯ 𝑎""

4 , 𝑎!! ∑ 𝑎!!!!"!

6


Once the synthesized matrix is computed, priority vectors could be calculated by getting the average of each row. Assume 𝑏 is the synthesized form of 𝑎, the calculation will be:

8𝑏!! ⋯ 𝑏!"⋮ ⋱ ⋮𝑏"! ⋯ 𝑏""

9 =(∑ 𝑏!#)/𝑛

"#$!…

(∑ 𝑏"#)/𝑛"#$!

.

At this time, the priority vector is calculated and should add to 1. The last part of the process is to verify and validate the judgements with a consistency ratio to see if the results are consistent. This analysis is discussed and presented in the results chapter of this manuscript.

Figure 3.1 Structure of Pair Wise comparison (Shaded boxes were Completed by Survey Respondents)

An electronic survey was deployed to safety researchers and KYTC safety personnel using Qualtrics Survey Software. The survey consists of 19 questions, some of which are basic biographic questions. Participants were asked to rate the relative importance of each dimension compared to the other selected dimensions based on a five-points Likert scale as shown in Table 3.2. A sample of the survey questions is shown in Table 3.3. The survey included instructions, definition of each dimension, and examples demonstrating the procedures to answer the questions. The results of the relative importance weighting can be seen in Table 4.1. Table 3.2 Relative Importance Comparison Scale Used in Survey

Degree of Comparison Equal Moderate Strong Very Strong Extreme

If X is more important than Y 1 2 3 4 5

If X is less important than Y 1 -2 -3 -4 -5

Table 3.3 Sample of Survey Questions

Dimensions Relative Importance

How much more valuable is Management Leadership than Employee Participation? How much more valuable is Management Leadership than Training & Competence?

Alternative 1

Alternative 2

….

Alternative j

Alternative k

Alternative 1

1

Desirability of Alt. 1 versus Alt. 2

….

Desirability of Alt. 1 versus Alt. j

Desirability of Alt. 1 versus Alt. k

Alternative 2

Desirability of Alt. 2 versus Alt. 1

1

….

Desirability of Alt. 2 versus Alt. j


….

….

….

1

….

….

Alternative j

Desirability of Alt. j versus Alt. 1

Desirability of Alt. j versus Alt. 2

….

1

Desirability of Alt. j versus Alt. k

Alternative k

Desirability of Alt. k versus Alt. 1


….

Desirability of Alt. j versus Alt. k

1


How much more valuable is Management Leadership than Hazards Identification & Prevention?

How much more valuable is Hazard Identification & Prevention than Evaluation & Improvement?

How much more valuable is Training and Competence than Evaluation & Improvement?

3.3 Phase II: Selection of Leading Indicators After identifying and weighing the main dimensions of be evaluated, it was necessary to identify the leading indicators that reflects safety performance within each dimension of the program. Selecting leading indicators that suit specific safety program is a complicated task. The complexity in selecting, measuring, and applying leading indicators to evaluate safety performance in a specific safety program stem from different issues. The significant number of variables in a safety system makes it difficult to obtain an accurate forecast of safety accidents. According to Manuele (2009), this issue poses a significant obstacle to the use of leading indicators. In addition, the difficulty in defining leading indicators and gaining workers and management buy-in is attributed to the absence or lack of well-established industry parameters, ambiguity in defining leading indicators, lack of familiarity with leading indicators, lack of understanding of the benefits, and the presumptions associated with additional cost and efforts (Akroush & El-Adaway, 2017; Hinze et al., 2013; Wehle & Hinze, 2009). To alleviate the effects of these issues and to ensure the validity of the selected leading indicators, an expert panel from the research team and KYTC safety professional was formed. The goal of the panel was to ensure that the selected indicators are measurable, reliable, practical, and tailored to the KYTC safety program. A comprehensive systematic review of different studies and performance evaluation tools was conducted to select the leading indicators within each dimension. The review covered literature sources that discuss leading indicators, proactive safety performance assessment tools, safety operational excellence, and other related studies. The review yielded an initial list of 330 safety leading indicators. Through multiple rounds of review and discussion, the panel shorten the list to a total of 65 leading indicator. Table 3.4 shows an example of the indicators selected for each dimension. Indicators were selected based on the following criteria: 1- The selected indicator should be assumption-based, complete, and consistent (Akroush & El-Adaway, 2017;

Hale, 2009; Leveson, 2015). 2- The selected indicator can be measured (Akroush & El-Adaway, 2017; Biggs et al., 2010; Leveson, 2015; Stricoff,

2000). 3- The selected indicator can be tailored to KYTC safety model (Akroush & El-Adaway, 2017; Leveson, 2015; Stricoff,

2000).


Table 3.4 Sample of Selected Leading Indicators Main Dimension Indicator

Management leadership and commitment

The district management allocates the necessary resources (staff, budget, safety information, tools, equipment, PPE, etc.) for employees to work safely The district management communicates safety expectations, employees’ rights, responsibilities, and authorities, and ensures that all employees have a full understanding of them.

Employee empowerment and worker participation

Maintenance crews who were promoted for their safety suggestions, initiatives, feedback, etc. On a district level, there is a safety committee that include maintenance employees, superintendent, and management

Training and Competence Maintenance employees who had First aid/CPR training. Maintenance employees who completed OSHA 10-hour safety training.

3.4 Phase III: Design and Development of the Scorecard After identifying the main dimensions and the safety leading indicators within each dimension of the program, there were two measurement issues that needed to be addressed. One issue was related to the weight of the contribution each dimension makes to the program safety performance. This issue was addressed through utilizing the safety expertise of researchers and KYTC practitioners through the AHP analysis. Each dimension was evaluated by experts based on their relative importance to safety performance. The second issue was to assign a performance score to each selected leading indicator within each dimension. The literature lacks research dedicated to address the scoring criteria of leading indicators as this topic is relatively new and has not matured. There is no consensus over a single approach to assign scores to individual leading indicators. However, the research team utilized a scoring criterion previously adopted by Ahmad (2000) to develop a scoring system to a proactive safety performance measurement tool. The criterion is practical, has a robust scientific basis, and can be modified to suit the scorecard.

3.5 Scoring Indicator Achievement The performance scoring criterion adopted by Ahmad (2000) is similar to the criterion used in the Objectives Matrix (OMAX) that was introduced by Felix and Riggs (1983) to measure productivity performance. In this approach, the performance scales are contained in a matrix and range from 0 to 10. Predetermined benchmark scores are assigned to different performance levels of each indicator. These levels are contained between the following two scores:

• Minimum benchmark: score 0 • Maximum benchmark: score 10

The minimum benchmark value of 0 indicates no safe performance exists for the corresponding indicators. In contrast, the maximum benchmark value of 10 indicates the ultimate achievement targeted in the corresponding indicator. As the benchmarks determine the minimum and maximum scores of performance, the levels of performance between them are assigned scores in equal increments.

Due to the nature of selected leading indicators, performance levels were divided in three different ways. For indicators that are measured in percent, the performance measurement is eventually transformed to a value between 0 and 10. For indicators that are measured with a five increment Likert scale, the performance measurement is eventually transformed to a five-equal increment between 0 and 10. For indicators that are measured with a nominal scale, the performance measurement is transformed to a value of 0 or 10.


3.6 Scorecard Levels To improve the effectiveness of the leading indicators in predicting safety performance, evaluate safety performance across different levels of the program, and to ensure there is a methodology to reduce the bias in reporting, the scorecard was designed to address safety performance at the three following levels:

• District Level • Area Level • Program level

On a district level, the scorecard assesses safety performance based on leading indicators collected from highway maintenance workers, foremen, superintendents, and safety personnel within each district. On an area level, safety performance is assessed based on the performance of multiple districts associated with the area office as well as the leading indicators collected from engineering, safety, and administration personnel from the area office. On the program level, safety performance is assessed based on the performance of all districts, areas, and the indicators collected from safety and administration personnel on a program level. On each individual level, the weighted score of each dimension is calculated by averaging the scores of leading indicators within each dimension and multiplying the average score by the weight of the dimension. The final score is calculated by averaging the weighted scores of all dimensions. This calculation criterion is performed on a district level, area level and program level.


Chapter 4 Results and Discussion 4.1 Weighting Results of the Main Dimensions To determine the weight of contribution each main dimension makes to safety performance, the AHP survey was deployed to 20 participants including KYTC safety coordinators and safety administration personnel as well as construction safety researchers. The potential participants were contacted and informed about the research and its objective. Eighteen out of the 20 members contacted responded and completed the survey. Of the 18 participants: fourteen (77.8 %) are KYTC personnel; twelve of them are safety coordinators with (8 – 22) years of experience, and two safety administration personnel with more than 30 combined years of experience. The rest of participants are four construction safety researchers with (5 - 15) years of experience. The collected data were analyzed by applying the AHP decision-aiding method (Al-Harbi, 2001). The results of the analysis are shown in Table 4.1. Table 4.1 Relative Importance / Weight of Main Dimensions

Performance Metrics Weight

Management leadership and commitment 24.8% Employee empowerment and worker participation 23.1% Training and Competence 18.9% Hazards Identification and Control 20.5% Evaluation and improvement 12.7%

Management commitment, employee engagement, and hazard identification and prevention scored the highest, respectively. In other words, the relative importance of these metrics to safety performance is higher than the rest of metrics. Such results are consistent with what has been reported in the literature. Metrics like management commitment, workers engagement, and hazards identification and prevention have been reported as significant indicators of safety performance in the construction industry (Akroush & El-Adaway, 2017; Michael et al., 2005; Wachter & Yorio, 2014). Management commitment in particular scored the highest among all metrics according to experts rating. Such result is also expected as management commitment has been highlighted as an important driver not only for safety performance but also for adopting and implementing proactive safety programs that involve leading indicators (Hinze et al., 2013). The implementation of such programs require management that is committed to dedicate the require resources, take ownership of the program, and play a role model for crews to encourage the use of leading indicators and proactive measures.

4.2 Final Scorecard Design After analyzing the collected data, calculating the weight of main dimensions, and selecting the leading indicators or the measures within each dimension, the research team utilized Microsoft Excel to develop a macro-enabled scorecard that uses all of this information to assess safety performance on a district, area, and a program level. The scorecard should be used by KYTC highway maintenance workers, superintendents, safety coordinators, and other safety personnel at districts offices, area offices, and the cabinet office. The design of the scorecard is user-centered and relies on the following criteria:

1- The scorecard should be developed in a platform that is accessible to all KYTC employees 2- The scorecard should be simple, intuitive, and easy to use and understand 3- The scorecard should allow performance measures input from different levels 4- There should be a cross check criterion to validate the results between different levels

The scorecard consists of three main sections in addition to the instructions. Instructions are written in simple language, and a supplementary instructional video tutorial was provided to first time users to ensure a full understanding of the flow and functionality of the tool. The main introductory interface of the scorecard is an excel sheet with general use and navigation instructions and three macro-enabled buttons that direct the user to the district, area, or the program interfaces Figure 4.1.


Figure 4.1 Main User Interface of Scorecard The first section of the scorecard is the district section. The users of this section are maintenance crews of each district as well as the safety coordinator of the district. Maintenance crews provide their input to the scorecard via a survey generated from the scorecard and distributed by the district safety coordinators. The coordinators collect the completed surveys and input the data in the district worksheet. The district section also requires the coordinators to provide their input in a separate user interface. The minimum number of completed surveys for each district depends on the total number of maintenance employees in the district. A table and instructions on the required sample size of surveys to be collected were provided with the scorecard. The second section of the scorecard is the area section. The users of this section are KYTC safety personnel in each area office. This section utilizes the proactive metrics to assess safety performance at a higher level compared with the district level. Since each area office is responsible for multiple districts, safety performance of maintenance crews in any area can be evaluated by examining the performance of districts within the area as well as the performance metrics of the area in this section. This mechanism is useful to validate the results in two different levels. For example, management commitment can be examined from workers perspective using the district data and from middle management perspective using the area data. The third section of the scorecard is the program or the cabinet section. The users of this section are the administrators of the KYTC safety program. The purpose of including this section in the scorecard is to incorporate the input of management at the program level in the evaluation of safety performance. The performance at the program level can be evaluated using the results from this section, the area section, and the district section. The results of all sections are displayed in the results sheet of the scorecard. Results are displayed in three different ways as shown in Figure 4.3. For each level, the results are displayed for every performance metric in a table and a radar chart. This helps users to understand which main dimension needs attention. The overall score for the program, the area, and the district are also shown on the results sheet.


Figure 4.2 A Sample of the District Results Sheet

4.3 Results of Pilot Study In a pilot try to examine the usability of the scorecard, the research team coordinate a pilot implementation in different districts of Kentucky. A total of 558 KYTC employees from eight different districts and three areas of the program participated in the pilot implementation of the scorecard. Table 4.2 shows the number and percent of participants. Safety coordinators distributed the surveys on maintenance crews, collected the data, and filled in the district section of the scorecard. The area and program sections were filled by safety personnel at the area offices and the cabinet office. The overall score of each main metric on a district level is presented in Table 4.3. Management leadership and commitment was ranked the highest (88%) among the metrics followed by training and competence (83%) and evaluation and improvement (81%). However, employee engagement scores relatively low (65%). Although such scores are informative to examine a holistic picture of the program performance on a district level, a closer look of the scores on area and program levels paints a slightly different image. As Figure 4.3 shows, management leadership and commitment score on a program level (43%) is significantly different from the score on a district level (88%). This indicates that management commitment from workers perspective is different from top management perspective. The same conclusion can be drawn about the evaluation and improvement dimension. While it was ranked relatively high by maintenance crews at the district level, the safety management at the area and the program level provide a different perspective. On the other hand, the scores of hazards identification and prevention and employee engagement metrics were consistent across the three levels. While the results of the pilot implementation were helpful to assess the usability of the scorecard, they are incomplete and cannot be used to evaluate safety performance. It is also worth mentioning that the research team utilized the feedback received from users to improve the scorecard.

Table 4.2 Number of Participants in the Pilot Implementation

Level Avg. Employees Surveyed % Surveyed

D1 62 26%

D2 58 19%

D3 109 39%

D4 51 18%

D5 54 22%


D6 21 10%

D7 17 7%

D12 181 60%

Area 1 1

Area 2 1

Area 3 1

Cabinet-1 1

Cabinet-2 1

Table 4.3 Average Score of Main Metrics on a District Level

Indicator Average Score

Management Leadership and Commitment 88%

Training & Competence 83%

Evaluation & Improvement 81%

Hazards Identification & Prevention 77%

Employee Engagement 65%


Figure 4.3 Pilot Implementation Results for District, Area, and Program Level Ideally, the results of the scorecard should be consistent with the reactive traditional measures over the same period of time. However, because the pilot implementation was for usability testing purposes, it was not viable to compare the results of the pilot implementation to the traditional measures used by KYTC. When the scorecard districts scores were compared to the 2019 TRIR numbers for the same districts, there appear to be an apparent inconsistency as shown in Figure 4.4. This can be attributed to the incompleteness of the scorecard results and the difference in the timeframe. The comparison would be available in the same timeframe with complete results of all districts.

Figure 4.4 Comparison of the Pilot Trial Results with the 2019 TRIR of Districts

87.63%

64.75%

83.00%77.00%

81.00%

80.00%

71.33%

53.33%58.67%

30.00% 43.00% 62.50%

89.50%

71.00%

50.00%

ManagementLeadership

EmployeesEngagement

Training &Competence

Hazards Identification& Controls

Evaluation &Improvement

Level Average Score District Average(%)Area Average(%)

Cabinet Average(%)

1

2

3

4

5

6

7

8District District District District District District District District

Rank

(low

er n

umbe

r is b

ette

r)

Rank of Districts in Scorecard Results and TRIR

Metrics Rank TRIR Rank


Chapter 5 Conclusion

The construction industry has been criticized not only for its poor safety performance but also for its reactive nature to safety problems. One of the highly criticized issues is the use of traditional reactive measures of safety performance. Measures that assess performance based on historical safety metrics, such as the number of work-related injuries. Such metrics have been criticized for being incapable of predicting safety performance and for being measures of the system failure. These shortcomings triggered extensive research efforts to develop proactive safety performance measures and replace or assist the traditional methods in evaluating safety performance. Although the industry has significant area of improvement in this path, significant strides have been made to adopt such measures in the construction industry. However, the use of proactive safety performance metrics in the highway sectors is still in its infancy. The majority, if not all, State DOTs rely on the traditional methods in measuring and evaluating workers safety performance. Minimal research exists to address this issue. Utilizing previous research in this subject and the safety expertise in highway maintenance work, researchers developed a proactive safety scorecard to evaluate safety performance of KYTC highway maintenance crews. The study utilizes 65 leading indicators to assess safety performance in five main areas including management commitment, employee engagement, training and competence, hazards identification and prevention, and evaluation and improvement. The final product of this study is a proactive scorecard that predict safety performance at a district level, area level, and a program level based on the aforementioned metrics. The scorecard provides KYTC maintenance crews with a mechanism to predict areas of improvement in their safety program. It is not intended to replace the current methods used to evaluate safety performance. Instead, it can be used as a proactive tool to assist in identifying safety issues and areas that need attention prior to any system failure. The pilot study of the safety metrics scorecard resulted in data from eight of the twelve districts, three area administrators, and two executive level responses. Overall, high scores (indicating strong performance of the metrics) were seen in the dimensions based on surveys of the districts. However, the area and executive level responses showed lower values. The employee engagement dimension had the lowest score, and a statistical grouping of the dimensions, found that it was in the lowest tier by itself. This indicates a need to seek more opportunities to involve employees in the safety program. Examples of such engagement include formal, active, and empowered safety committees, involvement of employees in the writing of safety policies and procedures, and involvement in hazard control practices like delivering toolbox talks, job hazard analyses (JHAs), or safety incident reviews.


Chapter 6 References

Ahmad, R. K. (2000). Developing a proactive safety performance measurement tool (SPMT) for construction sites Loughborough University].

Akroush, N. S., & El-Adaway, I. H. (2017). Utilizing construction leading safety indicators: Case study of tennessee. Journal of Management in Engineering, 33(5), 06017002.

Al-Harbi, K. M. A.-S. (2001). Application of the AHP in project management. International journal of project management, 19(1), 19-27.

Al-Shabbani, Z., Sturgill, R., & Dadi, G. B. (2018). Developing a pre-task safety briefing tool for Kentucky maintenance personnel. Transportation research record, 2672(12), 187-197.

ASTM. (1995). Standard Practice for Applying the Analytic Hierarchy Process to Multiattribute Decision Analysis Investments Related to Buildings and Building Systems ASTM E 1765-95. West Conshohocken, Pennsylvania.

Awolusi, I. G., & Marks, E. D. (2017). Safety activity analysis framework to evaluate safety performance in construction. Journal of construction engineering and management, 143(3), 05016022.

Biggs, H. C., Dinsdag, D., Kirk, P. J., & Cipolla, D. (2010). Safety culture research, lead indicators, and the development of safety effectiveness indicators in the construction sector. International Journal of Technology, Knowledge and Society, 6(3), 133-140.

Chang, C.-W., Wu, C.-R., Lin, C.-T., & Chen, H.-C. (2007). An application of AHP and sensitivity analysis for selecting the best slicing machine. Computers & Industrial Engineering, 52(2), 296-307. doi:http://dx.doi.org/10.1016/j.cie.2006.11.006

Choudhry, R. M., Fang, D., & Mohamed, S. (2007). Developing a model of construction safety culture. Journal of Management in Engineering, 23(4), 207-212.

Costella, M. F., Saurin, T. A., & de Macedo Guimarães, L. B. (2009). A method for assessing health and safety management systems from the resilience engineering perspective. Safety Science, 47(8), 1056-1067.

Felix, G. H., & Riggs, J. L. (1983). Productivity measurement by objectives. National Productivity Review, 2(4), 386-393.

Grabowski, M., Ayyalasomayajula, P., Merrick, J., & Mccafferty, D. (2007). Accident precursors and safety nets: leading indicators of tanker operations safety. Maritime Policy & Management, 34(5), 405-425.

Guo, J., Zhou, M., Li, Z., & Xie, H. (2014). Green design assessment of electromechanical products based on group weighted-AHP. Enterprise Information Systems(ahead-of-print), 1-22.

Guo, B. H., Yiu, T. W., González, V. A., & Goh, Y. M. (2016). Using a pressure-state-practice model to develop safety leading indicators for construction projects. Journal of construction engineering and management, 143(2), 04016092.

Guo, H., & Yiu, T. W. (2013). How traditional construction safety performance indicators fail to capture the reality of safety. The 38th Austrlasian Universities Building Education Association Conference, Auckland, New Zealand,

Hale, A. (2009). Why safety performance indicators? Safety Science, 4(47), 479-480.


Hallowell, M. R., Hinze, J. W., Baud, K. C., & Wehle, A. (2013). Proactive construction safety control: Measuring, monitoring, and responding to safety leading indicators. Journal of construction engineering and management, 139(10), 04013010.

Hinze, J. (2002). Safety plus: making zero injuries a reality. Construction Industry Institute, 160-111.

Hinze, J., Thurman, S., & Wehle, A. (2013). Leading indicators of construction safety performance. Safety Science, 51(1), 23-28.

Howell, G. A., Ballard, G., Abdelhamid, T. S., & Mitropoulos, P. (2002). Working near the edge: a new approach to construction safety. Annual conference on lean construction,

Leveson, N. (2015). A systems approach to risk management through leading safety indicators. Reliability Engineering & System Safety, 136, 17-34.

Lin, Z.-C., & Yang, C.-B. (1996). Evaluation of machine selection by the AHP method. Journal of Materials Processing Technology, 57(3), 253-258.

Manuele, F. A. (2009). Leading & lagging indicators. Professional Safety, 54(12), 28.

Mengolini, A., & Debarberis, L. (2008). Effectiveness evaluation methodology for safety processes to enhance organisational culture in hazardous installations. Journal of hazardous materials, 155(1-2), 243-252.

Michael, J. H., Evans, D. D., Jansen, K. J., & Haight, J. M. (2005). Management commitment to safety as organizational support: Relationships with non-safety outcomes in wood manufacturing employees. Journal of safety research, 36(2), 171-179.

OSHA. (2016). Recommended Practices for Safety & Health Programs in Construction. OSHA. Retrieved 05/01/2020 from https://www.osha.gov/shpguidelines/

Podgorski, D. (2015). Measuring operational performance of OSH management system - A demonstration of AHP-based selection of leading key performance indicators. Safety Science, 73, 146-166. doi:10.1016/j.ssci.2014.11.018

Robson, L. S., Macdonald, S., Gray, G. C., Van Eerd, D. L., & Bigelow, P. L. (2012). A descriptive study of the OHS management auditing methods used by public sector organizations conducting audits of workplaces: Implications for audit reliability and validity. Safety Science, 50(2), 181-189.

Saaty, T. L. (1971). On polynomials and crossing numbers of complete graphs. Journal of Combinatorial Theory, Series A, 10(2), 183-184.

Sgourou, E., Katsakiori, P., Goutsos, S., & Manatakis, E. (2010). Assessment of selected safety performance evaluation methods in regards to their conceptual, methodological and practical characteristics. Safety Science, 48(8), 1019-1025.

Shea, T., De Cieri, H., Donohue, R., Cooper, B., & Sheehan, C. (2016). Leading indicators of occupational health and safety: An employee and workplace level validation study. Safety Science, 85, 293-304.

Sinelnikov, S., Inouye, J., & Kerper, S. (2015). Using leading indicators to measure occupational health and safety performance. Safety Science, 72, 240-248.

Shapira, A., & Simcha, M. (2009). AHP-based weighting of factors affecting safety on construction sites with tower cranes. Journal of Construction Engineering and Management, 135(4), 307-318.


Skibniewski, M. J., & Chao, L.-C. (1992). Evaluation of advanced construction technology with AHP method. Journal of Construction Engineering and Management, 118(3), 577-593.

Stricoff, R. S. (2000). Safety peformance measurement: Identifying prospective indicators with high validity. Professional Safety, 45(1), 36.

Teo, E. A. L., & Ling, F. Y. Y. (2006). Developing a model to measure the effectiveness of safety management systems of construction sites. Building and Environment, 41(11), 1584-1592.

Toellner, J. (2001). Improving safety & health performance: identifying & measuring leading indicators. Professional Safety, 46(9), 42.

Wachter, J. K., & Yorio, P. L. (2014). A system of safety management practices and worker engagement for reducing and preventing accidents: An empirical and theoretical investigation. Accident Analysis & Prevention, 68, 117-130.


Appendix A: District Scorecard Questions Employee Survey

Read each statement and circle (Yes) if you agree with the statement or (No) if you disagree.

Statement Agree Disagree

1- The district management ensures that KYTC safety policies, goals, and objectives are communicated to all employees on time through different media, such as meetings, safety newsletter, posters, etc.

Yes

No

2- The district management communicates safety expectations, employees’ rights, responsibilities, and authorities, and ensures that all employees have a full understanding of them.

Yes

No

3- The district management have an open-door policy where all employees can communicate with managers about safety issues without fear of discrimination or reprisal.

Yes

No

4- The district management allocates the necessary resources (staff, budget, safety information, tools, equipment, PPE, etc.) for employees to work safely

Yes No

5- Workers participate in making decisions associated with safety issues, initiatives, and programs.

Yes No

6- Maintenance workers participate in the regular inspection of work equipment and tools.

Yes No

7- Maintenance crews have the authority to stop the work and do the necessary changes they identify when they encounter safety concerns, such as unexpected potential hazards.

Yes

No

8- On a district level, there is a safety committee that include maintenance employees, superintendent, and management.

Yes No

9- Maintenance crews routinely review safety issues (potential hazards, risk mitigation means, etc.) associated with the work in progress and planned work.

Yes

No

10- Maintenance crews inspect workplace, equipment, and tools for potential hazards prior to the workday.

Yes No

11- Workers and supervisors are aware of and fully understand the formal emergency plan when a safety incident occurs.

Yes No

12- The district has a well-defined formal plan for incidents investigations that immediately begin after an incident occur.

Yes No

13- The district safety committee meets routinely to discuss hazards trends and establish, monitor, and update control and prevention plans.

Yes No

Part II: Read each statement and mark the appropriate choice.


Statement Mark your choice

14- Maintenance crews routinely provide feedback on hazards, close calls, improvement, opportunities, and or observations.

� Never � Rarely � Sometimes � Often � Always

15- Maintenance workers routinely participate in work planning. � Never � Rarely � Sometimes � Often � Always

16- Maintenance crews participate in investigations of incidents and near misses and the development of corrective actions and programs.


17- Injured employees are involved in incident review process. � Never � Rarely � Sometimes � Often � Always

18- Crew leaders encourage, document, and promote workers' safety initiatives. � Never � Rarely � Sometimes � Often � Always

19- Crew leaders and supervisors encourage, document, and communicate workers feedback.


20- Maintenance crews routinely hold pre-work briefing for new work tasks. � Never � Rarely � Sometimes � Often � Always

21- Maintenance crews routinely review job hazards analysis JHAs. � Never � Rarely � Sometimes � Often � Always

22- Work vehicles are routinely inspected prior to work. � Never � Rarely � Sometimes � Often � Always

23- Maintenance crews hold daily work-specific safety briefings/talks by competent supervisor before starting the workday.

� Never � Rarely � Sometimes


� Often � Always

24- Inspection of PPE is routinely performed and documented. � Never � Rarely � Sometimes � Often � Always

25- Safety observation, near misses, and incidents reports are used in the development of safety briefings, toolbox talks, and/or safety refreshers


Documents

Development, Implementation, and Tracking of Preventative