1

SAFETY MANAGEMENT MOK SHING CHI

Department of Civil Engineering National University of Singapore

10 Kent Ridge Crescent Singapore 119260

INTRODUCTION In October 1994,the requirement by law for construction/building worksites, where the contract sum of work to be carried out is S$10 million or more, to implement a safety management system (SMS) was introduced1 with the intention of promoting the concept of �self-regulation� of safety and health in contractors. The SMS of worksites where the contract sum of work to be carried out is S$30 million or more is also to be audited by independent external auditors at least once every six months to highlight shortcomings in the SMS implemented and recommend measures for further enhancement. The general objective of this study is to locate the critical elements of the SMS that are responsible for a good safety performance for the selected MRT and road related projects that are under Land Transport Authority (LTA) supervision. The specific objectives of the research includes:

1. To analyze the correlation between project SMS elements and the Accident Frequency Rate (AFR) grading given in audits.

2. To analyze the correlation between project SMS elements and the AFR recorded in LTA

Safety Information System (SITS) database. MATERIALS AND METHODS The data used for the 1st research objective is collected from the audits done by external companies for LTA projects. The format of data is shown as follows: Table 1: Type of data used for Decision Tree Model *LTA indicates V Good: 85% and above, Good: 70% and above but below 85%, Acceptable: 50% and above but below 70%, Poor: 30% and above but below 50% and V Poor: below 30%. The contractors� performance in each of the 5 elements of the SMS is based on the score they obtain for the respective elements during safety audits conducted for their project. The contractor�s safety performance given in the audit is based on the Accident Frequency Rate and Severity Rate (AFR & SR). The actual score for every element were kept confidential; the data made available has been classified into 5 categories, as shown above. A decision tree analysis was done based on the classification of the 5 elements of the SMS and the safety performance, indicated by AFR & SR. This is to test which elements of the SMS are responsible for a good safety performance. The key elements would be nearer the root node, and levels denote how far these factors are from the root node. An example of decision tree2 is shown below.

S/N Attribute Type Example 1 Audit ID Label 21 2 Year Label 2000 3 Contract Number Label RC008 4 Project Type MRT, RDS RDS 5 Workplace V Good, Good, Acceptable, Poor, V Poor Good 6 Workforce V Good, Good, Acceptable, Poor, V Poor Acceptable 7 Management V Good, Good, Acceptable, Poor, V Poor Poor 8 Documentation V Good, Good, Acceptable, Poor, V Poor Good 9 Communication V Good, Good, Acceptable, Poor, V Poor V Good 10 AFR & SR V Good, Good, Acceptable, Poor, V Poor Good

2

Figure 1: Example of a Decision Tree

The data used for the 2nd research objective is collected from the LTA SITS. It is a computerized system designed to capture accidents/incidents that had occurred on LTA construction sites. This data is used to obtain a set of new AFR values, while the element classification from the 1st data set will remains. Since the LTA SITS data contains all accidents occurred for all audit periods, a new AFR score for an audit can be obtained using the following formula: AFR per 50000 man-hours worked = Number of accidents x 50000

Total Man Hours Worked See 5 Release 1.18 is used as the analysis program. 206 audit scores are inputted into the program as data file, and the table above is used as the name file. A name file serves as the platform or the design of analysis where the data is to be analyzed. RESULTS AND DISCUSSION A total of 11 tests were done with variations on elements and AFR classes� arrangement. This was done so that comparison can be made and errors can be identified. The table below summarizes the respective variations, error rate of the test and the rationale of each test variation. The full decision trees are shown in Appendix A. Test 1 Error: 39.9% S/N Attribute Type 1 Audit ID Label 2 Year Label 3 Contract Number Label 4 Project Type MRT, RDS 5 Workplace V Good, Good, Acceptable, Poor, V Poor 6 Workforce V Good, Good, Acceptable, Poor, V Poor 7 Management V Good, Good, Acceptable, Poor, V Poor 8 Documentation V Good, Good, Acceptable, Poor, V Poor 9 Communication V Good, Good, Acceptable, Poor, V Poor 10 AFR & SR V Good, Good, Acceptable, Poor, V Poor

Test 2 S/N Attribute Type 5 Workplace V Good, Good, Acceptable, Poor, V Poor 6 Workforce V Good, Good, Acceptable, Poor, V Poor 7 Management V Good, Good, Acceptable, Poor, V Poor 8 Documentation V Good, Good, Acceptable, Poor, V Poor 9 Communication V Good, Good, Acceptable, Poor, V Poor 10 AFR & SR V Good, Not V Good Error: 39.9% Attribute 1-4 remains. AFR & SR reduces the number of classes from 5 to 2. From the data, 128 out of 216 AFR & SR belongs to the class of V Good. To enhance prediction power, the remaining 4 classes have been combined to a class called �Not V Good�.

3

Test 3 S/N Attribute Type 5 Workplace Continuous 6 Workforce Continuous 7 Management Continuous 8 Documentation Continuous 9 Communication Continuous 10 AFR & SR V Good, Not V Good Error: 24.9% See 5 program has different tree splitting algorithms for discrete and continuous attributes hence different results will be yielded. Discrete values V Good, Good, Acceptable, Poor, V Poor are changed to continuous values 1,2,3,4,5. Test 4 S/N Attribute Type 5 Workplace V Good, Not V Good 6 Workforce V Good, Not V Good 7 Management V Good, Not V Good 8 Documentation V Good, Not V Good 9 Communication V Good, Not V Good 10 AFR & SR V Good, Not V Good Error: 39.9% The class of the 5 attributes has been reduced from 5 to 2. Test 5 S/N Attribute Type 5 Workplace Continuous 6 Workforce Continuous 7 Management Continuous 8 Documentation Continuous 9 Communication Continuous 10 AFR & SR V Good, Good, Acceptable, Poor, V Poor Error: 39.9% Continuous attributes and 5 AFR & SR classes. Test 6 S/N Attribute Type 5 Workplace Good, Not V Good 6 Workforce Good, Not V Good 7 Management Good, Not V Good 8 Documentation Good, Not V Good 9 Communication Good, Not V Good 10 AFR & SR Good (0-85%), Not V Good (85-100%) Error: 36.6% From the result of no. of accident over Man Hours worked, th AFR values lie from 0-3. According to the standard given by LTA, 0-30% is considered V Poor, 30-50% is Poor, 50-70% is Acceptable, 70-85% is Good and 85-100 is considered V Good. Hence adopting a similar model, score of >0.45 is Not V good, 0-0.45 is considered V Good.

4

Test 7 S/N Attribute Type 5 Workplace Good, Acceptable, Poor 6 Workforce Good, Acceptable, Poor 7 Management Good, Acceptable, Poor 8 Documentation Good, Acceptable, Poor 9 Communication Good, Acceptable, Poor 10 AFR & SR Good, Acceptable, Poor(0-50%, 50-70%, 70-

100%) Error: 28.8% From the result of no. of accident over Man Hours worked, th AFR values lie from 0-3. According to LTA�s rating, 0-30% is V Poor, 30-50% is Poor, 50-70% is Acceptable, 70-85% is Good and 85-100 is considered V Good. Hence adopting a similar model, score of >0.9 is Poor, 0.9-1.5 is Acceptable, and 0-0.9 is considered Good. Test 8 S/N Attribute Type 5 Workplace V Good, Good, Acceptable, Poor, V Poor 6 Workforce V Good, Good, Acceptable, Poor, V Poor 7 Management V Good, Good, Acceptable, Poor, V Poor 8 Documentation V Good, Good, Acceptable, Poor, V Poor 9 Communication V Good, Good, Acceptable, Poor, V Poor 10 AFR & SR V Good, Good, Acceptable, Poor, V Poor Error: 34.1% By adopting a similar model, score of > 2.1 is V Poor, 1.5-2.1 is Poor, 0.9-1.5 is Acceptable, 0.45-0.9 is Good and 0-0.45 is considered V Good. Test 9 S/N Attribute Type 5 Workplace V Good, Good, Acceptable, Poor, V Poor 6 Workforce V Good, Good, Acceptable, Poor, V Poor 7 Management V Good, Good, Acceptable, Poor, V Poor 8 Documentation V Good, Good, Acceptable, Poor, V Poor 9 Communication V Good, Good, Acceptable, Poor, V Poor 10 AFR & SR 0-85% (Not V Good), 85-100% (V Good) Error: 30.3% Splitting to only 2 classes makes the prediction potentially more accurate. The data pool has 205 training cases. A large number of classes mean that there are less data available to predict each class. Test 10 S/N Attribute Type 5 Workplace V Good, Good, Acceptable, Poor, V Poor 6 Workforce V Good, Good, Acceptable, Poor, V Poor 7 Management V Good, Good, Acceptable, Poor, V Poor 8 Documentation V Good, Good, Acceptable, Poor, V Poor 9 Communication V Good, Good, Acceptable, Poor, V Poor 10 AFR & SR 0-85% (Not Good),85-99% (Good),100% (V

Good) Error: 34.9% From the new AFR scores, there are about 20% of the data belongs to V Good range (ie. Zero accidents). Therefore, a split of 85-100% to the mentioned 2 classes may churn a more accurate decision tree. It is because grouping class with a distinct AFR score (zero) reduces the possibility of the tree choosing randomly.

5

Test 11 S/N Attribute Type 5 Workplace V Good, Good, Acceptable, Poor, V Poor 6 Workforce V Good, Good, Acceptable, Poor, V Poor 7 Management V Good, Good, Acceptable, Poor, V Poor 8 Documentation V Good, Good, Acceptable, Poor, V Poor 9 Communication V Good, Good, Acceptable, Poor, V Poor 10 AFR & SR (0-13%, 13-42%, 43-71%, 71-99%, 100%) Error: 50.2% Firstly, a graph (shown below) of Probability Density Function was drawn. From the graph, we can divide the AFR score into 3 regions; with each region signify by its unique gradient. The middle portion of the graph has a relatively linear increment. This means that each AFR score has a similar probability of occurring, and any way of classification within this range of values will not be fruitful, as the choice of class will base on randomness.

P(X<x)

0

0.5

1

1.5

0 0.5 1 1.5 2 2.5 3 3.5

AFR score

Prob

abili

ty

P(X<x)

According to the consistently high error shown, using the decision tree methods yield some unsatisfactory results. The following are highlighted as the possible reasons for such results. LTA Data vs. MCLM Data The AFR & SR class specification given by LTA does not match with the MCLM data. E.g. For Audit ID 61, LTA class the audit as Poor, but it has zero accidents over 1223990 Man Hours worked. The full chart is shown in Appendix B for comparison. The large amount of discrepancy means that either of the data lack accuracy. Therefore, it shows that either the LTA rating is biased, or the MCLM data is incomplete. Weak Correlation After completing the tests above with unsatisfactory results, a contingency table highlighting the correlation between each attribute and AFR classes was done. Through the P values, we know that the correlation between them is weak. Assume that classification of elements has r levels and that AFR classification has c levels. We will let Oij be the observed frequency for level I of the classification of elements and level j on the AFR classification. Such a table is called r x c contingency table. We are interested in testing the hypothesis that row-and-column classifications are independent. If we rejects this hypothesis, we can conclude there is some interaction between the two criteria of classification. Let Pij be the probability that a randomly selected element falls in the ijth cell, given that a randomly selected element falls in row class I and Vj is the probability that a randomly selected

6

element falls in column class j. Now, assuming independence, the expected frequency of each cell is Eij = nUiVi. Then, for large n, the statistic has an approximate chi square distribution with (r-1)(c-1) degrees of freedom if the null hypothesis is true. Therefore, we would reject the hypothesis of independence if the observed value of the test statistic exceeds the particular percentage point of the Chi Square Distribution. Contingency Table and Chi Square tests have been done on each element vs. AFR class. The Chi Square Test hypothesis is as follows: - H0: The Element and AFR classifications are independent of each other, indicated by a P-value >= 0.05 -H1: The Element and AFR classifications has some interaction between them, indicated by a P-value < 0.05 From the results shown in Appendix C, each of ALL comparison shows that The Element and AFR classifications are independent of each other, as ALL p-values exceeded 0.05. Therefore, H0 is accepted and the Element and AFR classifications are considered independent of each other. CONCLUSION Decision Tree analysis has been used in this study to understand the general safety management performance of the selected MRT and road related projects that are under Land Transport Authority (LTA) supervision. From the analysis done, there is a high degree of difficulty in predicting the correct AFR class from the 5 elements that are given. Various tests have been used but the error generated by prediction is always larger then 30%. This study has also shown that the correlation between the AFR class and 5 elements are not strong. It was demonstrated by the Contingency Table in Appendix C. With this lack of correlation, key elements cannot be identified as factors affecting AFR on site. This implies that having a good safety management system does not mean having a good safety performance. Moreover, this study also shows that the data provided may not be reliable as there are discrepancies between the data given by LTA and MCLM data. Therefore, further verification on data�s validity could be made. ACKNOWLEDGMENT I would like to express my heartfelt gratitude to my Supervisor Associate Professor Chua Kim Huat, David, for giving me the opportunity to embark on this research; his patience and advice are invaluable through out. I would also like to thank Mr. Goh Yang Miang for imparting his deep knowledge and his constant guidance during the course of study. REFERENCE

1. The Factories Regulations 1994 Building Operations and Works of Engineering Construction (Amendment)

2. http://www.wu-wien.ac.at/taxmanagement/Institut/Mitarbeiter/Hoermann/downloads/treenote.doc