Project APOLLO/W2/AISE

Final Report on Injury Severity Assessment Module

Validation of the new ICD9-AIS extended method and calculation of patients injury severity in Europe.

Alessio Pitidis, Giuseppe Balducci Italian National Institute of Health - Department of Environment

and Primary Prevention;

Osvaldo Chiara, Stefania Cimbanassi General Hospital “Niguarda Ca’ Granda” Milan - Trauma

Team;

Stefano Calderale, Sergio Ribaldi General Hospital ”Umberto I” Rome - Emergency Department;

Maria Segui-Gomez, Rafael Heredero University of Navarra – Department of Preventive Medicine

and Public Health;

Marco Dalmasso, Carlo Mamo Piedmont Region – Inter-Territorial Epidemiologic Observatory;

Gabriella Poli, Carlo Olivari Liguria Region - Health Information System;

Adriano Murgano Abruzzo Region – Epidemiologic Observatory;

Rome November 25th 2008

1

Final Report

AIM

The final phase of the injury severity assessment module have been dedicated to the

preparation of:

1) Review Report on the state of the art in injury severity assessment

methods;

2) Selection, among different methods based on anatomical or physiological

(or their combination) measures of the patient health status, of those

indicators more efficient in the prediction of an outcome measure (i.e.

lethality rate) and applicable with less information cost to the existing

clinical documents or hospital information systems.

3) Validation of the new ICD9-AIS extended method and calculation of patients injury severity in Europe by mean of developing an automatic software application for severity assessment on current data extracted from Hospital Discharge Register databases or from other ICD-9/ICD-9-CM databases.

Our analysis starts from the consideration that a precise injury classification by type and

severity is fundamental to the study of trauma relevance, diffusion and determinants, from

both statistical/epidemiological and clinical point of view. Furthermore, classification of

injuries and individual assessment of trauma patients by severity is central to the research

of injured patients outcome evaluation.

2

THE STATE OF THE ART

The first phase of the injury severity assessment module have been dedicated to the

drafting of a review report on the state of the art of trauma severity assessment methods.

Our analysis starts from the consideration that a precise description of the injury and its

damages is necessary for clinical and epidemiological aims, more in general we can name

them evaluation purposes. The need for a common language in trauma description has led

through the years to the development of trauma dictionaries based on two fundamental

language approaches: specific for trauma and non-specific.

In the first category we can include the Abbreviated Injury Scale (AIS) method and the

assessment methodologies deriving from it, in the second group we can include the

methods based on the International Classification of Diseases (ICD) system.

The construction of a common dictionary for trauma description fulfils two fundamental

needs, on one hand it consents to describe accurately and unequivocally a determined

trauma typology, on the other hand it defines an information base for the development of

injury severity indicators. This last item in particular interests us in our analysis.

In the AIS system the severity score attribution is based on the agreement among groups

of experts. This severity scale is based essentially on the opinion of the clinicians and

other experts that developed it. It can be defined, then, a subjective iso-severity patient

classification system.

Nevertheless, theoretically such an indicator should be an objective measure of a well

determined outcome such as the survival probability or the disability level. It should be an

indicator predictive of outcome expressed as individual or population health status

measure (i.e. mortality or morbidity probability ratio) or health care productivity measure

(i.e. inpatient length of stay or average cost).

Concerning clinical outcome an injury can be described on two basic dimensions:

anatomical damage and physiological consequences.

Using a simplified scheme, the main variables determining the outcome can be included,

as Osler indicated, in the following model:

OUTCOME = f(A, F, H) (1)

3

where

A = ANATOMICAL DAMAGE

F = PHYSIOLOGICAL DAMAGE

H = GENERAL HEALTH STATUS

The concrete functional form of the model depends obviously on the measures effectively

used.

The outcome more immediate measure is the survival (death) probability.

Physiological consequences of trauma can be measured by dedicated scales such as

Revised Trauma Score (RTS).

The anatomical damage can be measured i.e. by the AIS method and its extension to the

whole patient: the Injury Severity Score (ISS).

With regard to the pre-existing health status of the patient a proxy variable generally used

is the age of the patient, in the hypothesis that that it is a fundamental component of his

general health status.

Until now we have discussed about objective or subjective methods expressed by mean

of indexes of scales considering the validity of such indicator in terms of prediction

capacity of the outcome. We investigated the relationship between severity indicator (i.e.

ISS score) and outcome measure (i.e. mortality rate), but the trauma severity level can be

directly expressed by mean of an outcome measure. As examples can be indicated at

least two methods measuring the severity of trauma directly by mean of a survival

probability.

The first one is the TRISS method (TRauma score, Injury Severity Score, age combination

index) that combines all the outcome determinants considered in the above indicated

model: ISS as anatomic measure, RTS for physiological damage, age as proxy of the

general health status of the patient (physiological reserve). The basic idea of this indicator

is using simultaneously health status measures describing different aspects of the damage

for having a greater accuracy in outcome prediction than the one obtainable by measuring

each single dimension alone.

In the TRISS method you start from anatomical and physiological measures of trauma

severity for estimating their relationship with the survival probability in a determined

population and then using that probability as measure of global injury severity.

Although survival probability can be used in itself for directly grading the different levels of

severity of a determined pathology without necessarily calculating its relationship with

4

anatomical and physiological damages. That is the Rutledge idea for ICISS (International

Classification of diseases-9 Injury Severity Score) indicator that assesses the trauma

patient severity as product of the survival probabilities calculated for each ICD-9-CM

diagnosis as survival ratio observed in a wide study population.

REVIEW OF THE STATE OF ART IN INJURY SEVERITY ASSESSMENT METHODS

INTRODUCTION

In the period from the early ‘70s up to the late ‘90s an increasing need for a common

language in trauma description has internationally led the researchers toward the

development of trauma dictionaries and methodologies useful for an accurate assessment

of trauma severity based on two fundamental language approaches: specific for trauma

and non-specific.

In the first category we can include the Abbreviated Injury Scale (AIS)1 method and the

assessment methodologies (e.g., ISS, NISS, etc.) deriving from it as well as the

assessment methods related to physiological descriptors (e.g., RTS). The second category

is represented by methods such as ICISS2, based on universal languages used for the

description of every kind of pathology e.g. the International Classification of Diseases3

system.

INJURY SEVERITY

Growth over the past two decades in the number and variety of injury severity scales

reflects recognition that severity classification is critical for surveillance, epidemiological

5

investigations and evaluations of programs and policies aimed at mitigating the impact of

injury at both the individual and social levels4,5.

Injury severity generally describes the impact of an injury in terms of the extent of tissue

damage (that is, the pathologic evidence of trauma) and/or the physiologic response of the

body to that damage. Host factors are known to mediate the effect of injury severity on

outcome, such as: age, gender, and the presence of certain pre-existing conditions,

including intoxicants.

Since many of the early indices were initially developed for either patient triage or for use

as covariates in evaluating the impact of services or systems on patient survival, these

indices defined severity in terms of a "threat to life" criterion. Given the increased survival

rates of trauma patients due to advances in injury control and emergency medical

services, the scope of evaluation research has been broadened to include studies in which

functional outcomes and disability are of increasing interest. As discussed below, current

initiatives in severity measurement are focused on developing indices that measure these

non-fatal outcomes. Interest has also been directed towards evaluating currently available

indices (or developing new ones) for predicting consumption of health care resources and

the associated costs.

Here in after a detailed analysis of the main adopted trauma severity assessment systems

specific for trauma or non specific.

A. TRAUMA-SPECIFIC APPROACHES

1. Severity Scoring Systems Based On Anatomic Descriptors

a) Abbreviated Injury Scale (AIS)

The Abbreviated Injury Scale (AIS) was introduced in 19711 and is in its 5th revision6. It is

a specialized trauma classification of injuries based mainly on anatomical descriptors of

the tissue damage caused by the injury. It was originally developed for use by

multidisciplinary vehicular crash investigators in the 1970s as a standardized injury

severity assessment tool. Subsequently, it gained acceptance with the trauma community

for clinical research application. Currently, it is generally applied directly to records in

6

specialized trauma registries. To this day, a large AIS-user population consists of non-

clinical specialists in government (other than health agencies), academia and the

automotive industry.

The AIS classification system was designed to distinguish between types of trauma of

clinical importance as well as types of trauma of interest to vehicle designers and research

engineers. It has been shown to provide a good basis for valid measurement of probability

of death. The revisions to the classification of the injuries were also designed to provide a

basis to support severity measures framed in terms other than probability of death, for

example, probability of disability or functional impairment. However, to date, there has

been little empirical data to support the notion that AIS does well in terms of probability of

disability. Predicting functional impairment or disability was an important consideration in

the 2005 Revision of the AIS classification system6.

The AIS has two components: (1) the injury descriptor (often referred to as the ‘pre-dot”

code) which is a unique numerical identifier for each injury description; and (2) the severity

score (can be referred to as the ‘post-dot’ code).

The severity score ranges from 1 (relatively minor) to 6 (currently untreatable), and is

assigned to each injury descriptor. The severity scores are consensus assessments

assigned by a group of experts and implicitly based on four criteria: threat to life,

permanent impairment, treatment period, and energy dissipation. AIS values are

considered to be well-ordered within body region, while there are documented

discrepancies of the symmetry in scores between body regions (e.g. an AIS of 4 to the

head may not be equal to an AIS of 4 to the extremities)7.

Injuries should be coded to the AIS by trained staff of trauma services or by specialists in

injury data collection. Assigning AIS scores based on medical records is time consuming;

it is not routinely done outside of trauma centers.

The AIS is a severity measure for a single injury. Many derivative severity scales have

been developed to combine multiple injuries to create a single composite score for each

patient including: Injury Severity Score (ISS)8, The New Injury Severity Score (NISS)9, the

Anatomic Profile (AP)10 and the Anatomic Profile Scale (APS)11.

AIS is the most widely used severity score based on anatomic descriptors of injury; it is the

official injury data collection tool of the National Highway Traffic Safety Administration

(NHTSA) crash investigation teams. Although the reliability and validity of the AIS (and its

derivatives for assessing the combined effect of multiple injuries) have been

7

demonstrated, their utility for population-based research is limited due to the time it takes

to assign adequate scores12.

b) Injury Severity Score (ISS)

Until recently, the Injury Severity Score (ISS)13 was the accepted method of classifying

anatomic injury severity in cases of multiple injury. The ISS is derived from the

Abbreviated Injury Scale (AIS)1, which assigns a severity score to an injury on a scale

from 1 (minor) to 6 (major). An AIS 1 injury will, generally, not require inpatient hospital

treatment, whereas an AIS 6 injury is almost certainly fatal. The ISS score takes values from 0 to 75. If an injury is assigned an AIS of 6 (unsurvivable injury),

the ISS score is automatically assigned to 75. The ISS score is virtually the only anatomical

scoring system in use and seemed to correlate linearly with mortality and morbidity rates, hospital

stay and other measures of severity8.

It's weaknesses are that any error in AIS scoring increases the ISS error, many different injury

patterns can yield the same ISS score and injuries to different body regions are not weighted. Also,

as a full description of patient injuries is not known prior to full investigation and operation, the ISS

(along with other anatomical scoring systems) is not useful as a triage tool.

For that and other reasons, in more recent years, a number of other AIS-based measures have

been suggested as alternatives to the ISS. These include the New Injury Severity Score (NISS)9,

introduced in 1997; the modified Anatomic Profile (mAP)10 and the Anatomic Profile Score (APS)11.

It has been suggested that these measures better predict the probability of a patient surviving than

does the ISS.

The AIS classification, which forms the basis of all of these approaches, requires a manual review

of the record of each patient, which is estimated to take 10 to 20 minutes12. This makes it

prohibitively expensive for the coding of injury severity in large datasets.

MacKenzie et al. made possible the application of AIS coding to large datasets by developing the

software package ICDMAP-8512, released in 1986. This provided a map for conversion of

International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM)14

diagnosis codes to AIS codes. Therefore, ISS could be calculated simply by knowing the ICD

diagnosis codes of a patient’s injuries. A revised version, ICDMAP-9015, was released in 1997 after

changes to the AIS16. To differentiate between injury severity measures based on AIS codes

derived using ICDMAP and those coded by hand, the ICDMAP-based measures have been given

the prefix “ICD/” (e.g., ICD/ISS when ISS is based on AIS codes from ICDMAP).

8

c. New Injury Severity Score (NISS) Recently, researchers have proposed a new injury severity score (NISS)9 which, unlike the ISS,

considers the three most severe injuries, regardless of body region. The NISS is computed as the

simple sum of squares of the three most severe AIS (1990 revision)16 injuries. To date, two studies

have reported that the NISS is more predictive of survival and performs better, statistically, than

the ISS9,17.

d. modified Anatomic Profile (mAP) The mAP has four separate characterizations of the patient’s injury severity: the maximum AIS

value and the square root of the summed squares of the AIS values for all serious injuries (AIS ≥ 3) in each of three body regions (head/brain and spinal cord, thorax and front of neck, all other)11.

e. Anatomic Profile Score (APS) The APS is a weighted sum of the four mAP components, defined as APS _ 0.3199 (A) _ 0.4381

(B) _0.1406 (C) _ 0.7961 (maximum AIS). These four weights come from data from level 1 trauma

centers participating in the Major Trauma Outcome Study19.

2. Severity Scoring Systems Based On Physiological Descriptors

a. Revised Trauma Score (RTS) The Revised Trauma Score (RTS) is a physiological scoring system, with high inter-rater reliability

and demonstrated accurracy in predicting death. It is scored from the first set of data obtained on

the patient, and consists of Glasgow Coma Scale, Systolic Blood Pressure and Respiratory Rate.

RTS =

0.9368 GCS + 0.7326 SBP + 0.2908 RR

Glasgow Coma Scale

(GCS)

Systolic Blood Pressure

(SBP)

Respiratory Rate

(RR)

Coded Value

13-15 >89 10-29 4

9-12 76-89 >29 3

6-8 50-75 6-9 2

4-5 1-49 1-5 1

3 0 0 0

9

Values for the RTS are in the range 0 to 7.8408. The RTS is heavily weighted towards the

Glasgow Coma Scale to compensate for major head injury without multisystem injury or major

physiological changes. A threshold of RTS < 4 has been proposed to identify those patients who

should be treated in a trauma centre, although this value may be somewhat low. The RTS

correlates well with the probability of survival19:

10

B. NON SPECIFIC APPROACHES

1. International Classification of Diseases (ICD) The International Classification of Diseases (ICD) is a general purpose classification of diagnoses

and related matters for all health conditions and includes diagnostic codes for both nature of injury

and external causes of injury. The ICD does not incorporate an explicit severity dimension. The

ICD is widely used to classify health conditions in the clinical, administrative, public health

promotion, and research settings in the U.S. and abroad. The Barell Matrix is a framework for ICD-9 CM injury codes by anatomic region and type, but does not consider relative severity20. In the

United States of America (USA), mortality data are currently coded using the 10th revision of the

ICD, ICD-1021 and morbidity data are currently coded using the clinical modification of the 9th

revision of the ICD, ICD-9-CM14. In many European countries (i.e. ltaly and other southern

European countries) mortality data are yet coded using the 9th revision of the ICD, while morbidity

data are coded using the Clinical Modification of 9th revision of the ICD (ICD-9 CM). Due to its

widespread use, there has long been interest in basing the assessment of injury severity on the

ICD taxonomy.

2. ICDMAP

One approach for using the ICD for severity assessment has been the development of software

called ICDMAP that translates ICD-9 CM coded discharge diagnoses into AIS pre-dot codes, injury

descriptors, and severity scores15. The software then uses the AIS severity scores to compute ISS,

NISS, the components of the AP and the APS for the injured individual. The mapping does result

in some loss of information due to differences in the injury classification systems. Resulting

severity scores referred to as ICD/AIS scores are considered to be conservative measures of injury

severity.

ICDMAP has been validated and shown to be useful in categorizing the severity of injuries when

only ICD rubrics are available7,12,22. However, ICDMAP is becoming out of date; there is no map

available for the updated ICD-9 CM, (or ICD-10) codes and there is no map to the AIS2005.

Both the AIS and ICDMAP are proprietary. This poses a major barrier for many potential users.

There is a strong and growing consensus that both should be non-proprietary.

Furthermore, and often because of the proprietary nature of the AIS and its map to ICD,

researchers have been developing empirically derived measures of severity based on ICD

11

diagnoses and short term outcomes such as hospital mortality coded with ICD-9 CM. Although the

development of such measures was attempted as early as the 1970s23,24, not until recently have

we had sufficiently large enough databases and the statistical tools to develop and validate this

approach to severity classification.

3. Measure Of The Severity Of Trauma Directly By Mean Of A Survival Probability

Until now we have discussed about objective or subjective methods expressed by mean

of indexes of scales considering the validity of such indicator in terms of predictive

capacity of the outcome. We investigated the relationship between severity indicator (i.e.

ISS score) and outcome measure (i.e. mortality rate), but the trauma severity level can be

directly expressed by mean of an outcome measure. As an example can be indicated at

least two methods measuring the severity of trauma directly by mean of a survival

probability. a) Trauma Score - Injury Severity Score (TRISS)

The first one is the TRISS method (TRauma score, Injury Severity Score, age combination

index) that combines all the outcome determinants considered in the above indicated

model: ISS as anatomic measure, RTS for physiological damage, age as proxy of the

general health status of the patient (physiological reserve). The basic idea of this indicator

is using simultaneously health status measures describing different aspects of the damage

for having a greater accuracy in outcome prediction than each single dimension measure

alone. In the TRISS method you start from anatomical and physiological measures of

trauma severity for estimating their relationship with the survival probability in a determined

population and then using that probability as measure of global injury severity.

TRISS25 determines the probability of survival (Ps) of a patient from the ISS and RTS

using the following formulae:

Where 'b' is calculated from:

The coefficients b0 - b3 are derived from multiple regression analysis of the Major Trauma

Outcome Study (MTOS)15 database. AgeIndex is 0 if the patient is below 54 years of age

or 1 if 55 years and over. b0 to b3 are coefficients which are different for blunt and

12

penetrating trauma. If the patient is less than 15, the blunt coefficients are used regardless

of mechanism.

The TRISS calculator

determines the

probability of survival

from the ISS, RTS

and patient's age. ISS and RTS scores can be inputted independently or calculated from

their base parameters. b) ICD-9 Injury Severity Score (ICISS) One family of empirically derived measures of injury severity based on the ICD is the ICD-9 Injury

Severity Score, referred to as ICISS. The development of this approach to injury severity

assessment is on-going and shows great promise.

ICISS, first proposed by Osler and colleagues using the North Carolina Hospital Discharge Data, is

based on the calculation of survival risk ratios (SRRs) for each ICD-9 CM code2,26. The SRR’s are

derived by dividing the number of patients that survive a given ICD-9 CM injury diagnosis code by

the number of patients with that diagnosis code. ICISS, then, is the product of the SRRs

corresponding to a patient’s set of injuries. ICISS was shown to outperform several important

competitors, including the Injury Severity Score which is based on the AIS11,27-30.

There are several problems/issues with this early direct estimation approach that have been

addressed: 1) SRRs are database-specific and it cannot be assumed that SRRs from one source

will sustain accuracy and prediction in other sets of patients; 2) SRRs calculated in this manner are

not independent and are contaminated by the presence of outcomes from other injuries in multiple

trauma cases; 3) Interactions of injuries, which are known to exist, are not accounted for. That is, in

some cases, two injuries may combine synergistically to produce a greater effect than the

combination of the two acting independently. [This is equally likely to be a problem for scoring

systems based on AIS and ICD.]

To address some of these concerns, Meredith et al. calculated SRR’s from the National Trauma

Data Bank and highlighted differences between these SRR’s and the original set from NCHDD,

underscoring the need for comparability when using SRR’s from different sources28 Other sets of

SRR’s have since been calculated from various trauma registries. Also, Meredith et al. calculated

and described a set of independent SRRs that were derived from patients with only a single

injury29. These independent SRR’s were shown to be better predictors of survival than their

traditional counterparts. The accurate measurement and incorporation of injury interactions into

trauma severity models has been elusive, though some alternatives have been proposed31.

Blunt Penetrating B0 -0.4499 -2.5355 B1 0.8085 0.9934 B2 -0.0835 -0.0651 B3 -1.7430 -1.1360

13

More recently, Kilgo et al. has shown that modelling a patient’s most severe SRR rather than the

full multiplicative ICISS score is more predictive of survival32. This “worst-injury” approach may be

superior because most trauma patients are either so minimally injured as not to pose a direct threat

to life or so severely injured that they cannot be saved. These cases compose approximately 97%

of all trauma cases and the worst-injury approach does very well for these patients.

It could be argued that the SRR is exclusively an empirical observation of hospital survival given

hospital admission, and lacks the independent descriptive value of AIS in terms of body system

involvement. The SRR may therefore be less flexible for modelling outcomes other than hospital

survival.

All measures of prediction (ICISS as well as methods based on AIS) are affected by the large

proportion of deaths that occur before hospital admission as well as the large proportion of deaths

in older patients that occur after hospital discharge. These probabilities will also change over time,

and may be affected by extraneous factors like utilization and service delivery.

One major limitation in applying the ICISS to NCHS data is the dependence of the data on the

system from which it was derived. ICISS scores could be derived from the NCHS National Hospital

Discharge Survey. While the literature has shown that the ICISS can be internally valid, potential

lack of external validity is often mentioned as a caveat. One study of interest to NCHS is the

comparison of national level data from New Zealand and Australia showing both good external and

internal validity33. This study derived SRR’s based on the Australian clinical modification of ICD10

(ICD-10-AM) 33. In the United States, experts involved in its most recent revision of the AIS were

consulted during the development of the injury chapter of ICD-10 CM. This is likely to improve both

the ICDMAP software (should a new version be developed) as well as the ICISS measures of

injury severity.

CONCLUSIONS

In conclusion we have examined two fundamental approaches for injury severity

assessment, the first one based on injury description languages specific for trauma, the

second one referred to universal languages used for the description of every kind of

pathology.

The first approach is very costly for the necessary training of specialized codifying

personnel, the complexity of clinical instrumental information to be registered, the volume

of information to collect and elaborate. The universal languages based systems have

normally a lesser information costs.

14

That difference has determined the development of two main evolution paths in the

patient injury severity classification systems.

On one side a year 1980 population study by Champion24 calculated the real death

probability for each HICDA-8 (Hospital International Classification of Diseases) codified

diagnosis by mean of the analysis of correlation among lethality and ICD based severity

scores. That study opened the way to systems based on routine clinical information

currently registered, with trauma severity levels attributed directly on the base of observed

outcomes, ICISS index is the most recent of them. Those systems have a good outcome

predictive capacity and a fundamental advantage: can be applied on current existing

clinical data.

That intrinsic advantage has led some researches to follow a second approach in injury

severity assessment, simply consisting in the development of a conversion dictionary

among ICD-9 and trauma specific severity scores such us AIS. After many attempts the

introduction of ICD-9-CM incremented the diagnosis description specificity consenting to

Mackenzie the development of an ICD-9-CM/AIS-90 conversion table (ICDMAP-90)15.

Nevertheless because of the lesser diagnosis description specificity of ICD than AIS there

is a high discordance among manually calculated and automatic (conversion table based)

severity scores for abdominal internal organs and internal head, face and neck injuries

(52% of non appropriately assigned scores).

In our study we will make a selection of indicators to be applied to European data and try

to improve existing methods.

REFERENCES

1. AIS – “Rating the severity of tissue damage: I. The Abbreviated Injury Scale”. JAMA (1971); 215:277-280.

2. Osler T, Rutledge R et al.: "ICISS: an international classification of disease-9 based injury severity score". J Trauma (1996); 41(3): 380-6; discussion 386-8.

3. WHO (1977): International Classification of Diseases 1975 (IX Conference Revision). Geneva, World Health Organization.

4. MacKenzie EJ: "Injury severity scales: overview and directions for future research". Am J Emerg Med (1984); 2(6): 537-49.

5. Rivara FP (2001). Injury control: a guide to research and program evaluation. Cambridge; New York, Cambridge University Press.

15

6. Gennarelli T, Wodzin E (2005). The Abbreviated Injury Scale - 2005. Des Plaines, Il, Association for the Advancement of Automotive Medicine.

7. Clark DE, Winchell RJ: "Risk adjustment for injured patients using administrative data". J Trauma (2004); 57(1): 130-40; discussion 140.

8. Baker SP, O’Neill B, Haddon W, Long WB: "The injury severity score: a method for describing patients with multiple injuries and evaluating emergency care". J Trauma (1974); 14(3): 187-96.

9. Osler T, Baker SP et al.: "A modification of the injury severity score that both improves accuracy and simplifies scoring". J Trauma (1997); 43(6): 922-5; discussion 925-6.

10. Copes WS, Champion HR et al.: "Progress in characterizing anatomic injury". J Trauma (1990); 30(10): 1200-7. 11. Sacco WJ, MacKenzie EJ et al.: "Comparison of alternative methods for assessing injury

severity based on anatomic descriptors". J Trauma (1999); 47(3): 441-6; discussion 446-7.

12. MacKenzie EJ, Steinwachs DM, Shankar BS: “Classifying severity of trauma based on hospital discharge diagnosis: validity of a ICD-9CM to AIS-85 conversion table”. Med Care (1989); 27:412–422.

13. Baker SP, O’Neill B, Haddon W Jr: "The Injury Severity Score: a method for describing patients with multiple injuries and evaluating emergency care". J Trauma (1974); 14:187-196;

14. United States National Center For Health Statistics. The International Classification of Diseases, Ninth Revision, Clinical Modification. Vol 1. Ann Arbor, MI: Commission on Professional and Hospital Activities; 1979.

15. MacKenzie EJ, Sacco W. ICDMAP-90: A User’s Guide. Baltimore, MD. The Johns Hopkins University School of Public Health and Tri-Analytics, Inc, 1997.

16. Association for the Advancement of Automotive Medicine. The Abbreviated Injury Scale, 1990 Revision. Des Plaines, IL: Association for the Advancement of Automotive Medicine; 1990.

17. Brenneman FD, Boulanger BR, Mclellan BA et al.: “Measuring injury severity: time for a change?” J Trauma (1998); 44:580–2

18. Champion HR, Copes WS, Sacco WJ: “Major trauma outcome study: establishing national norms for trauma care”. J Trauma (1990); 30:1356.

19. Champion HR, Sacco WJ, Copes WS et al.: “A revision of the Trauma Score”. J Trauma (1989); 29: 623-629.

20. Barell V, Aharonson-Daniel L et al.: "An introduction to the Barell body region by nature of injury diagnosis matrix". Inj Prev (2002); 8(2): 91-6.

21. World Health Organization. (1992). International statistical classification of diseases and related health problems (Tenth revision). Geneva, World Health Organization.

22. Mullins RJ, Veum-Stone J et al.: "Outcome of hospitalized injured patients after institution of a trauma system in an urban area". JAMA (1994); 271(24): 1919-24.

23. Levy PS, Mullner R et al.: "The estimated survival probability index of trauma severity". Health Serv Res (1978); 13(1): 28-35.

24. Champion HR, Sacco WJ et al.: "An anatomic index of injury severity." J Trauma (1980); 20(3): 197-202.

25. Boyd CR, Tolson MA, Copes WS: "Evaluating Trauma Care: The TRISS Method". J Trauma (1987); 27:370-378.

26. Rutledge R, Hoyt DB et al.: "Comparison of the Injury Severity Score and ICD-9 diagnosis codes as predictors of outcome in injury: analysis of 44,032 patients". J Trauma (1997); 42(3): 477-87; discussion 487-9.

27. Hannan EL, Farrell LS et al.: "Predictors of mortality in adult patients with blunt injuries in New York State: a comparison of the Trauma and Injury Severity Score (TRISS) and

16

the International Classification of Disease, Ninth Revision-based Injury Severity Score (ICISS)". J Trauma (1999); 47(1): 8-14.

28. Meredith JW, Kilgo PD et al.: "A fresh set of survival risk ratios derived from incidents in the National Trauma Data Bank from which the ICISS may be calculated". J Trauma (2003); 55(5): 924-32.

29. Meredith JW, Kilgo PD et al.: "Independently derived survival risk ratios yield better estimates of survival than traditional survival risk ratios when using the ICISS". J Trauma (2003); 55(5): 933-8.

30. Hannan EL, Waller CH et al.: "A Comparison Among the Abilities of Various Injury Severity Measures to Predict Mortality With and Without Accompanying Physiologic Information". J Trauma (2005); 58(2): 244-251.

31. West TA., Rivara FP et al.: "Harborview assessment for risk of mortality: an improved measure of injury severity on the basis of ICD-9-CM". J Trauma (2000); 49(3): 530-40; discussion 540-1.

32. Kilgo PD, Osler TM et al.: "The worst injury predicts mortality outcome the best: rethinking the role of multiple injuries in trauma outcome scoring". J Trauma (2003); 55(4): 599-606; discussion 606-7.

33. Stephenson S, Henley G et al.: "Diagnosis based injury severity scaling: investigation of a method using Australian and New Zealand hospitalisations". Inj Prev (2004); 10(6): 379-83.

17

SELECTION OF EFFICIENT INDICATORS

Information cost of severity assessment methods At the state of the art we can classify then two fundamental approaches for injury severity

assessment, the first one based on injury description languages specific for trauma, the

second one referred to universal languages used for the description of every kind of

pathology.

The first approach lead to the definition of iso-severity groups of patients and is very costly

for the necessary training of specialized codifying personnel, the complexity of clinical

instrumental information to be registered, the volume of information to collect and

elaborate.

Table 1. Workload and costs estimated for the implementation of ISS and ICISS systems

Severity assessment process ISS ICISS

Multiple trauma (up to 5 injuries)

Admission

Diagnosis and treatment

Diagnosis ICD-9-CM coding 30 Minutes 30 Minutes

Diagnosis AIS coding 30 Minutes Unnecessary

RTS+Age+ISS coding for TRISS

Comparison expected/observed outcome

Personnel unit training AIS/ISS use 20 hours/year Unnecessary

Total time estimated on 2,000 patients/

hospital/year

2,000 hours/year 1,000 hours/year

Total additional costs ISS vs. ICISS on

2,000 patients/ hospital/year

10 hours/week

$ 6,250,000 hospital/year

Total additional costs ISS implementation

in 500 trauma centers/year

$3,125,000/year

Total additional costs ISS implementation

in ,.000 ospedali/anno

$18,750,000/year

Source: adapted from Rutledge et al.

18

An estimate of those costs has been provided by Rutledge et al. (1997)1. This example

shows clearly that the information costs of a manual trauma specific classification system

such as the AIS/ISS are largely greater than the ones deriving from an automated system

based on universal classification of pathologies (and trauma).

The universal languages based systems are expected to have a lower information cost.

Severity assessment based on survival probability

The above illustrated difference has determined the development of two main evolution

paths in the patient injury severity classification systems.

On one side in a year 1980 study Champion2 calculated on a sample of 2,165 trauma

patients the real death (survival) probability for each HICDA-8 (Hospital International

Classification of Diseases) codified diagnosis by mean of the analysis of correlation among

lethality and ICD based severity scores. The method was applicable to 1,154 of them and

demonstrated a Pearson correlation index of 0,6 similar to the one calculated for the ISS.

That appreciable result opened the way to the development of iso-severity classification

systems based on the ICD codes or anyway based on routine clinical information currently

registered, with trauma severity levels attributed directly on the basis of observed

outcomes, ICISS index is the most recent of them. It can be seen as an evolution of the

systems based on the analysis of the relation among survival probability and

anatomical/physiological indicators such as the TRISS which is an extension to

penetrating injuries of the Champion’s method investigating the aforesaid relation on a

sample of 2,000 blunt trauma patients3. While the TRISS is still linked to anatomical and physiological measures in order to predict

the survival probability, the ICISS directly lies on the ground of the observed probability of

survival (death) used as indicator of the severity of trauma.

1 Rutledge R, Hoyt DB, Eastman B, Sise MJ, Velky T, Canty T, Wachtel T, Osler TM (1997): Comparison of the Injury Severity Score and ICD-9 diagnosis codes as predictor of outcome in Injury analysis of 44.032 patients. The Journal of Trauma; vol 42, 3: 477-488. 2 Champion HR, Sacco WJ, Lepper RL et al. (1980): An anatomic index of injury severity. The Journal of Trauma; 20: 197. 3 Champion HR, Sacco WJ, Carnazzo AJ et al. (1981) : Trauma Score. Critical Care Medicine; 9:672-676.

19

Rutledge et al.4 used the ICD-9-CM classification for coding the diagnoses and

interventions, grading the severity of injuries by mean of the death (survival) probability per

diagnosis/intervention group observed in a sample of 10,000 inpatients (DRR: Death Risk

Ratio). The observational variable used in this model for the prediction of survival can be

summarized as follows:

Dependent Variable Independent Variable

Survival (1=alive, 0=dead) DRR1 – main diagnosis

DRR2 – second diagnosis

DRR3 – third diagnosis

DRR4 – main surgical procedure

DRR5 – second surgical procedure

DRR6 – external cause

Because the outcome variable have been considered dichotomous, logistic regression

methods where appropriate in order to investigate the functional relation among ICD-9

diagnoses and survival, estimating the functional parameters. The outcome has been

defined as ratio between the probability of survival and the one of death: that consented

the regression analysis by mean of a logit transformation for the prediction of association

and predictive power of the independent variables. In formal terms:

ln lo( )= lnSRRp

DRRp

⎛

⎝ ⎜ ⎜

⎞

⎠ ⎟ ⎟ = b0 + bd

d =1

k

∑ xd (2)

with

k = maximum number of indipendent variables per patient (in this case k = 6)

p = 1, 2,….., n

and

4 Rutledge R, Fakhry S, Baker C, Oller D (1993): Injury severity grading in trauma patients: a simplified technique based upon ICD-9 coding. The Journal of Trauma; vol 35, 4:497-507.

20

SRRp =eb0 + bd∑ x d

1 + eb0 + bd∑ x d (3)

where

lo = survival odds

SRRp = alive patients / dead patients = E(Y|x) = survival probability of the p-th typology of

patient

DRRp = 1 - SRRp = 1 - E(Y|x) = death probability of the p-th typology of patient

xd = severity level of the d-th ICD-9 codes group graded by specific DRRd value of the group

(independent variables)

bd = coefficients of the independent variables

b0 = intercept

For each typology of patient is defined a pattern of diagnoses and procedures (surgical

and medical) each one associated with its own probability of death identifying the specific

typology of patient and determining his probability of survival. The patterns and their DRRd

if they are observed in sufficiently large sample not affected by selection biases, can be

applied to any other population of trauma patients. The theoretical basis of this approach is

frequentistic and lies on the idea that the mortality ratio for a specific trauma is stable when

the socio-economic contexts observed are similar. Because the mechanism of the

accident and the medical treatment of trauma are similar. In conclusion the mortality ratio

observed in a large population of injured people is sufficiently stable during the time and

across different territorial contexts so that it can be extrapolated to another population. The

literature reports a good prediction capacity for these systems based on the observed

survival probability as we evidenced in the review of the state of the art.

A system based on the above illustrated concepts has been developed by Osler and

Rutledge5 it is named ICISS (International Classification of diseases-9 Injury Severity

Score) and is based on the idea of working on survival probability. The development of this

5 Osler T, Rutledge R, Deis J, Bedrick E (1996): ICISS an international classification of disease-9 based Injury Severity Score. The Journal of Trauma; vol 41, 3: 380-388.

21

idea started in the precedent studies by Champion et al. and Levy & Goldberg. ICISS

expresses the severity of trauma as multiplicative effect of the survival probabilities

calculated for each diagnosis (coded with the ICD-9-CM system) on the ground of the

survival ratio observed in a large enough sample. In formal terms:

∏=

==m

ddp SRRSRRICISS

1 (4)

with 0 ≥ SRRd ≤ 1 and p = 1,…….., n

where

d = number of diagnoses per typology of patient

SRRd = alive/totale patients = E(X) = survival probability for the d-th diagnosis

SRRp = survival per typology of patient

p = typology of patient

It is evident that the severity of trauma in this case is directly expressed in function of the

survival probability per diagnosis. This indicator takes into account the ICD coded

diagnoses only, excluding the medical and surgical procedures and any other clinical

information. The implementation of ICISS is an on-going process, but it results are

showing good promises. As we reported in the review of the state of the art, apart the

results presented by its authors, the comparison studies in literature indicate it outperforms

the existing alternative methods for assessing injury severity whether based on anatomic

descriptors (such as ISS) or including physiologic information (such as TRISS).

Nevertheless there are two main concerns affecting this indicator. The first one regards

the exportability of the SRRs (Survival Risk Ratios) from the reference source to other data

sets. In other words a group of SRRs observed in the reference sample could be specific

of that population and not guarantee accuracy and prediction capacity if applied to other

data sets.

Meredith et al.6 evidenced differences among the SRRs calculated in the National Trauma

Data Bank and the correspondent ones observed in the North Carolina State Hospital

6 Meredith JW, Kilgo PD et al.: "A fresh set of survival risk ratios derived from incidents in the National Trauma Data Bank from which the ICISS may be calculated". J Trauma (2003); 55(5): 924-32. Kilgo PD, Osler TM et al.: "The worst injury predicts mortality outcome the best: rethinking the role of multiple injuries in trauma outcome scoring". J Trauma (2003); 55(4): 599-606; discussion 606-7.

22

Discharge Database (NCHD), that originally was used by Osler and Rutlegde for the

development of ICISS.

The second one concerns the effect of multiple trauma. The proposed multiplicative model

could not be appropriate to measure this effect because it does not take into account that

in multiple trauma the single SRR for ICD diagnosis is not independent from the others

because influenced by the outcomes of the other injuries. The interaction among injuries is

implicitly considered in such multiplicative model, but not separately from the independent

effect of each injury. So that that the effect of multiple trauma could be alternatively

underestimated or overestimated depending from the patterns of ICD diagnoses included

in the calculation of ICISS

Kilgo et al. have shown that modelling a patient’s most severe SRR rather than the full

multiplicative ICISS score is more predictive of survival. This “worst-injury” approach may

be superior because most trauma patients are either so minimally injured as not to pose a

direct threat to life or so severely injured that they cannot be saved. These cases compose

approximately 97% of all trauma cases and the worst-injury approach does very well for

these patients.

Finally one can argue that the consideration of diagnoses only, in the model, could not

capture the effect of factors deriving not from the need of medical assistance (clinical

condition of the patient), but on the contrary depending from the organization of the health

care services and their utilization by the people.

Despite of these concerns Rutledge et al. in a comparison study7 showed an high

correlation among ICISS scores computed on the same patients (test sample) by mean of

SRRs calculated in different reference samples. The test sample was made of 44,032

patients registered during 1985-87 in the SDTR (San Diego Trauma Registry) while the

databases in which the SSRs have been calculated were: SDTR itself, North Carolina

State Hospital Discharge Database (NCHD), North Carolina Trauma Registry Database

(NCTR) and Agency for Health Care Policy Research’s Health Care Utilization Project

Database (HCUP). The sample dimension varies from thousand cases in the trauma

registries (TRs) up to millions of observations in the Hospital Discharge Registers (HDRs),

Results in table 2 indicate a good level of association among the test sample and the other

7 Rutledge R, Hoyt DB et al.: "Comparison of the Injury Severity Score and ICD-9 diagnosiscodes as predictors of outcome in injury: analysis of 44,032 patients". J Trauma (1997); 42(3): 477-87; discussion 487-9.

23

external ones, in general good pairwise association is shown between two trauma

registries or a trauma registry and a HDR. Consequently it seems reasonable to

extrapolate SRRs from a trauma registry to another one or from a HDR to a trauma

registry (the bigger the HDR the better the extrapolation).

Table 2. Pearson R correlation coefficient – ICISS SDTR patients (n.=44,032) period

1985-87

Data Base SDTR SRR NCHD SRR NCTR SRR p < 0,0001

NCHD SRR 0,5982 Yes

NCTR SRR 0,9092 0,6617 Yes

HCUP SRR 0,8888 0,4882 0,8260 Yes

Source Osler, Rutledge et al.

In general one can assume and literature shows internal validity for SRRs at limited

distance of time from reference SRRs observation, while lack of external validity has been

mentioned as a caveat.

Nevertheless more recently a comparison study on national level data in Australia and

New Zeeland has shown both good internal and external validity8.

The concern about the assumption of a multiplicative functional relation among injuries in

multiple trauma and about the independence of survival probability for each injury

maintains its validity. The accurate measurement and incorporation of injury interactions

into trauma severity models is still elusive in the existing studies.

In order to avoid potential problem of external validity we will consider the whole trauma

inpatients in a determined country; for that reason we analyzed the National HDRs of Italy

and Spain extracting from there all the patient having at least one injury coded in the main

or secondary diagnoses. The data from these two countries will be used for calculating

SRRs for ICD-9-CM codes while the data from the Netherlands and Austria will be used for

ICD-10-CM.

We will follow the approach of “worst-injury” calculating the most severe SRR per patient

and comparing it with ICISS. The most severe SRR method appears us promising

8 Stephenson S, Henley G et al.: "Diagnosis based injury severity scaling: investigation of a method using

Australian and New Zealand hospitalisations". Inj Prev (2004); 10(6): 379-83.

24

because in over 1 million trauma inpatients observed in Italy during 2002 the 76 percent

had a single diagnosis coded and 90 percent not more than two diagnoses.

Furthermore in order to overcome the problems implicit in a multiplicative model such as

ICISS we will use a clustering method computing a global SRR per patient directly in

groups of patients having the same pattern of diagnoses (main and secondaries ). This

approach provides a global survival probability for each type of patient directly by

observing it, without assuming any hypothesis about the functional relation among the

single diagnoses forming the pattern: not a single SRR for each diagnosis, but a total SRR

for the whole pattern of diagnoses is directly calculated: SRRp computed not in

multiplicative way but for the pattern as a whole . These patterns can be then them too

aggregated when showing similar survival probabilities. In the Italian 2002 HDR trauma

records we observed 9,621 patterns of diagnoses made of at least 5 persons each one

and representing 89 percent of the whole trauma inpatients.

A serious concern regards the fact that all the trauma severity assessment methods

(based on survival probability and/or anatomical/physiological descriptors) in using

mortality for the validation of injury scaling are affected by biases due to the large

proportion of deaths that occur before hospital admission as well as the large proportion of

deaths in older patients occurring after hospital discharge. We estimate that in Italy in 2002

on the total fatalities caused by trauma at least 50 percent occurred out of hospital.

In order to take into control this issue we decided to include in the analysis mortality data

from the death certificates, being normally ICD codes used for mortality less specific than

ICD clinical modification codes used in HDR. We decided to distribute out of hospital

deaths in the same proportion within each ICD-CM category, formally:

( )11+

⋅+

=kDIS

SSRRii

ia (5)

where:

( )∑∑

+=

iii

ii

DIS

DOk

25

SRRa = Survival Risk Ratio adjusted by out of hospital mortality

Ì = i-th diagnosis code within the j-th diagnoses category

S = number survived people

DI = number dead people in hospital

DO = number dead people out of hospital

In conclusion the systems based on survival probability have a good prediction capacity of

outcome and a fundamental advantage: can be (automatically) applied on large data sets

of current existing clinical data.

Severity assessment based on anatomic descriptors

The afore said intrinsic advantage has led some researches to follow a second approach

in injury severity assessment based on anatomic descriptors, simply consisting in the

development of a conversion dictionary among ICD and trauma specific severity scores

such us AIS. After many attempts the introduction of ICD-9-CM incremented the diagnosis

description specificity consenting to Mackenzie9 the development of an ICD-9-CM/AIS-85

conversion table.

In order to validate the ICD based conversion AIS and ISS obtained by mean of the

conversion table were compared with those calculated manually by trained coders

reviewing the complete medical charts for 1,120 trauma inpatients. Almost all the observed

cases were convertible (98.2 percent) and show globally a good agreement among ISS

scores calculated with the two methods; mean values were 10.6 and 13.1 for ICD

conversion and chart review respectively. The ISS scores assigned with the two methods

were within ± 5 points for 70 percent of the cases. Grouping the ISS scores in three ranges

(1-12; 13-19; 20 +), a classification often used in evaluation and research, the two

methods of ISS scoring agreed in 68 percent of the cases. In a previous study10 average

9 MacKenzie EJ, Steinwachs DM, Shankar BS: “Classifying severity of trauma based on hospital

discharge diagnosis: validity of a ICD-9CM to AIS-85 conversion table”. Med Care (1989); 27:412–422. 10 MacKenzie EJ, Shapiro S, Eastham J et al. : Reliability of the AIS-80. Med Care (1985); 23:844.

26

pairwise agreement in ISS among 15 coders reviewing the same medical charts in was 75

percent. These global results show that while the automatic conversion is not perfect, it

provides reasonably good information on severity for large population-based data sets.

Nevertheless, because of the lesser specificity in diagnosis description for ICD with

respect to AIS, there is a high discordance among manually calculated and automatic

(conversion table based) severity scores for abdominal internal organs and internal head,

face and neck injuries (52 percent of non appropriately assigned scores for head and neck

and 46 percent for abdomen) as you can see in table 3. Even though a significant

proportion of inappropriate severity coding (10 percent on average) depends from errors in

abstracting medical charts and using ICD classification. Furthermore for many head

injuries in the MacKenzie ICD-AIS map were assigned ranges of AIS values for the same

ICD code regarding. These ranges were retained in the final ICD-AIS conversion table to

allow the user to choose the higher or lower values depending on the application.

Table 3. Observed and maximum potential agreement between MAXAIS and ISS scores assigned

by medical charts review versus ICD conversion table.

Inpatients (n=1.120) in two Trauma Centers of Maryland 1989.

Body Region N. Cases Observed agreement

(%)

Maximum potential

agreement* (%)

Head/Neck 398 48 67

Face 173 54 79

Thorax 137 63 85

Abdomen 139 54 82

Spine 94 51 79

External/Burns 420 70 72

Extremities 446 74 87

Grouped ISS 1.099 68 78

*Achievable if errors in abstracting were minimized

Source: Mackenzie et al.

27

DEVELOPMENT OF AN EXTENDED AUTOMATIC SEVERITY SCORES ASSIGNMENT ALGORITHM.

Among the systems for the classification of the inpatients we choose those based on the

clinical condition of the patient, that as you know can be divided in two groups: a) the

universal ones; b) those specific for trauma. Within the group a) we will use as afore said a

severity assessment approach based directly on the observed survival probability, while

for the group b) we adopt trauma specific severity indicators based on anatomic

descriptors. We prefer the injury anatomic standardized description approach rather than

physiological measures because the first is more easily observable and stable during the

time. In other words there is more clinical agreement in the definition of anatomic damage

and its changes during the natural course of trauma are more progressive and easily

observable.

We can describe now more precisely the aim of our study. In the model expressed in

equation (1) the outcome of a trauma is a function of three factors: the anatomic injury, the

physiological damage, the general health status of the patient. We use for the anatomic

description the AIS/ISS method; as a proxy of the health status of the patient we can use

his age which on average is a good indicator of his general conditions before the trauma.

We preferred to ignore physiological measures because more difficult to observe and

record, for RTS in example it is controversial the moment in which the physiological

parameters have to be recorded, because they change rapidly during the course of

trauma. Physiological parameters are dynamic measures by definition.

In assigning AIS scores the method of reviewing complete medical charts by trained

coders has a high information cost so that we decided to use the automatic ICD-AIS

conversion method to be applied for the last year available in Italy and Spain to the whole

national hospital discharge records with a trauma indicated as main diagnosis or within the

secondary ones.

In order to solve the problem of specificity for the scoring of head/neck and abdominal

trauma we decided to revise the ICD-AIS conversion map. The main aim of this revision

was to extend to the head/neck body region the bi-univocal correspondence between ICD

codes and AIS scores: in order to improve the automation process to each ICD code has

to correspond not a range, but only one AIS score.

28

AIS scores were assigned to all ICD-9-CM rubrics between 800.00 and 959.90 excluding

foreign body through an orifice codes (939 to 939), traumatic complications (958) and late

effects of injures (905 to 909). In the end 1.250 codes resulted bi-univocal convertible.

A panel of clinicians has been formed within the Departments of Orthopedics and

Emergency Surgery of the General Hospital of the First Sate University of Rome, surgeons

from the Trauma Team of the Niguarda General Hospital of Milan were included in the

panel. This panel has taken as reference and revised the Guide Lines for Severity

Assessment published by the Public Health Agency of Latium Region.

We omit for space reasons the entire conversion table and indicate that – except for

head/neck, gastro-intestinal tract and other particular injuries - the general conversion

criterion has been conservative: were multiple AIS scores were attributable to the same

ICD-9-CM code the lowest score has been used if the difference in the range of scores

was not greater than 1 otherwise the ICD code has been excluded from the conversion

table. For head/neck, gastro-intestinal tract and other particular injuries a specific

conversion algorithm has been developed. The algorithm for the choice of an AIS score

(within a range) per ICD code lies on the probability to observe a specific score within the

range of possible values, the score with the highest relative frequency will be chosen.

This algorithm and the conversion table process have been automated by software

procedures running in a MS-Windows environment. The computing of SRRs has also been

included in the automatic procedures. Examples of the software user friendly interfaces

are reported in the following images.

29

Fig. 1 The software application main panel

Once validated the conversion process, the national hospital discharge records are being

processed in order to be inputted in the above illustrated severity assessment software.

The complete data process flow is charted in figure 4.

In particular have been converted the diagnoses for inpatients having at least one

diagnosis of trauma (within the range above specified) among the main or secondary

diagnoses. The trauma inpatients of three Italian Regions (Piedmont, Liguria and Abruzzo)

and of the whole Spain have been considered in the year 2004. Up to six diagnoses per

patient have been examined for the Italian admissions and up to three for the Spanish

ones.

30

Fig. 2 The ICD-9 diagnoses selection panel

Fig. 3 The web style injury severity report: Maximum AIS, ISS, SRR

31

Fig. 4 AISE project data processing flow

VALIDATION OF THE CONVERSION TABLE In order to validate the ICD-AIS revised conversion table a retrospective study has been

conducted. Complete medical charts were examined by expert coders and AIS (ISS)

scores were assigned to 1,054 inpatients in two major hospitals in the metropolitan areas

of Rome and Milan. The retrospective sample was formed by 315 inpatients recorded in

1998 within the Department of Emergency Surgery and the Intensive Care Unit of the First

University General Hospital in Rome, 439 cases observed during 2003 in the Departments

of Orthopedics and Emergency Surgery and the Intensive Care Unit of the same hospital,

300 hospitalized patients recorded during 2005 in the Trauma Centre of the Niguarda

General Hospital in Milan. The expected severity of injuries in the observed hospital units

ranged from low to high severity cases in the two major metropolitan areas of Italy.

The 91% of the cases in the whole sample resulted bi-univocal convertible by our

algorithm, so that a single AIS code was assigned per each ICD-9-CM code.

The validation of the ICD-AIS revised conversion table has being conducted comparing the

AIS-ISS scores manually assigned by medical trained personnel reviewing the complete

32

medical charts for each patient in the sample with the scores automatically assigned by

the conversion software.

In order to assess the goodness of the conversion procedure we calculated the

percentage of concordant maximum AIS scores assigned by body region in the validation

sample, respectively by the manual coders in the chart review versus the automatic

conversion procedure of ICD-9-CM diagnosis codes, based on the discharge abstract

data. In the assessment of agreement (or disagreement) we assumed as reasonable the

adoption of two different criteria of validating the conversion table: the one adopted by

McKenzie et Al. 11 in their ICDMAP procedure and a broader one that we propose in this

study.

In accordance to the Mackenzie et Al. criteria, the identification of significant injuries can

rely on the definition of injuries scored as AIS > 1. Once selected the injuries for which

both methods agree on the presence of an injury, for instance as afore said, comparisons

can be made on the agreement of maximum AIS by body region (MAXAIS) or on ISS

scores per patient. We considered the ISS body regions, not the AIS ones, so that direct

comparisons cannot be made for spine injuries.

We prefer to use an approach that includes in calculations even the disagreement about

the presence of a significant injury between the manual and automatic met. So that we

include in the analysis even the effect of errors due to the different sensibility in detecting

the presence of a significant injury. In other words, assigning the value equal to 0 to a

diagnosis of trauma that we could not convert into an AIS score either by manual or

automatic methods, even those zero values have been included in the analysis of

agreement.

Furthermore, based on the definition of significant injury (AIS score > 1), we introduce a

limit of tolerance in conversion agreement between automatic and manual scores, the limit

has been fixed within +1 and -1 difference in AIS scores assigned for the same diagnosis

by the two methods. It is introduced for testing the sensitivity of the scoring procedure in

varying the conversion method from manual to automatic scoring. In conclusion in the

analysis of agreement between the two methods we suggest to consider explicitly the

possible errors due to the sensibility of the methods, but reducing by construction the level

of sensitivity, with the adoption of an interval of tolerance, in order to avoid the agreement

measure being effected by slight variations along the AIS scale: variations equal to one

11 MacKenzie EJ, Steinwachs DM et Al., cited paper.

33

point on the AIS scale; value at which, according to Mackenzie et Al. an injury is not be

considered significant.

RESULTS

We can anticipate that the obtained results indicate that the extension of the ICD-AIS map

toward a revised conversion algorithm assigning univocal AIS scores even for head/neck

and gastro-intestinal tract maintains the information validity of the conversion process,

despite the extension of bi-univocal code conversion to these body parts.

Applying the above discussed first criterion to the validation sample, in order to see if

significant injuries are detected by both methods, we obtain the following frequency of

cases in which the MAXAIS by body region did not match the definition of significant injury

(AIS > 1) in either manual or automatic methods.

Table 4. Percentage of cases in which there was a disagreement as to the presence of a

significant injury (AIS > 1) to the Body Region. Manual vs. Automatic method (n. = 1.051)

Body Region % MAXAIS manual > 1 and

MAXAIS automatic < 1

% MAXAIS manual < 1 and

MAXAIS automatic > 1

Head/Neck 35 2

Face 23 4

Thorax 21 1

Abdomen 29 5

Extremities 16 2

External/Burns 3 1

In comparing each inpatient by maximum AIS score we observe an agreement of the 49,7

percent of patients between the manual and automatic scoring. If we reduce the sensitivity

of the test by applying a limit of tolerance of + 1 or -1 AIS scores to the difference between

the two methods, the agreement rises up to 82,7 percent of cases. This assumption is

acceptable in our opinion, when you consider that, by the MacKenzie et Al. definition, an

injury must have a severity score greater than one point in order to be considered

significant.

34

The same comparison can be made by body region for MAXAISs equal or greater than

one obtaining the agreement results reported in table 5. These data show that our revised

ICD-AIS map procedure vs. the original one performs slightly better for head/neck and

face, worse for thorax, abdomen and extremities, much better for the external area.

Nevertheless only the difference in proportions of agreements for the extremities (– 10

percentage points p = 0.001) and the external area / burns (+ 28 percentage points p =

0.0000) are statistically significant, given the power of samples. It as also to be considered

that we included the spine injuries in the specific body area, while in the original study they

were considered apart.

Once adopted the above mentioned tolerance limit of 1 point of AIS score the proportions

of agreement in our sample rises up to more than 80 percent in any body region. It

depends on the fact that our manual coders tend to attribute superficial injuries to the

specific body region instead of the external area. It is reflected in the fact that in our

sample the number of cases presenting external injuries is just one third of the cases

observed in the MacKenzie et Al. sample. The tolerance limit implicitly eliminates that

effect, given that the external injuries are in general of minor severity. Even if we include in

the analysis the cases that could not be converted by the automatic procedure, while

maintaining the same tolerance limit, the agreement proportion in each body region is

greater than the one observed in the original sample.

At global level in our sample the ISS scores difference between the two methods ranged

within ± 5 points for 64.6 percent of patients (95% C.I.: 61.6 – 67.7) in comparison to the

70 percent observed by MacKenzie. Grouping the scores in the three classes used in the

above indicated original study the agreement proportion range from 76.0 percent (95%

C.I.: 72.5 – 79.2) in the ISS group 1-12 to 41.9 percent (95% C.I.: 33.8 – 50.3) in the

ISS group 13-19 and 39.5 percent (95% C.I.: 31.9 – 47.5) for ISS greater than 20.

Ranking the patients by the above indicated ISS groups the total observed agreement

between chart review and (revised-extended) conversion table rises up to 72.7 percent

(95% C.I.: 69.8 – 75.5) in comparison to the 68 percent observed by MacKenzie et al. in

their ICD-AIS conversion map. Our percent value of ISS-grouped scores agreement (72.7)

between automatic and manual method is very near to the interrater average agreement

(75 percent) among manual coders reviewing the same complete medical charts, used by

MacKenzie et al. as reference for validating the conversion table. This agreement value

could even be incremented by eliminating errors in abstracting medical charts. In the HDR

35

records we often observed an incomplete reporting of ICD codes in comparison to the

injuries reported in the complete medical charts and scored by manual coders.

Anyway having the ICD-9-CM diagnosis codes less specificity in the description of trauma

than the AIS ones the agreement could never be perfect. In the following figure we

observe in the validation sample that automatically assigned MaxAIS scores based on the

conversion of ICD-9-CM codes from discharge abstracts tend to underestimate the

severity of injury in comparison to the MaxAIS scores manually assigned by coders from

charts review. It can be noticed how the automatic AIS scores frequencies are

systematically higher than the manual ones for low severity scores and the contrary for the

intermediate and very severe ones.

Table 5. Observed agreement between MAXAIS and ISS scores assigned by medical charts review

versus ICD conversion table.

Inpatients (n=1.051) in two Major Hospitals of Italy 1998-2005.

Body Region MAXAIS >= 1 MAXAIS >= 0 and tolerance

difference +/- 1 AIS score

MAXAIS >= 1 and

tolerance difference

+/- 1 AIS score

N.

Cases

Observed

agreement (%)

N.

Cases

Observed agreement

(%)

Observed agreement

(%)

Head/Neck 244 51 405 54 84

Face 90 56 170 63 100

Thorax 240 53 337 65 82

Abdomen 122 44 202 55 84

Extremities 505 64 640 80 96

External/Burn

s

148 98 349 96 100

Grouped ISS 1.051 73

36

Fig. 5

In conclusion these results on the validation sample indicate that with the extension to

head of unequivocal ICD-AIS mapping, instead of loosing information on the severity of

injuries, we significantly improved quality of automatic versus manual scoring.

That consideration lead us to the application of the automatic conversion procedure to the

national population-based data from Hospital Discharge Registers. We obtained a bi-

univocal assignment of a single AIS score per each ICD-9-CM diagnosis on 60 percent of

the Italian inpatients (62% in Piedmont, 57% in Liguria and 61% in Abruzzo) and on 50

percent of the Spanish ones. More than 366.000 admissions have been considered for

Spain and more than 174.000 for the Italian Regions.

The sex age distribution was similar in the two populations as shown in the following

figures.

37

Fig. 6 Fig. 7

Fig. 8

The maximum AIS for each patient has been calculated; the overall MaxAIS and the ones

by body region present very similar frequency distrubutions in the two Countries. For the

overall MaxAIS the value 0 indicate patient with uncorvetible ICD-9-CM codes.

38

Fig. 9

Fig. 10

39

Fig. 11

Fig. 12

40

Fig. 13

Fig. 14

41

Fig. 15

There is a strong agreement in the two Countries among the overall MaxAIS relative

frequency distributions. We assigned the value 0 to the ICD-9-CM diagnoses uncovertible

into bi-univocal severity scores. The agreement is also strong in consideringe the MaxAIS

scores by body region. It is interesting to note how Head/Neck and Thorax show the

greatest proportions of intermediate and severe injuries in the trauma inpatients of both

countries.

In general the Spanish hospital data show a more selective style of admission for trauma

patients than the Italian ones. It is reflected in the fact that for head/neck, face and thorax

body regions the Spanish cases have a significantly greater frequency than the Italians of

intefrmediate severity injuries. It can be beter seen in the ISS distribution which has

different shape in the two countries. In Italy there is most of the cases have low severity

(more than 60% in the ISS class 0-4) whilst in Spain the main group is represented by

intermediate severity inpatients (more than 80% in the ISS classes 5-9 and 10-16).

42

Fig. 16

In order to check the applicability of our revised ICD-AIS map procedure we applied it to

10 samples of trauma inpatients in each Country. The samples consisting in 1.000 cases

each one, randomly extracted from the Hospital Discharge Register (HDR) data of the

examined Country.

The MaxAIS relative frequency distributions in the observed samples either Spanish or

Italian are more concentrated on low severity scores than the one of the validation sample,

as we expected because the validation sample has been extracted respectively from the

Emergency Departments of the biggest University General Hospital of Rome and the

Trauma Center of Milan, the largest metropolitan areas of Italy. These hospitals are highly

specialized and therefore considered second level emergency units because they attract

severe cases from the other smaller hospitals in which the patient firstly arrived.

The observation of the charts indicates us that the automatically converted samples (from

ICD-9-CM to AIS) are internally strictly concordant as to overall (per patient) MaxAIS

distribution within the same Country. In other words by examining different sets of cases,

43

casually selected and of the same dimension of the validation sample, we always obtain

the same severity distribution for trauma inpatient within the examined Country..

Fig. 17

Furthermore the severity distributions are strongly agreeing even when the samples

observed through the different Countries. Apart the difference in frequency between low

(expected higher in Italy) an intermediate severity scores ((expected higher in Spain),

because the style of admission. In other words our revised conversion procedure is

produces results that are consistent internally (within) and externally (through) to the

examined Countries

44

Fig. 18

In conclusion the revised procedure for automatic conversion of discharge abstract ICD-9-

CM coded diagnoses into AIS scores produced valid figures in comparison with the

severity scores manually assigned by trained coders, on complete medical chart review, to

inpatients of the same validation sample. Furthermore it proved high consistency of the

results when applied in different samples from two different Countries, so that its

requirements of validity and consistency are fully complied.