BMJ Open€¦ · interruptions.13 15 16 The results of these studies identified that equipment problems, telephone calls, conversation and environment problems (e.g., noise) were

For peer review only

Correlates of non-technical skills in surgery: A prospective study

Journal: BMJ Open

Manuscript ID bmjopen-2016-014480

Article Type: Research

Date Submitted by the Author: 27-Sep-2016

Complete List of Authors: Gillespie, Brigid; Griffith University, NHMRC Centre for Research Excellence in Nursing (NCREN),Centre for Health Practice Innovation (HPI) l Menzies Health Institute Qld (MHIQ) Harbeck, Emma; Griffith University, School of Applied Psychology Kang, Evelyn; Griffith University - Gold Coast Campus, NHMRC Centre for Research Excellence in Nursing (NCREN) Menzies Health Institute Queensland Steel, Catherine; Princess Alexandra Hospital, Division of surgery

Fairweather, Nicole; Princess Alexandra Hospital, Division of surgery Chaboyer, Wendy; Griffith University, National Centre of Research Excellence in Nursing Interventions for Hospitalised Patients, Centre for Health Practice Innovation, Menzies Institute for Health (Queensland)

<b>Primary Subject Heading</b>:

Surgery

Secondary Subject Heading: Communication, Anaesthesia

Keywords: miscommunications, interruptions, teamwork, non-technical skills, surgical team, NOTECHS

For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml

BMJ Open on M

arch 23, 2021 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2016-014480 on 30 January 2017. Dow

nloaded from

http://bmjopen.bmj.com/


Running header: Predictors of NOTSS in surgery

Page 1 of 24


Authors:

*1Brigid M. Gillespie PhD, RN, FACORN; 2Emma Harbeck BPsych (Hons), 1Evelyn Kang MScHlth

MHealthPrac, RN, 3Catherine Steel MN, RN;

3Nicole Fairweather FANZCA, MBBS &

1Wendy Chaboyer

PhD RN

1NHMRC Centre for Research Excellence in Nursing (NCREN), Menzies Health Institute Qld (MHIQ),

Griffith University, Parklands Drive, Gold Coast Campus QLD, AUSTRALIA

2School of Applied Psychology, Griffith University, Gold Coast Campus, QLD, AUSTRALIA

3 Division of Surgery, Princess Alexandra Hospital, Woolloongabba, Brisbane, QLD, AUSTRALIA

Email addresses:

2Emma Harbeck: [email protected]

1Evelyn Kang: [email protected]

3Catherine Steel: [email protected]

3Nicole Fairweather: [email protected]

1Wendy Chaboyer: [email protected]

*Corresponding author:

Professor Brigid Gillespie: [email protected]

G01_Rm 2.04, Griffith University

Parklands Dr, Southport

Gold Coast, Qld Australia 4222

Funding Statement

Brigid M. Gillespie acknowledges the financial support of the Australian Research Council, Early

Career Discovery Fellowship Scheme and the NHMRC Centre for Excellence in Nursing Research

(NCREN).

Page 1 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on March 23, 2021 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2016-014480 on 30 January 2017. D

ownloaded from




Page 2 of 24

Authors’ Contributions

BMG conceived of the study, assisted in data analysis, interpreted results and drafted the

manuscript. EH performed data analysis and assisted in interpretation. WC assisted in analysis and

interpretation. EK, CS, and NF assisted in recruitment and interpretation. All authors participated in

the coordination of the study and read and approved the final manuscript.

Conflict of interest

The authors have no declared conflict of interest.

Data sharing statement All relevant data have been included in the paper.

Page 2 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 3 of 24


ABSTRACT

Background: Communication and teamwork failures have frequently been identified as the

root cause of adverse events and complications in surgery. Few studies have examined

contextual factors that influence teams’ non-technical skills in surgery (NOTSS). The purpose

of this prospective study was to identify and describe correlates of NOTSS.

Methods: We assessed NOTSS of teams and professional role at two hospitals using the

revised 23-item NOTECHS and its subscales (communication, situational awareness, team

skills, leadership, and decision making). Over 6 months, two trained observers evaluated

teams’ NOTSS using a structured form. Inter-observer agreement across hospitals ranged

from 86%-95%. Multiple regression models were developed to describe associations

between operative time, team membership, miscommunications, interruptions, and total

NOTECHS and subscales scores.

Results: We observed 161 surgical procedures across eight teams. The total amount of

explained variance in NOTECHS and its five subscales ranged from 14% (adjusted R2 0.12, p <

.001) to 24% (adjusted R2

0.22, p < .001). In all models, inverse relationships between the

total number of miscommunications and total number of interruptions and teams’ NOTSS

were observed.

Conclusions: Miscommunications and interruptions impact on team NOTSS performance.

Key words: Miscommunications, interruptions, Non-technical skills, surgical team, NOTECHS,

Communication, teamwork

Page 3 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 4 of 24

Article Summary

Strengths and Limitations

• This is one of the largest of its kind in this area and included surgical procedures

observed spanned across two hospitals, and captured a wide array of surgeries

relative to complexity.

• The observational nature of this study allowed us to measure performance as it

happened, rather than a retrospective self-report.

• Individuals may have altered their practices in response to being observed during

the observational period.

• Measures upon which the observations were based may be considered somewhat

subjective as they rely on observers’ ability to interpret events. Observers,

however were experienced clinicians and were trained in observational research

and human factors.

Page 4 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 5 of 24

INTRODUCTION

Compared to other hospital settings, medical errors in the operating room (OR) can

have catastrophic consequences for patients. Adverse events and malpractice claims have

been linked to teamwork failures in surgery.1-4 Deficits in teamwork behaviours were

identified as a root cause in 63% of all the sentinel events reviewed by The Joint Commission

between 2004 and 2013.5 While human error is inevitable and cannot be completely

eliminated, the importance of linking the safety of surgery to team culture is increasingly

recognised.6-8

Fostering a climate of teamwork and collaboration, along with safety minded

work processes that focus on error prevention is the ultimate goal of healthcare

organisations.

Nevertheless, surgical errors need to be understood in the context of the surgical

team. Unique challenges stem from the overlapping but different inter-professional

expertise and roles among members, ad hoc team membership, unstructured and variable

communications, frequent distractions, technology, procedural complexity, and competing

priorities. 9-14 Several studies have described the sources and frequencies of intraoperative

interruptions.13 15 16

The results of these studies identified that equipment problems,

telephone calls, conversation and environment problems (e.g., noise) were major sources

of distractions that influenced team performance. It is therefore hardly surprising that as

much as 30% of information gets lost during case-related exchanges.8 17

More recent

research suggests that omissions in team communications related to providing members

with updates about the progress of an operation comprised up to 36% of all observed

communication errors. 18

As surgical teams often work together on an ad hoc basis, a lack

of prior working experience has the potential to impact on team dynamics. Team familiarity,

defined as a core group of individuals who work together regularly, and who share a similar

Page 5 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 6 of 24

mental model, 19 has been identified as an important element of effective teamwork.13 20 An

earlier observational study found that fewer miscommunications occurred in teams with a

history of working together.13 More recently, results of an Australian observational study

suggested a positive association between team familiarity and instrument nurses’ NOTSS

performance across 182 surgical procedures.9 Other studies, using retrospective designs

have found associations between team familiarity and reductions in postoperative

morbidity following cardiac and major abdominal surgeries.21 22

As a means to increase surgical safety, researchers have focussed on

communication, leadership, situational awareness and decision-making, termed collectively

as non-technical skills in surgery (NOTSS). NOTSS are the cognitive (i.e., decision making and

situational awareness) and interpersonal skills (i.e., communication, teamwork and

leadership) that complement the individual’s technical knowledge.23 Previous research

indicates that communication is key to the performance of successful teams. Effective and

timely transfer of information enables team processes and states such as coordination,

cooperation, conflict resolution and situational awareness.8 10 24 The development of astute

NOTSS is critical to patient safety yet surgical teams are challenged by the increasing

technical complexity of surgery and high acuity of patients, who are older, and have multiple

comorbidities.[8] Moreover there is a lack of research that examines the impact that

environmental factors have on teams’ NOTSS performance. In this prospective study, we

hypothesised that longer surgeries, limited team familiarity, miscommunications, and

interruptions negatively influenced teams’ use of NOTSS. A better understanding of the

factors that impinge on teamwork behaviours will help us to design strategies to improve

NOTSS performance.

Page 6 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 7 of 24

METHODS

This was a prospective, observational study of teams’ use of NOTSS during surgery.

Two Australian metropolitan hospitals 70 kilometres from each other, each with a similar

case mix, specialising in all surgical specialities, were included to generate results that would

be applicable across a variety of procedures. In each hospital, four surgical teams comprising

of anaesthetic and surgical consultants, their registrars, and instrument/circulating nurses,

were observed. Teams and surgical procedures across each hospital were purposively

chosen to ensure maximum variation relative to case complexity, particular procedures

within specialties, team membership, and surgical experience. In Hospital A, teams from

paediatric, thoracic, orthopaedic and general surgery were observed on a weekly basis

across 20-25 surgeries. In Hospital B, a similar number of surgeries were observed with

cardiac, vascular, upper gastro-intestinal, and hepatobiliary teams.

Observational data for each hospital were collected during 2015, with an observer

located at each hospital. Observations commenced when the patient entered the operating

room (prior to anaesthesia) and ended when the patient left the room. Both observers were

trained in human factors. To ensure methodologic consistency, interrater checks with 10%

of cases at each hospital site were performed during the observation period by the lead

author, also trained in human factors. Interrater agreement across hospital sites ranged

from 86%-95%. A single observer was present during each procedure and collected data

using pre-specified checklists and free-hand notes. During each surgical procedure,

observers documented explanatory field notes to supplement the structured observations

to better understand contextual factors. Observational data were collected in 2015 over 6-

months.

Page 7 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 8 of 24

Institutional ethics approvals were given by the participating hospitals and the

university. Participants signed a consent form and were advised of their right to

confidentiality and anonymity, and to withdraw at any time during data collection. Patients

whose operations were observed were informed of the likelihood of observations taking

place and given the chance to opt out.

Observational measures

We used the revised Non-TECHnical Skills (NOTECHS) scale, 25

which was originally

developed in the aviation industry for crew resource management. The NOTECHS provides

comprehensive behavioural descriptors for each of its subscales and so requires less training

prior to use. In surgery, it has been shown to differentiate between good and poor

behaviours, thus has demonstrated good construct validity.25 In the revised NOTECHS, five

subscales of NOTSS are assessed: A. communication and interaction; B. situational

awareness and vigilance; C. team skills; D. leadership and managerial skills; and, E. decision

making in a surgical crisis. Each domain is measured on a 7-point scale to rate each item,

with 1 = not done through to 6 = done very well, and, 0 = not applicable. 25

Total NOTECHS

scores range from 5-23 with higher scores indicative of better overall performance on all

five subscales. Scores for individual subscales were as follows; Subscales A and B scores

ranged from 4-24 while Subscales C to E scores ranged from 5-30. The “not applicable”

option meant that a specific item was not relevant or could not be rated on the basis that

the behaviour was not observed. However, participant NOTECHS scores were not affected

by a reduced score for non-observed behaviours. ‘Not applicable’ scores were replaced by

the participant’s individual item mean. In this study, as all subscales were considered of

equal importance, total NOTECHS scores were calculated by the number of items (i.e., 23) as

Page 8 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 9 of 24

the denominator. Scores for total NOTECHS and its individual subscales were calculated

using the mean of all individual team members NOTECHS total scores. We also calculated

the mean NOTECHS scores based on professional role (i.e., surgeon, anaesthetist, nurse).

In this study, team familiarity was defined as a core membership of three members

(i.e., surgeon, anaesthetist, instrument and/or circulating nurses) who had worked together,

weekly or fortnightly, for a minimum of three months. 26 Prior to commencement of each

surgical procedure, the senior nurse in the room was asked by the observer about

regularity, stability, and length of time individual team members had worked together. The

number of familiar team members for each procedure were tallied and recorded.

Miscommunications (i.e., audience, content, occasion, experience) and interruptions (i.e.,

procedural, conversational) were tallied. Operative time included the time from patient skin

preparation to the application of the final wound dressing.

Analyses

All analyses were performed using the Statistical Package for Social Sciences [SPSS]

(version 23, IBM, New York, NY, USA). Data were cleaned and a random sample of 20% was

checked for accuracy. Descriptive analysis included absolute (n) and relative (%) frequencies

to analyse categorical variables (discipline/role, surgical specialty), while means/standard

deviations [SD] or medians/interquartile range [IQR]) were used for continuous data (i.e.,

operative time, number of interruptions, miscommunications, NOTECHS scores).

Independent variables; operative time, team familiarity, number of interruptions and

miscommunications were subsequently included as covariates in simultaneous multiple

regression models with the dependent variable, NOTSS (measured by NOTECHS). A p-value

Page 9 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 10 of 24

of <.05 was considered significant and 95% confidence intervals (CI) were used. Cohen’s f2

was used to calculate effect size.

Sample size calculation

Our a priori sample size estimate was based on the 20:1 rule which states that the

ratio of the sample size to the number of parameters in a regression model should be at

least 20 cases for each predictor variable in the regression model.27 28 As 4 predictor

variables were proposed in this study, a sample size of 100 was considered sufficient in a

parsimonious regression model.

RESULTS

Across both hospital sites, a total of 161 operations were observed (Hospital A n=80;

Hospital B n=81). The number of surgeries observed for each team ranged from 20-25 with

the exception of the thoracic team. Owing to the retirement of the consultant surgeon in

the thoracic team, only 6 surgical procedures were observed in this specialty. In total, 481

individual participant’s observational data were collected (Hospital A n =243; Hospital B

n=238). The mean length of surgery across both sites was 116.3 minutes (±96.5) [Site A =

78.5 minutes, ±71.2; Site B, 153.7 minutes, ±103.8]. Across the 160 procedures we

observed, consistency in team membership ranged from 3-8 team members. Table 1 shows

case characteristics for each surgical specialty relative to number of procedures in each

specialty, operative time, team membership, and NOTECHS scores (by Subscales A-E and

mean total). Subscale E, decision-making during a surgical crisis was observed in only 40-

Page 10 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 11 of 24

50% of cases as these situations were often not observed during field work. Of the eight

teams observed, the Hepatobiliary team had the highest NOTECHS mean scores (20.7±2.3)

while the cardiac team had the lowest (19.1±3.5). Table 2 displays the descriptive results for

NOTSS performance based on professional role. Observed NOTSS performance among

surgeons and anaesthetists was comparable however, nurses scores were somewhat lower.

Page 11 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 12 of 24

Table 1: Case Characteristics (n=161 Surgical Procedures)

Surgical

Specialty

Number of

Procedures

Observed

in each

Specialty

(n/total %)

Operative

Time (mins)

Mean (SD)

Team

Member-

ship

Mdn (IQR)

*Total

NOTECHS

Scores

Mean (SD)

^Subscale A

Communi-

cation and

interaction

Mean (SD)

^Subscale B

Vigilance

/situation

awareness

Mean (SD)

^Subscale C

Team skills

Mean (SD)

^Subscale D

Leadership

and magmt

skills

Mean (SD)

^Subscale 5

Decision

making in a

crisis

Mean (SD)

General 25 (15.5) 119.5 (85.7) 4 (1) 18.7 (3.1) 19.8 (3.5) 20.0 (3.6) 23.9 (4.5) 23.8 (4.7) 24.3 (4.7)

Orthopaedic 25 (15.5) 82.3 (66.6) 4 (2) 20.3 (2.4) 21.0 (3.1) 21.8 (2.6) 25.9 (3.1) 26.7 (3.5) 26.3 (4.0)

Paediatric 25 (15.5) 35.0 (26.1) 3 (2) 20.5 (2.7) 21.5 (2.8) 21.9 (3.0) 26.1 (4.5) 26.5 (4.1) 27.1 (3.5)

Thoracic 6 (3.7) 74.1 (55.8) 4 (3) 19.6 (2.3) 21.6 (3.0) 20.8 (3.0) 25.2 (3.1) 24.6 (3.7) 25.1 (2.4)

Cardiac 20 (12.4) 234.4 (97.5) 8 (3) 18.4 (2.6) 19.1 (3.5) 20.7 (2.7) 22.8 (4.3) 22.5 (4.4) 24.8 (4.2)

Hepatobiliary 20 (12.4) 165.3 (122.2) 5 (1) 20.7 (2.3) 22.1 (2.4) 22.1 (2.5) 26.7 (3.3) 25.7 (3.8) 27.1 (4.0)

Upper GI 20 (12.4) 109.8 (78.6) 4 (2) 20.5 (2.6) 22.1 (2.9) 22.1 (2.5) 26.2 (3.7) 25.1 (4.1) 27.0 (4.7)

Vascular 20 (12.4) 105.4 (51.7) 6 (2) 20.1 (2.4) 22.1 (2.5) 21.9 (2.3) 25.3 (4.3) 23.9 (4.6) 26.6 (4.1)

Note: *Total NOTECHS Scores range 1-23; ^Subscales A and B scores in domain range 4-24, Subscales C, D and E Scores in domain range 5-30.

Page 12 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on March 23, 2021 by guest. Protected by copyright. http://bmjopen.bmj.com/ BMJ Open: first published as 10.1136/bmjopen-2016-014480 on 30 January 2017. Downloaded from




Page 13 of 24

Table 2: Descriptives of NOTSS performance based on professional role (n=481)

Surgeon

Consultant/

Registrar

Anaesthetic

Consultant/

Registrar

Scrub /

Scout Nurse

Total NOTECS

n 161 158 160

Mean 20.5 20.6 18.9

SD 2.1 2.4 3.2

95% CI 20.1-20.8 19.8-20.6 18.4-19.4

Range 14.5-23.0 11.64-23.0 10.04-23.00

Subscale A, Communication

and interaction

n 161 158 160

Mean 21.4 21.5 20.4

SD 2.8 2.87 3.7

95% CI 20.9-21.8 21.0-22.0 19.81-20.96

Range 10.0-24.0 10.0-24.0 10.00-24.00

Subscale B , Vigilance /

situational awareness

n 161 158 160

Mean 22.2 21.3 20.8

SD 2.2 2.6 3.6

95% CI 21.8-22.5 20.9-21.7 20.3-21.4

Range 16.0-24.0 11.0-24.0 8.0-24.0

Subscale C, Team skills

n 161 158 160

Mean 25.9 25.9 24.1

SD 3.5 4.0 4.6

95% CI 25.3-26.4 25.2-26.5 23.3-24.8

Range 15.0-30.0 11.00-30.0 10.0-30.0

Subscale D, Leadership and

management skills

n 161 158 160

Mean 25.5 25.5 23.8

SD 4.1 3.9 4.8

95% CI 24.9-26.2 24.9-26.1 23.0-24.6

Range 14.0-30.0 12.5-30.0 10.0-30.0

Subscale E , Decision making

in a crisis

n 161 158 160

Mean 27.5 27.0 23.6

SD 2.83 3.16 5.2

95% CI 27.1-28.0 26.6-27.6 22.8-24.4

Range 18.0-30.0 17.0-30.0 9.0-30.0

Note: *Total NOTECHS Scores range 1-23; ^Subscales A and B scores in domain range 4-24, Subscales C, D and

E Scores in domain range 5-30.

Page 13 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 14 of 24

During each surgical procedure, the observers recorded field notes to better

understand and explain the contextual happenings during assessment of teams’ NOTSS. The

following two field notes are provided as exemplars of team communications from the

highest and lowest performing teams on the NOTECHS. Ensuring that both the anaesthetic

and surgical teams had a similar mental model in relation to the procedure was important:

Prior to commencing a liver resection procedure, the Consultant and Registrar

Surgeons and the Anaesthetic Consultant participated in a detailed prebriefing

about the patient’s medical history and anticipated difficulties/challenges from

their discipline perspectives. These physicians had never worked together

before. Prebriefings between the lead surgeon and anaesthetist were

commonplace in this room and were observed to occur in 70% of the cases

observed. (Hepatobiliary: Hepatectomy, Case # 18).

The following field note illustrates an observed miscommunication between the

surgeon and perfusionist:

Consultant Surgeon to Perfusionist, “Give pledgia.”

Perfusionist: “Give another one?”

Consultant Surgeon: ”‘Have you finished with the previous one?”

Perfusionist: “Yes”. Consultant Surgeon appears to be unaware of pledgia

delivery time. There was no further inquiry from the Consultant Surgeon.

(Cardiac: CABGS x 4, Case # 9).

Across the 161 procedures, the number of miscommunications totalled 436 (Hospital A

n=133; Hospital B n=303). The highest number of miscommunications was observed in

cardiac surgery (n=121). Throughout the observed procedures, interruptions occurred in

106/161 (65.8%) cases. Of the 106 procedures where interruptions were observed,

procedural interruptions occurred at least once in 92 procedures (86.8%) (Hospital A n=118;

Page 14 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 15 of 24

Hospital B n=76). The number and types of miscommunications and interruptions for each

surgical specialty appear in Figures 1 and 2.

Multivariate regression analyses

Table 3 shows the six multiple regression models for total NOTECHS scores and its

individual subscales (A-E). The total amount of explained variance in NOTECHS and its

individual subscales ranged from 14% (adjusted R2 0.12, p < .001) to 24% (adjusted R2 0.22,

p < .001). In all six regression models, the total number of miscommunications and

interruptions were consistently significant predictors of teams’ NOTSS (Table 3). Operative

time and team membership were non-significant.

Page 15 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 16 of 24

Table 3: Regression models for predictors of total NOTSS and each NOTSS domain

(n=161 Surgical Procedures)

95% CI

Model Predictor Variable B

Std

Error β t Sig

Lower

Bound

Upper

Bound a

Team NOTECHS Constant 20.70 .38 - 55.01 <.001 19.96 21.45

Team Familiarity .01 .08 .01 .14 .893 -.15 .18

Operative Time .00 .00 .10 .97 .334 -.00 .01

Miscommunications -.27 .06 -.41 -4.82 <.001 -.38 -.16

Interruptions -.29 .12 -.19 -2.44 .016 -.52 -.05 bSubscale A

Communication and

interaction Constant 22.17 .44 - 50.18 <.001 21.30 23.05

Team Familiarity -.04 .10 -.04 -.42 .674 -.23 .15

Operative Time -.00 .00 -.07 -.66 .512 .00 .01


Interruptions -.35 .14 -.21 -2.54 .012 -.62 -.08 c Subscale B

Vigilance/situation

awareness Constant 21.76 .41 - 55.62 <.001 20.96 22.56

Team Familiarity .09 .09 .09 1.04 .299 -.08 .27

Operative Time .00 .00 .14 1.40 .163 -.00 .01


Interruptions -.36 .13 -.23 -2.86 .005 -.61 -.11 d

Subscale C

Team skills Constant 26.72 .58 - 46.19 <.001 25.58 27.87


Operative Time .00 .00 .04 .41 .686 -.01 .01


Interruptions -.30 .18 -.13 -1.65 .101 -.66 .06 e Subscale D

Leadership and

management skills Constant 26.67 .56 - 47.72 <.001 25.57 27.78

Team Familiarity -.13 .12 -.09 -1.09 .277 -.38 .11

Previous training .01 .00 .15 1.59 .115 -.00 .01


Interruptions -.21 .18 -.09 -1.23 .222 -.56 .13 f Subscale E

Decision making

in a crisis Constant 26.65 .56 - 47.92 <.001 25.55 27.75


Operative Time .01 .01 .03 .26 .793 -.01 .01


Interruptions -.45 .17 -.21 -2.58 .011 -.79 -.11

Note model results

a R=.43, R² .18, R² Adj .16, F= 8.65 df(4/160), p<.001, (f

2)=.22

Page 16 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 17 of 24

b R=.37, R² .14, R² Adj .12, F= 6.22 df(4/160), p<.001, (f

2)=.16

c R=.37, R² .14, R² Adj .12, F= 6.23 df(4/160), p<.001, (f

2)=.16

d R=.41, R² .17, R² Adj .15, F= 7.82 df(4/160), p<.001, (f

2)=.20

e R=.49, R² .24, R² Adj .22, F=12.27df(4/160), p<.001, (f

2)=.32

f R=.38, R² .15, R² Adj .12, F= 6.56 df(4/160), p<.001, (f

2)=.18

DISCUSSION

To the best of our knowledge, this is the first study to examine the correlates of

teams’ NOTSS. This study is also one of the largest single observational studies in this field.

Notably, we found inverse associations between the number miscommunications and

interruptions and team NOTSS across all NOTECHS subscales, suggesting that the fewer

miscommunications and interruptions, the higher teams’ NOTSS performance. These results

seem intuitive but, this study is the first to provide evidence generated through structured

observations conducted in real time (rather than in simulated environments). In this study,

we observed fewer interruptions as compared with miscommunications; with the highest

number of interruptions seen in the general surgery team. Many interruptions may be

considered acceptable when there are no immediate demands from patient care, but are

clearly less appropriate at busy times or when problems occur.29 Some interruptions are

essential for information sharing, or to talk to and reassure patients, but managing

interruptions and distractions is a crucial skill and requires individuals to refocus on their

primary task.13 Interruptions have identified as a major contributor to loss of vigilance in

anaesthetists.29 While teams and individuals scored reasonably highly on the NOTECHS and

its subscales, the lowest NOTSS performance was observed in relation to vigilance/situation

awareness across all teams. Clearly, miscommunications and interruptions have the

potential to erode individual and distributed situational awareness in surgery.13 29

Page 17 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 18 of 24

The hepatobiliary team had the highest NOTSS performance, as indicated by their

NOTECHS scores. The hepatobiliary team also had the lowest number of

miscommunications during the fieldwork period. In field notes, the observer described

routine preoperative discussions that occurred between physicians prior to case start, the

low levels of environmental and conversational noise, and frequent occasions of closed loop

communications between members, which heightened levels of distributed situational

awareness among team members. Taken together, these features contributed to the

smooth coordination of team tasks and patient care processes during these lists.

Conversely, the cardiac team demonstrated the lowest NOTSS performance, which

was unexpected given that this team had clearly defined roles and a small repertoire of

procedures that were ‘routine’ and well-rehearsed. Remarkably, this team also had the

greatest number of observed miscommunications during the field work period. During

observations, the cardiac team experienced considerable environmental, technological and

team-related challenges: The observer noted high noise levels in this room, attributed to

team communications and technology, e.g., cross conversations, repeated requests from

the surgeon to the perfusionist who was distracted by other team members and/or

equipment problems, incessant alarms during the intraoperative period. Additionally,

procedural and conversational interruptions as a result of the entry of external team

members into the room to ask questions, the referral of cell phone calls that occasionally

demanded the recipient to leave the room, contributed to lower observed NOTSS in the

cardiac team.


Page 18 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 19 of 24

This study has several strengths but we also recognise some limitations: First,

surgical teams’ NOTSS were evaluated through direct observation. Although most research

in this area has been largely observational and has focussed on refining this methodology, 8,

15, 16, 26, 30 individuals may have altered their practices in response to being observed.

Nevertheless contemporaneous observation is a preferred method to self-report which

could be flawed (giving rise to response bias). The observational nature of the NOTECHS

allowed us to measure performance as it happened, rather than a retrospective self-report.

Second, the measures upon which the observations were based may be considered

somewhat subjective as they rely on observers’ ability to interpret events. Yet, observers

were experienced clinicians, trained in observational research and in human factors.

Interrater consistency between observers was acceptable. Additionally, the measures we

used have been previously validated in this field. 25, 26, 30, 31 Third, surgical teams were

purposively selected based on participants’ willingness to be observed. Thus there is the

potential for selection bias. Notwithstanding, the surgical procedures observed spanned

across two hospitals, and captured a wide array of surgeries relative to complexity, and

there was variability in NOTECHS scores. Finally, in this sample the amount of explained

variance in NOTSS and its subscales while reasonable indicates that there are unknown

predictors that warrant further exploration. Despite these limitations, our results contribute

to identifying interventions that specifically target minimising miscommunications and

interruptions, both of which are modifiable with the ultimate goal of improving NOTSS in

surgery.

CONCLUSIONS

Page 19 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 20 of 24

Our observational results suggest that effective communication and interruptions

were consistent correlates of surgical teams’ NOTSS performance. Across teams, we

observed examples of good and poor NOTSS performance. Notwithstanding, these

correlates of team performance are amenable to improvement or change. Implementation

of interdisciplinary team training may contribute to improvements in NOTSS. However such

training programs need to be underpinned by behaviour change frameworks that focus on

sustained improvements in NOTSS performance. It is reasonable to propose that the

behavioural indicators of success for overall performance are transferrable across surgical

specialties and can consequently, be developed.

Words: 3,000 (including Abstract, excluding reference list)

Page 20 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 21 of 24

REFERENCES

1. Rogers S, Gawande A, Kwaan M, et al. Analysis of surgical errors in closed malpractice claims at 4

liability insurers. Surgery 2006;140(1):8.

2. Kohn L, Corrigan J, Donaldson M. To err is human: Building a safer health system. Washington:

National Academy Press, 2000.

3. Mazzocco K, Petitti DB, Fong KT, et al. Surgical team behaviors and patient outcomes. Am J Surg

2009;197(5):678-85.

4. Kable A, Gibberd R, Spigelman A. Adverse events in surgical patients in Australia. Int J Qual Health

Care 2002;14(4):269-76.

5. JCAHO. Sentinel event data: root causes by event type (2004-second quarter 2011) Galveston, TX:

University of Texas; 2011 [Available from:

http://www.utmb.edu/emergency_plan/plan/appendix/jcaho/.

6. Morgan L, Hadi M, Pickering S, et al. The effect of teamwork training on team performance and

clinical outcome in elective orthopaedic surgery: a controlled interrupted time series study.

BMJ Open 2015;5:e006216.

7. Gillespie B, Gwinner K, Chaboyer W, et al. Team communications in surgery—creating a culture of

safety. J Interprof Care 2013;27(5):287-93.

8. Lingard L, Regehr G, Cartmill C, et al. Evaluation of a preoperative team briefing: a new

communication routine results in improved clinical practice. BMJ Qual Saf 2010;doi:

10.1136/bmjqs.2009.032326

9. Kang E, Massey D, Gillespie B. Factors that influence the non-technical skills performance of scrub

nurses: A prospective study. J Adv Nurs 2016;71:2846–57.

10. Gillespie B, Marshall AP, Gardiner T, et al. The impact of workflow on the use of the Surgical

Safety Checklist: a qualitative study. ANZ J Surg 2016.

11. Sutcliffe KM, Lewton E, Rosenthal MM. Communication failures: An insidious contributor to

medical mishaps. Acad Med 2004;79(2):186-97.

12. Catchpole K. Task, team and technology integration in the paediatric cardiac operating room.

Prog Pediatr Cardiol 2011;32:85-88.

13. Gillespie B, Chaboyer W, Fairweather N. Interruptions and miscommunications in surgery: An

observational study. AORN Journal 2012;95(5):576-90.

14. Wahr JA, Prager RL, Abernathy JH, et al. Patient Safety in the Cardiac Operating Room: Human

Factors and Teamwork. A Scientific Statement From the American Heart Association

2013;128(10):1139-69.

15. Healey A, Primus C, Koutantji M. Quantifying distraction and interruption in urological surgery

Qual Saf Health Care 2007;16:135-39.

16. Sevadalis N, Healey A, Vincent C. Distracting communications in the operating theatre. J Eval Clin

Pract 2007;13:390-94.

17. Lingard L, Regehr G, Orser B, et al. Evaluation of a Pre-operative Checklist and team briefing

among surgeons, nurses, and anaesthesiologists to reduce failures in communication. Arch

Surg 2008;143(1):12-17.

18. Halverson A, Casey J, Andersson K, et al. Communication failure in the operating room. Surgery

2011;149:305-10.

19. Mathieu J, Heffner T, Goodwin G, et al. The influence of shared mental models on team process

and performance. J Appl Psychol 2000;85(2):273-83.

20. Kurmann A, Keller S, Tschan-Semmer F, et al. Impact of Team Familiarity in the Operating Room

on Surgical Complications. World J Surg 2014;38:3047–52.


on Surgical Complications. World J Surg 2014;38:3047-52.

22. ElBardissi A, ElBardissi A, Duclos A, et al. Cumulative team experience matters more than

individual surgeon experience in cardiac surgery. J Thorac Cardiovasc Surg 2013;145:328-33.

Page 21 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 22 of 24

23. Armitage-Chan EA. Human factors, non-technical skills, professionalism and flight safety: their

roles in improving patient outcome. Vet Anaesth Analg 2014;41(3):221-23.

24. Mesmer-Magnus J, DeChurch L. Information Sharing and Team Performance: A Meta-Analysis. J

Appl Psychol 2009;94:535-46.

25. Sevdalis N, Davis R, Koutantji M, et al. Reliability of a revised NOTECHS scale for use in surgical

teams. Am J Surg 2008;196:184-90.

26. Gillespie B, Chaboyer W, Fairweather N. Factors that influence the expected length of operation:

Results of a prospective study. Qual Saf Health Care 2012;21(1):3-12.

27. Polit D. Statistics and data analysis for nursing research. Second ed. Upper Saddle River: Pearson,

2010.

28. Department of Biostatistics VU. Statistical Problems to Document and to Avoid Nashville,

Tennessee: Department of Biostatistics, Vanderbilt University; 2014 [Available from:

http://biostat.mc.vanderbilt.edu/wiki/Main/ManuscriptChecklist.

29. Campbell G, Arfanis K, Smith A. Distraction and interruption in anaesthetic practice. Br J Anaesth

2012;109:707-15.

30. Healey A, Undre S, Vincent C. Developing observational measures of performance in surgical

teams. Qual Saf Health Care 2004;13:7.

31. Lingard L, Regehr G, Epsin S, et al. A theory-based instrument to evaluate team communication

in the operating room: balancing measurement authenticity and reliability. Qual Saf Health

Care 2006;15:422-26.

Page 22 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 23 of 24

Table Legend


Table 2: Descriptives of NOTSS performance based on professional role (n=481)

Table 3: Regression models for predictors of NOTSS and each NOTSS domain (n=161 Surgical

Procedures)

Figure Legend

Figure 1: Total number of miscommunications across 8 specialties

Figure 2: Total number of interruptions across 8 specialties

Page 23 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 24 of 24

Page 24 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




Figure 1: Categories and total number of miscommunications across 8 specialties in Hospitals A and B

477x207mm (150 x 150 DPI)

Page 25 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



Figure 2: Categories and total number of interruptions across 8 specialties in Hospitals A and B

466x199mm (150 x 150 DPI)

Page 26 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



Correlates of NOTSS in surgery

1

STROBE Statement—checklist of items that should be included in reports of observational studies

Item

No Recommendation

Checklist/page

Title and abstract 1 (a) Indicate the study’s design with a commonly used term in the

title or the abstract

3

(b) Provide in the abstract an informative and balanced summary

of what was done and what was found

3

Introduction

Background/rationale 2 Explain the scientific background and rationale for the

investigation being reported

5

Objectives 3 State specific objectives, including any prespecified hypotheses 6

Methods

Study design 4 Present key elements of study design early in the paper 7

Setting 5 Describe the setting, locations, and relevant dates, including

periods of recruitment, exposure, follow-up, and data collection

7

Participants 6 (a) Cohort study—Give the eligibility criteria, and the sources and

methods of selection of participants. Describe methods of follow-

up

Case-control study—Give the eligibility criteria, and the sources

and methods of case ascertainment and control selection. Give the

rationale for the choice of cases and controls

Cross-sectional study—Give the eligibility criteria, and the

sources and methods of selection of participants

7-8

(b) Cohort study—For matched studies, give matching criteria and

number of exposed and unexposed

Case-control study—For matched studies, give matching criteria

and the number of controls per case

Variables 7 Clearly define all outcomes, exposures, predictors, potential

confounders, and effect modifiers. Give diagnostic criteria, if

applicable

8

Data sources/

measurement

8* For each variable of interest, give sources of data and details of

methods of assessment (measurement). Describe comparability of

assessment methods if there is more than one group

7-8

Bias 9 Describe any efforts to address potential sources of bias 8,19

Study size 10 Explain how the study size was arrived at 10

Quantitative variables 11 Explain how quantitative variables were handled in the analyses. If

applicable, describe which groupings were chosen and why

9

Statistical methods 12 (a) Describe all statistical methods, including those used to control

for confounding

9-10

(b) Describe any methods used to examine subgroups and

interactions

10

(c) Explain how missing data were addressed 9-10

(d) Cohort study—If applicable, explain how loss to follow-up was

addressed

Case-control study—If applicable, explain how matching of cases

and controls was addressed

NA

Page 27 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




2

Cross-sectional study—If applicable, describe analytical methods

taking account of sampling strategy

(e) Describe any sensitivity analyses 9-10

Results

Participants 13* (a) Report numbers of individuals at each stage of study—eg numbers potentially

eligible, examined for eligibility, confirmed eligible, included in the study,

completing follow-up, and analysed

10-11

(b) Give reasons for non-participation at each stage

(c) Consider use of a flow diagram

Descriptive data 14* (a) Give characteristics of study participants (eg demographic, clinical, social) and

information on exposures and potential confounders

11,13

(b) Indicate number of participants with missing data for each variable of interest

(c) Cohort study—Summarise follow-up time (eg, average and total amount)

Outcome data 15* Cohort study—Report numbers of outcome events or summary measures over

time

NA

Case-control study—Report numbers in each exposure category, or summary

measures of exposure

Cross-sectional study—Report numbers of outcome events or summary measures

Main results 16 (a) Give unadjusted estimates and, if applicable, confounder-adjusted estimates

and their precision (eg, 95% confidence interval). Make clear which confounders

were adjusted for and why they were included

10

(b) Report category boundaries when continuous variables were categorized

(c) If relevant, consider translating estimates of relative risk into absolute risk for a

meaningful time period

Other analyses 17 Report other analyses done—eg analyses of subgroups and interactions, and

sensitivity analyses

11,12,16

Discussion

Key results 18 Summarise key results with reference to study objectives 17

Limitations 19 Discuss limitations of the study, taking into account sources of potential bias or

imprecision. Discuss both direction and magnitude of any potential bias

19

Interpretation 20 Give a cautious overall interpretation of results considering objectives, limitations,

multiplicity of analyses, results from similar studies, and other relevant evidence

18

Generalisability 21 Discuss the generalisability (external validity) of the study results 18

Other information

Funding 22 Give the source of funding and the role of the funders for the present study and, if

applicable, for the original study on which the present article is based

1

*Give information separately for cases and controls in case-control studies and, if applicable, for exposed and

unexposed groups in cohort and cross-sectional studies.

Note: An Explanation and Elaboration article discusses each checklist item and gives methodological background and

published examples of transparent reporting. The STROBE checklist is best used in conjunction with this article (freely

available on the Web sites of PLoS Medicine at http://www.plosmedicine.org/, Annals of Internal Medicine at

http://www.annals.org/, and Epidemiology at http://www.epidem.com/). Information on the STROBE Initiative is

available at www.strobe-statement.org.

Page 28 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Journal: BMJ Open

Manuscript ID bmjopen-2016-014480.R1


Date Submitted by the Author: 02-Nov-2016

Complete List of Authors: Gillespie, Brigid; Griffith University, School of Nursing & Midwifery, Menzies Health Institute Queensland; Gold Coast University Health Services District Harbeck, Emma; Griffith University, School of Applied Psychology Kang, Evelyn; Griffith University - Gold Coast Campus, NHMRC Centre for Research Excellence in Nursing (NCREN) Menzies Health Institute Queensland Steel, Catherine; Princess Alexandra Hospital, Division of surgery Fairweather, Nicole; Princess Alexandra Hospital, Division of surgery

Chaboyer, Wendy; Griffith University, National Centre of Research Excellence in Nursing Interventions for Hospitalised Patients, Centre for Health Practice Innovation, Menzies Institute for Health (Queensland)


Surgery




BMJ Open on M


http://bmjopen.bm

j.com/

BM



nloaded from




Page 1 of 21

Correlates of non-technical skills in surgery: A prospective study 1

ABSTRACT 2

Background: Communication and teamwork failures have frequently been identified as the 3

root cause of adverse events and complications in surgery. Few studies have examined 4

contextual factors that influence teams’ non-technical skills (NTS) in surgery. The purpose of 5

this prospective study was to identify and describe correlates of NTS. 6

Methods: We assessed NTS of teams and professional role at two hospitals using the 7

revised 23-item NOTECHS and its subscales (communication, situational awareness, team 8

skills, leadership, and decision making). Over 6 months, two trained observers evaluated 9

teams’ NTS using a structured form. Inter-observer agreement across hospitals ranged from 10

86%-95%. Multiple regression models were developed to describe associations between 11

operative time, team membership, miscommunications, interruptions, and total NOTECHS 12

and subscales scores. 13

Results: We observed 161 surgical procedures across eight teams. The total amount of 14

explained variance in NOTECHS and its five subscales ranged from 14% (adjusted R2 0.12, p < 15


0.22, p < .001). In all models, inverse relationships between the 16

total number of miscommunications and total number of interruptions and teams’ NTS were 17

observed. 18

Conclusions: Miscommunications and interruptions impact on team NTS performance. 19

20

21

Page 1 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 2 of 21

INTRODUCTION 22

Compared to other hospital settings, medical errors in the operating room (OR) can 23

have catastrophic consequences for patients. Adverse events and malpractice claims have 24

been linked to teamwork failures in surgery.1-5 Deficits in teamwork behaviours were 25

identified as a root cause in 63% of all the sentinel events reviewed by The Joint Commission 26

between 2004 and 2013.6 While human error is inevitable and cannot be completely 27

eliminated, the importance of linking the safety of surgery to team culture is increasingly 28

recognised.7-9

Fostering a climate of teamwork and collaboration, along with safety minded 29

work processes that focus on error prevention is the ultimate goal of healthcare 30

organisations. 31

Nevertheless, surgical errors need to be understood in the context of the surgical 32

team. Unique challenges stem from the overlapping but different interprofessional 33

expertise and roles among members, ad hoc team team membership, unstructured and 34

variable communications, frequent distractions, technology, procedural complexity, and 35

competing priorities.10-15 Several studies have described the sources and frequencies of 36

intraoperative interruptions.14 16 17

The results of these studies identified that equipment 37

problems, telephone calls, conversation and environment problems (e.g., noise) were major 38

sources of distractions that influenced team performance. It is therefore hardly surprising 39

that as much as 30% of information gets lost during case-related exchanges.9 18

More recent 40

research suggests that omissions in team communications related to providing members 41

with updates about the progress of an operation comprised up to 36% of all observed 42

communication errors.19

As surgical teams often work together on an ad hoc basis, a lack of 43

prior working experience has the potential to impact on team dynamics. Team familiarity, 44

defined as a core group of individuals who work together regularly, and who share a similar 45

Page 2 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 3 of 21

mental model20, has been identified as an important element of effective teamwork.14 21 An 46

earlier observational study found that fewer miscommunications occurred in teams with a 47

history of working together.14 More recently, results of an Australian observational study 48

suggested a positive association between team familiarity and instrument nurses’ NTS 49

performance across 182 surgical procedures.10

Other studies, using retrospective designs 50

have found associations between team familiarity and reductions in postoperative 51

morbidity following cardiac and major abdominal surgeries. 21 22 52

As a means to increase surgical safety, researchers have focussed on 53

communication, leadership, situational awareness and decision-making, termed collectively 54

as non-technical skills (NTS) in surgery. NTS are the cognitive (i.e., decision making and 55

situational awareness) and interpersonal skills (i.e., communication, teamwork and 56

leadership) that complement the individual’s technical knowledge.23 Previous research 57

indicates that communication is key to the performance of successful teams. Effective and 58

timely transfer of information enables team processes and states such as coordination, 59

cooperation, conflict resolution and sitational awareness.9 11 24 The development of astute 60

NTS is critical to patient safety yet surgical teams are challenged by the increasing technical 61

complexity of surgery and high acuity of patients, who are older, and have multiple 62

comorbidities.8 Moreover there is a lack of research that examines the impact that 63

environmental factors have on teams’ NTS performance. In this prospective study, we 64

hypothesised that longer surgeries, limited team familiarity, miscommunications, and 65

interruptions negatively influenced teams’ use of NTS. A better understanding of the factors 66

that impinge on teamwork behaviours will help us to design strategies to improve NTS 67

performance. 68

69

Page 3 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 4 of 21

METHODS 70

This was a prospective, observational study of teams’ use of NTS during surgery. Two 71

Australian metropolitan hospitals 70 kilometers from each other, each with a similar case 72

mix, specialising in all surgical specialities, were included to generate results that would be 73

applicable across a variety of procedures. In each hospital, four surgical teams comprising of 74

anaesthetic and surgical consultants, their registrars, and instrument/circulating nurses, 75

were observed. Teams and surgical procedures across each hospital were purposively 76

chosen to ensure maximum variation relative to case complexity, particular procedures 77

within specialties, team membership, and surgical experience. In Hospital A, teams from 78

paediatric, thoracic, orthopaedic and general surgery were observed on a weekly basis 79

across 20-25 surgeries. In Hospital B, a similar number of surgeries was observed with 80

cardiac, vascular, upper gastro-intestinal, and hepatobiliary teams. 81

Observational data for each hospital were collected during 2015, with an observer 82

located at each hospital. Prior to the observation period, both observers underwent specific 83

training in the use of the observational tool which included the NOTECHS system. The 84

observers pilot tested the tool and minor changes made to its formatting. During the 85

piloting process, regular meetings were held with the co-resesrchers to ensure greater 86

clarification of recorded events and refine coding. Both observers were trained in human 87

factors and observational research methods. To ensure methodologic consistency, 88

interrater checks with 10% of cases at each hospital site were performed during the 89

observation period by the lead author, also trained in human factors. Interrater agreement 90

across hospital sites ranged from 86%-95%. A single observer was present during each 91

procedure and collected data using pre-specified checklists and free-hand notes. 92

Observations commenced when the patient entered the operating room (prior to 93

Page 4 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 5 of 21

anaesthesia) and ended when the patient left the room. During each surgical procdure, 94

observers documented explanatory field notes to supplement the structured observations 95

to better understand contextual factors. Observational data were collected in 2015 over 6-96

months. 97

Institutional ethics approvals were given by the participating hospitals and the 98

university. Participants signed a consent form and were advised of their right to 99

confidentiality and anonymity, and to withdraw at any time during data collection. Patients 100

whose operations were observed were informed of the likelihood of observations taking 101

place and given the chance to opt out. 102

103

Observational measures 104

We used the revised Non-TECHnical Skills (NOTECHS) scale 25, which was originally 105

developed in the aviation industry for crew resource management. The NOTECHS provides 106

comprehensive behavioural descriptors for each of its subscales and so requires less training 107

prior to use. In surgery, it has been shown to differentiate between good and poor 108

behaviours, thus has demonstrated good construct validity.25

In the revised NOTECHS, five 109

subscales of NTS are assessed: A. communication and interaction; B. situational awareness 110

and vigilance; C. team skills; D. leadership and managerial skills; and, E. decision making in a 111

surgical crisis. Each domain is measured on a 7-point scale to rate each item, with 1 = not 112

done through to 6 = done very well, and, 0 = not applicable 25. Total NOTECHS scores range 113

from 5-23 with higher scores indicative of better overall performance on all five subscales. 114

Scores for individual subscales were as follows; Subscales A and B scores ranged from 4-24 115

while Subscales C to E scores ranged from 5-30. The “not applicable” option meant that a 116

specific item was not relevant or could not be rated on the basis that the behaviour was not 117

Page 5 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 6 of 21

observed. However, participant NOTECHS scores were not affected by a reduced score for 118

non-observed behaviours. ‘Not applicable’ scores were replaced by the participant’s 119

individual item mean. In this study, as all subscales were considered of equal importance, 120

total NOTECHS scores were calculated by the number of items (i.e., 23) as the denominator. 121

Scores for total NOTECHS and its individual subscales were calculated using the mean of all 122

individual team members NOTECHS total scores. We also calculated the mean NOTECHS 123

scores based on professional role (i.e., surgeon, anaesthetist, nurse). 124

In this study, we drew on the literature for definitions and measurement of the 125

observational variables relative to team familiarty, miscommunication and interruption 126

events. Team familiarity was defined as a core membership of three members (i.e., surgeon, 127

anaesthetist, instrument and/or circulating nurses) who had worked together, weekly or 128

fortnightly, for a minimum of three months. 26 Prior to commencement of each surgical 129

procedure, the senior nurse in the room was asked by the observer about regularity, 130

stability, and length of time individual team members had worked together. The number of 131

familiar team members for each procedure were tallied and recorded. We used Lingard et 132

al’s.18 27

taxonomy to classify miscommunications (i.e., audience, content, occasion, 133

experience). Interruptions were classified according to Healey et al’s.16 28 framework (i.e., 134

procedural, conversational). For each procedure, we tallied the number of 135

miscommunications and interruptions in each of their respective categories. In some 136

instances, it was possible that a single miscommunication or interruption could be placed 137

into more than one category. As such, the primary prompt of the miscommunication or 138

interruption was deamed to categorise the event. Operative time included the time from 139

patient skin preparation to the application of the final wound dressing. 140

141

Page 6 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 7 of 21

Analyses 142

All analyses were performed using the Statistical Package for Social Sciences [SPSS] 143

(version 23, IBM, New York, NY, USA). Data were cleaned and a random sample of 20% was 144

checked for accuracy. Descriptive analysis included absolute (n) and relative (%) frequencies 145

to analyse categorical variables (discipline/role, surgical specialty), while means/standard 146

deviations [SD] or medians/interquartile range [IQR]) were used for continuous data (i.e., 147

operative time, number of interruptions, miscommunications, NOTECHS scores). 148

Independent variables; operative time, team familiarity, number of interruptions and 149

miscommunications were subsequently included as covariates in simultaneous multiple 150

regression models with the dependent variable, NTS (measured by NOTECHS). A p-value of 151

<.05 was considered significant and 95% confidence intervals (CI) were used. Cohen’s f2 was 152

used to calculate effect size. 153

154

Sample size calculation 155

Our a priori sample size estimate was based on the 20:1 rule which states that the 156

ratio of the sample size to the number of parameters in a regression model should be at 157

least 20 cases for each predictor variable in the regression model.29 30 As 4 predictor 158

variables were proposed in this study, a sample size of 100 was considered sufficient in a 159

parsimonious regression model. 160

161

162

Page 7 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 8 of 21

RESULTS 163

Across both hospital sites, a total of 161 operations were observed (Hospital A n=80; 164

Hospital B n=81). The number of surgeries observed for each team ranged from 20-25 with 165

the exception of the thoracic team. Owing to the retirement of the consultant surgeon in 166

the thoracic team, only 6 surgical procedures were observed in this specialty. In total, 481 167

individual participant’s observational data were collected (Hospital A n =243; Hospital B 168

n=238). The mean length of surgery across both sites was 116.3 minutes (±96.5) [Site A = 169

78.5 minutes, ±71.2; Site B, 153.7 minutes, ±103.8]. Across the 160 procedures we 170

observed, consistency in team membership ranged from 3-8 team members. On average 171

there were 7 team members present across all procedures including 2 surgeons, 2 172

anaesthetists, and 3 nurses. Table 1 shows case characteristics for each surgical specialty 173

relative to number of procedures in each specialty, operative time, team membership, and 174

NOTECHS scores (by Subscales A-E and mean total). Subscale E, decision-making during a 175

surgical crisis was observed in only 40-50% of cases as these situations were often not 176

observed during field work. Of the eight teams observed, the Hepatobiliary team had the 177

highest NOTECHS mean scores (20.7±2.3) while the cardiac team had the lowest (19.1±3.5). 178

Table 2 displays the descriptive results for NTS performance based on professional role. 179

Observed NTS performance among surgeons and anaesthetists was comparable however, 180

nurses scores were somewhat lower. 181

During each surgical procedure, the observers recorded field notes to better 182

understand and explain the contextual happenings during assessment of teams’ NTS. The 183

following two field notes are provided as exemplars of team communications from the184

Page 8 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 9 of 21

Table 1: Case Characteristics (n=161 Surgical Procedures) 185

Surgical

Specialty

Number of

Procedures

Observed

in each

Specialty

(n/total %)

Operative

Time (mins)

Mean (SD)

Team

Member-

ship

Mdn (IQR)

*Total

NOTECHS

Scores

Mean (SD)

^Subscale A

Communi-

cation and

interaction

Mean (SD)

^Subscale B

Vigilance

/situation

awareness

Mean (SD)

^Subscale C

Team skills

Mean (SD)

^Subscale D

Leadership

and magmt

skills

Mean (SD)

^Subscale E

Decision

making in a

crisis

Mean (SD)

General 25 (15.5) 119.5 (85.7) 4 (1) 18.7 (3.1) 19.8 (3.5) 20.0 (3.6) 23.9 (4.5) 23.8 (4.7) 24.3 (4.7)

Orthopaedic 25 (15.5) 82.3 (66.6) 4 (2) 20.3 (2.4) 21.0 (3.1) 21.8 (2.6) 25.9 (3.1) 26.7 (3.5) 26.3 (4.0)

Paediatric 25 (15.5) 35.0 (26.1) 3 (2) 20.5 (2.7) 21.5 (2.8) 21.9 (3.0) 26.1 (4.5) 26.5 (4.1) 27.1 (3.5)

Thoracic 6 (3.7) 74.1 (55.8) 4 (3) 19.6 (2.3) 21.6 (3.0) 20.8 (3.0) 25.2 (3.1) 24.6 (3.7) 25.1 (2.4)

Cardiac 20 (12.4) 234.4 (97.5) 8 (3) 18.4 (2.6) 19.1 (3.5) 20.7 (2.7) 22.8 (4.3) 22.5 (4.4) 24.8 (4.2)

Hepatobiliary 20 (12.4) 165.3 (122.2) 5 (1) 20.7 (2.3) 22.1 (2.4) 22.1 (2.5) 26.7 (3.3) 25.7 (3.8) 27.1 (4.0)

Upper GI 20 (12.4) 109.8 (78.6) 4 (2) 20.5 (2.6) 22.1 (2.9) 22.1 (2.5) 26.2 (3.7) 25.1 (4.1) 27.0 (4.7)

Vascular 20 (12.4) 105.4 (51.7) 6 (2) 20.1 (2.4) 22.1 (2.5) 21.9 (2.3) 25.3 (4.3) 23.9 (4.6) 26.6 (4.1)

Note: *Total NOTECHS Scores range 1-23; ^Subscales A and B scores in domain range 4-24, Subscales C, D and E Scores in domain range 5-30. 186

187

188

189

Page 9 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960





Page 10 of 21

Table 2: Descriptives of NOTECHS performance based on professional role (n=481) 190

Surgeon

Consultant/

Registrar

Anaesthetic

Consultant/

Registrar

Scrub /

Scout Nurse

Total NOTECS

n 161 158 160

Mean 20.5 20.6 18.9

SD 2.1 2.4 3.2

95% CI 20.1-20.8 19.8-20.6 18.4-19.4

Range 14.5-23.0 11.64-23.0 10.04-23.00


and interaction

n 161 158 160

Mean 21.4 21.5 20.4

SD 2.8 2.87 3.7

95% CI 20.9-21.8 21.0-22.0 19.81-20.96

Range 10.0-24.0 10.0-24.0 10.00-24.00



n 161 158 160

Mean 22.2 21.3 20.8

SD 2.2 2.6 3.6

95% CI 21.8-22.5 20.9-21.7 20.3-21.4

Range 16.0-24.0 11.0-24.0 8.0-24.0


n 161 158 160

Mean 25.9 25.9 24.1

SD 3.5 4.0 4.6

95% CI 25.3-26.4 25.2-26.5 23.3-24.8

Range 15.0-30.0 11.00-30.0 10.0-30.0


management skills

n 161 158 160

Mean 25.5 25.5 23.8

SD 4.1 3.9 4.8

95% CI 24.9-26.2 24.9-26.1 23.0-24.6

Range 14.0-30.0 12.5-30.0 10.0-30.0


in a crisis

n 161 158 160

Mean 27.5 27.0 23.6

SD 2.83 3.16 5.2

95% CI 27.1-28.0 26.6-27.6 22.8-24.4

Range 18.0-30.0 17.0-30.0 9.0-30.0

Note: *Total NOTECHS Scores range 1-23; ^Subscales A and B scores in domain range 4-24, Subscales C, D and 191

E Scores in domain range 5-30. 192

Page 10 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 11 of 21

highest and lowest performing teams on the NOTECHS. Ensuring that both the anaesthetic 193

and surgical teams had a similar mental model in relation to the procedure was important: 194

Prior to commencing a liver resection procedure, the Consultant and Registrar 195

Surgeons and the Anaesthetic Consultant participated in a detailed prebriefing 196

about the patient’s medical history and anticipated difficulties/challenges from 197

their discipline perspectives. These physicians had never worked together 198

before. Prebriefings between the lead surgeon and anaesthetist were 199

commonplace in this room and were observed to occur in 70% of the cases 200

observed. (Hepatobiliary: Hepatectomy, Case # 18). 201

The following fieldnote illustrates an observed miscommunication between the 202

surgeon and perfusionist: 203

Consultant Surgeon to Perfusionist, “Give pledgia.” 204

Perfusionist: “Give another one?” 205

Consultant Surgeon: ”‘Have you finished with the previous one?” 206

Perfusionist: “Yes”. Consultant Surgeon appears to be unaware of pledgia 207

delivery time. There was no further inquiry from the Consultant Surgeon. 208

(Cardiac: CABGS x 4, Case # 9). 209

Across the 161 procedures, the number of miscommunications totalled 436 (Hospital A 210

n=133; Hospital B n=303). The highest number of miscommunications was observed in 211

cardiac surgery (n=121). Throughout the observed procedures, interruptions occurred in 212

106/161 (65.8%) cases. Of the 106 procedures where interruptions were observed, 213

procedural interruptions occurred at least once in 92 procedures (86.8%) (Hospital A n=118; 214

Hospital B n=76). The number and types of miscommunications and interruptions for each 215

surgical specialty appear in Figures 1 and 2. 216

217

Page 11 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 12 of 21

Multivariate regression analyses 218

Table 3 shows the six multiple regression models for total NOTECHS scores and its 219

individual subscales (A-E). The total amount of explained variance in NOTECHS and its 220

individual subscales ranged from 14% (adjusted R2 0.12, p < .001) to 24% (adjusted R2 0.22, 221

p < .001). In all six regression models, the total number of miscommunications and 222

interruptions were consistently significant predictors of teams’ NTS (Table 3). Operative 223

time and team membership were non-significant. 224

225

DISCUSSION 226

To the best of our knowledge, this is the first study to examine the correlates of 227

teams’ NTS. This study is also one of the largest single observational studies in this field. 228

Notably, we found inverse associations between the number miscommunications and 229

interruptions and team NTS across all NOTECHS subscales, suggesting that the fewer 230

miscommunications and interruptions there are, the higher teams’ NTS performance. These 231

results seem intuitive but, this study is the first to provide evidence generated through 232

structured observations conducted in real time (rather than in simulated environments). In 233

this study, we observed fewer interruptions as compared with miscommunications; with the 234

highest number of interruptions seen in the general surgery team. Many interruptions may 235

be considered acceptable when there are no immediate demands from patient care, but are 236

clearly less appropriate at busy times or when problems occur.31 Some interruptions are 237

essential for information sharing, or to talk to and reassure patients, but managing 238

interruptions and distractions is a crucial skill and requires individuals to refocus on their 239

Page 12 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 13 of 21

Table 3: Regression models for predictors of total NOTECHS and each NOTECHS domain (n=161 Surgical Procedures) 240

95% CI


Std

Error β t Sig

Lower

Bound

Upper

Bound a



Operative Time .00 .00 .10 .97 .334 -.00 .01



Communication and



Operative Time -.00 .00 -.07 -.66 .512 .00 .01



Vigilance/situation



Operative Time .00 .00 .14 1.40 .163 -.00 .01


Interruptions -.36 .13 -.23 -2.86 .005 -.61 -.11 d

Subscale C



Operative Time .00 .00 .04 .41 .686 -.01 .01


Interruptions -.30 .18 -.13 -1.65 .101 -.66 .06

Page 13 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960





Page 14 of 21

e Subscale D

Leadership and






Decision making



Operative Time .01 .01 .03 .26 .793 -.01 .01


Interruptions -.45 .17 -.21 -2.58 .011 -.79 -.11

Note model results 241 a

R=.43, R² .18, R² Adj .16, F= 8.65 df(4/160), p<.001, (f2)=.22

b R=.37, R² .14, R² Adj .12, F= 6.22 df(4/160), p<.001, (f

2)=.16

c R=.37, R² .14, R² Adj .12, F= 6.23 df(4/160), p<.001, (f

2)=.16

d R=.41, R² .17, R² Adj .15, F= 7.82 df(4/160), p<.001, (f

2)=.20

e R=.49, R² .24, R² Adj .22, F=12.27df(4/160), p<.001, (f

2)=.32

f R=.38, R² .15, R² Adj .12, F= 6.56 df(4/160), p<.001, (f

2)=.18

Page 14 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960





Page 15 of 21

primary task.14 Interruptions have identified as a major contributor to loss of

vigilance in anaesthetists.31

While teams and individuals scored reasonably highly on the

NOTECHS and its subscales, the lowest NTS performance was observed in relation to

vigilence/situation awareness across all teams. Clearly, miscommunications and

interruptions have the potential to erode individual and distributed situational awareness in

surgery.14 31

The hepatobiliary team had the highest NTS performance, as indicated by their








Conversely, the cardiac team demonstrated the lowest NTS performance, which was

unexpected given that this team had clearly defined roles and a small repertoire of


greatest number of observed miscommunications during the field work period. Notably, the

degree of difficulty and complexity, technical skills, stress, and patients’ instability and

acuity may be highest in cardiac surgery.13 32 Observer described (in field notes) the

considerable environmental, technological and team-related challenges experienced by the

cardiac during the surgery, which added to case complexity. For instance, the high noise

levels in this room, attributed to team communications and technology, e.g., cross

Page 15 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 16 of 21

conversations, repeated requests from the surgeon to the perfusionist who was distracted

by other team members and/or equipment problems, incessant alarms during the

intraoperative period. Additionally, procedural and conversational interruptions as a result

of the entry of external team members into the room to ask questions, the referral of cell

phone calls that occasionally demanded the recipient to leave the room, contributed to

lower observed NTS in the cardiac team.

Although we had good sampling across surgical specialties and procedures, it is

difficult to speculate about whether the differences in NTS performance can be attributed

to hospital sites, specialties, surgical teams, or individuals. The two hospital sites chosen

were similar in relation to case mix, patient acuity and surgical activity. However the

selection of specialties varied in each hospital, which may in part, explain the differences in

NTS we observed across teams. The observed diffierences may also be attributed to

particular individuals, i.e., good leadership of the consultant surgeon has been linked with

effective team behaviour and task accomplishment.33 Arguably, surgeons may establish

aspects of leadership prior to the start of the procedure to condition intraoperative team

performance. For instance, using the surgical safety checklist or having a team briefing can

contribute to building the team’s shared mental model, and hence increasing distributed

situational awareness.34


This study has several strengths but we also recognise some limitations: First, while

we found relationships between miscommunications, interruptions and surgical teams’ NTS,

Page 16 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 17 of 21

temporal order and causality cannot be established. Thus there may be some competing

explanations for these results. Notwithstanding, the design does allow statistical

associations and identification of some potential confounders; but, not all have necessarily

been identified. Second, surgical teams’ NTS were evaluated through direct observation.

Although most research in this area has been largely observational and has focussed on

refining this methodology 9 16 17 26 35, individuals may have altered their practices in response

to being observed. Nevertheless contemporaneous observation is a preferred method to

self report which could be flawed (giving rise to response bias). The observational nature of

the NOTECHS allowed us to measure performance as it happened, rather than a

retrospective self report. Third, the measures upon which the observations were based may

be considered somewhat subjective as they rely on observers’ ability to interpret events.

Yet, observers were experienced OR nurses, trained in observatimal research and in human

factors. Interrater consistency between observers was acceptable. Additionally, the

measures we used have been previously validated in this field. 25-27 35 Fourth, surgical teams

were purposively selected based on participants’ willingness to be observed. Thus there is

the potential for selection bias. Notwithstanding, the, and there was variability in NOTECHS

scores. Finally, in this sample the amount of explained variance in NTS and its subscales

while reasonable, indicates that there are unknown predictors that warrant further

exploration. Despite these limitations, our results contribute to identifying interventions

that specifically target minimising miscommunications and interruptions, both of which are

modifiable with the ultimate goal of improving NTS in surgery.

CONCLUSIONS

Page 17 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 18 of 21

Our observational results suggest that effective communication and interruptions

were consistent corrrelates of surgical teams’ NTS performance. Across teams, we observed

examples of good and poor NTS performance. Notwithstanding, these correlates of team

performance are amenable to improvement or change. Implementation of interdisciplinary

team training may contribute to improvements in NTS. However such training programs

need to be underpinned by behaviour change frameworks that focus on sustained

improvements in NTS performance. It is reasonable to propose that the behavioural

indicators of success for overall performance are transferrable across surgical specialties

and can consequently, be developed.

Words: 3,389

Page 18 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 19 of 21

REFERENCES





3. Mazzocco K, Petitti DB, Fong KT, et al. Surgical team behaviors and patient outcomes. The

American Journal of Surgery 2009;197(5):678-85.

4. Kable A, Gibberd R, Spigelman A. Adverse events in surgical patients in Australia. International

Journal of Quality in Health Care 2002;14(4):269-76.

5. Raman J, Leveson N, Samost A, et al. When a checklist is not enough: How to improve them and

what else is needed. The Journal of thoracic and cardiovascular surgery 2016;152(2):585-92.

6. JCAHO. Sentinel event data: root causes by event type (2004-second quarter 2011). Secondary

Sentinel event data: root causes by event type (2004-second quarter 2011) 2011.




BMJ Open 2015;5:e006216.


safety. J Interprofessional Care 2013;27(5):287-93.



10.1136/bmjqs.2009.032326

10. Kang E, Massey D, Gillespie B. Factors that influence the non-technical skills performance of

scrub nurses: A prospective study. J Adv Nurs 2016;71:2846–57.








observational study. AORN Jounral 2012;95(5):576-90.



2013;128(10):1139-69.




Pract 2007;13:390-94.



Surg 2008;143(1):12-17.


2011;149:305-10.




on Surgical Complications. World J Surg 2014;38:3047–52.

Page 19 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 20 of 21


individual surgeon experience in cardiac surgery. he Journal of thoracic and cardiovascular

surgery 2013;145:328-33.


roles in improving patient outcome. Veterinary Anaesthesia And Analgesia 2014;41(3):221-

23.

24. Mesmer-Magnus J, DeChurch L. Information Sharing and Team Performance: A Meta-Analysis.

Journal of Applied Psychology 2009;94:535-46.


teams. The American Journal of Surgery 2008;196:184-90.


Results of a prospective study. Quality & Safety in Healthcare 2012;21(1):3-12.



Care 2006;15:422-26.

28. Healey A, Sevadalis N, Vincent C. Measuring intra-operative interference from distraction and

interruption observed in the operating theatre. Ergonomics 2006;49(5-6):589-604.


2010.

30. Department of Biostatistics VU. Statistical Problems to Document and to Avoid. Secondary

Statistical Problems to Document and to Avoid 2014.


31. Campbell G, Arfanis K, Smith A. Distraction and interruption in anaesthetic practice. British

Journal of Anaesthesia 2012;109:707-15.

32. Gurses A, Kim G, Martinez E, et al. Identifying and categorising patient safety hazards in

cardiovascular operating rooms using an interdisciplinary approach: a multisite study. BMJ

Qual Saf 2012;21:810-18.

33. Siu J, Maran N, Patterson-Brown S. Observation of behavioural markers of nontechnical skills in

the operating room and their relationship to intra-operative incide. The Surgeon 2014.

34. Henrickson Parker S, Flin R, McKinley A, et al. Factors Influencing Surgeons’ Intraoperative

Leadership: Video Analysis of Unanticipated Events in the Operating Room. World J Surg

2014;38:4-10.



Page 20 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 21 of 21

Table Legend


Table 2: Descriptives of NOTECHS performance based on professional role (n=481)

Table 3: Regression models for predictors of NOTECHS and each NOTECHS domain (n=161

Surgical Procedures)

Figure Legend



Page 21 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



Figure 1

477x207mm (300 x 300 DPI)

Page 22 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



Figure 2

466x199mm (300 x 300 DPI)

Page 23 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




1


Item

No Recommendation

Checklist/page



3



3

Introduction



5


Methods




7



up






7-8







applicable

8

Data sources/

measurement




7-8





9


for confounding

9-10


interactions

10



addressed



NA

Page 24 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




2




Results




10-11





11,13




time

NA







10






11,12,16

Discussion




19



18


Other information



1








Page 25 of 25


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Journal: BMJ Open

Manuscript ID bmjopen-2016-014480.R2


Date Submitted by the Author: 05-Dec-2016

Complete List of Authors: Gillespie, Brigid; Griffith University, National Center of Research Excellence in Nursing (NCREN), Menzies Health Institute Queensland, School of Nursing and Midwifery; Gold Coast University Hospital, Gold Coast Hospital and Health Service, Nursing and Midwifery Education and Research Unit Harbeck, Emma; Griffith University, School of Applied Psychology Kang, Evelyn; Griffith University - Gold Coast Campus, National Centre of Research Excellence in Nursing, Menzies Health Institute Queensland, School of Nursing and Midwifery

Steel, Catherine; Princess Alexandra Hospital, Division of surgery Fairweather, Nicole; Princess Alexandra Hospital, Division of surgery Chaboyer, Wendy; Griffith University, National Center of Research Excellence in Nursing (NCREN), Menzies Health Institute Queensland, School of Nursing and Midwifery,


Surgery




BMJ Open on M


http://bmjopen.bm

j.com/

BM



nloaded from




Page 1 of 26


Authors: 2

1,2,3*Brigid M. Gillespie PhD, RN, FACORN; 4Emma Harbeck BPsche (Hons), 1Evelyn Kang 3

MHth (Hons) RN, 5Catherine Steel MN, RN; 5Nicole Fairweather FANZCA, MBBS; & 3Wendy 4

Chaboyer PhD RN 5

6

7

1School of Nursing and Midwifery, Griffith University 8

2 Gold Coast University Hospital, Gold Coast Hospital and Health Service 9

3 National Centre for Research Excellence in Nursing (NCREN), Menzies Health Institute Qld 10

(MHIQ), Griffith University, Parklands Drive, Gold Coast Campus QLD, AUSTRALIA 11

4 School of Applied Psychology, Griffith University, Gold Coast Campus, QLD, AUSTRALIA 12

5 Division of Surgery, Princess Alexandra Hospital, Woolloongabba, Brisbane, QLD, 13

AUSTRALIA 14

15

Email addresses: 16

4Emma Harbeck: [email protected] 17

3Evelyn Kang: [email protected] 18

5Catherine Steel: [email protected] 19

5Nicole Fairweather: [email protected] 20

3Wendy Chaboyer: [email protected] 21

22

23

*Corresponding author: 24

Page 1 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 2 of 26

1,2,3 Professor Brigid Gillespie: [email protected] 25

G01_Rm 2.04, Griffith University 26

Parklands Dr, Southport 27

Gold Coast, Qld Australia 4222 28

Tel: + 61 7 552 9718 29

30

*Corresponding author:

31

Professor Brigid Gillespie: [email protected] 32

33

Competing Interests 34

The authors declare that they have no competing interests. 35

36

Funding Statement 37

Brigid M. Gillespie acknowledges the financial support of the Australian Research Council, 38

Early Career Discovery Fellowship Scheme and the NHMRC Centre for Excellence in Nursing 39

Research (NCREN). 40

41

Authors’ Contributions 42

BMG conceived of the study, assisted in data analysis, interpreted results and drafted the 43

manuscript. EH performed data analysis and assisted in interpretation. WC assisted in 44

interpretation, and edited the manuscript for important intellectual content. EK, CS, TKW, 45

KS and NF assisted in recruitment and assisted in interpretation. All authors participated in 46

the coordination of the study and read and approved the final manuscript. 47

Page 2 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 3 of 26

48

49

50

Page 3 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 4 of 26


ABSTRACT 52

Background: Communication and teamwork failures have frequently been identified as the 53

root cause of adverse events and complications in surgery. Few studies have examined 54

contextual factors that influence teams’ non-technical skills (NTS) in surgery. The purpose of 55

this prospective study was to identify and describe correlates of NTS. 56

Methods: We assessed NTS of teams and professional role at two hospitals using the 57

revised 23-item NOTECHS and its subscales (communication, situational awareness, team 58

skills, leadership, and decision making). Over 6 months, two trained observers evaluated 59

teams’ NTS using a structured form. Inter-observer agreement across hospitals ranged from 60

86%-95%. Multiple regression models were developed to describe associations between 61

operative time, team membership, miscommunications, interruptions, and total NOTECHS 62

and subscales scores. 63

Results: We observed 161 surgical procedures across eight teams. The total amount of 64

explained variance in NOTECHS and its five subscales ranged from 14% (adjusted R2 0.12, p < 65


0.22, p < .001). In all models, inverse relationships between the 66

total number of miscommunications and total number of interruptions and teams’ NTS were 67

observed. 68

Conclusions: Miscommunications and interruptions impact on team NTS performance. 69

Strengths and limitations 70

• While we found relationships between miscommunications, interruptions and 71

surgical teams’ non-technical skills, the causal sequence between predictors and the 72

Page 4 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 5 of 26

outcome cannot be established. However, the design allowed us to describe 73

statistical associations and identification of some potential confounders. 74

• Surgical teams’ non-technical skills were assessed using direct observation and so it 75

is possible for individuals to alter their practices giving rise to the potential for the 76

Hawthorne effect Nevertheless contemporaneous observation is preferable to self 77

report which gives rise to an unintentional reporting bias. 78

• Measures upon which observations were based may be considered somewhat 79

subjective as they relied on observers’ ability to interpret events. However, 80

observers were experienced in observational research and trained in observational 81

research and human factors. 82

• There is potential for selection bias as surgical teams were purposively selected 83

based on participants’ willingness to be observed. Despite this, there was variability 84

in NTS scores. 85

Key words: teamwork, communication, interruptions, human factors, surgery 86

87

88

Page 5 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 6 of 26

INTRODUCTION 89

Compared to other hospital settings, medical errors in the operating room (OR) can 90

have catastrophic consequences for patients. Adverse events and malpractice claims have 91

been linked to teamwork failures in surgery.1-5 Deficits in teamwork behaviours were 92

identified as a root cause in 63% of all the sentinel events reviewed by The Joint Commission 93

between 2004 and 2013.6 While human error is inevitable and cannot be completely 94

eliminated, the importance of linking the safety of surgery to team culture is increasingly 95

recognised.7-9

Fostering a climate of teamwork and collaboration, along with safety minded 96

work processes that focus on error prevention is the ultimate goal of healthcare 97

organisations. 98

Nevertheless, surgical errors need to be understood in the context of the surgical 99

team. Unique challenges stem from the overlapping but different interprofessional 100

expertise and roles among members, ad hoc team team membership, unstructured and 101

variable communications, frequent distractions, technology, procedural complexity, and 102

competing priorities.10-15 Several studies have described the sources and frequencies of 103

intraoperative interruptions.14 16 17

The results of these studies identified that equipment 104

problems, telephone calls, conversation and environment problems (e.g., noise) were major 105

sources of distractions that influenced team performance. It is therefore hardly surprising 106

that as much as 30% of information gets lost during case-related exchanges.9 18

More recent 107

research suggests that omissions in team communications related to providing members 108

with updates about the progress of an operation comprised up to 36% of all observed 109

communication errors.19

As surgical teams often work together on an ad hoc basis, a lack of 110

prior working experience has the potential to impact on team dynamics. Team familiarity, 111

defined as a core group of individuals who work together regularly, and who share a similar 112

Page 6 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 7 of 26

mental model20, has been identified as an important element of effective teamwork.14 21 An 113

earlier observational study found that fewer miscommunications occurred in teams with a 114

history of working together.14 More recently, results of an Australian observational study 115

suggested a positive association between team familiarity and instrument nurses’ NTS 116

performance across 182 surgical procedures.10

Other studies, using retrospective designs 117

have found associations between team familiarity and reductions in postoperative 118

morbidity following cardiac and major abdominal surgeries. 21 22 119

As a means to increase surgical safety, researchers have focussed on 120

communication, leadership, situational awareness and decision-making, termed collectively 121

as non-technical skills (NTS) in surgery. NTS are the cognitive (i.e., decision making and 122

situational awareness) and interpersonal skills (i.e., communication, teamwork and 123

leadership) that complement the individual’s technical knowledge.23 Previous research 124

indicates that communication is key to the performance of successful teams. Effective and 125

timely transfer of information enables team processes and states such as coordination, 126

cooperation, conflict resolution and sitational awareness.9 11 24 The development of astute 127

NTS is critical to patient safety yet surgical teams are challenged by the increasing technical 128

complexity of surgery and high acuity of patients, who are older, and have multiple 129

comorbidities.8 Moreover there is a lack of research that examines the impact that 130

environmental factors have on teams’ NTS performance. In this prospective study, we 131

hypothesised that longer surgeries, limited team familiarity, miscommunications, and 132

interruptions negatively influenced teams’ use of NTS. A better understanding of the factors 133

that impinge on teamwork behaviours will help us to design strategies to improve NTS 134

performance. 135

136

Page 7 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 8 of 26

METHODS 137

This was a prospective, observational study of teams’ use of NTS during surgery. Two 138

Australian metropolitan hospitals 70 kilometers from each other, each with a similar case 139

mix, specialising in all surgical specialities, were included to generate results that would be 140

applicable across a variety of procedures. In each hospital, four surgical teams comprising of 141

anaesthetic and surgical consultants, their registrars, and instrument/circulating nurses, 142

were observed. Teams and surgical procedures across each hospital were purposively 143

chosen to ensure maximum variation relative to case complexity, particular procedures 144

within specialties, team membership, and surgical experience. In Hospital A, teams from 145

paediatric, thoracic, orthopaedic and general surgery were observed on a weekly basis 146

across 20-25 surgeries. In Hospital B, a similar number of surgeries was observed with 147

cardiac, vascular, upper gastro-intestinal, and hepatobiliary teams. 148

Observational data for each hospital were collected during 2015, with an observer 149

located at each hospital. Prior to the observation period, both observers underwent specific 150

training in the use of the observational tool which included the NOTECHS system. The 151

observers pilot tested the tool and minor changes made to its formatting. During the 152

piloting process, regular meetings were held with the co-resesrchers to ensure greater 153

clarification of recorded events and refine coding. Both observers were trained in human 154

factors and observational research methods. To ensure methodologic consistency, 155

interrater checks with 10% of cases at each hospital site were performed during the 156

observation period by the lead author, also trained in human factors. Interrater agreement 157

across hospital sites ranged from 86%-95%. A single observer was present during each 158

procedure and collected data using pre-specified checklists and free-hand notes. 159

Observations commenced when the patient entered the operating room (prior to 160

Page 8 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 9 of 26

anaesthesia) and ended when the patient left the room. During each surgical procdure, 161

observers documented explanatory field notes to supplement the structured observations 162

to better understand contextual factors. Observational data were collected in 2015 over 6-163

months. 164

Institutional ethics approvals were given by the participating hospitals and the 165

university. Participants signed a consent form and were advised of their right to 166

confidentiality and anonymity, and to withdraw at any time during data collection. Patients 167

whose operations were observed were informed of the likelihood of observations taking 168

place and given the chance to opt out. 169

170

Observational measures 171

We used the revised Non-TECHnical Skills (NOTECHS) scale 25, which was originally 172

developed in the aviation industry for crew resource management. The NOTECHS provides 173

comprehensive behavioural descriptors for each of its subscales and so requires less training 174

prior to use. In surgery, it has been shown to differentiate between good and poor 175

behaviours, thus has demonstrated good construct validity.25

In the revised NOTECHS, five 176

subscales of NTS are assessed: A. communication and interaction; B. situational awareness 177

and vigilance; C. team skills; D. leadership and managerial skills; and, E. decision making in a 178

surgical crisis. Each domain is measured on a 7-point scale to rate each item, with 1 = not 179

done through to 6 = done very well, and, 0 = not applicable 25. Total NOTECHS scores range 180

from 5-23 with higher scores indicative of better overall performance on all five subscales. 181

Scores for individual subscales were as follows; Subscales A and B scores ranged from 4-24 182

while Subscales C to E scores ranged from 5-30. The “not applicable” option meant that a 183

specific item was not relevant or could not be rated on the basis that the behaviour was not 184

Page 9 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 10 of 26

observed. However, participant NOTECHS scores were not affected by a reduced score for 185

non-observed behaviours. ‘Not applicable’ scores were replaced by the participant’s 186

individual item mean. In this study, as all subscales were considered of equal importance, 187

total NOTECHS scores were calculated by the number of items (i.e., 23) as the denominator. 188

Scores for total NOTECHS and its individual subscales were calculated using the mean of all 189

individual team members NOTECHS total scores. We also calculated the mean NOTECHS 190

scores based on professional role (i.e., surgeon, anaesthetist, nurse). 191

In this study, we drew on the literature for definitions and measurement of the 192

observational variables relative to team familiarty, miscommunication and interruption 193

events. Team familiarity was defined as a core membership of three members (i.e., surgeon, 194

anaesthetist, instrument and/or circulating nurses) who had worked together, weekly or 195

fortnightly, for a minimum of three months. 26 Prior to commencement of each surgical 196

procedure, the senior nurse in the room was asked by the observer about regularity, 197

stability, and length of time individual team members had worked together. The number of 198

familiar team members for each procedure were tallied and recorded. We used Lingard et 199

al’s.18 27

taxonomy to classify miscommunications (i.e., audience, content, occasion, 200

experience). Interruptions were classified according to Healey et al’s.16 28 framework (i.e., 201

procedural, conversational). For each procedure, we tallied the number of 202

miscommunications and interruptions in each of their respective categories. In some 203

instances, it was possible that a single miscommunication or interruption could be placed 204

into more than one category. As such, the primary prompt of the miscommunication or 205

interruption was deamed to categorise the event. Operative time included the time from 206

patient skin preparation to the application of the final wound dressing. 207

208

Page 10 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 11 of 26

Analyses 209

All analyses were performed using the Statistical Package for Social Sciences [SPSS] 210

(version 23, IBM, New York, NY, USA). Data were cleaned and a random sample of 20% was 211

checked for accuracy. Descriptive analysis included absolute (n) and relative (%) frequencies 212

to analyse categorical variables (discipline/role, surgical specialty), while means/standard 213

deviations [SD] or medians/interquartile range [IQR]) were used for continuous data (i.e., 214

operative time, number of interruptions, miscommunications, NOTECHS scores). 215

Independent variables; operative time, team familiarity, number of interruptions and 216

miscommunications were subsequently included as covariates in simultaneous multiple 217

regression models with the dependent variable, NTS (measured by NOTECHS). A p-value of 218

<.05 was considered significant and 95% confidence intervals (CI) were used. Cohen’s f2 was 219

used to calculate effect size. 220

221

Sample size calculation 222

Our a priori sample size estimate was based on the 20:1 rule which states that the 223

ratio of the sample size to the number of parameters in a regression model should be at 224

least 20 cases for each predictor variable in the regression model.29 30 As 4 predictor 225

variables were proposed in this study, a sample size of 100 was considered sufficient in a 226

parsimonious regression model. 227

228

229

Page 11 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 12 of 26

RESULTS 230

Across both hospital sites, a total of 161 operations were observed (Hospital A n=80; 231

Hospital B n=81). The number of surgeries observed for each team ranged from 20-25 with 232

the exception of the thoracic team. Owing to the retirement of the consultant surgeon in 233

the thoracic team, only 6 surgical procedures were observed in this specialty. In total, 481 234

individual participant’s observational data were collected (Hospital A n =243; Hospital B 235

n=238). The mean length of surgery across both sites was 116.3 minutes (±96.5) [Site A = 236

78.5 minutes, ±71.2; Site B, 153.7 minutes, ±103.8]. Across the 160 procedures we 237

observed, consistency in team membership ranged from 3-8 team members. On average 238

there were 7 team members present across all procedures including 2 surgeons, 2 239

anaesthetists, and 3 nurses. Table 1 shows case characteristics for each surgical specialty 240

relative to number of procedures in each specialty, operative time, team membership, and 241

NOTECHS scores (by Subscales A-E and mean total). Subscale E, decision-making during a 242

surgical crisis was observed in only 40-50% of cases as these situations were often not 243

observed during field work. Of the eight teams observed, the Hepatobiliary team had the 244

highest NOTECHS mean scores (20.7±2.3) while the cardiac team had the lowest (19.1±3.5). 245

Table 2 displays the descriptive results for NTS performance based on professional role. 246

Observed NTS performance among surgeons and anaesthetists was comparable however, 247

nurses scores were somewhat lower. 248

During each surgical procedure, the observers recorded field notes to better 249

understand and explain the contextual happenings during assessment of teams’ NTS. The 250

following two field notes are provided as exemplars of team communications from the251

Page 12 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 13 of 26

Table 1: Case Characteristics (n=161 Surgical Procedures) 252

Surgical

Specialty

Number of

Procedures

Observed

in each

Specialty

(n/total %)

Operative

Time (mins)

Mean (SD)

Team

Member-

ship

Mdn (IQR)

*Total

NOTECHS

Scores

Mean (SD)

^Subscale A

Communi-

cation and

interaction

Mean (SD)

^Subscale B

Vigilance

/situation

awareness

Mean (SD)

^Subscale C

Team skills

Mean (SD)

^Subscale D

Leadership

and magmt

skills

Mean (SD)

^Subscale E

Decision

making in a

crisis

Mean (SD)

General 25 (15.5) 119.5 (85.7) 4 (1) 18.7 (3.1) 19.8 (3.5) 20.0 (3.6) 23.9 (4.5) 23.8 (4.7) 24.3 (4.7)

Orthopaedic 25 (15.5) 82.3 (66.6) 4 (2) 20.3 (2.4) 21.0 (3.1) 21.8 (2.6) 25.9 (3.1) 26.7 (3.5) 26.3 (4.0)

Paediatric 25 (15.5) 35.0 (26.1) 3 (2) 20.5 (2.7) 21.5 (2.8) 21.9 (3.0) 26.1 (4.5) 26.5 (4.1) 27.1 (3.5)

Thoracic 6 (3.7) 74.1 (55.8) 4 (3) 19.6 (2.3) 21.6 (3.0) 20.8 (3.0) 25.2 (3.1) 24.6 (3.7) 25.1 (2.4)

Cardiac 20 (12.4) 234.4 (97.5) 8 (3) 18.4 (2.6) 19.1 (3.5) 20.7 (2.7) 22.8 (4.3) 22.5 (4.4) 24.8 (4.2)

Hepatobiliary 20 (12.4) 165.3 (122.2) 5 (1) 20.7 (2.3) 22.1 (2.4) 22.1 (2.5) 26.7 (3.3) 25.7 (3.8) 27.1 (4.0)

Upper GI 20 (12.4) 109.8 (78.6) 4 (2) 20.5 (2.6) 22.1 (2.9) 22.1 (2.5) 26.2 (3.7) 25.1 (4.1) 27.0 (4.7)

Vascular 20 (12.4) 105.4 (51.7) 6 (2) 20.1 (2.4) 22.1 (2.5) 21.9 (2.3) 25.3 (4.3) 23.9 (4.6) 26.6 (4.1)

Note: *Total NOTECHS Scores range 1-23; ^Subscales A and B scores in domain range 4-24, Subscales C, D and E Scores in domain range 5-30. 253

254

255

256

Page 13 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960





Page 14 of 26

Table 2: Descriptives of NOTECHS performance based on professional role (n=481) 257

Surgeon

Consultant/

Registrar

Anaesthetic

Consultant/

Registrar

Scrub /

Scout Nurse

Total NOTECS

n 161 158 160

Mean 20.5 20.6 18.9

SD 2.1 2.4 3.2

95% CI 20.1-20.8 19.8-20.6 18.4-19.4

Range 14.5-23.0 11.64-23.0 10.04-23.00


and interaction

n 161 158 160

Mean 21.4 21.5 20.4

SD 2.8 2.87 3.7

95% CI 20.9-21.8 21.0-22.0 19.81-20.96

Range 10.0-24.0 10.0-24.0 10.00-24.00



n 161 158 160

Mean 22.2 21.3 20.8

SD 2.2 2.6 3.6

95% CI 21.8-22.5 20.9-21.7 20.3-21.4

Range 16.0-24.0 11.0-24.0 8.0-24.0


n 161 158 160

Mean 25.9 25.9 24.1

SD 3.5 4.0 4.6

95% CI 25.3-26.4 25.2-26.5 23.3-24.8

Range 15.0-30.0 11.00-30.0 10.0-30.0


management skills

n 161 158 160

Mean 25.5 25.5 23.8

SD 4.1 3.9 4.8

95% CI 24.9-26.2 24.9-26.1 23.0-24.6

Range 14.0-30.0 12.5-30.0 10.0-30.0


in a crisis

n 161 158 160

Mean 27.5 27.0 23.6

SD 2.83 3.16 5.2

95% CI 27.1-28.0 26.6-27.6 22.8-24.4

Range 18.0-30.0 17.0-30.0 9.0-30.0

Note: *Total NOTECHS Scores range 1-23; ^Subscales A and B scores in domain range 4-24, Subscales C, D and 258

E Scores in domain range 5-30. 259

Page 14 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 15 of 26

highest and lowest performing teams on the NOTECHS. Ensuring that both the anaesthetic 260

and surgical teams had a similar mental model in relation to the procedure was important: 261

Prior to commencing a liver resection procedure, the Consultant and Registrar 262

Surgeons and the Anaesthetic Consultant participated in a detailed prebriefing 263

about the patient’s medical history and anticipated difficulties/challenges from 264

their discipline perspectives. These physicians had never worked together 265

before. Prebriefings between the lead surgeon and anaesthetist were 266

commonplace in this room and were observed to occur in 70% of the cases 267

observed. (Hepatobiliary: Hepatectomy, Case # 18). 268

The following fieldnote illustrates an observed miscommunication between the 269

surgeon and perfusionist: 270

Consultant Surgeon to Perfusionist, “Give pledgia.” 271

Perfusionist: “Give another one?” 272

Consultant Surgeon: ”‘Have you finished with the previous one?” 273

Perfusionist: “Yes”. Consultant Surgeon appears to be unaware of pledgia 274

delivery time. There was no further inquiry from the Consultant Surgeon. 275

(Cardiac: CABGS x 4, Case # 9). 276

Across the 161 procedures, the number of miscommunications totalled 436 (Hospital A 277

n=133; Hospital B n=303). The highest number of miscommunications was observed in 278

cardiac surgery (n=121). Throughout the observed procedures, interruptions occurred in 279

106/161 (65.8%) cases. Of the 106 procedures where interruptions were observed, 280

procedural interruptions occurred at least once in 92 procedures (86.8%) (Hospital A n=118; 281

Hospital B n=76). The number and types of miscommunications and interruptions for each 282

surgical specialty appear in Figures 1 and 2. 283

284

Page 15 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 16 of 26

Multivariate regression analyses 285

Table 3 shows the six multiple regression models for total NOTECHS scores and its 286

individual subscales (A-E). The total amount of explained variance in NOTECHS and its 287

individual subscales ranged from 14% (adjusted R2 0.12, p < .001) to 24% (adjusted R2 0.22, 288

p < .001). In all six regression models, the total number of miscommunications and 289

interruptions were consistently significant predictors of teams’ NTS (Table 3). Operative 290

time and team membership were non-significant. 291

292

DISCUSSION 293

To the best of our knowledge, this is the first study to examine the correlates of 294

teams’ NTS. This study is also one of the largest single observational studies in this field. 295

Notably, we found inverse associations between the number miscommunications and 296

interruptions and team NTS across all NOTECHS subscales, suggesting that the fewer 297

miscommunications and interruptions there are, the higher teams’ NTS performance. These 298

results seem intuitive but, this study is the first to provide evidence generated through 299

structured observations conducted in real time (rather than in simulated environments). In 300

this study, we observed fewer interruptions as compared with miscommunications; with the 301

highest number of interruptions seen in the general surgery team. Many interruptions may 302

be considered acceptable when there are no immediate demands from patient care, but are 303

clearly less appropriate at busy times or when problems occur.31 Some interruptions are 304

essential for information sharing, or to talk to and reassure patients, but managing 305

interruptions and distractions is a crucial skill and requires individuals to refocus on their 306

Page 16 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 17 of 26

Table 3: Regression models for predictors of total NOTECHS and each NOTECHS domain (n=161 Surgical Procedures) 307

95% CI


Std

Error β t Sig

Lower

Bound

Upper

Bound a



Operative Time .00 .00 .10 .97 .334 -.00 .01



Communication and



Operative Time -.00 .00 -.07 -.66 .512 .00 .01



Vigilance/situation



Operative Time .00 .00 .14 1.40 .163 -.00 .01


Interruptions -.36 .13 -.23 -2.86 .005 -.61 -.11 d

Subscale C



Operative Time .00 .00 .04 .41 .686 -.01 .01


Interruptions -.30 .18 -.13 -1.65 .101 -.66 .06

Page 17 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960





Page 18 of 26

e Subscale D

Leadership and






Decision making



Operative Time .01 .01 .03 .26 .793 -.01 .01


Interruptions -.45 .17 -.21 -2.58 .011 -.79 -.11

Note model results 308 a

R=.43, R² .18, R² Adj .16, F= 8.65 df(4/160), p<.001, (f2)=.22

b R=.37, R² .14, R² Adj .12, F= 6.22 df(4/160), p<.001, (f

2)=.16

c R=.37, R² .14, R² Adj .12, F= 6.23 df(4/160), p<.001, (f

2)=.16

d R=.41, R² .17, R² Adj .15, F= 7.82 df(4/160), p<.001, (f

2)=.20

e R=.49, R² .24, R² Adj .22, F=12.27df(4/160), p<.001, (f

2)=.32

f R=.38, R² .15, R² Adj .12, F= 6.56 df(4/160), p<.001, (f

2)=.18

Page 18 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960





Page 19 of 26

primary task.14 Interruptions have identified as a major contributor to loss of

vigilance in anaesthetists.31

While teams and individuals scored reasonably highly on the

NOTECHS and its subscales, the lowest NTS performance was observed in relation to

vigilence/situation awareness across all teams. Clearly, miscommunications and

interruptions have the potential to erode individual and distributed situational awareness in

surgery.14 31

The hepatobiliary team had the highest NTS performance, as indicated by their








Conversely, the cardiac team demonstrated the lowest NTS performance, which was

unexpected given that this team had clearly defined roles and a small repertoire of


greatest number of observed miscommunications during the field work period. Notably, the

degree of difficulty and complexity, technical skills, stress, and patients’ instability and

acuity may be highest in cardiac surgery.13 32 Observer described (in field notes) the

considerable environmental, technological and team-related challenges experienced by the

cardiac during the surgery, which added to case complexity. For instance, the high noise

levels in this room, attributed to team communications and technology, e.g., cross

Page 19 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 20 of 26

conversations, repeated requests from the surgeon to the perfusionist who was distracted

by other team members and/or equipment problems, incessant alarms during the

intraoperative period. Additionally, procedural and conversational interruptions as a result

of the entry of external team members into the room to ask questions, the referral of cell

phone calls that occasionally demanded the recipient to leave the room, contributed to

lower observed NTS in the cardiac team.

Although we had good sampling across surgical specialties and procedures, it is

difficult to speculate about whether the differences in NTS performance can be attributed

to hospital sites, specialties, surgical teams, or individuals. The two hospital sites chosen

were similar in relation to case mix, patient acuity and surgical activity. However the

selection of specialties varied in each hospital, which may in part, explain the differences in

NTS we observed across teams. The observed diffierences may also be attributed to

particular individuals, i.e., good leadership of the consultant surgeon has been linked with

effective team behaviour and task accomplishment.33 Arguably, surgeons may establish

aspects of leadership prior to the start of the procedure to condition intraoperative team

performance. For instance, using the surgical safety checklist or having a team briefing can

contribute to building the team’s shared mental model, and hence increasing distributed

situational awareness.34


This study has several strengths but we also recognise some limitations: First, while

we found relationships between miscommunications, interruptions and surgical teams’ NTS,

Page 20 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 21 of 26

temporal order and causality cannot be established. Thus there may be some competing

explanations for these results. Notwithstanding, the design does allow statistical

associations and identification of some potential confounders; but, not all have necessarily

been identified. Second, surgical teams’ NTS were evaluated through direct observation.

Although most research in this area has been largely observational and has focussed on

refining this methodology 9 16 17 26 35, individuals may have altered their practices in response

to being observed. Nevertheless contemporaneous observation is a preferred method to

self report which could be flawed (giving rise to response bias). The observational nature of

the NOTECHS allowed us to measure performance as it happened, rather than a

retrospective self report. Third, the measures upon which the observations were based may

be considered somewhat subjective as they rely on observers’ ability to interpret events.

Yet, observers were experienced OR nurses, trained in observatimal research and in human

factors. Interrater consistency between observers was acceptable. Additionally, the

measures we used have been previously validated in this field. 25-27 35 Fourth, surgical teams

were purposively selected based on participants’ willingness to be observed. Thus there is

the potential for selection bias. Notwithstanding, the, and there was variability in NOTECHS

scores. Finally, in this sample the amount of explained variance in NTS and its subscales

while reasonable, indicates that there are unknown predictors that warrant further

exploration. Despite these limitations, our results contribute to identifying interventions

that specifically target minimising miscommunications and interruptions, both of which are

modifiable with the ultimate goal of improving NTS in surgery.

CONCLUSIONS

Page 21 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 22 of 26

Our observational results suggest that effective communication and interruptions were

consistent corrrelates of surgical teams’ NTS performance. Across teams, we observed

examples of good and poor NTS performance. Notwithstanding, these correlates of team

performance are amenable to improvement or change. Implementation of interdisciplinary

team training may contribute to improvements in NTS. However such training programs

need to be underpinned by behaviour change frameworks that focus on sustained

improvements in NTS performance. It is reasonable to propose that the behavioural

indicators of success for overall performance are transferrable acr

Words: 3,523 (excluding references, figures and tables)

Contributors

BMG conceived of the study, assisted in data analysis, interpreted results and drafted the

manuscript. EH performed data analysis and assisted in interpretation. WC assisted in

analysis and interpretation. EK, CS, and NF assisted in recruitment and interpretation. All

authors participated in the coordination of the study and read and approved the final

manuscript.

Funding

Brigid M. Gillespie acknowledges the financial support of the Australian Research Council,

Early Career Discovery Fellowship Scheme and the NHMRC Centre for Excellence in Nursing

Research (NCREN).

Competing interest

None declared.

Ethics approval

Page 22 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 23 of 26

Metro South Health Human Research Ethics Committee

Page 23 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 24 of 26

REFERENCES





3. Mazzocco K, Petitti DB, Fong KT, et al. Surgical team behaviors and patient outcomes. Am J Surg

2009;197(5):678-85.

4. Kable A, Gibberd R, Spigelman A. Adverse events in surgical patients in Australia. Int J Qual Health

Care 2002;14(4):269-76.

5. Raman J, Leveson N, Samost A, et al. When a checklist is not enough: How to improve them and

what else is needed. J Thorac Cardiovasc Surg 2016;152(2):585-92.

6. JCAHO. Sentinel event data: root causes by event type (2004-second quarter 2011). Secondary

Sentinel event data: root causes by event type (2004-second quarter 2011) 2011.




BMJ Open 2015;5:e006216.


safety. J Interprof Care 2013;27(5):287-93.



10.1136/bmjqs.2009.032326

10. Kang E, Massey D, Gillespie B. Factors that influence the non-technical skills performance of

scrub nurses: A prospective study. J Adv Nurs 2016;71:2846–57.








observational study. AORN Journal 2012;95(5):576-90.



2013;128(10):1139-69.




Pract 2007;13:390-94.



Surg 2008;143(1):12-17.


2011;149:305-10.




on Surgical Complications. World J Surg 20134;38:3047-52.


individual surgeon experience in cardiac surgery. J Thorac Cardiovasc Surg 2013;145:328-33.

Page 24 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 25 of 26


roles in improving patient outcome. Vet Anaesth Analg 2014;41(3):221-23.

24. Mesmer-Magnus J, DeChurch L. Information Sharing and Team Performance: A Meta-Analysis. J

Appl Psychol 2009;94:535-46.


teams. Am. J. Surg. 2008;196:184-90.


Results of a prospective study. Qual Saf Health Care 2012;21(1):3-12.



Care 2006;15:422-26.

28. Healey A, Sevadalis N, Vincent C. Measuring intra-operative interference from distraction and

interruption observed in the operating theatre. Ergonomics 2006;49(5-6):589-604.


2010.

30. Department of Biostatistics VU. Statistical Problems to Document and to Avoid. Secondary

Statistical Problems to Document and to Avoid 2014.


31. Campbell G, Arfanis K, Smith A. Distraction and interruption in anaesthetic practice. Br J Anaesth

2012;109:707-15.

32. Gurses A, Kim G, Martinez E, et al. Identifying and categorising patient safety hazards in

cardiovascular operating rooms using an interdisciplinary approach: a multisite study. BMJ

Qual Saf 2012;21:810-18.

33. Siu J, Maran N, Patterson-Brown S. Observation of behavioural markers of nontechnical skills in

the operating room and their relationship to intra-operative incide. The Surgeon 2014.

34. Henrickson Parker S, Flin R, McKinley A, et al. Factors Influencing Surgeons’ Intraoperative

Leadership: Video Analysis of Unanticipated Events in the Operating Room. World J Surg

2014;38:4-10.



Page 25 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Page 26 of 26

Table Legend


Table 2: Descriptives of NOTECHS performance based on professional role (n=481)

Table 3: Regression models for predictors of NOTECHS and each NOTECHS domain (n=161

Surgical Procedures)

Figure Legend



Page 26 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



Figure 1

477x207mm (300 x 300 DPI)

Page 27 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



Figure 2

466x199mm (300 x 300 DPI)

Page 28 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




1


Item

No Recommendation

Checklist/page



3



3

Introduction



5


Methods




7



up






7-8







applicable

8

Data sources/

measurement




7-8





9


for confounding

9-10


interactions

10



addressed



NA

Page 29 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




2




Results




10-11





11,13




time

NA







10






11,12,16

Discussion




19



18


Other information



1








Page 30 of 30


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from


Documents

BMJ Open€¦ · interruptions.13 15 16 The results of these studies identified that equipment problems, telephone calls, conversation and environment problems (e.g., noise) were