Marci Lee Nilsen, MSN, RNRebecca Nock, SN

Lauren Terhorst, PhD, MA Susan Sereika, PhD

Mary Beth Happ, PhD, RN, FAAN

Assessing Reliability and Validity of a Communication Interaction

Behavior Instrument in Critically Ill, Mechanically Ventilated Adults

Overview

1. Background

2. Aims

3. Methods

– Definition refinement

– Sample

– Coding Procedure

– Analysis

5. Coding Exemplar

6. Results

7. Conclusion

Background• Nurse’s interaction behaviors can affect the tone of

communication interactions with critically ill, mechanically ventilated (MV) patients

• Positive actions by the nurse tended to yield positive patient reactions and negative nurse actions yielded negative patient reactions1

• Higher GCS scores were associated with more positive nurse actions and less negative nurse actions2

1) Salyer, J., & Stuart, B. J. (1985). Heart & Lung, 14(1), 20-24

2) 2) Hall, D. S. (1996). American Journal of Critical Care, 5(4), 293-297.

Background• Nurse training can increase positive behaviors and

decrease negative behaviors utilized by nurses during an interaction3

– Improving nurse-patient interaction may affect both the patient’s recovery

• Psychological (patient satisfaction, ability to engage)3

• Medical (need for support materials and equipment, length of stay)3

• There is no psychometrically evaluated instrument to measure nurse-patient interaction behaviors in the intensive care unit (ICU)

3) de los Ríos Castillo, J. L., & Sánchez-Sosa, J. (2002). Salud Mental, 25(2), 21-30.

Objective• To develop and psychometrically evaluate a

communication interaction behavior instrument derived from prior observational research in the ICU

– Salyer and Stuart, 1985

– Hall, 1996

– de los Ríos Castillo and Sánchez-Sosa, 2002

Methods

• The initial pilot version of the Communication Interaction Behavior Instrument covered 33 behaviors

• Behaviors were divided into 4 categories:

– Positive nurse (17)

– Negative nurse (3)

– Positive patient (10)

– Negative nurse (3)

Pilot Communication Interaction Behavior Instrument

Nurse Interaction Behaviors

Positive Sharing, Praising, Visual Contact, Brief Contact, Brief Contact with Speech, Proximity, Proximity with Speech, Physical Contact, Social Politeness, Preparatory Information, Expanded Preparatory Information, Delayed Preparatory Information, Delayed Expanded Preparatory Information, Smiling, Modeling, Laughing, Augmenting

Negative Disapproving, Yelling, Ignoring

Patient Interaction Behaviors

Positive Acceptance, Instruction Following, Full Visual Contact, Partial Visual Contact, Physical Contact, Request, Smiling, Maintaining Attention, Laughing, Praise

Negative Disagreement, Disgust, Ignoring

Definition Refinement

• Nurse Positive Behavior– Sharing: Facing the patient, the nurse offers him/her such items as

a glass of water, prescribed food, special urinals, the patient’s CD or MP3 player or some other object used to support the patient’s well-being or treatment.

– Revised Sharing: Facing the patient, the nurse offers him/her such items as a blanket, pillow, ice chips, prescribed food (ex: ice cream), the patient’s CD, TV (ex: on/off or closer), or MP3 player or some other objects* used to support the patient’s well-being or treatment. *objects should not include medication or treatments (Example: offering pain medication or suction)"

Definition Refinement• Patient Positive Behavior

– Acceptance: After nurse offers or performs a health-related or comfort providing function, the patients says “yes”; “mmhm”; thanks the nurse; or nods affirmatively with the head, eyes, or hand, expressing agreement, acceptance, or satisfaction.

– Revised Acceptance: After the nurse offers or performs a health-related or comfort providing function, the patient nods affirmatively with the head, eyes or hand, expressing agreement, acceptance or satisfaction. *May receive a “present” if the nurse provides the patient with a plan of care for the rest of the shift (example: Nurse offers to change patient’s bed/ linens after the physicians comes back and finishes the procedure. The patient nods yes and accepts the plan the nurse has offered.)

Sample

• 5 critically ill, MV adult patients and their nurses were randomly selected from the Study of Patient-Nurse Effectiveness with Communication Strategies (SPEACS)

• All patients were under 60 years of age

• Nurse ranged from 22 to 55 years of age and had at least 1 year critical care experience

Coding Procedure

• The instrument was used to rate four 3-minute video-recordings of the 5 nurse-patient dyads (N=20 sessions)

• Two raters coded each session independently

– Raters achieved 90% agreement before proceeding to coding the 5 cases

Coding Procedure

• Raters came together to review coding differences and jury

– To jury, coders review difference(s), watch the video session again, discuss definitions and come to a consensus on whether a behavior was present or absent.

– If the two coders can not come to a consensus, a third coder reviews the session in question and provides feedback

Analysis

• Experts reviewed the instrument for face validity

– Dr. Mary Beth Happ and Dr. Ann Kowlanowaski (Penn State University)

• Percent agreement was calculated for each item, by subscale, and for the total instrument

• Kappa Coefficients were calculated for each item

*Picture courtesy of Robert Wood Johnson Foundation

Nurse: Are you having trouble breathing?

Patient: Shakes head (No)

Nurse: Any pain in your chest?

Patient: Shakes Head (No)

Nurse: Are you feeling ok?

Patient: Nods Head (Yes)

Coding Exemplar

Coding Exemplar

• Nurse Positive– Proximity with

Speech

– Visual Contact

– Physical Contact

• Patient Positive– Visual Contact

– Instruction Following

– Maintaining Attention

Results• Total score agreement ranged from 76-100%

(Mean=90.4%)

• Mean subscale agreement

– 90.5% (Range: 80-100%) for nurse behaviors

– 90.4% (Range: 76-100%) for patient behaviors

• Mean item agreement – 80-100% (Mean=90.3%) for nurse behaviors

– 65-100% (Mean=91.2%) for patient behaviors

Results

• Kappa coefficients ranged from 0.222 to 0.875

• 11 behavior definitions were modified based on feedback from coders and experts

– Nurse Behaviors: Sharing, Praising, Proximity with Speech, Preparatory Information, Expanded Preparatory Information, Preparatory Information with Delayed Action, Expanded Preparatory Information with Delayed Action

– Patient Behaviors: Acceptance, Instruction Following, Full Visual Contact, Disgust

Results

• 4 behaviors were removed from the instrument

– Nurse behaviors: Brief Contact, Brief Contact with Speech, Proximity

– Patient behaviors: Partial Visual Contact

• Final Communication Interaction Behavior Instrument includes 29 behaviors

– 17 nurse behaviors

– 12 patient behaviors

17 Nurse Interaction Behaviors

14 Positive Sharing, Praising, Visual Contact, Proximity with Speech, Physical Contact, Social Politeness, Preparatory Information, Expanded Preparatory Information, Delayed Preparatory Information, Delayed Expanded Preparatory Information, Smiling, Modeling, Laughing, Augmenting

3 Negative Disapproving, Yelling, Ignoring

12 Patient Interaction Behaviors

9 Positive Acceptance, Instruction Following, Visual Contact, Physical Contact, Request, Smiling, Maintaining Attention, Laughing, Praise

3 Negative Disagreement, Disgust, Ignoring

Conclusion

• Preliminary results suggest that the Communication Interaction Behavior Instrument has good reliability and face validity in a critically ill, MV adult population

• The next step will be to test the instrument in a larger population (38 mechanically ventilated older adults (>60 years of age) and their nurses for a total of 152 sessions)

Funding

• National Institute of Child Health and Human Development (5R01-HD043988) (Happ)

• National Institute of Nursing Research (K24-NR010244) (Happ)

• National Institute of Nursing Research (F31 NR012856) (Nilsen)

Questions