Explorations in Internet-Based Collaborative Informatics Research: A

Cognitive Evaluation

Vimla L. Patel, PhD, DScDepartments of Medical Informatics and Psychiatry

College of Physicians and SurgeonsColumbia UniversityNew York, New York

Asynchronous Synchronous

• Telephone• Group Conference Calls• Video Conferencing• Face-to-Face Meetings

• Email - personal• Email Dist’n Lists• World Wide Web• Regular Mail

Communication

McGillEvaluation

Studies

STANFORD COLUMBIA

BWH - HARVARD MGH - HARVARD

• Academic Research and Practice• Graduate Degree Program• Clinical Information Systems• Medical Logic Standards• Decision-Support Systems• Clinical Workstations• Knowledge-Based Vocabularies• Cholesterol Guidelines

• Academic Research• Graduate Degree Program• Knowledge Representation• Advanced Server Technology• Decision-Support Systems• Knowledge Acquisition• Clinical Trials Workstations• Breast Mass Protocols

• Academic Research• Medical Informatics Training• Component-Based Software• Clinical Workstations• Integrated Guidelines• Decision-Support Systems• Knowledge Authoring Tools• Breast Cancer Guidelines

• Academic Research and Practice• Integrated Clinical Training• Decision-Support Systems• Clinical Workstations• Integrated Guidelines• Clinical Information Systems• Electronic Medical Records• Flu Vaccine Guideline

Shortliffe Cimino

BarnettGreenes

Patel

Sources of Data

• Electronic Interviews• Audio-Taped Conference Calls• Email

– Current & Archived• Progress Reports• Papers, Presentations, and Proposals• Developments at Intermed WWW Sites• WWW Access Statistics

– Access of Resources, Documents, etc..

– File Transfers Between Sites• Audiotapes of Mini Workshop(s)

Perspectives on Collaboration: E-Mail Interview

PLEASE COMMENT ON THE FOLLOWING TWO STATEMENTS:

a InterMed is a Highly Integrated Collaborative Endeavor with Largely Shared Assumptions. Work on Common Projects Require Shared Efforts and a High Degree of InterPersonal and Intergroup Integration

b InterMed is a More Loosely Coupled Cooperative Project which Each Team Provides Shared Expertise and Resources towards some Common Goal and Towards Institution-Specific Goals. Progress is Largely a Function of Individual Intiative Efforts, which are Later Coordinated within the Larger InterMed Context.

Modes of Communication Best Suited To Specific Purposes

Individual

A

B

C

D

E

F

G

Plan

CC

CC

CC

CC

CC

FTF

CC

Present

WWW

WWW

WWW

FTF

WWW

EM

WWW

Brainstorm

FTF

CC

EM

EM

CC

FTF

FTF

Conflicts

FTF

EM

CC

CC

CC

CC

FTF

Technical

EM

EM

EM

EM

EM

EM

EM

Critique

EM

EM

EM

EM

EM

EM

EM

STANFORD

COLUMBIA

DSG

PURPOSE

Categories of Activity During Conference Call

ACTIVITY Feb. 7 Feb. 21Total Time of Call 68'45" 31'35"Number of Participants 20 14Active Participants (5 Turns) 9 9Number of Turns 455 253Alternatives 17 15Clarification 193 157Criterion 2 4Goal 3 0Issue 45 18Meeting Management 14 11Project Management 74 22Summary 2 0Walkthrough 2 0

Frequency of Guideline-RelatedFrequency of Guideline-Related Email Messages per WeekEmail Messages per Week

WEEKS FROM JULY 1994 TO APRIL 1996July 94 April 96Jan 95 July 95 Dec 95 Feb 96

Numberof

Emails

0

5

10

15

20

25

30

35

40

451 32 46 49 54 65 68 76 80 82 84 86 88 90 92 94

Voca

bula

ry G

uide

lines

Exp

.

HPC

C M

eetin

g,C

omm

on G

uide

lines

Rep

rese

ntat

ion

Inte

nsiv

e W

ork

Beg

ins

onG

uide

lines

Mod

el/D

etai

led

Age

nda

AM

IA F

all C

olla

bora

tive

Subm

issi

ons

Min

i Wor

ksho

p on

Gui

delin

es

37

35

33

31

29

27

25

23

21

19

17

15

13

11

9

7

5

3

11 3 5 7 9 11 13 15 17 19 21 23

Sequence of Goals in Progress Report

Specific Goals

In Progress

Complete

MONTHS FROM JULY 1994 TO APRIL 1996July 94 April 96

Common Guideline Model

Adapt And Display Guidelines on WWW

PrototypicalGuideline/ WWW CGI

C

A B E

G

F

H

J

I ML

D

K

Sociometric Analysis of Email Communication Patterns

Stanford

DSG

Columbia

MGH

InterMedCentral

Stanford Columbia

MGH

C

AB E

F

ML

D

K

Email Communication Patterns (Jan-Feb 96)Email Communication Patterns (Jan-Feb 96)Episode 1: Vocabulary Server and Guidelines Episode 1: Vocabulary Server and Guidelines

InterMedCentral

Stanford

BWH

Columbia

MGH

C

AB E

G

F

H

J

I

D

K

Email Communication Patterns (Feb-Mar 96)Email Communication Patterns (Feb-Mar 96)Episode 2: Guidelines RepresentationEpisode 2: Guidelines Representation

InterMedCentral

Conclusions• Significant Evolution in Communication

Patterns–Greater Degrees of Integration–Focused Communication

• Increased Use of Richer Representations –Diagrams, WWW

• Integrated Plans and Coordinated Actions• Sustained Collaboration Requires Sustained

Commitment• Different Modalities — Cognitive & Social

Consequences• Learning Curve in Collaboration

• Collaboration is difficult!