Why Collect Data?

November 2, 2007

Colleen Cook, Dean of Libraries, Texas A&M University

NISO Usage Data ForumDallas, TX

Why do I do assessment at Texas A&M?

Spending millions wiselyLearn from mistakesQuality managementAccreditationGood citizens of associationsThe right thing to doIn the end….a Culture of Assessment

What do I assess?

…just about everything I can

What are the characteristics of qualitative methods?

• The observer/researcher inseparable from the study

• Consists of a set of interpretive practices that try to make sense of a cultural context

• Data sources: field notes, interviews, conversations, photographs, recordings, and memos to the self

• Study a natural setting, attempting to make sense of, or to interpret, phenomena in terms of the meanings people bring to them

How do I assess at Texas A&M?

Multiple Methodsof Listening to Customers

• Transactional surveys*

• Mystery shopping

• New, declining, and lost-customer surveys

• Focus group interviews

• Customer advisory panels

• Service reviews

• Customer complaint, comment, and inquiry capture

• Total market surveys*

• Employee field reporting

• Employee surveys

• Service operating data capture

Note. A. Parasuraman. The SERVQUAL Model: Its Evolution And Current Status. (2000).

Paper presented at ARL Symposium on Measuring Service Quality, Washington, D.C.

Historical circulation status for stack items(Circulation data from 1980-)

Circulated and browsed

38%

Circulated not browsed

21%

Not browsed not circulated

38%

Browsed not circulated

3%

Historical circulation status for monograph stack items (data from 1980-)

Circulated and browsed

44%

Circulated not browsed

20%

Not browsed not circulated

35%

Browsed not circulated

1%

Historical circulation status for monograph stack items by LC class (data from 1980-)

0%

20%

40%

60%

80%

100%

A B C D E F G H J K L M N P Q R S T U V Z

Not circulated not browsedNot circulated browsed

Circulated not browsedCirculated browsed

Time lapse to first circulation for stack items added between 1/1/2000 and 1/1/2002

60%17%

9%

6%

4% 4%

One year

Two years

Three years

Four years

Five years

More than five years

0

50000

100000

150000

200000

250000

300000

2000 2001 2002 2003 2004 2005 2006

Total circulations ofStack items

Unique Stack itemscirculated during year

0

50000

100000

150000

200000

250000

300000

2000 2001 2002 2003 2004 2005 2006

Calendar year

Total circulations ofmonograph stack items

Unique monograph stackitems circulated duringyear

Shelving pickups in TAMU main library and annex

0

50,000

100,000

150,000

200,000

250,000

2000 2001 2002 2003 2004 2005 2006

Pickups

Total Circulation

Note. M. Kyrillidou and M. Young. (2005). ARL Statistics 2003-04. Washington, D.C.: ARL, p.6.

Time lapse to first circulation for monograph stack items added between 1/1/2000 and

1/1/2002

63%16%

8%

5%

4% 4%

One year

Two years

Three years

Four years

Five years

More than five years

Annual circulation of monograph stack items in sciences

05000

100001500020000250003000035000400004500050000

Q R S T

Library of Congress classification

Circulations

2000

2001

2002

2003

2004

2005

2006

Annual circulation of serial stack items in the sciences

0

1000

2000

3000

4000

5000

6000

7000

8000

Q R S T

Library of Congress classification

Circulations

2000

2001

2002

2003

2004

2005

2006

Annual circulation of monograph stack items in humanities

05000

1000015000200002500030000350004000045000

B C D E F M N P

Library of Congress classification

Circulations

2000

2001

2002

2003

2004

2005

2006

Annual circulation of serial stack items in humanities

0100200300400500600700800900

1000

B C D E F M N P

Library of Congress classification

Circulations

2000

2001

2002

2003

2004

2005

2006

Annual circulation of monograph stack items in social sciences

0

5000

10000

15000

20000

25000

30000

35000

A G H K L U V Z

Library of Congress classification

Circulations

2000

2001

2002

2003

2004

2005

2006

Annual circulation of serial stack items in social sciences

0

500

1000

1500

2000

2500

3000

A G H J K L U V Z

Library of Congress classification

Circulations

2000

2001

2002

2003

2004

2005

2006

Annual circulations of monograph items by major patron group

0

20000

40000

60000

80000

100000

120000

Faculty/staff Graduate Undergraduate ILS Lending

Patron group

2000

2001

2002

2003

2004

2005

2006

3.753.88

3.583.77

4.05

1

2

3

4

5

Climate for Customer Service

Climate for Customer Service Means

Texas Library 1 Library 2 Library 3 Library 4

Climate for Customer Service

68% 67%

74%

64%

73%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percent Agreement

Agreement Ratings of Trust in Leader

Texas Library 1 Library 2 Library 3 Library 4

Leadership Climate

76%71%

82%79%

75%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percent Agreement

Agreement Ratings of Leader-Member Relationship Quality

Texas Library 1 Library 2 Library 3 Library 4

Leadership Climate

54% 55%52%

35% 37%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percent Agreement

Agreement Ratings of Climate for Interpersonal Treatment: Team-level

Interpersonal Treatment

Texas Library 1 Library 2 Library 3 Library 4

Climate for Interpersonal Treatment

55% 53% 52%47%

56%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percent Agreement

Agreement Ratings of Climate for Interpersonal Treatment: Manager's

Interpersonal Treatment of Employees

Texas Library 1 Library 2 Library 3 Library 4

Climate for Interpersonal Treatment

The EndThe Endwoofwoof

LoadedPT:P1:01xxxxxxxxxxxxxx.txt,S:\Admin\Colleen\ServQual Interviews\TEXT Only\01xxxxxxxxx.txt (redirected: c:\zz\atlasti\fred

Historical circulation status for serial stack items (data from 1980-)

Circulated and browsed

23%

Circulated not browsed

24%

Not browsed not circulated

46%

Browsed not circulated

7%

How did qualitative methods evolve?

• Beginnings in Sociology: 1920s and 30s in the “Chicago School;” in Anthropology: in the studies by Boas, Mead, Benedict, Bateson, Evans-Pritchard, Radcliffe-Browne, and Malinowski

• Through seven moments (Denzin & Lincoln, 2001)

• Today: influences of poststructuralism and postmodernism from textual studies

How do qualitative and quantitative methods differ?

• Multiple realities, not a single one “out there” to be discovered

• Value laden, subjective rather than objective• Seeks closeness with the investigated

through interviewing and observation rather than abstract relationships

• Inductive rather than deductive• Purposeful sampling chosen for diversity

rather than random sampling

• Thick descriptions rather than crisp and terse background information

• Comfort with contradictions, ambiguity• Representations include ethnographic

prose, historical narratives, first-person accounts, still photographs, life histories, biographical and graphs, third-person narratives autobiographical materials rather than mathematical models, statistical tables

What data are collected by the qualitative researcher(researcher as bricoleur, montage maker)?

• Case studies, personal experience, introspection, life story, interview, artifacts, cultural texts and productions, observational, historical, interactional, and visual texts, statistics that describe routine and problematic moments and meanings in individuals’ lives.

• Inherently multimethod in focus: triangulation

What fields of study are included in qualitative methods?

• Ethnomethodology, phenomenology, hermeneutics, feminism, deconstructionism, ethnography, interviews, psychoanalysis, cultural studies, survey research, participant observation

Qualitative research: the method of choice

LibQUAL+ as a research example

• Purposeful sampling• Unstructured interviews - “conversations with

a purpose”• Peer review• Immediate and continuous analysis informing

further exploration• Journal• Member checks• Audit review

Establishing Trustworthiness: A Comparisonof Conventional and Naturalistic Inquiry

Criterion ConventionalTerm

NaturalisticTerm

NaturalisticTechniques

Truth value Internal validity Credibility Prolonged engagementPersistent observationTriangulationReferential adequacyPeer debriefingMember checksReflexive journal

Applicability External validity Transferability Thick descriptionPurposive samplingReflexive journal

Consistency Reliability Dependability Dependability auditReflexive journal

Neutrality Objectivity Confirmability Confirmability auditReflexive journal

Adapted from Lincoln & Guba, 1985.

Randolph High School Stability Within Transition

IndexCard Photos

ArtifactsB: Base D:District

S: School C: Calendar A: Annuals

N:Newspaper

I. A Rich History (p.44)A. Location of the base/the TAJB. Base description/accessC. Base housing descriptionD. History of the baseE. Mission of the base/Education of personnelF. History of the school district/funding

II. The High School (p.47)A. Perimeter Road

1. Description2. Stadium3. Trees

B. Campus descriptionC. High school office description/secretariesD. Teachers/ lounge descriptionE. Patio area descriptionF. Main classroom building description/display cases

III. The Principal—”Do what’s best for the kid!” (p. 51)A. Physical description

1. “Conservative”2. “Clean-cut”3. “Honest”4. “Country Gentleman”5. “Western-cut clothes”6. “Never having a hair out of place”7. “Trim”

IA,IBIBIB,IC

IE(1-3)IF(1-2)

IIA8,98IIB(1-2)IIC(1-6)

IIEIIF

IIIA2(1-2)IIIA2(1)IIIA2(2),IIIA3IIIA4IIIA4IIIA2(1)IIIA6(1-2)

4,5,66

B1,D2

7

10,11,14,80,8165,6667,6815-18,24-2830-44,49

2,90

2,90

2,902,9090,92

B1B1

A(1,2),D2,P32N(34,35,41)

A(1-5,12)

A16

The Audit TrailThe Audit Trail

Excerpted from Skipper, 1989.

Affect of Service

“I want to be treated with respect. I want you to be courteous, to look like you know what you are doing and enjoy what you are doing. … Don’t get into personal conversations when I am at the desk.”

Faculty member

Library as Place

“One of the cherished rituals is going up the steps and through the gorgeous doors of the library and heading up to the fifth floor to my study. … I have my books and I have six million volumes downstairs that are readily available to me in an open stack library.”

Faculty member

Library as Place

“I guess you’d call them satisfiers. As long as they are not negatives, they won’t be much of a factor. If they are negatives, they are a big factor.”

Faculty member

Information Control

“By habit, I usually try to be self-sufficient. And I’ve found that I am actually fairly proficient. I usually find what I’m looking for eventually. So I personally tend to ask a librarian only as a last resort.”

Graduate student

Information Control

“…first of all, I would turn to the best search engines that are out there. That’s not a person so much as an entity. In this sense, librarians are search engines [ just ] with a different interface.”

Faculty member

Dimensions ofLibrary Service Quality

Empathy

InformationControl

Responsiveness

Symbol

Utilitarian space

Assurance

Scope of Content

Ease of Navigation

Self-Reliance

Library as Place

LibraryServiceQuality

Model 3

Refuge

Affect of Service

Reliability

Convenience

Timeliness

Equipment

So…..what did we do at Texas A&M?

What are we doing now?