1

Telling the storyQualitative approaches to measuring the performance

of emerging library services

Professor Peter BrophyCentre for Research in Library & Information

Management (CERLIM)Manchester Metropolitan University

United Kingdom

2

Library goodness

• How good is this library?– Quality– Effectiveness– Efficiency– Economy– Equity

• How much good does this library do?– Impact and outcomes– Value

Orr (1973)

3

The future library (2001)….• The invisible intermediary

– “Every reader his/her information object”

• The memory institution– “The collected wisdom of the centuries”

• The learning centre– “Where people meet and exchange ideas”

• The community resource– “The street-corner, drop-in, citizens’ centre”

• The traditional (but hybrid) library– “The legacy of the past and the promise of the future”

BUT ALL ARE FOCUSED ON THE USER

4

Libraries: a threat to their users?

ImageThree men in a boat

5

“Library” users in …

1: The networked landscape– Personal search replaces ‘ask a librarian’– Global search of the global library

• Google, flickr, del.icio.us; PubMed; Google Scholar

– If there’s no response in 3 seconds, try elsewhere• Then: Resources scarce, attention abundant• Now: Attention scarce, resources abundant [Dempsey]

– ‘Phone a friend’• Social networking services, wikis, …. [SCONUL VAMP]

– Where’s the text?• Discovery to delivery (d2d) essential

– Satisficing• Then: what is the best source of authoritative information?• Now: which is the most readily accessible source of adequate

information?

– Network tools which are used are those embedded in workflow

6

“Library” users in …

2: The virtual learning environment– Self-contained online environments

• all you need to study xology in one place

7

You log into WebCT

8

and that’s where you stay

PS Where’s the library?

9

“Library” users in …

• The virtual learning environment– Self-contained

• all you need to study xology in one place

– Mandated by institutions• If you study here you have to use it

– Mandated by teachers• Used for course development• Used for delivery• Used for debate• Used for assessment

– Inhospitable to embedding the library– Environment is embedded in learning workflow

(‘learnflow’)

10

“Library” users in …

3: Research practice– Big ‘science’ shifting to computational use of

huge and dynamic datasets (satellites, particle accelerators, human genome)

• Astronomy, physics, medicine, geology

• Linguistics, history

• Cross-disciplinary

– Need is for ‘curation’ throughout the lifecycle– Librarians challenged re.

skills/knowledge/organisation– Data is embedded in research practice

11

“Library” users in …

4: Scholarly publication– Pressure for open access through ‘self’ publication

from some academics– Libraries prominent in running repositories– Take-up by academic authors patchy– Because it’s not embedded in their workflow?

12

“Library” users in …

5: Personal creativity– Shift from

‘information chain’ to ‘creative circle’

– d2d [discovery to delivery] to c2c – [creation to creation]?

Create

Publish

Enquire

DiscussLearn

Enjoy

View

Acquire

13

Supporting creativity

Helping people tell their own stories– and publishing them

for others to see – and their responses

14

Supporting creativity

Create

Publish

Enquire

DiscussLearn

Enjoy

View

Acquire

Identify/Acquire Describ

ePreserv

e

Provide

Share

Enable

Teach

DesignAssess

Validate

Interpret

Affirm

Disseminate/Communicate

15

What do all these have in common?

• The E word: Embedding• ‘Users’ need library services which are

embedded in their– Workflows– Learnflows– Leisureflows– Lifeflows

16

Emerging new/revised roles for the librarian

Supporting learning, research and life where they occur – and surfacing that support within the users’ lifeflows

17

What’s distinctive about libraries?

1. They provide access to paid-for content

2. They have ‘intelligence’ about users

Useruniverse User

population

InformationuniverseInformation

populationLibrary as intermediary

Userintelligence

Informationintelligence

User interface Source interface

18

So, getting close to the users is critical

– Engage with the users’ languages and their processes of learning, research, leisure and … life

– Because they are not going to come to you

19

Linguistic codes

• The way a ‘language’ divides up the world – it’s take on ‘how things are’

• e.g. the way scientists speak of ‘matter’ having 16 dimensions

• or the fact that Hopi Indians have a conception of time that can’t be explained in English

20

“Let us imagine a language ...The language is meant to serve for communication between a builder A and an assistant B. A is building with building-stones; there are blocks, pillars, slabs and beams. B has to pass the stones, and that in the order in which A needs them. For this purpose they use a language consisting of the words 'block', 'pillar', 'slab', 'beam'. A calls them out; - B brings the stone which he has learnt to bring at such-and-such a call. - Conceive of this as a complete primitive language.”

[Wittgenstein: Philosophical Investigations]

Language games

21

Being part of the dialogue

Learning the language game– Terminology– Concepts– Shared meaning

Playing the language game– Credibility– Trust

22

Academic disciplines as social spaces “An academic discipline ……is not primarily content, in the sense of facts and principles. It is rather primarily a lived and historically changing set of distinctive social practices. It is in these practices that ‘content’ is generated, debated and transformed via certain distinctive ways of thinking, talking, valuing, acting and, often, writing and reading.”

Gee, 2003

23

Learning as social practice

“Learning is a social process that occurs through interpersonal interaction within a cooperative context. Individuals, working together, construct shared understandings and knowledge.”

Johnson et al.

24

To share in the these social networks, you need …

• A deep understanding of how the discipline or activity is practised

• leading to credibility as a contributor to the enterprise

• and thus to the standing and peer regard to supply solutions

• The following could be well-placed– Subject librarian– Community librarian– Knowledge officer

25

So how do we measure performance?

"Sometimes I lie awake at night, and I ask, 'Where have I gone wrong?'

Then a voice says to me, ‘This is going to take more than one night’."

Charles Schultz/Peanuts

26

So how do we measure performance?

• How do you measure the performance of any social system?– Ethnographic approaches

• Living in the lifeflow

– Moderated by independent, expert assessment

• Assisted self-assessment

– Painting rich pictures• Use of narrative and story

27

Ethnography• Culture. The focus of concern is on the shared culture of

groups of people, examining and seeking to understand the shared patterns of behaviour, values, norms and standards.

• Holism. The focus is on the whole system being observed, rather than on an isolated part. Meaning and purpose can only be discerned within the context of the whole system.

• In-depth studies. Ethnography is not interested in the data collected by counting or even by questioning, but relies on ‘living in’ the group that is being investigated.

• Chronology. Ethnography is interested in change, in how people and societies alter and reinvent themselves over time.

[after Sarantakos, 1998]

28

Methodological requirements1. Speak the language

• Investigators judgements are not credible unless they themselves have credibility with the target group

• Therefore an element of self-evaluation is needed

2. Understand the problem space• Investigators must understand the potential of library &

information services so as to be able to offer improved solutions to users’ problems

• Therefore LIS expertise is also needed

3. Maintain independence• Findings are not credible unless an investigator with no

vested interest can sign them off• Therefore an element of independent evaluation is needed

29

What kinds of methods?

• Observation• Conversation• Self-reflection• Textual analysis

• Questionnaires• Interviews• Focus groups

Internal

External

30

How do we construct meaning when faced with complexity?

• Use of narrative“One reason why we live in a soup of narratives, why

narratives permeate our lives and understanding, is that resorting to narratives is the way in which we have learned to cope with our world … Even while scientists and schoolteachers have been telling us to abandon these unscientific approaches, and adopt linear abstract thinking, the human race has used its common sense and stubbornly – to some extent surrepticiously – stuck with narratives as the most usable tool to cope with complexity”

Stephen Denning (2001)

31

Unveiling meaning through narrativeYou could make an objective list of the actions you performed over the last week, but if it were simply a “factual” account, it would not mean anything. But if you told us what you had done in the last week, not only would your story acquire meaning, but in telling it, both you the narrator and we the listeners would be compelled to reflect on it in order to gain a greater understanding of what had gone on. ……Just as history does not exist in nature, but is created in the telling, so, too autobiography and the medical case history emerge out of transactions which mean they are at the same time both less and more than the ‘facts’ of the case.

Greenhalgh & Hurwitz

32

Unveiling meaning through narrative

“Steadily increasing recognition of the importance of narrative in mainstream management is now inevitable. …… Narrative thinking is contributing to an emerging view of organizations that more accurately reflects not only the traditional structural, process-oriented, control-based aspects of an organization but also the living, flowing aspects of organizations – where talking, thinking, dreaming, feeling human beings work and play and talk and laugh and cry with each other, in a way that is organic and self-adjusting and naturally innovative.”

(Denning in Brown et al.)