Embed Size (px)
DESCRIPTION
Simon will consider how we can use a deeper understanding of value and impact to survive in an ever more competitive and confusing digital landscape. How do the cultural, heritage or creative sectors cope with the twin challenges of meeting the public desire for digital content whilst maintaining their curatorial responsibilities within what could be considered an unfunded mandate? Simon will investigate the values and benefits of digital with a consideration of the risks we face in what he refers to as the Digital Death Spiral. Simon will propose one solution in particular, The Balanced Value Impact Model (BVI Model) that he has recently developed. The BVI Model draws evidence from a wide range of sources to provide a compelling account of the means of measuring the impact of digital resources and using evidence to advocate how change benefits people. Simon will argue that putting people at the centre of our strategic thinking is both the most challenging and satisfying action we can take in securing our digital futures.
Citation preview
Today’s Agenda!
Why am I here?
The Digital Death Spiral (duh, dah, daaah)
Avoiding the Death Spiral
Impact – understanding how you have made a difference
Some thoughts on value
The case for Impact
We are more effective and efficient in delivering change and tangible benefits (Internal Impact);
Our organisation is gaining strategic advantage through the innovation inherent in this digital activity (Innovation Impact);
We are delivering a strong economic benefit to our community that demonstrate the worth and value of our endeavours in clear monetary terms (Economic Impact); and
the community has been changed by the resource in beneficial ways that can be clearly identified (Social Impact)
Digital Humanities:
the application of digital technology to humanities disciplines
reflection upon the impact of digital media upon humanity
> 50 academics & researchers
~ £2.5 million research income per annum
5+ million digital objects in 107+projects
200+ million hits over the last 5 years
www.kcl.ac.uk/ddh/
http://www.youtube.com/watch?v=tp4y-_VoXdA
Digital Humanities methods for historical analysis of
Irish Immigrants in 19th Century London, England
http://bit.ly/datasea2
Data Sea 2.0: a real-time artistic
representation of the Radiosphere
simon-tanner.blogspot.co.uk http://bit.ly/1cJBBDr
The Midnight Run
Is the value in the wine, the glass or the drinking?
www.kdcs.kcl.ac.uk/innovation/inspiring.html
http://www.kdcs.kcl.ac.uk/innovation/impact.html
Measuring the Impact of Digitized Resources: The Balanced Value Model
A Digital Death Spiral?
“digitisation = funding”
“Digital is everything today”
“who knows how much it’ll cost, but digital’s bound to be wonderful”
“Planning is so 20th Century, let’s be Agile”
“cos our competition / Google / my mate is doing it”
“cos if we build it, they will come!”
Signs you are in the Digital Death Spiral
Digital Death Spiral: the shocking & stealthy
Curation Challenges & Unfunded Mandates
Digitisation
Web Archiving
Collection Development
Material heritage
Intellectual
heritage
Digital Preservation
Virtual
heritage
Web 2.0 /
Interactive heritage
Born digital
Preservation
&
Conservation
“the measurable outcomes arising from the existence of a
digital resource that demonstrate a change in the life or life
opportunities of the community”
www.kdcs.kcl.ac.uk/innovation/impact.html
The case for Impact
We are more effective and efficient in delivering change and tangible benefits (Internal Impact);
Our organisation is gaining strategic advantage through the innovation inherent in this digital activity (Innovation Impact);
We are delivering a strong economic benefit to our community that demonstrate the worth and value of our endeavours in clear monetary terms (Economic Impact); and
the community has been changed by the resource in beneficial ways that can be clearly identified (Social Impact)
www.kdcs.kcl.ac.uk/innovation/impact.html
www.kdcs.kcl.ac.uk/innovation/impact.html
Impact taskforce
Simon Tanner
Aggregated
• 2200+ content
providers
• 173 aggregators
• 26.9 million objects
Network
• 600+ individual
members, working
in taskforces and
on strategy
Breakdown
Galleries: 2
Libraries: 111
Archives: 26
Museums: 60
National Aggs: 22
Publishers: 2
Creative Ind: 5
Research: 78
Ministries: 9
Other: 174
www.kdcs.kcl.ac.uk/innovation/impact.html
PERSPECTIVE
+ VALUE
DRIVER OBJECTIVES STAKEHOLDERS AREAS MEASURED METHODS INDICATORS
CURRENT
USE (SOCIAL)
Users
Those with
an interest
in the
intellectual
content find
it useful to
their
research
Users of the
Code-
breakers
resource.
1. Discovery
2. Engagement
3. Usefulness
Analytics review
Site surveys
sampling users
on the
Codebreakers
microsite.
Tracking of
recommendatio
ns to others or
reviews.
User panels.
Citations of
content on the
site.
1. Discovery of resource:
a. Web visits/visitors
b. Views to digitised content
c. Relative use to historical use figures (where
applicable)
1. Engagement with resource:
a. Average time spent on digitised content
b. Repeat visitors to Codebreakers
c. Downloads of content
d. User journeys across Codebreakers
1. Usefulness of resource:
a. Site surveying to collect reported usage and
utility of Codebreakers resource
b. Citation indices – this can only be a very long
term measure due to research and publishing
timescales
c. User panel – recruited from actual users of
the Codebreakers resource. Engage in enquiry of
the function, content and discoverability of
Codebreakers.
d. Desk research to find innovative use of the
content
e. Online media monitoring to capture people’s
mentions and recommendations of Codebreakers.
PERSPECTIVE +
VALUE DRIVER
OBJECTIVES STAKEHOLDERS AREAS MEASURED METHODS INDICATORS
CURRENT USE (SOCIAL)
Community
Peer
organisation
s and
members of
our
professional
community
have
changed
their policy
or practice
concerning
digitisation
projects.
Practitioners
, peer
organisation
s and
members of
our
professional
community
who have
been
influenced
by the
project
1. Awareness
of the
project
2. Take up of
methods/
approaches
/ standards
3. Impact of
take up on
partner and
peer
organisatio
ns
Survey of
partner
organisations
who worked on
the project
Survey of event
attendees / key
peer
organisations
Desk research
1. Awareness:
a. Number attending digitisation open days/events
held at Wellcome Library
b. Contacts from peer organisations/practitioners
c. Online media monitoring for blogs, conference
presentations, events, workshops, open days,
social media etc.
d. Citations/references to the project
1. Take up:
a. Survey of partner and peer organisations, and
practitioners identified in stage 1.
b. Desk research to identify stakeholders influenced
by the Codebreakers project (e.g. Ronan Deazley
work on archives and copyright)
1. Impact:
a. Survey of partner and peer organisations, and
practitioners identified in stage 1.
b. Desk research
PERSPECTIVE
+ VALUE
DRIVER OBJECTIVES STAKEHOLDERS AREAS MEASURED METHODS INDICATORS
POTENTIAL
USE (INNOVATION)
Users
The
Codebreakers
project has
enabled new
potential
activities and
research
methods for
those
interested in
the intellectual
content.
Potential users
of the
Codebreakers
resource
1. Delivery of
the planned
functionality/
usability of
the
Codebreakers
resource.
2. User
understandin
g of the new
research
enabled by
Codebreakers
.
3. Unforeseen
potentials for
new research.
Evaluation of
functional
capabilities of the
finished site against
our initial goals.
Heuristic evaluation
of Codebreakers
resource.
Site survey.
Focus group with
potential users
1. Delivery:
a. Desk work to check the functionality of the site against our
initial specification.
b. Heuristic evaluation of usability
1. User understanding:
a. Site survey of users on Codebreakers microsite
b. Focus group research
c. Usability research with potential users
d. Heuristic evaluation
1. Unforeseen potentials:
Staff interviews to discover where unexpected benefits
occurred during the build of the site.
Focus group recruited from potential Codebreakers users to
discuss new opportunities for researchers.
Community
The extent to
which
Codebreakers
has created
new
possibilities for
organisations
and
professional
members of
the cultural
heritage
community.
Practitioners,
peer
organisations
and members
of our
professional
community
who may be
influenced by
the project
1. The
accessibility
of technical
developments
.
2.
Awareness of
access to
developments
.
3. Uptake of
practices
initiated in the
project as
industry
standard.
4. Unforeseen
potentials.
Desk work to
determine
accessibility of
technical and
process
developments.
Qualitative
measures such as
questionnaires,
desk research,
structured
interviews. A
comparison of our
initial goals with the
final site.
1. Accessibility of developments:
a. Desk work to check accessibility of technical developments
to the peer community
b. Availability of documentation of process/organisational
developments
1. Awareness of access:
a. Survey of partner and peer organisations, and practitioners
identified in Current Community Awareness.
1. Uptake as industry standards:
a. As described in Current Community uptake
1. Unforeseen potentials:
a. Survey of partner and peer organisations, and practitioners
identified in Current Community Awareness.
b. Staff interviews to discover where unexpected benefits
occurred during the build of the site.
PERSPECTIVE
+ VALUE
DRIVER OBJECTIVES STAKEHOLDERS AREAS MEASURED METHODS INDICATORS
INTERNAL
Development
How have the
staff of the
Wellcome
Trust had their
skills, abilities,
capacity and
knowledge
enhanced by
developing
Codebreakers
Staff of the
Wellcome
Library and
Trust.
1. Changes in
individual
knowledge or
skills.
2. Changes in
working
practices and
behaviours
3. Changes in
organisation
al capacity or
ability.
Survey of Trust
staff connected
with the project.
Interviews with
line-managers.
Interviews with
senior managers.
1. Individual knowledge:
a. Survey of Trust staff connected to Codebreakers activity.
b. Interviewing line-managers of staff involved in the
Codebreakers project.
1. Working practices and behaviours:
a. Survey of Trust staff connected to Codebreakers activity.
b. Interviewing line-managers of staff involved in the
Codebreakers project.
1. Changes to organisation:
a. Interviews with senior managers.
Inheritance /
Bequest
How does
Codebreakers
represent the
inheritance of
the Wellcome
Trust Library’s
activities since
collecting
began and
how does it
prepare the
Library for the
future and
bequeath
benefits to
future
generations?
Staff and
members of
the Wellcome
Trust.
1. Change in
usage
enabled by
Codebreaker
s resource.
2. Value for
future
digitisation
activity
3. Benchmarkin
g against
peer
organisations
.
4. Comparison
to historical
strategic
direction of
Library.
Google Analytics
Interviews with
senior managers
Review of peer
organisation
activity
Desk research
Change in usage:
a. See Current Users usage
Value for future digitisation activity:
a. Interviews with senior managers
b. Data from Internal Development
Benchmarking:
a. Review of peer organisations – desk research and
interviews to compare the Wellcome Library’s digital
status in comparison with its peers worldwide.
Comparison to historical strategic direction:
a. Desk research
b. Interviews with senior managers
PERSPECTIVE
+ VALUE
DRIVER
OBJECTIVES STAKEHOLDERS AREAS MEASURED METHODS INDICATORS
ECONOMIC
Users
What is the
net
economic
effect of
making the
content
freely
available
online?
Users of the
Codebreaker
s resource.
Economic gain
to individual
users of the
resource.
Economic value
generated for
organisations
that are end-
users of the
resource.
Google Analytics
review
Site survey
User panels.
This will be based on methodology developed by the
British Library in their 2013 economic evaluation. The
full British Library report is available here. It will include:
1. Comparison of Codebreakers usage with archive
usage records over the last 5 years with an
assessment of the cost of use.
2. User time spent on the Codebreakers resource
3. Users’ geographic location.
4. Equivalent cost implication for users consulting
across collections previously held in physically
separate locations.
5. Contingent valuation questions included in site
survey and user panels.
Thanks!
Thank you to all the folks at the Wellcome Library for allowing me to share this with you. Contacts Alexander Green Email: [email protected] Christy Henshaw Email: [email protected]
www.kdcs.kcl.ac.uk/innovation/impact.html
http://simon-tanner.blogspot.com/
http://simon-tanner.blogspot.com/
“That’s the first time, in that room, that I’ve written what I feel, responded to those questions and left it up there for anybody else to read – for the first time in the last 10 years. I didn’t let myself worry about being judged or whether it was good enough, whatever, I just left it out there. And there was some peace came with that.... I just allowed myself to be and I feel enriched, I feel energised by that and empowered by that.”
With thanks to Alice Maggs for the Impact illustrations [email protected]
