Upload
mrinal
View
37
Download
1
Embed Size (px)
DESCRIPTION
Evaluating Information & Knowledge Services Using Narrative Techniques - a case study. Nerida Hart Chair, Knowledge Management Division Special Libraries Association Information Online 2009. Presentation Outline. What was this project all about? Potential solutions - PowerPoint PPT Presentation
Citation preview
Evaluating Information & Knowledge Services Using
Narrative Techniques - a case study
Evaluating Information & Knowledge Services Using
Narrative Techniques - a case study
Nerida Hart
Chair, Knowledge Management Division
Special Libraries AssociationInformation Online 2009
Nerida Hart
Chair, Knowledge Management Division
Special Libraries AssociationInformation Online 2009
Presentation OutlinePresentation Outline
What was this project all about? Potential solutions What are narrative techniques? Knowledge & Information Services approach How K&IS used narrative techniques Limitation Results Conclusions
What was this project all about? Potential solutions What are narrative techniques? Knowledge & Information Services approach How K&IS used narrative techniques Limitation Results Conclusions
BackgroundBackground
2005 managing a special library (knowledge
service) with approximately 40,000 clients 5 agencies being serviced - FaCS, Centrelink,
Child Support Agency, Social Security Appeals Tribunal, Indigenous Affairs
> 40 staff grouped into 5 service delivery sections
2005 managing a special library (knowledge
service) with approximately 40,000 clients 5 agencies being serviced - FaCS, Centrelink,
Child Support Agency, Social Security Appeals Tribunal, Indigenous Affairs
> 40 staff grouped into 5 service delivery sections
What was the problem we needed to solve?
What was the problem we needed to solve?
We were due for the dreaded review !!!! How did we prove our worth within our client
groups? Time to do this was limited and we didn’t want
to boil the ocean but concentrate on our biggest value add service - “research services”
We were due for the dreaded review !!!! How did we prove our worth within our client
groups? Time to do this was limited and we didn’t want
to boil the ocean but concentrate on our biggest value add service - “research services”
How did we go about proving our worth…How did we go about proving our worth…
Library statistics only measure input and output and we needed to show how we contributed to client outcomes
Need to cater for emergent benefits (the real benefits of the research service)
What were the real experiences of our clients Need to encourage good practices and disrupt poor
practices Looking for ways of continuously improving products
and services
Library statistics only measure input and output and we needed to show how we contributed to client outcomes
Need to cater for emergent benefits (the real benefits of the research service)
What were the real experiences of our clients Need to encourage good practices and disrupt poor
practices Looking for ways of continuously improving products
and services
Potential methodologiesPotential methodologies
Surveys - great for the number devoted managers who need ‘hard’ evidence
Focus groups - can be manipulated Interviews - time was against us Narrative - excellent - would give us
‘depth’ but which narrative technique to use
Surveys - great for the number devoted managers who need ‘hard’ evidence
Focus groups - can be manipulated Interviews - time was against us Narrative - excellent - would give us
‘depth’ but which narrative technique to use
Cynefin Complexity Framework
Cynefin Complexity Framework
Why Anecdote Circles as a narrative technique?
Why Anecdote Circles as a narrative technique?
Anecdote circles use anecdotes of personal experiences of workers to gain evidence of what is really happening in a complex organisation
They are more open-ended than focus groups and allow for the unexpected.
Focus groups are more concerned with opinions and judgements than anecdotes which are more concerned with personal experiences
Anecdotes reveal the values and behaviours of the staff – make sense of the organisation’s culture
Anecdotes link events in a meaningful way
Anecdote circles use anecdotes of personal experiences of workers to gain evidence of what is really happening in a complex organisation
They are more open-ended than focus groups and allow for the unexpected.
Focus groups are more concerned with opinions and judgements than anecdotes which are more concerned with personal experiences
Anecdotes reveal the values and behaviours of the staff – make sense of the organisation’s culture
Anecdotes link events in a meaningful way
Components of Narrative Approach using Anecdote
Circles
Components of Narrative Approach using Anecdote
Circles
Employing Narrative approach required dividing the work into 4 phases: Preparation (1/2 day) Discovery – Anecdote circles (90-120 minutes
each) Sensemaking (1 day) Intervention Design – using complexity principles
(i.e. action planning and implementation) (1/2 to 1 day)
Employing Narrative approach required dividing the work into 4 phases: Preparation (1/2 day) Discovery – Anecdote circles (90-120 minutes
each) Sensemaking (1 day) Intervention Design – using complexity principles
(i.e. action planning and implementation) (1/2 to 1 day)
Knowledge & Information Services Approach
Knowledge & Information Services Approach
Project was part of project management certification for a staff member
Parallel survey – as a control group
Narrative – use of anecdotes
Full skills and knowledge transfer from our consultants Anecdote Pty Ltd (www.anecdote.com.au)
Project was part of project management certification for a staff member
Parallel survey – as a control group
Narrative – use of anecdotes
Full skills and knowledge transfer from our consultants Anecdote Pty Ltd (www.anecdote.com.au)
Use of Narrative in K & ISUse of Narrative in K & IS
Preparation: Identification of themes:
Client information seeking behaviour Quality Accuracy and authenticity Client experiences with research staff
Preparation: Identification of themes:
Client information seeking behaviour Quality Accuracy and authenticity Client experiences with research staff
DiscoveryDiscovery
Discovery – Anecdote circles Four sessions including one regional
session 28 participants in total (6-12 per session) All sessions recorded and transcribed Anecdotes extracted from the transcripts All participants were not personally
identified in the report
Discovery – Anecdote circles Four sessions including one regional
session 28 participants in total (6-12 per session) All sessions recorded and transcribed Anecdotes extracted from the transcripts All participants were not personally
identified in the report
Survey results vs Anecdotes
Survey results vs Anecdotes
Was the information you received what you required?
Relevant/useful
Too much
Too little
Just enough
Other
Was the information you received what you required?
Relevant/useful
Too much
Too little
Just enough
Other
“We were thinking about FaCS policy priorities. The bibliography the library sent me, at very short notice, was concentrated on sources like the OECD and other government publications, because I think there was an acknowledgement that it we were going to try to introduce it to our Department we had to have sources that would be credible.”
“We were thinking about FaCS policy priorities. The bibliography the library sent me, at very short notice, was concentrated on sources like the OECD and other government publications, because I think there was an acknowledgement that it we were going to try to introduce it to our Department we had to have sources that would be credible.”
Anecdote example : Depth of Information
Anecdote example : Depth of Information
“When I got the material it was not really what (I wanted). In the end I didn’t use very much of what was suggested to me. Given that I didn’t go back and define more closely and try to work it through with the person… so it was a time factor more than anything else. I would have gone back. In the end it was easier for me to go the way I did go.”
“When I got the material it was not really what (I wanted). In the end I didn’t use very much of what was suggested to me. Given that I didn’t go back and define more closely and try to work it through with the person… so it was a time factor more than anything else. I would have gone back. In the end it was easier for me to go the way I did go.”
SensemakingSensemaking
Sensemaking One day workshop Used an independent facilitator
Objectivity Distancing ourselves from the results and emotional
involvement Enables future credibility
Identifying themes, behaviours and archetypes
Sensemaking One day workshop Used an independent facilitator
Objectivity Distancing ourselves from the results and emotional
involvement Enables future credibility
Identifying themes, behaviours and archetypes
Interventions / Action Planning
Interventions / Action Planning
Interventions Marketing for greater awareness of our services Follow up after research completed Clarification of client research requirements Revamping parts of our intranet site Enhancing our role as an information and
knowledge sharing catalyst (connectors)
Interventions Marketing for greater awareness of our services Follow up after research completed Clarification of client research requirements Revamping parts of our intranet site Enhancing our role as an information and
knowledge sharing catalyst (connectors)
LimitationsLimitations
Geographic dispersal of the clients Getting the numbers and timing right –
availability of participants (Welfare to Work priorities of FaCS and Centrelink)
Christmas looming Convincing our clients that this was important
– benefits to both us and them
Geographic dispersal of the clients Getting the numbers and timing right –
availability of participants (Welfare to Work priorities of FaCS and Centrelink)
Christmas looming Convincing our clients that this was important
– benefits to both us and them
ConclusionsConclusions
Benefits to K&IS and our clients: Tangible evidence of K&IS contribution to core
business through real examples of time saving, authenticity of information, contribution to the big picture goals of the client agencies
Skills and knowledge development of K&IS staff Stronger relationships with K&IS clients and increase
in return business Improvements in our training methods for K&IS clients Improvements in research processes which then
benefits the client groups
Benefits to K&IS and our clients: Tangible evidence of K&IS contribution to core
business through real examples of time saving, authenticity of information, contribution to the big picture goals of the client agencies
Skills and knowledge development of K&IS staff Stronger relationships with K&IS clients and increase
in return business Improvements in our training methods for K&IS clients Improvements in research processes which then
benefits the client groups
ReferencesReferences
How to run this process www.rkrk.net.au
Anecdote Pty Ltd www.anecdote.com.au
Dave Snowden’s Cynefin Framework www.cognitive-edge.com
How to run this process www.rkrk.net.au
Anecdote Pty Ltd www.anecdote.com.au
Dave Snowden’s Cynefin Framework www.cognitive-edge.com
Questions and Comments
Contact details: [email protected] in: www.linkedin.com/in/neridahart
Skype: Neridahartau