Upload
merlien-institute
View
377
Download
2
Embed Size (px)
Citation preview
Archiving qualitative data for policy research: meeting challenges and establishing
best practices
Aileen O’Carroll
Manager, Irish Qualitative Data Archive
Irish Qualitative Data Archive
• Funded by the Irish Government under the Programme for Research in Third Level Institutions (Cycle 4)
• Several major datasets being archived, including –• Growing up in Ireland study
• Life Histories and Social Change in Ireland in 20th Century Ireland
• Re-use & Archiving of Complex Community-based Evaluation Research (RACcER) project explores issues specific to archiving community-based evaluation data, also aims to raise awareness throughout research community
Tallaght West Childhood Development Initiative (CDI)
� 1 of 3 Prevention and Early Intervention Programmes jointly funded by government & philanthropy
� Aims to improve the education and well-being outcomes for children and families in historically disadvantaged areas of Tallaght West
� 5 constituent projects: early years, literacy, pro social behaviour, healthy schools, community safety
� Major emphasis on evaluation
RACcER� RACcER co-funded by IRCHSS and CDI: demonstrator project for qualitative archiving
� Objectives:
� Scope concerns, issues and requirements in qualitative data archiving through interviews
� Explore issues and challenges for researchers, participants, funders, potential data users
� Test feasibility of archiving evaluation data
� Develop protocols and procedures for archiving evaluation data
� Research Process
Consultation > Pilot Strategy > Consultation > Archiving process
Findings – qualitative data gathered in evaluations
� Range of qualitative material collected
� Variety of participant groups
� Most material recorded, anonymised and transcribed
Findings – archiving policy
� Evaluators’ research centres didn’t generally have archiving policies for qualitative material
� Range of experience of what happens to qualitative data after project
� No evidence of researchers sharing datasets informally
� Default ethics position is to destroy after 5 years
Findings – archiving experience
� No evidence of formal archiving
� Limited experience of accessing qualitative archives,
� One example for use in teaching
� Some had archived quantitative material
� Some had accessed quantitative archives
� Range of views on resource implications of prepare qualitative material for archiving
Findings – issues with archiving
� Difficulties of anonymising in small community
“Just to follow on really from that, archiving to do what really? Archiving to make something as a resource that can be used again. In the context of some of my own work, to anonymise it sufficiently is to make it meaningless. ”
“There’s interconnections, there’s politics, and there’s alliances, and there’s you know, sort of, tensions, between people and characters and actors and you know there’s a finite cast of characters. I think in that context, it is challenging about how anonymise that work and then allow it to be available and maybe you can’t, I don’t know, maybe that’s not possible”.
“I think there are enormous issues, if you talk about archiving do you anonymise transcripts to make... so then you do all this anonymisation activity, you invest time and money in it, and what you come out with is nothing. If you remove all the contextual information and so on. Then you’ve got a set of data which would essentially be akin to what was in a report. ”
Findings – issues with archiving
Findings – issues(Consent)
� “…. could you actually be happy that you would have informed consent from somebody? That they actually fully understand what it means to have this available and what the implications are. Because, to a degree, you don’t actually know what the implications are.”
� “If we were to put on our consent forms or our information sheets that data generated from interviews will be available, anonymised, as part of an archive, we’re screwed. That’s basically it. Sorry for using such
unparliamentary language, we would not get responses, not in a country like ours. ”
Findings – issues(3)
� Wide perception of fundamental difference between evaluation research and broader topics
“we’re generally speaking about evaluation and evaluation is different. It’s not asking people whether the sky is blue or green or how you perceive that, you’re talking about somebody else’s actions or activities and putting some, essentially, value on those in what you say. So therefore, there’s another level that you have to add into the mix.”
Findings – issues(4)
� Lack of clarity on who can access data and how it could be used generates fear and reduces inclination to archive
� Concern over data being used out of context, ‘twisted’, sensationalising populations, journalists gaining access
Findings – issues(5)
� Researchers concerned about being opened up to scrutiny
� PhD/masters students using archived material could reduce valuable learning opportunities gained from fieldwork
� Resource implication of archiving
Findings – advantages
� Potential to explore alternative research questions
� Contributing to cumulative knowledge of implementation of children’s services
� Facilitate comparative analysis over time or place
Findings – advantages (2)� A valuable teaching resource
� Increase the standing of qualitative research if it is open to external examination, improved validity and reliability
� Contribute to corporate memory of commissioning organisation
Findings - risks
� Reputational risk for organisations involved at all levels
� Negative publicity risk for researched community
� User agreement could be breached
� Anonymity could be compromised –
� But...
� Need to consider likelihood of risk occurring and how serious it would be if it happened and develop approperiate archiving strategies
Findings – mitigating risks� Strict controls on user access
� Archive to facilitate collaboration between researchers
� Hierarchy of access, tailored depositor and user agreements
Consultation
With
CDI Board
Building
Informed Consent
Data
Collection
Data
CostingData
Anonymisation
Process of archiving CDI datasets
Accessing Sensitivity level of the data
Risk of Identification Risk Of Harm Sensitivity Level
Little Low Low
Some Low Medium
Any High High
Archiving Strategy Appropriate to the Level of Sensitivity
Sensitivity
Level
Participant
Consent
Anon-
ymised
Access Remove
Sensitive
Text
Embargo
(Anonymised
Sensitive
Text
Removed)
30 Year
Embargo
(Sensitive
Text Not
Removed
Low Yes Yes Restricted N/A No No
Medium Yes Yes Restricted No No No
High Yes Yes Restricted Yes No No
Thank You
RACcER Team
�Irish Qualitative Data Archive
�Jane Gray
�Julius Komolafe
�Hazel O’Byrne
�Aileen O’Carroll
�Tallaght West CDI
�Tara Murphy