AGN Seminar 11 May 2006AGN Seminar 11 May 2006

The Economic and Social The Economic and Social Data Service (ESDS)Data Service (ESDS)

Louise CortiLouise CortiUK Data ArchiveUK Data Archive

AGN SeminarAGN Seminar11 May 200611 May 2006

What is the ESDS?What is the ESDS?

national data archiving and dissemination service national data archiving and dissemination service running from 1 Jan. 2003 – 2012running from 1 Jan. 2003 – 2012

Support social science -related research, learning and Support social science -related research, learning and teachingteaching

jointly supported by: jointly supported by: Economic and Social Research Council (ESRC)Economic and Social Research Council (ESRC)

Joint Information Systems CommitteeJoint Information Systems Committee (JISC)(JISC)

PartnersPartners

UK Data Archive, Essex UK Data Archive, Essex

Manchester Information & Associated Services (MManchester Information & Associated Services (MIMAS), IMAS), ManchesterManchester

Cathie Marsh Centre for Census and Survey Research Cathie Marsh Centre for Census and Survey Research (CCSR), Manchester (CCSR), Manchester

Institute for Social and Economic Research (ISER), EssexInstitute for Social and Economic Research (ISER), Essex

ESDS OverviewESDS Overview

provides access and support for key economic and social provides access and support for key economic and social datadata

distributed service, bringing together centres of expertise distributed service, bringing together centres of expertise in data creation, dissemination, preservation and usein data creation, dissemination, preservation and use

provides seamless and easier access to a range of provides seamless and easier access to a range of disparate resourcesdisparate resources for UK Higher and Further Education for UK Higher and Further Education sectors and beyondsectors and beyond

Sources / type of data supportedSources / type of data supported

Data for research and teaching purposes and used in all Data for research and teaching purposes and used in all sectors and for many different disciplinessectors and for many different disciplines

official agencies - mainly central governmentofficial agencies - mainly central government

international statistical time seriesinternational statistical time series

individual academics - research grantsindividual academics - research grants

market research agenciesmarket research agencies

public records/historical sourcespublic records/historical sources

qualitative and quantitative qualitative and quantitative

links to UK census datalinks to UK census data

access to international dataaccess to international data via links with other data archives worldwidevia links with other data archives worldwide

Collections heldCollections held

UKDA UKDA

5,000+5,000+ datasets in the collectiondatasets in the collection

200+200+ new datasets are added each yearnew datasets are added each year

18,000+18,000+ datasets distributed worldwide per yeardatasets distributed worldwide per year

History data service in-house (AHDS)History data service in-house (AHDS)

Census Registration Service in-house (CRS)Census Registration Service in-house (CRS)

databanksdatabanks

MIMASMIMAS

30+30+ series of international macro databanks series of international macro databanks

International networkInternational network

ESDS overviewESDS overview

core archiving service (UKDA) plus four core archiving service (UKDA) plus four specialist dataspecialist data servicesservices

ESDS Government (CCSR/UKDA)ESDS Government (CCSR/UKDA) ESDS International (MIMAS/UKDA)ESDS International (MIMAS/UKDA) ESDS Longitudinal (ISER/UKDA)ESDS Longitudinal (ISER/UKDA) ESDS Qualidata (UKDA)ESDS Qualidata (UKDA)

These provide:These provide:― value-added data and documentation― service-specific web pages and help― support and training ― teaching datasets/online samplers― discussion lists/newsletters― publicity and promotion promotion

Core ServiceCore Service

run by UKDA

overall integration and management of ESDS

acquiring, processing, preserving and disseminating data

central registration service operating across the ESDS

central 'first stop' help desk service

front line user support

cataloguing and describing data

maintaining and developing web presence

publicity and training

Data InData In

Data acquisitionData acquisition offers and proactive scopingoffers and proactive scoping data evaluationdata evaluation

Data ingestData ingest checking, verifyingchecking, verifying converting, formatting, processingconverting, formatting, processing documenting and contextualisingdocumenting and contextualising

Data preservationData preservation long-term data managementlong-term data management Preservation PolicyPreservation Policy UKDA is a TNA (National Archives) Recognised Place of UKDA is a TNA (National Archives) Recognised Place of

DepositDeposit

Data outData out registration using Athens including agreement to an End User registration using Athens including agreement to an End User

Licence, fine-grained access controlLicence, fine-grained access control

download service (SPSS, STATA, ASCII, RTF etc)download service (SPSS, STATA, ASCII, RTF etc)

request data on CDrequest data on CD

online data analysis, includingonline data analysis, including Simple data analysis, visualisation, downloading and subsetting Simple data analysis, visualisation, downloading and subsetting

via Nesstarvia Nesstar ESDS Qualidata Online – interview transcriptsESDS Qualidata Online – interview transcripts ESDS Government Vital Statistics onlineESDS Government Vital Statistics online International macro data via Beyond 20/20 and visualisation International macro data via Beyond 20/20 and visualisation

interfaceinterface Census dataCensus data

Access to dataAccess to data

End User LicenceEnd User Licence data can only be used by registered users (some data can only be used by registered users (some

exceptions relating to teaching)exceptions relating to teaching)

to preserve the confidentiality of, and not attempt to to preserve the confidentiality of, and not attempt to identify, individuals, households or organisations in the identify, individuals, households or organisations in the data data

to use the correct methods of citation and to use the correct methods of citation and acknowledgement in publications acknowledgement in publications

to supply the bibliographic details of any published work to supply the bibliographic details of any published work based on the data collections based on the data collections

to offer for deposit any new data collections which have to offer for deposit any new data collections which have been derived from the materials suppliedbeen derived from the materials supplied

Government dataGovernment data

General Household Survey/Continuous Household Survey (NI)General Household Survey/Continuous Household Survey (NI) Labour Force Survey/NI LFSLabour Force Survey/NI LFS Health Survey for England/Wales/Scotland Health Survey for England/Wales/Scotland Family Expenditure Survey/NI FESFamily Expenditure Survey/NI FES British/Scottish Crime SurveyBritish/Scottish Crime Survey Family Resources Survey Family Resources Survey National Food Survey/Expenditure and Food Survey National Food Survey/Expenditure and Food Survey ONS Omnibus Survey ONS Omnibus Survey Survey of English Housing Survey of English Housing British Social Attitudes/Scottish Social Attitudes/Young People’s Social British Social Attitudes/Scottish Social Attitudes/Young People’s Social

Attitudes/NI Life & TimesAttitudes/NI Life & Times National Travel SurveyNational Travel Survey Time Use SurveyTime Use Survey Vital StatisticsVital Statistics for England and Wales

Benefits of the large-scale Benefits of the large-scale government datasetsgovernment datasets

good quality datagood quality data produced by experienced research organisationsproduced by experienced research organisations UK/GB - usually nationally representative with large samples. UK/GB - usually nationally representative with large samples.

Interviewers all over countryInterviewers all over country good response ratesgood response rates well documented, NISRA, ONS, UKDA, QB etc.well documented, NISRA, ONS, UKDA, QB etc.

continuous datacontinuous data e.g. Continuous Household Survey 1983, allows comparison over timee.g. Continuous Household Survey 1983, allows comparison over time data is largely cross-sectional data is largely cross-sectional

hierarchical datahierarchical data individual and household intra-household differencesintra-household differences household effects on individualshousehold effects on individuals

Nesstar serviceNesstar service

browse detailed information (metadata) about these data browse detailed information (metadata) about these data sources, including links to other sourcessources, including links to other sources

do simple data analysis and visualisation on microdatado simple data analysis and visualisation on microdata

bookmark analysesbookmark analyses

download the appropriate subset of data in one of a download the appropriate subset of data in one of a number of formats (e.g. SPSS, Excel)number of formats (e.g. SPSS, Excel)

ESDS Longitudinal Data ESDS Longitudinal Data

five main studies, that are primarily UK Research Council: five main studies, that are primarily UK Research Council:

British Household Panel Survey (BHPS)British Household Panel Survey (BHPS) British Birth Cohort studies:British Birth Cohort studies: National Child Development Survey (NCDS)National Child Development Survey (NCDS) British Cohort Study 1970 (BCS70)British Cohort Study 1970 (BCS70) Millennium Cohort Study (MCS) Millennium Cohort Study (MCS) English Longitudinal Study of Ageing English Longitudinal Study of Ageing possible forthcoming Medical Research Council population study possible forthcoming Medical Research Council population study

datasets datasets – 1946 Birth Cohort1946 Birth Cohort– ALSPACALSPAC

Longitudinal dataLongitudinal data

longitudinal surveys involve repeated surveys of longitudinal surveys involve repeated surveys of the same individuals at different points in timethe same individuals at different points in time

allow researchers to analyse change at the allow researchers to analyse change at the individual levelindividual level

more complex to analysemore complex to analyse

British Household Panel Survey British Household Panel Survey collected and deposited by the ULSC at Essexcollected and deposited by the ULSC at Essex

follows the members of 5500 households first sampled in 1991follows the members of 5500 households first sampled in 1991

interviews conducted annuallyinterviews conducted annually

become a major resource for understanding the dynamics of British become a major resource for understanding the dynamics of British householdshouseholds

coverage includes:coverage includes: income, labour market behaviour, social and political values, income, labour market behaviour, social and political values,

health, education, housing and household organisationhealth, education, housing and household organisation

recently large new samples were introduced in Scotland, Wales and recently large new samples were introduced in Scotland, Wales and Northern IrelandNorthern Ireland

British Birth Cohort StudiesBritish Birth Cohort Studies

impact of childhood conditions on later life and understanding impact of childhood conditions on later life and understanding children and families in the UKchildren and families in the UK

national Child Development Study follows a cohort born in a single national Child Development Study follows a cohort born in a single week in 1958 - data collected at birth & ages 7, 11, 16, 23, 33, 42 week in 1958 - data collected at birth & ages 7, 11, 16, 23, 33, 42

1970 British Cohort Study follows a cohort born in a single week in 1970 British Cohort Study follows a cohort born in a single week in 1970 - data collected around birth & ages 5, 10, 16, 26 and most 1970 - data collected around birth & ages 5, 10, 16, 26 and most recently at age 30recently at age 30

Millennium Cohort Study focuses on children born in 2000/ 2001 - Millennium Cohort Study focuses on children born in 2000/ 2001 - first sweep at 9 months, second sweep at 3 yearsfirst sweep at 9 months, second sweep at 3 years

wide range of social, economic, health, medical and psychological wide range of social, economic, health, medical and psychological issuesissues

ESDS International data portfolioESDS International data portfolio

regularly updated macro-economic time series datasets from selected regularly updated macro-economic time series datasets from selected major international statistical databanks that collectively chart over 50 major international statistical databanks that collectively chart over 50 years of global economic, industrial and political change:years of global economic, industrial and political change:

the International Monetary Fund the International Monetary Fund the OECD the OECD the United Nationsthe United Nations the World Bank the World Bank EurostatEurostat the International Labour Organisationthe International Labour Organisation the UK Office for National Statisticsthe UK Office for National Statistics access for UK HE/FE onlyaccess for UK HE/FE only

access to micro data surveysaccess to micro data surveys EurobarometersEurobarometers International Social Survey ProgrammeInternational Social Survey Programme other social data via other national data archivesother social data via other national data archives

International data themesInternational data themes

economic performance and developmenteconomic performance and development trade, industry and marketstrade, industry and markets employmentemployment demography, migration and healthdemography, migration and health governancegovernance human development human development social expendituresocial expenditure educationeducation science and technology science and technology land use and the environmentland use and the environment

ESDS QualidataESDS Qualidata

access and support for a range of qualitative datasets, hosted by access and support for a range of qualitative datasets, hosted by the UK Data Archivethe UK Data Archive

data from National Research Council (ESRC) individual and data from National Research Council (ESRC) individual and programme research grant awards programme research grant awards

data from ‘classic’ social science studiesdata from ‘classic’ social science studies

other funders/sourcesother funders/sources

focus on DIGITAL Collections, but also facilitate paper-based focus on DIGITAL Collections, but also facilitate paper-based archivingarchiving

Types of qualitative dataTypes of qualitative data

diverse data types:diverse data types: iin-depth interviewsn-depth interviews ; ; semi-structured semi-structured interviewsinterviews; f; foocucus groups; s groups; oral histories; mixed methods oral histories; mixed methods datadata; ; oopen-ended survey qpen-ended survey questionuestionss; c; case notes/records ase notes/records of meetingsof meetings; d; diariesiaries/ / research diariesresearch diaries

multimedia: multimedia: audio, video, photosaudio, video, photos and t and textext (most (most common is interview transcriptions)common is interview transcriptions)

formats: formats: digital, paper, analogue audio-visualdigital, paper, analogue audio-visual

data structures - differ across different ‘document data structures - differ across different ‘document typestypes

Classic study datasetsClassic study datasets

Peter Townsend – Poverty, old agePeter Townsend – Poverty, old ageand Katherine Buildingsand Katherine Buildings

Paul Thompson – oral history and EdwardiansPaul Thompson – oral history and Edwardians

Ray Pahl –Hertfordshire Villages studiesRay Pahl –Hertfordshire Villages studies

National Social Policy and Social Change ArchiveNational Social Policy and Social Change Archive

Online access to dataOnline access to data

Other dataOther data

Many other sources across disciplinesMany other sources across disciplines

Can do catalogue search or browse by subjectCan do catalogue search or browse by subject

Major studies pageMajor studies page

New releasesNew releases

Catalogue record metadata Catalogue record metadata

Study documentation Study documentation

Help for usersHelp for users

Help desk and dedicated data Help desk and dedicated data pagespages

JISCmail list JISCmail list

regularly updated web-based FAQs regularly updated web-based FAQs

programme of training courses and publicity eventsprogramme of training courses and publicity events

news bulletins and articles news bulletins and articles

resources (links to other sites)resources (links to other sites)

teaching datasets and/or exemplarsteaching datasets and/or exemplars

enhanced documentation e.g.enhanced documentation e.g.

dataset and software guidesdataset and software guides

statistical guides (SPSS, Stata, weighting)statistical guides (SPSS, Stata, weighting)

variables consistent over time on specific surveys (ESDS Government)variables consistent over time on specific surveys (ESDS Government)

thematic guidesthematic guides

The Census Registration Service The Census Registration Service

the Census Registration Service provides one-stop the Census Registration Service provides one-stop registration and support for access to: registration and support for access to:

Census Dissemination Unit from MIMAS – aggregate Census Dissemination Unit from MIMAS – aggregate tables/Caswebtables/Casweb

Census Geography Data Unit (UKBORDERS) from Census Geography Data Unit (UKBORDERS) from EDINA – boundaries data EDINA – boundaries data

Census Interaction Data Service (Universities of Leeds Census Interaction Data Service (Universities of Leeds and St Andrews) - flow dataand St Andrews) - flow data

Samples of Anonymised Records from CCSR – micro Samples of Anonymised Records from CCSR – micro datadata

CHCC - Historical Census Collection from AHDS CHCC - Historical Census Collection from AHDS HistoryHistory

AHDS HistoryAHDS History

AHDS History is one of the five Subject Centres of the AHDS History is one of the five Subject Centres of the Arts and Humanities Data Service. ahds.ac.ukArts and Humanities Data Service. ahds.ac.uk

AHDS History collects, preserves, and promotes the use AHDS History collects, preserves, and promotes the use of digital resources, which result from or support of digital resources, which result from or support historical research, learning and teachinghistorical research, learning and teaching

The AHDS History collectionThe AHDS History collection

particularly strong in 19th and 20th century economic and social particularly strong in 19th and 20th century economic and social historyhistory

census data (1881 100% sample; 1851 2% sample; lots of local census data (1881 100% sample; 1851 2% sample; lots of local census returns)census returns)

Great Britain Historical Database onlineGreat Britain Historical Database online taxation materialstaxation materials large-scale datasets of Welsh and Irish historical statistics large-scale datasets of Welsh and Irish historical statistics electoral data (poll books for local areas) electoral data (poll books for local areas) criminal court records (e.g. a collection of datasets on violent crimes criminal court records (e.g. a collection of datasets on violent crimes

1600-1900) 1600-1900) agricultural statistics (prices, output) agricultural statistics (prices, output) surveys of Scottish witchcraftsurveys of Scottish witchcraft state finance datastate finance data economic indicators/industrial production dataeconomic indicators/industrial production data

Secondary analysis potentialSecondary analysis potential

• descriptive materialdescriptive material

• comparative research, restudy or follow-up studycomparative research, restudy or follow-up study

• re-analysis/secondary analysis re-analysis/secondary analysis

• research design and methodological advancement research design and methodological advancement

• replication of published statisticsreplication of published statistics

• teaching and learningteaching and learning

Using ESDS data for teachingUsing ESDS data for teaching

tutor registers with ESDS and downloads datatutor registers with ESDS and downloads data

registers a usage (for micro data)registers a usage (for micro data)

tutor makes data available to students (if online only via tutor makes data available to students (if online only via secure network)secure network)

tutor asks students to register or sign access agreement tutor asks students to register or sign access agreement for teaching (not always necessary, but to be for teaching (not always necessary, but to be encouragedencouraged

Data repurposed for teaching and trainingData repurposed for teaching and training

teaching datasets (available for download and via teaching datasets (available for download and via Nesstar) based on ELSA, NCDS, BHPS, HSE, BCS, Nesstar) based on ELSA, NCDS, BHPS, HSE, BCS, LFS, GHS)LFS, GHS)

X4L Survey Data in TeachingX4L Survey Data in Teaching CHCC Learning and Teaching MaterialsCHCC Learning and Teaching Materials Practical Exemplars on the Analysis of SurveysPractical Exemplars on the Analysis of Surveys Understanding Social StatisticsUnderstanding Social Statistics Teaching Resources and Materials for Social Scientists Teaching Resources and Materials for Social Scientists

(TRAMSS)(TRAMSS)

Questions?Questions?

Repurposing data – issues to considerRepurposing data – issues to consider

UKDA is guardian for the data/does not own the dataUKDA is guardian for the data/does not own the data

data can usually only be accessed by registered users data can usually only be accessed by registered users

most data can be incorporated into teaching packages/courses but access most data can be incorporated into teaching packages/courses but access restricted to the relevant students and not made publicly available e.g. on restricted to the relevant students and not made publicly available e.g. on the web (some exceptions)the web (some exceptions)

any derived data must be offered for deposit with UKDAany derived data must be offered for deposit with UKDA

permission of data owners – where data are repurposed and to be made permission of data owners – where data are repurposed and to be made publicly available, permission of data owners must be soughtpublicly available, permission of data owners must be sought

copyright e.g. LFS teaching dataset Crown copyright held jointly with the copyright e.g. LFS teaching dataset Crown copyright held jointly with the ESDSESDS

IPR of authors of repurposed materialIPR of authors of repurposed material

information on how to cite and acknowledge datainformation on how to cite and acknowledge data

ContactContact

www.esds.ac.ukwww.esds.ac.uk

help@esds.ac.ukhelp@esds.ac.uk

ESDS FAQ: ESDS FAQ: www.esds.ac.uk/about/faq.aspwww.esds.ac.uk/about/faq.asp