Dr Juliet Hassard Deputy Director, Centre for Sustainable Working Life Lecturer in Occupational...

Dr Juliet Hassard

Deputy Director, Centre for Sustainable Working Life

Lecturer in Occupational Health Psychology

* Secondary Data Analysis: An Introduction

*Overview of presentation

*What is secondary data analysis?

*Types and sources of data

*Opportunities, limitations, and challenges

*Ethics

*Thinking forward: funding and publishing.

*Secondary data analysis: why

and what it is?

*The use of secondary data, or existing data that are freely available to researchers who were not involved in the original study, has a long and rich tradition in the social sciences [1].

*Sociology, economics, etc.

*Why collect new data, given the wealth of existing data sets that can be used to answer important questions?

*Longitudinal & large sample sizes.

* Traditionally, the field of psychology (any many of those within it) have dismissed the importance and value of studies using secondary data.

*But times are changing…….

*Why use secondary data?

*To ask and answer important questions. For example,

*To understand the longitudinal nature of relationships.

*To understand group differences, trends over time?

*To explore new and emerging social phenomena.

*Why secondary data analysis?

*More data (and types of data) are being collected (and available!) then ever before.

*There is a unique opportunity to explore this ever growing source(s) of data, and to ask important research questions.

*Types and sources of data….

*Let’s get creative……..

*In small groups of 3-5. Discuss and outlines 4-5 different types of data/ types of information that could be used to investigate an important psychological research question.

Online support groups

Second life

App technology

Blogs Chat forums

Published business reports

*Where do I find data?

*The UK Data Service

* https://www.ukdataservice.ac.uk/

* Census data

* International macrodata

* Longitudinal studies

* Qualitative/mixed methods

* UK surveys

*The National Data Service

* http://www.nationaldataservice.org/about/

* Individual studies may have different access points.

* E.g., Whitehall II Study, UCL.

*Secondary data is everywhere –

some in the public forum.

*Examples

*Online support groups:

* COULSON, N.S., 2015. Exploring patient's engagement with web-based peer support for Inflammatory Bowel Disease: forums or Facebook? Health Psychology Update. 42(2), 3-9.

*Longitudinal data (Whitehall II survey)

* Kouvonen, A., et al . (2011). Negative aspects of close relationships as a predictor of increased body mass index and waist circumference: the Whitehall II study. American journal of public health, 101(8), 1474-1480.

*Twitter, Instragram…..

* Whiting, R., & Pritchard, K. (2015). “Big Data? Qualitative Approaches to Digital Research", Qualitative Research in Organizations and Management: An International Journal, Vol. 10 Iss: 3, pp.296 - 298

*Advantages, Limitations, &

Challenges

Low response

rate

Small sample size

Reliance on convenience

samples

Access to high quality

measures

Limited money &

resources to collect

primary data

Limited scope for extensive

comparative research (across

groups or internationally)

Correlation does not equal causation

‘Traditional’ Challenges in Psychological

Research

High attrition rates

*Advantages

*The data has already been collected.

*Save time – primary researcher does not have to design study and collect new set of data.

*The types of data that are typically collected tend to be higher quality than could be obtained by individual researchers.

* Typically longitudinal, have large sample sizes that have been obtained using elaborate sample plans.

Ref: Trzesniewski et al., 2011

*Advantages

*Learning how to work with, manage and analyse secondary data can provide individual researchers with the raw materials to make important contributions to the scientific literature

*… using data sets with impressive levels of external validity.

Ref: Trzesniewski et al., 2011

*Advantages

*Open-source approach to research

*Replicate findings using similar analyses

*Encourages careful reporting and justification of analytical decisions.

*Allows researchers to test alternative explanations and competing models.

*Encourages transparency, which in turns help facilitates good science.

Ref: Trzesniewski et al., 2011

*Disadvantages

*The data has already been collected!!!

*You may not have all the information on how or why certain types of information was collected.

*You may not know of any particular problems that occurred during data collection.

*Sometimes you are left wanting more …..

*Disadvantages

*The temptation: a statistical fishing trip.

*Great research is driven by a good research question that is strongly underpinned and shaped by theory.

*The purpose of analysing data is to refine the scientific understanding of the world and to develop theories by testing empirical hypotheses.

* “Mo Money Mo Problems” - Mo Data, Mo Temptations ?

*A note about statistical power. Ref: Trzesniewski et al., 2011

*Disadvantage

*Considerable time and effort:

* is invested by the researcher to understand the nature and structure of a data set.

* is needed by the researcher to explain and justify the theoretical and analytical approached used.

*Although, I would argue there is real advantages to the time invested in doing this.

Ref: Trzesniewski et al., 2011

*Disadvantage

*Measures in these datasets are often abbreviated. Often because the projects themselves were designed to serve multiple purposes and to support a multidisciplinary team.

* Shortened measures, mix-levels of data, and single items measures.

*These datasets often have impressive levels of breadth (many constructs are measured), but often with an associated cost in terms of depth of measurement.

*Therefore, measurement issues are ~ therefore ~ one of the major issues in secondary data analysis

* These issues often require quite a bit of conceptual consideration & defending in the peer-review process.

Ref: Trzesniewski et al., 2011

*Challenges

*A good grounding in psychometrics and Classic Test Theory.

*You need to carefully consider and evaluate the trade-offs in reliability and validity.

*You need to defend your position when writing up.

*You need to understand how measurement issues frame your findings; and, in turn, your interpretation of your findings. Ref: Trzesniewski et al., 2011

*Practical & Methodological

Challenges

*Creating and managing data files

* Data inventory

* Research journal

*Approach to missing data and data screening procedures

*Use of and/or development of constructs

*Use of proxy variables

*Development & testing of composite measures

* Single item measures

*Accounting for the data structure in your analysis

*Case study: An Example

*MODELLING GENDER-RELATED DIVERSITY IN PSYCHOSOCIAL PROCESSES

AND WORK-RELATED WELLBEING: PATHWAYS AND MECHANISMS

*The aim of the doctoral thesis was to develop and test a theoretical model seeking to describe the aetiological role of psychosocial processes, in and out of the workplace, in predicting gender-related diversity issues in men’s and women’s health at a structural/population level.

*An iterative multi-stage methodology was utilised to develop and test the proposed theoretical model.

*Case Study: Methodology

Stage one

• Literature review – Theoretical framework

Stage two

• Identification of suitable source of data

Stage

three

• Data review (data inventory)• Measurement development and testing• Data cleaning

* Case study

*European Working Conditions Survey

*Pan-European cross sectional survey of working conditions, worker’s health and safety, and living conditions (n = over 40, 000 workers)

*Now on the 6th wave of data collection.

*The survey as evolved over time asking more questions.

*Survey items are informed and based on contemporary theory

*The measures used are not always based on a validated psychometric measures

*Single items vs. composite measures?

*Case study: Single item

measures

*The vast majority of latent conceptual constructs are complex and multifaceted in nature.

*Consequently, the use of a single item as a theoretical concept may not yield an accurate, comprehensive, and reliable measurement of the given construct of interest.

*Case study: Measurement

error

*The guiding premise by many in the scientific community is that multiple responses reflect the “true” response more accurately than does a single response.

*Imprecision in measurement is one of the key causes (although not the sole cause) of measurement error.

*Measurement error creates ‘noise’ to the observed variables.

*Case study: Implications of poor measurement

*Inaccurate and unreliable measurement of a concept results in key concerns regarding the overall validity and reliability of the hypotheses tested using this (or these) given measurement(s).

* It is generally agreed/ suggested that research findings that are valid, reliable and generalizable, are built on a solid foundation of accurate and consistent measurement.

*Composite measures

*The primary objective of creating a series of summated (or composite) scales is to avoid the exclusive use of, or dependence on, single item constructs where possible.

*The use of several variables as indicators provides an opportunity to represent differing facets of a given concept, with the aim of yielding a more well-rounded perspective and, arguably, a better measurement of the given concept

*Thinking about ethics

*A note about ethics

*Researchers need to ask: how was consent obtained in the original study? Where sensitive data is involved, we cannot/ should not assume informed consent.

*Given that it is usually not feasible to seek additional consent, a professional judgement may have to be made about whether the use of secondary data violates the contract made between subjects and the primary researchers.

*Growing interest in secondary data make it imperative that researchers in general now consider obtaining consent, which covers the possibility of secondary analysis as well as the research in hand.

* This is consistent with professional guidelines on ethical practice

Heaton, J (1998). Secondary analysis of qualitative data. Social Research Update (issue 22). See: http://sru.soc.surrey.ac.uk/SRU22.html

*Some thoughts on writing up

*Can you publish secondary data analysis – yes!

*Never forget: the central role of theory.

*Be detail orientated!

* Justifying your research question is important, but you also need to be prepared to justify and outline the logic of your analysis framework and approach.

*Understand and reflect on how the research design or any experienced methodological issues of your secondary data may impact or frame the interpretation of your results.

*Conclusion

*Secondary data analysis is an important and useful research methodology.

*There are many benefits and strengths to using secondary sources of data.

*But there also important pragmatic and methodological challenges that face researchers.

*Suggested reading

* Trzesniewski, K. H., Donnellan, M., & Lucas, R. E. (2011). Secondary data analysis: An introduction for psychologists. American Psychological Association.

* Vartanian, T. P. (2010). Secondary data analysis. Oxford University Press.

* Heaton, J. (2008). Secondary analysis of qualitative data: An overview. Historical Social Research/Historische Sozialforschung, 33-45.

* Hinds, P. S., Vogel, R. J., & Clarke-Steffen, L. (1997). The possibilities and pitfalls of doing a secondary analysis of a qualitative data set. Qualitative Health Research, 7(3), 408-424.

j.hassard@bbk.ac.uk

*Thank you for listening