ESTABLISHING TRUSTWORTHINESS

Qualitative Data Management

We enhance reliability or rigor of our data analysis by:

Comparing our categories to pre-existing frameworks. Having an additional person redo the analysis. Comparing notes from more than one source. Using more than one type of qualitative data in our analysis

(observation, interviews, document analysis). Supplementing the qualitative analysis with information from

another quantitative source (for example, a survey). Keeping a record (audit) of how you established data

categories and identified themes. Establishing a feedback loop so that participants can verify

whether or not the analysis is accurate enough to reflect their views (member checking).

We call this process in qualitative research as “trustworthiness.”

Padgett (1998) defines trustworthiness as:

A trustworthy study is one that is carried out fairly and ethically and whose findings represent as closely as possible the experiences of the respondents (p. 92)

Lack of trustworthiness comes from:a) reactivityb) researcher biasesc) respondent biases/ such as withholding information or the halo effect.

Qualitative Data Analysis

Looks for common themes and patterns.Sample quotations are used.Categories of responses are identified and

the number of responses that fall within these categories are identified.

No statistical analysis is required – but demographic information may be expressed in percentages or placed in tables.

Example of using quotes (from gang study in Sin City)

One fourth of the youths surveyed said that they didn’t like gangs or didn’t want to join a gang. Almost 40% said that being in a gang was bad or stupid. One youth described being in a gang as a “dead end choice.” Nine percent felt that being in a gang was dangerous or scary. One respondent said being in a gang was scary because “they always talk about killing or beating up other people.” Only 6% of the respondents thought being in a gang was fun or was necessary for protection. However, one youth said that being in a gang made him feel “safe and good, cool to be around.”

In the previous quote there were categories:

Didn’t like gangsGangs are bad or stupidBeing in a gang is fun

One approach to writing narratives can also include:

The writer’s own thoughts, values, and beliefs.

An interpretation of the research participant’s behavior or thoughts.

For example, this quotation from Fadiman, A. (1997). The spirit catches you and you fall down. New York:

Farrar, Straus, & Giroux.

While Foua was telling me about the dozens of tasks that constituted her “easy” work in Laos, I was thinking that when she said she was stupid, what she really meant was that none of her former skills were transferable to the U.S. – none, that is, except for being an excellent mother to her nine surviving children. It then occurred to me that this last skill had been officially contradicted by the American government, which had legally declared her a child abuser.

One approach to writing a narrative is “thick description” – creation of a picture of observed events, people

involved, rules associated with certain activities, and social context or environment. Thick description

can also incorporate the researcher’s perspectives.

Clients articulate their belief that the welfare system is not designed to help them succeed or care for their families….Often it feels as if the information they received from workers is blatantly wrong. In one focus group, participants talked assuredly of the misinformation they had received ….As one women said“[The policy] is a lie. This what happens in the welfare system”….Such a lack of trust raises serious questions about whether or not clients will heed front-line staff. From Sandfort, Kalil, & Gottschalk (1999). The mirror has two faces. Journal of Poverty, 3 (3), 71-91.

A narrative or text-based summary should include

Identification of common themes in responses.

Patterns of behaviorCultural or other symbols found in the

setting or described by respondents.Identification/description of cultural normsCommon words or phrases used by many

respondents with sample quotationsMinority responses with sample quotations

According to Jorgensen (1989 as cited in Seidel (1998)

Analysis is a breaking up, separating, or dissassembling of research materials into pieces, parts, elements, or units. With facts broken down into manageable pieces, the researcher sorts and sifts them, searching for types, classes, sequences, processes, patterns, or wholes. The aim of this process is to assemble or reconstruct the data in a meaningful or comprehensible fashion (p. 107).

Steps in Data Analysis

Choose a unit of analysis (word, phrase, sentence, paragraph, entire transcript).

Identify possible categories to classify these data elements.

Classification can take place using one or both of the following methods:- Categories are developed for each of the questions to which respondents gave answers.- Categories are developed for the responses as a whole. These categories are also called themes.

The process for doing this is often called constant comparative method

You make comparisons among data elements.You may need to take three or four attempts

to analyze and interpret the data.Your categories or themes may change as you

examine additional transcripts. You can also use a process called “negative

case analysis.” Are there responses that don’t fit the other categories. How does this change your analysis?

Additional Steps

Make sure that you have been able to classify all your data.

May develop alternative categories and then choose one that best indicate a pattern or theme.

You probably will need to make a count of all the possible data elements in order to ensure that you’ve been able to classify all of them.

Data Management Techniques

Cut and paste transcripts on note cards and then put all elements that fit into one category in a pile. This allows you to reshuffle the cars.

Use magic marker or crayon to color code your categories.

Just use a scissor to cut and paste data elements.“Cut and paste” on the computer – some qualitative

software packages allow you to do these or allow you to search for similar words and phrases.

Develop a numerical code and place these codes within a copy of the transcript.

First stage of Coding is “Open-Coding, identifying what you see in a transcript:

Text Code

I always wanted to be the thinnest, the prettiest. I wanted to look like the girls in the magazines. I’m going to have so many boyfriends, and boys are going to be so in love with me. I won’t have to work and I’ll be taken care of for the rest of my life.

ThinnestPrettiestLook like girls in magazinesBoys will love mePositive body imageProvides economic resourcesThin RationaleThin as a means of securityMedia creates standards

Important to Keep Notes

Rationale for Coding Category (could be on transcript)

Need notes on your reasoning for classification.Keep track of themes in data.Try to tie categories together. Are there cause and

effect relationships in the data? Can keep analytic memos Can keep memos on your own reaction to the

data.[Note – all these elements can be incorporated into

write up]

For example, if we were to use the following interview guide, we would transcribe all responses underneath each question in a WORD

processing program. (Sample interview guide)

1. Can you describe how you first became aware of your deafness?

Respondent #1

Respondent #2

Respondent #3

2. How do you see yourself today, in terms of your deafness?Respondent #1

Respondent #2

Respondent #3

From Janesick, V. (1998). "Stretching" exercises for qualitative researchers. Thousand Oaks, CA: Sage, p. 75.