1

Class 1

Concept Development and Concept Definitions

September 20, 2007

Anita L. Stewart, Ph.D.Institute for Health & Aging

University of California, San Francisco

2

Overview of Entire Class (EPI 225) How to identify, critique and select good

measures for your research Homework – weekly Two discussion sessions (optional)

– 2:15-3:00 and 4:30-5:00 Focus on one concept of interest to you

– Health dimension or a determinant of health– Measurable (need to find 2-3 measures to review)

3

Inappropriate Measures can Result in:

Conceptual inadequacy Poor data quality (e.g. missing data) Poor variability Poor reliability and validity Inability to detect associations

– Correlations, mean differences

4

Overview of Class 1

Concept development for new measures Complexity of most concepts Importance of defining concepts prior to

selecting existing measures Goal

– Provide ideas for developing your own concept before considering measures

5

Typical Sequence of Developing New Self-Report Measures

Develop concept

Create item pool

Pretest/revise

Field survey

Psychometric analyses

Final measures

6

Typical Sequence of Developing New Self-Report Measures

Develop concept

Create item pool

Pretest/revise

Field survey

Psychometric analyses

Final measures

7

Concept/Construct

A variable that is relatively abstract as opposed to concrete– e.g. health status, stress, decisional conflict,

acculturation Cannot be assessed directly An abstraction based on observations of

certain behaviors or characteristics

8

Concept, Construct, Variable, Latent Trait, Latent Variable

Terms defined differently by different investigators/authors

Meaning is very similar– considerable overlap among them

– often used interchangeably

9

Latent Trait

Latent - present but not visible, unobservable

Latent trait - unobservable set of characteristics that can be empirically inferred and estimated through answers to a set of questions

10

Measures of Concepts

Concepts and constructs are defined and operationalized in terms of observed indicators

Measures are “proxies” for the latent variables we cannot directly observe

11

Depicting Latent Variables and Measures (Manifest Variables)

Variable A

Measure A

Variable B

Measure B

12

Complexity of Concepts

By definition, due to abstract nature of concepts, most (if not all) are complex

Complexity means– Hard to define

– Usually multidimensional

13

Dimension, Domain, Sub-domain, Component

Dimension: a distinct component of a multidimensional construct that can be theoretically or empirically specified (e.g., physical and mental health)

Terms somewhat interchangeable

14

“Health” as a Concept

What first comes to mind when you think of the word “health”?

Health

15

Some Common Terminology of “Health”

Health indicators Health status Health outcomes Clinical status Functional status Functioning and well-being Quality of life Health-related quality of life

16

Definition Varies by Perspective

Health

Bench scientist

Individual

Populationscience Clinician

17

Perspectives on Health Bench

Scientist Clinician Individual

Molecular, genetic, cellular X ??Anatomic, physiologic signs XSymptoms X XDiagnosed conditions X XSeverity of conditions X XFunctioning in daily life XWell-being X

18

What is Perfect Health?

An ideal toward which people are oriented – not a state they expect to attain.

“A healthy person is someone who has been inadequately studied” (p. 31)

AC Twaddle, The concept of health status, Soc Sci Med 1974;8

19

Health as a continuum?

Perfect health

Less than perfect health

Illness

20

What are Normal Health States?

Perfect health (abnormal)

Less than perfect health (normal)

Illness (abnormal)

21

Contradictory?Or Just Different Perspectives?

Signs of illness (MD)

Perceive oneself to be in “excellent” health (individual)

No signs of illness (MD)

Perception of being in poor health

Cellular abnormalities (bench scientist)

No signs of illness (MD)

22

Concept of Quality of Life

What first comes to mind when you think of the word “quality of life”?

Quality of Life

23

Quality of Life Terminology

Satisfaction with life “as a whole” Well-being Global well-being Subjective well-being Sense of well-being Global happiness

24

Dimensions or Domains of Quality of Life

Quality of …. Marriage Family life Health Neighborhood Friendships Job

City Housing Standard of living National government Local government

…. and many more

25

Health-Related Quality of Life

Domains of life quality relevant to health, disease, and medical care

Well-being– Physical (malaise, symptoms)– Mental (psychological well-being)

Functioning– Limited in activities because of health problems

or treatments

26

MOS Framework: Functioning and Well-Being From the Patient’s Perspective

Physical functioning

Pain Energy/fatigue Sleep problems Cognitive functioning Psychological distress/well-being Social activity limitations due to health Role limitations due to physical health Role limitations due to emotional problems Health perceptions

Stewart AL, reading for week 1.

27

MOS Framework: Functioning and Well-Being From the Patient’s Perspective

Physical functioning

Pain Energy/fatigue Sleep problems Cognitive functioning Psychological distress/well-being Social activity limitations due to health Role limitations due to physical health Role limitations due to emotional problems Health perceptions

Stewart AL, reading for week 1.

Health-related quality of life?

28

MOS Framework: Domains are Indicators of Physical and Mental Health

Physical Mental

Indicators Health Health Physical functioning X Pain X Energy/fatigue X X Sleep problems X X Cognitive functioning X Psychological distress/well-being X Social activity limitations due to health X X Role limitations due to physical health X Role limitations due to emotional problems X Current health perceptions X X

29

MOS Framework: Physical and Mental Health are “Latent Variables”

Physicalhealth

Mentalhealth

30

Physical Health

MOS Physical Health: Dimensionality

Physicalfunctioning

Physicalfunctioning

Health perceptions

Health perceptions

PainPainEnergy &

fatigueEnergy &

fatigue

Role limitationsdue to physical

health

Role limitationsdue to physical

health

31

Methods For Developing Concepts

Qualitative (formative research) Mixed methods (iterative process of

qualitative and quantitative methods to develop and test measures)

32

What are Qualitative Methods?

Data consist of words (text), not numbers Questions are open-ended Focus on inductive analytic approaches Many types: ethnography, participant-

observation, direct observation, focus groups, in-depth interviews

33

Focus Groups

Open-ended guided group discussion with probing of responses

Listen to perspective of likely “subjects”– How they discuss “concept” in their own

words Participants stimulate comments of others

34

Focus Group Methods

Led by experienced moderator– usually 1 - 2 hours

Purposeful sampling of 6-10 homogenous participants per group

Audio-record and transcribe discussion

35

Example: Developing Concept of Expectations Regarding Aging (ERA)

Construct thought to reflect how older adults think about aging– Do they expect to decline on various

dimensions?

– Would their expectations affect decisions about health and health care?

36

Concept Development ERA : Qualitative Methods

Conducted 5 focus groups of older adults 10 vignettes describing older adults

experiencing common age-associated changes. Probed “does this describe changes you would expect to occur with aging?”

Analyzed transcripts using qualitative methods Results: 26 “domains”

Sarkisian CA et al. Med Care. 2003;39:1025-36.

37

Examples of “Expectations About Aging” Domains

Physical function Cognitive function Social function Pain Sexual function Personal autonomy Loneliness

Appearance Energy Urinary incontinence Falling Happiness Spirituality Being treated differently

38

Mixed Methods

Iterative process integrating qualitative and quantitative methods

39

Combining Qualitative and Quantitative Methods in Concept Development

Prior to quantitative: to develop concepts, framework, hypotheses, and content for structured survey items or interventions

After quantitative: – to help identify reasons for survey items not

performing well quantitatively– to explore possible explanations for

unexpected results

40

Mixed Methods: Iteratively Refine Concept Based on Quantitative Results

Develop concept

Create item pool

Pretest/revise

Field survey

Psychometric analyses

Final measures

41

Concept Development of Interpersonal Processes of Care: Mixed Methods

Interpersonal processes between physicians and patients, emphasis on vulnerable patients

Basis for concept:– Review literature, satisfaction with care measures – Clinical experience of investigators

Initial conceptual framework included three broad domains: communication, decision making, and interpersonal style, each with subdomains

Stewart et al., 1999 in readings

42

Concept Development of Interpersonal Processes of Care: Mixed Methods

Second phase:– Items written to reflect all domains

– Administered to diverse patient sample

– Empirical scaling analyses identified 13 scales

Stewart et al., 1999 in readings

43

“Empirically Confirmed” Conceptual Framework of IPC

I. COMMUNICATION III. INTERPERSONAL STYLE General clarity Respectfulness Elicitation/responsiveness Courteousness Explanations of Perceived discrimination -processes Emotional support -self care -medications Empowerment II. DECISION MAKING Responsiveness to patient preferences Consideration of ability to comply

44

Pathway to First IPC “Empirically Confirmed” Conceptual Framework

Develop concept

Create item pool

Pretest/revise

Field survey

Psychometric analyses

Final measures ??

Stewart et al., MilbankQuart, 1999

45

Second Round of IPC Studies

Dissatisfied with resulting framework– Not able to measure several concepts

important to patients Obtained funding for another round of

measurement studies

46

Mixed Methods: Revisit Original Concept

Develop concept

Create item pool

Pretest/revise

Field survey

Psychometric analyses

Final measures

47

Developing Revised IPC Concept

Draft IPC II

conceptualframework

IPC Version I frameworkin Milbank Quarterly

19 new focus groups -African American, Latino,and White adults

Updated literature review of quality of care in diverse groups

48

IPC: Second Conceptual Framework

Wrote additional items Pretested, reduced item set Administered to large sample of diverse

patients Analyzed data for best measures

http://medicine.ucsf.edu/cadc/cores/measurement/ipc/IPCframework_4.13.06.pdf

49

Pathway to Current IPC Conceptual Framework

Develop concept

Create item pool

Pretest/revise

Field survey

Psychometric analyses

Final measures

Stewart et al., Health Serv Res, 2007

Stewart et al., MilbankQuart, 1999

50

Concept Development: Cultural Sensitivity of Physicians

For minority patients and those with limited English proficiency– Sensitivity of providers to their cultural

perspective hypothesized to be an important aspect of interpersonal processes of care

Hypothesized scale did not work in either of our IPC studies

51

New Qualitative Analyses

19 focus groups of ethnically diverse adults– Groups homogeneous on ethnicity, language

(English/Spanish), gender, and age Two open-ended questions:

– What does the word culture mean to you?– What do your doctors understand about your culture

that might affect your visits? Transcripts analyzed using qualitative methods

Nápoles-Springer AM et al. Health Expectations. 2005;8:4-17.

52

Results: Multiple Dimensions of Cultural Factors Affecting the Medical Encounter

Examples … Values, beliefs, and

attitudes Communication

including sensitivity to language access

Complementary and alternative medicine

Privacy and modesty Ethnic concordance

of physician and patient

Empowerment and submissiveness

53

Overview of Class 1

Concept development for new measures Complexity of most concepts Importance of defining concepts prior

to selecting existing measures Goal

– Provide ideas for developing your own concept before considering measures

54

Basic Steps in Selecting Appropriate Measures for Your Studies

1. Specify role of concept in research question2. Define concept for your study 3. Specify context 4. Identify and review potential measures for

a) conceptual and psychometric adequacyb) practicality and acceptability

5. Select best candidates6. Pretest selected measures in your groups7. Choose best ones based on pretest results OR8. Adapt if necessary to address problems

Thisweek

55

Role of Concept in Your Research

Evaluate intervention (outcome) Predict an outcome Describe population Predict future event (e.g. health, use of

emergency care) Covariate

56

Define Concept for Your Research

A methodological task before identifying appropriate measures

Need to clarify concept definition prior to selecting any measures – Define the concept YOU are interested in

measuring

57

Importance of Defining Concept

Distinction between “label” or name of a measure and what it actually contains

May find “good” measure of your concept – But mismatched

» Measures something different than you need» Measures only part of what you need (missing

elements)

58

Define Concept

Define it first from your point of view – How you would define it based on your experience

and understanding of your research question For dependent variables

– Describe how intervention or independent variables might affect it - types of changes you expect

For independent (predictor) variables– Describe how it might be predictive

59

Example of Complexity of Concept

Shared decision making

60

Shared Decision Making: Some Brief “Definitions”

Patient and physician share in the process of deciding on treatment

Patient takes active role in deciding Physician encourages patient to take active role Physician is responsive to patient preferences

regarding participating in decision

61

Varying Definitions Suggest Complex Concept

Process of shared decision making

MD propensityto encourage shared DM(participatory DM style)

Patient desire for shared DM

(active patient orientation)

62

Varying Definitions Suggest Complex Concept

Process of shared decision making

MD propensityto encourage shared DM(participatory DM style)

Patient desire for shared DM

(active patient orientation)

63

Definitions of “Participatory Decision-Making Style”

“Propensity of physicians to involve patients in treatment decisions by providing treatment options, a sense of control over treatment decisions, and a sense of responsibility for care.”– Kaplan SH et al., Med Care, 1995;33:1176

“Propensity (of physicians) to offer patients choices among treatment options and to give them a sense of control and responsibility for care.”– Kaplan SH et al., Ann Int Med, 1996;124:497

64

Varying Definitions Suggest Complex Concept

Process of shared decision making

MD propensityto encourage shared DM(participatory DM style)

Patient desire for shared DM

(active patient orientation)

65

Process of Shared Decision Making in ICU: Several Dimensions

Information exchange: – MD provides medical information relevant to decision– Surrogate provides relevant information re patient’s

values/preferences Deliberation

– MD and surrogate discuss which treatment option is most appropriate given patient’s values and condition

Treatment decision– MD and surrogate make clear what they think is correct

treatment choice

Douglas White, MD, Dept of Med, Div of Pulmonary and Critical Care

66

Homework: Define Concept

Summarize briefly your research question Choose one concept and clarify its role (e.g.

dependent, independent variable) Define concept from your point of view

– Is it multidimensional? What are the dimensions?

Summarize 1-2 “state-of-the-art” definitions– How similar is your own definition to these?