FHOP MCAH Orientation 9-07EpiINFO Developing an Effective Planning Process: A Guide for Local MCAH Programs zNew edition published in 2003 zReviews the traditional health planning

FHOP Orientation for MCAH Program Consultants

AGENDA

9:45 Sign in 10: 00 Introductions and Logistics 10:15 Who is FHOP and what do we do? - Gerry Oliva 10:45 Navigating the FHOP website

Public Health Data CA MCAH Resources Planning Tools- Planning Guide Software – EpiBC and Epi HOSP Training

11:30 Practice time: Can you find it? 12:00 Lunch 1:00 Orientation to the Databooks 1:15 Understanding and Using the Indicator Rate Tables - Gerry Oliva 1:45 Practice looking at rate tables for counties and finding data templates 2:15 How Data Quality Issues Impact the Validity of County Level Indicator

Rates - Linda Remy 2:45 Break 3:00 Do we have a Trend? - Linda Remy 3:35 Practice looking at trend graphs 3:50 Where we go from here? 4:00 Evaluations

The Family Health The Family Health Outcomes Project: Outcomes Project:

Orientation Orientation

Gerry Oliva M.D., MPHFamily Health Outcomes Project

September 25, 2007

FHOP MissionFHOP Mission

To improve the health of children and their families and communities by supporting the development and implementation of comprehensive community planning, data-driven policies, evidence-based interventions, and effective evaluation strategies.

FHOP BackgroundFHOP Background

Located at the University of California, San Francisco, Department of Family and Community MedicineAffiliated with the Institute for Health Policy StudiesMCAH project funded by State, Federal, and Foundation grants

FHOP ApproachFHOP Approach

CollaborationLiterature ReviewConsulting ExpertsCommunity Advisory Groups

FHOP Areas of ActivityFHOP Areas of Activity

Web siteAutomated Resources and ToolsTrainingResearch and DevelopmentAnalytic MethodsEvaluationTechnical Assistance

FHOP Activities:FHOP Activities:Improving Data AccessibilityImproving Data Accessibility

Work with state to develop standardized reportsDisseminate data reports (e.g. annual birth and death data)Disseminate electronic data (e.g. hospital discharge file)Data resources library on FHOP website

Data Sources on the WebData Sources on the Web

FHOP Website: www.ucsf.edu/fhop

• Data tables and spreadsheets

• Links to other sites with relevant data CDPH Center for Health Statistics

CDPH Communicable Disease Branch

UCLA/California Health Interview Survey

CDPH EPIC Center- Injury Data

CADSS Foster Care Data

• Data and links organized by topic area

Areas of Activity:Areas of Activity:Automated Resources Automated Resources

EXCEL Templates – Calculate Rates and confidence intervals – Calculate Risk statistics

Analysis – EpiInfo Based software – EpiBC– EpiHOSP

#DIV/0!#DIV/0!#DIV/0!#DIV/0!#DIV/0!#DIV/0!011.0%169,2742003-2005




Upper 95% C.L.

Lower 95% C.L.Ratio

Upper 95% C.L.

Lower 95% C.L.PercentEventsPercentEvents3 Year Aggregates

County/State ComparisonCountyCalifornia

#DIV/0!#DIV/0!#DIV/0!#DIV/0!#DIV/0!#DIV/0!11.2%59,2252005












Upper 95% C.L.

Lower 95% C.L.Ratio

Upper 95% C.L.

Lower 95% C.L.PercentEventsPercentEventsYear


FHOP Data Templates

FHOP Local FHOP Local DatabooksDatabooksEXCEL Spreadsheets that display County-level data over 12 years for different indicators

Display the comparison of rates between county data, the State and Healthy People 2010 Objectives

Contain a data quality tab to alert counties to missing or unlikely values and how they may affect accuracy of rate calculations

Perform trend tests- “are things getting better or worse?” “how does progress in my county compare to the state?”

FHOP Automated Resources:FHOP Automated Resources:EpiBCEpiBC

Based on public domain EpiInfo 2005Easy-to-use windows versionAnalyzes local birth certificate data and generates reportsGenerates graphs and mapsBuilt-in tutorials, references, documentationUpdated version for 2005

FHOP Activities:FHOP Activities:Research and DevelopmentResearch and Development

Develop public health indicators

Develop methods and guidelines and analytic tools

Perform literature reviews for risk and protective factors and evidence-based practices

Study trends in hospitalizations for children and youth for specific diagnoses

Perform research and evaluation of innovative intervention strategies for high risk women

PerinatalPerinatal and Child Health and Child Health SurveySurvey

Developed in conjunction with the MCAH Action Rural Caucus committeeContains a core survey along with 5 optional modulesStand-alone Adolescent health surveyData entry and analysis template in EpiINFO

Developing an Effective Developing an Effective Planning Process: A Guide for Planning Process: A Guide for

Local MCAH ProgramsLocal MCAH ProgramsNew edition published in 2003Reviews the traditional health planning cycle with a focus on MCAH activitiesProvides tools to facilitate planning during each phase of the processProvides tools to simplify data analysis

Analytic Statistical Guidelines Analytic Statistical Guidelines 20022002

Data StandardsData Standards

A 2003 set of guidelines for the collection, coding and reporting of race and ethnicity on all data sets maintained by the CADHSGoal is to be able to compare data by uniformly defined categories that are consistent with Federal Office of Management and Budget census categories so that standard rates can be calculated

Race/Ethnicity GuidelinesRace/Ethnicity Guidelines

A set of guidelines for the collection, coding and reporting of race and ethnicity on all data sets maintained by the CADHSDeveloped through a cooperative process with the Center for Health Statistics and the Prevention DivisionNew revision for 2002

FHOP Areas of Activity: FHOP Areas of Activity: TrainingTraining

Topics and schedule developed in collaboration with state MCAH staff and consultation with CCLDMCAHFormat focuses on interactive skill developmentAim to target trainings toward current MCAH needs

FHOP Areas of Activity:FHOP Areas of Activity:Technical AssistanceTechnical Assistance

FHOP staff available for assistance by phone 5 days a week (415-476-5283)– Data related questions– Assistance in use of automated toolsGuidance in planning process

FHOP StaffFHOP StaffGeraldine Oliva MD, MPH, DirectorLinda Remy, PhD, Assoc. Director, ResearchJudith Belfiori, MA, MPH, Planning and Evaluation Jennifer Rienks, PhD, Research Associate Gosia Pellarin, Administrator and Web masterJaime Sanchez, Administrative Assistant

Looking at Indicator Rates: Is There a Difference?

Gerry Oliva M.D., MPH

September 25th, 2007

FHOP Orientation for MCAH Program Consultants, Sacramento, CA

Family Health Outcomes Project, UCSF

Member of the Board of Supervisors Listening to County Presentation on Indicator Rates

How Do We Use Rates in the Analysis of a Public Health Problem

• To determine a benchmark or starting point before addressing an area of concern

• To see how we’re doing compared to a national, state or regional rate

• To compare ourselves with a standard such as the HP 2010

• To determine which population(s) are at highest risk

How Do We Use Rates in the Analysis of a Public Health Problem

• To determine which risk or causal factor(s) are most strongly associated with a condition or outcome of interest

• To determine which factor(s) contributes most to that outcome or condition

Purpose of this Presentation

• To review the basics of risk analysis– confidence intervals– risk analysis– the use of relative risk/ risk ratios in determining the

strength of the association with the identified problem

– the use of attributable risk and population attributable risk in quantifying the impact of an outcome or condition

• Demonstrate how the FHOP webstie can assist in this process– databooks– small numbers guidelines– data templates

Comparing RisksMost public health reports compare Most public health reports compare percents or rates of occurrence of percents or rates of occurrence of conditions, problems or outcomes among a conditions, problems or outcomes among a number of groupsnumber of groups

These differences are used to identify and These differences are used to identify and quantify disparities in health by age, quantify disparities in health by age, geography, race/ethnicity or social classgeography, race/ethnicity or social class

In order to be meaningful, statistical tests In order to be meaningful, statistical tests need to be applied to determine the need to be applied to determine the likelihood that observed differences are not likelihood that observed differences are not due to chancedue to chance

Normal Distribution –Can the Result Be Due to Chance?

95% CI 95% CI

Using Confidence Intervals (CI)

• A confidence interval is the range of values within which the “true” value is likely to fall

• Ninety-five percent is the most commonly used CI

• A 95% CI indicates that there is a 95% chance that the “true” value of the estimate (rate) is included in the interval

• All risk calculations should be accompanied by a CI

Effects of Sample Size on the Confidence Interval

• The smaller the sample, the larger the confidence interval and the harder it is to establish if observed differences are statistically significance

• The larger the sample, the smaller the confidence interval

95% Confidence Interval

Confidence Interval Width & Number of Events

4.00%

5.00%

6.00%

7.00%

8.00%6,

000

LBW

and

100,

000

Tota

lBi

rths

600

LBW

and

10,0

00 T

otal

Birth

s

60 L

BW a

nd1,

000

Tota

lBi

rths

Low

Birt

hwei

ght P

erce

nt

Upper 95% CI

LBW Percent

Lower 95% CI

The percent of live births less than 37 weeks gestation

9.7 8.9 9.3 1,919 11.3 11.2 11.2 59,225 2005

9.6 8.8 9.2 1,880 11.1 10.9 11.0 55,738 2004

10.2 9.4 9.7 2,050 10.9 10.7 10.8 54,311 2003

10.5 9.7 10.1 2,110 10.6 10.4 10.5 52,067 2002

10.6 9.8 10.2 2,169 10.5 10.4 10.4 51,974 2001

10.2 9.4 9.8 2,096 10.5 10.4 10.5 52,522 2000

10.7 9.9 10.3 2,052 10.7 10.5 10.6 51,807 1999

10.6 9.7 10.1 2,035 10.7 10.5 10.6 52,441 1998

10.6 9.7 10.1 2,034 10.5 10.3 10.4 51,875 1997

10.3 9.4 9.8 1,965 10.4 10.2 10.3 53,022 1996

10.7 9.9 10.3 2,095 10.4 10.2 10.3 54,453 1995

10.7 9.9 10.3 2,142 10.4 10.2 10.3 56,064 1994

Upper C.L. Lower C.L. Rate Births Upper C.L. Lower C.L. Rate Births Year

LocalCalifornia





Upper 95% C.L.

Lower 95% C.L.Ratio

Upper 95% C.L.

Lower 95% C.L.PercentEventsPercentEvents3 Year Aggregates














Upper 95% C.L.

Lower 95% C.L.Ratio

Upper 95% C.L.

Lower 95% C.L.PercentEventsPercentEventsYear


FHOP Data Templates

Risk Difference (RD)

• Definition: the differences in rates of outcomes for subgroups of the population ( e.g. race or income groups)

• Calculation: Rate in one group minus the rate in the comparison group

Risk Difference Calculation

If the infant mortality rate is 14 per 1000 in African- American infants and 7 per 1000 in White infants the risk difference is:

14 per 1000 minus 7 per 1000 = 7 per thousand

This means that African American infants have a 7 per thousand more or twice the risk of dying than white infants

Limitations of Rate Comparisons

• The numbers may be too small to ever get a stable rate with which to compare differences using CIs

• A rate difference may be significant but the actual increase in risk may be minimal

• A rate difference may be significant but the contribution of a small group to the total number of cases in terms of actual numbers may be minimal in a jurisdiction

Risky Analysis

What are the chances of this man getting hit compared to another who is not a data analyst?

What Do We Mean by Risk?

The probability of a disease-free individual developing a given disease over a specified period

or in public health

The probability of occurrence of a negative outcome in a population in a specified time

and

The probability of occurrence of this outcome in one group compared to another group

and

The proportion or number of adverse events that can be attributed to each group of affected individuals

Organizing Data for Risk Analysis : The 2x2 Table

a b

c d

Yes

Group A

a + c b + d

a+b (n1)

c+d (n2)

Group B

NoPoor Outcome

Calculating Risk

• Risk in exposed = a / a+b

• Risk in non exposed = c / c+d

• Risk can vary from zero to one (or in the case of a protective factor from minus 1 to one)

Organizing Data for Risk Analysis - Example

Infant Death Yes No Births(n)

Group A 240 (a) 19,760 (b) 20,000 (a+b)

Group B 160 (c) 19,840 (d) 20,000 (c+d)

Total 400 39,600 40,000(a+c) (b+d)

Risk Analysis Questions

• What is the risk of infant death in each group?

• How does the risk in Group A compare to the risk in Group B?

Calculating Risk - Example

Group A 240 ÷ 20,000 = .012

Group B 160 ÷ 20,000 = .008

÷ =Infant Deaths in group

Total Births in group

Risk of Death in group

Reporting Risk Results

• These results can be expressed as rates:

– the risk of infant death for Group A infants is 12 deaths per 1000 births

– for Group B infants, the risk of infant death is 8 deaths per 1000 births

• However, in a risk analysis, risks are usually reported using measures that compare the risks among two or more groups.

Relative Risk/Risk Ratio (RR)

• Definition: the relative risk is the calculated ratio of the incidence rate of an outcome in two groups of people, one with the risk factor of interest and the other without that risk factor

• Calculation: a / a+b (n1) divided by c/ c+d (n2)

• Measurement: the RR can vary from zero to infinity

Relative Risk Questions

• How do the infant death rates of Group A babies compare to those in Group B?

OR• How much more likely is it that

Group A babies will die compared to babies in Group B?

Relative Risk (RR) for Infant Death in Group A Infants

Risk RRGP A .012 (a/a+b)

= 1.5 GP B .008 (c/c+d)

The risk of infant death in Group A babies is 1.5 times than that in Group B babies

Just how many cases does this increased risk account for?

• Definition: The portion of the incidence of an outcome among people exposed to a risk factor that can be attributed to the exposure to the risk factor

• Calculation: The incidence rate among exposed minus (-) the incidence rate among non-exposed

Attributable Risk

The Attributable Risk is a Measure of Excess Risk

Attributable Risk Calculation (AR)

Risk for GP A babies = a/a+b = .012

Risk for GP B = c/c+d = .008

AR = .012 - .008 = .004

Group A accounts for an extra 4 deaths per 1000 births. A program to target this group could reduce the infant mortality rate by up to 4 per 1000

Attributable Risk Percent (AR%)

• Definition - The percent of deaths (cases) among Group A members that can be attributed to being part of Group A (or other risk exposure)

• Calculation:

(AR GP A –AR not GP A) / AR GP A x 100

= (.012 - .008) / .012

= .333 x 100

= 33.3%

33% of the deaths in the counties infants can be attributed to being part of the GP A population

Population Attributable Risk: Definition

Population attributable risk (PAR) is the proportion of cases with a negative outcome that can be attributed to belonging to a particular risk group or having exposure to a negative condition

or conversely

the reduction in the proportion of cases that could be achieved if the outcomes in the group of interest are improved

Population Attributable Risk (PAR) Calculation

PAR = p0 ( the proportion of the total population with the outcome of interest minus p2 ( the proportion of those in Group B [or those without exposure] with that outcome)

= p0 - p2 o = (p1-p2) x n1/N = AR x n1/N= .004 x 20,000/40,000= .004 x .5= .002

An excess of 2 deaths per thousand infants can be attributed to the Group A population

Population Attributable Risk Percent (PAR%)

Definition: % of cases with a negative outcome in the total population that can be attributed to a group or a factor of interest

Calculation:

PAR% = PAR divided by the overall risk times 100

= .002 / .010 x 100

= 20%

Being part of Group A is associated with 20% of infant deaths in total population

ATTRIBUTABLE RISK CALCULATIONS Population: Describe the Study Population. Exposure: Describe the Exposure. Note that membership in a group (race, ethnicity, age) can be an exposure. Outcome: Describe the disease. Note that "disease" refers to the outcome associated with the exposure. ENTER THE FOLLOWING NUMBERS: <--- The total population, including the exposed and the non-exposed. <--- The number exposed. <--- The total number (exposed and non-exposed) with the disease.

<--- The number of the exposed with the disease.

Disease

Yes No Total

Yes 0 0 0 Exposure No 0 0 0

Total 0 0 0 #DIV/0! Proportion of the total population with the exposure. #DIV/0! Risk of disease for the total population. #DIV/0! Risk of disease for the exposed population.

#DIV/0! Risk of disease for the non-exposed population.

#DIV/0! Risk Ratio = Relative Risk

The risk for the exposed divided by the risk for the non-exposed. The risk for the exposed is: #DIV/0! times the risk for the non-exposed.

#DIV/0! Attributable Risk Among the Exposed = Risk Difference = Excess Risk The risk for the exposed minus the risk for the non-exposed. The risk for the exposed is: #DIV/0! greater than the risk for the non-exposed.

#DIV/0! Upper limit of 95% confidence interval.

#DIV/0! Lower limit of 95% confidence interval.

#DIV/0! Relative Confidence Interval (at 95% level) #DIV/0! Number of cases attributable to the exposure.

#DIV/0! Upper limit of 95% confidence interval.

#DIV/0! Lower limit of 95% confidence interval.

Take Home

1. Risk differences supplemented by confidence intervals are essential to understanding and quantifying health disparities

2. Relative risk (RR), a measure of the strength of the association or causal link between a risk factor and an outcome is an important adjunct in determining a focus for an intervention

Take Home (cont.)

• We recommend Population Attributable Risk Percent (PAR %) which quantifies the contribution of a population group at risk or a risk factor to the outcome and can thus help direct interventions.

• FHOP tools are available to make these analysis simple !!!!!

So Relax and Have Fun !!!

1

Impact of Data Quality on Birth Indicators 1989-2005: Preterm Delivery

Sacramento CASeptember 25 2007

2

Linda Remy, MSW PhDGerry Oliva, MD MPH

Ted Clay, MS

Family Health Outcomes ProjectDepartment of Family and Community Medicine

University of California, San Francisco

3

California Data Quality

• The National Center for Health Statistics (NCHS) calculates data quality for required birth certificate variables.

• In 2004, California did not meet NCHS standards on mother’s Hispanic origin, education, and month and year of last menstrual period.

• The Center for Health Statistics (CHS) began a targeted training program for birth clerks in hospitals providing poor quality data.

• Significant improvements are reported to have resulted from this approach.

4

FHOP’s Data Quality Reports

• Given the importance of certain variables for calculating health indicators, FHOP has been exploring the impact of improbable data (missing, out of range) on the calculation of health indicators.

• We do routine quality checks for perinatal indicators counties monitor for Title V.

• Among others, these include preterm birth, first trimester entry into prenatal care, adequacy of prenatal care (APNCU), low and very low birthweight, each by mother’s ethnicity.

5

Utility of Data Quality Reports

• Data quality reports help counties: – assess the potential impact of data errors on

the accuracy of indicators – give health department staff information to

work with providers and hospitals to improve data quality.

• For rural counties, a few missing or unlikely values can result in misleading conclusions about the quality and adequacy of prenatal care or the effectiveness of outreach.

• Using the statewide average to gauge problems may not be helpful for a state as large and diverse as California.

6

Purpose

• Describe birth certificate data quality variation in between 1989 and 2005 for gestational age (GA).

• Examine preterm birth rate differences using GA as reported and as cleaned.

• Show how data quality impacts assessment of race/ethnic disparities.

7

Improbable ValuesGestational Age (GA)

Improbable Gestational Age (Weeks)Year Missing Lt 18 Gt 47 Total Pct

1989 17,484 306 6,397 24,187 4.21990 17,009 273 5,718 23,000 3.81991 16,768 292 5,387 22,447 3.71992 17,018 150 4,950 22,118 3.7

… … … … … …2002 30,124 286 3,378 33,788 6.42003 34,093 204 2,853 37,150 6.92004 33,482 262 2,756 36,500 6.72005 18,537 324 3,408 22,269 4.1

8

Between 1989 and 2005

• Improbable GA (missing, less than 18 weeks, more than 47 weeks) ranged from a low of 3.7% (1992) to 6.9% (2003).

• Of records with improbable GA, 72% were due to missing data in 1992, compared with 92% in 2004.

• In 2005, after CHS training began for selected hospitals, the statewide number of records with improbable GA dropped 64% compared with 2004, and number of cases fully missing gestational age dropped 80%.

9

Change in Improbable GA

-

2

4

6

8

1992 2003 2005

10

• The 1992 county range for improbable GA was 0.0%-13%, median 3.2%. The 2003 range was 0.0%-17.7%, median 5.3%.

• In 1992, most counties with data quality problems were rural. In 2003, more counties had data quality problems and most were larger.

• In 1992, only 5 counties with 5,000 or more births had more than 5% of records with improbable data. Of 20 counties with 5,000 or more births in 2003, 13 had improbable GA above 5%.

Improbable GA distributed unevenly

11

• To understand the impact of data quality on preterm birthrates reported in FHOP databooks, we imputed GA when it was missing or out of range by using a section of Kotelchuck’s algorithm to calculate adequacy of prenatal care.

• This allows us to estimate “true” preterm birthrates if data were of higher quality.

• The Federal government uses a similar algorithm to calculate state-level statistics for their reports.

Imputing Gestational Age

12

California Preterm Birth Rates Reported and Imputed 1989-2005

-

3

6

9

12

15

18

21

24

27

1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Observed Imputed HP2010

13

Local Preterm Birth Rates Reported and Imputed 1989-2005

-

3

6

9

12

15

18

21

24

27

1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005


14

Local Asian Preterm Birth Rates Reported and Imputed 1989-2005

-

3

6

9

12

15

18

21

24

27

1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005


15

Local Black Preterm Birth Rates Reported and Imputed 1989-2005

-

3

6

9

12

15

18

21

24

27

1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005


16

Local Hispanic Preterm Birth Rates Reported and Imputed 1989-2005

-

3

6

9

12

15

18

21

24

27

1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005


17

Local White Preterm Birth Rates Reported and Imputed 1989-2005

-

3

6

9

12

15

18

21

24

27

1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005


18

Implications

• Local jurisdictions must carefully review their data quality reports for a given indicator, both to understand the impact of quality data on results in a given year or trends over time.

• FHOP has prepared a new spreadsheet presenting annual preterm birth data quality. It is available on the website.

• Local jurisdictions are advised to consult that spreadsheet before reporting their preterm birth rates.

19

Conclusions

• Data quality issues result in significant under-estimates of California’s true performance with respect to achieving Healthy People 2010 objectives.

• The shift of poor quality data from smaller to more populous counties has a greater impact on the accuracy of state rates.

• Before concluding that population-based rates are changing, it is important to evaluate and understand the impact of data quality.

20

For Further Information:

• Linda Remy, MSW, PhD– Email: [email protected]

• Gerry Oliva, MD, MPH– Email: [email protected]

• Mail: Family Health Outcomes Project– 3333 California Street, #365– San Francisco, CA 94118

• Website: www.ucsf.edu/fhop

21

Improbable ValuesBirthweight

Improbable Birthweight (Grams)Year Births Missing Lt 250 Gt 4999 Total Pct

1992 600,838 104 57 1,325 1,486 0.25 1993 584,483 85 82 1,281 1,448 0.25 1994 567,034 78 64 1,135 1,277 0.23

…2001 527,371 5 70 965 1,040 0.20 2002 529,241 7 77 929 1,013 0.19 2003 540,827 13 82 944 1,039 0.19

22

Between 1992 and 2003:

• The number of births to California residents dropped 11.1% from 600,838 to 540,827.

• Improbable BWT (missing, less than 250 grams and more than 4999 grams) dropped from 1,486 to 1,039 births.

• Most of the decrease was associated with BWT greater than 4999 grams.

• In 1992, improbable BWT represented 0.25% of records. In 2003, this was 0.19%.

23

Improbable birthweightdistributed unevenly

• In 1992, county-level improbable BWT values ranged from 0.0%-2.4%, median 0.27%.

• The 2003 range was 0.0%-0.9%, median 0.2%. The median was little changed, reflecting that jurisdictions tended to improve data quality on this measure with time.

• Several reabstraction studies have found BWT is one of the most reliably coded variables. Unlikely values are not a significant factor in calculating low birthweight rates.

24

Local Asian Preterm Birth Rates Reported and Cleaned 1992 and 2003

9.0

9.5

10.0

10.5

11.0

11.5

12.0

12.5

Reported 1992 Cleaned 1992 Reported 2003 Cleaned 2003

25

African-American Preterm Birth RateState and Local 1992 and 2003

-

3

6

9

12

15

18

21

St-1992 Loc-1992 St-2003 Loc-2003

1

Do We Have a Trend?

Sacramento CASep 25 2007

Linda Remy, MSW, PhDGeraldine Oliva, MD, MPH

2

How is Trend Analysis Different from Comparison of Rates?

• When comparing rates we are looking at two points and their confidence intervals at specific times

• Rates may differ due to random fluctuation rather than a true difference

• Trend analysis uses many data points and controls for random variation using various statistical techniques

3

To Eyeball or Not to Eyeball: That is the Question

YES! NO!

4

Why Trend Analysis?• Allows us to look at rates over the time

period of 1994-2005 and see if upward or downward trends on key indicators are significant (as opposed to the inaccurate eyeballing approach)

• Useful for identifying and monitoring trends in disparities among racial/ethnic groups

• Allows for comparisons with statewide trends on key indicators for different race/ethnic groups

5

Why Trend Analysis?

• Useful for identifying whether a problem affects people across groups or disproportionately impacts subgroups

• Allows local health jurisdictions to track their progress toward reaching Healthy People 2010 Goals

6

What Do Trend Charts and Tables Tell Us?

• If there are significant upward or downward trends in rates over time in the local jurisdiction

• If there are significant upward or downward trends in rates over time at the state level

• If the local trend and the state trend are significantly different

7

What Do Trend Charts and Tables Tell Us?

• If the local or state trend is linear or curvilinear. When state or local trend is curvilinear, we cannot test if the curvilinear trend is significantly different from a linear trend

• Were local average rates at period start - say 1994-1996 --significantly different from average rates at the state level

• Were average rates at period end - say 2003-2005 - at the local level significantly different from average rates at the state level

8

Definition of Terms for Regression Analysis of Trends

• Slope – the average rate of change over time where:

zero = flat

(+) = increasing trend

(-) = downward trend

• Intercept – the rate at the start of the study period - where the line intercepts the vertical (x) axis

9

Slope: Change in Rate per Population

0

20

40

60

80

100

1991 1992 1993 1994 1995 1996 1997 1998 1999 2000

COUNTY: y = 70.3 - 0.75x

STATE: y = 82.8 - 1.5x

10

Change in the Intercept

0

2

4

6

8

1991 1992 1993 1994 1995 1996 1997 1998 1999 2000

COUNTY: y = 7.5 - 0.13x

STATE: y = 5.9 + 0.04x

11

Statistical Terms for Trend Regression Analysis

• P-value - likelihood that model results are statistically significant

• Standard Error - the confidence interval around the slope and intercept

• Both are used for significance of indicator trend and differences among comparison groups

12

Factors Impacting Analysis

• Sample size and the number of time periods being examined

• Presence of outliers• Availability of numerators and

denominators• Confounding - changes over time in

factors related to the indicator of interest

13

Small Numbers and Data Points

• For large sample sizes at the federal and state level simple linear plots describing changes are treated as error free

• For the usual county, city or district, the problem of unstable rates due to small numbers requires tests of statistical significance and other statistical techniques

• For the impact of a particular program or policy on individuals, small numbers problems apply

14

Sample Size Issues

• Minimum number for stable rate may require statistical techniques such as aggregating or averaging years

• Many years of data may be needed to obtain at least 3 usable data points for a trend analysis

• One size does not fit all• Approach differs between one jurisdiction

and another and between one indicator and another. This can make comparisons difficult

15

1. No trend, no difference in rates

• Neither state nor local had a statistically significant trend.

• Local was not different from state.

• At period start and period end, Local rate was not significantly different from State rate.

• Neither State nor local rates met the HP2010 goal

0

5

10

15

20

25

1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005California Local HP 2010 Goal

Trend Regression ResultsIntercept Slope

Level Date Range Est. Std. Err Est. Std. Err P-Value Sig?State 1994-2005 15.72 0.11 0.00 0.02 0.857 NoLocal 1994-2005 13.47 1.79 0.27 0.28 0.370 No Different? 0.343 No State Avg 1994-1996 15.77 0.11 Local Avg vs State 15.34 1.89 0.821 NoState Avg 2003-2005 15.82 0.13 Local Avg vs State 16.48 1.94 0.734 No

16

2. No trend, Local below State at period start and end

• Neither Local nor State had statistically significant trends.

• Local trend not significantly different from State trend.

• At period start and end, Local rate was lower than State rate.

• Neither State nor Local rates met the HP2010 goal.

0

3

6

9

12

15

18

21


Trend Regression Results

Intercept SlopeLevel Date Range Est. Std. Err Est. Std. Err P-Value Sig?State 1994-2005 15.72 0.11 0.00 0.02 0.857 NoLocal 1994-2005 13.54 0.56 -0.15 0.09 0.142 No Different? 0.126 No State Avg 1994-1996 15.77 0.11 Local Avg vs State 13.61 0.87 0.014 YesState Avg 2003-2005 15.82 0.13 Local Avg vs State 11.95 1.00 0.000 Yes

17

3. No trend, Local same as State at start, above at end

• Neither Local nor State had statistically significant trends.

• Local trend not significantly different from State trend.

• At period start Local rate was not significantly different from State rate.

• At period end, Local rate was significantly higher than State rate

• Neither State nor Local rates met the HP2010 goal.

0

3

6

9

12

15

18

21



Level Date Range Est. Std. Err Est. Std. Err P-Value Sig?State 1994-2005 9.94 0.12 0.02 0.02 0.231 NoLocal 1994-2005 10.49 0.37 0.08 0.06 0.185 No Different? 0.337 No State Avg 1994-1996 10.04 0.07 Local Avg vs State 10.56 0.48 0.280 NoState Avg 2003-2005 10.27 0.07 Local Avg vs State 11.27 0.47 0.034 Yes

18

4. Local trend, local trend different, local rates different

• State trend was flat while Local had a statistically significant downward trend.

• Local trend significantly different from State trend.

• At period start and end, Local was significantly higher than State.

• Neither State nor Local rates met the HP2010 goal but Local trend moving toward the goal.

0

5

10

15

20

25

1994 1996 1998 2000 2002 2004California Local HP 2010 Goal


Intercept SlopeLevel Date Range Est. Std. Err Est. Std. Err P-Value Sig?State 1994-2005 9.94 0.12 0.02 0.02 0.231 NoLocal 1994-2005 19.03 0.90 -0.53 0.13 0.003 Yes Different? 0.000 Yes State Avg 1994-1996 10.04 0.07 Local Avg vs State 19.35 1.10 0.000 YesState Avg 2003-2005 10.27 0.07 Local Avg vs State 14.12 1.05 0.000 Yes

19

5. Upward trend, Local lower start and end

• Both Local and State had statistically significant upward trends.

• Trends cannot be compared because State trend was curvilinear.

• At period start and end, Local rate significantly lower than State rate.

• State and local trends are moving away from the HP2010 goal

0

2

4

6

8

10

12

1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

California Local HP 2010 Goal


Level Date Range Est. Std. Err Est. Std. Err P-Value Sig?State 1994-2002 9.00 0.04 0.08 0.01 0.000 Yes 2002-2005 7.39 0.49 0.28 0.05 0.001 YesLocal 1994-2005 8.14 0.08 0.13 0.01 0.000 Yes State Avg 1994-1996 9.06 0.04 Local Avg vs State 8.28 0.08 0.000 YesState Avg 2003-2005 10.18 0.04 Local Avg vs State 9.53 0.09 0.000 Yes

20

6. Upward trend, Local lower start, same at end

• Both Local and State had statistically significant upward trends.


• At period start, Local rate significantly lower than State rate.

• At period end, Local rate was not significantly different from State rate.

• State and local trends moving away from HP2010 Goal.

0

2

4

6

8

10

12



Level Date Range Est. Std. Err Est. Std. Err P-Value Sig?State 1994-2002 9.00 0.04 0.08 0.01 0.000 Yes 2002-2005 7.39 0.49 0.28 0.05 0.001 YesLocal 1994-2005 8.17 0.18 0.19 0.03 0.000 Yes State Avg 1994-1996 9.06 0.04 Local Avg vs State 8.43 0.14 0.000 YesState Avg 2003-2005 10.18 0.04 Local Avg vs State 10.24 0.17 0.760 No

21

7. Upward trend State, Lower trend Local, Local start same, Local different end

• State had statistically significant upward trends.

• Local had statistically significant downward trend.


• At period start, Local rate not significantly different from State rate. At period end, Local rate significantly lower than State rate.

• Neither State nor Local rates meet the HP2010 goal, but Local trend moving toward the goal.

0

2

4

6

8

10

12



Level Date Range Est. Std. Err Est. Std. Err P-Value Sig?State 1994-2002 10.31 0.06 0.03 0.01 0.060 No 2002-2005 8.75 0.64 0.22 0.06 0.010 YesLocal 1994-2005 10.36 0.15 -0.08 0.02 0.006 Yes State Avg 1994-1996 10.30 0.02 Local Avg vs State 10.14 0.12 0.192 NoState Avg 2003-2005 11.00 0.03 Local Avg vs State 9.42 0.12 0.000 Yes

22

8. No trend, Local rate higher than State, then same

• Neither state nor local had a statistically significant trend

• Local trend not different from state.

• At period start, Local rate was significantly higher than State rate.

• At period end, Local rate was not significantly different from State rate.

• Neither State nor local rates meet the HP2010 goal.

0

5

10

15

20

1994 1996 1998 2000 2002 2004California Local HP 2010 Goal


Intercept SlopeLevel Date Range Est. Std. Err Est. Std. Err P-Value Sig?State 1994-2005 15.72 0.11 0.00 0.02 0.857 NoLocal 1994-2005 16.67 0.29 -0.06 0.04 0.201 No Different? 0.226 No State Avg 1994-1996 15.77 0.11 Local Avg vs State 16.66 0.34 0.012 YesState Avg 2003-2005 15.82 0.13 Local Avg vs State 16.30 0.34 0.184 No

23

9. No Trend, Local trend different from State, Local rate higher then same

• Both State and Local trends were statistically non-significant but moved in different directions.

• As a result, Local trend is significantly different from State trend.

• At period start, Local rate was higher than State rate.

• At period end, Local rate was same as State rate.

• Neither Local nor State rates meet HP2010 Goal.

24

Exercise Time!

25


Level Date Range Est. Std. Err Est. Std. Err P-Value Sig?State 1994-2005 15.72 0.11 0.00 0.02 0.857 NoLocal 1994-2005 13.75 0.44 0.09 0.07 0.203 No Different? 0.173 No State Avg 1994-1996 15.77 0.11 Local Avg vs State 13.93 0.42 0.000 YesState Avg 2003-2005 15.82 0.13 Local Avg vs State 15.22 0.44 0.183 No

African-American

02468

1012141618

1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Ca lifornia Loc a l HP 2 0 10 Goa l

26


Level Date Range Est. Std. Err Est. Std. Err P-Value Sig?State 1994-2002 9.00 0.04 0.08 0.01 0.000 Yes 2002-2005 7.39 0.49 0.28 0.05 0.001 YesLocal 1994-2005 8.29 0.21 0.11 0.03 0.006 Yes State Avg 1994-1996 9.06 0.04 Local Avg vs State 8.32 0.16 0.000 YesState Avg 2003-2005 10.18 0.04 Local Avg vs State 9.39 0.18 0.000 Yes

White

0

2

4

6

8

10

12

1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005


27


Level Date Range Est. Std. Err Est. Std. Err P-Value Sig?State 1994-2002 9.00 0.04 0.08 0.01 0.000 Yes 2002-2005 7.39 0.49 0.28 0.05 0.001 YesLocal 1994-2000 8.25 0.09 0.16 0.03 0.006 Yes 2000-2003 9.65 1.39 -0.08 0.19 0.703 No 2003-2005 4.70 1.98 0.47 0.19 0.065 No State Avg 1994-1996 9.06 0.04 Local Avg vs State 8.35 0.13 0.000 YesState Avg 2003-2005 10.18 0.04 Local Avg vs State 9.43 0.14 0.000 Yes

White

0

2

4

6

8

10

12

1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005


28

For Further Information:

• Linda Remy, MSW, PhD– Email: [email protected]

• Gerry Oliva, MD, MPH– Email: [email protected]

• Mail: Family Health Outcomes Project– 3333 California Street, #365– San Francisco, CA 94118

• Website: www.ucsf.edu/fhop

Documents

FHOP MCAH Orientation 9-07EpiINFO Developing an Effective Planning Process: A Guide for Local MCAH Programs zNew edition published in 2003 zReviews the traditional health planning