How Well Do Birth Certificates Describe the Pregnancies They Report? The Washington State Experience with Low-Risk Pregnancies

Maternal and Child Health Journal, Vol. 2, No. 3, 1998

How Well Do Birth Certificates Describe the PregnanciesThey Report? The Washington State Experience withLow-Risk Pregnancies

Sharon A. Dobie,1,2 Laura-Mae Baldwin,1 Roger A. Rosenblatt,1 Meredith A. Fordyce,1C. Holly A. Andrilla,1 and L. Gary Hart1

Objectives: Birth certificates are a major source of population-based data on maternal andperinatal health, but their value depends on the accuracy of the data. This study assessesthe validity of information recorded on the birth certificates for women in Washington Statewho were considered to be low risk at entry into care. Methods: Birth certificates werematched to data abstracted from prenatal and intrapartum clinic and hospital records of asample of 1937 Washington State obstetrical patients who were considered to be low riskat the beginning of their pregnancies. Accuracy of a variety of pregnancy characteristics(e.g., complications, procedures) on the birth certificate was analyzed using percentageagreement and sensitivity with record abstracts as the "gold standard." Next, we weightedthe data from each source to produce estimates of pregnancy characteristics in the population.We compared these estimates from the two data sources to see whether they provide similarpictures of this subpopulation. Results: Missing data for specific items on the birth certificatesranged from 0% to 24%. The birth certificate accurately captured gravidity and parity, butwas less likely to report prenatal and intrapartum complications. The population estimatesof the two data sources were significantly different. Conclusions: Because birth certificatessignificantly underestimated the complications of pregnancies, number of interventions, num-ber of procedures, and prenatal visits, use of these data for health policy development orresource allocation should be tempered with caution.KEY WORDS: Birth certificates; low-risk pregnancies; validity.

Birth certificates provide invaluable informationon maternal and perinatal health. As population-based data, they support health policy research, pro-gram planning, and evaluation. In 1989, the NationalCenter for Health Statistics recommended changesto the birth certificate form with an explicit goal toincrease the reliability and completeness of the re-porting of each birth (1). This is crucial because stud-ies use birth certificate data to describe estimates of

1Department of Family Medicine, University of Washington, Seat-tle, Washington.

2Address correspondence and reprint requests to Sharon Dobie,M.D., University of Washington, Department of Family Medicine,Box 356390, Seattle, Washington 98195-6390; e-mail: [email protected]

pregnancy and medical risk for selected subpopula-tions (2,3). In the current era of evidence-based med-icine and managed care, the use of large, readilyavailable data sets may actually increase as a wayto estimate the incidence and prevalence of medicalconditions and outcomes. In this environment, thebirth certificate is potentially a key data source tomonitor some aspects of maternal and perinatalhealth.

A number of studies used hospital charts andbilling data to examine the validity and reliability ofbirth certificates (4-16). When compared with thesedata sources, birth certificates historically underre-port complications. Variables such as birthweight andgestational age at delivery were more likely to agree

1451092-7875/98/0900-0145$15.00/0 C 1998 Plenum Publishing Corporation

146 Dobie, Baldwin, Rosenblatt, Fordyce, Andrilla, and Hart

with hospital data, but missing data were often aproblem (4-7). Some work suggests that the 1989changes in the birth certificate resulted in continuedcapture of demographic information and some im-provements in reporting prenatal and intrapartumcomplications (7,10,11). Despite this improvement,when compared to hospital chart data, the birth cer-tificate is still shown to underreport pregnancy com-plications, procedures, and congenital anomalies (2,8-14). In addition, the hospital record used in thesestudies is an incomplete record of the pregnancy andmay not accurately capture the true antenatal history.It is often the source for birth certificate completion;high agreement with the reference source would beexpected, but does not necessarily reflect accuracy.These studies may actually underestimate the inaccu-racies of the birth certificates (17). Only one study(18) used prenatal records to evaluate the accuracyof the adequacy, timing, and amount of prenatal careon the birth certificate. It found that birth certificatesoverestimate adequacy by underestimating the pro-portion of women with late entry into care or toofew visits.

This study takes the next step in evaluating valid-ity and reliability of information recorded on birthcertificates by looking not only at the intrapartumhospital record, but also at extensive prenatal re-cords. We compare a number of variables from Wash-ington State birth certificates to data abstracted fromprenatal and intrapartum clinic and hospital recordsof a sample of women who were low risk at entryinto care. Then we measure agreement between thetwo data sources and the proportion of conditionsaccurately reported by the birth certificate. Second,through a comparison of the population estimates ofpregnancy characteristics created by each data set,we explore the implications of using birth certificatedata to provide estimates of the subpopulation stud-ied and its health service utilization.

MATERIALS AND METHODS

The data for this study come from two sources:birth certificates and prenatal and intrapartum medi-cal records.

Washington State Birth Certificates

This study examined Washington State birth cer-tificates from 1989 and 1990. The birth certificate

form in Washington State is consistent with the rec-ommendations of the National Center for Health Sta-tistics. It contains demographic information on thebirth parents, prior obstetrical history, and prenatalcare and birth information. The lower section of thecertificate is a series of checklists for medical riskfactors in the index pregnancy, other risk factors,obstetrical procedures, method of delivery, complica-tions of labor and delivery, conditions of the new-born, and congenital anomalies. In this section, per-sons completing the form check a box if the conditionexisted. If no condition on the list occurred, there isa "none" box to check.

Prior to implementation of the 1989 revisions, afield representative from the State of WashingtonCenter for Health Statistics visited nearly all hospitalsin the state to educate staff about new items andrevisions of the form. New staff are also trained bythe field representative. There is only one field repre-sentative, which limits the availability of training.Staff are encouraged to use the prenatal record todocument last menstrual period and number of pre-natal visits. How often this record is available at thehospital is not known. Staff are given a data diction-ary to facilitate matching terms between the medicalrecord and the birth certificate. At the time of thisstudy, data for the forms were supplied by both moth-ers and medical records, and there was variabilityacross the state both in which data were suppliedfrom either source and in the handling of incompletedata. Completed forms were reviewed by countyhealth departments prior to submission to the state.Again, there was variability among counties in thethreshold for querying incomplete or questionabledata. State data entry staff coded missing items asmissing, not with inferred values. The electronic birthcertificate was implemented after this study.

Chart Abstracted Data

Data were abstracted from the outpatient andhospital records of a sample of Washington Statewomen who were low-risk at the start of their prena-tal care. These data were used to assess the accuracyof the birth certificates. The sampling methodologyfor chart abstraction cases is described in detail inprior work (19). A synopsis of the approach follows.The 2036 obstetricians, family physicians, and certi-fied nurse-midwives in Washington State were di-vided according to specialty and practice location(rural or urban). Because there are very few rural

Accuracy of Birth Certificates 147

certified nurse-midwives in Washington State, thisgroup was not included in the study. The providersin the five strata were assigned and sorted by a com-puter-generated random number. Providers werethen called sequentially by random number to deter-mine eligibility (minimum attendance of 10 birthsin calendar year 1988). Once the target number ofeligible providers was identified, no further providerswere contacted. A network of respected obstetricproviders throughout Washington State invited andencouraged the participation of selected providers,resulting in a 91% participation rate (20, 21).

Data were collected from all of the prenatal andintrapartum office and hospital records of a randomsample of up to 11 obstetric patients who were consid-ered to be low risk at entry into care and who initiatedobstetric care with the study provider during thestudy year, from September 1, 1988, to August 31,1989. If there were fewer than 11 in a practice, allwere sampled. Low risk at entry into care was definedas being between 18 and 34 years of age and having:private insurance or Medicaid coverage, a first prena-tal visit during the first trimester, no more than threeprevious live births, no prior stillbirths, not morethan four previous spontaneous abortions before 14weeks' gestation, no previous births of less than 36weeks' gestation, no prior obstetrical complications(i.e., gestational diabetes, preterm labor) or cesareandeliveries, no history of alcohol or drug abuse, andno history of concurrent major medical conditions(i.e., hypertension). Of the 12,450 patients of theseproviders, 43.3% (or 5385) met these criteria. Thesample of up to 11 women per practice was drawnfrom this group.

Trained medical records technicians abstractedthe complete prenatal and intrapartum records forthis low-risk sample of 2350 women. Prenatal recordabstraction took place in the provider's office. Intra-partum records were abstracted at the hospital. Therewere complete prenatal and intrapartum records for87.4% (2054) of the sampled women. Some of thepatients moved, were lost to follow-up during theindex pregnancy, had spontaneous abortions or in-duced terminations, or the hospital chart for the pa-tient identified in the antenatal sample could not belocated. None of the reasons for losing completerecords varied meaningfully by provider type or lo-cation.

The abstraction instrument captured approxi-mately 6000 items. Abstractors retrieved data fromclinic problem lists, notes, perinatal flow sheets, labreports, procedure reports, hospital provider notes,

intrapartum records, discharge diagnoses, and nurs-ing notes. These included virtually all the outpatientand inpatient antepartum diagnostic test results, vis-its, and therapeutic interventions; intrapartum man-agement and interventions; delivery method; and in-formation about the status of the mother and babyat birth. For the analyses, prenatal visits were thosefor which the visit was noted as a prenatal visit. Visitswith presenting problems unrelated to pregnancywere not counted as prenatal visits. Complicationsof the pregnancy and the intrapartum period anddiagnoses were constructed by algorithms that in-clude objective data abstracted from the chart (i.e.,fasting glucose greater than 110, abnormal fluid vol-ume on ultrasound report) and diagnoses listed bythe provider in the prenatal record or hospital chart.All hospital discharge diagnoses were recorded.

Matching Birth Certificates to Patient Records

Using three variables—birth date, hospital, andhour of birth (within 1 hour before or after)—1246of the 2054 abstracted cases were linked to birthcertificates. The remaining 808 cases either had oneitem missing or had multiple possible matches. Ahand match of two other variables, birthweight(within 1 ounce) and same age of mother, resultedin 691 additional matches. These 1937 matched caseswere analyzed.

Variables were considered for analysis if theywere captured by both data sources and if the defini-tion and content was the same in both data sets (TableI). This limited the variables considered to those inTable I. Those variables used to match the cases werenot analyzed.

Data Analyses

The patient served as the unit of analysis. Wefirst examined the missing value rate for each variablein each database. Next we measured the accuracy ofthe birth certificates compared with the chart ab-stractions by assessing agreement and sensitivity.Agreement was calculated as the percentage of casesin which both data sets reported the same result forthe variable of interest. Cases in which either dataset had a missing value for that variable were ex-cluded. This results in greater agreement for somevariables than would be found if cases with missing


values were included in the analysis and were consid-ered nonagreeing.

Because many of the data items have skeweddistributions, we used Cohen's kappa statistic to mea-sure the strength of agreement between the two datasets. The kappa measures the degree to which thereis agreement between the birth certificates and thechart abstractions beyond what would be expectedby chance (22). Kappa ranges between 0 and 1. Weconsidered a score above 0.75 as excellent agreement;a score of 0.40 to 0.75 as fair to good; and a scorebelow 0.40 as poor agreement beyond what wouldbe expected by chance alone. Kappa is employed inthis paper as a general indicator of agreement beyondchance and is used in combination with measures ofapparent agreement and sensitivity. Because use ofkappa has been shown to sometimes produce some-what inflated estimates of agreement beyond chanceand to have other limitations as well, results shouldbe viewed with caution (23, 24).

For uncommon conditions like pregnancy-in-duced hypertension, in which fewer than 10% of thewomen have the condition, agreement between the

databases may appear high because it is driven bythe frequency with which both data sets say the condi-tion did not exist. There may be substantial disagree-ment, however, between the birth certificates and themedical records for those cases in which the conditionmight exist. For this reason, we explored the sensitiv-ity of the birth certificate, or the proportion of theoccurrences, conditions, or procedures known to existfrom the chart abstraction data that were also re-ported by the birth certificate. Here we consideredthe medical record to be the "gold standard." Forexample, the sensitivity of the birth certificate formaternal diabetes would be the proportion of casesof diabetes found through the chart abstraction thatwere reported on the birth certificate.

Last, for both databases, we compared the over-all population estimates of the pregnancy characteris-tics examined in this study. In this analysis, we deter-mined how accurately the birth certificates describethis patient population compared to the medical re-cords' estimates. Estimates are based on all nonmiss-ing data for each data source. Hence, the number ofcases for each variable and data source varies.

Table I. Variables Used to Compare the Washington State Birth Certificates with the Chart Abstracted Data, by Source of Data

Source of data

GravidityNumber of prior live birthsLast menstrual periodNumber of prenatal visitsComplications

PrenatalDiabetesPregnancy-induced hypertension

Chronic hypertension

Placenta previaAbnormal fluid volume

IntrapartumCephalopelvic disproportion

Placental abruption

Fetal distress

ProceduresAmniocentesisUltrasoundFetal monitoringInduction and/or augmentation

How recorded on birth certificate(appendix)

Total prior pregnancies recordedTotal prior live births recordedRecorded on certificateTotal number recorded

Check-box formatCheck-box format

Check-box format

Check-box formatCheck-box format (polyhydramnios

or oligohydramnios)

Check-box format

Check-box format

Check-box format

Check-box formatCheck-box formatCheck-box formatCheck-box format (induction or

stimulation)

Chart abstraction source

Provider documentation in prenatal chartProvider documentation in prenatal chartProvider documentation in prenatal chartCount of charted encounters for prenatal care

Abnormal lab resultsProvider documentation in prenatal and/or

hospital chartProvider documentation in prenatal and/or

hospital chartUltrasound reportUltrasound report documenting oligohydram-

nios or polyhydramnios

Provider documentation in hospital chart,ICD-9 codes

Ultrasound report or provider documentationin hospital chart, ICD-9 codes

Provider documentation in hospital chart,ICD-9 codes

Operative noteReport of resultsHospital nursing documentationHospital nursing and provider documentation,

medication documentation


Compensatory patient weights were created toproduce unbiased estimates that are representativeof the study population. These weights corrected fortwo biases: (1) stratum differences in specialty/loca-tion strata sampling and participation rates; and (2)interphysician differences in patient sampling rates.Although these weights produce unbiased populationestimates, the associated variances may be biased.To adjust the variance estimates, a random effectsmodel (i.e., SUDAAN program) was applied (25).The resulting variance adjustments should result inconfidence intervals that are both unbiased and con-servative.

Statistical power was high. For example, in thepaired test of the significance of the difference be-tween the abstract and birth certificate data (TableIV) (n = 1878) on the percentage of women withfrom zero to five prenatal visits, there is a 92% powerto detect a 1% difference at the 0.05 level of signifi-cance using a two-tailed test. In addition, the two-tailed confidence intervals around the patient-by-patient agreement measures (in Table III) arerelatively narrow.

RESULTS

The average age of the 1937 women included inthe study was 26.3 years; approximately 47.6% werenulliparous; 85.4% were married at the start of theirpregnancies and 87.0% were privately insured.

Missing data were relatively uncommon fromthe birth certificates. For prenatal care, missing dataranged from 0% (prior live births) to 24.0% (lastmenstrual period). Prenatal complications were miss-ing 10.1%, intrapartum complications were missing9.1%, and procedures were missing 7.6% (Table II).There were virtually no missing data for these vari-ables from the chart abstractions. However, last men-strual period was missing for 3.3% of the chart ab-straction cases.

Agreement between the birth certificates andthe chart abstractions ranged from 22.4% (numberof prenatal visits) to 99.7% (presence or absence ofplacenta previa) (Table III). There was agreement ofgreater than 90% in 11 of the 16 variables. Agreementbetween the two data sets is highest (greater than95%) for number of prior live births, all antenatalcomplications, cephalopelvic disproportion, placen-tal abruption, and amniocentesis.

Despite the agreement noted, the kappas suggestthat for many of the variables, the agreement is not

much greater than would be expected by chance (Ta-ble III). Specifically, only for gravidity, prior livebirths, cephalopeivic disproportion, and amniocente-sis is the kappa large enough (i.e., greater than 0.75)to suggest that the agreement is excellent. For manyof the data items, the agreement beyond chance ispoor (e.g., prenatal visits, chronic hypertension, fetaldistress, fetal monitoring).

The ability of the birth certificate data to accu-rately report conditions reported in the chart abstrac-tion (sensitivity) was highly variable. The birth cer-tificates correctly identified prior live births andgravidity to four prior pregnancies more than 90%of the time. The reporting of complications of preg-nancy and labor was much lower. Only 7.3% ofwomen with chronic hypertension, 21.5% with fetaldistress, and 29.7% of women with documented poly-or oligohydramnios were identified by the birth cer-tificates as having these conditions. Approximatelyhalf of the cases of diabetes, pregnancy-induced hy-pertension, placenta previa, and placental abruptionwere captured by the birth certificates (Table III).

Table IV compares the population estimates de-rived from each data source for this study's low-riskpopulation. The birth certificates reported signifi-cantly fewer prenatal visits than the medical record.In general, the birth certificates also identified moreprimiparous women, fewer pregnancy complications,and fewer procedures than the chart abstractions.This was most striking for fetal distress, reported on4.8% of the birth certificates compared with 21.1%of the chart abstractions.

Table II. Proportion of Cases with Missing Data from the BirthCertificates for Matched Low-Risk Women in Washington State

GravidityPrior live birthsLast menstrual perioda

Number of prenatal visitsPrenatal complications'Intrapartum complicationsb

Procedures'Number of matched cases = 1937

Percentage of casesmissing from birth

certificates

1.50

24.05.1

10.1c

9.1c

7.6c

aLast menstrual period is considered missing if month, day, or yearis missing.

bSee Table I for included variables.cIf no box, including the "none" box, was checked on this sectionof the birth certificate (Appendix), each condition in the sectionwas counted as having missing data.

DISCUSSION

Birth certificates provide large data sets on peri-natal health that are well maintained and accessible.They have been used for decades to monitor anddescribe the perinatal health of communities and sub-populations and to support interventions to improveperinatal health and health services. In general, birthcertificates have been thought to portray prenatalhistory and care variables adequately and to reportcomplications poorly. In the current era, the use ofsecondary data sets is likely to increase. Knowing thestrengths and weaknesses of the data is critical beforewe use them to justify or modify programs. In thisstudy, we evaluate the Washington State birth cer-tificate against a large primary data set collected on asample of low-risk obstetrical patients in Washington

State. This study goes beyond other work, however,because it compares the birth certificate informationnot only with hospital data, but also with a numberof variables abstracted from provider office records.

This study identified few variables recorded ac-curately on the birth certificate when measuredagainst careful abstraction of medical office and hos-pital records. On the surface, agreement appears tobe reasonably good; the kappa statistics, however,suggest that agreement is often not much better thancould be expected by chance. It is encouraging thattwo important variables describing women's preg-nancies—gravidity and prior live births—had a highdegree of agreement, high kappa statistic, and highsensitivity. However, the population estimates tell usthat when there is disagreement between the birthcertificates and the record abstracts, the birth certifi-


Table III. Agreement of Birth Certificate Data with Chart Abstraction Data for Low-Risk Women in Washington Statea

Prior live births0123

GravidityGravid 1Gravid 2-4Gravid >5

Last menstrual periodWithin 3 daysWithin 7 days

Number of prenatal visitsWithin 1Within 2

ComplicationsPrenatal

Placenta previaChronic hypertensionDiabetesAbnormal amniotic fluid volumePregnancy-induced hypertension

IntrapartumPlacental abruptionCephalopelvic disproportionFetal distress

ProceduresAmniocentesisInduction or augmentationUltrasoundFetal monitoring

Percent agreement (95%confidence interval)

97.8 (96.8,98.8)99.0 (98.5,99.5)98.1 (97.1,99.0)98.8 (98.1,99.6)99.7 (99.2,100.1)93.0 (91.1,94.9)93.3 (91.3,95.3)91.7 (89.5,94.0)96.8 (95.0,98.5)71.0 (66.7,75.3)85.2 (82.2,88.2)91.0 (88.7,93.3)22.4 (19.7,25.1)53.3 (49.7,56.9)72.1 (68.6,75.6)

99.7 (99.4,100.0)98.0 (96.8,99.2)97.9 (97.0,98.8)96.4 (94.9,97.9)95.3 (93.8,96.8)

99.5 (99.0,100.0)97.9 (97.0,98.8)83.7 (80.7,86.7)

98.7 (98.1,99.3)86.0 (83.1,88.9)68.5 (63.3,73.7)77.8 (73.2,82.4)

Agreement beyondchance (kappa)

0.960.980.960.950.950.830.880.850.780.70nana

0.12nana

0.590.120.660.430.58

0.580.800.29

0.780.680.360.05

Percent of conditionaccurately reported in the

birth certificate(sensitivity)

na99.796.595.891.9na

99.491.966.1nananananana

49.07.3

52.029.758.6

45.976.621.5

69.171.763.078.4

na = not applicable.aThe number of cases for each data element is from 1525 to 1820.

cates will estimate a greater proportion of women intheir first pregnancy.

The birth certificate's underreporting of condi-tions found in the medical record of these womenwho began their pregnancies without risk is consis-tent with the findings of other research (4, 5, 7, 8,10-13). The risk for epidemiologists and health plan-ners is the underestimation particularly of intrapar-tum conditions and interventions most likely to con-sume the greatest resources.

Number of prenatal visits and timing of preg-nancy care initiation are two variables frequentlyused to evaluate the access of populations to prenatalcare. Compared to the medical record, we found thatthe birth certificates significantly underestimate thenumber of prenatal visits. This disagrees with thework by Clark et al., who found the birth certificateto overestimate the number of prenatal visits (18).Our study abstracted all visits to the prenatal pro-vider. Although we attempted to count only those


Table IV. Population Estimates Derived from the Chart and Birth Certificate Data for Low-Risk Women in Washington State

Prenatal CareNumber of prenatal visits

0-56-1011-1516-20>21

GravidityGravid 1Gravid 2-4Gravid >5

Prior live births0123>3

ComplicationsPrenatal

DiabetesPregnancy-induced hypertensionChronic hypertensionPlacenta previaAbnormal fluid volume

IntrapartumCephalopelvic disproportionPlacental abruptionFetal distress

ProceduresAmniocentesisAt least one ultrasoundFetal monitoringInduction and/or augmentation

Population estimates" based on:

Chart % (CI)b Birth certificate % (CI)C

0.4 (0.0,0.9)17.7 (14.2,21.2)71.6 (67.9,75.3)9.8 (7.5,12.1)0.5 (0.0,1.0)

31.8 (28.5,35.1)63.3 (59.7,66.9)4.8 (3.2,6.4)

47.6 (43.5,51.7)33.3 (29.6,37.0)15.3 (12.5,18.1)3.8 (2.6,5.0)0.0 (0.0,0.0)

3.9 (2.7,5.1)6.3 (4.4,8.2)1.8 (0.7,2.9)0.4 (0.0,0.8)4.8 (3.2,6.4)

6.2 (4.4,8.0)0.7 (0.2,1.2)

21.1 (17.7,24.5)

3.8 (2.5,5.1)77.7 (73.0,82.4)97.8 (96.7,98.9)36.4 (32.8,40.0)

1.3 (0.6,2.0)26.5 (21.331.7)64.7 (60.0,69.4)7.2 (5.2,9.2)0.2 (0.0,0.4)

36.4 (32.6,40.2)60.3 (56.6,64.0)3.3 (2.2,4.4)

48.4 (43.3,52.5)32.9 (29.0,36.8)15.2 (12.4,18.0)3.5 (2.3,4.7)0.0 (0.0,0.0)

2.3 (1.3,3.3)5.8 (3.9,7.7)0.4 (0.1,0.7)0.2 (0.0,0.5)1.3 (0.5,2.1)

5.4 (3.7,7.1)0.4 (0.1,0.7)4.8 (3.0,6.6)

2.6 (1.4,3.8)51.7 (46.2,57.2)77.9 (73.4,82.4)29.5 (25.3,33.7)

Significance of paireddifferencesd

0.000e

0.0040.0000.0050.0590.3320.000e

0.0000.0010.0080.000e

0.0020.4400.7600.1970.320

0.0000.5890.0140.4180.000

0.1500.1680.000

0.0100.0000.0000.001

aConfidence intervals available on request.bThe numbers of cases on which the birth certificate estimates are based are as follows (top to bottom): 1902; 1878 (all prenatal visits);1915 (all gravidity and prior live birth estimates); 1820; 1820; 1820; 1832; 1820; 1832 (for all intrapartum complication estimates); and1847 (for all procedures).

cThe number of cases on which the chart estimates are based is 1937, except for abnormal fluid volume (1618), fetal monitoring (1936),and induction/augmentation (1925).

dThe estimates for the two data sources displayed in the two columns to the left are based on all available data from each data source.Significance levels are based on paired r-tests wherein cases are only included if data are nonmissing for both data sources. However,substituting abstract data for missing birth certificate data and recomputing the birth certificate estimates did not meaningfully changethe analysis findings.

eOverall chi-square across categories.


that were actually prenatal visits, we might have over-estimated the number of prenatal visits. These couldhave important effects on the Kessner and Kotel-chuck measurements of the adequacy of prenatalcare, which incorporate both of these measures (26,27). Additional work examining whether this under-reporting of the number of prenatal visits varies bysociodemographic or other characteristics would beimportant. In particular, it would be important toassess this reporting in high-risk, resource-intensivepregnancies.

Several items examined in this study were addedto the birth certificate in 1989 (i.e., augmentation andinduction of labor, ultrasound, and fetal monitoring).These procedures may be underreported because ofcoders' unfamiliarity with them. Accuracy of re-porting on the birth certificate can be expected toincrease over time with increased use of the new birthcertificate form.

The underreporting of fetal distress by the birthcertificate was particularly striking. There are twopossible explanations for this difference. The chartabstraction database identified fetal distress if it wasreported as an ICD-9 discharge code. This dischargecoding may overreport fetal distress because of itsimportance in justifying billing codes. Alternatively,those completing the birth certificate form (nurses,clinicians, and others) may have had different stan-dards for recording the diagnosis of fetal distress onthe birth certificate. It is also likely that those com-pleting the birth certificate forms are not capturingall cases of fetal distress.

This study is limited by several factors. The data-base from the medical records was created as part ofanother research study describing the interspecialtyand rural-urban differences in the content of obstet-rical care (28, 29). We drew a stratified random sam-ple of obstetric providers in Washington State, fromwhose practices we selected a random sample of low-risk women. This is not a population-based sampleof all pregnant women in Washington State. If ourrosters of providers or patients were incomplete, ei-ther systematic or random bias might have been in-troduced, the effect of which we cannot estimate.However, the data have been weighted to be repre-sentative of the population of Washington State low-risk women and we believe that our rosters of provid-ers and patients were accurate.

Second, the generalizability of this study is lim-ited. The study that generated the abstracted dataincluded only women who were considered to below risk at entry into prenatal care, raising questions

about the amount of agreement we might find if oursample had included women entering care later intheir pregnancies or with more complicated pregnan-cies (8). The work of others, however, suggests thatthe accuracy of the birth certificates is poorest forcomplications, procedures, and congenital anomalies(7-11). Generalizability is also limited to states withreporting systems and processes similar to those ofWashington State. Other states may have more fieldrepresentatives and more comprehensive systems toretrieve incomplete data. These states could be ex-pected to have better quality data. In addition, weonly analyzed those variables that were clearly com-parable in both data sets. Accuracy of other variablesmay be better or worse.

Third, the chart abstraction and birth certificateare in part limited by reliance on the written record.Major variations in the recording behaviors of pro-viders could result in either understated or overstatedestimates of various conditions. Also, electronic birthregistration was not available in Washington Stateuntil 1992. Its effect on accuracy is unknown.

Fourth, we have accepted the medical recordas our "gold standard." For certain conditions, theabstracted record is truly a "gold standard." We haveactual lab or ultrasound values for several of theconditions that we have analyzed (diabetes, ultra-sound, amniocentesis, fluid volume, placenta previa).For other conditions, however, such as pregnancy-induced hypertension, chronic hypertension, and fe-tal distress, provider designation or hospital dis-charge coding could either over- or understate thetrue prevalence of the condition.

This work adds to other state-level investigationsof birth certificate validity and completeness by ana-lyzing the wealth of data abstracted from prenatalrecords in addition to hospital records. With thismore complete database about the care delivered tothe sampled women, we were able to closely scruti-nize the birth records for completeness and accuracyof selected variables. Consistent with the findings ofother authors, there is better agreement and sensitiv-ity for certain descriptive variables, such as gravidityand prior live births, and poorer and more variableagreement and sensitivity for those items reportedby the check-box format. This is the only study toprovide population estimates comparing the birthcertificate's estimates to those of another more com-plete data source. The picture estimated by the birthcertificates is one of less prenatal care, fewer compli-cations, and fewer interventions than the estimatesderived from the chart abstractions.


If taken at face value, these estimates (e.g., offewer prenatal visits) could influence departments ofhealth or insurers to develop unnecessary interven-tions to increase compliance with perinatal guide-lines. In addition, the underestimation of complica-tions and interventions could result in lowerallocation of resources to perinatal care by plannersand insurers.

The quality of the birth record can be improvedin a number of ways. First, the electronic record pro-vides some advantages. These systems can provideimmediate feedback to the person completing therecord. Continuous training and quality improve-ment are more possible if electronic systems are pro-grammed to return incomplete records for comple-tion and, to provide surveillance of data for outlierresponses. Second, in Washington State several hos-pitals are experimenting with collecting some dataprenatally. In addition, others have suggested uni-form training for those completing records, as wellas the allocation of more time for staff to completerecords. Finally, several states and the National Cen-ter for Health Statistics are exploring transfer of somedata directly from the hospital computer system tothe vital statistics records. Research is needed to de-termine which interventions actually improve thequality of the records.

Birth certificates are a major source of data usedto analyze and monitor perinatal outcomes andhealth. Although changes in format increased accu-racy and completeness of reporting, the current studysuggests that the inaccuracies of birth certificates,when measured against data collected in the prenataland intrapartum records of pregnant women, haveimportant implications. Indeed, use of these birthcertificate data for health policy development and forresource allocation should be tempered with caution,recognizing the underreporting of those conditionsand interventions requiring greater resources.

ACKNOWLEDGMENTS

This study was supported through the WWAMIRural Health Research Center, which is supported bythe Health Resources and Services Administration'sFederal Office of Rural Health Policy and the Agencyfor Health Care Policy and Research. Preparation ofthis manuscript was assisted by a grant from the Rob-ert Wood Johnson Foundation, Princeton, New Jer-sey. The data were originally collected per a grantfrom the Agency for Health Care Policy and Re-

search (RO1 HSO6166). The opinions and conclu-sions are those of the authors and do not necessarilyrepresent the views of the supporting agencies andfoundation. We thank the many other faculty, staff,and participating providers for their help in this study.We thank Dr. Paula Diehr for her advice on statistics.We also thank Dr. Patricia Starzyk, Research Investi-gator for the Center for Health Statistics, WashingtonState Department of Health, for background infor-mation on the collection of birth-related data inWashington State. Of course, the authors are solelyresponsible for any errors that occur in the methodsand text.

REFERENCES

1. Tolson GC, Barnes JM, Gay GA, Kowaleski JL. The 1989Revision of the U.S. Standard Certificates and Reports. Hyatts-ville, MD: National Center for Health Statistics, 1991. (Vitaland health statistics (4), no. 28)

2. Woolbright LA, Harshbarger DS. The revised standard cer-tificate of live birth: Analysis of medical risk factor data frombirth certificates in Alabama, 1988-92. Pub Health Rep1995:110:59-62.

3. Centers for Disease Control. Pregnancy risks determinedfrom birth certificate data—United States, 1989. JAMA1992;268:1831-2.

4. Lilienfield AM, Parkhurst E, Patton R, Schlesinger ER. Accu-racy of supplemental medical information on birth certificates.Pub Health Rep 1951:66:191-8.

5. Green HG, Nelson CJ, Gaylor DW, Holson JF. Accuracyof birth certificate data for detecting facial cleft defects inArkansas children. Cleft Palate J 1979;16:167-70.

6. David R. The quality and completeness of birthweight andgestational age data in computerized birth files. Am J PubHealth 1980;70:964-73.

7. Frost F, Starzyk P, George S, McLaughlin JF. Birth complica-tion reporting: The effect of birth certificate design. Am JPub Health 1984:74:505-6.

8. Ethen MK, Selwyn BJ, Borders SB. Hospital reporting prac-tices and their impact on Texas birth certificate data quality.In: Department of Health and Human Services, ed. Proceed-ings of 25th Public Health Conference on Records and Statis-tics. 1995:405-10. (DHHS publication no. (PHS) 96-1214).

9. Buescher PA, Taylor KP, Davis MH, Dowling JM. The qualityof the new birth certificate data: A validation study in NorthCarolina. Am J Pub Health 1993;83:1163-5.

10. Piper JM, Mitchell EF, Snowden M, Hall C, Adams M, TaylorP. Validation of the 1989 Tennessee birth certificates usingmaternal and newborn hospital records. Am J Epidemiol1993:137:758-68.

11. Parrish KM, Holt VM, Connell FA, Williams B, LoGerfoP. Variations in the accuracy of obstetric procedures anddiagnoses in birth records in Washington State. Am J Epide-miol 1993:138:119-27.

12. Hexter AC, Harris JA, Roeper P, Croen LA, Krueger P, GantD. Evaluation of the hospital discharge diagnoses index andthe birth certificate as sources of information on birth defects.Pub Health Rep 1990;105:296-307.

13. Snell LM, Little BB, Knoll KA, Johnston WL, Rosenfeld CR,Gant NF. Reliability of birth certificate reporting of congenitalanomalies. Am J Perinatal 1992;9:219-22.


14. Watkins ML, Edmonds L, McClearn A, Mullins L, MulinareJ, Khoury M. The surveillance of birth defects: The usefulnessof the revised U.S. standard birth certificate. Am J PubHealth 1996;86:731-4.

15. Fingerhut LA, Kleinman JC. Comparability of reportingbetween the birth certificates and the 1980 National NatalitySurvey. Hyattsville, MD: National Center for Health Sta-tistics, 1985. (Data Evaluation and Methods Research (2),No. 99)

16. Schoendorf KC, Parker JD, Kiely JL. Vital and Health Statis-tics: Comparability of the Birth Certificate and 1988 Maternaland Infant Health Survey. Hyattsville, MD: National Centerfor Health Statistics, 1993. (Data Evaluation and MethodsResearch (2), No. 116)

17. Calle EE, Khoury MJ. Completeness of the discharge diagno-ses as a measure of birth defects recorded in the hospital birthrecord. Am J Epidemiol 1991;134:69-77.

18. Clark K, Fu CM, Burnett C. Accuracy of birth certificate dataregarding the amount, timing, and adequacy of prenatal careusing clinic medical records as referents. Am J Epidemiol1997;145:68-71.

19. Baldwin L-M, Raine T, Jenkins LD, Hart LG, RosenblattRA. Do providers adhere to ACOG standards? The case ofprenatal care. Obstet Gynecol 1994;84:549-56.

20. Kosekoff J, Kanouse DE, Rogers WH, McCloskey L,Winslow CM, Brook RH. Effects of the NIH ConsensusDevelopment Program on physician practice. JAMA1987;258:2708-13.

21. Dobie SA, Hart LG, Fordyce M, Rosenblatt RA. Do womenchoose their obstetric providers based on risks at entry into

prenatal care? A study of women in Washington State. ObstetGynecol 1994;84:557-64.

22. Fleiss JL. Statistical Methods for Rates and Proportions. 2ndedition. New York: John Wiley & Sons, 1981.

23. Lantz CA, Nebenzahl E. Behavior and interpretation of theK statistic: Resolution of the two paradoxes. J Clin Epide-miol 1996;49:431-4.

24. Guggenmoose-Holzmann I. The meaning of kappa: Probablis-tic concepts of reliability and validity revisited. J Clin Epide-miol 1996;49:775-82.

25. Shah BV, Folsom RE, LaVange LM, Wheeless SC, Boyle KE,Williams RL. Statistical methods and mathematical algorithmsused in SUDAAN. Research Triangle Park, NC: ResearchTriangle Institute, 1993.

26. Kessner DM, Singer J, Kalk CE, Schlesinger ER. Chapter 2.In: Institute of Medicine, National Academy of Sciences, ed.Infant Death: An Analysis by Maternal Risk and Health Care.Washington, DC: Institute of Medicine and National Acad-emy of Sciences, 1973.

27. Kotelchuck M. An evaluation of the Kessner adequacy ofprenatal care index and a proposed adequacy of prenatal careutilization index. Am J Public Health 1994;84:1414-20.

28. Hart LG, Dobie SA, Baldwin L-M, Pirani MJ, Fordyce M,Rosenblatt RA. Rural and urban differences in physician re-source use for low-risk obstetrics. Health Sen Res1996;31:429-52.

29. Rosenblatt RA, Dobie SA, Hart LG, Schneeweiss R, GouldD, Raine RT, Benedetti TJ, Pirani MJ, Perrin EB. Interspe-cialty differences in the obstetric care of low-risk women. AmJ Pub Health 1997;87:344-51.

Documents

How Well Do Birth Certificates Describe the Pregnancies They Report? The Washington State Experience with Low-Risk Pregnancies