The Long-Term Effects of Building Strong Families: A …...Long-Term Effects of Building Strong Families 447 when Congress voted to provide $75 million annually for marriage education

ROBERT G. WOOD AND QUINN MOORE Mathematica Policy Research

ANDREW CLARKWEST Mathematica Policy Research∗

ALEXANDRA KILLEWALD Harvard University∗∗

The Long-Term Effects of Building Strong Families:

A Program for Unmarried Parents

The authors present findings from a large-scale, random-assignment evaluation of Build-ing Strong Families (BSF), a program offeringgroup sessions on relationship skills educationto low-income, unmarried parents who wereexpecting or had recently had a baby. Findingsbased on a 3-year follow-up survey of over 4,000couples indicate that BSF did not succeed in itscentral objectives of improving the couple rela-tionship, increasing the quality of coparenting,or enhancing father involvement. In fact, theprogram had modest negative effects on someof these outcomes. BSF also had little impacton child well-being, with no effect on children’sfamily stability or economic well-being and onlya modest positive effect on children’s socioe-motional development. Impacts varied acrossthe 8 study sites. Although attendance at groupsessions was relatively low, there is little evi-dence of program effects even among coupleswho attended sessions regularly.

Mathematica Policy Research, P.O. Box 2393, Princeton,NJ 08543-2393 ([email protected]).∗Mathematica Policy Research, 1100 1st Street, NE, 12thFloor, Washington, DC 20002–4221.∗∗Department of Sociology, Harvard University, 648William James Hall, 33 Kirkland St., Cambridge, MA02138.

Key Words: child well-being, coparenting, fatherhood,low-income families, marriage education, relationshipeducation.

In 2010, over 40% of U.S. births were tounmarried parents (Martin et al., 2012). Mostof the parents were in romantic relationshipswhen their children were born; about half wereliving together (Carlson, McLanahan, England,& Devaney, 2005; Sigle-Rushton & McLana-han, 2002). Unmarried parents are typicallyoptimistic about their future together, includingthe likelihood that they will eventually marry(Carlson et al., 2005). But these hopes are oftenunrealized. Most are no longer in a romanticrelationship 5 years after their child’s birth(Center for Research on Child Wellbeing, 2007).

The disruption of unmarried parents’ relation-ships is significant because children growingup in households that do not include boththeir biological parents are at greater riskof poor outcomes (Amato, 2005; Carlson &Corcoran, 2001; Kim, 2011; McLanahan &Sandefur, 1994). For this reason, there is con-siderable interest among policymakers in sup-porting unmarried parents’ efforts to maintaintheir relationships. The 1996 Personal Respon-sibility and Work Opportunity ReconciliationAct made increasing the number of childrenraised in two-parent families an explicit pol-icy objective. In 2001, the Administration forChildren and Families (ACF), U.S. Departmentof Health and Human Services, launched theHealthy Marriage Initiative. In 2005, Congressapproved $100 million in annual funding forprograms designed to encourage and strengthenmarriage. This funding was continued in 2010

446 Journal of Marriage and Family 76 (April 2014): 446–463DOI:10.1111/jomf.12094

Long-Term Effects of Building Strong Families 447

when Congress voted to provide $75 millionannually for marriage education.

Building Strong Families (BSF) was one ofthe first major projects under ACF’s HealthyMarriage Initiative. The project developed,implemented, and tested voluntary programsdesigned to help unmarried, economicallydisadvantaged parents who were expectingor had just had a baby strengthen theircouple relationship, with the ultimate goalof helping them create a stable and healthyhome environment for their children. The maincomponent of the BSF model was relationshipskills education offered to couples in groupsessions. The model was implemented in eightsites and tested using a random-assignmentresearch design involving over 5,000 couples.To track couples’ outcomes, telephone surveyswere conducted with both mothers and fathers15 and 36 months after they applied for BSF.

Results based on the 15-month follow-upwere released in 2010 (Wood, McConnell,Moore, Clarkwest, & Hsueh, 2010). That anal-ysis, which focused on interim outcomes—primarily the couples’ relationship status andquality—found that BSF had no effect onthese relationships when results from the eightevaluation sites were combined. However, theresults varied across the eight sites, with oneprogram having a consistent pattern of positiveeffects and another having a number of negativeeffects. The other six programs generally hadlittle or no effect on relationships.

In this article, we present final BSF impactresults based on the 3-year follow-up. Theanalysis examines how the somewhat mixedpicture that emerged at 15 months has evolvedover the longer term. It is important toexamine long-term impacts because changesin relationship status can take time to unfold.Moreover, in this article we extend the analysisto additional domains associated with childwell-being that were not examined in the 15-month analysis. The ultimate aim of BSF was toimprove child well-being by improving parents’relationship stability and quality. To examineBSF’s success in this area, we estimated impactson the well-being of the children who madethese couples eligible for BSF services: thoseborn around the time couples applied for theprogram and who were about 3 years old at thefinal follow-up.

In addition, we examined BSF’s effectson the couples who actually received the

program’s core service of relationship skillseducation. Getting couples to attend sessionsproved challenging, and 45% of couples whowere offered program services never attendedthese sessions. Because of this large proportionof nonattending couples, we used quasi-experimental techniques to examine BSF’seffects on two groups: (a) couples who attendedany group sessions and (b) couples who attendedat least half of these sessions.

BACKGROUND

Related Literature

Relationship skills education programs haveexisted for several decades. Evidence has sug-gested that these programs can be effective atimproving the relationship quality of middle-class, married couples (Dion, 2005; Hawkins,Blanchard, Baldwin, & Fawcett, 2008; Mark-man, Renick, Floyd, Stanley, & Clements,1993; Reardon-Anderson, Stagner, Macomber,& Murray, 2005). Until recently, little rigor-ous research existed on the effectiveness ofsimilar programs for low-income couples orunmarried parents, and few studies have exam-ined long-term impacts of these programs. A2010 meta-analysis of research on relationshipskills programs serving low-income couples,both married and unmarried, found generallypositive impacts. The authors noted that theresearch summarized was typically not rigor-ous. Most studies used pre–post designs with nocomparison group, sample sizes were small, andoutcomes were typically measured shortly afterthe end of the program (Hawkins & Fackrell,2010).

Several recent studies of relationship skillsprograms serving low- and moderate-incomecouples have used experimental researchdesigns. A study of the Prevention and Relation-ship Enhancement Program (PREP) for StrongBonds, a relationship skills education programserving married military couples, found that theprogram lowered the risk of divorce during the2 years after the end of the program; no impactswere found on self-reported relationship quality(Stanley et al., 2013). The Fatherhood, Rela-tionship, and Marriage Education project testeda version of the PREP for Strong Bonds cur-riculum adapted for low-income parents andfound positive effects on conflict managementmeasured 2 weeks after the program ended

448 Journal of Marriage and Family

(Wadsworth et al., 2011). The Supporting FatherInvolvement study examined the effectiveness ofa curriculum delivered to low-income couples ingroup sessions. The Supporting Father Involve-ment program focused primarily on increasingfathers’ involvement with their children but alsoaimed to improve the couple relationship. Theauthors reported positive impacts on relationshipsatisfaction at the 18-month follow-up (Cowan,Cowan, Pruett, Pruett, & Wong, 2009). TheACF-sponsored Supporting Healthy Marriage(SHM) evaluation tested a program similar toBSF but serving low-income married couples.The study found relatively small positive effectson couple’s relationship quality at the 12-monthfollow-up, but no effect on relationship stabil-ity (Hsueh et al., 2012). These studies includedprimarily—and, in the case of both SHM andPREP for Strong Bonds, exclusively—marriedcouples.

Unmarried parents face distinct relationshipchallenges that could influence the effectivenessof relationship skills education programs.These challenges include lower levels ofcommitment, higher rates of infidelity, morefrequent multiple-partner fertility, and greatereconomic vulnerability (Carlson & Furstenberg,2006; Edin, England, & Linnenberg, 2003).Very little rigorous research has examined theeffectiveness of relationship skills educationprograms serving primarily unmarried parents.The Young Parenthood Program, an 8- to 12-week intervention designed to improve therelationship skills of coparenting adolescents,was evaluated with a random assignmentresearch design. The study found short-term,marginally statistically significant reductions inintimate partner violence that did not persistat the final follow-up 18 months after thechild’s birth (Florsheim, McArthur, Hudak,Heavin, & Burrow-Sanchez, 2011). To date,BSF is the only random-assignment evaluationof a relationship skills education programserving primarily unmarried adult parents. Inaddition, it is one of only a handful ofstudies of relationship skills programs thatused a large research sample and among thefew that has a follow-up period extending3 years after study enrollment. Thus, the analysisof the 3-year BSF follow-up data presentedhere provides important new information onthe long-term effects of offering relationshipskills education to low-income, unmarriedparents.

The BSF Program

BSF was a voluntary program for new unmarriedparents. Couples were eligible to enroll ifthey had had a baby in the last 3 monthsor were expecting one, were both at least18 years old, and were romantically involvedand unmarried at the time their baby wasconceived. Eight organizations were selected toimplement the BSF program model, which laidout a set of research-based program guidelines(Hershey, Devaney, Dion, & McConnell, 2004).Four sites (those in Florida, Indiana, andHouston and San Angelo, Texas) added BSFservices to their Healthy Families programs,which aim to promote positive parentingvia staff visiting and educating new andexpectant parents in their homes. Other programs(those in Baltimore, Maryland, and BatonRouge, Louisiana) were offered by agenciesthat provided a mix of services to low-income families. The infrastructure for programsin Atlanta, Georgia, and Oklahoma City,Oklahoma were built from the ground upspecifically for BSF. Across the eight programs,key recruitment sources included hospitalmaternity wards, prenatal clinics, health clinics,and clinics for the Special Nutritional Programfor Women, Infants, and Children.

The core component of BSF was curriculum-based group education on relationship skills.Programs chose one of three curricula thathad been adapted by the developers to theneeds of unmarried parents. These adaptationswere aimed to better address issues specificto low-income unmarried couples and enhancethe cultural sensitivity of the intervention. Forexample, early focus groups revealed that manypotential participants had negative interactionswith educational institutions and wished toavoid being lectured. In response, curriculawere modified to use a less didactic approachand to allow couples to share their ownexperiences and learn from each other. Inaddition, topics were added to the curricula thatresearch suggests are particularly important forlow-income, unmarried couples, including skillsfor building trust and commitment and managingrelationship issues related to multiple-partnerfertility (McConnell et al., 2006).

The program in Oklahoma City chose the“Becoming Parents for Low-Income and Low-Literacy Couples” curriculum, developed byPamela Jordan. The San Angelo and Houstonprograms chose “Love’s Cradle,” developed


by Mary Ortwein and Bernard Guerney. Theother five programs chose “Loving Couples,Loving Children,” developed by John andJulie Gottman. These curricula covered astandard set of topics. However, the emphasisvaried somewhat, with “Becoming Parents”emphasizing the transition to parenthood. Inaddition, “Becoming Parents” included only30 hours of material, compared with 42 hoursfor the other two curricula, and could thusbe delivered in fewer sessions. Under allthree curricula, group sessions usually metweekly and ranged in length from 2 to5 hours, with shorter sessions typically heldon weeknights and longer sessions held onweekends. Depending on the format and thenumber of hours of instruction offered, thecurriculum could take as little as 6 weeksor as much as 5 months to complete. Thesessions were led by facilitators who weretrained directly by the curriculum developers.The developers also provided ongoing technicalassistance in an effort to promote fidelity ofprogram implementation (Dion et al., 2008).

BSF complemented the group relationshipskills education with other supports, including afamily coordinator who reinforced relationshipskills, provided emotional support, and encour-aged participation in the group sessions. Thefamily coordinator also assessed family mem-bers’ needs and referred them for appropriatesupport services, such as education, employ-ment, and mental health services. In the fourHealthy Families programs that adopted BSFprograms, home visitors were assigned to fill theBSF family coordinator role and continued pro-viding Healthy Families services during homevisits.

The BSF evaluation included an extensiveimplementation study of the delivery of pro-gram services in the eight BSF evaluation sites(Dion et al., 2008; Dion, Avellar, & Clary,2010). The study found that the eight sites suc-cessfully implemented BSF’s core componentsas specified by the program model. In addi-tion, program participants in focus groups andin-depth interviews reported benefiting from thecommunication and conflict management tech-niques taught in group sessions as well as thesocial network offered by the groups (Dion et al.,2010). Despite these positive views of programservices, the implementation study also revealedthat sites had difficulty getting couples to attendgroup sessions regularly.

Programs devoted substantial effort to pro-moting attendance at group sessions, includingfree meals, on-site child care, and transporta-tion assistance. Some programs also offeredcash incentives, gift cards, or baby productsto promote attendance. Despite these efforts,across the eight programs, only 55% of the cou-ples assigned to the program group attended agroup session. Couples who attended at leastone session typically attended multiple sessionsand received a substantial dose of this programcomponent, averaging 21 hours of attendance.In a later section of this article we examineimpacts for this group of couples as well asthose who attended at least half of the sessions.Most couples who did not attend group sessionsreceived other services from the program, suchas help from a family coordinator or referrals tosupport services. Overall, 90% of couples whoenrolled in BSF received some service from theprogram.

Analytic Approach

The main pathway by which BSF was expectedto improve the lives of participating familieswas by improving the quality of the couple’sromantic relationship. The hope was thatimproved relationship quality would increasethe likelihood that couples remained together ina healthy relationship, which would ultimatelyimprove child well-being. It was also thoughtthat the program could improve coparenting andfather involvement either directly or indirectlythrough its effects on the couple relationship.BSF’s potential effects on coparenting and fatherinvolvement could in turn contribute to theprogram’s intended influence on child well-being.

The BSF intervention has the potentialto affect multiple aspects of couples’ lives.Examining a large number of outcomes in animpact analysis increases the risk of findinga statistically significant result by chance(Schochet, 2009). To address this multiple-comparison concern, our analysis focused onlyon the outcome domains BSF was intendedto influence most directly and made use of arelatively small set of outcomes within eachoutcome domain.

We organized our analysis around the threebroad areas that BSF hoped to affect: (a) thecouple relationship, measured by relationshipquality and relationship status; (b) coparenting


and father involvement; and (c) child well-being, measured by family stability, economicwell-being, and socioemotional development.An examination of impacts on these outcomesserved as a test of whether the programsucceeded in its primary objectives of improvingcouples’ relationships, their parenting, and theirchildren’s well-being.

METHOD

Study Design and Research Sample

Couples who applied to the BSF program wereassigned randomly to either the BSF groupthat was offered admission to the program orto a control group that was not. Across theeight programs, 5,102 couples were randomlyassigned from July 2005 to March 2008, with2,553 assigned to the BSF group and 2,549assigned to the control group. As illustratedin Table 1, random assignment created tworesearch groups (BSF couples and control groupcouples) with very similar characteristics atbaseline.

Most couples in the research sample werein stable relationships and aspired to marriagewhen they applied for the program. Acrossall programs, 7% of couples were married atprogram application, having wed after their babywas conceived but before applying for BSF (seeTable 1). About 6 in 10 were unmarried andreported living together all of the time. Amongcouples unmarried at application, almost 60%reported that they both thought there was either“a pretty good” or “an almost certain” chancethat they would marry each other.

The couples who applied for BSF facedmany potential stresses to their relationships.Their earnings were generally low, with thecombined earnings of the mother and fatheraveraging about $20,000 in the year prior toBSF application. Only 37% of couples includedtwo members with high school diplomas. Innearly half of all couples applying for BSF, atleast one of the parents had a child from a priorrelationship.

BSF served a racially and ethnically diversepopulation. About half the couples were AfricanAmerican, about one quarter were Hispanic, and12% were White (see Table 1). The rest wereinterracial couples or couples in which bothparents considered themselves neither White,nor African American, nor Hispanic.

Data Sources

We used data from two sources: (a) a baselineform completed by all parents when they appliedto BSF and (b) telephone surveys conductedabout 3 years after program application. In bothresearch groups, 80% of mothers and 69% offathers responded to the 36-month follow-upsurvey. At least one parent responded in 4,247couples (85% of all couples). This response ratewas the same for the two research groups. Weused the sample of couples for which at least onemember responded to estimate most programimpacts. If only one member responded, weused a multiple-imputation strategy to imputethe response of the other partner.

Measures Related to the Couple’s Relationship

Relationship status. We examined impacts onthree measures of the couple’s relationship statusat follow-up: (a) whether both members of thecouple reported being romantically involved,(b) whether both reported living together (eithermarried or unmarried) all or most of the time,and (c) whether both reported being marriedto each other. When couple members disagreedabout their relationship status, we used a simplerule: A couple was categorized as having aparticular status only if both members of thecouple reported that status. When there wasa discrepancy between the two responses, thecouple was assigned to the “no” category forthat particular outcome.

In the large majority of couples, the partnersagreed on their relationship status. The rateof disagreement between partners was 9% forromantic involvement, 9% for coresidence, and2% for marital status. Similarly high ratesof partner disagreement on relationship statusamong unmarried parents were found in thenationally representative data from the FragileFamilies and Child Well-Being Study (Knab &McLanahan, 2007). For all relationship statusmeasures, rates of mother–father disagreementwere similar in the two research groups.

Relationship quality. Using the 37 relationshipquality questions from the follow-up survey,we conducted a factor analysis to develop asmall set of key relationship quality measuresfor the impact analysis. Five distinct rela-tionship quality measures emerged from thisanalysis:


Table 1. Baseline Characteristics of the Building Strong Families (BSF) Program and Control Group Couples

CharacteristicBSF

couplesControl group

couples

Socioeconomic and demographic characteristicsRace/ethnicity (%)

Both partners are Hispanic 25.7 25.4Both partners are Black, non-Hispanic 47.5 46.8Both partners are White, non-Hispanic 11.5 11.5All other couples 15.4 16.2

High school diploma receipt (excluding GEDs; %)Both partners have diploma 36.4 37.8One partner has diploma 36.9 36.3Neither partner has diploma 26.8 25.9

Average age (years)Mother’s age 23.6 23.5Father’s age 26.0 25.8

Couples’ total earnings in past year $20,651 $19,866∗

Either partner received TANF or SNAP in past year (%) 46.0 45.2Relationship characteristics

Couple’s relationship status (%)Married to each other 6.6 7.0Unmarried, cohabiting full time 59.9 57.3∗

Unmarried, not cohabiting full time 33.5 35.7Relationship qualitya

Highest tercile (%) 31.6 32.8Middle tercile (%) 35.6 33.4Lowest tercile (%) 32.9 33.9Average scale value (range: 1–4) 3.26 3.25

Both partners expect to marry (%) 59.5 57.8Baby born prior to BSF application (%) 43.9 43.8Either partner has a child from a prior relationship (%) 47.6 46.7Pregnancy intendedness (%)

Intended by both partners 25.0 24.2Wanted by both partners, but considered mistimed 52.8 53.1Unwanted by at least one partner 22.2 22.7

Mental health, attitudes, and religiosityEither partner has psychological distressb (%) 38.0 38.3Both partners agree with the statement “It is better for children if parents are married” (%) 61.2 59.9Attendance at religious services (%)

Both attend more than monthly 24.5 23.6One attends more than monthly 28.5 28.9Neither attends more than monthly 47.0 47.5

Sample size 2,553 2,549

Note: Data are from BSF baseline information forms and BSF eligibility forms. The eight BSF programs are weightedequally for these calculations. TANF = Temporary Assistance for Needy Families; SNAP = Supplemental Nutrition AssistanceProgram.

aThis scale is based on nine items asking respondents to rate on a 4-point scale, which ranged from strongly disagreeto strongly agree, the extent to which their partner (1) shows love and affection, (2) gives encouragement, and (3) listens;(4) respondents’ satisfaction with how the couple resolves conflict; (5) whether the couple enjoys doing things together;respondents’ (6) marriage expectations, (7) confidence in partner’s fidelity, (8) confidence in wanting to be with partner in thefuture, and (9) feeling that the relationship with their partner is the most important thing to them. bPsychological distress wasassessed using the Kessler-6 scale (Kessler et al., 2003).

∗p < .10. ∗∗p < .05.


1. Relationship happiness. This measure isbased on a single survey question in responseto which each member of the couple rated hisor her level of happiness in the relationshipon a 0-to-10 scale, with 10 being thehappiest. This widely used measure was, asexpected, positively correlated with the otherrelationship quality measures we examined.

2. Support and affection. This scale is basedon each partner’s level of agreement with 12statements concerning the relationship, suchas “My partner shows love and affection tome” and “My partner respects me.” The scaleranges from 1 to 4; 4 represents both partnersstrongly agreeing with all 12 statements and 1represents both partners strongly disagreeingwith all of them. The measure has stronginternal consistency (α = .94).

3. Constructive conflict behavior. This scale isbased on eight survey questions concern-ing how frequently couples used specificconstructive behaviors for managing con-flict, such as “Even when arguing, we cankeep a sense of humor” and “My partneris good at calming me when I get upset.”The scale ranges from 1 to 4; 4 correspondsto often exhibiting all the behaviors and 1corresponds to never exhibiting any of thebehaviors (α = .88).

4. Avoidance of destructive conflict behavior.This scale is based on nine survey questionsconcerning how frequently couples engagedin destructive conflict management behav-iors, such as “When we argue, one of uswithdraws and refuses to talk about it any-more” and “When we argue, I feel personallyattacked by my partner.” The scale rangesfrom 1 to 4; 4 corresponds to never exhibitingany of these behaviors and 1 corresponds tooften exhibiting all these behaviors (α = .87).

5. Fidelity. This measure is a binary indicatorthat takes a value of 0 if either member ofthe couple indicated that he or she or his orher partner had been unfaithful during thefollow-up period. Otherwise, it has a valueof 1.

We created these five measures by combiningthe survey responses of both members of the cou-ple into a single couple-level measure. For all themeasures except fidelity, we averaged the valueof the measure for the mother and the father tocreate a composite couple-level measure. Part-ners’ assessments of the quality of their relation-ship were positively correlated; however, reports

could differ substantially, with between-partnercorrelations ranging from .28 to .39 across thefour continuous relationship quality measures.We combined mothers’ and fathers’ reports ofrelationship quality because we conceptualizedthese measures as couple level (rather thanindividual level), with the quality of the couplerelationship properly characterized by the aver-age of the quality assessment of the two partners.To confirm the robustness of our results, as partof a sensitivity analysis we examined impactson mothers’ and fathers’ reports separatelyand found very similar results (Moore, Wood,Clarkwest, Killewald, & Monahan, 2012).

With the exception of the fidelity measure,the relationship quality measures were notdefined for all sample members. Relationshiphappiness and the support-and-affection scalewere available only for the 59% of couplesstill romantically involved at the 36-monthfollow-up. The two conflict behavior scales wereavailable only for the 80% of couples still inregular contact.

Conditioning experimental impact esti-mates on outcomes that occur after randomassignment and that could be influencedby the intervention—such as relationshipbreakup—has the potential to bias results,because the two research groups may no longerbe comparable (McConnell, Stuart, & Devaney,2008). As described by Moore et al. (2012),we assessed the risk of this sort of trun-cation bias using established techniques forassessing acceptable levels of sample attritionin experimental evaluations. This assessmentindicated that analysis of the two conflict behav-ior measures has a low risk of bias. Analysisof the relationship happiness and support-and-affection measures had a moderate risk of biasand should thus be interpreted somewhat morecautiously. To address this concern, we alsoexamined an abbreviated six-item version of thesupport-and-affection scale that could be askedof all couples, including those who were nolonger romantically involved. Impact estimatesusing the full sample and this alternative support-and-affection measure were almost identical toour main results (Moore et al., 2012).

Measures Related to Parenting and FatherInvolvement

Coparenting relationship. The coparentingmeasure examined in the 36-month impact


analysis is a single summary index of 10 itemsdrawn from the Parenting Alliance Inventory(Abidin & Brunner, 1995) and was chosen inconsultation with one of the developers. Theseitems indicate whether respondents think thatthey and their partner communicate well in theircoparenting roles and are a good coparentingteam. Items were asked of all mothers andfathers regardless of whether the couple hadremained romantically involved. Using a5-point scale (ranging from strongly disagreeto strongly agree), respondents were askedto state their level of agreement with the 10positive statements concerning the coparentingrelationship. The scale was created by averagingthe responses to the 10 items. The measure hasa high level of internal consistency (α = .95).The quality of the coparenting relationship usedin the impact analysis was defined in a mannerparallel to couples’ romantic relationship qual-ity, averaging mothers’ and fathers’ responsesto create a couple-level outcome.

Father involvement. Four measures capturedfathers’ contributions of both time and money tochildren’s well-being:

1. Father lives with child. Information fromboth fathers’ and mothers’ surveys was usedto define a measure of father’s residentialstatus at follow-up. The father was consideredto live with the focal child (the child bornaround the time the couple applied for BSFand who made them eligible for the program)only if both parents reported that he did.

2. Father provides substantial financial supportfor child. To measure a father’s financialsupport for children, we relied on mothers’responses to the question “How much of thecost of raising [CHILD] does [FATHER]cover?” The measure is a binary indicator setto 1 if the mother reported that the fathercovered at least half the cost of raising thechild and to 0 otherwise.

3. Father regularly spends time with child.Fathers’ time with their children wasmeasured on the basis of parents’ responsesto a question asking how often the fatherwas in contact with the focal child for anhour or more during the previous month. Themeasure is a binary indicator set equal to 1if both parents reported that the father spentan hour or more with the focal child everyday or almost every day during the previousmonth and to 0 otherwise.

4. Father’s engagement with child. Fatherswere asked to report the frequency oftheir engagement with the focal child in12 activities spanning three domains: (a)caregiving (e.g., feeding or diapering thechild), (b) physical play (e.g., rolling a ball orplaying chasing games with the child), and (c)cognitive and social play (e.g., singing songsor reading stories). Responses were recordedon a 6-point scale ranging from 6 (morethan once a day) to 1 (not at all). Fathers’responses to the 12 items were averaged toform a single scale of father engagement(α = .95).

Measures Related to Child Well-being

Family stability. We examined a binary mea-sure that indicates whether the child had alwayslived with both parents since birth. The measureis based on both parents’ responses.

Economic well-being. We examined three mea-sures in this domain:

1. Poverty status. Poverty status is a binarymeasure indicating whether the monthlyincome of the child’s household—definedas the income of coresidential parentsand their coresidential partners, if any—isbelow the poverty threshold. Respondentswere asked about income from earnings,child support, the Supplemental NutritionAssistance Program (SNAP), TemporaryAssistance for Needy Families (TANF),unemployment insurance, disability benefits,money from friends and relatives, and moneyfrom any other sources. When children livedwith both parents and both parents respondedto the survey, the responses of the two parentswere averaged when calculating parents’ totalincome.

2. Material hardship. As an additional measureof economic distress, we measured materialhardship with a binary measure of whether theresidential parent reported the child’s familyexperienced any of the following: inability topay the full amount of rent or the mortgage,having utilities shut off, or eviction. If thechild lives with both parents, the familyis considered to have experienced materialhardship if either parent reports experiencingany of the three hardships.

3. Reliance on public assistance. Reliance onpublic assistance was measured with a binary


variable that indicated whether the child’sfamily received either SNAP or TANF inthe past month. If the child lives with bothparents, public assistance receipt is indicatedif either residential parent reports receivingSNAP or TANF in the past month.

Socioemotional development. We examinedBSF’s impacts on two measures of children’ssocioemotional development: (a) behavior prob-lems and (b) socioemotional development. Eachof these measures is based on 36-month follow-up survey data collected from the parent whowas the primary caretaker of the child (typicallythe mother).

1. Behavior problems. To assess the prevalenceof problem behaviors among focal children,the BSF 36-month follow-up survey included26 items from the version of the BehaviorProblems Index (Peterson & Zill, 1986;Zill, 1985) included in the Panel Surveyof Income Dynamics, Child DevelopmentSupplement. Parents were asked to reportwhether their children exhibited each of26 problem behaviors often, sometimes, ornever. These items included such behaviorsas lying, losing his or her temper easily,demanding a lot of attention, or crying orworrying too much. A summary behavior-problems score was created by averagingresponses on all 26 items, with higherscores indicating a greater level of behaviorproblems (α = .90).

2. Emotional insecurity. The impact on achild’s emotional insecurity in the presenceof parental conflict was measured using10 developmentally appropriate items fromthe Emotional Reactivity and BehavioralDysregulation subscales of the Security in theMarital Subsystem—Parent Report Inventory(Davies, Forman, Rasi, & Stevens, 2002).The 80% of parents who at the 3-year follow-up were still in regular contact (defined asseeing or talking to each other at least afew times a month) were asked to rate howfrequently various behaviors had occurredduring the past month in response to the childseeing arguments or disagreement betweenthe BSF partners, using one of four responses(often, sometimes, rarely, and never or notapplicable). A summary score was createdby averaging responses on all 10 items,with higher scores indicating a greater level

of emotional insecurity in the presence ofparental conflict (α = .87).

Statistical Models

Regression models can improve the statisticalprecision of impact estimates based on random-assignment research designs and adjust forsmall initial differences between research groupsthat can occur by chance or through surveynonresponse. Therefore, all impact estimatespresented in this article were generated usingregression models that included a large numberof baseline control variables, including measuresof the couple’s initial relationship statusand quality, demographic characteristics, andvarious contextual factors.

These models included data pooled acrossall eight BSF programs. Each regression modelincluded a series of binary variables indicatingeach of the eight BSF programs includedin the study as well as a set of binaryinteraction variables indicating whether thecouple had applied to a given BSF programand had been assigned to the BSF programgroup. The program-specific impact estimatesare the regression coefficients associated withthese interaction variables. The pooled impactestimate for a given outcome was obtainedfrom a mean of the eight program-specificimpact estimates in which each program wasweighted equally. This method answers thepolicy-relevant question: What is the averageeffect of programs that implemented the BSFmodel? See Moore et al. (2012) for moreinformation on these statistical models.

Estimating Impacts on Couples Who AttendedBSF Group Sessions

As described earlier, only 55% of couplesassigned to the BSF program group ever attendeda group relationship skills session, the program’score service. The primary analysis sample usedin this study included all couples who applied forBSF, irrespective of their level of participationin the program. Thus, the impact estimatesrepresent the average effect on all programapplicants of being offered BSF services.Such intent-to-treat (ITT) impact estimates arewidely used in large-scale random-assignmentevaluations for two reasons. First, becauseeveryone randomly assigned is included in theanalysis, one can be confident that the program


and control groups were similar at baselineand that statistically significant differences inoutcomes that emerge can be attributed tothe program. Second, ITT estimates address apolicy-relevant research question: What is theeffect of offering a program in the “real world,”where not everyone in the target population willparticipate in all program services?

Even so, measuring BSF’s impact on coupleswho actually received the program’s core serviceis also of wide interest. Estimating thesetreatment-on-the-treated (TOT) effects poses achallenge because couples who attended groupsessions were systematically different fromthose who did not. For example, these coupleshad higher levels of relationship commitment,were more educated, and were more likely toexpect to marry when they applied for BSF(Wood, Moore, & Clarkwest, 2011). Calculatingan accurate impact estimate requires that onecompare these couples to similar control groupcouples who would have attended had they beenoffered access to BSF.

To identify these couples, we used twopropensity score matching approaches—atraditional approach and a “likely atten-der” approach—following recommendations bySchochet and Burghardt (2007) for TOT anal-ysis within an experimental research design. Inthe traditional approach, program group coupleswho attended sessions are matched to couples inthe control group who appear to have a similarpropensity to attend based on a statistical modelusing their initial characteristics to predict theirlikelihood of attendance. This method shouldgenerate two research groups that are similar intheir observed initial characteristics. It is pos-sible, however, that the groups still differ onunmeasured characteristics, such as their levelof motivation to improve their relationships.The “likely attender” approach uses propensityscores to identify couples in both research groupswho are most likely to attend group sessionsif they are offered to them. Thus, these TOTestimates are based not on couples in the BSFprogram group who actually attended but oncouples whose baseline characteristics indicatethat they would be likely to attend—regardlessof whether they actually attended. This approachensures that the two research groups are similaron both observed and unobserved characteris-tics, because both groups are based entirely oninitial characteristics and not on post-random-assignment behavior. If the propensity score

model cannot accurately predict who actuallyattends, however, and the likely attenders in theprogram group are a substantially different set ofcouples from those who actually attended, thenthe results will not yield an accurate estimate ofthe effects of BSF on those who attended groupsessions.

Although these two approaches differ, thereliability of both depends on the extent to whichthe propensity models identify sample memberswho would choose to attend group sessions ifoffered the opportunity. If the predictive powerof the model is high, then the two approacheswill yield similar results that are likely to reflectBSF’s effects on those who attended groupsessions. Conversely, if the propensity model haslittle predictive power, these approaches tend toyield different results, neither of which is likelyto represent the program’s effects for attenders(Schochet & Burghardt, 2007). Therefore, anexamination of the degree to which resultsfrom these two methods are similar can suggesthow much confidence one can have in theestimates.

For this analysis, we examined impacts on twogroups of participants: (a) the 55% of coupleswho attended at least one group session and (b)the 29% of couples who attended at least half ofthe group sessions. Examining these two groupsallowed us to determine how sensitive BSFimpacts appear to be to the size of the “dose” ofgroup sessions. For brevity, we limited our TOTanalysis to impacts on the couple relationship.Results were similar for other outcomes.

RESULTS

Impacts on the Couple Relationship

BSF did not make couples more likely to staytogether or get married. In fact, BSF coupleswere somewhat less likely to still be together atthe 3-year follow-up, with 57% still romanticallyinvolved at this point compared to 60% of controlgroup couples, a difference that is marginallystatistically significant (see Table 2). Similarly,BSF couples were somewhat less likely thancontrol group couples to live together (marriedor unmarried) at the 3-year follow-up (47%and 50%, respectively), a difference that isalso marginally statistically significant. BSF andcontrol group couples were equally likely to bemarried to one another at the 3-year follow-up, with 21% of each group married at thispoint.


Table 2. Impacts of the Building Strong Families (BSF) Program on Key Outcomes at the 3-Year Follow–up

OutcomeBSF

groupControlgroup

Estimatedimpact p

Effectsize

Relationship statusRomantically involved (%) 57.4 60.5 −3.2∗ .053 −0.079Living together (married or unmarried) (%) 46.9 49.5 −2.6∗ .100 −0.064Married (%) 20.6 20.9 −0.3 .817 −0.011

Relationship qualityRelationship happiness scale 8.29 8.30 −0.01 .868 −0.008Support and affection scale 3.43 3.43 0.00 .989 0.001Use of constructive conflict behaviors scale 3.22 3.22 −0.01 .770 −0.011Avoidance of destructive conflict behaviors scale 2.75 2.78 −0.03 .130 −0.054Neither member of the couple was unfaithful

since random assignment (%)58.2 59.0 −0.8 .628 −0.020

CoparentingQuality of coparenting relationship scale 4.19 4.21 −0.02 .510 −0.022

Father’s involvement and parenting behaviorFather lives with child (%) 50.1 51.8 −1.7 .308 −0.040Father regularly spends time with child (%) 52.4 56.1 −3.6** .032 −0.089Mother reports that father provides substantial

financial support for raising child (%)62.8 65.6 −2.8∗ .096 −0.072

Engagement with child 4.22 4.26 −0.04 .429 −0.031Family stability

Both parents lived with child since birth (%) 42.3 42.7 −0.4 .810 −0.010Economic well-being

Family’s monthly income below povertythreshold (%)

46.5 46.9 −0.4 .824 −0.010

Family experienced difficulty meeting housingexpenses during past year (%)

44.1 44.0 0.1 .956 0.002

Family receiving TANF or SNAP benefits (%) 66.4 65.4 0.9 .564 0.025Child socioemotional development

Behavior Problems Index 1.38 1.41 −0.02** .040 −0.078Emotional insecurity amid parental conflict 1.40 1.42 −0.02 .430 −0.032

Sample sizeAll couples 2,129 2,118Couples still romantically involved 1,233 1,253Couples still in regular contact 1,713 1,740Mothers 1,997 1,984Fathers 1,719 1,707

Note: Data are from BSF 3-year follow-up surveys. The difference between the BSF and control group means may notequal the estimated impact because of rounding. Only the 59% of couples who were still romantically involved were includedin the analysis of relationship happiness and support and affection. Only the 79% of couples who were still in regular contactwere included in the analysis of conflict management measures. The measure of father provides substantial financial supportwas based on the mother’s report and is defined for couples in which the mother responded to the survey. The measure offather engagement with child was based on the father’s report and is defined for couples in which the father responded tothe survey. The measure of emotional insecurity amid parental conflict is defined for children of couples who were still inregular contact at the 3-year survey. Other measures are defined for all couples for whom at least one partner responded to thefollow-up survey. TANF = Temporary Assistance for Needy Families; SNAP = Supplemental Nutrition Assistance Program.

∗p < .10. **p < .05.


BSF did not improve the quality of couples’relationships (see Table 2). Among thosewho were still romantically involved at the3-year follow-up, BSF and control groupcouples reported being equally happy in theirromantic relationships, with average ratingsof 8.29 and 8.30 respectively on the 0-to-10 relationship happiness scale (SD = 1.41 forBSF couples, 1.38 for controls). Similarly,among those still romantically involved, couplesin both research groups reported identicallevels of supportiveness and affection intheir relationships, with average support-and-affection-scale values of 3.43 for both BSFand control group couples on the 1-to-4 scale(SD = 0.41 for both groups).

Similarly, BSF did not improve couples’ability to manage their conflicts. Among the8 in 10 couples who were still in regularcontact at the 3-year follow-up, the averagescore on the 1-to-4 scale measuring the useof constructive conflict behaviors was 3.22 forboth BSF and control group couples (SD = 0.58for both groups). There was no differencebetween the research groups in the avoidanceof destructive conflict behaviors among coupleswho remained in regular contact; the averagescale scores were 2.75 for BSF couples and 2.78for control group couples on the 1-to-4 scale(SD = 0.61 for BSF couples, 0.64 for controls).BSF also had no effect on how faithful coupleswere to each other. At the time of the 36-monthfollow-up survey, 58% of BSF couples reportedno instances of infidelity by either partner sinceapplying for the program, compared with 59%of control group couples, a difference that is notstatistically significant.

Impacts on Parenting and Father Involvement

BSF and control group couples reported that theircoparenting relationships were of similarly highquality (see Table 2). The average coparentingscale score was 4.19 on the 1-to-5 scale formembers of the BSF group and 4.21 formembers of the control group, a difference thatis not statistically significant (SD = 0.78 for BSFcouples, 0.77 for controls). These average scoresindicate that in both research groups, couplestypically agreed or strongly agreed with the 10positive statements about coparenting includedin the scale.

At the 3-year follow-up, BSF had smallnegative effects on some aspects of father

involvement: Fifty-two percent of BSF fathershad spent an hour or more with the focal childon a daily basis during the previous month,compared to 56% of control group fathers,a statistically significant difference. Similarly,63% of BSF mothers reported that the fathercovered at least half the cost of raising the child,compared to 66% of mothers in the controlgroup, a marginally statistically significantdifference. There was no statistically significantdifference between BSF and control groupfathers in the likelihood that they lived withtheir children 3 years after program application(50% and 52%, respectively).

Although BSF fathers spent somewhat lesstime with their children than fathers in the controlgroup did, BSF had no impact on fathers’ self-reported engagement with their children (seeTable 2). The average score on the fatherengagement scale was 4.22 for BSF fathersand 4.26 for control group fathers on the 1-to-6 scale, a difference that is not statisticallysignificant (SD = 1.25 for BSF couples, 1.23 forcontrols). The average scores indicate that, inboth research groups, fathers reported that theytypically engaged in these activities with theirchildren between a few times a week and oncea day.

Impacts on Child Well-being

BSF did not increase the likelihood that childrenlived with both their biological parents throughage 3. At the time of the 3-year follow-up,42% of BSF children and 43% of children inthe control group had lived with both parentscontinuously since birth (see Table 2). BSFalso had no effect on the economic well-beingof children. At the 3-year follow-up, 47%of children in both research groups lived inpoverty, and 44% of children in both researchgroups lived in a family that had experiencedmaterial hardship during the previous year.In addition, virtually identical percentages ofBSF and control group children were living infamilies that received public assistance (66%and 65%, respectively).

BSF led to modest reductions in behaviorproblems at the 3-year follow-up. BSF parentsreported slightly fewer behavior problems fortheir children than did parents in the controlgroup. The average Behavior Problems Indexscore on the 1-to-3 scale was 1.38 for BSFchildren and 1.41 for children in the control


group (SD = 0.28 for both groups), a statisticallysignificant difference that represents an effectsize of −0.08. A negative impact on thisoutcome represents an effect of the program inthe desired direction.

Why might BSF have improved children’sbehavior if it had no positive effects oncouples’ relationships? One possibility is thatthis effect originates from services offered byBSF programs that did not specifically aim toimprove the couple relationship. Four of theeight BSF programs were also Healthy Familiesprograms. At these four sites, BSF couplesreceived home visits focused on promotingpositive parenting behaviors in addition to therelationship skills education offered in BSFgroup sessions. Control group couples did notreceive these home visits. BSF’s effect onbehavior problems was concentrated in thesefour Healthy Families programs, suggesting thatthe effect may be related to the home visitsoffered in these sites. When impact estimateswere pooled across these four programs, theeffect size on the Behavior Problem Indexwas −0.14, which is statistically significant.In contrast, in the four BSF sites that didnot offer Healthy Families home visits thepooled effect size was −0.02, not statisticallysignificant. The difference in the impact on thisoutcome between Healthy Families sites and

other sites is statistically significant (results notshown).

BSF had no effect on children’s emotionalreactions to parental conflict. Among the 8 in10 couples in the full research sample who werestill in regular contact at the 3-year follow-up,the average score on the composite measure ofthe child’s emotional insecurity amid parentalconflict measured on the 1-to-4 scale was 1.40in the BSF group and 1.42 in the controlgroup (SD = 0.51 for both groups). These levelsindicate that, in both groups, parents who werestill in regular contact typically reported thattheir children never or only rarely responded toparents’ conflicts in these ways.

Impacts for Couples Who Attended GroupSessions

The TOT estimates revealed no strong evidenceof effects of BSF on relationship outcomes(either positive or negative) among coupleswho attended group sessions at all or attendedthem regularly (see Table 3). Moreover, thetwo methods we used for estimating theseTOT effects—the traditional approach and the“likely attender” method—generally yieldedsimilar results, suggesting that these findingsare robust. Among the 55% of couples whoattended at least one group session, there were

Table 3. Sign and Statistical Significance of Impacts on Relationship Outcomes for Couples Attending Building StrongFamilies Group Sessions

Attended at leastone session

Attended at leasthalf of sessions

OutcomeTraditional

methodLikely attender

methodTraditional

methodLikely attender

method

Relationship statusRomantically involved © © © ©

Living together (married or unmarried) © © + ©

Married © © © ©

Relationship qualityRelationship happiness scale © © © ©

Support and affection scale © © © ©

Use of constructive conflict behavior scale © © © ©

Avoidance of destructive conflict scale © © © †

Neither member of the couple was unfaithfulsince random assignment

© © © ©

Note: Data are based on information from BSF 3-year follow-up surveys. No impacts were statistically significant at the.05 level. += statistically significant positive impact at the .10 level; †= statistically significant negative impact at the .10level; ©= no statistically significant impact.


no statistically significant effects on any keyrelationship outcomes using either method. Inaddition, effect sizes for these estimates weresmall (the largest was −0.07) and were as likelyto be negative as positive (results not shown).

When we estimated impacts for couples whoattended at least half of the group sessions, theestimates still showed no impacts of BSF onmost relationship quality and status outcomes.However, there are two marginally statisticallysignificant effects—one negative (on avoidanceof constructive conflict behaviors, using the“likely attender” method) and one positive(on coresidence, using the traditional matchingmethod)—revealing no clear pattern of impacts.Therefore, BSF did not appear to have asubstantial effect on couples’ relationships,even among those who attended group sessionsregularly. See Moore et al. (2012) for a moredetailed discussion of these results.

Separate Impacts for the Eight BSF Programs

By design, the BSF program model gaveorganizations implementing BSF considerableflexibility in organizing and operating theirprograms. The goal was to allow for somedegree of experimentation with service deliverystrategies, because at the time the evaluationbegan little evidence existed concerning howbest to deliver relationship skills educationto unmarried parents. As documented bythe BSF implementation study, this flexibilityled to considerable variation across sites inimplementation, with sites choosing differentrecruitment strategies, different curricula forgroup sessions, and different approaches tothe one-on-one support provided by familycoordinators (Dion et al., 2010). Given thisprogram variation across sites, we thought ituseful to examine BSF’s impacts for the eightprograms separately.

At the 15-month follow-up, results variedacross the eight sites. The BSF programin Oklahoma City had a consistent patternof positive effects on relationship outcomes,whereas the Baltimore program had a numberof negative effects (Wood, McConnell, Moore,Clarkwest, & Hsueh, 2012). At the 3-yearfollow-up, the pattern of impacts across siteschanged substantially. The negative impactsobserved in Baltimore had faded and weregenerally not statistically significant (results notshown). Similarly, most of the positive effects

in Oklahoma City observed at 15 months didnot persist; however, a positive impact onfamily stability had emerged. At the 3-yearfollow-up, 49% of BSF children in OklahomaCity had lived with both their biologicalparents since birth compared to 41% of controlgroup children, a difference that is statisticallysignificant. Although the impacts observed inBaltimore and Oklahoma City faded, numerousnegative impacts emerged in the Florida sitebetween the 15-month and 3-year follow-ups.For example, only 55% of BSF couples inFlorida were still romantically involved after3 years, compared with 67% of control groupcouples. In contrast, at the 15-month follow-upthe Florida BSF program had no statisticallysignificant impacts (either positive or negative)on the key outcomes examined. The otherevaluation sites generally had little or noeffect at either follow-up. Therefore, of theeight evaluation sites, only the program inOklahoma City showed evidence of positiveeffects, although these impacts generally did notpersist at the 3-year follow-up.

Potential Moderators of Program Effects

Examinations of BSF’s average effects mightmask heterogeneity in the program’s effec-tiveness for certain subgroups of couples. Toevaluate this possibility, we identified key sub-groups prior to analysis based on the follow-ing baseline characteristics: relationship quality,multiple-partner fertility, fathers’ earnings, raceand ethnicity, and age. At 3 years, none of thesesubgroups had a strong pattern of effects (resultsnot shown). The absence of strong subgroupfindings at 3 years differs from the pattern foundat the 15-month follow-up, when BSF had pos-itive impacts for African Americans on four ofthe eight primary relationship quality and sta-tus measures (Wood et al., 2010). At the 3-yearfollow-up, BSF did not have a significant impacton any of these eight measures for African Amer-ican couples. See Moore et al. (2012) for moredetails on this analysis.

DISCUSSION

BSF represented a new approach to address-ing the needs of low-income, unmarried parentsand their children. Many new, unmarried par-ents report that they want and expect to marryeach other (Carlson et al., 2005). BSF aimed to


help these parents achieve this goal by offer-ing them services designed to teach relationshipskills. The hope was to improve the quality andstability of couples’ relationships and ultimatelyimprove outcomes for their children. Althoughrelationship skills education had been shown tobe successful in improving relationship qualityamong middle-class and married couples, theapproach had not yet been implemented on alarge scale with low-income, unmarried parents,and its effectiveness with this population had notyet been rigorously tested. The BSF evaluationtook a thorough look at this approach’s potentialfor success, rigorously testing in multiple loca-tions the effects of relationship skills programsthat had been specifically developed for unmar-ried parents and adapted to their particular needs.

Our results suggest that it is challengingto make this approach work with unmarriedparents. At the end of the 3-year follow-up period, BSF had no positive effects oncouples’ relationships or fathers’ involvementwith their children. In fact, the program hadmodest negative effects on some outcomes inthese areas. In addition, BSF did not affect mostaspects of child well-being that we examined,although it did lead to modest improvementsin children’s behavior. As discussed earlier, thelack of positive effects on couple outcomes, aswell as the concentration of this particular effectin BSF sites that also offered Healthy Familieshome visits, suggests that this effect was mostlikely generated not by the group relationshipskills education services but by the additionalparenting services offered in those sites.

What factors may have limited BSF’s suc-cess? A key result from the BSF implementationstudy is that, despite concerted efforts made atthe sites, promoting regular attendance at groupsessions was a major challenge, and many cou-ples never attended these sessions at all (Dionet al., 2008, 2010). Some have suggested thatcouples’ poor attendance at group sessions wasthe central reason for the program’s lack ofpositive impacts; however, even among thosecouples who received a substantial dose of theseservices we found little evidence of effects onrelationship outcomes. These findings suggestthat, although regular attendance at group ses-sions may be a key element of a successfulprogram of this type, it does not guarantee posi-tive impacts. These results also suggest that weneed to consider other explanations for BSF’slimited success.

The BSF results differ from findings fromtwo other recent studies of similar relationshipskills education programs that served low- andmoderate-income married couples. As describedearlier, a study of a relationship skills programfor married military couples, PREP for StrongBonds, found that the program reduced thelikelihood that couples divorced in the 2 yearsafter the program ended (Stanley et al., 2013).In addition, the SHM evaluation, which testedprograms similar to BSF but that served low-income married couples, found a pattern of smallpositive effects on relationship quality but noeffect on marriage stability (Hsueh et al., 2012).

Why might relationship skills education pro-grams have less success with unmarried parentsthan with married couples? One contributingfactor may be the relatively low levels of trustand commitment among unmarried low-incomeparents (Edin & Kefalas, 2005). The behavioralchanges required to improve a couple’s rela-tionship may involve substantial personal effort.Partners who are less committed to a relationshipor distrustful of the commitment of their partnermay be more reluctant to do the hard work rela-tionship improvement may require (Van Langeet al., 1997). Thus, on average, unmarried par-ents may be less likely than married couplesto put newly learned relationship skills to use ifdoing so requires considerable effort on their partand if they are uncertain about their own or theirpartner’s commitment to the relationship. Otherdifferences in the characteristics of married andunmarried parents, such as higher rates of eco-nomic disadvantage among unmarried parentsand the more frequent occurrence of multiple-partner fertility in these families, may also playa role. These additional stresses may make itdifficult for some unmarried parents to focus onputting their newly learned relationship skills touse. Future programs may want to place greateremphasis on directly addressing these stresses.

A noteworthy finding from the BSF eval-uation is the fact that a program that aimed toincrease relationship stability and father involve-ment instead led to small reductions in thelikelihood that couples remained together andthat fathers regularly spent time with theirchildren or provided them with substantial finan-cial support. Perhaps BSF helped some coupleswith particularly negative or hostile relation-ships recognize this fact and break up soonerthan they otherwise would have, an outcomethat may be an appropriate one for these couples.


In addition, qualitative research with BSF cou-ples indicated that the need for fathers to stepup and be more responsible was one of thestrongest messages that couples took from theprogram (Dion et al., 2010). This expectationmay have led some fathers in particularly dis-advantaged circumstances to instead distancethemselves from their partner and children. Forexample, if men do not see themselves as capa-ble of being economically supportive or meetingother expectations of responsible fatherhood,they may reduce engagement with their childrenin order to protect themselves from a sense offailure or to “shield their children from their ownpersonal failing” (Young, 2011, p. 120). Con-sistent with that hypothesis, recent research thatused BSF data to examine negative impacts ofthe Baltimore BSF program at 15 months foundthat BSF fathers at that site were more likely thancontrol group fathers to blame themselves—andespecially their own financial, criminal justice,and substance abuse problems—for a rela-tionship breakup, even though their objectiveoutcomes related to earnings, arrests, and sub-stance use were no worse than those of controlgroup fathers (Clarkwest, Killewald, & Wood,in press). Thus, program messages concerningwhat is involved with being a good father andpartner may have led some men to believe theycould not meet those expectations and to insteadwithdraw from these relationships. Future pro-grams serving unmarried parents should givecareful attention to the messages they conveyto fathers and be sure that goals for good par-enting and partnering are presented to fathers inways that make these goals appear realistic andattainable.

BSF was implemented by eight organizationsfollowing a common program model. Sevenof the programs did not achieve the centralobjective of improving couples’ relationships.The one exception was the Oklahoma Cityprogram, which at the 15-month follow-up hadpositive effects on relationship quality, romanticinvolvement, coparenting, and father involve-ment. These impacts had generally faded by the3-year follow-up. The Oklahoma program did,however, increase the likelihood that childrenlived with both their biological parents until age3. Given that increasing family stability was oneof BSF’s central goals, this result is noteworthy.

As described earlier, organizations were givenflexibility in how they implemented the BSFmodel, and approaches varied across the eight

study sites. Given this fact, as well as themore positive impacts in Oklahoma City, anexamination of the distinctive elements of thissite’s approach is warranted. Oklahoma Citywas the only BSF program to use the “Becom-ing Parents” curriculum, which covered similartopics as the other curricula but placed a greateremphasis on the transition to parenthood. Thecurriculum also prescribed groups twice as largeas those recommended in the other two BSFcurricula and covered the material in less time(30 rather than 42 hours). The Oklahoma Cityprogram offered weekly group sessions in twoformats, 3 or 5 hours long, whereas other BSFprograms typically offered only 2-hour weeklysessions (Dion et al., 2010). This difference,combined with use of a shorter curriculum,allowed BSF couples in the Oklahoma Cityprogram to complete the group sessions in 6or 10 weeks, whereas couples in other programsneeded 5 months to finish. In addition, the Okla-homa City program offered more financial incen-tives to encourage group attendance than otherprograms did (Dion et al., 2010). These factorsmay have played a role in the Oklahoma Citysite’s greater success at getting couples to com-plete the program. Finally, although only unmar-ried parents were eligible for the BSF researchsample, the Oklahoma City program also servedlow-income married parents and included bothmarried and unmarried parents in the same groupsessions. No other BSF program served parentswho were married before their child was con-ceived. The presence of married couples mayhave influenced how the group sessions in Okla-homa City functioned as well as how effectivethey were in improving the outcomes of the cou-ples in the BSF research sample. New programsthat plan to offer relationship skills servicesto unmarried parents may want to examine theapproach used in Oklahoma City so that they canbuild on the program’s success. They should alsoconsider adaptations to increase the likelihoodthat impacts are sustained over the longer term.

The decision to marry can be complex forcouples with limited economic prospects. Qual-itative research suggests that many low-incomecouples want both parents to be in a stableeconomic position before they consider marriage(Cherlin, 2009; Edin & Kefalas, 2005; Edin &Reed, 2005). In addition, recent research on low-income fathers underscores the importance offathers’ perceptions of their economic success intheir ability to be engaged and supportive parents


(Young, 2011). These factors may have limitedthe success of the BSF program model. Recentprogram efforts have placed greater emphasison approaches that offer low-income couplesboth employment and relationship services(Zaveri & Hershey, 2010). ACF is currentlysponsoring the Parents and Children Togetherevaluation, which will examine the effectivenessof programs that offer both employment andrelationship services. Perhaps these integratedapproaches will have greater success inimproving the outcomes of unmarried parents.

REFERENCES

Abidin, R. R., & Brunner, J. F. (1995). Develop-ment of a parenting alliance inventory. Jour-nal of Clinical Child Psychology, 24, 31–40.doi:10.1207/s15374424jccp2401_4

Amato, P. (2005). The impact of family formationchange on the cognitive, social, and emotionalwell-being of the next generation. Future ofChildren, 15, 75–96. doi:10.1353/foc.2005.0012

Carlson, M. J., & Corcoran, M. E. (2001). Familystructure and children’s behavioral and cognitiveoutcomes. Journal of Marriage and Family, 63,779–792. doi:10.1111/j.1741-3737.2001.00779.x

Carlson, M., & Furstenberg, F. F. (2006). The preva-lence and correlates of multipartnered fertilityamong urban U.S. parents. Journal of Marriageand Family, 68, 718–732. doi:10.1111/j.1741-3737.2006.00285.x

Carlson, M., McLanahan, S., England, P., & Devaney,B. (2005). What we know about unmarried parents:Implications for Building Strong Families pro-grams. Retrieved from http://www.mathematica-mpr.com/publications/PDFs/bsfisbr3.pdf

Center for Research on Child Wellbeing. (2007).Parents’ relationship status five years after anon-marital birth. Fragile Families Research BriefNo. 39, Center for Research on Child Wellbeing,Princeton, NJ.

Cherlin, A. (2009). The marriage-go-round: The stateof marriage and the family in America today. NewYork: Knopf.

Clarkwest, A., Killewald, A., & Wood, R. (inpress). Stepping up or stepping back: Low incomeBaltimore parents’ responses to the BuildingStrong Families program. In O. Patterson, & E.Fosse (Eds.), The cultural matrix: New approachesto the problems of Black youth. Cambridge, MA:Harvard University Press.

Cowan, P. A., Cowan, C. P., Pruett, M. A., Pruett,K. D., & Wong, J. J. (2009). Promoting fathers’engagement with children: Preventive interven-tions for low-income families. Journal of Marriageand Family, 71, 663–679. doi:10.1111/j.1741-3737.2009.00625.x

Davies, P. T., Forman, E. M., Rasi, J. A., & Stevens, K.I. (2002). Assessing children’s emotional securityin the interparental relationship. Child Devel-opment, 73, 544–562. doi:10.1111/1467-8624.00423

Dion, R. (2005). Healthy marriage programs:Learning what works. The Future of Children,15, 139–156. doi:10.1353/foc.2005.0016

Dion, R., Avellar, S., & Clary, E. (2010). The BuildingStrong Families Project: Implementation of eightprograms to strengthen unmarried parent families.Washington, DC: Mathematica Policy Research.

Dion, R., Hershey, A., Zaveri, H., Avellar, S.,Strong, D., Silman, T., & Moore, R. (2008).Implementation of the Building Strong FamiliesProgram. Washington, DC: Mathematica PolicyResearch.

Edin, K., England, P., & Linnenberg, K. (2003,September). Love and distrust among unmarriedparents. Paper presented at the National PovertyCenter Conference on Marriage and Family For-mation among Low-Income Couples. Retrievedfrom http://apps.olin.wustl.edu/macarthur/papers/england-LoveDistrust.pdf

Edin, K., & Kefalas, M. (2005). Promises I can keep:Why poor women put motherhood before marriage.Berkeley: University of California Press.

Edin, K., & Reed, J. (2005). Why don’t theyjust get married? Barriers to marriage amongthe disadvantaged. The Future of Children, 15,117–137. doi:10.1353/foc.2005.0017

Florsheim, P., McArthur, L., Hudak, C., Heavin,S., & Burrow-Sanchez, J. (2011). The YoungParenthood Program: Preventing intimate partnerviolence between adolescent mothers and youngfathers. Journal of Couple & Relationship Ther-apy, 10, 117–134. doi:10.1080/15332691.2011.562823

Hawkins, A. J., Blanchard, V. L., Baldwin, S. A.,& Fawcett, E. B. (2008). Does marriage and rela-tionship education work? A meta-analytic study.Journal of Consulting and Clinical Psychology,76, 723–734. doi:10.1037/a0012584

Hawkins, A. J., & Fackrell, T. A. (2010). Doesrelationship and marriage education for lower-income couples work? A meta-analytic studyof emerging research. Journal of Couple &Relationship Therapy, 9, 181–191. doi:10.1080/15332691003694927

Hershey, A. A., Devaney, B., Dion, M. R., &McConnell, S. (2004). Building Strong Families:Guidelines for developing programs. Princeton,NJ: Mathematica Policy Research.

Hsueh, J., Alderson, D. P., Lundquist, E., Michalopou-los, C., Gubits, D., Fein, D., & Knox, V. (2012).The Supporting Healthy Marriage Evaluation:Early impacts on low-income families. New York:MDRC.


Kessler, R. C., Barker, P. R., Colpe, L. J., Epstein, J.F., Gfroerer, J. C., & Hiripi, E. (2003). Screeningfor serious mental illness in the general population.Archives of General Psychiatry, 60, 184–189.

Kim, H. S. (2011). Consequences of parental divorcefor child development. American SociologicalReview, 76, 487–511. doi:10.1177/0003122411407748

Knab, J. T., & McLanahan, S. (2007). Measuringcohabitation: Does how, when, and who youask matter? In S. L. Hofferth & L. M. Casper(Eds.), Handbook of measurement issues in familyresearch (pp. 19–33). Mahwah, NJ: Erlbaum.

Markman, H. J., Renick, M. J., Floyd, F. J., Stanley,S. M., & Clements, M. (1993). Preventing maritaldistress through communication and conflictmanagement training: A 4- and 5-year follow-up.Journal of Consulting and Clinical Psychology,61, 70–77. doi:10.1037/0022-006X.61.1.70

Martin, J. A., Hamilton, B. E., Ventura, S. J.,Osterman, M. J. K., Wilson, E. C., & Mathews,T. J. (2012). Births: Final data for 2010. NationalVital Statistics Reports, 60(1). Hyattsville, MD:National Center for Health Statistics.

McConnell, S., Stuart, E. A., & Devaney, B.(2008). The truncation by death problem. Eval-uation Review, 32, 157–186. doi:10.1177/0193841X07309115

McConnell, S., Wood, R., Devaney, B., Avellar, S.,Dion, R., & Zaveri, H. (2006). Building StrongFamilies: The evaluation design. Princeton, NJ:Mathematica Policy Research.

McLanahan, S., & Sandefur, G. D. (1994). Growingup with a single parent: What hurts, what helps.Cambridge, MA: Harvard University Press.

Moore, Q., Wood, R., Clarkwest, A., Killewald, A.,& Monahan, S. (2012). The long-term effects ofBuilding Strong Families: A relationship skillseducation program for unmarried parents (Tech-nical Supplement). Princeton, NJ: MathematicaPolicy Research.

Peterson, J. L., & Zill, N. (1986). Marital disruption,parent–child relationships, and behavior problemsin children. Journal of Marriage and the Family,48, 295–307. doi:10.2307/352397

Reardon-Anderson, J., Stagner, M., Macomber, J. E.,& Murray, J. (2005). Systematic review of theimpact of marriage and relationship programs.Washington, DC: Administration for Children andFamilies, U.S. Department of Health and HumanServices.

Schochet, P. Z. (2009). An approach for addressingthe multiple testing problem in social policy impactevaluations. Evaluation Review, 33, 539–567.doi:10.1177/0193841X09350590

Schochet, P. Z., & Burghardt, J. (2007). Using propen-sity scoring to estimate program-related subgroupimpacts in experimental program evaluations.Evaluation Review, 31, 95–120. doi:10.1177/0193841X06288736

Sigle-Rushton, W. & McLanahan, S. (2002). Theliving arrangements of new unmarried mothers.Demography, 39, 415–433. doi:10.1353/dem.2002.0032

Stanley, S. M., Rhoades, G. K., Loew, B. A., Allen,E. S., Carter, S., Osborne, L. J., Prentice, D., &Markman, H. J. (2013). Two-year outcome resultsfrom a randomized clinical trial of relationshipeducation with U.S. Army couples. Manuscriptsubmitted for publication.

Van Lange, P. A. M., Rusbult, C. E., Drigotas, S. M.,Arriaga, X. B., Witcher, B. S., & Cox, C. L. (1997).Willingness to sacrifice in close relationships.Journal of Personality and Social Psychology, 72,1373–1395. doi:10.1037/0022-3514.72.6.1373

Wadsworth, M. E., Santiago, C. D., Einhorn, L., Etter,E. M., Rienks, S., & Markman, H. (2011). Pre-liminary efficacy of an intervention to reduce psy-chosocial stress and improve coping in low-incomefamilies. American Journal of Community Psy-chology, 48, 257–271. doi:10.1007/s10464-010-9384-z

Wood, R., McConnell, S., Moore, Q., Clarkwest,A., & Hsueh, J. (2010). Strengthening unmarriedparents’ relationships: The early impacts of Build-ing Strong Families. Princeton, NJ: MathematicaPolicy Research.

Wood, R., McConnell, S., Moore, Q., Clarkwest, A.,& Hsueh, J. (2012). The effects of Building StrongFamilies: A healthy marriage and relationshipskills education program for unmarried parents.Journal of Policy Analysis and Management,31, 228–252. doi:10.1002/pam.21608

Wood, R., Moore, Q., & Clarkwest, A. (2011).BSF’s effects on couples who attended group rela-tionship skills sessions: A special analysis of 15-month data. Princeton, NJ: Mathematica PolicyResearch.

Young, A. (2011). Comment: Reactions from theperspective of culture and low-income fatherhood.Annals of the American Academy of Politicaland Social Science, 635, 117–122. doi:10.1177/0002716210390316

Zaveri, H., & Hershey, A. (2010). Initial implemen-tation of a couple-focused employment program.Washington, DC: Mathematica Policy Research.

Zill, N. (1985). Behavior problem scales developedfrom the 1981 Child Health Supplement to theNational Health Interview Survey. Washington,DC: Child Trends, Inc.

Documents

The Long-Term Effects of Building Strong Families: A …...Long-Term Effects of Building Strong Families 447 when Congress voted to provide $75 million annually for marriage education