Measuring progress toward self-reliance · Measuring progress toward self-reliance 1 May 2002 Visible progress toward self-reliance Even in a time of slowed economic growth, which

Measuring progresstoward self-reliance

Early results of the Self-Reliance Achievement Scale

Minnesota Community

Action Association

Minnesota Department

of Children, Families &

Learning - Office of

Economic Opportunity

Wilder Research

Center

The Self-Reliance Achievement Scale is a joint project of:

MMiinnnneessoottaa CCoommmmuunniittyy AAccttiioonn AAssssoocciiaattiioonn

www.mncaa.org

MMiinnnneessoottaa DDeeppaarrttmmeenntt ooff CChhiillddrreenn,, FFaammiilliieess && LLeeaarrnniinngg

Office of Economic Opportunity

http://cfl.state.mn.us/OEO

WWiillddeerr RReesseeaarrcchh CCeenntteerr

www.wilder.org/research

Funding provided by the U.S. Department of Health and Human Services. May 2002

This summary presents highlights of Improving the Self-Reliance of Low-IncomeMinnesotans: Measurement Strategies and Effectiveness Results, March 2002.

To request additional copies, contact the Minnesota Community Action Association at651-645-7425. For more information about the project, contact Tarryl Clark at the same number, or Connie Greer at the Minnesota Department of Children, Families & Learning - Office of Economic Opportunity, 651-582-8569.

How do you chart the complex journey toward

self-reliance, with its real-world mix of

setbacks and progress?

How do you tell the story of not just one family,

but of many families in many communities, in a

way that is both reliable and meaningful?

Minnesota Community Action Agencies havebeen testing a new way to tell their story – todescribe changes in the lives of low-incomepeople they work with.

Policy-makers and funders look increasinglyfor solid evidence of results. The agenciesthemselves want better information aboutwhich of their efforts are most and leasteffective with which types of participants.

The Self-Reliance Achievement Scale is anew way to measure progress toward self-reliance and stability of low-income people.It is designed to document the progress oflow-income individuals and families in 15essential aspects of economic stability, calleddomains. A yearlong pilot test of the Scaleshows promising results.

Measuring progress toward self-reliance 1 May 2002

Visible progress toward self-relianceEven in a time of slowed economic growth,which tends to create serious setbacks forlow-income people, the Self-RelianceAchievement Scale shows clear evidence ofpositive change in the lives of manyparticipants in the first year of the pilot test.

Preliminary data on 456 participants fromseven different Community Action Agenciesprovides a testimony both to the efforts ofparticipants to improve their own situation,and to the value of services that CommunityAction Agencies provide to low-incomeMinnesotans.

The link between services andprogressIn 11 of the 15 domains of self-reliancecovered on the Self-Reliance AchievementScale, participants who received CommunityAction services showed greater progress thanthose who did not.

In all but one domain (Income Sources), therelationship with the Community ActionAgency had a positive impact on theprogress of participants.

Measuring progresstoward self-relianceEarly results of the Self-RelianceAchievement Scale

The link between agency services andimproved self-reliance was much stronger insome domains than in others. These tended tobe issues that can respond relatively quicklyto intervention. For example:

n 37% of those who received services to helpthem work more hours did increase theirhours of employment by 10 hours or more.

n 43% of those who received services to helpthem gain enough income to pay for foodand shelter were later able to meet thosebasic needs

n 70% of those who received services relatedto being homeless or in temporary housingwere later in market rate or subsidizedrental housing or were homeowners.

n 34% of those who at first could not meettheir daily transportation needs reportedlater that they were meeting those needs.

It will take more time and wider use of theSelf-Reliance Achievement Scale tounderstand the impact of Community Actionservices in domains that typically requiremore complicated or long-term efforts. Forexample, the domains that showed weakerprogress for those who received servicesincluded Child Support Income, EnglishProficiency, Informal Social Support, andEducation. Measurable progress in theseareas can take considerable time, so a longertrack record is needed to examine theeffectiveness of services.

How the scale worksThe Self-Reliance Achievement Scale covers15 essential aspects of economic stability,called domains:


n Hours of employment n Income sources n Child support income n Earned Income Tax Credit n Adequacy of income for food and sheltern Financial credit n Housing stability n Housing affordability n Household health insurance coverage n Child care n English proficiency n Education n Transportation n Informal social supportn Number of jobs (this domain was tested in

the pilot year but has since been dropped,since it proved less useful than the othermeasures)

Each domain has three to six levels of self-reliance. The levels are designed to be veryconcrete and observable, so that they can beused consistently among different locationsand staff members.

For example, for Income Sources the levelsof self-reliance include:

n No incomen Public cash benefits/no earned incomen More than 50% public cash benefits with

some earned incomen More than 50% earned income with some

public cash benefitsn Earned income/no public cash benefits

Agency staff also record any servicesprovided to the participant. This makes itpossible to look for correlations betweenservices and improvements in theparticipant’s situation.

When completed for larger numbers ofparticipants over time, the Self-RelianceAchievement Scale will provide agencieswith a new depth of insight into the impact oftheir work. It may also help communitiesidentify local issues that pose persistentdifficulties for low-income families.

Putting the tool to the testSeven of Minnesota’s 29 Community ActionAgencies began to use the Self-RelianceAchievement Scale in September 2000. In thefirst full year, they completed the Scale atleast twice with 456 participants. These wereparticipants who had ongoing contact with theagency and received more than one type ofservice. Reports from agency staff have beenencouraging. They have found the Scale clearand easy to use. Most completed it in lessthan 15 minutes. They also found the Scale tobe thought-provoking and said it spurred newcommunication and coordination of services.

Twenty-nine Minnesota Community ActionAgencies began to use the Self-RelianceAchievement Scale in 2002.

Services aimed at self-relianceDuring the yearlong pilot test of the Self-Reliance Achievement Scale, participantsreceived an average of eight services.

In each domain, the most services went to theparticipants with the farthest to go to achieveself-reliance. This was particularly striking inthe domains of Adequate Income for Foodand Shelter, Transportation, and Child Care.


One hallmark of local Community Actionprograms is that they not only provide directservices, but also guide people to many otherservices available in the community. This roleof “information and referral” can be as simpleas sharing information, or as intensive aswalking someone through the process ofapplying for and receiving services.

In fact, this information and referral role is sostrong that during the pilot test period, itaccounted for nearly half of all servicesprovided in all 15 domains of the Self-Reliance Achievement Scale.

Who participated in the pilot test?More than 500 participants in the seven testsite communities agreed to help CommunityAction Agencies test the Self-RelianceAchievement Scale.

Of these, 456 had at least two checkpointswhen the Scale was completed, allowing usto look at change in their situation over time.

Looking at some basic demographics of thetest group:n 87% were womenn 87% had at least one child in the

household; 28% had three or more childrenn 59% were the only adult in the householdn 30% were married or in a marriage-like

relationshipn 34% had less than a high school education;

4% had a college degreen 32 was the average age; the majority were

age 26 to 45n 81% were White/Caucasian; 12% were

Black/African American; 7% were ofanother race

n 13% were of Hispanic/Latino origin

EARNED INCOME TAX CREDITn BETTER n NO CHANGE n WORSE

HOUSING STABILITYn BETTER n NO CHANGE n WORSE

CHILD CAREn BETTER n NO CHANGE n WORSE


Increasing levels of self-reliance:n Eligible, no benefitn Eligible, applied for benefitn Eligible, received or receiving benefit

Increasing levels of self-reliance:n Homelessn Emergency shelter, doubled up, or notice

of eviction or foreclosuren Transitional housing (time-limited)n Subsidized rental housingn Market rate rental housingn Home ownership

Increasing levels of self-reliance:n No child care availablen Child care available but inadequaten Child care available and adequate

with subsidyn Child care available and adequate

without subsidy

38

1

26

1

Changes in self-relianceEach chart shows the number of participants whose situation in that domain improved,remained the same, or worsened from their first checkpoint to their last during the pilot year.The domains listed first are those that show the strongest progress for clients who receivedCommunity Action services. On the right side of the page are the categories found in the Self-Reliance Achievement Scale.

18

94

3

75

42

5

5 100

27 22

0

NO SERVICES 1 OR MORE SERVICES

1 OR MORE SERVICESNO SERVICES

NO SERVICES 1 OR MORE SERVICES

NU

MB

ER

OF

PA

RT

ICIP

AN

TS

60

120

NU

MB

ER

OF

PA

RT

ICIP

AN

TS

60

120

NU

MB

ER

OF

PA

RT

ICIP

AN

TS

60

120

2747

ADEQUACY OF INCOME n BETTER n NO CHANGE n WORSE

HOURS OF EMPLOYMENTn BETTER n NO CHANGE n WORSE

TRANSPORTATIONn BETTER n NO CHANGE n WORSE

INCOME SOURCESn BETTER n NO CHANGE n WORSE


Increasing levels of self-reliance:n Unable to meet food AND housing

expenses during the last monthn Able to meet food OR housing expenses

during the last monthn Able to meet BOTH food and housing

expenses during the last month

Increasing levels of self-reliance:n Working less than 15 hours / weekn 15-19 hours / weekn 20-24 hours / weekn 25-29 hours / weekn 30-34 hours / weekn 35-40 hours / week

Increasing levels of self-reliance:n Transportation not adequate to meet

daily needsn Adequate to meet some but not all

daily needsn Adequate to meet daily needs

Increasing levels of self-reliance:n No incomen Public cash benefits, no earned incomen More than 50% public cash benefits

with some earned incomen More than 50% earned income with

some public cash benefitsn Earned income, no public cash benefits

2110

0

114

79

3


NU

MB

ER

OF

PA

RT

ICIP

AN

TS

60

120

28

73

14

107

84

21


NU

MB

ER

OF

PA

RT

ICIP

AN

TS

60

120

1030

1

7786

2


NU

MB

ER

OF

PA

RT

ICIP

AN

TS

60

120

68 67

7

64 58

14


NU

MB

ER

OF

PA

RT

ICIP

AN

TS

60

120

SOCIAL SUPPORT n BETTER n NO CHANGE n WORSE

HEALTH CARE COVERAGEn BETTER n NO CHANGE n WORSE

HOUSING AFFORDABILITYn BETTER n NO CHANGE n WORSE


Increasing levels of self-reliance:n No support from family and friendsn One family member or friend to turn to

in a crisisn Two or more family members or friends

to turn to in a crisis

Increasing levels of self-reliance:n No insurance for any household membersn Public health insurance for some

household membersn Public health insurance for all household

membersn Mix of public and private insurance for

some household membersn Mix of public and private insurance for

all household membersn Private insurance for some household

membersn Private insurance for all household

members

Increasing levels of self-reliance:n Pays more than 50% of household

income for housingn Pays less than 50% but more than

30% of incomen Pays less than 30% of income

14

56

0

2134


NU

MB

ER

OF

PA

RT

ICIP

AN

TS

60

120

14

37

524

36

7


NU

MB

ER

OF

PA

RT

ICIP

AN

TS

60

120

28

99

2

33

76

5


NU

MB

ER

OF

PA

RT

ICIP

AN

TS

60

120

FINANCIAL CREDIT n BETTER n NO CHANGE n WORSE

CHILD SUPPORT INCOMEn BETTER n NO CHANGE n WORSE

ENGLISH PROFICIENCYn BETTER n NO CHANGE n WORSE

EDUCATIONn BETTER n NO CHANGE n WORSE


Increasing levels of self-reliance:n No creditn Poor creditn Restoring credit or beginning to

establish creditn Good credit or restored credit

Increasing levels of self-reliance:n Eligible for child support, no income

benefitn Eligible for child support, partial or

irregular benefitn Eligible for child support, receives full

benefit

Increasing levels of self-reliance:n No English skills or inadequate

English skillsn Limited English skillsn Functional English skills with

some supportn Functional English skills, no support

needed

Increasing levels of self-reliance:n Formal education not adequate to meet

employment needsn Formal education adequate for current

employment but not for work advancement

n Formal education adequate for currentemployment and advancement

15

125

3

36

92

1


NU

MB

ER

OF

PA

RT

ICIP

AN

TS

60

120

14

103

216

50

3


NU

MB

ER

OF

PA

RT

ICIP

AN

TS

60

120

2 5 0 523


NU

MB

ER

OF

PA

RT

ICIP

AN

TS

60

120

10

123

3

31

179

7


NU

MB

ER

OF

PA

RT

ICIP

AN

TS

60

120

At the end of the study, a greater percentage of participants were at the highestlevel of self-reliance.

ENGLISH PROFICIENCY

SOCIAL SUPPORT

EARNED INCOME TAX CREDIT

ADEQUATE INCOME

TRANSPORTATION

NUMBER OF JOBS

INCOME SOURCES

HOUSING AFFORDABILITY

CHILD CARE

CHILD SUPPORT INCOME

HOUSING STABILITY

EDUCATION

FINANCIAL CREDIT

HEALTH INSURANCE

HOURS OF EMPLOYMENT


10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

92%93%

72%77%

45%70%

50%66%

54%62%

40%54%

39%45%

33%42%

20%26%

21%23%

20%23%

22%22%

20%22%

10%10%

5%8%

n FIRST CHECKPOINT n LAST CHECKPOINT

About Minnesota Community Action AgenciesCommunity Action Agencies were established in the 1960s to fight America’s War onPoverty. Each agency is locally based, with a local governing board of directors made upof one-third low-income individuals, one-third government officials, and one-thirdbusiness community members. The agencies design strategies to fight poverty andadvocate for low-income people. In Minnesota, 29 Community Action Agencies and 11tribal governments receive state and federal funds to do this kind of work and operate astatewide network.

Specifically, Community Action Agencies in Minnesota provide children and familyservices, economic self-sufficiency services, housing services, senior services,transportation services, emergency and energy assistance and other programs and services.Find more information at www.mncaa.org.

Collaborators in creating the Self-Reliance Achievement ScaleThe collaborative team that developed the Scale includes staff from local CommunityAction Agencies, the Minnesota Community Action Association, the MinnesotaDepartment of Children, Families & Learning - Office of Economic Opportunity, andWilder Research Center.

The team came together in the mid-1990s in response to a new federal emphasis onResults Oriented Management and Accountability (ROMA) for Community Actionefforts nationwide.

The U.S. Congress directed the Community Action Network to focus on measurement ofprogress and continuous improvement in order to achieve these ultimate goals:

n Low-income people become more self-sufficient.n The conditions in which low-income people live are improved.n Low-income people own a stake in their community.n Those who support and provide services to low-income people work in partnership.n Agencies increase their capacity to achieve results.n Low-income people, especially vulnerable populations, achieve their potential by

strengthening family and other supportive systems.

MEASURING PROGRESS TOWARD SELF-RELIANCE MAY 2002

Documents

Measuring progress toward self-reliance · Measuring progress toward self-reliance 1 May 2002 Visible progress toward self-reliance Even in a time of slowed economic growth, which