AgrAbility NTW McGill QOL Minneapolis, MN April 9, 2013 11:15-12:00

AgrAbility NTW McGill QOLMinneapolis, MN

April 9, 201311:15-12:00

By Robert J. Fetsch, Extension Specialist & Professor EmeritusDirector, Colorado AgrAbility Project, Human Development & Family Studies

Colorado State University& Robert Aherin,

Professor & Illinois AgrAbility Program DirectorDepartment of Agricultural & Biological Engineering

University of Illinois Urbana ChampaignAANTWMcGillQOL4.0913 (Rev. 3.2813)

How Effective Are 9 SRAPs at Increasing Their Clients’ Quality of Life & Independent Living and Operating Levels?

By Robert J. Fetsch (CSU), Robert Aherin (UIL),Sheila Simmons (KU),

Vicki Janisch (UW), Vincent Luke (CSU/Goodwill Denver),

Rick Peterson (TAMU),Toby Woodson (UAR),

Kirk Ballin (ESVA),Inetta Fluharty (WVU),

Sharry Nielsen (UN),& Diana Sargent (OSU)

Our AgrAbility Mission

“The AgrAbility Mission is to enhance and protect quality of life and preserve livelihoods. It’s about supporting and promoting growth and independence. Ultimately it’s about hope.”

Source: National AgrAbility Project. (2011). It’s about hope [DVD]. Author: Purdue University.

NAPEC Produced Results• Published two refereed journal articles and

submitted another.– Christen, C. T., & Fetsch, R. J. (2008). Colorado AgrAbility:

Enhancing the effectiveness of outreach efforts targeting farmers and ranchers with disabilities. Journal of Applied Communication, 92(1&2), 1-12.

– Jackman, D. M., Fetsch, R. J., & Collins, C. L. (2013). Quality of life and independent living and operating levels of farmers and ranchers with disabilities. Manuscript submitted for publication.

– Meyer, R. H., & Fetsch, R. J. (2006). National AgrAbility Project impact on farmers and ranchers with disabilities. Journal of Agricultural Safety and Health, 12(4), 275-291.

Brief Review of the Literature on the MQOL• Meyer and Fetsch (2006) reported the impacts of

AgrAbility on 618 clients from 8 states.• Jackman, Fetsch, and Collins (2013) reported that

the pre-survey QOL levels of 313 farmers and ranchers with disabilities were statistically significantly lower than those of other samples (Cohen, et al., 1997; Fetzer, 2010).

Source: Meyer, R. H., & Fetsch, R. J. (2006). National AgrAbility Project impact on farmers and ranchers with disabilities. Journal of Agricultural Safety and Health, 12(4), 275-291.

Jackman, D. M., Fetsch, R. J., & Collins, C. L. (2013). Quality of life and independent living and operating levels of farmers and ranchers with disabilities. Manuscript submitted for publication.

Brief Review of the Literature on the MQOL• Today’s report is on the third multi-state

AgrAbility study of matched pre- and post-survey differences—this time with 137 farmers and ranchers with disabilities.

• Last time our NAP E.C. presented results to you like this was via Webinar on 11/28/12 when we had 98 matched pre-post surveys.

• Our 15 SRAPs continued to work hard to collect 39 new matched pre-post cases (N=98+39=137).

History of National AgrAbility Evaluation Committee

• Early 2006—Kathryn Pereira, Evaluation Specialist NAP U of WI, invited all SRAP’s to join in an AgrAbility evaluation study.

• The National AgrAbility Evaluation Committee (NAEC) met approximately bi-monthly (2007-Present) via teleconference/face-to-face (N = 6-25 participants/meeting).

How many SRAPs are collecting QOL and ILOS data from their new clients?

History of National AgrAbility Evaluation Committee

• As of 2/28/13, 15/24 SRAPs (62.5%) are on the NAP Evaluation Committee (AR, CO, KS, ME, MN, MO, NC, NE, OH, OK, TX, UT, VA, WI, & WV).

Please join us!

Control SRAPs?

Currently unfunded

History of National AgrAbility Evaluation Committee

• As of 2/28/13 15/24 SRAPs (62.5%) are on the NAP Evaluation Committee.

• 10 SRAPs have provide pre-survey McGill data (AR, CO, KS, NC, OH, OK, TX, VA, WI, & WV).

• 9 SRAPs provided matched pre- and post-survey data.

• 2 have IRB approval (ME & NC).• 4 are seeking IRB approval (MN, MO, OH, & UT).

History of National AgrAbility Evaluation Committee

• Who is an AgrAbility Client? An AgrAbility client is an individual with a disability engaged in production agriculture as an owner/operator, family member, or employee who has received professional services from AgrAbility project staff during an on-site visit.

History of National AgrAbility Evaluation Committee

• 5 Questions:– Do our AgrAbility clients increase their QOL?– Are our AgrAbility clients more able to live on, operate,

and manage their farms/ranches if they choose?– Are our group mean scores the same as those from the

population group’s mean scores?– Is the McGill QOL Survey sensitive to the effects of

AgrAbility information, education, & service?– Who else will join us?

History of National AgrAbility Evaluation Committee

• Nine SRAP’s conducted a 5.67-year study to answer the first 4 questions (June 2007-February 28, 2013).

• McGill QOL—AR, CO, KS, NE, OK, TX, VA, WI & WV

Measures Used in 9-State Study• McGill Quality of Life Survey & AgrAbility

Independent Living & Operating Survey (ILOS)

• NAP Demographic Data

Protocol• Procedure—9 SRAPs mailed each new

client the Pre-Survey, a cover letter, McGill Pre-Survey, and a stamped, self-addressed envelope with an invitation to complete and return it.

Protocol• Participants were given the choice of

completing the survey themselves or of having the items read aloud by the AgrAbility team member. No one was coerced to complete and return their surveys.

History of National AgrAbility Evaluation Committee

• By February 28, 2013 AR, CO, KS, NE, OK, TX, VA, WI, & WV entered their 137 matched pre-post-survey data into Excel files and e-mailed them to CO for entering and analyzing.

– KS 60– WI 34– CO 24– TX 9– AR 3– VA 3– WV 2– NE 1– OK 1 – Total 137

Who Were the Participants in the Study? (N = 137)

• All 137 clients completed matched pre-post surveys.

• 92 (67%) were male; 35 (26%) were female. 10 did not report gender (7%).

• 87 (64%) were new; 29 (21%) were on-going; six (4%) were re-opened; six (4%) were closed in current grant year, and 9 (7%) were missing.

Who Were the Participants in the Study? (N = 137)

• Ages ranged from 11 to 95 (M = 58.5; SD = 17.5; N = 95).

• For U.S. farmers and ranchers, the average age was 57.1 in 2007.*

• Original disability occurred 1934-2012.*Source: Retrieved April 27, 2007 from http://nass.usda.gov/census/

What Was the Range and Average Length of Time with AgrAbility?

• The amount of time spent with AgrAbility ranged from 1 to 39 months (M = 13.6; SD = 8.2; N = 134).

What Were Clients’ Major Symptoms (N=137)?• Pain 16 12%• Vision Issues 15 11%• Walking 10 7%• Back Pain 8 6%• Mobility 8 6%• None 8 6%• Immobility 7 5%• Tiredness 7 5%• Hearing Issues 6 4%• Bladder Control 5 4%• Memory Issues 5 4%• Being Able to Do Work 4 3%• Foot Pain 4 3%• Sleep Issues 4 3%

What Were Clients’ Primary Disabilities (N=126)?• Arthritis 17 12%• Visual Impairment 15 11%• Back Injury 11 8%• Joint Injury 8 6%• Orthopedic Injury 8 6%• Other 8 6%• Cerebral Vascular Accident Stroke 7 5%• Leg Amp Above Knee 6 4%• Multiple Sclerosis 6 4%• Cardiovascular Disease 6 4%• Spinal Paraplegic 5 4%• Traumatic Brain Injury 5 4%• Hearing Impairment 4 3%

What Were the Purposes of This 9-State Study? (N = 137)

• To determine pre-post service changes in clients’ QOL levels and in their ability to live on, operate, and manage their farms/ranches.

• To determine whether the McGill QOL and the AgrAbility ILOS were sensitive to the effects of AgrAbility.

Q: Do Our AgrAbility Clients Increase Their QOL?A: ?

0

2

4

6

8

10

Pre-Survey Post-Survey

McGill SIS (N=129)***

Physical Well Being (N=125)***

Physical Symptoms (N=120)***

McGill Pre- Post-Survey Changes(Single item Scale, Physical Well-Being, & Physical

Symptoms)

0

2

4

6

8

10

Pre-Survey Post-Survey

Support (N=133)***

Experiential Well Being (N=133)***

Psychological Well-Being (N=132)***

McGill Pre- Post-Survey Changes(Support, Experiential Well Being &

Psychological Well-Being)

0123456789

10

Pre-Survey Post-Survey

MQOL Total Score (N=104)***

McGill Pre- Post-Survey Changes(Total Score)

Q: Do Our AgrAbility Clients Increase Their QOL?A: Yes, they report improvements on the Total QOL Scale plus on all 6 subscales!

Q: Are Our AgrAbility Clients More Able to Live on, Operate, and Manage Their Farms/Ranches if They Choose?A: ?

0

1

2

3

4

5

Pre-Survey Post-Survey

Manage farm (N=128)***

Complete chores (N=131)***

Operate machinery (N=131)***

AgrAbility Independent Living & Operating Survey (ILOS)(Manage Farm, Complete Chores, & Operate Machinery)

0

1

2

3

4

5

Pre-Survey Post-Survey

Live in home on farm/ranch (N=129)**

Access workspaces (N=129)***

Modify machinery (N=127)***

AgrAbility Independent Living & Operating Survey (ILOS)(Live in Home, Access Workspaces & Modify Machinery)

0

5

10

15

20

25

Pre-Survey Post-Survey

ILOS Total Score (N=124)***

AgrAbility ILOS Changes(Total Score)

Q: Are Our AgrAbility Clients More Able to Live on, Operate, and Manage Their Farms/Ranches if They Choose?A: Yes, they report improvements on the Total ILOS Scale plus on all 6 items!

AgrAbility provided me with info/recommendations I used:

Yes No

To do my farm/ranch work better/more easily than before AgrAbility? (N=86)

84% 16%

To continue farming/ranching in part/whole, without help I would not have been able to do so? (N=84)

71% 29%

To continue to live in my home independently? (N=84)

45% 55%

To continue to live on the farm/ranch, but successfully take up another occupation? (N=79)

5% 95%

AgrAbility did not provide me with help. (N=78)

8% 92%

I am able to: SA/A Neither D/SD NA

Complete chores (N=133) 80% 5% 10% 5%

Operate machinery (N=133) 66% 6% 7% 21%

Manage farm/ranch (N=132) 79% 6% 6% 9%

Access workspaces (N=131) 83% 8% 7% 2%

Live in my home on the farm/ranch (N=133)

92% 3% 2% 2%

Change/modify machinery (N=132)

51% 13% 8% 28%

Receive useful assistance info (N=132)

84% 5% 8% 4%

Follow thru on AgrAbility recommendations (N=130)

85% 8% 7% 1%

The Top Reasons Clients Were Unable to Follow AgrAbility Recommendations (N = 48)

1. Unable to obtain funding (n = 10/46 = 22%) (n = 10/137 = 7%)

2. Health conditions changed (n = 8/48 = 17%)3. My financial situation changed (n = 5/47 = 11%) (n

= 5/137 = 4%)4. Recommendations did not work for me (n = 2/47

= 4%) (n = 2/137 = 1%)5. Chose a different career (n = 1/47 = 2%)

Q: What do these 9 SRAPs do well?A: They have pre-post survey data that show statistically significant increases in:-QOL levels (p < .001)-ILOS levels (p < .01; p < .001)

Q: What do the results say we can improve?A: We can do more as we assist farm and ranch families:-”to live in their homes on the farm/ranch” (p <.01) (M=4.064.43).-”to assist them in obtaining funding.”

Q: Is the McGill QOL Survey Sensitive to the Effects of AgrAbility Information, Education, & Service?A: Yes!

These results look promising,

BUT…

How do we know these results are not due to something other

than our AgrAbility services?

“Good News”• AgrAbility is among the 45 federally

funded programs that supported employment for people with disabilities in fiscal year 2010.

• AgrAbility is among the 10/45 programs with a review or study to evaluate the program’s effectiveness.

Source: U.S. Government Accounting Office. (2012). Employment for people with disabilities; Little is known about the effectiveness of fragmented and overlapping programs (GAO Publication No. 12-677). Washington, DC. (p. i).

“Good News”• “…The Department of Agriculture’s

AgrAbility program conducted a review of its activities between 1991 and 2011 and found that 11,000 clients had been served, and that 88 percent of those clients continued to be engaged in farm or ranch activities.”

Source: U.S. Government Accounting Office. (2012). Employment for people with disabilities; Little is known about the effectiveness of fragmented and overlapping programs (GAO Publication No. 12-677). Washington, DC. (p. 27).

“Bad News”• “However, this study did not

determine whether other factors may have contributed to participants’ positive outcomes.”

• “No impact study.”Source: U.S. Government Accounting Office. (2012). Employment for people with disabilities; Little is known about the

effectiveness of fragmented and overlapping programs (GAO Publication No. 12-677). Washington, DC. (pp. 27, 80).

Brad Rein asked us to help respond.• So far 15 SRAP’s are working to collect

data from AgrAbility clients with an on-site visit (AR, CO, KS, ME, MN, MO, NC, NE, OH, OK, TX, UT, VA, WI, & WV).

• We welcome the rest of you to join us.

Let’s Turn to Bob Aherin for an Update on:

• Where he and Chip are with the Control Group at UIUC and

• How we can assist their efforts with collecting data for the Control Group.

Experimental Group (N = 200 with matching pre- and post-surveys)

Control Group (N = 100 withmatching pre- and post-surveys)

Control Group (N = 100)• Cannot be receiving any type of

AgrAbility program services or onsite visits regardless of whether they are in USDA funded or Affiliate States.

Where do we find 100 ranchers and farmers for the Control Group?

1. Non-funded AgrAbility Affiliate States previously funded, but not currently funded to provide AgrAbility services, e.g. IA, ID, IL, MI, MT, and PA.

2. Farmers and ranchers with disabilities who call NAP’s 1-800-825-4264 for information from non-funded states.

Where do we find 100 ranchers and farmers for the Control Group?

3. Ranchers and farmers with disabilities who call in to currently funded SRAP’s but who decide not to become clients at this time.

4. Wait list group?5. Fingerlake, NY Goodwill?6. Where else?

Please join us!

Control SRAPs?

Currently unfunded

Why Join Us?1. Document your project’s effectiveness at

increasing QOL and ILOS.2. Enhance your chances of receiving funding

next time with empirical evidence of your SRAP’s quality and effectiveness.

3. Increase your chances for outside funding by demonstrating your accountability.

4. Contribute to AgrAbility’s Mission.

Remember Our AgrAbility Mission

“The AgrAbility Mission is to enhance and protect quality of life and preserve livelihoods. It’s about supporting and promoting growth and independence. Ultimately it’s about hope.”

Source: National AgrAbility Project. (2011). It’s about hope [DVD]. Author: Purdue University.

Won’t You Join Us? Here’s how:1. Send an email to

robert.fetsch@colostate.edu.2. Seek IRB approval from your Land-Grant

University.3. Study and use the same protocol.4. Adapt CO to __ on pp. 1-2 & mail.5. Enter your data into an Excel file that we

will provide, proof perfectly & email to me.

Thank you very much!

AgrAbility across the U.S.(source: National AgrAbility Project)

How Reliable Are the Subscales?• A common measure of reliability is

Cronbach’s alpha.• Subscale Pre Post• Physical Symptoms .59 .85• Psychological WB .90 .93• Experiential WB .92 .93• Support .82 .83• MQOL Total .79 .87