Opportunities for Postsecondary Success Project Center for Community Partnerships Department of Occupational Therapy Colorado State University Goal Attainment

Opportunities for Postsecondary Success ProjectCenter for Community Partnerships Department of Occupational TherapyColorado State University

Goal Attainment Scaling

AHEAD 2012 Conference

7/11/12

Cathy Schelly, M.Ed., OTR

Assistant Professor

Director, Center for Community Partnerships

Patti Davies, PhD., OTR

Professor

Research Director

Opportunities for Postsecondary Success (OPS)

Implementation site: Center for Community Partnerships (CCP)

OPS Students:◦ CSU & Front Range Community College

students with Asperger’s/ASD, TBI, ID◦ Transitioning youth with ID from local

school district (partnership with Project Search)

◦ Young adults with ID seeking community employment

Establishing measurable goals with participants: Goal Attainment Scaling

Goal Attainment Scaling

Goal attainment scaling (GAS): an individualized approach for measuring the achievement of goals (King et al., 1999).

Originally developed to assess adults in a community mental health setting, but has since been applied to numerous practice areas, including education, health, and social work (MacKay, Somerville, & Lundie, 1996).

Original goal attainment scaling method

The original GAS scale uses a 5-point scale, ranging from -2 to +2, with zero representing the expected level of performance after intervention.

Levels Kiresuk, Smith, & Cardillo (1994)

- 2 Much less than expected outcome

- 1 Somewhat less than expected outcome

0 Projected level of performance

+1 Somewhat more than expected outcome

+2 Much more than expected outcome

Alternative methodsLevels

King et al. (1999)

- 2 Baseline

- 1 Less than expected

0 Expected level

+1 Greater than expected

+2 Much greater than expectedLevel

sSteenbeek et al. (2007)

+2 A lot more

+1 A little more

0 As expected

-1 Partially achieved

-2 Same as baseline

-3 Worse

Levels

1 No change

2 Small improvement

3 Interest in goal

4 Takes some action towards goal

5 Maintains new behavior

6 Committed to achieving goal

7 Achieves goal

OPS Scale

Numerical Level of Goal Attainment

Descriptive Level of Goal Attainment

I lost ground.0

Where I am right now. (Baseline)1

I’m getting close to my goal. 2

I reached my goal!3

I did even better than expected!4

OPS Goal Attainment Scaling

GOAL: Implement effective study strategies.Numerical Level of Goal

AttainmentDescriptive Level of Goal Attainment

I lost ground.0

Not willing or interested in discussing/considering effective study strategies.

Where I am right now.1

Willing to discuss/consider potential effective study strategies but not implementing any strategies.


Implementing effective study strategies with most difficult course, with cueing and reminders.

I reached my goal!3

Implementing effective study strategies with all courses, with cueing and reminders.


Independently implementing effective study strategies with all courses.

Applying GAS in practice

1. Identify the overall objective.2. Identify the specific problem area(s) that

the individual wants/needs to address.3. Identify the behaviors or events that will

indicate improvement in each area.4. Determine the methodology to collect the

desired information.5. Identify outcomes for scaling.6. Determine the individual’s current status

(baseline). 7. Determine how progress will be

documented.

What is the overall goal?

Setting your expected level of outcome or the desired goal:◦For example, you and the student may

agree that developing self-advocacy skills is the overall program goal.

◦Document the student’s current status (baseline).

◦Establish incremental changes in behavior from baseline to depict improvements or ‘losing ground’ as student pursues goal.

What are the specific problem areas?

Prioritize areas of concern and formulate SMART goals.

Prioritized goals are given a weight, which is used to convert the scale score into a standard score.

What are SMART goals?

Specific= a specific goal has a much greater chance of being achieved. It is the who, what, where, when, which, and why of the goal.

Measurable=establish concrete criteria for measuring progress.

Attainable=help client’ set goals that are meaningful to them and that they want to achieve.

Relevant=a goal must represent an objective toward which you are both willing and able to work.

Time-specific=a goal should be grounded within a time-frame.

Tips for scaling goals…

Select the expected level of performance.

Identify the least favorable outcome and the most favorable outcome.

Identify the intermediate levels of performance.

Set SMART goals.Develop scale with equally interval

levels.

Important considerations

Where is my client’s current level of performance (baseline)?

When/how often should I follow-up to determine and document client’s change in status?

What behaviors and events will indicate improvement?

Create a scale that will be useful for measuring changes in performance.

For the student working on self-advocacy skills, this may involve creating a scale based on the number of times the student attends his/her professor’s office hours.

For student who is wanting to make friends in the residence hall (RH), this may involve creating a scale based on the number of RH activities he participates in.

GAS Goal: Recognize and address issues that

result in lack of persistence spring semester. Numerical Level of Goal Attainment


I lost ground.0

Withdraws from all courses spring semester with no warning, no support.


Planning on withdrawing from challenging courses spring semester – seeking support but unwilling to discuss issues that are causing urge to withdraw.


Willing to discuss issues that cause the urge to withdraw from challenging courses.

I reached my goal!3

Recognizes and begins addressing issues that are causing urge to withdraw from challenging courses.


Proactively addresses issues that cause urge to withdraw from challenging courses and persists with all courses.

GAS Goal: Implement study strategies. Numerical Level of Goal Attainment


I lost ground.0

Not willing to develop study strategies.


Not participating in Academic Advancement Center (AAC) to learn study strategies and not using any study strategies, but willing to consider development of strategies.


Attending AAC study strategy sessions to learn about and develop study strategies.

I reached my goal!3

Fully implementing study strategies with OPS support during study sessions.


Independently implementing study strategies.

GAS Goal: Self-advocacy for learning accommodation

needs. Numerical Level of Goal Attainment


I lost ground.0

Not willing to speak with professor.


States a willingness to speak with professor regarding accommodation needs, but afraid to follow through.


Made list of learning accommodation needs in preparation for meeting with with professor(s).

I reached my goal!3

With mentor assistance, set up appointment(s) with professor and kept appointment(s), advocating for learning accommodation needs and with mentor attending meeting(s).


Independently set up appointment(s) with professors and kept appointments, advocating for learning accommodation needs with behind the scenes encouragement from mentor.

GAS Goal: Completes new tasks when

assigned jobs are finished. Numerical Level of Goal Attainment


I lost ground.0

Leaves work area when assigned jobs are finished.


Sits down and stops working when assigned jobs are finished, even when additional tasks are not completed.


Asks job coach what to do next when assigned jobs are finished and works on additional tasks as directed.

I reached my goal!3

Asks supervisor and/or natural support staff if there are additional tasks to be done and then works on these tasks as directed.


Recognizes additional tasks after assigned jobs are finished and independently completes these tasks.

Process we useDevelop relationship with student.Connect student with mentor.Gather information re: needs, wants of

student, and areas where there are challenges (assessment process).

Develop semester goal with student – review with team to ensure that goal is SMART.

Enter goal into GAS database.Begin goal attainment activities,

documenting incremental changes.At end of semester, regroup and establish

new goal(s) for next semester.

CCP Assessment ProcessFunctional, community-

referenced process◦Identify strengths, challenges,

barriers in all domains◦Identify support needs◦Formulate student/participant driven

goal(s)

Challenges/Support NeedsEvaluation Observed Indicator (actual

behavior, action or circumstance)

Fatigue (physical and cognitive)

Student describes inability to stay focused and awake in afternoon classes. Student complains that fatigue is unrelenting, even after 8 hours of sleep each night. Student states that mornings are best for being alert and focused.

Confusion (short-term memory challenges)

Student spends unusually long periods of time studying for exams and completing assignments, but complains of inability to remember content even after reading numerous times.

Assessment process is ongoing

• Assessment is the critical starting point that paves the way to successful planning, educational mentoring, and provision of necessary supports that facilitate success.

• Functional assessment continues after the student is working towards goals to continually guide planning, solve problems, improve academic performance and, ultimately, persistence, retention and graduation.

• Same guidelines apply for students pursuing employment/careers.

Canadian Occupational Performance Measure (COPM)Under consideration, to complement

GAS.Assesses individual’s perceived

occupational performance in life areas.Process:

◦Conduct semi-structured interview to identify wants/needs of client.

◦Based on wants/needs, create goals.◦Supplement interview information through

direct observation, environmental assessment, or standardized tests.

Benefits of COPMMeasures client’s perceived progress in

goal attainment between pre-COPM, intervention and post-COPM (designed as outcome measure).

Easy to administer ~20 minutes.Demonstrated reliability and validity.Similar to GAS, useful with a wide

variety of clients in many different settings.

Can weight and prioritize goals (again, consistent with GAS).

Audience DiscussionWhat type of assessment

process(es) are you using?◦What are the advantages of

your assessment process?◦What are the disadvantages, if

any, of your assessment process?

What population do you serve?

Practice session on writing goalsReview GAS worksheets.Develop goals as per worksheet,

OR, for student you are working with.◦Connect goals to domain areas.◦Develop SMART goals.

Discussion◦Actual goals that we established with

our students/participants

How will data be collected?

What process will your group implement to collect and analyze data?

How will information be collected and who will collect it? Role of the Transition Coordinator Role of the Mentee Role of the Mentor

Our GAS ResearchExciting results from pilot study fall

2011 and spring 2012Noted progress in following areas:

◦Academics◦Self-advocacy◦Socialization◦Persistence◦Retention◦Graduation◦Employment

Preliminary DataParticipants - total 23

◦12 students at CSU◦5 students at FRCC◦6 students enrolled in Project Search

Number of goals written – total 43◦15 in Fall semester 2011 ◦28 in Spring semester 2012

Students enrolled in Project Search6 students

Goals written for a 3 month internship

8 goals written ◦ 1.3 goals on average per student

83% of the students made steady improvement on their goals during their internships

16% of the students displayed a decline in their GAS score at the end of the internship

100% of students were hired after completing their internships

Students in College (FRCC) and University (CSU)

17 students

Goals written for a semester

35 goals written ◦1.8 goals on average per student

47% of the students made steady improvement on their goals across the semester

24% of the students displayed a decline in their GAS score at the end of the semester

Changes in Goal Attainment Scores Before and After Intervention

T Score

MeanStandard Deviation

Pre Intervention

38.12 2.23

Post Intervention

58.90 7.92

Changes in Goal Attainment Scores Before and After Intervention

ANOVA

Statistic: F(1,20) = 136.17, p < .0005

Effect Size:Partial η² = .87

Converting Goal Attainment Scales Scores to T ScoresSum the GAS score for the goal(s)

◦If a participant only has one goal the sum will be the score for that one goal.

◦If a participant has more than one goal the score for each goal will be added together.

Convert the summed score to a T Score◦Formula in Kiresuk & Sherman (1968)

More precision

◦Tables in Kiresuk, Smith, & Cardillo (1994) Less precision

Advantages of Using T ScoresHelps to equate scores across

participants when they have different number of goals.

Provides variability to pre-intervention scores which is necessary for conducting statistical analyses.

Goals can be Weighted To reflect

◦Relative value of each goal based on intervention outcomes

◦Relative importance of each goal based on participants’ perspectives

◦Relative to the difficulty of the goalWeights do not need to sum to a

particular fixed totalExamples

◦1, 2, 3 or 2, 4, 6 or 10, 20, 30Equal weighting of goals lose little

information

Goal Attainment Scaling as an Intervention Tool

Weighting and selecting goals based on client’s desires

Setting goals in collaboration with client

Using improvement on GAS scores as motivation for the client to work hard

Can be used in combination with the COPM ◦Determine goal areas◦Weight goals

Goal Attainment Scaling as an Research Tool

The evaluation and establishment of goals may be conducted by someone different than the person providing the intervention.

Follow-up assessment on goals may be conducted by someone other than the person providing the intervention.

Consider comparing intervention programs or using a control group.

Consider using other evaluation tools in addition to GAS.

Goal: University GraduationSuccess!

Goal: College Graduation

Success!

Advantages of using GAS with the OPS project

Provides a quantitative measure of outcome.Can be used to compare a student’s

progress over time.Can be used to compare performances

across clients in the same program but who have different skills, challenges (Ottenbacher & Cusick, 1989).

Allows for a collaborative approach-working with the student to set realistic goals.

Provides flexibility for measuring diverse outcomes (Brown, 2009).

Limitations of GASScaling can be a time-consuming

process.Biases can occur in goal setting,

scaling, and rating.Temptation to modify goals

throughout the course of intervention.

NOTE: Good goals are really hard to write!!

References King, G.A., McDougal, J., Palisano, R.J., Gritzan, J., & Tucker, M.A. (1999). Goal

attainment scaling: its use in evaluating pediatric therapy programs. Physical and Occupational Therapy in Pediatrics, p. 31-52.

Kiresuk, T. & Sherman, R. (1968). Goal attainment scaling: A general method of evaluating comprehensive mental health programs. Community Mental Health Journal, 4 (6), 443-453.

Kiresuk, T.J., Smith, A., & Cardillo, J.E. (1994). Goal Attainment Scaling: Application, Theory, & Measurement. Hilldale, N.J.: Lawrence Erlbaum Associates.

Mailloux, Z., May-Benson, T.A., Summers, C.A., Miller, L.J., Brett-Green, B., Burke, J.P., et al. (2007). The Issue Is-Goal Attainment Scaling as a measure of meaningful outcomes for children with sensory integration disorders. American Journal of Occupational Therapy, 61, 254-259.

McKay, G., Somerville, W., & Lundie, J. (1996). Reflections on goal attainment scaling (GAS): cautionary notes and proposals for development. Education Research, 38,2, 161-172.

Ottenbacher, K.J. & Cusick, A. (1990) Goal attainment scaling as a method of clinical service evaluation. American Journal of Occupational Therapy, 44, 6, 519-525.

Steenbeek, D., Ketelaar, M., Galama, K., & Gorter, J.W. (2007). Goal attainment scaling in pediatric rehabilitation: a critical review of the literature. Dev Med Child Neurol, 49, 550-556.

Turner-Stokes, L. & Williams, H. (2010). Goal attainment scaling: a direct comparison of alternative rating methods. Journal of Clinical Rehabilitation, 24, 66-73.

Documents

Opportunities for Postsecondary Success Project Center for Community Partnerships Department of Occupational Therapy Colorado State University Goal Attainment