Olmsted-County_outcomes_6-08.ppt

7/27/2019 Olmsted-County_outcomes_6-08.ppt

http://slidepdf.com/reader/full/olmsted-countyoutcomes6-08ppt 1/42

Appendix A.

The Incredible Years:

Parent, Child and Teacher Training Series



The Incredible Years

Award winning ―exemplary best practices‖ program

with demonstrated effectiveness. [Office of Juvenile Justice

and Delinquency Prevention, Substance Abuse and Mental Health

Services Administration]

Comprehensive—targets home, school and child.

Designed to prevent, reduce and treat conductproblems and to increase children‘s social

competence.

Parent skills training uses group discussion,

videotape modeling, and homework exercises .

Child series (Dina Dinosaur) uses puppets and

videotapes to teach social skills, conflict

management, and successful classroom behavior.



Research indicates that children‟s emotional, social and

behavioral adjustment is as important for school success ascognitive and academic preparation.

Child program promotes children‘s cognitive problem-

solving strategies, increases use of pro-social skillsand reduces conduct problems at home and school.

Parent program helps strengthen parenting skills,reduces parenting stress and increases parent‘sinvolvement in school activities.

Teacher program strengthens classroommanagement skills, reduces classroom aggressionand improves teachers‘ ability to focus on student‘ssocial, emotional and academic competence.

The Incredible Years Outcomes



CMHRC commitment to excellence, demonstrated

through:

Training and certification of facilitators, dedicated FTcoordinator for Incredible Years programming.

Purchase of all required curriculum and programmaterials.

Use of treatment fidelity groups, peer review,videotape feedback.

Continued use of ongoing training and technicalsupport from the Incredible Years developers

Ongoing evaluation of parent and child groups.

Incredible Years Adherence to Model Fidelity



1. Implement a comprehensive model that employsevidence-based multiple strategies for providing

supportive environments and that accommodates

ethnic, cultural and economic diversity.

2. Expand the target population to universal andearlier prevention—reaching children with certain

family characteristics that put them at particular risk

for developing conduct problems, delinquency and

substance abuse.

3. Strengthen partnerships with local agencies that

serve at-risk populations and, with the community at

large, promoting a holistic view of health that includes

physical and mental well-being.

CMHRC Goals for Incredible Years



The Incredible Years and Ben

Franklin School Partnership sponsored through a two-year grant from

the Rochester Area Foundation First Steps Initiative

Implements the comprehensive model-teacher, child

and parent.

First year, Franklin Kindergarten teachers and para-professionals receive extensive training and

mentoring from certified Incredible Years trainers.

Second year, Franklin Kindergarten teachers

implement independently with guidance and supportfrom CMHRC IY staff.

Second year, Franklin 1st Grade teachers will receive

comprehensive training and mentoring from CMHRC

IY staff (like Kindergarten teachers this year).



Why offer Dina Dinosaur in kindergarten classes atBen Franklin school?

Social and emotional competencies are importantpredictors of school readiness and academic

success. Prevalence of aggressive behavior in early

childhood 10% and may be as high as 25% for socio-economically deprived children.

Dina program cost effectively reaches high number of at-risk youth early in life when behavior is mostmalleable and before severe behavior problemsarise.

Incredible YearsDina Dinosaur Prevention



Goals for program at Ben Franklin [Kindergarten]:

Prevent early behavior problems fromescalating.

Integrate children into peer group –classroomas ―family‖.

Improve social competence for entire class,provide common vocabulary and problem

solving steps to use in everyday conflict. Magnify dosage of IY intervention through

teacher reinforcement of key conceptsthroughout the school day.

Incredible YearsDina Dinosaur Prevention



Implementation at Ben Franklin duringSchool Year 2007-2008:

Offered 2 times a 6-day cycle in 5kindergarten classrooms

Lessons over entire school year, seven unitsin sequential order [rules/behavior, feelings,problem solving, anger management, peer relations etc.]

Led by Children‘s Mental Health IncredibleYears staff with Ben Franklin teachersparticipating in planning and delivery of lessons

Incredible YearsDina Dinosaur Prevention - Children



Incredible YearsTeacher Training at Ben Franklin

Teacher training at Ben Franklin during School Year 2007-2008 included:

Five full day interactive classroom managementworkshops offered throughout the school year, and 3

days of training from Seattle IY trainer on DinaDinosaur curriculum.

Sessions on promoting pro-social behavior andschool readiness, reducing non-cooperation andaggressive behaviors, and ways to collaborate with

parents to support school involvement and school-home consistency.

Teachers implementing a classroom action plan withongoing support from CMH IY staff.



Incredible YearsParent Training at Ben FranklinParent training at Ben Franklin during School Year 2007-2008 included:

A 12-week parent group with two CMH IYfacilitators meeting once a week. Child care, supper and transportation offered to program participants.

Used a collaborative process to promote positiveparent-child relationships through play, praise andattention.

Worked on effective limit-setting, use of incentives,

building social competence and handlingmisbehavior.

All Kindergarten parents received detailed handoutson the Dina Dinosaur curriculum throughout the year

and received tips on how to use the Dina techniquesat home.



Evaluation of Incredible Years at

Ben Franklin

Evaluation on all three components of the model, datacollection includes:

Classroom Environment Scale [pre/post]

Social Competence Scale on Student Behavior [pre/post]

Teacher Classroom Strategies [pre/post]

Teacher Workshop Satisfaction Surveys

Parent Dina Dinosaur Satisfaction Surveys

Parent Stress Index [parent group, pre/post]

Eyberg Child Behavior Inventory [parent group, pre/post]

Parent Group Satisfaction Surveys



Classroom

Environment Scale

Teachers filled out the scale in October 2007 and again

in May 2008. Instructions were to:

Think about the general atmosphere of your classroom

during the last month. Use the scale below where „10‟ stands for always, „0‟ stands for never and „5‟ stands for

half the time.

0.…1 .... 2….3.…4….5.…6….7.…8.…9….10 Rarely Sometimes Almost always



Classroom Environment Scale

October 2007 versus May 2008

8.9 8.8 8.6 8.8

5.6 5.5

2.3

4.2

Q1. Follows RulesQ2. Work Cooperatively

Q3. Problem Solve Conflic tsQ4. M anage Anger

0

2

4

6

8

10

Rarely

Some-times

Almost Always

Teachers rate their classrooms on scale of 1 through 10 for behaviors exhibited in past month (N=8 Classrooms)

Q1. Students follow

classroom rules

Q2. Students work

cooperatively (helpingsharing, teamwork).

Q3. Students attempt

to problem solve

when conflicts arise(talk about problem,

discuss feelings,suggest solutions).

Q4. Students

manage anger (uses

words not fists, takes

deep breaths, remains

calm).

p≤.002 p≤.001 p≤.000 p≤001 Statistically

significant =

Source: Classroom Environment Scale database



Rarely

Some-times

Almost Always

Teachers rate their classrooms on scale of 1 through 10 for behaviors exhibited in past month (N=8 Classrooms)

Q5. Students

express feelings

appropriately

(positive or negative)

Q6. Students handle

transitions well.

Q7. Students stay on

task during classactivities.

Q8. Students show

awareness and

concern for individual

differences

8.99.1

8.9 8.9

4.6

5.4 5.6 5.8

Q5. Express feelings Q6. Handle transitions Q7. On Task Q8. Show concern

0

2

4

6

8

10

p≤.000 p≤.002 p≤.003 p≤000 Statistically

significant =






How confident are you

in managing currentbehavior problems inyour classroom?

How confident are youin managing futurebehavior problems inyour classroom?

Confident7

88%

Somewhat Confident1

13%

October

Confident2

25%

Very Confident6

75%

May

Teachers rate their confidence on six point scale: very unconfident,unconfident, somewhat unconfident, somewhat confident,

confident, very confident (N=8 Classrooms)

Confident6

75%

Somewhat Confident2

25%

October

Confident2

25%

Very Confident6

75%

May






Average TOTAL s core on Classroom Environm ent

Scale by Program (maximum =80 poin ts)

7073.5

69.7 70.1

44.749.5

26.7

39.1

AM classes

PM classes

Extended Day ALL classes0

10

20

30

40

50

60

70

80



By Franklin Program (AM, PM, Extended Day)




Social Competence

Scale – Teacher*Teachers rate individual student behavior on 25 questions

using scale: ‗not at all‘, ‗a little‘, ‗moderately well‘, ‗well‘ ‗very

well‘ .

Ratings occurred in October 2007 and again in May 2008.

Questions look at:

Emotional regulation

Problem solving skills

Attentiveness and initiation

N= 89 students

(17 students moved before post data was collected and are not included in the

analysis.)

* Tool used by Incredible Years Programs – Denver Colorado School District

S i l C t S l T h



Social Competence Scale Teacher

Emotional Regulation

73 75

80

7073

7983

7673

1511

21

1511

17

25

15 17

2. Accepts things

3. Copes w/ failure

6. Accepts l imits

7. Expresses needs

8. Thinks before acting

11. Calms down

12. Wait patiently

13. Understands other's feelings

14. Effect behavior others

0

20

40

60

80

100

Source: Social Competence Scale Teacher database

N=89 students

Percentage who do “well” or “very well”


S i l C t S l T h



Social Competence Scale Teacher

Social Skills/Peer Interaction

66

73

7982 81 80

7672

8

1620

2619

2518

14

9. Resolves peer problems

15. Works well group

16. Plays by rules

19. Shares materials

20. Cooperates peers

22. Helpful others

23. Listens others

24. Not bossy

0

20

40

60

80

100


N=89 students

Percentage who do “well” or “very well”






Social Competence Total Score by Franklin Program

(AM, PM, Extended Day)


Average TOTAL score by Program (maximum =125 points)

98

112.9

99.3103.9

59.8

78.6

52.2

64.1

AM classesPM classes


20

40

60

80

100

120

All statistically significant at p≤000

N=89 students




Social Competence Total Score by Ethnicity



104.1

93.397.7

108.8 108.5

65.861.5

51.5

6370.2

CaucasianAfrican American

African Nativ eHispanic

Asian0

20

40

60

80

100

120

All statistically significant at p≤01 Source: Social Competence Scale Teacher database

N=42

N=10N=6

N=20

N=8

1 Native American

pre 75, post 125



Social Competence Total Score by Gender / Ethnicity/

Disability: October 2007 versus May 2008


99.7

110.3104.1 104.4

55.5

107.8

59.6

70.265.8 62.7

34.3

66.2

MaleFemale Caucasian

Minority Dev . DisabilityNo Disability

0

20

40

60

80

100

120


N=6

N=83

N=51

N=38 N=42

N=47



Social Competence Total Score by Teacher



82.3

112.9

105.6

114.1

58.463.5

50.4

95.7

Frana Ilstrup Moskalik Schlake0

20

40

60

80

100

120


N=18 N=20 N=33 N=18



Social Competence Total Score by Gender / Ethnicity/

Disability: October 2007 versus May 2008


99.7

110.3104.1 104.4

55.5

107.8

59.6

70.265.8 62.7

34.3

66.2

MaleFemale Caucasian

Minority Dev . DisabilityNo Disability

0

20

40

60

80

100

120


N=6

N=83

N=51

N=38 N=42

N=47



Social Competence Total Score by Ethnicity



104.1

93.397.7

108.8 108.5

65.861.5

51.5

6370.2

CaucasianAfrican American

African Nativ eHispanic

Asian0

20

40

60

80

100

120


N=42

N=10N=6

N=20

N=8

1 Native American

pre 75, post 125



Social Competence Total Score by Franklin Program

(AM, PM, Extended Day)



98

112.9

99.3103.9

59.8

78.6

52.2

64.1

AM classesPM classes


20

40

60

80

100

120

All statistically significant at p≤000

N=89 students




Social Competence Total Score by Teacher



82.3

112.9

105.6

114.1

58.463.5

50.4

95.7

Frana Ilstrup Moskalik Schlake0

20

40

60

80

100

120


N=18 N=20 N=33 N=18

S S *



Teacher Strategies Scale*

Teachers rate how often they use various teaching strategies

on 34 questions using scale: ‗rarely/never‘, ‗sometimes‘, ‗half

the time‘, ‗often‘ ‗very often‘ .

Ratings occurred in October 2007 and again in May 2008.

Questions divided into five scales**:

Positive Approaches with Parents

Praise and Incentives

Proactive Strategies

Limit-Setting Strategies

Inappropriate Strategies (lower score better)

N= 5 teachers, parent scale =4 teachers

*Tool used by Incredible Years Programs – Denver Colorado School District

** Incredible Years in process of completing psychometric analyses, currently

using these scales



Average total score on each subscale, n=5 (parent n=4)33.2

26.227.7

16.4

11.8

29.7

18.4

22.4

18.4 18

Positive ParentPraise/Incentives

Proactive StrategiesLimit-Setting Innapropriate Strategies

0

5

10

15

20

25

30

35

Source: Teacher Strategies database

Teacher Strategies Scale

Fall versus Spring

Increase in proactive strategies and decrease in inappropriate strategies

significant at p≤05, increase in praise marginally significant p≤06

T h T i i S ti f ti



Teacher Training: Satisfaction

with Workshops 1-5

Very helpful8 5

86%

Helpful1 3

13%

Neutral1

1%

Con t e n t Total: 99

Very helpful5 0

58%

Helpful3 3

38%

Neutral3

3%

V i d eo s Total: 86

Very helpful6 7

68%

Helpful2 9

29%

Neutral3

3%

Ro le P l ays

Total: 99

There were no

“not helpful” answers

Source: Teacher workshop satisfaction database

Ben Franklin Staff Only

Teacher Training Satisfaction




with Workshops 1-5

Very helpful9 6

97%

Helpful3

3%

Leader's Teaching Total: 99

Very helpful9 0

91%

Helpful9

9%

Group Discussion Total: 99

There were no “neutral” or “not helpful” answers

Source: Teacher

workshop

satisfactiondatabase


T h T i i S ti f ti




by Workshop

Average scoreScale: 1=not helpful, 2=neutral, 3=helpful, 4=very helpful

WORK

SHOPCONTENT VIDEOS LEADER’S

TEACHING

GROUP

DISCUSSION

ROLE

PLAYS

1

N=18

3.9 3.4 3.9 3.9 3.8

2

N=18

3.7 3.3 3.9 3.9 3.5

3

N=19

3.8 3.2 4.0 3.9 3.6

4

N=22

4.0 3.8 4.0 3.9 3.7

5

N=22

3.9 3.9 4.0 4.0 3.7

Source: Teacher workshop satisfaction database


P t F db k S



Parent Feedback Survey

Dinosaur School

Parents were asked to read a description of the Dinosaur School (puppets named Dina and Wally and the lessons being taught)

They were then asked to think about the last three months and rate

the following questions on a 4- point scale “very often”, “somewhat

often”, “once in awhile”, “not at all”.

1. How often does your child talk about the things he or she is

learning in Dinosaur School?

2. How often have you noticed your child doing the following:

a. Talks about his/her feelings

b. Identifies / recognizes when there is a problem

c. Tries to solve problems when they come up (uses solutions/ideas

from Dinosaur School)

d. Shares or helps friends/siblings



Tools Used to Measure Outcome:

1. Pre and Post Parenting Stress Index

[PSI] Paper/pencil assessment parent fills out to assess parent-

child relationship (three subscales and total stress score)

2. Pre and Pos t Eyberg Chi ld Behavior Inventory [ECBI] Paper/pencil behavioral rating scale

parent fills out to measure their perception of child conduct problems

and the extent to which it is a problem for the parent.

Parent BASIC GroupWeekly parent group using Incredible Years curriculum to

strengthen parenting skills and reduce stress.

Nine parents completed [3 fathers, 4 mothers, 1 stepparent,

1 grandparent]



Significant Drop in Percentage of Parents at a ―Clinical Level‖ of Parental Stress (TSS*)

Average stress level for the Ben Franklin parent group [beginning

and end] is less than for the parent “treatment groups” held at the

CMHRC.

Average Entry Score –

81.2 (SD=8)

Average Exit Score -

47.7 (SD=13)

t{5}=6.5, significant at

p<.001

0

3

0

1

2

3

4

5

6

NumberatClinicalLevel

Pre Post

*Total Stress Score (TSS) is an indication of the stress level experienced within therole as parent.

Clinical level is at or

above the 90 th

percentileon PSI scale.

N=6 with pre and post PSI*

Parenting Stress Index

Pre to Post Program

Source: PSI / ECBI Database

Eyberg Child Behavior Inventory



Significant Improvement in Behavior Intensity andChild Behavior Problems (as perceived by parents)

INTENSITY

Average Entry Score - 132

(SD=19)

Average Exit Score - 89 (SD=8)

t{8}=8, significant at p<.001

PROBLEM

Average Entry Score – 12 (SD=10)

Average Exit Score - 1 (SD=3)

T{8}=3 , significant at p<.01

0 0

3

4

Intensity Problem0

1

2

3

4

5

NumberatClin

icalLevel

Pre Post

Eyberg Child Behavior Inventory

[ECBI] Pre to Post Program

N=9 with pre and post ECBI

Average ECBI levels for the Ben Franklin parent group [beginning

and end] is lower than for the parent “treatment groups” held at the

CMHRC.

Source: PSI / ECBI Database



Parent Satisfaction with BASIC IY

Parent Program, n=9 parents

44% 33%100%

56%

100%

67%

Original problem

Us e of methods

Child's progress

Other family issues

0

10

20

30

40

50

6070

80

90

100

Percentage

helped sl ightly helped greatly helped1. Problem(s) that

originally prompted

me to take this

program for my

child….

2. Use of methods to

try and change

child‘s problems…

3. My feelings about

my child‘s

progress…

4. Has helped with

other

personal/family

problems…

Source: Tally sheet for parent BASIC satisfaction surveys



Parent Satisfaction with BASIC

Parent Program


Of 9 respondents: All would ―strongly recommend‖ the program to a

friend or relative.

All felt ―positive‖ [3] or ―very positive‖ [6] about

achieving their goal in the program for child and family.

All were ―confident‖ [5] or ―very confident‖ [4] they

could manage current behavior problems.

All were ―optimistic‖ [1] or ―very optimistic‖ [8] aboutgood results from the program.

All felt the parenting approach used to change child

behavior was ―appropriate‖ [1] or ―greatly appropriate‖

[8].

P S i f i i h BASIC




Parent Program


33% 33% 22% 22%

67% 67%

78% 78%

Teac hing 1

Teaching 2 Preparation 1

Preparat ion 2

0

10

20

30

40

5060

70

80

90

10 0

Percenta

ge

Sl igh tly Above Average High Superior

Two facilitators:

1. I feel the leader‘s teaching was…

2. The leader‘s preparation was…

N=9 responses

All participants felt ―very

supported‖ [6] or ―supported‖

[3] by the group.

Eight of nine programparticipants would like to keep

meeting as a group. (The one

participant who did not want to keep

meeting was a grandparent).

Seven felt it was ―likely‖ or

―somewhat likely‖ they would

continue meeting. The other

two were ―neutral‖

Response to Facilitators Response to Group

Parent Satisfaction ith BASIC




Parent Program

What did you see as the main benefit of the IncredibleYears program?

I have become so much better with my children in all the ways that

a father should be. I feel this will help me stay that #1 dad that I‟ve

always wanted. Thank you very much.

Positive talk was the most beneficial for me. Also the skills taught

– how and when to ignore, using calming down techniques.

How easy it is to make small changes and have such wonderful

results. When the program is broken down the way it was,

everything makes so much sense. Learning skills to improve communication, bringing parents into

the children‟s setting really helped my kids. They have been so

proud and excited to have me here every Monday.

Consistency and reinforcing positive/expected behaviors.

Documents

Olmsted-County_outcomes_6-08.ppt