2013-14 Formative Evaluation Report

Florida International Academy After-School and Summer Camp Program

21stCentury Community Learning Center

Formative Evaluation Report 2013-14

Project Number: 13J-2444A-4PCC1

February 28, 2014

Table of Contents

Introduction ........................................................................................................................................ 1

Formative Evaluation Report .............................................................................................................. 1

Description of Evaluation Methodology ......................................................................................... 2

Evaluation Design ........................................................................................................................ 2

Data Collection Efforts ................................................................................................................ 2

FIA 21st CCLC Program Staff ......................................................................................................... 7

Activities Conducted ................................................................................................................... 8

Student Attendance.................................................................................................................. 13

Student Demographic Information ........................................................................................... 14

Assessment of Program Objectives ................................................................................................... 15

Site Visits ....................................................................................................................................... 29

Progress toward Sustainability .......................................................................................................... 30

Dissemination of Findings ................................................................................................................. 31

Conclusion ......................................................................................................................................... 32

1 FIA 21st CCLC Program Formative Evaluation Report | 2013-14

Introduction

Florida International Academy (FIA) is a Title I charter school that was established in 1998

as a middle school. To accommodate the growing number of enrollees, in August 2010, FIA

moved from its former location to

its new home in Opa Locka,

Florida, an urban northeast

Miami neighborhood. FIA is now

located on a six-acre campus and

serves over 840 students in

grades kindergarten through

eighth.

The racial and ethnic composition of FIA’s student body comprises Blacks (93%), Hispanics

(6%), and a few students from other races/ethnicities (1%). Most of the students are girls

(61%), qualify for free or reduced lunch (93%), and reside in single-parent homes (72%).

Additionally, approximately 3% of the students have Individualized Education Plans (IEPs)

for special needs, primarily for a learning disability.

FIA attributes much of its success to its 21st Century Community Learning Centers (21st CCLC)

program. Since 2004, FIA has operated a 5-day-a-week, year-round 21st CCLC afterschool

program. During the 2012-13 school year, this program served over 100 high-risk students.

FIA’s 21st CCLC program meets the needs of these students by not only providing them with

a safe place to go during afterschool hours, but also provides them with remedial education

and academic enrichment in reading, language arts, math, and science; homework help;

indoor and outdoor sports and fitness activities; character education and social skills

building; dropout prevention; and project based learning. Moreover, all academic

enrichment activities are provided by certified teachers. This is the final year of the contract

award period for FIA’s 21st CCCL program.

Formative Evaluation Report

The purpose of this formative report is to describe ongoing evaluation efforts that took place

from September 3, 2013 through February 28, 2014. This report will outline preliminary

findings of FIA’s progress towards meeting its goals and objectives for the 2013-14 program

year. Specifically, the report will include:

Description of evaluation design

Activities conducted

An overview of contracted measurable objectives

Results of data analyses

Conclusions and recommendations for continuous quality improvement


Description of Evaluation Methodology

Evaluation Design

Q-Q Research Consultants (QQRC) was contracted by FIA to conduct all evaluation efforts

for their 21stCCLC program. QQRC sought to examine FIA’s 21st CCLC program processes to

evaluate their continuous quality improvement efforts. QQRC also examined how well FIA’s

21stCCLC program served its students between September 3, 2013 and February 28, 2014

by assessing performance on a number of predetermined outcomes. By using both of these

approaches, QQRC was able to provide preliminary findings as well as provide

recommendations that are useful to the program’s internal operations and FIA’s external

stakeholders.

Data Collection Efforts

Data were collected from a variety of sources. Additional data will be collected throughout

program implementation. The table below outlines the measureable objectives and the

data sources used to examine progress towards meeting the goals and the scheduled data

collection intervals.

Measureable Objectives Data Sources

Academic Enrichment

(1) Seventy-five percent (75%) of regularly

participating students (i.e., students who attend

the program 30 days or more) will increase their

reading comprehension skills as evidenced by

scores on standardized reading tests.

Students’ 2013 FCAT reading scores have

been collected for the pretest. Students will

complete the 2014 FCAT, and the scores will

be compiled for the posttest.

(2) Seventy-five (75%) of regularly participating

students will increase their reading fluency skills as

demonstrated by the Oral Reading Fluency (ORF)

scores.

Students completed the ORF pretest

assessment. The posttest will be

administered at the end of the year.


participating students will increase math skills as

evidenced by scores on standardized math tests

and report cards.

Students’ 2013 FCAT math scores have been

collected and compiled for the pretest.

Students will complete the 2014 FCAT, and

the scores will be compiled for the posttest.

Students’ report card grades for 1stquarter

were collected. Report card grades for

subsequent quarters will be collected.




participating students will increase science skills as

evidenced by scores on standardized science tests

and report cards.

Students will complete the 2014 FCAT

science, and the scores will be compiled for

the posttest.

Students’ report card grades for 1st quarter

were collected. Report card grades for

subsequent quarters will be collected.

Personal Enrichment

(5) Ninety percent (90%) of regularly participating

students will improve their fitness levels as

demonstrated by the Progressive Aerobic

Cardiovascular Endurance Run (PACER) Multi-Stage

Shuttle Run.

FIA 21st CCLC staff conducted the PACER

pretest. The posttest will be administered at

the end of the year.

(6) Seventy percent (70%) of regularly participating

students will show increased positive attitudes

toward physical fitness as demonstrated by survey

on attitudes towards physical fitness.

Students completed the pre and midtest

surveys on attitudes towards physical

fitness. The posttest will be administered at


(7) Eighty percent (80%) of regularly participating

students will demonstrate improved conflict

resolution skills, social and resistance skills, and

emotional competency as demonstrated by the

Too Good for Drugs and Violence (TGFD&V) After-

School Youth Survey.

Students completed the TGFD&V pre and

midtests. The posttest will be administered

at the end of the year.

In addition to the TGFD&V survey, students

also completed the Normative Beliefs

about Aggression and Aggressive Behavior

Scale pre and midtests. The posttest will be

administered at the end of the year.

(8) Eighty percent 80% of regularly participating

students will demonstrate improved goal setting,

decision making, and communication as

demonstrated by the TGFD&V After-School Youth

Survey.

Students completed the TGFD&V pre and

midtests. The posttest will be administered

at the end of the year.

(9) Ninety-five percent (95%) of regularly

participating students will have no more than three

unexcused absences in the regular day school as

demonstrated by school attendance records.

Students’ school attendance records for the

1stquarterswere collected. Students’ records

for subsequent quarters will be compiled.



(10) Ninety percent (90%) of regularly participating

students will demonstrate an increased sense of

responsibility for completing homework

assignments as demonstrated by a parent survey

on homework completion.

Parents completed homework pretest

surveys. The posttest will be administered at


Adult Family Member Services

(11) Sixty percent (60%) of parents/guardians of

regular participating students will participate in at

least two Adult Family Member Services activities

as demonstrated by the attendance logs.

Data from attendance logs for parent

workshops will be compiled at the end of

the school year.

(12) Eighty percent (80%) of parents of regularly

participating students will demonstrate a better

understanding of their role in supporting their

child’s education as evidenced by scores on pre

and posttest surveys.

Pre and posttest assessments from parent

workshops will be compiled at the end of

the school year.


Frequency of Formative/Ongoing Evaluation

The formative evaluation for FIA’s 21stCCLC program will entail a continuous data collection

process. For this report, pre and midtest data collected between the start of the program

until January and other program activities completed up until the beginning of February

were used to examine progress towards achieving goals and objectives. Analyses included

descriptive analyses on program and participant characteristics (e.g. mean FCAT reading

scores among students, number of student activities, number of unexcused absences, etc.).

Findings from these analyses will be used to inform FIA’s current programming and aid in

the overall success of the program.

Data Collection, Analysis, and Reporting Timeline

Data were collected in 5 areas: (1) program characteristics and activities (e.g., average daily

attendance, types of activities, etc.), (2) students’ demographic information, (3) students’

academic progress, (4) students’ social development and behavior, and (5) parental

involvement. These data were collected daily (e.g., attendance), monthly (e.g., parental

involvement), quarterly (e.g., report cards), three times a year (i.e., pre, mid, and post

surveys), and bi-annually (i.e., pre and post surveys). Most data are updated weekly and

reviewed monthly to monitor progress. Data on program staff (e.g., staff development

activities, staff turnover, etc.) are also collected and reviewed monthly. Staff members are

kept informed of programs progress during monthly staff meetings. Other stakeholders are

also kept abreast of the FIA 21st CCLC program’s progress.

The following paragraphs provide more detail on data collection intervals for the current

program year.

1. Program Characteristics and Activities

To ensure that each student obtains all grant proposed and approved program activities,

individual students’ attendance records are closely monitored on a quarterly basis for each

program activity to evaluate appropriate dosage of programmatic elements. This program

year, to enhance programming, a field trip will be offered during the school year and two

additional field trips will be offered during the summer. Program activities will also include

inviting various cultural art groups to the FIA 21st CCLC program for interactive programming

with students. Students’ and teachers’ reactions to these activities will be noted in

subsequent reports.

2. Demographic Information

Students’ demographic information is obtained upon students’ admission into the program

and is verified to ensure accuracy. Verification of self-reported information is facilitated

through comparison to information obtained from the FIA school office.


3. Academic Progress (Reading, Mathematics, and Science)

FCAT scores are obtained for each students from the previous year upon acceptance into

the program (with the exception of science).1 The prior year scores are obtained from the

school’s Miami-Dade County Public Schools (MDCPS) database and are then compared to

the present year’s scores. Furthermore students’ academic progression on reading,

mathematics, and science will be obtained and evaluated on a quarterly basis via program’s

approved use of each student’s academic records (quarterly report cards).

Additionally, students’ reading is assessed using District-approved tests (i.e., Oral Reading

Fluency). The assessments are administered and the data are analyzed twice per year via pre

and posttest surveys.

4. Social Development and Behavior

Students’ self-report data is collected on their perceptions of their (a) conflict resolution

skills, social and resistance skills, and emotional competency; (b) goal setting, decision

making, and communication skills; (c) beliefs about aggression and aggressive behaviors;

and (d) attitudes towards physical fitness. Students’ self-report data is collected at the

beginning (pretest), middle (midtest), and end of the program (posttest).

5. Parental Involvement

Participation in program activities and knowledge gained through participation is tracked

on parents and other adult family members of regularly participating students. At each

Adult Family Member Services activity, a sign-in sheet is circulated to capture attendance.

At some of the didactic workshops, surveys are administered to the parents and other adult

family members to capture any knowledge gained through participation in the workshop.

1 Testing for FCAT Science begins in 8th grade; therefore, only 2013-14 FCAT Science scores will be collected.


FIA 21st CCLC Program Staff

The FIA 21st CCLC program has a total of 9 staff. For this contract year, two staff have left

the program. This is the Program Director’s second year with the program, and she hired a

Program Assistant three weeks ago. All instructional staff members are certified teachers.

All, but one, are also employees at FIA’s day school. By assuming dual roles, staff is afforded

great opportunities to build rapport with the students and gain an understanding of each

of the student’s unique needs. Staff dedication to participating students is vividly displayed

in their interactions with the students. In previous program years, participating students

have expressed their satisfaction with having FIA staff serve in the 21st CCLC program. They

appreciate having adults with whom they are familiar and trust.

For the 2013-14 program year, there have been eight professional development workshops

held, to date.

Professional Development Offered Date

Overview of the 21st CCLC for the current program year

August 2013

A review of project-based learning concepts August 2013

Program startup planning meeting and review of the proposed activities for the year

September 2013

Attendance at the FASA conference October 2013

Presentation on infusion of science into project-based activities

November 2013

Presentation on utilizing social studies-related activities to enhance programming

November 2013

Student showcase development and updates on implementing project-based learning activities

December 2013

Developing lesson plans for second semester utilizing project-based learning activities

January 2014


Activities Conducted

The FIA 21st CCLC program operates year-round with an afterschool component operational

during the corresponding school year and a summer component that operates during the

summer which will be June 16, 2014 to July 25, 2014 this program year. This program year,

the FIA 21st CCLC after-school program commenced on September 3, 2013 and from that

day all afterschool curricula have been fully implemented. In addition, the program staff has

developed new processes and tools to enhance program success. For example, some

creative project-based learning activities have been added to the existing program. These

activities not only enhance the program’s academic enrichment component, but are also

being used as a parent engagement strategy wherein students’ projects will be showcased

to the parents at the end of the year.

Summer 2013 Operation

Total number of weeks THIS site

was open:

Typical number of days per

week THIS site was

open:

Typical number of hours per week THIS site was open on

WEEKDAYS

WEEKDAY EVENINGS

WEEKENDS

FIA 21st CCLC Summer Program

6 5 35 0 0

2013-14 Academic Year Operation

Total number

of weeks THIS site

was open:

Total number of days

THIS site was

open.

Typical number of days per

week THIS site was

open.

Typical number of hours per week THIS site was

open

TOTAL number of days THIS site operated

Bef

ore

Sch

oo

l

Du

rin

g Sc

ho

ol

Aft

ersc

ho

ol

Wee

ken

ds/

Ho

liday

s

Bef

ore

Sch

oo

l

Du

rin

g Sc

ho

ol

Aft

ersc

ho

ol

Wee

ken

ds/

Ho

liday

s

FIA 21st CCLC Afterschool Program

20 89 5 0 0 15 0 0 0 89 0


Academic Enrichment

Reading/Language Arts, Math, and Science

The project-based learning activity, The Panda Quarterly has been implemented. Students

have worked to develop a quarterly newsletter. The first edition of the publication included,

poems, riddles, articles on current events, tribute to culture awareness, and an interview

conducted with the program director. Students were tasked with completing all elements

of the publication including generating original material, compiling images, editing, and the

overall layout. The first volume of the Panda Quarterly was disseminated to parents and

community partners who attended the first student showcase of the year. This activity is

incorporated into the regular activities which are conducted for one hour, five days per

week for 36 weeks.

Since the start of the year, students have been engaged in the Math Bowl, a project based

activity that affords students an opportunity to reinforce math skills attained across the

year in a creative and interactive competition. Students are placed in teams comprised of

students with various mathematical abilities, and they are challenged to respond to

problems generated by teacher in a jeopardy-like fashion (i.e., problems are timed and

increase in difficulty for additional points). Quarterly semifinals are hosted prior to student

showcase, and the final event is conducted live at the student showcase. This activity is

incorporated into the regular activities which are conducted for one hour, five days per

week for approximately 36 weeks.

For the Panda Spoken Word Café, students will write spoken-word poetry and scripts for

skits dramatizing issues that are important to them revolving around conflict resolution,

anger management, problem solving, team work, and drug and alcohol abuse based on

discussions from the Too Good for Drugs and Violence curriculum. Students have two

different venues to exhibit their projects from the Panda Spoken Word Café: the Panda

Quarterly newsletter and the student showcases. This activity is incorporated in the regular

activities which are conducted for one half hour, five days per week for 36 weeks.

The Mini Greenhouse project-based activity is scheduled to begin during final quarter of the

year. Students will create a “mini” greenhouse to demonstrate the Greenhouse Effect;

relate the greenhouse to the Earth’s atmosphere; and identify local, national, and global

environmental concerns. They will also investigate the carbon cycle and create a recycling

project for the school. The mini projects created by students during these activities will be

showcased during a special exhibit on Earth Day 2014 (April 22), to which parents and the


FIA community will be invited. In addition to physical models and other exhibits, students

will make oral presentations on their projects. This activity is incorporated in the regular

activities which are conducted for one hour, five days per week for nine weeks.

Additional Learning Tools Utilized

In addition to the project-based learning activities outlined above, FIA 21st CCLC program

staff utilize curricula to reinforce learning. Two curricula are utilized for the reading

component: Scholastic After School Learning Program for Grades 6 to 8 and Time for Kids.

The Scholastic After School Learning Program combines highly motivational reading in the

core content areas of reading/language arts, science, math, social studies and art, combined

with hands-on activities, and audio books. The After School Fun & Learning Center

component provides teachers with access to 200 fiction and non-fiction books to use in

their daily lessons and 20 audio CD books that students can listen to on CD players with

headsets. Students use laptops provided by FIA to access the online After School Fun and

Learning Center that includes news articles arranged by content area, educational games

and puzzles, and a “Homework Hub.”

Time for Kids is used as a reading supplement, and it is a weekly classroom news magazine

designed for kids. These magazines provide a variety of real-world subject matter that aligns

nicely across many curricula. Teachers are provided with a guide, worksheets, mini-lessons,

and a comprehension quiz.

The After-School Math PLUS is used for the math component. In collaboration with the New

York Hall of Science and the St. Louis Science Center, the After-School Math PLUS was

developed by the Educational Equity Center (EEC) at the Academy for Educational

Development. The After-School Math PLUS curriculum addresses mathematics using four

different modalities: jumping rope, music, art, and the built environment (i.e., exploration

of mathematical concepts within their environment, including buildings, streets, etc.).

Students are offered informal math learning activities where they create objects or engage

in physical activities related to mathematical concepts.

The After-School Science PLUS is a science curriculum also developed by EEC. Designed for

afterschool environments, the After-School Science PLUS curriculum develops higher order

thinking skills (i.e. problem-solving, creative thinking, decision-making, and spatial

relations) while providing hands-on activities focused on inquiry-based science and literacy.

Personal Enrichment

Physical Education

To address physical fitness and education, the SPARK program was implemented. The

SPARK program is an evidenced-based program designed to improve the health and fitness

of students through a variety of physical activities. The curriculum includes 24 core activity


units. Instructors are provided with suggestions for managing children in physical education

classes. Instructors are provided with indoor and outdoor activities as well as instructions

for strength and conditioning warm-up. The SPARK curriculum is used for 30 minutes, five

days per week during the school year, and 65 minutes, five days per week during the

summer program.

Drug and Violence Prevention and Dropout Prevention

In an effort to improve social skills, reduce the risk factors associated with school drop-out,

increase refusal skills for engaging in drugs and violence, and improve positive character

traits, the Too Good for Drugs and Violence (TGFD&V) curriculum is being used. TGFD&V,

developed by the Mendoza Foundation in Tampa, Florida, is recognized as an evidence-

based program by the Substance Abuse and Mental Health Administration’s National

Registry of Evidence-based Programs and Practices for its demonstrated impact on

promoting personal and pro-social behaviors and protective factors related to conflict and

violence among at-risk K-12 students. The TGFD&V curriculum is used for 30 minutes, five

days per week during the school year, and 65 minutes, five days per week during the

summer program.

Tutoring and Mentoring

During the afterschool program, staff members provide students with homework help for

40 minutes. Staff members provide students requiring additional homework assistance with

individual tutoring services.

Adult Family Members Services

This year, the FIA 21stCCLC program staff decided to implement quarterly student

showcases to highlight students’ work products and offer parents an opportunity to interact

with other parents and learn more about the program. In addition to the student

showcases, the21stCCLC Program Director, Yvonne Morris has coordinated a variety of

workshops that address topics such as family literacy, parenting skills, drug and alcohol use,

coping with stress during the FCAT, and other topics. Many of these workshops are

facilitated by local experts on the topics. It is expected that following participation in the

workshops, adult family members will be more informed about what their children are

learning in school and the 21stCCLC program and how they can best support their children’s

progress and personal development as adolescents.

Additionally, the FIA 21st CCLC program has an Advisory Board. The first meeting of the year

was hosted in October 2013. Four parents, three students, and four program staff were in

attendance, and they discussed ways to enhance community partnerships. One of the

outcomes from the meeting was a linkage from a parent for a field trip for the school year.

Another outcome was an Advisory Board member’s connection with a local Juvenile Court


Judge, whom the Program Director invited to address the students and staff.

The following table offers a summary of these activities, as well as the program’s present

operational status.

Status of Approved Activities

Activities Status

Panda Quarterly Fully implemented

Math Bowl Fully implemented

Panda Spoken Word Cafe Fully implemented

Mini Greenhouse Will be implemented in 4th quarter

Supplemental academic curricula Fully implemented

Tutoring and homework help sessions Fully implemented

Too Good for Drugs and Violence curricula Fully implemented

Outdoor and indoor sports activities using the SPARK After School curriculum

Fully implemented

Monthly Adult Family Members services Fully implemented


Student Attendance

To date, there have been 155 regularly attending middle school students in the FIA 21st CCLC

afterschool program. The targeted number to enroll for the 2013-14 contract year is 90, so

FIA 21st CCLC program staff has exceeded their enrollment target. Additionally, the

program’s average daily attendance (ADA) percentages have been exemplary and much

improved over last program year. Program staff is to be commended for implementing

proactive and creative strategies to maintain consistent program participation this year. The

ADA for the months of September to January are shown in the table below.

Average Daily Attendance

September October November December January

87% 83% 83% 88% 89%

Student Enrollment: Total and Regularly Participating Students Academic Year 2013-14

Total Enrolled Attending (at least one day)

Regularly Participating Enrollment (30 days or more)

Only Summer

2013

Fall 2013 Only

Summer and Fall

2013

Total Only Summer

2013

Fall 2013 Only

Summer and Fall

2013

Total

72 127 43 242 0 56 99 155 Note: Unduplicated counts shown.


Student Demographic Information

The FIA 21st CCLC program is currently serving students in the 6th (45 students), 7th (43

students), and 8th grades (47 students).The majority of students are males (n =69) and

identify as Black (n = 128). Ninety percent of the students receive free lunch. None of the

students were identified as homeless. With respect to ESE status, there were five students

who were identified. Thirty-three students were English Language Learners. See below for

more information on student demographics.

Student Demographics

33%

32%

35%

Grade

Sixth Seventh Eighth

51%49%

Sex

Male Female

95%

3%

1% 1%

Race/Ethnicity

Black Hispanic

White non-Hispanic Native American

90%

5%5%

Lunch Status

Free Reduced Paid


Assessment of Program Objectives

The FIA 21st CCLC program has four measureable objectives that target academic

enrichment for participating students in the areas of reading, math, and science. Pretest

surveys were administered in October and midtest surveys were administered in January.

Descriptive analyses were employed for objectives where a mid or post was not available

at the time of the report. Inferential analyses, specifically, paired samples t-tests and

ANOVAs, were used to evaluate changes from pre to post and differences among students

in different grades.

Objective 1: Seventy-five percent (75%) of regularly participating students will

increase their reading comprehension skills as evidenced by scores on standardized

reading tests and report cards.

FCAT reading scores for the 2012-13 school year have been recorded for participating

students. Some of the students (math: n=19; reading: n=10) had missing FCAT reading

scores because either (1) the students did not take the FCAT during the 2012-13 school

year due to their private school enrollment or (2) staff were not able to access scores. Pre

data indicates that 66% of the students scored below grade level (level 2 or lower) on the

FCAT reading during the 2012-13 school year. FCAT reading data by grade is shown in the

table below. The majority of the students across all grades were not on grade level

according to exam results. Scores for the 2013 FCAT test will be compared to the 2014

scores for the Summative Evaluation report.

69%

58%

70%

31%

43%

30%

0% 20% 40% 60% 80% 100%

Eighth

Seventh

Sixth

Reading FCAT 2012-13

Level 1 or 2 Level 3 or higher

AC

AD

EM

EIC

EN

RIC

HM

EN

T


Students’ reading report card grades indicate that overall, 93% of the students

demonstrated a “C” average or above on their 1st quarter report card. Comparisons by grade

show that a lesser percentage of sixth graders are receiving average or better grades in

reading compared to students in the other grades. First quarter report card grades will be

compared to the grades received on report cards collected at the end of the school year.

Objective 2: Seventy-five percent (75%) of regularly participating students will increase

their reading fluency skills as demonstrated by Oral Reading Fluency (ORF) test scores.

The Oral Reading Fluency (ORF) test was administered to students and pretest scores were

recorded. According to the results of research studies for establishing the norms of the ORF

test, test results should be greater than or equal to 137 for all grades. Based on the average

scores shown below, none of the grades were on target at pretest. Pretest scores will be

compared to posttest scores at the end of the year.

Average ORF Scores by Grade

Sixth Grade Seventh Grade Eighth Grade

108 (33.58) 118 (21.90) 135 (38.68)

Note: Standard deviations noted in parentheses.

100%

97%

84%

75% 80% 85% 90% 95% 100%

Eighth

Seventh

Sixth

"C" or Better in Reading

7%

54%32%

4% 3%0%

20%

40%

60%

A B C D F

Overall Reading Grades


Students are placed in risk classes according to their ORF test results. Overall, the majority

of students in all grades scored within the moderate to low risk range; however, a third of

the sixth graders demonstrated high risk.

24%

15%

33%

28%

50%

38%

48%

35%

29%

0% 20% 40% 60% 80% 100%

Eighth

Seventh

Sixth

Oral Reading Fluency Levels

High Risk Medium Risk Low Risk



their math skills as evidenced by scores on standardized math tests and report cards.

FCAT math scores for the 2012-13 school year have been recorded for participating students.

Pre data indicates that 55% of the students scored below grade level (level 2 or lower) on

the FCAT math portion during the 2012 -13 school year. FCAT math data by grade is shown

in the table below. While academic performance on the math portion of the FCAT was

better than reading, there are still significant numbers of youth in all grades who are not on

grade level as evidenced by last year’s FCAT math results.

Students’ math report card grades indicate that overall, 97% of the students demonstrated

a “C” average or above on their 1stquarter report card. Comparisons by grade show that a

lesser percentage of sixth graders are receiving average or better grades in math compared

to students in the other grades.

49%

41%

75%

51%

59%

25%

0% 20% 40% 60% 80% 100%

Eighth

Seventh

Sixth

Math FCAT 2012-13

Level 1 or 2 Level 3 or higher

100%

100%

93%

88% 90% 92% 94% 96% 98% 100%

Eighth

Seventh

Sixth

"C" or Better in Math

9%38%

50%

2% 1%0%

20%

40%

60%

A B C D F

Overall Math Grades



their science skills as evidenced by scores on standardized science tests and report

cards.

FCAT science scores for the 2012-13 school year will be collected at the end of the school

year to evaluate current level of peformance in science. Students’ science report card

grades indicate that overall, 93% of the students demonstrated a “C” average or above on

their 1stquarter report card. Comparisons by grade show that a lesser percentage of seventh

graders are receiving average or better grades in science compared to students in the other

grades.

98%

87%

91%

80% 85% 90% 95% 100%

Eighth

Seventh

Sixth

"C" or Better in Science

3%

37%52%

3% 4%0%

20%

40%

60%

A B C D F

Overall Science Grades


The FIA 21st CCLC program has six measureable objectives that target social enrichment for

participating students in the areas of physical education and recreational activities,

tutoring, drug and violence prevention, and dropout prevention and character education.

Objective 5: Ninety percent (90%) of regularly participating students will improve their

fitness levels as demonstrated by the Progressive Aerobic Cardiovascular Endurance

Run (PACER) Multi-Stage Shuttle Run.

The SPARK Physical Education curriculum is being utilized for the physical fitness

component of the program. The Progressive Aerobic Cardiovascular Endurance Run (PACER)

Multi-Stage Shuttle Run pretest was administered to participating students. Pretest data

shows that the average number of laps successfully completed by students was 9.13. A

situps/curl ups fitness test was also administered. On average, students completed 29 situps

in a minute for the pretest.The graphs below show the average scores by grade. Post-test

data will be collected at the end of the school year, and changes from pre to posttest will

be evaluated.

Objective 6: Seventy percent (70%) of regularly participating students will show

increased positive attitudes toward physical fitness as demonstrated by survey on

attitudes towards physical fitness.

Participating students were administered a physical fitness survey that queried their

perceptions of physical activity and fitness. To evaluate progress toward the respective

performance objective, pre and midtest data were examined to ascertain students’

attitudes toward physical fitness, on average. Scores on the survey range from one to five

with lower scores indicating more positive attitudes toward physical activity and fitness.

9.79 7.96 9.23

0

5

10

15

Eighth Seventh Sixth

Shuttle Run Laps Completed

26.6 27.6 32.7

0

20

40


Situps

PE

RSO

NA

L E

NR

ICH

ME

NT


There are two graphs below. The first shows the percentage of students who improved or

maintained their self-reported attitudes (if attitudes already favorable) toward physical

activity from pre to mid. The second graph shows the average scores on the physical fitness

survey by grade. As shown in the graph, on average, participating students demonstrated

fairly favorable attitudes toward physical activity and fitness at the pre and midtest surveys.

Note: A little fewer than half (44%) of the regularly participating students completed the fitness midtest.

52%48%

Improvement on Attitudes toward Physical Activity

Improved or maintained Did not improved

1.83

1.61

1.711.67

1.81

1.65

1.5

1.6

1.7

1.8

1.9


Physical Fitness Scores

Pre Mid


Objective 7: Eighty percent (80%) of regularly participating students will demonstrate

improved conflict resolution skills, social and resistance skills, and emotional

competency as demonstrated by the Too Good for Drugs and Violence (TGFD&V) After-

School Youth Survey.

The Too Good for Drugs and Violence (TGFD&V) curriculum is being used for the FIA 21st

CCLC program. To evaluate regularly participating students’ improvement in the areas of

conflict resolution, social and resistance skills, and emotional competency skills, the

TGFD&V After-School Youth Survey was administered at pre and mid. Scores range from

one to five, with higher scores suggesting more positive perceptions, attitudes, and skills

with scores around 2.5 deemed as average. Overall, pre and mid data indicate that

participating students scored above average. As can be seen from the graph, scores were

fairly similar by grade and from pre to mid. There were no significant differences among the

various grades. Additionally, while some of the scores were slightly higher on the pre than

the mid, there were no significant differences between pre and mid survey results.

3.84

3.33

3.84

3.79

3.56

3.63

3 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9

Pre

Mid

Pre

Mid

Pre

Mid

Co

nfl

ict

Res

olu

tio

n

Soci

al a

nd

Res

ista

nce

Skill

s

Emo

tio

nal

Co

mp

ete

ncy

Skill

s

Too Good for Drugs and Violence Survey Results


In addition to the TGFD&V survey, participating students were administered the pre-test

for the Normative Beliefs about Aggression and Aggressive Behavior Scale. Average scores

on pre-test surveys were examined to evaluate progress on the conflict resolution and

social and resistance skills performance objective. Youth were asked to report their

perceptions on (1) a number of scenarios on interpersonal conflict and (2) general beliefs

on aggression and use of aggressive behavior. Lower scores or scores closer to one

represent appropriate reactions to interpersonal conflict. There were no significant

differences by grade or in the scores from pre to mid. Student’s scores were slightly

elevated on the subscale for general approval of aggression.

While not a primary objective of the FIA 21st CCLC program, data on students’ suspensions

and detentions are collected to monitor behavioral concerns and make appropriate and

timely referrals. To date, none of the regularly participating students have received

detentions. The graph below shows the number of student suspensions by grade. Of the

seven students who have received suspensions, only one student received more than one

suspension.

Suspensions Received


None One More than one



42 1 1 39 2 0 41 3 0

2.5

2.52

1.95

1.95

2.06

2.07

0 0.5 1 1.5 2 2.5 3

Pre

Mid

Pre

Mid

Pre

Mid

Ge

ner

al A

pp

rova

lo

f A

ggre

ssio

nA

pp

rova

l of

Ret

alia

tio

nTo

tal S

core

Average Scores on the Normative Beliefs about Aggression and Aggressive Behavior Scale


Objective 8: Eighty percent (80%) of regularly participating students will demonstrate

improved goal setting, decision making, and communication as demonstrated by the

TGFD&V After-School Youth Survey.

To evaluate regularly participating students’ improvement in the areas of (1) goal setting

and decision making and (2) communication, the TGFD&V After-School Youth Survey was

administered. Scores range from one to five, with higher scores suggesting more positive

perceptions, attitudes, and skills with scores around 2.5 deemed as average. Overall, pre

and mid data indicate that participating students scored above average. As can be seen

from the graph, scores were fairly similar by grade and from pre to mid. There were no

significant differences among the various grades. Additionally, while some of the scores

were slightly higher on the pre than the mid, there were no significant differences between

pre and mid survey results.

3.12

3.19

4.28

4.24

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5

Pre

Mid

Pre

Mid

Co

mm

un

icat

ion

Skill

sG

oal

Se

ttin

g an

dD

ecis

ion

Mak

ing

Too Good for Drugs and Violence Survey Results


Objective 9: Ninety-five percent (95%) of regularly participating students will have no

more than three unexcused absences in the regular day school as demonstrated by

school attendance records.

Attendance data were available for majority of the regularly participating students. Data

were examined to evaluate progress toward the performance objective regarding less than

three unexcused absences. Attendance data demonstrated that the goal is not currently on

track with reports of approximately 20% of the students having more than three unexcused

absences. Program staff will encourage students’ attendance in day school in an effort to

decrease the number of unexcused absences by the end of the program year.

Number of Students’ Unexcused Absences from School


0 Absences

1-3 Absences

>3 Absences

0 Absences

1-3 Absences

>3 Absences

0 Absences

1-3 Absences

>3 Absences

18 22 10 24 16 8 23 18 9


Objective 10: Ninety percent (90%) of regularly participating students will demonstrate

an increased sense of responsibility for completing homework assignments as

demonstrated by a daily homework checklist.

Homework completion data was collected from teachers. Daily, teachers check each

participating students’ homework and documented whether the homework was

completed, incomplete, or not completed at all. Data from teachers’ daily reports collected

for the months of September and January are displayed below. The data show that the

percentage of students submitting completed homework assignments decreased from

September to January. One reason for this decrease could be attributed to the large amount

of distractions during homework assistance time observed during the site visit. Students in

the FIA 21st CCLC program shared a physical space with FIE 21st CCLC-ELT program for

elementary students. During the homework assistance period, the elementary students

have snacks while talking and playing loudly.

0%

27%

73%

9%

44%

47%

0% 20% 40% 60% 80%

Homework Not Done

Homework Incomplete

Homework Completed

Homework Completion

January September


The FIA 21stCCLC program has two measureable objectives that target the program’s ability

to increase involvement of participating students’ parents in their children’s education and

to equip them with tools to be supportive of their children’s learning.

Objective 11: Sixty percent (60%) of parents/guardians of regular attending students

will participate in at least two Adult Family Member Services activities as demonstrated

by the attendance logs.

This year, quarterly student showcases are held to highlight students’ work products and

offer parents an opportunity to interact with other parents and learn more about the

program. The first student showcase was successful in that about half of the parents of

regularly attending students came out to support their children. The showcase entailed

an introduction of program staff to parents and other visitors; student dance, spoken

word, rap, and drama performances; and a live math bowl competition.

In addition to the student showcases, this year, there was a Family and Friends Fun Friday

event hosted. Over 20 family members/parents attended the event comprised over

various interactive sport activities. Parents and their children participating in friendly

competitions with other students and their families.

In January 2014, a local judge, Orlando Prescott who handles juvenile offenses made a

presentation to parents, family members, and some participating students on the nature

of the offenses most often committed by youth, the consequences of the offenses, and

what parents can do to protect against their children’s engagement in such offenses.

About a quarter of the parents of participating children were in attendance.

Formative attempts have been made to engage parents/guardians, and this year’s

attendance appears to be much improved over last year’s. Program staff should be

commended for their diligence in utilizing various forms of media to attract parents’

participation. Additionally, a raffle was held at the first student showcase to incentivize

parent participation in the event. This objective will be fully evaluated at the end of the

program year; however, for now, this objective appears to be on target evidenced by

attendance ranging from 20% from 50% of parents attending the first few events of the

program year.

Objective 12: Eighty percent (80%) of parents of regularly participating students will

demonstrate a better understanding of their role in supporting their child’s education

as evidenced by scores on pre- and post-test surveys.

Because surveys have not been administered, this objective could not be fully evaluated at

the time of report. However, based on reports from the Program Director, parents have

expressed their opinions of the workshops and have found them to be helpful. For

AD

UL

T F

AM

ILY

ME

MB

ER

SE

RV

ICE

S


example, the last workshop facilitated by the local judge was so informative that parents

requested a second installment of the workshop.

Exit slips, or brief 3-item surveys have been developed and provided to program staff.

The exit slips will used to evaluate parents/guardians’ knowledge of workshop topics at

future workshops. Data will be collected, analyzed, and presented in Summative

Evaluation report for FIA’s 21st CCLC program.

Topic: Topic Name Here

Family’s Name___________________ Child/children’s Name: __________________________________

Did you find this workshop helpful? Not at all Somewhat helpful Very helpful

Did you learn anything new from this workshop? Yes No

Comments:

______________________________________________________________________________________

______________________________________________________________________________________

______

EXIT SLIP


Site Visits This program year, three site visits for FIA’s 21st CCLC program have been conducted. The

first site visit was formal and included an interview with the Program Director and

observation of a typical day in the afterschool program. A monitoring tool was utilized to

evaluate the program’s operations among student-centered and staff-centered activities,

including academic enrichment, social enrichment, and project-based activities. A brief

summary of the commendations and recommendations from the site visit is displayed

below.

Recommendations:

Although parent attendance was reported to be poor for activities held earlier in the academic year, a subsequent visit showed good parent attendance at the showcase. To foster consistent attendance by parents at future activities, staff could consider serving food, giving a gift card to the first 10 families that attend, or other incentives.

The noise level in the cafeteria during the Too Good for Drugs and Violence component was noted as an issue at the time of the site visit. However, conversations with program staff indicated that this was later solved by moving the class outdoors.

During the tutoring component, noise level was an issue. This has been due to the FIE program sharing the cafeteria with FIA during snack time. There is not enough supervision on the FIE side of the cafeteria, which plays a significant factor in the students’ noise level. It may be beneficial to move the tutoring component to the end of the schedule. Another possible solution is for students to spend the tutoring time with the first teacher they are in a classroom with. This would also facilitate the transition from tutoring to classroom activities.

Noteworthy Highlights:

There was a sign for the program that was painted and created as a collaborative effort by the students displayed in the cafeteria. It showed a student at a desk, the 21st CCLC logo, and words that were used to describe the program. The Program Director stated that it was a project that the students were really excited about and proud to have displayed.

The second site visit was informal and comprised observation of a few components of the

afterschool components. The third site visit was during the first quarterly student showcase.

The showcase included skits and raps from various students showing the dangers of drugs.

Students were very supportive of their classmates performing and cheered for every group.

The math bowl was also held, which garnered more participation from students than was

expected. Students were eager to show off their math skills, especially the ones whose

parents were in attendance. Parents were even seen trying to solve math problems

presented to the students. Display boards with students’ science projects were available for

viewing and the Panda Quarterly newsletter was distributed.


Progress toward Sustainability

FIA continues to make great strides towards maintaining a sustainable afterschool and

summer program. FIA engages a number of local businesses and organizations. This

program year, FIA has maintained its relationships with three organizations: DreamCatchers

for the Soul (DreamCatchers), Kappa Alpha Psi, Miami Chapter, and the Resource Room. A

new relationship with a new community partner, the office of Judge Orlando Prescott, has

also been established.

DreamCatchers--a local organization that seeks to educate and empower youth through

education, arts, and counseling services--is providing gender-specific programming.

Specifically, DreamCatchers conducts mentoring workshops with participating girls to

empower them to be peer mentors.

The Miami Chapter of Kappa Alpha Psi, the local division of the national fraternity, was

chartered in 1946, and since that time, the organization has been active throughout the

South Florida community. Last year, the organization connected with the FIA 21st CCLC

program to provide mentoring services for participating boys. The organization conducts

workshops for the boys related to positive youth development and provides informal

mentoring.

The Resource Room is a nonprofit organization that aims to empower families through

education, technology, personal safety, recreation, and leadership. For the FIA 21st CCLC

program, the Resource Room will provide participating youth with training and certification

for the Teen Community Emergency Response Team (Teen-CERT). The Teen-CERT training

will equip youth with the skills necessary to assist in community stabilization efforts

following an emergency or natural disaster. Additionally, the youth will be trained to

provide training for other youth attending neighboring schools.

The office of Judge Orlando Prescott has been secured as key partner for this program year.

Judge Prescott has committed himself and his office to the facilitation of some informative

parent-centered workshops. Additionally, workshops for students and informal mentoring

will also be provided to aid in students’ social development and decrease their risk for

school dropout and engagement in violence and other problematic behavior.

FIA 21st CCLC program staff continues to make great effort o develop new partnerships with

businesses and organizations in the Opa Locka area so that they may expand the services

provided through their 21st CCLC program. The goal is to gain financial stability and find

additional means to sustain the services provided through the 21st CCLC funding.


Dissemination of Findings

The FIA 21st CCLC program plans to disseminate this Formative Evaluation Report through

four avenues:

1. Teachers/Staff Meetings

After school staff meetings are hosted at least once a month. During these meetings, FIA’s

21st CCLC program objectives and progress towards attaining these objectives are

discussed. The Formative Evaluation Report will be included on the agenda for next

months’ meeting, and will be discussed in depth for continuous program quality

improvement. An agenda and sign in sheets will be used to provide evidence of

dissemination. Additionally, a copy of the report will be given to each of program staff and

a signed log of receipt will be used as documentation that staff received the report.

2. Backpack Mailers

The program will make a copy of the report available to every parent, upon request. A

letter will be sent home with each child, which includes a check box where

parents/guardians will be able to select an option to request a hard copy of the Formative

Evaluation Report. Once the request form is sent back to the program’s office, all

requested copies will be sent home to parents/guardians.

3. Email Blasts

The Program Director has developed an email distribution list. Flyers and announcements

are blasted to parents on the list. This method of dissemination began this program year,

and already, staff has seen the benefits of using this low-cost method for communicating

with parents.

4. The FIA’s Website

FIA’s website recently launched a new interactive site (http://www.fiacs.org/) that now

displays all FIA 21st CCLC program announcements, reports, and documents. The school’s

website serves as an optimal environment to publish the Formative Evaluation Report

since it is the gateway through which FIA and its 21st CCLC program communicates with

parents/guardians, teachers, supporting staff, students, and the community.

http://www.fiacs.org/)


Conclusions

The FIA 21st CCLC program continues to be dedicated to providing the community with a

quality afterschool program that positively impacts at-risk youth residing in Opa Locka and

its neighboring communities. Continuous data collection will be a priority for demonstrating

program effectiveness. The FIA 21st CCLC program has made the necessary provisions to

assess progress towards achieving the objectives and goals. The FIA 21st CCLC program has

implemented approved activities and has completed data collection activities for pre

administration points.

The data have been reviewed and areas of improvement and refinement have been noted.

The table below shows progress on each of the program’s measureable objectives. Green

faces indicate objectives that are not yet achieved, though good progress has been made.

Yellow faces indicate objectives that are not yet achieved, though moderate progress has

been made. Red faces indicate objectives that are not yet achieved, though little progress

has been made. Following display of the table is a narrative comprising a few

recommendations for improved programming.

Measureable Objectives Progress

Academic Enrichment

(1) Seventy-five percent (75%) of regularly participating students (i.e., students who

attend the program 30 days or more) will increase their reading comprehension skills

as evidenced by scores on standardized reading tests.

(2) Seventy-five (75%) of regular participating students will increase their reading

fluency skills as demonstrated by the Oral Reading Fluency (ORF) scores.

(3) Seventy-five percent (75%) of regularly participating students will increase math

skills as evidenced by scores on standardized math tests and report cards.

(4) Seventy-five percent (75%) of regularly participating students will increase science

skills as evidenced by scores on standardized science tests and report cards.

Personal Enrichment

(5) Ninety percent (90%) of regularly participating students will improve their fitness

levels as demonstrated by the Progressive Aerobic Cardiovascular Endurance Run

(PACER) Multi-Stage Shuttle Run.

(6) Seventy percent (70%) of regularly participating students will show increased

positive attitudes toward physical fitness as demonstrated by survey on attitudes

towards physical fitness.


Measureable Objectives Progress

(7) Eighty percent (80%) of regularly participating students will demonstrate

improved conflict resolution skills, social and resistance skills, and emotional

competency as demonstrated by the TGFD&V After-School Youth Survey.

(8) Eighty percent 80% of regularly participating students will demonstrate improved

goal setting, decision making, and communication as demonstrated by the TGFD V

After-School Youth Survey.

(9) Ninety-five percent (95%) of regularly participating students will have no more

than three unexcused absences in the regular day school as demonstrated by school

attendance records.

(10) Ninety percent (90%) of regularly participating students will demonstrate an

increased sense of responsibility for completing homework assignments as

demonstrated by a daily homework checklist.

Adult Family Member Services

(11) Sixty percent (60%) of parents/guardians of regular attending students will

participate in at least two Adult Family Member Services activities as demonstrated

by the attendance logs.

(12) Eighty percent (80%) of parents of regularly participating students will

demonstrate a better understanding of their role in supporting their child’s education

as evidenced by scores on pre and posttest surveys.

Program staff developed and implemented creative strategies to improve student and

parent attendance. The program enhanced its program delivery model to make learning

activities more engaging and even added in field trips, which were not offered in previous

years for budgetary reasons. For the Adult Family Member Services component, in addition

to workshops designed to meet parents’ needs, staff hosts quarterly student showcases

which are mutually enjoyable and beneficial for parents and their children. It is expected

that consistent attendance for both students and their parents will be maintained for the

remainder of the program year.

Program attendance is good, but students’ attendance in day school is not currently on

target. Program staff should develop some strategies and incentives to reinforce students’

attendance in day school and reduce the number of unexcused absences.

While the report card grades for reading were excellent, FCAT reading scores demonstrate

that the majority of the students are currently below grade level. ORF test scores also

suggest that more attention is needed to assist students with their reading skills. Robust

academic coaching is suggested to improve students’ reading skills and help them attain


achievement in reading at the appropriate grade level.

Similarly, for math, report card grades were outstanding; however, only 25% of 6th graders,

59% of 7th graders, and 51% of 8thgraders are on grade level according to FCAT scores. It is

recommended that FIA make formidable efforts to determine which strategies may aid in

the improvement of students’ math performance, particularly 6th graders. Instructional staff

may want to consider providing more intensive instruction in math for the various grade

levels. Another method for improving both reading and math proficiency could include

additional assistance with students’ homework.

The percentage of students who completed their homework based on teachers’ reports was

fairly low, with less than half of all participating students completing their homework while

in the program last month. One reason for low percentage could be attributed to the large

amount of distractions during homework assistance time observed during the site visit.

Students in the FIA 21st CCLC program shared a physical space with FIE 21st CCLC-ELT

program for elementary students. During the homework assistance period, the elementary

students have snacks while talking and playing loudly. FIA and FIE staff should develop a

strategy for maximizing activities for both programs.

The vast majority of the students appear to possess good social skills and attitudes towards

aggression and aggressive behavior. Last year, the FIA 21st CCLC program implemented

gender-specific mentoring from community partners. The positive impact of the mentoring

was evident at the end of the 2012-13 year. Preliminary reports for this program year

indicate continued benefits of having students engaged in mentoring activities. Students’

reports on general approval of aggression was slightly elevated, so staff should encourage

the community partners providing mentoring to work with students on identifying more

pro-social alternatives to aggression.

Overall, the FIA 21st CCLC program appears to be on track for a successful final year in their

contract cycle. Based on quantitative and qualitative data amassed, it appears that the

program is providing children with a safe, enriching afterschool environment that has a

positive impact on their academic and social development. The Summative Evaluation

report will include an evaluation of the program for the 2013-14 year, as well as information

from previous evaluations conducted on the other four years that this program has been

funded.

1444 Biscayne Blvd. Suite #115-24

Miami, FL 33132 ( 305 ) 999 - QQRC (7772)

[email protected] www.qqresearchconsultants.com

Documents

2013-14 Formative Evaluation Report