Data collection & Analysis Practices

C A I T L I N R E A S H – U N I V E R S I T Y O F M O U N T U N I O N

DATA COLLECTION & ANALYSIS PRACTICES

DEMOGRAPHICSStudent Enrollment: 440 students

9.6% Black, 4.5% Multi-Racial, 85.2% White 62.7% Economically Disadvantaged 20.9% Students with Disabilities

Faculty: 30 General Education teachers, 7 Special Education teachers, 4 Title One teachers (2 reading, 2 math), 1 Guidance Counselor, and 1 Principal.

100% hold a Bachelor’s Degree – 66.7% hold a Master’s DegreeSchool: Designated as a Medium-High Poverty school

Many students receive breakfast at the school daily and are on Free and Reduced lunch

Parent involvement would be considered low by most teachersLRC rating: Continuous Improvement

Value-Added was metCulture of Data Collection and Analysis: In this school, data collection and

analysis is mainly done by classroom teachers, with a few exceptions. Teachers are required to administer, score, analyze, and make the data available to parents (if applicable). Teachers understand the purpose for most of the assessments, but still feel that assessments take up a significant amount of their teaching time. Data is typically received and reviewed in a timely manner, as the results are available online the same day the test is given for some of the tests. Teachers use Teacher Based Team meetings to review and analyze the data for various purposes. Assessments are used for student achievement, progress monitoring, and for intervention strategies. They are also used for grouping students based on skill sets.

Process AnalysisGathering the Data: To gather the data, I spoke with the principal first to ensure that I had her approval before continuing. This was done during a face to face conversation, and at that time, I also generated a list of the assessments that were administered in the building. The next day, I went to a teacher from each grade level and asked her if she would be willing to be informally interviewed about the assessments in her grade level. All teachers that I asked were willing to speak with me, so I let them know that I would follow up with them shortly, which I did through face to face interaction. When I met with the teachers I spoke with, we reviewed each of the assessments that they administer whether it is a standardized assessment, unit (reading series) component, or one that they created as a grade level. In a couple instances, there was more than one teacher present during the interview (which I had let the teacher I talked with know was alright). This was helpful because it provided me with a more comprehensive view of the assessments, along with their grade level feelings towards them.

Stakeholders: Throughout this project, I spoke with the Principal in the building, along with the Special Education Director who also serves as a testing organizer (which I did not know until this process), and eight teachers from various grade levels and positions.

Effectiveness: The way in which data was collected for this project took some time, but was effective in that I was able to talk with teachers at each grade level about the assessments they administer. Had I only talked with teachers from a grade level or two, I would not have the understanding that I do about the beliefs of each grade level team when it comes to their assessments. During this process, I found out that there was a testing organizer, which I feel was something that should have been communicated to all teachers. If teachers needed the testing information for each grade level quickly, there is not a document or way for them to get it without talking to individuals in the building. This project allowed me to create a chart that will make the information much more accessible if it is used.

Kindergarten Second Fourth

Monthly:Pre and Post Tests Math and Reading-

aligned with curriculum

Beginning of the Year:

KRAL (August)DIBELS (September)

Middle of the Year:DIBELS (January)

End of the Year:DIBELS (March)

STAR Early Literature (APR)

Running Records (MAY)

CTBS: GIFTED (BASED ON REFERRAL)

Monthly:Pre and Post Tests Math and Reading-

aligned with curriculum


DIBELS (September)STAR Math & Reading

(September)Running Records

(Sept)


STAR Math & Reading(January)

Running Records (Jan)

End of the Year:DIBELS (April)

STAR Math & Reading State Diagnostic

(May)Running Records

(May)


First ThirdMonthly:

Post Unit Assessments aligned with Treasures


DIBELS (September) CTBS (September)

STAR Math & Reading(September)




STAR Math & Reading (May)

State Diagnostic (May)



(September)OAA Language Arts Practice(October)




OAA (May)STAR Math & Reading

(May)


Weekly:Reading Checks –

Aligned with Series

Monthly:Pre and Post Tests-

MathUnit Tests aligned with Treasures CurriculumCriterion Referenced (SS, SCI, and MATH)



(September) IOWA (September)OAA Language Arts Practice (October)




OAA (May)STAR Math & Reading

(May)

Grade Level Assessments Kindergarten-4th

TEST KRA-L DIBELS STARState

Diagnostic

Running Records

Unit Pre/Post

TestsCTBS IOWA OAA

Description

Diagnostic:Literacy test

used for students

entering K

Diagnostic:Reading/

Sounds/Letter Identification

Test

Diagnostic:Reading and Math Levels

Diagnostic:Reading and

Math

Formative:Informal Reading Check

Summative/Formative based on delivery


Math

Diagnostic:Reading, Math, and Science


Math

Purpose1. Placement of students

2. Determine students’

strengths/weaknesses

To determine student levels

for reading intervention

(Title 1 services)

1. Progress Monitoring

2. Also used for

intervention purposes

To determine student levels

at the conclusion of the academic

year

To determine students’ current

reading level

To drive instruction and track student

growth or need for

intervention

Given to identify

students who may be gifted

To determine student

achievement

To determine student

achievement

Gravel Level(s) Kindergarten K-4 End of K-4 K-2 End of K-1

K-4 (test varies by grade level)

2nd gradeOther grades:

(referral)4th 3-4

Dates of Admin.

August(make-ups, by

Oct.1)

Fall (Sept)Winter (Jan)Spring (Apr)

Fall (Sept)Winter (Jan)Spring (May)

Spring (May)Fall (Sept)

Winter (Jan)Spring (May)

Monthly (for most grade

levels)

September(can vary

based on ref)September

Practice (LA): OctoberAll: May

Scores are

Calculated

Same Day Same Day Same DaySame

Day/Next Day Same DayVaries, but typically

within the week

November DecemberPractice: Nov/Dec

Actual: June

Teachers Receive Scores

AugustSame day as assessment

Same Day as the

assessment(by end of

school day)

Immediately

At the conclusion of testing for the

grade level(same week)

Same Day because teachers

complete the assessment

ImmediatelyDue to

teachers administering

the assessment

Varies based on how it is being given (referral vs. grade level)

FebruaryPractice: Nov/Dec

Actual: June

Distributed to

Parents (If it

applies)

Generally, immediately following the test if time

permits.

For K:Within the

month(due to 3rd

grade reading guarantee)

For 1-4:Within the

month(due to 3rd

grade reading guarantee)

N/A N/A (Can be

discussed at conferences)

N/A (Can be

discussed at conferences)

Varies by teacher

N/A On State Report Card

Who Analyzes

Kindergarten Teachers

Title 1 Teachers

Teachers/Title One Math Teachers

Teachers/title One Teachers

Teachers Teachers Outside Testing

Service/Gifted Teacher

Outside Testing

Service/Teacher

Outside Testing

Service/Teachers

When are they

analyzed

The day after testing is

complete for all students

At conclusion of K-4

Assessments (varies)

At the following Teacher

Based Team Meeting

At this point, they are not

Same Day At the following Teacher

Based Team Meeting

When results are received



Next Steps

Students placed based

scores.Reported to

EMIS

Intervention/Title One

groups are determined

Growth is charted and intervention groups are

made

A spreadsheet is submitted

to the state to prove tests were given

Teachers assign books

based on student’s

level

Intervention groups are

determined.Growth is charted

Students identified as gifted begin

receiving services

Interventions to increase students’

scores

Interventions to increase students’

scores

AUG SEPT OCT NOV DEC JAN FEB MAR APR MAY

DIBELS NaN 2 NaN NaN NaN 6 NaN NaN 9 NaN

STAR NaN NaN NaN NaN NaN NaN NaN NaN NaN 10

Running Records NaN NaN NaN NaN NaN NaN NaN NaN NaN 10

Diagnosic NaN NaN NaN NaN NaN NaN NaN NaN NaN 10

Pre/Post 1 2 3 4 5 6 7 8 9 10

KRAL 1 NaN NaN NaN NaN NaN NaN NaN NaN NaN

1.5

2.5

3.5

4.5

5.5

6.5

7.5

8.5

9.5

Kindergarten Assessments

Mon

th in

Sch

ool



STAR NaN 2 NaN NaN NaN 6 NaN NaN 9 NaN

Running Records NaN 2 NaN NaN NaN 6 NaN NaN 9 10


Pre/Post 1 2 3 4 5 6 7 8 9 10

1.5

2.5

3.5

4.5

5.5

6.5

7.5

8.5

9.5

First Grade Assessments

Mon

th in

Sch

ool




CTBS NaN NaN 3 NaN NaN NaN NaN NaN NaN NaN


Post Tests 1 2 3 4 5 6 7 8 9 10

1.5

2.5

3.5

4.5

5.5

6.5

7.5

8.5

9.5

Second Grade Assessments

Mon

th in

Sch

ool




OAA NaN NaN 3 NaN NaN NaN NaN NaN NaN 10

CTBS NaN NaN NaN NaN NaN NaN NaN NaN NaN 10

Pre/Post 1 2 3 4 5 6 7 8 9 10

1.5

2.5

3.5

4.5

5.5

6.5

7.5

8.5

9.5

Third Grade AssessmentsM

on

th in

Sch

ool




Weekly Reading Checks

1 2 3 4 5 6 7 8 9 10

Terra Nova NaN NaN NaN NaN NaN NaN NaN NaN NaN 10

Pre/Post Math 1 2 3 4 5 6 7 8 9 10

Reading Unit Test 1 2 3 4 5 6 7 8 9 10

Criterion Referenced 1 2 3 4 5 6 7 8 9 10

OAA NaN NaN 3 NaN NaN NaN NaN NaN NaN 10

1.5

3.5

5.5

7.5

9.5

Fourth Grade Assessments

Mon

th in

Sch

ool

The KRA-L is used by teachers to determine the levels of students entering Kindergarten.

Results are reported to the state. Testing occurs during the first three days of school for the rest of the building (1-4).

Kindergarten begins three days later than grades 1-4 to allow time for testing/placement. Once testing is complete, Kindergarten teachers come together to analyze the data. Teacher have two to three hours to review scores, group children based on scores,

and distribute into classes. Many of the Kindergarten teachers felt rushed to group students and be prepared

for parents to come the same afternoon that data is analyzed/students placed. During a teacher based team meeting, Kindergarten teachers, along with the school

principal and county representative, discussed changes that could be made regarding the KRA-L assessment. Overall, teachers felt that the KRA-L was not comprehensive and representative of what

students should be tested on. Additional time was suggested in order to also administer a teacher created assessment

for math and other reading skills (not yet created). An increase in the time to analyze scores and the way in which children were grouped was

also shared. Teachers would like heterogeneous groupings instead of the current “high, medium high,

medium low, low” groupings.

KRA-LKindergarten Readiness Assessment - Literacy

Kindergarten August Reading levels when entering Kindergarten

DIBELS is administered by Title One reading teachers three times a year. Results are made available to teachers the same day as the test is given to

his/her class. The information collected is analyzed by the Title One teachers at the

completion of testing for grades K-4. Based on the students’ scores, reading intervention groups are determined

to provide supports to at-risk students. Currently, a two day pull-out, two day push-in model is being used to

provide necessary interventions. In Kindergarten only, parents receive a notice as to whether their child has

met benchmark or not. Title One teachers, along with classroom teachers, are currently

questioning the effectiveness of the current push-in model because of the success/student improvement that was achieved through the five day pull-out model. The main reason for this is that in the current model, the student to teacher

ratio is 13:1, whereas in the pull out model, the ratio is at most 4:1.

DIBELSDynamic Indicators of Basic Early Literacy Skills

Kindergarten-4th Sept, Jan, & AprLiteracy Skills/Title One Reading

Intervention

The STAR test is administered by grades K-4. At the end of Kindergarten, students take the STAR Early Literacy to determine their reading level entering First Grade.

Results are made available to teacher immediately due to the electronic format of the tests.

The scores are then analyzed by teachers, including Title One math, in order to monitor progress and determine intervention groupings. This also helps to determine the skills to be addressed during both math and reading

interventions. Based on math scores, the Title One teachers conduct mini lessons (pull-out) in

addition to the push-in math intervention time. Student scores and analysis are shared with parents after each time the test is

administered. Teachers interviewed are pleased with the types of questions used, the

speed/ease of getting results, and the data/interventions suggested based on the student scores.

Teachers also find that this is an effective way to monitor progress. Second grade teachers are considering using this test monthly because of the data

provided and how its use in effective interventions.

STAREarly Literacy, Reading, and Math

End of K-4th Sept, Jan, & MayProgress Monitoring/Intervention

Groupings

The State Diagnostic test is required by Ohio to be administered in Kindergarten through 2nd grade at the conclusion on the academic year.

Teachers administer the test and results are made available immediately.

At the current time, these results are not analyzed, but a spreadsheet is made in order to show the state that tests were administered.

The school principal said that this can help to place students in appropriate classes the next year.

Teachers interviewed feel that the current way it is being used is not effective, and they do not understand the need for it other than it is required.

State DiagnosticRequired by Ohio in grades K-2

K-2 May Determine student achievement levels

Running records are completed by classroom teachers in order to determine students’ current reading levels.

Given three times a year, results are also used to show progress made by students.

Results are immediate due to the fact that teachers assess each student.

Information is used by teachers in order to select the appropriate texts for the students.

Running records used to be done in additional grade levels, but in talking to the principal, when reading coaches were dismissed, running records were not given except in the end of Kindergarten and during First Grade.

A first grade teacher felt that they are an accurate way to formatively assess how her students are reading and to ensure that she is providing reading instruction that will benefit all of her students at their various reading levels.

Running RecordsReading

End of K-1 Sept, Jan, & May Track reading ability/Show reading growth

In some way, each grade level administers pre and post tests for math and reading. Kindergarten: Teacher created math and reading assessments that align to the

CCSS (Common Core State Standards) The reading assessment is a hybrid of the Treasures’ Unit Tests and additional skills that

the teachers want to monitor. First Grade: Reading Series Unit Tests along with a CCSS aligned math assessment Second Grade: Post Tests aligned to CCSS Third Grade: Pre/Post aligned to CCSS Fourth Grade: Pre/Post aligned to CCSS for math, Treasures’ Unit Tests for reading,

Criterion-Referenced Tests for Science and Social Studies, and formative Weekly Reading Checks that are from Treasures as well.

The Pre/Post data is used to drive instructional choices, monitor progress, and to determine intervention groups for both reading and math instruction.

Some grade levels do not see the need for unit assessments in addition to the other tests that are given.

On the other hand, grade levels feel that it does help to make instructional decisions that will benefit all learners.

Pre/Post TestsReading and Math

K-4 Monthly Progress Monitoring / Drive Instruction

The CTBS is a diagnostic test that the school uses in order to determine student achievement in second grade, along with gifted identification.

The principal says that the school no longer uses the Terra Nova test, and that the CTBS has taken its place.

Based on teacher referral, students are given the CTBS in order to see if they qualify for gifted services. Referrals can be made by teachers starting in Kindergarten

Second grade teachers feel that the information is not communicated well.

CTBSCalifornia Test of Basic Skills

Grade 2 and also Based on

Referral

September / Can Vary

Student Achievement / Gifted Identification

The IOWA Basic Skills Test is administered to 4th grade students. Results are compiled by an outside testing service and then provided to

the school. This year, the test was given in September, and results were received in

December. As of right now, the principal was the only one who had seen the results. She mentioned that she was not sure when the tests were submitted and that

could be a reason for the long return time (Sept-Dec).

Student achievement is the purpose of these assessments.

IOWATest of Basic Skills

4th Grade September Student Achievement

Starting in third grade, the OAA is administered to track student achievement.

In October, students are given the reading portion as a practice test. The actual test is not given until May, and this is when all portions are

administered. Results from the practice test are used to determine areas of need, as

well as to provide a pre test to use for student growth in the Spring. After testing is complete, results are compiled by an outside testing

service and returned to the school within a month typically. Teachers are working to improve student achievement through

intervention strategies and the implementation of a new reading series. This is only type of Value-Added data currently provided.

The data is provided on the State Report Card

OAAOhio Achievement Assessment

Grades 3-4 Practice: OCTActual: MAY

Student Achievement / Value Added

ConclusionsThis building is in the very early stages of effective data collection that is easily accessible and known to all teachers. Currently, there is not an electronic system in place to allow for easy access to assessment information or student data.

Recommendation: To remedy this, for testing information, I feel that using a Google Doc or some other electronic editing tool would make the information

easily accessible to everyone in the school. Not only could they see the information, but they could edit it when changes occur (for instance if second grade was going to begin using the STAR instead of Pre and Post Test to drive instruction).

For student data, I would suggest that we utilize our Progress Book (electronic system) more effectively. We could input student scores of that system rather than in a “data folder” in their student file folder. The next year, teachers would be able to see all of their students’ information without having to pull each student’s file folder.

From grade level to grade level, there is disconnect about the assessments that are given, and while teachers in the grade level understand the purpose of each assessment, they do not always feel that they are necessary.

Recommendation: Have grade level meetings that are vertical to discuss the use of each assessment in their give grade levels. At these meetings they could also discuss the purpose of the assessment and if one could be use in its place for the same purpose. Currently, the State Diagnostic tests are not used for anything except to prove to the state that they were completed, so having a team of teachers brainstorm uses for those assessments would be beneficial. An article from Learning Point Associates said, “Working in a team, individuals can discover new ideas and views by collaborating with their

teammates—discoveries they would never have made on their own” (2004).

ConclusionsTests that are found to be most effective by teachers are the STAR series and Pre and Post tests. The STAR is an efficient tool for monitoring progress, and Pre and Post tests help to drive instructional decisions and organize intervention groupings. The DIBELS is also an assessment that monitors progress, but because of the computerized nature of the STAR, it ranks higher among teachers than the DIBELS, especially with the Common Core State Standards coming down the pike and the computerized testing that will go along with them.

Recommendation: Since the CCSS is going to be a sort of computerized test, I would encourage the use of the STAR

since it is electronic. The STAR is currently being used as a progress monitoring tool, which is one of its designed uses (Renaissance Learning, 2011). In addition, I would explore using it in place of pre and post tests to drive instructional choices. Even if the questions are not directly related to the

reading series (like the pre and post tests are), the skills that the students are lacking in

could be addressed using those reading series components. In Delaware, they are using computer-adaptive testing for standardized and noticing that there a number of benefits including faster results, less time for students to take the tests, and better

result data (Davis, 2012).

The amount of testing in third grade is much less than the testing that takes place in fourth grade. On the state report card, grade four is performing higher than grade three when it comes to reading (Ohio Department of Education, 2011).

Recommendation: Grade three should do reading checks and interventions based on students’ reading levels. Since literacy coaches were let go,

Running Records became used much less infrequently. If teacher do not want to do running records for their students, then weekly reading checks (from the series) like those done in fourth grade could be beneficial for reading improvement. (Again, vertical teams could discuss

items such as these if the school had them,)

ConclusionsTeachers feel overwhelmed and that there is not enough teaching time.

Recommendation: A first grade teacher suggested a substitute to come in and administer the running records or teach the class while the

teacher does them in a quiet setting instead of at recess, which is when she currently gives them. When possible, teachers could use formative assessment instead of the typical paper and pencil pre and post tests. This “assessment” would then take place during teaching time instead of taking away from it.

ReferencesDavis, M. R. (2012, October 12). Adaptive testing evolves to assess

common-core skills. Education Week. Retrieved from http://

www.edweek.org/dd/articles/2012/10/17/01adaptive.h06.html

Ohio Department of Education. (2011). Building report card 2010-2011. Columbus, OH. Retrieved from http://www.ode.state.oh.us/

reportcardfiles/2010-2011/BUILD/019380.pdf

Learning Point Associates. (2004). Guide to using data in school improvement efforts. Naperville, IL. Retrieved from http://www.learningpt.org/pdfs/datause/guidebook.pdf

Renaissance Learning. (2011). The foundation of the STAR assessments: The science of STAR. Wisconsin Rapids, WI. Retrieved from http:// doc.renlearn.com/KMNet/R003957507GG2170.pdf

Documents

Data collection & Analysis Practices