CHAPTER 11 Data Analysis - Wikispacescalicoaches.wikispaces.com/file/view/Chapter+11... · CHAPTER 11 Data Analysis ... Powerful Designs for Professional Learning National Staff Development

CHAPTER 11

Data Analysis

Handouts for this chapter include:

1. Multiple measures of data

2. Magnolia Middle School case study

3. Data analysis study questions

4. Guiding list of demographic data possibilities

5. Steps in analyzing results

Powerful Designs for Professional Learning National Staff Development Council �� www.nsdc.org


Multiple measures of data

H A N D O U T 1 C H A P T E R 1 1 � D A T A A N A L Y S I S

1. Demographics2. Perceptions3. School Processes4. Student Learning

1. Demographic data provide:• Descriptive information about the school and sets its context.• The perspective for understanding all other data.• Understanding of the impact of the educational system on all groups of students through

disaggregating other data by demographics (e.g. student learning by gender or ethnicity).

2. Perceptions data:• Help us understand what students, parents, teachers, and the community think about the

learning environment.• Are important to understand since people act in congruence with what they believe, perceive, or

think about different topics.• From students can tell us what motivates students to learn.• From staff can tell us what is possible within the school.

3. School process data:• Define what school staff members are doing to get the results they are getting.• Include programs, instructional strategies, assessment strategies, classroom practices, and the

implementation of standards.

4. Student learning data:• Describe the results of our educational system.• Include results of norm-referenced tests, criterion-referenced tests, standards assessments, teacher-

given grades, and authentic assessments.


Magnolia Middle School case study


Magnolia Middle School is part of the Great View School District, one of the largest school systems in the nation.

Magnolia Middle School currently serves 781 students. As the graph below shows, the number of students enrolledchanged very little since 1998-99, until 2002-03, when the school population increased by 59 students from theprevious year.

Magnolia’s population by gender shows more females than males overall, in every grade level, and for every year. Thetable below shows the gender numbers and percentages, by grade level, over time.


H A N D O U T 2 / p. 2 C H A P T E R 1 1 � D A T A A N A L Y S I S

The pie graph below shows that the current student population consists of 557 White (71.3%), 179 Black (22.9%), 26Hispanic/Latino (3.3%), and 19 Asian (2.4%) students.

Over the past five years, as shown in the graph below, the percentages of students by ethnicity have remained about thesame. The number of White students dipped from 526 (73%) in 1998-99, to 505 (72%) in 1999-00, increased to 529(74%) in 2000-01, dipped again to 522 (72%) in 2001-02, and increased to 557 (71%) in 2002-03. The Black studentpopulation has steadily increased from 157 students (22%) from 1998-99 to 179 (23%) in 2002-03. The number ofHispanic/Latino students has also increased over time, from 11 students (2%) in 1998-99 to 26 students (3%) in 2002-03. The Asian population of students has ranged between 26 (4%) and 16 (2%) over the same time span, settling in at19 (2%) in 2002-03.



The graph below shows the number of students enrolled by ethnicity for the same time period.

One can see the grade-level population remained fairly steady over time until 2002-03, as shown in the schoolenrollment by grade-level graph below.

Note: Looking at the same grade level over time is called grade-level analysis.



Reorganizing the data in the graph below we can see the groups of students progressing through the grades together overtime. Magnolia’s cohorts show some fluctuation in numbers over time.

Note: Following groups of students over time is called cohort analysis. The numbers beneath the cohort name is thenumber of students that remained the same over time. This analysis is called matched cohort analysis. On average, about75% of the 6th-grade students were still at Magnolia as 8th-graders. That would mean that Magnolia has a mobility rateof around 25%.



The table below shows the number of free/reduced lunch students by grade level, ethnicity, and gender.



Over the past five years, Magnolia Middle School has served between 75 (10%) and 100 (14%) students classified asneeding special education. The majority of students receiving special education assistance were classified as LearningDisabled, followed by Emotionally Disabled, and Educable Severely Handicapped. One-hundred students — less than14% of the school enrollment — were classified as special education in 2000-01, as shown in the graph below. Thatnumber decreased to 75 (10%) students in 2001-02 and increased to 96 (12%) students in 2002-03. These percentagesare average for middle schools in the Great View School District.



The table below shows the spread of special education numbers by learning impairment, grade level, ethnicity, andgender, over time. (Percentages could not be calculated because it is not known how many students had multipledescriptors.)



The three tables that follow show the number and percentage of gifted students for the last four years by grade level,ethnicity, and gender, by free/reduced lunch, and by all four disaggregations combined. (Gifted data were not availablefor 2002-03.)



Students have the opportunity to take honors classes, enabling them to advance and enroll as sophomores in high schoolwhen leaving Magnolia. Approximately 24% of the Magnolia students are enrolled in high school credit courses,including:• Algebra I• Geometry• Spanish I• Keyboarding• English I• Physical Science

Attendance rates at Magnolia Middle School have remained steady over the past five years. Magnolia has an average dailystudent attendance rate of 95.8% and a 95.2% attendance rate for teachers.



Retentions and SupportApproximately 16 students are older than average for their grade, which implies they may have been retained at somepoint in their educational career.

Support systems provided for students beyond the classroom include:• Media Center• After-school Assistance Program (ASAP)• Teen Connection (Urban League)• Individual Tutoring• Group Tutoring (two-teacher teams)• Peer Tutoring — Collaborative pairs and small groups• Counseling (provided by the guidance counselors)• Student Council• Beta Club• Junior Achievement

DisciplineAll teachers at Magnolia post classroom rules that are outlined in the student handbook and are consistently enforcedthroughout the school. Students are encouraged and expected to maintain appropriate behavior at all times to avoiddisruptions in their and other students’ learning.

The interpretations of suspended or expelled students is low — 0.8% in 2002-03 — up from 0.7% in the previous year.The percentage of suspended students in the other Magnolia City middle schools is 1.3%.

The StaffThe 2002-03 staff at Magnolia included: 50 regular teachers, 1 instructional coach, 1 media specialist, 5 specialeducation teachers, and 2 guidance counselors. The majority of the teachers are female. Seven are male. Ninety-fourpercent are White. The other 6% are Black and Hispanic/Latino. Magnolia has five National Board Certified Teachers.One-half of the teachers have advanced degrees. No teachers are teaching out of their certificated field. The graph thatfollows shows the number of teachers has remained fairly consistent since 1998-99. The increase of 3 teachers in 2002-03 is connected to the enrollment increase that year. Class size is approximately 16 to 1.



There are currently 5 new full-time teachers and 3 new part-time teachers. Ninety percent of the current teaching staffhas been in the school for the past 10 years. The number of years of teaching experience, by grade level, is shown in thetable below.

Additional personnel include the school principal and assistant principal, secretary, instructional assistants, clericalsupport staff, custodial staff, library technician, campus supervisors, and food services workers. This is the third year atMagnolia Middle School for the principal and the ninth year for the assistant principal. The previous principal was at theschool for three years. The principal, assistant principal, and instructional coach are White females.



PERCEPTIONSMagnolia Middle School students, staff, and parents completed Education for the Future questionnaires. Students andstaff also answered two open-ended questions. The results follow (not all graphs are shown).

(Note: Some of the subgroup numbers do not add up to the total number of respondents because some respondents didnot identify themselves by this demographic, or identified themselves by more than one demographic.)

Student Questionnaire ResultsEighty-six percent of the students in grades six through eight at Magnolia Middle School responded to a questionnairedesigned to measure how they feel about their learning environment in May 2003. Students were asked to respond toitems using a five-point scale: 1 = strongly disagree; 2 = disagree; 3 = neutral; 4 = agree; and 5 = strongly agree.Responses to each item on the questionnaire were graphed by the average and disaggregated by gender, grade level, andethnicity. A summary of results is shown below.

Average Student ResponsesAverage student responses to all the items in the student questionnaire were in agreement with the statements, with theexception of three items that were closer to neutral. These items were: I have freedom at school, I have choices in what Ilearn, and Students are treated fairly by teachers.

Student Responses by GenderData disaggregated by gender revealed responses very similar to each other and clustered around the overall average.(Graph not shown here.)

Student Responses by Grade LevelThe questionnaire results were also disaggregated by grade level (see graph). There were 229 6th graders (85% responserate); 215 7th graders (85% response rate); and 217 8th graders (83% response rate) responding. The graph revealedsome slight differences when disaggregated by grade level. Sixth graders were generally the highest in agreement, while8th graders were generally in lowest agreement.



Student Responses by EthnicityStudent questionnaire data were also disaggregated by ethnicity (122 Black; 28 American Indian/Pacific Islander; 18Asian; 414 White; 34 Hispanic/Latino; and 97 Other students responded). The graph that follows shows responses weresimilar and revealed few differences. Asian students generally responded less positively than other subgroups. While thesubgroup of students marking themselves as Asian were relatively small (n=18), we still need to follow up with them.





Student Open-Ended ResponsesIn May 2003, Magnolia Middle School students were asked to respond to two open-ended questions: What do you likeabout your school? and What do you wish was different at your school? The top 10 responses are shown below. (Note: Whenanalyzing open-ended results, one must keep in mind the number of responses that were written in. Open-endedresponses often help us understand the multiple choice responses, although caution must be exercised around smallnumbers of respondents.)

1. What do you like about your school?• Good teachers (145)• This school has a very strong related arts program (98)• My friends (51)• The class choices we have for electives (30)• Gym (27)• Nothing! (23)• Learning can be fun (14)• The people (14)• The band program (10)• My work is challenging (9)

2. What do you wish was different at your school?• Less strict dress code (65)• Teachers more respectful of all students (59)• Better lunches and more food in the cafeteria (53)• Bigger hallways (37)• Cleaner bathrooms with locks on the doors (32)• Less homework and fewer projects (25)• Students would treat each other better (20)• No teaming (14)• Be able to chew gum (8)• More freedom (7)



Staff Questionnaire ResultsMagnolia staff (n=49, 98%) responded to a questionnaire designed to measure their perceptions of the schoolenvironment in May 2003. Members of the staff were asked to respond to items using a five-point scale: 1 = stronglydisagree; 2 = disagree; 3 = neutral; 4 = agree; and 5 = strongly agree.

Average responses to the items on the questionnaire were graphed and disaggregated by job title and number of yearsteaching.

Average Staff ResponsesOverall, the average responses to the staff questionnaire were in agreement with most of the statements in thequestionnaire.

Staff Responses by Number of Years TeachingStaff questionnaire data were also disaggregated by the number of years teaching (one to three years, n=5; four to sixyears, n=5; seven to 10 years, n=8; and 11 or more years, n=31). Results graphed by this demographic (see three-pagegraph) reveal some apparent differences in responses, although the size of the Ns needs to be taken into consideration.





��





Staff Open-Ended ResponsesMagnolia Middle School staff completed two open-ended questions in May 2003: What are the strengths of this school?and What needs to be improved? The top 10 results are shown below.

1. What are the strengths of this school?• Dedicated staff (32)• Extremely talented students (10)• The arts program (8)• The school’s reputation (2)• The teachers’ love for their students and their curriculum• Appropriate facilities for our unique program• The parents of the children we teach are wonderful and helpful• Majority of students are motivated to learn• Principal has a strong vision for student achievement• Student diversity

2. What needs to be improved?• Incorporate shared decision making so teachers have a voice (11)• Teacher morale (3)• Communication among all staff members (3)• Standards-based instruction with higher-order assessments• Teacher leadership• Learning styles• Literacy instruction for below-grade-level readers• Smaller class sizes for academic subjects• Too many demands on our time — little time for reflection and planning• Establish clear communication



Parent Questionnaire ResultsParents of students attending Magnolia Middle School completed a questionnaire designed to measure their perceptionsof the school environment in May 2003 (N=350; 75% response rate). Parents were asked to respond to items using afive-point scale: 1 = strongly disagree; 2 = disagree; 3 = neutral; 4 = agree; and 5 = strongly agree.

Average responses to each item on the questionnaire were graphed by the average and disaggregated by ethnicity,children’s grade levels, number of children in the household, and number of children in the school.

Average Parent ResponsesOverall, the average responses to the items in the parent questionnaire were in agreement.

Parent Responses by EthnicityWhen parent questionnaire data were disaggregated by ethnicity: 274 White, 32 Black, 18 Hispanic/Latino, 10 Asian,and 16 Other. Results show that Black and Other parents were generally less positive in their responses. Some differenceswere noted (see graph), although most averages were in agreement. The lowest averages were neutral. Once again, wemust take the small Ns into consideration.





Parent Responses by Children’s Grade LevelsParent questionnaire responses disaggregated by children’s grade levels revealed few differences. No significantdistinguishing pattern emerged when looking at the data by these subgroups. (Graph not shown here.)

Parent Responses by Number of Children in the SchoolParent questionnaire data disaggregated by the number of children in the school clustered around the overall average andreveal few differences in responses. (Graph not shown here.)

Parent Responses by Number of Children in the HouseholdParent questionnaire data were also disaggregated by the number of children in the household (one child, n=62; twochildren, n=158; three children, n=82; and four children, n=24). Results graphed by this demographic revealed somedifferences in responses. (Graph not shown here.)



What are our results?

STUDENT ACHIEVEMENTThe State Criterion-referenced Assessment Test (SCAT) is the test used by this state for grades six, seven, and eight inEnglish/language arts (ELA) and mathematics. Student results are reported by proficiency levels. The two tables thatfollow summarize the proficiency results by grade level and subgroups for English/language arts and mathematics for2001-02 and 2002-03.


H A N D O U T 2/ p. 25 C H A P T E R 1 1 � D A T A A N A L Y S I S


Data analysis study questions


Demographic Data

1. What are your school’s demographic strengths and challenges?

Strengths Challenges

2. What are some implications for the school improvement plan?

3. Looking at the data presented, what other demographic data would you want to answer the question Who are we?

School: ______________________________________________


Continuous Improvement Continuums Data

1. What came out of the CIC analyses as strengths and challenges?



3. Looking at the data presented, what other CIC data would you want to answer the question How do we do business atour school?


Data analysis study questionsSchool: ______________________________________________




Analysis of Student, Staff, and Parent Questionnnaire Results

StudentQuestionnaire

StaffQuestionnaire

ParentQuestionnaire

AgreementsAcross

Questionnaires

DisagreementsAcross

Questionnaires

General Feel ofQuestionnaire(positive, neutral,negative)

Most PositiveItems

Neutral Items

Negative Items

On which itemsare theredifferences insubgroups?(i.e., disaggregatedresponses)

Comments



Standards Assessment Data

1. What are the school’s strengths and challenges that came out of the standards assessment?



3. Looking at the data presented, what other standards assessment data would you want to answer the question How dowe do business at our school?




Student Learning Data

1. What are your student learning strengths and challenges?



3. Looking at the data presented, what other student learning data would you want to answer the question How arewe doing?




Guiding list of demographic data possibilities

STUDENTS OVER TIME AND BY GRADE LEVEL

Extracurricular activity participation/clubs/service learning by gender,ethnicity, language fluency, free/reduced lunch

Number of participants in programs such as AP, IB, Honors, UpwardBound, Gear-up, college-prep, vocational

Number of home schoolers associated with school

Number of students electing to come to the school from out-of-attendance area

Number of bus riders

Student employment

* Discipline indicators (e.g. suspensions, referrals, types of incidents,number of students carrying weapons on school property)

* Graduation rates by gender, ethnicity, language proficiency, free/reducedlunch, migrant, and special education (where students go, what they do)

Number of students concurrently enrolled in college courses

Number of students meeting college course entrance requirements, bygender, ethnicity, language fluency, free/reduced lunch

Number of scholarships by gender, ethnicity, language proficiency,free/reduced lunch

Number of students completing GEDs

Adult education program

Number and percentage of students going on to college, postgraduatetraining and/or employment

Grade-point average in college

Number of graduates ending up in college remedial classes

Demographic data Number of yearsof data we have

Number of yearsof data desired

* Required for NCLB (includes the numbers required to understand the disaggregated numbers required by NCLB)



* Required for NCLB (includes the numbers required to understand the disaggregated numbers required by NCLB)

Demographic data Number of yearsof data we have

Number of yearsof data desired

STAFF OVER TIME

* Number of teachers, administrators, instructional specialists, support staff,by roles

* Years of experience, by grade level and by role in teaching

Ethnicity, gender, languages spoken

Retirement projections

* Types of certifications/ licenses/ teacher qualifications/ percentage of timeteaching in certified area(s)

Grades/subjects teachers are teaching

Degrees

* Educational training of paraprofessonals

Teacher-student ratios by grade level

Teacher turnover rates

Attendance rates

Teacher involvement in extracurricular activities, program participation

* Number of teachers receiving high-quality professional development

* Percent of teachers qualified to use technology for instruction

National Board for Professional Teaching Standards (NBPTS) teachers

PARENTS

Educational levels, home language, employment, socioeconomic status

Involvement in their child’s learning

Involvement in school activities

Incarceration

WHAT OTHER DATA?



Steps in analyzing results

1. One participant serves as facilitator to lead the whole group through this process. Another

participant serves as a recorder of the whole group’s work.

2. Individuals independently read the demographic section of the report and jot down their

thoughts about strengths, challenges, and implications of the data for school improvement. They

note additional data to gather.

3. Individuals form small groups (five to seven people) to share their thoughts about demographics.

One member of each small group serves as recorder for the small group discussion.

4. The facilitator brings the small groups together to merge their thinking, with the whole-group

recorder capturing their ideas.

5. The facilitator uses a round-robin style of reporting and recording, beginning with one of the

small groups. Each small group’s recorder presents all of that group’s ideas for the group recorder

to capture on chart paper. Subsequent groups can acknowledge ideas that already have been

contributed (and the group recorder can make a tick mark beside those items) and then presents

new ideas.

Repeat this process with the other three aspects of data: perceptions, student learning, and school processes.

Documents

CHAPTER 11 Data Analysis - Wikispacescalicoaches.wikispaces.com/file/view/Chapter+11... · CHAPTER 11 Data Analysis ... Powerful Designs for Professional Learning National Staff Development