Project SAILSS: Final Evaluation Report€¦ · 410-570-0315 Submitted by: Sonia Jurich, MD, Ed.D., Senior Research Associate Laura Taylor, M.A., Research Associate RMC Research Corporation

Project SAILSS: Final Evaluation Report

Anne Arundel County Public Schools

September 15, 2013

Project SAILSS: Final Evaluation Report


Submitted to:

Lori Snyder, NBCT

Office of Advanced Studies and Programs


2644 Riva Road

Annapolis, Maryland 21401

410-570-0315

Submitted by:

Sonia Jurich, MD, Ed.D., Senior Research Associate

Laura Taylor, M.A., Research Associate

RMC Research Corporation

1501 Wilson Boulevard, Suite 1250

Arlington, VA 22209

Phone: 703.558.4000

Fax: 703.558.4823

September 30, 2013

Sculpture created by students and Kevin Reese, artist in residence,

integrating visual arts and mathematics

as part of the SAILSS project

Location: Entrance to Willey H. Bates Middle School

Annapolis, Maryland

AACPS Project SAILSS – Final Evaluation Report Page | i

TABLE OF CONTENTS

TABLE OF CONTENTS .............................................................................................................. i

LIST OF TABLES ....................................................................................................................... iii

LIST OF FIGURES ..................................................................................................................... iv

EXECUTIVE SUMMARY .......................................................................................................... v

INTRODUCTION......................................................................................................................... 1

BACKGROUND ........................................................................................................................... 3

The Grantee ......................................................................................................................................3

The Project .......................................................................................................................................4

IMPLEMENTATION OUTPUTS .............................................................................................. 7

Preparing Teachers for Arts Integration ..........................................................................................7

1. Off-campus professional development activities ................................................................ 7

2. In-house professional development activities...................................................................... 8

3. Modeling Arts Integration: Artists in Residence and Arts Teachers ................................... 9

From Professional Development to Classroom Application .........................................................11

From the Classroom to the Neighborhood .....................................................................................12

From Neighborhood to the Community .........................................................................................13

INTERMEDIATE OUTCOMES .............................................................................................. 14

Changes in School Organization ....................................................................................................15

Changes in Teachers’ Perceptions .................................................................................................17

1. Arts integration as a teaching strategy ........................................................................... 17

2. Arts integration and student engagement ....................................................................... 18

3. Arts integration and school climate ................................................................................ 20

Changes in Teaching Practices ......................................................................................................23

Changes on Students’ Engagement ................................................................................................26

1. Perceptions about arts integration ...................................................................................... 26

2. Observed behavior ............................................................................................................. 29

AACPS Project SAILSS – Final Evaluation Report Page | ii

PROJECT IMPACT ................................................................................................................... 31

Student Demographics ...................................................................................................................31

Academic Performance ..................................................................................................................33

1. Descriptive analysis........................................................................................................ 33

2. Comparison of means ..................................................................................................... 36

3. Effect size analysis ......................................................................................................... 37

4. Cohort Analysis .............................................................................................................. 38

Monitoring Project Objectives .......................................................................................................38

CONCLUSIONS ......................................................................................................................... 43

REFERENCES ............................................................................................................................ 47

APPENDIX A: EVALUATION DESIGN ................................................................................ 48

AACPS Project SAILSS – Final Evaluation Report Page | iii

LIST OF TABLES

Table 1: Average results for statewide assessments for Bates and AACPS ................................... 3

Table 2: Analysis of standards targeted in arts integration lessons .............................................. 12

Table 3: Comparison of means – student support scale (2009 and 2012 surveys) ....................... 23

Table 4: Integrating arts with core content areas .......................................................................... 28

Table 5: Students’ perceptions that arts improve understanding in core content areas ................ 29

Table 6: Average yearly attendance per school from SY 2009 to SY 2012 ................................. 30

Table 7: Average attendance by school and grade (SY 2013) ...................................................... 30

Table 8: Student demographics by school (SY 2012-2013) ......................................................... 31

Table 9: Results from the 2013 MSA Reading and Mathematics ................................................ 33

Table 10: Changes in percentages of students at or above proficient in the MSAs between 2008

and 2013 .................................................................................................................................. 35

Table 11: Results for the 2013 MSA disaggregated by grade and demographics ........................ 35

Table 12: Results of the comparison of means analysis (2013 MSA) .......................................... 36

Table 13: Effect size analysis for MSA Reading and Mathematics (SY 2009 and SY 2013) ...... 37

Table 14: Results in the MSA Reading and Mathematics for Grade 8 students disaggregated by

years in SAILSS (2012 and 2013 Cohorts)............................................................................. 38

Table 15: Percent of students with 5% or greater increases on the MSA mathematics ................ 39

Table 16: Changes in the 2012 and 2013 MSA mathematics for students at advance levels ....... 40

Table 17: Number of discipline referrals and referrals per student (SY 2009 to SY 2013) ......... 41

Table 18: Number of discipline referrals and referrals per student by grade (SY 2013) .............. 41

Table 19: Number of suspensions and average suspension per student (SY 2013) ...................... 42

Table 20: Number of suspensions and average suspension per student by grade (SY 2013) ....... 42

AACPS Project SAILSS – Final Evaluation Report Page | iv

LIST OF FIGURES

Figure 1: Proposed logic model for the SAILSS Project ................................................................ 6

Figure 2: Announcement of the Arts Showcase in February 2012 ............................................... 13

Figure 3: SAILSS' model for arts integration ............................................................................... 14

Figure 4: Use of arts integration (2010 Faculty Survey) .............................................................. 18

Figure 5: Influence of arts integration on student engagement (2010 Faculty Survey) ................ 19

Figure 6: Changes in responses to the student support scale between 2009 and 2012 ................. 22

Figure 7: Art as motivation to learning (percentage of agree/strongly agree responses) ............. 27

Figure 8: Percentage of Bates students disaggregated by race/ethnicity from 2005-2013 ........... 32

Figure 9: Percentage of FARM-eligible students at Bates across grant years .............................. 33

Figure 10: Percentage of students at or above proficient on the 2013 MSA Reading .................. 34

Figure 11: Percentage of students at or above proficient at the 2013 MSA Mathematics ............ 34

AACPS Project SAILSS – Final Evaluation Report Page | v

EXECUTIVE SUMMARY

In the summer of 2008, Anne Arundel County Public Schools (AACPS) applied for and

was awarded a grant from the U.S. Department of Education, Office of Innovation and

Improvement, to implement the Supporting Arts Integrated Learning for Student Success

(SAILSS) project. The project reflected a partnership among AACPS, Maryland Hall for the

Creative Arts, the Annapolis Arts Council, Arts Education in Maryland Schools Alliance

(AEMS), and Towson University. The grant extended from school year (SY) 2008-2009 through

SY 2012-2013. SAILSS project proposed a comprehensive cross-curricular, arts integrated

model for middle school students in high poverty, high mobility areas. The model included (1)

focused, systemic school-wide professional development; (2) mentoring of core content area

teachers by art teachers and artists in residence to build capacity related to arts integration; (3)

focused use of arts integration strategies to teach standards where students showed the greatest

challenges; and (4) involvement of families and community. RMC Research Corporation

conducted the evaluation of the program through its five years.

Main findings from the five grant years included:

Implementation

o All Bates teachers and administrators attended workshops and/or conferences related

to arts integration both off-campus and in-house; attendees to off-campus activities

presented to school faculty upon their return.

o Six Bates teachers completed the two-year Post-Baccalaureate Certificate in Arts

Integration at Towson University.

o All Bates teachers were offered opportunities to work side-by-side with artists in

residence and arts teachers to develop and implement arts integration lessons.

o Teacher mentoring followed three models – long-term (artists in residence worked

with specific teachers or classrooms for a full semester); short-term (artists in

residence worked with teachers and/or classrooms on a time-limited project); and

peer-to-peer (content area and arts teachers worked together on specific lessons).

o A position of Arts Integration Specialist (AIS) was created to help in the planning

and implementation of professional development, ensure availability of resources,

and support teachers as they designed and implemented arts integration lessons.

AACPS Project SAILSS – Final Evaluation Report Page | vi

o Approximately 700 arts integration lessons and projects for grades 6-8 were

developed and piloted; the lessons focused on standards where students

demonstrated greater difficulties and included core content areas, elective courses,

special education, and English as a Second Language.

o Through the grant years, the school implemented quarterly showcases and yearly

festivals as vehicles to display student work and involve the community. The

presentations were attended by families, members of the Board of Education and

County Board, invited artists and partners.

o The project director, the AIS, and Bates teachers presented in state and national

conferences and workshops to disseminate the model.

Intermediate Outcomes

o Creation of an Arts integration Specialist position at the central office to help arts

integration across all AACPS schools.

o Change in teachers’ perceptions – higher acceptance of arts integration as a viable

instructional strategy for improving student engagement, student ability to make

connections with other disciplines and overall academic performance.

o Changes in teachers’ practices – arts integration became an integral part of the

strategies available to teachers – the “Bates way” to address student’s needs.

o Changes in students’ engagement in learning – through the years, responses to the

students’ surveys indicated increased student engagement and better understanding

of concepts, particularly in English and Mathematics, when learning through arts.

o Classroom observations showed the majority of students actively engaged and

collaborating with each other.

o Attendance – Although already high at the beginning of the grant, attendance rate (as

percentage of days enrolled) has grown and is now 96.4%; attendance increase at

Bates was slightly larger than that at the comparison school.

Long-term outcomes

o Results in the 2013 Maryland State Assessments (MSA) Reading indicated that

Bates’ students outscored their peers in the comparison school and state averages in

all three grade levels.

AACPS Project SAILSS – Final Evaluation Report Page | vii

o At the 2013 MSA Mathematics, students at Bates did better than students in the

comparison school in grades 6, 7 and 8. For grade 7, Bates’ students outscored the

district and the state percentages.

o When Bates data are disaggregated by student subgroup, FARM-eligible students

scored lower than their non-Farm peers. Likewise, students from the two largest

minority groups (African American and Hispanic/Latino) tended to score lower than

White students. Mean differences were statistically significant with 95% confidence

level.

o An effect size analysis was conducted on the mean scores for the MSAs Reading and

Mathematics between 2009 and 2013. For Bates, medium effect size was found in

grades 6 and 7 Reading, and grade 7 Mathematics, while a small effect size was found

for grade 8 Reading. Grade 8 Mathematics showed no program effect on test results.

The comparison school, which represents a traditional approach (without arts

integration) showed negligible or negative effect sizes for all grades in Mathematics

and grades 6 and 8 in Reading. A small effect size was found in grade 7 Reading.

Therefore, arts integration appears to be more effective in increasing student learning

as measured by state assessments, in contrast to teaching strategies that exclude arts

integration.

o A cohort analysis was conducted on the MSAs results for students who have been in

Bates for one, two and three years. Results from the analysis suggested that time in

the school had no influence on test results.

o The analysis of disciplinary referrals showed improvements in both the treatment

(Bates) and the comparison schools. Between 2009 and 2013, discipline referrals in

Bates declined by 80% (from 746 to 152), and by 82% at the comparison school

(from 674 to 122). However, within the same time period, the number of suspensions

per student declined by 71% in Bates and 52% at the comparison school. These

results suggest that although the numbers of disciplinary referrals have declined

equally at both schools, the severity of referrals declined further at Bates (thus the

smaller number of suspensions).

AACPS Project SAILSS – Final Evaluation Report Page | 1

INTRODUCTION

The Arts in Education—Model Development and Dissemination (AEMDD) grant

program was authorized under the No Child Left Behind Act of 2001 (Public Law 107-110), Title

V, Part D, Subpart 15, Sec. 5551. The program is managed by the Office of Innovation and

Improvement (OII) at the U.S. Department of Education (USDE). Using a competitive grant

system, AEMDD provides funds to local education agencies and nonprofit or governmental arts

organizations to “support the enhancement, expansion, documentation, evaluation, and

dissemination of innovative, cohesive models that demonstrate effectiveness” to attain its goals

(USDE, 2008, p. 222). These goals are: (1) integrate standards-based art education into the core

elementary and middle school curricula; (2) strengthen standards-based arts instruction in those

grades; and (3) improve students’ academic performance, including their skills in creating,

performing, and responding to the arts. Applicants are encouraged to form partnerships with

school districts, independent schools (including charters), arts organizations, community

organizations, and/or institutions of higher education.

In the summer of 2008, Anne Arundel County Public Schools (AACPS) applied for and

was awarded a four-year AEMDD grant (award number U351D080013). The AACPS project,

Supporting Arts Integrated Learning for Student Success (SAILSS), reflected a partnership

among AACPS, Maryland Hall for the Creative Arts, the Annapolis Arts Council, Arts

Education in Maryland Schools Alliance (AEMS), and Towson University. RMC Research

Corporation was included in the proposal as the external evaluator for the project. At the end of

fourth project year, AACPS was awarded a one year no-cost extension. Therefore, the project

extended from school year (SY) 2008-2009 (2009) through SY 2012-2013 (2013).

The SAILSS project proposed a comprehensive cross-curricular, arts-integrated approach

for middle school students in high poverty, high mobility areas. The vision for the project was

based on the assumption that art, in its various forms, engage students’ sensory, intellectual,

physical, social, aesthetic, and emotional aptitudes. Consequently, art has the potential to

improve the learning process for all students.

Founded upon this theory of change, SAILSS was implemented in one middle school in

Anne Arundel County, Maryland, with the goal of improving mathematics learning for students

from grades 6 through 8. The project relied upon a four-prong foundation that included: (1)


professional development for all teachers in the school; (2) mentoring of core content area

teachers by art teachers and artists in residence to build capacity related to arts integration; (3)

focused use of arts integration strategies to teach standards where students showed the greatest

challenges; and (4) involvement of families and community.

This report reflects findings from evaluation activities conducted during the grant years,

from September 2008 through June 2013. The report focuses on themes rather than activities.

Descriptions of each evaluation activity implemented during the project years and their findings

have been detailed in the yearly evaluation reports, which are available at AACPS and/or RMC.

The report comprises five sections and an appendix, as such:

Background: Introduces the grantee and provides an overview of the project with a focus

on its theory of change.

Implementation Outputs: Summarizes the activities implemented and their immediate

outcomes (outputs).

Intermediate Outcomes: Describes changes in school organization, teachers’ perceptions

and school climate during the project years.

Project Impact: Analyzes the impact of the project on student behavior and academic

performances using a matched comparison design.

Conclusions: Provides an overview of the project implementation and discusses lessons

learned.

Appendix A: Describes the evaluation design and the methods for data collection and

analysis.

This report was written for a diverse audience that includes implementers, funders,

educators, and families. The purpose of the report is to provide readers with an overview of the

implementation of an arts integration project and its impact on teachers, students, and the school.

Graphics and tables are used to highlight results and to maintain the narrative flow. Technical

information is placed in the appendices.


BACKGROUND

The Grantee

Anne Arundel County Public Schools (AACPS) is the 5th

largest school district in

Maryland and one of the 50th

largest in the country. In 2008, at the time of the grant application,

AACPS was serving approximately 74,000 students in more than 120 schools. AACPS schools

are organized in feeder systems such that a number of elementary schools send their students to

two or three middle schools, which then feed into one high school. The school involved in the

SAILSS project, Wiley H. Bates Middle School (herein referred to as Bates), is part of the

Annapolis High School feeder system, which serves the eastern part of the county. This area is

characterized by wide socio-economic and cultural diversity. The school, located in Annapolis,

the state capital, is surrounded by five federal housing projects. Four of the five elementary

schools that feed into Bates are Title I schools.

In 2008, Bates served 567 students in grades 6 through 8. Half of the student population

was African American, 30% Caucasian, 15% Hispanic, and 3% Asian. Students who qualified

for Free and Reduced Meals (FARM) program comprised 45% of the school population and 17%

received special education services. Of the 45 teachers in the school, 44% held advanced

professional certification, 44% held the standard professional certification, and 12% had

provisional certifications. The school had not made Adequate Yearly Progress (AYP) for two

consecutive years and was classified as a school in need of improvement. The percentage of

Bates students who passed the Maryland School Assessment (MSA) was on average 13

percentage points below the district levels for both reading and mathematics, as displayed in

Table 1.

Table 1: Average results for statewide assessments for Bates and AACPS

SCHOOL

YEAR CONTENT

GRADE 6 GRADE 7 GRADE 8

Bates District Bates District Bates District

2006-2007 Reading 65.0 79.4 55.2 74.6 46.5 73.5

Mathematics 63.2 78.9 54.3 71.5 51.9 69.2

2007-2008 Reading 64.5 83.2 71.6 83.6 57.1 76.0

Mathematics 67.9 82.0 61.7 78.9 67.6 72.6

Source: Maryland Report Card website, http://www.mdreportcard.org/

http://www.mdreportcard.org/


The Project

Burnaford (2007) placed the origin of arts integration in the early 1900s, when educators

started to challenge the organization of the secondary school curriculum as a group of insulated

content areas. Proposals to reform secondary schools included reorganizing the curricula by

themes, rather than content areas; embedding interdisciplinary topics into content-specific

curricula; or adopting a full integration model. Gardner’s (1983) theory of multiple intelligences

was instrumental to the arts integration movement. In the early 1980s, Gardner advanced the

concept that human beings have different intelligences that account for their broader range of

potential. He identified eight types of intelligence: linguistic, mathematical, spatial, kinesthetic,

musical, interpersonal, intrapersonal, and naturalist. His theory highlighted the need to reach

students’ different intelligences by diversifying teaching methods (Armstrong, 2006). Art, with

its many forms (visual, dance, music, drama), addressed this need for diversification (Nelson,

2001). The integration of art into the academic content curricula seemed a logical approach to

address the variety of students’ intelligences that are reflected in their different learning styles.

SAILSS involved the design and implementation of a school-wide instructional model

that integrated the learning of different art forms into the core content areas. The project was

based on a triad of teacher-driven activities that included sustained and focused professional

development, curriculum writing, and curriculum implementation. Art-related enrichment

activities were also planned to bring the project beyond the school walls and involve the

community.

The SAILSS proposal described a number of professional development activities that

included three-week summer sessions in the first two years of the grant, access to Towson

University post-Baccalaureate Arts Integration Certification, and attendance at workshops,

conferences, and field trips. During the summer sessions, teachers were expected to develop and

pilot test lessons that integrated core content topics and various forms of art, including visual and

technical arts, dance, drama, and music. The lessons would incorporate interdisciplinary topics

and align with the AACPS Curriculum Frameworks and the Maryland State Voluntary

Curriculum. All Bates teachers were welcomed to the activities, although the emphasis was on

mathematics teachers. During the school year, arts teachers and artists in residence were to work

side by side with core content area teachers to implement arts integrated lessons and co-teach

lessons to expand the repertoire of arts integration techniques. Grant funds were geared toward


tuition and fees, stipends for attendance at professional development and curriculum writing

activities outside of regular work hours, substitute teachers when activities coincided with school

hours, and purchase of resources needed to implement the arts integration lessons.

SAILSS emphasized student participation in art-related extracurricular activities, such as

drama club, creative writing club, orchestra, and others. Students were offered a one-week art

summer camp that included, among other activities, the piloting of arts integration lesson plans.

Student work was expected to be showcased at the school and at additional community locations.

Annual art celebrations and related activities were planned to involve families and other

community members.

Partnerships with arts organizations, such as the Maryland Hall for the Creative Arts, the

Annapolis Arts Council, and others, were an integral part of the project. The purpose of these

partnerships was to bring a variety of enrichment opportunities to school staff, students, and the

community, such as fieldtrips to art museums and exhibits, workshops and speakers.

The project, as proposed to the USDE, had five objectives that were reduced to four as

more analysis occurred. The four objectives included:

By the end of each academic year, 100% of the students (500 students) at Bates will be

engaged in instructional activities designed to reflect their diverse learning styles and

multiple intelligences, as measured by teacher activity logs.

At least 20% of all Bate students will participate yearly in extracurricular art-related

activities, countywide music and dance competitions, and art portfolios, starting in the

2009 academic year, as measured by enrollment logs.

At least 40% of all Bates students will increase their scores on the MSA in mathematics

by 5% yearly, starting with the 2009 MSA in mathematics.

At the end of each academic year, starting with 2009, the number of discipline referrals at

Bates will decline by at least 5%.

The external evaluation was embedded into the project as a feedback mechanism to

inform the decision-making process. Figure 1, on the next page, displays the project’s logic

model and the role of the evaluation within the model.


Figure 1: Proposed logic model for the SAILSS Project

Supporting Arts Integration for Student Success (SAILSS) Project Logic Model

Resources

Par

tner

sE

du

cato

rsS

tud

ents

AACPS

Engagement and

Collaboration

Arts in Education

Partnership

Network

Facilities,

Equipment and

Materials

Educators

Skilled in Arts

Integration

Evidence-Based

Curriculum

Partners

Increased Commitment to

Arts in Education

Educators

Increased/Enhanced Arts

Integration Practices and

Techniques Among

Educators

Increased Educator

Perceptions of

Psychosocial Dimensions

in the School

Environment

Students

Arts Integration

Products and

Reflections

Educators

Curriculum

and Program

Development

Number

of Educators

Participating in

Professional

Development

Activities

Partners

Provide

Professional

Development

for Educators

Provide

Fieldtrips for

Educators and

Students

Educators

Attend

Professional

Development

Attend

Fieldtrips

to Art

Museums

Students

Perform

Classroom

Arts

Integration

Activities

Attend

Class

Fieldtrips

to Art

Museums

Partners

& Educators

Educators

Skilled and

Competent in

Arts

Integration

Practices and

Techniques

Educators

Applying

Integration

Practices and

Techniques in

the Classroom

Students

Students with

Positive

Attitudes

Toward

Learning

Activities Outputs Outcomes

Partners &

Educators

Partners &

Students

Number

of Students

Participating in Art

Activities Outside of

School

Impact

EvaluationFeedback Feedback

Students

Student Improved

Academic Performance

Decreased Discipline

Referrals and

Suspensions


Classroom Observations Instrument 7

IMPLEMENTATION OUTPUTS

This section focuses on the immediate results from the implementation of SAILSS.

Information was provided by the AIS or collected by the evaluators through classroom

observations and interviews.

Preparing Teachers for Arts Integration

1. Off-campus professional development activities

SAILSS funds allowed Bates faculty to attend more than 20 off-campus PD activities

across the project years. The activities included visits to arts museums and exhibits accompanied

by workshops. Upon return, attendees were required to share the information with school faculty

during staff meetings or professional development days. Attendees included administrators,

department chairs and core content area and art teachers. Grant funds paid for fees, per diem and

substitute teachers (when appropriate) for more than 100 individual participations in off-campus

workshops and conferences. In the fourth project year alone 22 Bates teachers (unduplicated

numbers), in addition to interns, other school staff, and other AACPS staff, attended workshops

and conferences in the Washington Metropolitan area for a total of 64 attendees.

During the first project year (2008-09), off-campus activities focused on mathematics and

involved teachers and students from grades 6 to 8. Workshops and field trips included the

Walters Arts Museum, in Baltimore for a presentation on Mosaic, Mummies and Math; the

Baltimore Museum of Art for a tour accompanied of a lecture on Math at the Museum; and the

National Building Museum in Washington, D.C., to learn about Architecture and Geodesic

Domes.

For the second project year (2009-2010), attendees explored different opportunities that

included the Crossing Borders Conference at the University of Maryland, Baltimore Campus

(UMBC); a meeting with the Maryland Arts Integration Network (MAIN); the Neuro-education

Summit at the Johns Hopkins University; the AEMS Tri County Conference (which involved

faculty from Montgomery, Howard and Anne Arundel Counties); the Picturing America

Workshop at the Philadelphia Museum; and Exploring the Early Americas at the Library of


Congress. Additionally, Bates hosted the 21st Century Institute, with the participation of the tri-

county consortium and MAIN.

The third project year (2010-2011) provided a mix of content-focused and systemic-

focused activities. Content-focused workshops included the Mathematics and Art, at the

Baltimore Museum of Art, and the Artifact Analysis at The Walters Art Museum, also in

Baltimore. Systemic perspectives were provided in the Changing Education through Arts

(CETA) workshops at the Kennedy Center, in Washington D.C, the Re-envisioning Education, at

the UMBC, and the AEMS Tri County Conference. The 21st Century Institute, initiated the

previous year under SAILSS’ auspices, became a self-sustained activity sponsored by AACPS

(from general funds) and partners. The Institute provides arts integration workshops to teachers

from Bates and other schools in and outside the county.

The fourth project year (SY 2011-2012) included attendance to the 21st Century Arts

Integration Institute, the Institute at the Freer Gallery, the Teaching Artist Institute III at the

Young Audience Arts for Learning Maryland (YAMD), and the Engaging with Arts Education at

Towson University. Workshops included Digital Viewing at the National Postal Museum, Story

of America Through Art at the Smithsonian American Art Museum, Cherry Blossoms lecture at

the Library of Congress, African Americans in the National Portrait Gallery at the National

Portrait Gallery, Washington, DC, and African American Artists at the Smithsonian American

Art Museum. Additionally, six Bates teachers completed the post-Baccalaureate certification in

arts integration, at Towson University. The cohort comprised the AIS, two reading teachers, two

English Language Arts teachers, and a mathematics teacher. The certification holders worked as

mentors to other teachers during the development and implementation of arts integration lessons.

2. In-house professional development activities

At the beginning of the second project year, Bates teachers were offered in-house

sessions provided by the AIS, artists in residence, and other arts integration experts. Four

sessions were provided per day to give all Bates teachers the opportunity to attend the

workshops. For the 2009-2010 project year, workshops included Introduction to Arts Integration,

Digital Documentation, Elements of Drama, Elements of Dance, Drama Games, Artful Thinking,

Artful Thinking and Higher Order Thinking Skills, and Bring your Best Arts Integration

Reflection. The Arts Integration Fridays continued in 2010-2011 with topics that included

Introduction to Arts Integration Resources, Digital Documentation Basic and Advanced, Priming


with Arts Integration, Drama Games, Music in your Classroom, Photography and Resources, Art

and Drama in the Classroom, Islamic Art and Culture, and Beyond I See, I Think and I Wonder.

Moving to Thursdays for the fourth project year, the workshops included Principles of

Design, Dance, Rap and Poetry, Artful Thinking, Using Digital Cameras in Classroom, Art

Inspire Writing in the Curriculum, Technology and Arts Integration. In addition to these

activities provided by Bates teachers, presentations from the artists in residence included

Movement, Process Drama, Summary Songs, and Pottery and Text. Arts teachers at Bates

worked with core content area teachers to model arts integration lessons with topics such as Jazz

and Harlem Renaissance, Hip-Hop Geometry, Printing Spanish, Pop Art Spanish, Math Scale

Drawings, and Science Watercolor. All content area teachers were also involved in half-day

hands-on activities related to arts integration.

During the final project year, in-house teachers provided workshops on Orientation to

Arts Integration in the Classroom. Presentations from artists in residence included: Arianna Ross

presented on Storytelling, Ali Kruger discussed Drama Games, Tim Gregory and Laura Groo

presented on Arts Integration and Common Core, and Michael Lamason presented on Story

Boarding.

The off-campus activity provided lead teachers and interested teachers at Bates with

opportunities to hear from experts and update their knowledge of arts integration. Concomitantly,

in-house activities introduced all teachers to the basics of arts integration and provided them with

knowledge and support in order to plan and implement lessons integrating arts.

3. Modeling Arts Integration: Artists in Residence and Arts Teachers

SAILSS used three models of mentoring: long-term, short-term, and peer-to-peer. The

two first models involved artists in residence. In the long-term model, the artist in residence

worked for a couple of months or a semester side by side with teachers. The objective was to

provide enough time for teachers and students to build rapport with the artist, and for the artist to

know the school and gradually build teachers’ capacity in arts integration. In the short-term

model, the artists came to the school for a couple of weeks and worked directly with a classroom

or a group of classrooms on a focused, specific project. The peer-to-peer model involved Bates

arts teachers working together use the core content area teachers to create and model arts

integration lessons


As an example of the long-term model, during SY 2008-09, Robert Levit worked year-

round with mathematics and art teachers to develop a variety of projects that involved

mathematics, music and visual arts. Examples of the short-term model for the same year

included: the artist David Cunningham and students created a large acrylic mural that adorns the

lobby of the school cafeteria on a project that involved mathematics and visual arts; Footworks

Percussive Dance Company provided a 10-day workshop to highlight the contributions of Irish

and African communities to the American culture – the workshop concluded with students

presenting to the community; Bob Benson, from Shiny Happy Things, worked with teachers to

create a mirror tree that integrated mathematics and visual arts. The tree is in permanent display

at the Bates Arts Center.

Bob Benson and Robert Levit returned for the second project year. Benson helped

mathematics teachers integrate glass sculpture into their lessons; while Levit created 13 lessons

integrating music and visual arts into mathematics. Other artists in residence who came to Bates

on short-term assignments included: Ali Oliver, who worked with the sixth grade classes to

integrate social studies and creative opera and drama; Ryan Sellers, who helped the seventh

grade English language arts class use mime and tableau to interpret myths; Christina Delgado,

who worked with the social studies classes on the use of photography to define culture; and

Amanda Pellerin, who integrated visual arts and Italian language into social studies classes.

During SY 2010-11, Robert Levit continued his collaboration with the mathematics

teachers for the entire school year. Kevin Reese, in collaboration with the AIS and mathematic

teachers, implemented a one-week residency with the students to integrate concepts of

engineering and visual arts. The result of this collaboration was the 15 foot sculpture of brushed

aluminum and wood that adorns the school entrance. As part of the in-house mentoring, the AIS

facilitated collaboration between art teachers – including visual arts, dance, drama and music –

and teachers from other content areas to create and implement arts integration lessons.

Six artists in residence worked with Bates teachers and students during SY 2011-12. For

two months, Kevin Martin helped integrate music into science and English lessons with themes

as Physics of Steel Drums and Steel Drums in Caribe Culture. Ali Oliver returned for a full

month to help social studies teachers and students create two operas on Qin Dynasty and

Mesoamerica. Other artists who stayed for two months working with core content area teachers

included Joan Gaither (visual arts), Tim Gregory (music and movement), and Sue Trainer


(music). Footworks returned for a workshop on American Diversity with social studies teachers

and students for grades 7 and 8. Visual arts and dance teachers at Bates co-taught about a dozen

lessons that integrated arts into social studies, English, mathematics, and foreign languages.

From Professional Development to Classroom Application

Since the start of the grant in SY 2008-09, time for collaborative planning was scheduled

for the development of arts integrated lessons and projects that involved teachers from core

content areas and art teachers or artists in residence. The teachers used the time to model the

lessons with each other and pilot with students. On many occasions, the AIS and arts experts

taught the lessons alongside the core content area teacher. Three teachers from Bates served on

district-wide curriculum writing teams that were responsible for the development of arts

integration units for Grades 6 and 8. The units are available for use by all AACPS middle school

teachers. It is of note that the process, while geared toward core content area teachers, ended up

by involving teachers from elective courses and specialized areas.

Teachers were encouraged to use arts integration lessons to target those standards where

students traditionally faltered. Department chairs, working with the AIS, were instrumental in

supporting faculty in these activities and in providing them with time for planning and review.

The AIS collected data from the quarterly benchmark tests and reported on the outcomes for the

targeted standards. Table 2, on the next page, displays information on benchmark results related

to standards taught with arts integration (from a 2012 presentation).

About 700 arts integration lessons and projects were developed and piloted during the

five project years. Lessons reflected all arts forms (visual, music, dance, drama) and all content

areas, including core areas (English, mathematics, science and social studies), foreign languages,

and specialized areas, such as English as a Second Language (ESOL) and special education. Arts

integration lessons also represented all grade levels. During the fourth grant period alone, 180

lessons were developed and implemented. Examples of such lessons include: Mehendi Art

Negative/Positive Spatial and Celestial Bodies, a collaboration between grade 6 science teachers

and the dance teacher; Area of Trapezoids with Bearden Collage, a lesson that integrate seventh

grade mathematics and visual arts; Silhouettes and Writing: Guatemalan Worry Dolls,

collaboration between the ESOL teacher and visual arts teacher; Harlem Renaissance Literature

and the Jazz Connection, for students in Grade 8, integrating English, social studies, and music;


Tableau – How to Show Angle Relationships and Find Missing Angles, integrating drama and

mathematics for students in grade 7.

Table 2: Analysis of standards targeted in arts integration lessons

Grade Arts integration

Activity Social studies Content Standard

Correct Answers

Pre Post

6 Monochromatic

Mapping/Value

2.C.1.a . Identify reasons why people migrate, such

as economic opportunity, climate, and political

reasons

32% 88%

7 Economies/Creative

Comparisons/Dance

4.B.1.a Examine how different economic systems,

traditional, command, market, and mixed, answer

the basic economic question of what, how, and for

whom to produce

31% 74%

8 Federalism/Visual

Arts

1.A.1.b Explain and summarize the principles of

federalism, and how they protect individual rights

and impact the functioning of government 22% 81%

8

War of 1812/

Musical

Expressions

5.C. 2.a. Explain why the US adopted a policy of

neutrality prior to the War of 1812.

5.C.2.b. Explain how the continuing conflict

between Great Britain and France influenced the

foreign policy of the United States

27% 86%

From the Classroom to the Neighborhood

One prong of SAILSS-sponsored activities had the objective of bringing families and

communities into the school to celebrate student work and create a sense of school pride. During

the five project years, the school promoted not only quarterly showcases and but also larger

celebrations, such as the Family Heritage and Multicultural Arts Integration Festival and the

International Arts Festival. All the activities involved displays of student work and student

presentations. Families, Board of Education members, politicians, and representatives of the

county and state administration were invited to attend, thus bringing much attention to the school

and arts integration. Figure 2 displays part of a brochure announcing the 2012 February

Quarterly Showcase.


Figure 2: Announcement of the Arts Showcase in February 2012

From Neighborhood to the Community

As Bates’ faculty became familiar with arts integration, a process of dissemination of the

proposed model was set in motion. This included presentations to faculty in other AACPS

schools and outside the school district. As early as the second project year, the cohort of Bates

teachers who were working toward their Arts Integration Certification at Towson University

presented to AACPS and Towson students on Art Forms and Artful Thinking. They also

presented at the Kennedy Center on Laying a Foundation: Defining Arts Integration and

Documenting Arts Integration.

During the third project year, for instance, Bates faculty presented before a total of 130

participants which included AACPS leadership, both principals and administrators, and attendees

to the Arts Education in Maryland Schools (AEMS) Conference, the Arts Integration Conference

at the University of Maryland, Baltimore Campus (UMBC), and the World Language County

Coordinators at the Maryland State Department of Education. Topics presented included Arts

Integration: Overview, Artful Thinking, Art of Storytelling, A Literacy Quartet and Arts

Integration and World Languages.

In the fourth project year, the AIS dedicated over 20 hours to presentations regarding arts

integration at diverse AACPS schools, the College Board Forum in New York, meetings for the

Pennsylvania and Maryland Foreign Language Associations, conferences for the National

Science Teachers and the American Council on the Teaching of Foreign Languages, and others.

Join us on Monday, February 13, 2012 to celebrate

Quarterly Arts Showcase

Black History Month

African American Parent Involvement Day

Valentine’s Day

The staff, students, and PTA invite all parents and guardians to join their Bates students to view our

artistic accomplishments and celebrate Black History Month.


Topics included Motivating with Arts Integration, How to Integrate the Arts, Artful Thinking, and

Thinking through Art.

In summary, SAILSS presented a model of arts integration that can be replicated in

schools in different settings. Figure 3 displays schematically the model of arts integration

proposed by SAILSS.

Figure 3: SAILSS' model for arts integration

Off-campus PD

Neighboring Community

Focused lessons

School District

Community at large

SYSTEMATIC SUSTAINED

FOCUSED


Bates has quarterly

benchmark tests, and we are

able to see what standard

was taught by arts

integration and it shows

improvement. The county is

very interested in the

results. The data is getting

teacher buy-in and the

principal is backing the

project 100%... Arts

Integration Specialist

INTERMEDIATE OUTCOMES

Intermediate outcomes are here defined as conditions that are expected to lead to changes

in academic performance. They include changes in school organization, teachers’ perceptions

and practices, and student engagement in learning. Data for this section were collected from

yearly faculty and student surveys, classroom observations, and interviews with project and

school staff. For details on the evaluation design and methods of data collection refer to

Appendix A. Findings from each independent activity are detailed in the evaluation reports

provided to AACPS in previous years. This section presents an overview of these activities and

changes through time.

Initially, surveys were conducted at Bates and a comparison school (School A). However,

School A started to implement arts integration at the very onset of SAILSS and soon became an

arts magnet school, thus invalidating any comparison. A second comparison school (School B)

was selected at the beginning of the third grant year but comparisons were limited to analysis of

student performance data.

Changes in School Organization

Changes in the school organization during the SAILSS

period included the creation of the Arts integration Specialist

(AIS) position and the opening of a magnet program in

Performing and Visual Arts (PVA). Both changes occurred in the

second project year (2009-10).

The AIS position was created to coordinate the schools’

activities and mentor teachers as they integrated arts into their

lessons. The role encompassed a number of functions that

included working with the teachers during the planning and

development of the lessons, helping in the implementation of the lessons through modeling and

co-teaching, and supporting the evaluation activities.

The teacher hired for the AIS position was working toward her Arts Integration

Baccalaureate Certification (which she completed). The position was fully supported with


SAILSS funds during the grant period, but AACPS was planning to sustain the position with

general funds after the grant expired. Eventually, the school district created a districtwide AIS

position to provide similar supports across its schools.

The PVA magnet program opened at Bates in September 2009. Technically, the program

is not an output from SAILSS, since it was planned before the grant was awarded. However,

according to PVA staff, SAILSS contributed to buy-in within and outside the school, and

facilitated the interaction between the magnet program and school wide activities.

PVA students come from feeder systems in the southern part of the county. As the name

indicates, the program focuses on visual and performing arts, and offers a variety of classes that

include instrumental music, dance, cinema, computer generated art, painting, sculpting, among

other activities. In addition to the core content areas (mathematics, English, science, social

studies), the program offers a course on World & Classical Languages with a focus on Italian.

Students must apply to the program and part of the application process includes an audition. The

students select one of four pathways: dance, instrumental music, visual arts and vocal arts.

Students who complete the three-year program gain high school credit in Algebra and foreign

languages. Qualified arts instructors and artists in residence provide studio instruction during the

Performance Plus sessions (Wednesdays and Thursdays from 3:30 to 4:30 pm). The program

also offers eight Saturday sessions that include field trips to museums and performances within

the Baltimore-Washington, DC area.

Interviews with teachers and parents suggest that the introduction of the magnet program

had initial challenges, as it created a divide between PVA and non-PVA students. SAILSS

helped this divide disappears, since all Bates students are involved with arts in core content

classes, and actively participate in showcases and festivals. SAILSS made “the playing field

more even,” commented a 7th

grade teacher.

During individual and group interviews, the teachers agreed that the transformation of

Bates into an arts integration school and the opening of the magnet program changed the way the

community sees the school. Bates is now perceived by the community as a school that provides

students with holistic and high quality educational opportunities. Parents who would previously

send their children to private schools are staying and the school has seen an influx of students

who are academically oriented. Applications for the magnet program are already exceeding the

slots and the school has been forced to institute a raffle system for acceptance. One teacher


explained that she left a position in the central office to teach at Bates because the school “has

developed a reputation for providing students with a unique opportunity to learn through the

arts,” and she wanted to be part of it.

In 2012 the school received three awards: The Award for Excellence and Innovation in

the Arts, from the College Board; the Maryland Schools of Excellence in Arts Education Award,

from the Arts Education in Maryland Schools Alliance (AEMS), and the Schools that Work

roster from Edutopia.

Changes in Teachers’ Perceptions

Bates faculty perceptions were assessed through two surveys: a faculty survey and a

school climate survey. The faculty survey, created by the AIS, focused on arts integration as a

teaching technique and its impact on teaching and learning. Using a pre/post design, the survey

was conducted in the fall and spring semesters of the second and third project years. The school

climate survey was conducted at the end of each project year and examined changes in

perceptions regarding the students and the school.1 Survey findings were complemented by

individual and group interviews with school faculty from different content areas and grade

levels.

1. Arts integration as a teaching strategy

Figure 4, on the next page, displays results from the pre/post survey conducted in the

third project year (2010). The responses indicate the percentage of teachers who gave moderate

to considerable ratings to the items. The survey results suggest that Bates teachers perceived

themselves as being moderately to considerably knowledgeable about arts integration, and saw

arts integration as a viable instructional strategy for improving achievement. They also felt

comfortable with integrating arts in their lessons. Yet, fewer than half of the survey respondents

perceived major changes in their teaching practices as a result of arts integration. Results were

similar for the 2009 and 2010 surveys.

1 Response rates for each survey were above 70%; for the faculty survey, paired-sample t-tests were used to compare

responses for a group of teachers who participated in the pre and post- application that year. T-tests and ANOVA

were used for the climate surveys. Analyses of the survey templates and findings are detailed in yearly evaluation

reports presented to AACPS.


Figure 4: Use of arts integration (2010 Faculty Survey)

During interviews, teachers commented that, as they started SAILSS, integrating arts into

their lessons “seemed daunting,” and the AIS was central to the success of the project. The AIS

was described as always supportive, planning lessons together, finding resources, modeling, and

helping them “in any way possible.” According to survey respondents, the two most important

obstacles to implementing arts integration were lack of time for planning and lack of training.

However, as SAILSS implementation matured, teachers became more familiar with arts

integration. One mathematics teacher commented during an interview,

We teach with the arts to find ways to get students interested and it becomes second

nature; it’s universal, it’s done everywhere in the school—there is no resistance.

2. Arts integration and student engagement

Teachers’ perceptions that student engagement increased in response to the use of arts

integration was a finding from surveys and interviews as early as the second grant year. The

consensus among teachers was that arts integration provided a natural way to differentiate

instruction and tended to engage most students, even those students who were traditionally

disengaged. Teachers from all grade levels commented during interviews that students responded

differently to arts. Students might resist arts activities that involved skills with which they did not

0%

20%

40%

60%

80%

100%

pre post pre post pre post pre post pre post

Knowledge aboutarts integration (AI)

Participation inprofessional

development aboutAI

Belief that AI is aviable instructional

strategy forimproving

achievement

Comfort level withintegrating the artsinto your classroom

Changes in teachingpractice as a result

of AIimplementation

57.4%

67.9%

48.0%

67.5%

81.3% 90.4%

77.1% 77.4%

47.9% 54.9%

Co

nsi

der

able

+Mo

der

ate

resp

on

ses


feel comfortable or which they saw as foolish or “uncool,” explained a teacher. Performance arts

activities were seen as particularly successful with hard-to-engage and shy students.

Alternatively, some teachers claimed that performance activities were too disruptive and

perceived their students responding better to “quieter” strategies, such as painting with

watercolors.

Interviewees suggested that lessons should provide options for students who were

reluctant to participate in one specific activity, such as inviting them to perform instead of

drawing, or vice versa. Overall, teachers agreed that their comfort level with arts influenced how

they utilized arts integration in the classroom and how well the students received it. As one

teacher commented, “if the teacher enjoys it, then the students will too.”

The pre/post teacher survey conducted in 2009 and 2010 included questions regarding

teachers’ perceptions of the influence of arts integration on students’ engagement in instructional

activities (engagement), ability to make connections across different content areas (connections),

motivation to learn (motivation), academic performance, and behavior. Figure 5 displays

“moderate to considerable positive” ratings for the survey conducted in the third grant year.

Figure 5: Influence of arts integration on student engagement (2010 Faculty Survey)

0%

20%

40%

60%

80%

100%

Pre Post Pre Post Pre Post Pre Post Pre Post

Engagement Connections Motivation Academicperformance

Behavior

84.5%

97.8%

66.7%

93.5%

82.2%

95.7%

68.2%

89.1%

77.8%

86.6%

Mo

der

ate

to L

arge

Po

siti

ve E

ffec

t (%

Res

po

nse

s)


I absolutely love arts

integration—it’s been

amazing…I was afraid and

uncomfortable at first, but then

the kids started teaching me.

We had an artist in residence

to teach steel drums. [A special

needs student] asked to be in

the final performance. He had

never participated in anything!

In another occasion, we

worked on describing scenes

and surroundings, and we

“stepped into a painting.”

Everybody was participating.

It was one of those moments

I’ll never forget. It gives the

students a feeling that they can

[learn]. Teacher, first year in

Bates

. A comparison of means test for the 2010 survey indicated robust increase between the

pre and post-application for engagement (t=2.524, df=31, p<0.017), connections (t=3.350, df=31,

p<0.002) and academic performance (t=2.048, df=31; p<.002). That is, within one year of using

arts integration, teachers were considerably more likely to perceive these techniques as

improving moderately to considerably students’ engagement, their ability to make connections

with other disciplines, and their academic performance. Teachers’ perception of the program

influence on student behavior was already high (78%) in the pre-survey.

3. Arts integration and school climate

Research indicates that teachers in art-rich schools demonstrated more interest in their

work, were more likely to become involved in professional development activities, and were

more innovative than teachers in art-poor schools (Burton, Horowitz, and Abeles, 1999). The

climate survey tested whether SAILSS would build a more congenial climate at Bates. The

survey, conducted in May of the first (2009), second (2010) and fourth (2012) project years, used

the School-Level Environment Questionnaire (SLEQ) developed by Fisher and Fraser (1990).2

The instrument includes eight scales: student support, affiliation, professional interest,

participation in decision-making, innovation, resource adequacy, work pressure and staff

freedom. A detailed discussion of the instruments and the

results through the years is found in the 2012 Evaluation

Report.

Overall, Bates faculty scored quite high in the

affiliation, professional interest, innovation and resource

adequacy scales since the first survey (May 2009). Faculty at

Bates felt part of a broader community that was supportive

and easily integrated new members (affiliation scale). They

also perceived the school as a place that encouraged

professional growth and collaboration (professional

development scale), and inspired creativity and innovation

(innovation scale). The majority of these perceptions

remained relatively constant across the years. However,

2 The SLEQ was used in this project with permission of the authors.


perceptions related to the professional development scale, albeit already strongly positive in the

2009 survey, showed robust improvements between the first and the fourth project years,

particularly for three statements: Teachers are keen to learn from each other – 2009 mean was

3.68 vs. 4.06 in 2012 (t=2.176; df=83; p<0.032); Teachers avoid talking with each other about

teaching and learning – mean of 1.97 in 2009 and 1.67 in 2012 (t=2.272; df=83; p<0.026); and

Teachers frequently discuss teaching methods and strategies with each other – mean of 3.82 in

2009 and 4.31 in 2012 (t=3.192; df=83; p<0.002). Compared to the beginning of SAILSS,

teachers in 2012 were more likely to see the school as strongly embracing professional growth.

Two scales showed lower means between the first and the fourth project years: resources

adequacy and work pressure. The resources adequacy scale measures teachers’ perceptions about

availability of resources in the school. Fewer teachers tended to perceive the schools as having

appropriate resources in 2012, when compared to 2009. However, these results may suggest

rather than a decline in resources, an increase in the demand for them, as more teachers utilize a

variety of materials for arts integration.

The work pressure scale measures how teachers perceive the pressure in their daily

working routine. Although responses for most items in this scale were similar for the two

surveys, increased agreement was found in one statement – There is constant pressure to keep

working. Mean response for this item in 2009 was 3.50 and 3.96 in 2012 (t=2.326; df=83;

p<0.022). Feelings of pressure may be a result of the large number of new teachers in the school,

the increase in student population, or changes in demand from central office. Yet, it can also be a

result from the longer time required to plan arts integration lessons, at least for first year

teachers, a comment heard in the interviews.

The participatory decision making scale measures the faculty’ perceptions regarding their

level of participation in the governance of the school. Responses to items in this scale showed

some ambivalence. Regarding decisions about the running of the school, more teachers perceived

that they were involved in the process in 2012, when compared to 2009. Regarding decisions

about teaching, more teachers indicated that they need to ask permission to the department head

before making decisions about teaching in 2012 when compared to 2009. That is, results from

the fourth project year suggest that Bates teachers see themselves as having more autonomy

regarding school governance than regarding decisions about instruction. However, large


percentages of “not sure” responses (above 10% for most statements) challenge any conclusion

based on the small numbers of respondents (55 teachers, 85% response rate).

The only scale that showed strong improvement in the climate survey between the first

and the fourth project years was the student support scale. Figure 6 displays the percentages of

agree/strongly agree responses in the student support scale for the 2009 and 2012 surveys.

Figure 6: Changes in responses to the student support scale between 2009 and 2012

Findings from the 2009 and 2010 climate surveys suggested that teachers at Bates had a

more pessimistic view of their students, when compared to a similar school that was being used

as comparison. In the 2009 survey, items that used adverse words to describe students (e.g.

disruptive, difficult, noisy, badly behaved, or needing strict discipline) tended to receive a higher

There are many disruptive, difficult students in theschool.

Most students are helpful and cooperative to teachers.

Most students are pleasant and friendly to teachers.

There are many noisy, badly behaved students.

Students get along well with teachers.

Most students are well-mannered and respectful to theschool staff

Very strict discipline is needed to control many of thestudents.

52.9%

70.5%

72.6%

39.2%

82.3%

66.7%

56.9%

83.4%

43.8%

39.6%

79.2%

43.8%

22.8%

85.5%

Agree/Strongly Agree Responses

2009

2010

2012


percentage of agree and strongly agree responses from Bates teachers than statements that used

constructive words (e.g. pleasant, friendly, helpful, well-mannered, respectful). In the 2012

climate survey, perceptions about the students had changed considerably. For instance, in the

2009 survey 83% of the teachers agreed/strongly agreed that there were many disruptive,

difficult students in the school. The percentage of agreed/strongly agreed responses for this item

fell to 56% in 2012, a 33% decline. Alternatively, the agree/strongly agree responses for the

statement “students get along well with their teachers” increased from 44% to 83%, a 47%

increase.

As displayed in Table 3, all seven items pertaining to the student support scale elicited

robust differences in responses between the two surveys, with statistical significance established

at 95% confidence level. By the fourth project year, Bates teachers were likely to agree/strongly

agree that their students were helpful, cooperative, pleasant, well-mannered and respectful.

Table 3: Comparison of means – student support scale (2009 and 2012 surveys)

STATEMENTS

Mean* t-test

2009 2012 t df p

There are many disruptive, difficult students in the

school. 4.29 3.35 4.481 83 0.000

Most students are helpful and cooperative to teachers. 3.03 3.57 2.458 83 0.016

Most students are pleasant and friendly to teachers. 3.18 3.69 2.408 83 0.018

There are many noisy, badly behaved students. 3.91 3.14 3.503 83 0.001

Students get along well with teachers. 3.06 3.80 4.715 83 0.000

Most students are well-mannered and respectful to the

school staff 2.65 3.49 3.730 83 0.000

Very strict discipline is needed to control many of the

students. 4.18 3.57 3.092 83 0.003

*Higher means indicate greater numbers of agree or strongly agree responses adverse wording statements are

expected to have lower means (more disagree/strongly disagree responses)

Changes in Teaching Practices

Already during the first school visit, in April 2009, the evaluators observed that the

classrooms and halls of the school displayed a number of reproductions of works by renowned

artists as well as students’ arts integration projects. Of the 14 teachers observed, 11 reported that

they had documented the arts integration activities implemented since the beginning of the

school year. They were also planning to showcase the student work throughout the school and


Students used to display

avoidance behavior in class,

but now they want to stay. We

are working on a science

project about water

conservation here and in

Kenya. We are comparing

water levels and using colors

to signify shortages. Then

students will weave together

the color patterns into Kente

cloth. Some students have

even taken it a step further—

they are developing their own

water conservation plan for

the school. It’s forcing kids to

think on a higher level.

Science teacher

use the work for instructional purposes in future classes. Art-related activities were integrated

into the core content area lessons in 12 of the 14 classrooms observed, including classes where

arts integration were not supposed to occur. The activities involved different forms of art. For

instance, in an English class students were asked to read a play and recreate scenes by using

musical instruments and ambient sounds. Students in a mathematics class were painting triangles

on the color wheel that corresponded to equations. An artist in residence was helping a social

studies teacher to explain the Irish famine through dance.

During the second project year, the evaluators visited 18

classrooms. At the time, student arts integration products were

displayed across the school, starting with the entrance lobby.

Arts – dance, drama, drawing, poetry, writing, photography –

were being integrated in 17 of the 18 classrooms observed. In

one English class, students were asked to discuss moods in the

colors of a painting and draw pictures that corresponded to the

poem they had finished reading, while in another English class,

students were asked to sing a poem and act out Latin roots of

the words learned. In two math classes, students learned how to

increase the proportions of a portrait and draw it to scale, and in

a third class, students were identifying patterns in artwork and

creating bracelets by following the pattern. In the Italian Language class, students were asked to

identify words that described different portraits, while in the Spanish Language class the students

created sentences to describe picture or role-played with partners on representing words.

Teachers continually invited students to participate in the lesson and, in all but one class,

learning in or through the arts was promoted. Teachers used modeling about 83% of the

classroom time, and made frequent connections with students’ previous learning and between

disciplines. Examples from the arts and other disciplines were used in 17 classes for illustration

or demonstration.

Two mini-case studies were conducted during the spring semester of 2010 to investigate

the implementation of arts integration at Bates and its impact on instruction. The observation

included four 7th

grade pre-algebra classes and two 6th

grade English classes. Two pre-algebra

and one English teacher taught the classes, with the agreement that they would use arts


Arts gives this school life, energy and respect…

it engages kids in a common interest—everyone

is equal. Arts integration is the tipping point of

bringing a broader range of kids back to the

school. PTA member

integration in one class (AI class) but not in the other (NAI class). The evaluators dropped the

observations of the English class in the second visit, when it became clear that arts integration

techniques were being used in both classrooms. Regarding the pre-algebra classes, AI classes

used more project-based strategies than NAI classes and therefore, student collaboration was

more frequent. Additionally, AI classes were more likely to connect the information being

learned to students’ previous learning experience, when compared to NAI classes. No significant

differences in benchmarks results were found between AI and NAI classes.

During SY 2010-11, another mini-study was conducted with three different teachers in

the mathematics department: a first year teacher, a veteran teacher who was seen by project staff

as resistant to arts integration, and another veteran teacher who was supportive of the project.

Pre-algebra and algebra classes were observed. Across the observed classes, all three teachers

showed similar levels of arts integration (about 10% of their class time). Three different arts

integration activities were observed during the visits: identify solutions of equations in a

Kandinsky painting; make metaphorical statements involving inequality relationships in a

Japanese painting, and draw relationships from a Harlem Renaissance painting. The activities

aimed at encouraging students to think creatively while making statements informed by

mathematics and logic. The teachers made comments about the paintings and urged students to

consider the cultural impacts of the Harlem Renaissance, thus introducing social studies themes

into mathematics. Two of the observed teachers also used contemporary songs to help students

remember the difference with inequalities.

Observed differences in student

behavior and participation were related to

content topic rather than use of arts

integration. Students in Algebra classes tended

to be more engaged than those in Algebra Readiness (pre-algebra) regardless of the teacher.

However, the more teachers integrated arts into their lessons, the more collaboration, critical

thinking and connection to previous learning experiences were observed. Benchmarks results

were not statistically different across teachers when content area was taken into account;

improvements were greater for the Algebra students than the Algebra Readiness students

regardless of teacher (Details of the mini-case studies are included in the 2010 and 2011

evaluation reports).


The findings in the mini-study were corroborated by a series of group interviews

conducted during the spring semester of 2011. Teachers from all grade levels and content areas

agreed that they were integrating a variety of arts forms into their lessons and perceived an

increase in student engagement. The reasons cited for integrating arts included increase

comprehension of mathematical concepts, novels, and historical events, encourage creativity, and

elicit higher-order thinking skills activities (associate, compare, explain). Examples of arts

integration strategies employed included: using a wheel as a time allotment; creating a tableau

for vocabulary; using dance to focus or to control stress; using music and sound waves to

understand frequencies; linking artwork to the period covered in social studies; using drawings to

reinforce the plot moments in a story; and using artwork to help students visualize historic

events.

Interview and surveys showed that artful thinking was the most frequently used strategy

for all grades and content areas. Teachers commented that artful thinking activities forced

students to think outside the box and helped shy students to come out of their shells. Arts

integration was used as entry event, as a theme throughout the lesson, as a complement to a

specific point in the lesson, or as final event.

Overall, teachers agreed that planning for arts integration took more time than planning

for regular lessons, but they seemed to be able to adjust. Additionally, teachers perceived that

some content areas, such as English, were easier to integrate than mathematics. Additionally,

mathematics teachers felt that the standards they had to cover were too broad and they were

afraid of losing instruction time when integration arts.

Changes on Students’ Engagement

1. Perceptions about arts integration

Bates students participated in three yearly surveys that probed their perceptions about arts

integration. 3 Survey responses indicate that, overall, Bates students are receptive to the use of

arts as a strategy for making learning more engaging. Figure 7, on the next page, displays the

3 Survey findings are described in details in the yearly evaluation reports. A total of 345 students in grades 6-8

participated in the 2009 survey (60% response rate) and 586 (79%) participated in the 2012 survey.


percentages of students who agreed/strongly agreed with 15 statements related to art as a

motivator for learning in the surveys conducted during the first and fourth project years.

Figure 7: Art as motivation to learning (percentage of agree/strongly agree responses)

I ask more questions in class when arts are part of thelesson

Sharing my art work with others is important for me

Presenting my art work to others makes me proud of myschoolwork

I feel energetic after I participate in arts activities

Sharing my art work reminds me of what I have learned

Describing what my art work means reminds me of what Ihave learned

I can concentrate best when I am participating in artsactivities

I pay more attention in class when arts are part of thelesson

Arts activities motivate me to participate in class

Expressing my knowledge through arts activities is exciting

I enjoy learning through the arts

I would like to do more activities involving arts in myclasses

Working with my classmates during arts activities is fun

Using arts to learn other subjects is enjoyable

Arts activities make learning fun

40.7%

46.5%

57.4%

58.7%

59.0%

61.3%

68.7%

71.3%

74.4%

77.3%

81.9%

82.7%

83.5%

84.9%

88.2%

32.7%

46.9%

60.9%

51.6%

56.7%

59.9%

64.6%

70.9%

74.7%

73.9%

79.7%

75.2%

83.4%

79.3%

75.3%

2009 2012


A comparison of means between the 2009 and 2012 surveys indicated [statistically

significant] differences in five items: Arts activities make learning fun (t=5.070; df=911;

p<.000); Using arts to learn other subjects is enjoyable (t=2.173; df=911;p<0.03); I would like to

do more activities involving arts (t=2.740; df=911; p<.006); I feel energetic after I participate in

arts activities (t=2.096; df=911; p<.03); and I ask more questions in class when arts are part of

the lesson (t=2.419; df=911; p<.01). Students were more likely to agree/strongly agree with these

items in the 2012 survey when compared to the 2009 survey.

Students were also asked how they perceived blending arts into core content areas. Once

more, students were more likely to perceive that art projects blend well with the lessons in the

core content areas in 2012, when compared to 2009. Between the two survey years, all four

content areas showed increases in positive perceptions, with mathematics having the largest

change (21%), followed by English (12%). A possible explanation for this finding is the fact that

teachers, being more familiar with arts integration, were improving the process of integrating arts

to support learning. Table 4 displays the percentage of agree/strongly agree responses for the

2009 and 2012 surveys and the percentage of change between the two surveys.

Table 4: Integrating arts with core content areas

CONTENT AREAS Agree/Strongly Agree Responses (%)

% Change 2009 2012

English 70.3 78.8 +12.1

Math 63.1 76.1 +20.6

Science 69.0 73.0 + 5.8

Social Studies 74.2 77.2 + 4.0

The survey also probed students’ perceptions about the influence of arts integration on

their understanding of content in the core content areas. Options included “yes,” “no” and “don’t

know.” Table 5 displays the percentage of “yes” responses in the 2009 and 2012 surveys.

Relative to the number of students responding to the survey, considerably more students in 2012

stated that arts integration helped them have a new understanding in all core content areas, when

compared to students in the 2009 survey. The percentage of change was large, particularly for

English (an increase of 48%) and Mathematics (41% increase).


I was working one on one with a new student from

[Latin America]. He had said he did not know

English. However, when asked to examine a painting,

he formed a list of English words describing the

painting: rooster, chicken, red, blue, yellow, green,

orange... He knew English, but didn’t feel comfortable

speaking. Arts integration allows for open

communication. Talking about the arts before going

into a reading or writing assignment gets students

ready. ESOL teacher

Table 5: Students’ perceptions that arts improve understanding in core content areas

CORE CONTENT AREAS “Yes” responses (%)

% Change 2009 2012

English 43.1 63.7 +47.8

Math 47.4 66.9 +41.1

Science 49.1 62.5 +27.3

Social Studies 54.6 71.2 +30.4

2. Observed behavior

During site visits, student engagement in the lesson was assessed through student body

language and behavior such as actively listening or watching the educator; questioning,

exploring, or discussing the topic; collaborating with each other or the educator on a project;

engaging in activities that require critical/creative thinking. During the visits, more than 70% of

the students in all classes were observed

to be engaged in instruction.

Instructional approaches that involved

peer collaboration generated the highest

level of observed engagement. Students

also were engaged during lessons that

linked prior learning to the task at hand.

The observers were not able to capture

differences in students’ levels of engagement during arts integration versus non-arts integration

classrooms. One reason for this difficulty was that arts integration was being used in most

classes, even those that had been reported to the observers as not integrating arts.

Attendance was also reviewed as a proxy for student engagement, based on the

hypothesis that students who feel engaged in learning are more likely to come to school and

remain for the day than students who feel disengaged. A review of the percentage of days

attended relative to days enrolled was conducted, as displayed in Table 6. It must be noted that a

large increase cannot be expected when the baseline is already high. However, growth in average

attendance at Bates surpassed that at School B in SY 2013. Average attendance for Bates grew

by 3% between SY 2009 and SY 2013, while the growth for School B was 2%. Both schools

surpassed the Maryland satisfactory standard for attendance (94%).


Table 6: Average yearly attendance per school from SY 2009 to SY 2012

School SY 2009 SY 2010 SY 2011 SY 2012 SY 2013

Bates 93.6 93.8 94.8 96.4 96.4

School B 93.9 94.4 94.3 95.8 95.7

Table 7 lists attendance disaggregated by grade in 2013. Attendance at School B was

higher for students in Grade 6 when compared to upper-grade students. At Bates, however,

attendance was relatively the same through all three grade levels.

Table 7: Average attendance by school and grade (SY 2013)

School Grade 6 Grade 7 Grade 8

Bates 96.5 96.5 96.2

School B 96.0 95.9 95.3


PROJECT IMPACT

This section starts with a summary of student demographics and changes across grant

years. It follows an analysis of results in the Maryland School Assessment (MSA) Reading and

Mathematics across time. Finally, the five-year progress toward the grant objectives is reviewed.

Comparisons are made, when appropriate, with another AACPS middle school that had a student

population similar to Bates and was not implementing arts integration (School B). Data for this

section was provided by AACPS, unless otherwise noted.

Student Demographics

Table 8 displays the number of students in Bates and School B disaggregated by grade

and demographic characteristics. Demographics included race/ethnicity, gender, and eligibility

for special education services, English Language Proficiency (ELP) services, and Free and

Reduced Meals (FARM) program, used as a proxy for low socio-economic status.

Table 8: Student demographics by school (SY 2012-2013)

Characteristics Bates School B

Number % Number %

Grade

6 243 32.6 235 32.7

7 246 33.0 229 31.8

8 256 34.4 255 35.5

Total 745 100.0 719 100.0

Race/

Ethnicity

Black/African American 230 30.9 387 53.8

White 286 38.4 114 15.9

Hispanic 177 23.8 134 18.6

Multi-racial 29 3.9 42 5.8

Other 23 3.0 42 5.8

Gender Female 405 54.4 359 49.9

Male 340 45.6 360 50.1

Free and Reduced Meals (FARM) 358 48.1 396 55.1

Special Education 63 8.5 80 11.1

English Language Proficiency (ELP) 73 9.8 53 7.4


As seen in the table, both schools have a high composition of students from minority

backgrounds and low socio-economic status. Between SY 2009 and SY 2013, School B saw a

67% increase in the percentage of FARM-eligible students (from 33% to 55%), while the

proportion of FARM-eligible students at Bates remained constant around 48%. School B has

also a higher percentage of African American students when compared to Bates, which has a

more diversified population.

Figure 8 displays changes in the racial/ethnic composition of Bates’ students across the

years. The project started in 2009; data from 2005 through 2008 are provided as baseline. As

seen in the graphic, although the student population at Bates is still mostly from minority

backgrounds, the minority composition has changed considerably across the years. Between

2005 and 2013, the percentages of students classified as African Americans declined by 52%

(from 60% to 31%), while the percentages of students classified as Hispanic/Latino and from

other minorities, particularly multi-racial, more than doubled (from 15% to 31%). Within this

same period, the school experienced a 41% increase in the numbers of white students relative to

the total student enrollment (from 27% to 38%).

Figure 8: Percentage of Bates students disaggregated by race/ethnicity from 2005-2013

*SAILSS started in 2009; 2005-2008 are baseline years

0%

20%

40%

60%

80%

100%

2005 (588) 2006 (558) 2007 (567) 2008 (530) 2009 (580) 2010 (612) 2011 (701) 2012 (748) 2013 (745)

African American White Hispanic Other


Figure 9 displays changes in the percentage of students classified as FARM-eligible

during the grant period. As seen in the graphic, FARM-eligibility remained relatively constant

across the project years (about half of the student population)

Figure 9: Percentage of FARM-eligible students at Bates across grant years

Numbers in parenthesis = school enrollment per year

Table 9: Results from the 2013 MSA Reading and Mathematics

Grade

Reading

Bates School B AACPS Maryland

N % N % N % N %

6 239 85.4 222 83.8 5,583 88.8 60,389 84.1

7 229 85.2 208 79.8 5.615 88.9 61,149 85.0

8 240 82.1 233 79.1 5,633 85.4 60,079 81.0

Mathematics

6 238 73.1 224 58.9 5,580 76.0 60,399 77.1

7 230 79.6 211 54.5 5,616 79.2 61,139 72.6

8 240 64.6 232 48.7 5,636 66.3 60,011 67.0 N= number of students taking the MSA

%= percentage scoring at or above proficient

Source: 2013 Maryland Report Card, http://www.mdreportcard.org/

Academic Performance

1. Descriptive analysis

Maryland middle school students take the Maryland School Assessments (MSAs) in

Reading and Mathematics at grades 6, 7 and 8. The assessments are conducted in April or May

of each year. Table 9, above, lists the number of students who took the 2013 MSAs Reading and

49.50% 50.20%

47.20% 48.80% 48.10%

30%

35%

40%

45%

50%

55%

60%

2009 (580) 2010 (612) 2011 (701) 2012 (748) 2013 (745)



Mathematics and the percentages who scored at or above proficient. For comparison, the table

displays information for Bates, School B, the district (AACPS) and the state (Maryland).

Figures 10 and 11 display the percentages of students who scored at or above proficiency

in the 2013 MSAs Reading and Mathematics. In reading, results for Bates were above those for

the comparison school and the state average for all three grade levels. In mathematics, students at

Bates also did better than students at School B in all three grades. For grade 7, Bates’ students

outscored the district and the state averages.

Figure 10: Percentage of students at or above proficient on the 2013 MSA Reading

Figure 11: Percentage of students at or above proficient at the 2013 MSA Mathematics

Data source: 2013 Maryland Report Card (http://www.mdreportcard.org)

85.4% 85.2% 82.1% 83.8%

79.8% 78.1%

88.8% 88.9% 85.4% 84.1% 85.0%

81.0%

0%

20%

40%

60%

80%

100%

Grade 6 Grade 7 Grade 8

Bates School B District State

73.1% 79.6%

64.6% 58.9% 54.5%

48.7%

76.0% 79.2%

66.3%

77.1% 72.6%

67.0%

0%

20%

40%

60%

80%

100%

Grade 6 Grade 7 Grade 8

Bates School B District State



Table 10 compares differences in the percentages of students at or above proficient in the

MSAs Reading and Mathematics at two points in time: 2008, the year before SAILSS started at

Bates, and 2013, the final project year. The table shows a strong decline in the differences in

percentages for Bates and AACPS between the 2008 and 2013 MSAs. For grade 7 mathematics,

Bates results exceeded those for the district in 2013.

Table 10: Changes in percentages of students at or above proficient in the MSAs between

2008 and 2013

MSA COMPARISON GRADE 6 GRADE 7 GRADE 8

2008 2013 Bates District Bates District

Reading

AACPS 83.2 88.8 83.6 88.9 76.0 85.4

Bates 64.6 85.4 71.7 85.2 57.1 82.1

Difference) 18.6 3.4 11.9 3.7 18.9 3.3

Mathematics

AACPS 82.0 76.0 78.9 79.2 72.7 66.3

Bates 67.9 73.1 61.7 79.6 67.6 64.6

Difference 14.1 2.9 17.2 -0.4 5.1 1.7

Source: Maryland Report Card website, http://www.mdreportcard.org/

Table 11: Results for the 2013 MSA disaggregated by grade and demographics

Grade Demographics Reading Mathematics

N % N %

6

All 239 85.4 238 73.1

African American 70 84.3 68 67.6

Hispanic 63 68.3 62 54.0

Male 93 81.7 93 62.4

Female 146 87.7 145 80.0

FARM 123 76.4 122 59.8

Non-FARM 116 94.8 116 87.1

7

All 229 85.4 230 79.6


Hispanic 53 79.2 53 64.2

Male 109 82.6 110 75.5

Female 120 87.5 120 83.3

FARM 96 67.7 96 60.4

Non-FARM * <95.0 134 93.3

8

All 240 82.1 240 64.6


Hispanic 43 67.4 43 46.5

Male 119 79.8 119 60.5

Female 121 84.3 121 68.6

FARM 113 67.3 113 47.8

Non-FARM * <95.0 127 79.5

N = Number of students in the subgroup %= percentage at or above proficient

* Numbers were not included in the report



Results in the 2013 MSAs for Bates were disaggregated by large student subgroups, here

defined as groups with 20 or more students. These included male and female students, and

students classified as African American, Hispanic, and FARM-eligible. Results are displayed in

Table 11, above. Differences in percentages of students who scored at or above proficient were

quite large for FARM and non-FARM students, and they tend to increase as students move to

upper grades. Similar trends are seen in results for minorities when compared to all students.

2. Comparison of means

A comparison of means analysis was conducted for students in the two schools using

Independent-samples t-tests with a 95% confidence level. As displayed in Table 12, mean scale

scores for students at Bates surpassed those for School B students at all grade levels and in both

assessments.

Table 12: Results of the comparison of means analysis (2013 MSA)

Grade MSA Mean t-test

Grade 6

MSA Reading Bates 417.51

t= 3.085; df= 457; p<.002 School B 407.77

MSA Mathematics Bates 421.14

t= 6.401; df= 460; p<.000 School B 402.67

Grade 7


t= 4.825; df= 434; p<.000 School B 401.57


t= 7.116; df= 448; p<.000 School B 410.70

Grade 8


t= 2.028; df= 469; p<.043 School B 413.79


t= 4.016; df= 474; p<.000 School B 407.38

A comparison of means was also conducted for test results between student subgroups

within the treatment school (Bates). Means for minority students (African American and

Hispanics) were considerably lower than means for white students in both reading and

mathematics at all three grade levels. Likewise, large differences were found when mean test

results for FARM and non-FARM students were compared, with FARM students showing

considerably lower means than their non-FARM peers. Mean test results for female students

were higher than those for male students, while differences attained statistical significance in all


three grade levels for Reading, and grade 6 for Mathematics. At grades 7 and 8, mean test

results were similar for both genders. [Test results are not displayed for reasons of space].

3. Effect size analysis

Table 13 displays results of the effect size calculations. By comparing a school that is

implementing arts integration (Bates) with a similar school that is not implementing the program

(School B), the effect size analysis informs the program effectiveness in facilitating improvements

in student achievement. The effect size (Cohen’s d) was calculated as the difference between the

means for the treatment and comparison group, divided by the standard deviations for the

population. Cohen’s d is interpreted as such: 0.3 to 0.5 = small; 0.5 to 0.8 = medium; greater than

0.8 = large. The table does not presents results that are negative or below 0.3.

Table 13: Effect size analysis for MSA Reading and Mathematics (SY 2009 and SY 2013)

Grade School MSA Reading MSA Mathematics

Mean St. Dev. Cohen’s d Mean St. Dev. Cohen’s d

Bates

6 2009 396.05 49.40

0.50 405.76 45.98

0.39 2013 417.51 35.76 421.14 33.76

7 2009 390.67 80.57

0.60 389.61 83.17

0.60 2013 426.78 38.18 426.55 39.99

8 2009 400.38 59.66

0.42 414.01 52.08

2013 419.90 34.31 419.46 33.02

School B

6 2009 413.94 56.87

399.55 50.13

2013 407.77 31.55 402.67 27.77

7 2009 393.96 76.85

0.31 399.16 76.18

2013 410.70 30.61 401.57 33.86

8 2009 414.55 60.43

402.74 58.12

2013 413.79 30.86 407.38 32.55

*All results are statistically significant at α=.05; numbers in italics = small effect size; bold = medium effect size.

For Bates, medium effect size was found in grades 6 and 7 Reading, and grade 7

Mathematics, while a small effect size was found for grade 8 Reading. Grade 8 Mathematics

showed no program effect on test results. School B, which represents a traditional approach

(without arts integration) showed negligible or negative effect sizes for all grades in Mathematics

and grades 6 and 8 in Reading. A small effect size was found in grade 7 Reading.


4. Cohort Analysis

A cohort analysis was also conducted with Bates students to examine potential effect of

time in the project, as displayed in Table 14. The 2012 Cohort represents students who were in

Grade 8 in 2012; the 2013 Cohort includes students who were in grade 8 in 2013. The students

were divided into subgroups according to their time at Bates: first year – students who entered

Bates that school year (2012 or 2013) and, therefore, had SAILSS only while in Grade 8; two

years – students who were exposed to the project in Grades 7 and 8; and three years – students

who had been at Bates since sixth grade and had SAILSS for three years (grades 6, 7 and 8). An

ANOVA was conducted using MSA results as the dependent variable, and years in Bates as the

factor. No relevant differences were found in mean scale scores for both the 2012 and 2013 MSAs

Reading and Mathematics across the three groups. In other words, time in Bates does not appear

to influence results in MSAs.

Table 14: Results in the MSA Reading and Mathematics for Grade 8 students

disaggregated by years in SAILSS (2012 and 2013 Cohorts)

2012 Cohort 1 Year 2 Years 3 Years

Numbers 28 21 168

Reading 417.2 397.7 415.9

Mathematics 406.6 402.3 421.9

2013 Cohort 1 Year 2 Years 3 Years

Numbers 16 26 200

Reading 415.4 423.2 419.8

Mathematics 404.1 417.2 421.3

Monitoring Project Objectives

Objective 1: At the end of the grant period, 100% of the students at Bates Middle School

will be engaged in instructional activities designed to reflect their diverse learning styles and

multiple intelligences, as measured by teacher activity logs.

Status: Attained

This objective was attained in the first project year and the involvement of students in

arts integration continued throughout the grant. Information from teacher logs, site visits and

interviews show that students are involved in arts integration activities regardless of teacher or

grade level.


Objective 2: At least 20% of all Bates students will participate yearly in extracurricular arts

activities, countywide music and dance competitions, and arts portfolios, starting in the 2009

academic year, as measured by enrollment logs.

Status: Attained

The majority of Bates students are participating in arts-related activities, including arts

clubs, countywide music and dance competitions and arts portfolios. Close to 70% of the

students who participated in the 2012 survey acknowledged being involved in arts activities

outside of school time.

Objective 3: At least 40% of all Bates students will increase their scores on MSA for

mathematics by 5% yearly, starting with the 2009 MSA for mathematics.

Status: Not attained

To determine whether this objective was achieved, results from the MSA for mathematics

were compared to results from the same assessment the year before. If the resulting value was

greater than 0.05, the objective was met. The analysis must exclude students in Grade 6, as their

previous year’s MSA scores (when they attended elementary schools) were not available to the

evaluators. Table 15 displays the percentage of students in Grade 7 and 8 who increased their

MSA mathematics scores by 5% or more across the project years.

Table 15: Percent of students with 5% or greater increases on the MSA mathematics

Year Grade 7 Grade 8 Number of students

2009 20.0 17.9 108

2010 10.5 17.4 291

2011 13.7 30.2 329

2012 8.6 16.5 325

2013 5.1 5.0 418

A problem with the objective is that students who achieve high scores in the previous

year’s assessment will have difficulty increasing their scores by 5% from one year to another.

Another problem is that assessments have different levels of difficulty across grades and across

years. For instance, results for the 2013 MSA mathematics were lower overall, when compared

to the 2012 results, what may explain the decline in the percentage of students who increased

their scores by 5% between 2012 and 2013.


To further explore these caveats, Table 16 displays changes in the 2013 MSA

Mathematics broke down by how the students performed in the 2012 assessment. As seen in the

table, among the 28 students who were in grade 6 in 2012 and did not achieved proficient in the

2012 assessment, 22 (78.6%) were not able to increase their scale scores by 5% between 2012

and 2013, while 6 (21.4%) did achieve it. For the 78 grade 6 students who achieved proficient in

2012, 77 (98.7%) could not increase their scale scores by 5% between 2012 and 2013, while 1

(1.3%) did. For the 91 students in grade 6 who attained advanced in the 2012 MSA, 88 (96.7%)

could not increase their scale scores by 5%, while 3 (3.3%) did. Similar results are displayed for

the 2012 grade 7 students. As a rule of thumb, the lower the scores in the previous assessment,

the greater was the probability of the requested 5% increase. As more Bates’ students attain

advanced levels in the state assessments, it becomes more difficult for the objective to be

achieved.

Table 16: Changes between the 2012 and 2013 MSA mathematics for students at advance

levels

2012 MSA 2013 MSA

Grade Level Grade Below 5%* 5% or more**

Total N % N %

6

Below proficient

7

22 78.6 6 21.4 28

Proficient 77 98.7 1 1.3 78

Advanced 88 96.7 3 3.3 91

Total 187 94.9 10 5.1 197

7

Below proficient

8

32 82.1 7 17.9 39

Proficient 129 97.0 4 3.0 133

Advanced 49 100.0 0 0.0 49

Total 210 95.0 11 5.0 221

*Scale scores did not change, declined, or changed by less than 5%

**Scale scores changed by 5% or more

Objective 4: At the end of each academic year, starting with 2009, the number of discipline

referrals at Bates will decline by at least 5%.

Status: Attained

Table 17 displays the total number of referrals for disciplinary actions and average

number of referrals per student for Bates from SY 2009 to SY 2013, and compares to numbers

from School B. As indicated in the table, the two schools experienced a large decrease in the


number of referrals in the past five years. Discipline referrals in Bates declined by 80% (from

746 to 152), and by 82% at School B (from 674 to 122).

Table 17: Number of discipline referrals and referrals per student (SY 2009 to SY 2013)

School Referrals SY 2009 SY 2010 SY 2011 SY 2012 SY2013

Bates

Sum1 746 1,223 698 169 152

Average2 1.29 2.00 0.99 0.23 0.20

Enrollment3 580 612 701 748 745

School B

Sum1 674 538 429 170 122

Average2 0.85 0.70 0.57 0.24 0.17

Enrollment3 784 768 756 721 719

1 Number of disciplinary referrals

2 Average= number of disciplinary referrals divided by total enrollment

3Total number of students enrolled in the school year

Table 18 disaggregates information on number of discipline referrals per grade for SY

2013. Grade 8 students at Bates had fewer discipline referrals than students in grades 6 and 7. At

School B, variations across grades were small

Table 18: Number of discipline referrals and referrals per student by grade (SY 2013)

School Suspension Grade 6 Grade 7 Grade 8 All

Bates

Sum 55 54 43 152

Average 0.22 0.22 0.17 0.20

Total Enrollment 243 246 256 745

School B

Sum 43 33 46 122

Average 0.18 0.14 0.18 0.17


Information on suspensions complement the information on disciplinary referrals, since

the overall number of discipline issues in the school may decline, but the severity of the actions

may increase (resulting in greater number of suspensions). Table 19, on the next page, displays

suspensions at both schools since SY 2009. In contrast to the data on disciplinary referrals,

where decline was similar to both schools, the number of suspensions per student declined by

71% in Bates between 2009 and 2013, compared to a decline of 52% at School B.


Table 19: Number of suspensions and average suspension per student (SY 2013)

School Suspensions SY 2009 SY 2010 SY 2011 SY 2012 SY 2013

Bates

Sum 249 225 233 99 71

Average 0.43 0.37 0.33 0.13 0.10

Enrollment 580 612 701 748 745

School B

Sum 183 255 177 98 88

Average 0.23 0.33 0.23 0.14 0.12

Enrollment 784 768 756 721 719

Table 20 displays suspensions disaggregated by grade for SY 2013. Differences in

numbers of students suspended by grades were small at both schools.

Table 20: Number of suspensions and average suspension per student by grade (SY 2013)

School Suspension Grade 6 Grade 7 Grade 8 All

Bates

Sum 21 29 21 71

Average 0.09 0.12 0.08 0.10


School B

Sum 35 21 32 88

Average 0.15 0.09 0.13 0.12



CONCLUSIONS

From SY 2008-2009 through 2012-2013, AACPS implemented Project SAILSS in a

middle school located in an urban setting. The project proposed a model of arts integration

founded upon systemic, focused professional development and mentoring support for all school

faculty. Arts integration lessons were geared toward standards where students demonstrated the

greatest difficulties. The model expanded from its core – the school – to the community at large

through the involvement of families, partners, and community members, and dissemination of

the model, and its success, in conferences and publications.

Answering the evaluation questions

This report summarizes evaluation findings from all five project years to address the

initial evaluation questions. Two series of questions were proposed:

How does the model of Arts Integration proposed by SAILSS work to bring quality across

the curriculum?

As described in the previous evaluation reports, the project implementation happened

“with a bang.” From year one, content area teachers, working side-by-side with arts teachers and

artists in residence, developed and implemented arts integration lessons with students at all grade

levels. By the end of the project, all Bates teachers had received professional development on

arts integration and the project was expanding its reach to teachers in other AACPS schools.

Close to 700 arts integration lessons were developed. These lessons included all types of arts

(visual, drama, music) and content areas (English, Mathematics, Social Studies, Science, Foreign

Languages). Interviews with teachers in the fourth project year showed that arts integration had

become “the Bates way” of teaching. Attempts to observe classes without arts integration classes

failed as basically in all observed classes arts integration was being used at some point.

How does the model impact student achievement and attitudes toward learning?

Analyses of student academic performance in the statewide assessments showed a steady

increase in the percentage of students who scored at or above proficient in reading and

mathematics. When compared to a school that did not implement arts integration and had a

similar student population, Bates’ results were considerably higher, particularly in mathematics.

An analysis of effect size using Cohen’s d showed that the project had a medium effect on test


results for grades 6 and 7 Reading and grade 7 Mathematics, and a small effect size for grade 8

Reading, with no effect on test results for grade 8 Mathematics.

Behavior has been improving across the years and the trend is seen in both schools.

Therefore, changes at Bates cannot be attributed solely to SAILSS. Indeed, AACPS has been

implementing a number of programs that focus on behavior management, such as Positive

Behavior Intervention Strategies (PBIS), Collaborative Decision Making (CDM) and others. The

role of these programs cannot be minimized when discipline is examined (School B implements

PBIS). Connections among these programs are certainly a winning strategy for AACPS.

Lessons learned

A reflection on SAILSS’ accomplishments highlights three important lessons that may

help stakeholders to implement future projects.

A project success depends on shared responsibility and collaboration

Strong support from administration and the presence of the Arts Integration Specialist

(AIS) were probably the two main factors responsible for overcoming the initial resistance to the

project from teachers, particularly veteran teachers. Even at the beginning of the project, the

school was already recognized as a place for learning, where the administration supported

innovation. Therefore, arts integration was another innovation that the administration was open

to pilot. The school district put its weight behind the project and in support of the school

administration. Teachers heard one voice and they knew they had to comply. The AIS position

was essential as both a voice for the project among peers and a mentor that created conditions for

success. The AIS was a lead teacher, highly respected among her peers, who went through the

Post-Baccalaureate Certification and became an arts integration expert. She worked closely with

all participants – content area teachers, arts teachers, and artist in residence – to ensure that the

grant’s objectives were not lost, but also to ensure teachers’ success as they developed arts

integration lessons. She also worked closely with the evaluators to ensure the availability of data

and even proposed further ways to collect data that informed decisions related to project

improvement.

An important strategy advanced by the AIS involved looking at standards to see changes

in benchmark results. As teachers saw the results of arts integration on benchmark tests and on

students’ engagement, buy-in was total and evaluators could no longer find classes that were not

integrating arts as a way to advance knowledge of new concepts. A major change in teachers’


perceptions that can be attributed to the project was a heightened sense of camaraderie not only

among teachers but also among teachers, families and students. Perceptions regarding students as

disciplined and engaged increased significantly between the first and the last climate survey.

Teachers who are now entering Bates already know that arts integration is an integral component

of the teaching strategies available and are receptive to its use.

Changes in education take time and patience

Changes in education take time, particularly changes in student academic performance as

measured by standardized tests. There are so many factors beyond the control of the schools that

influence the lives of both students and faculty (economic crisis and personal histories to name a

few). At Bates, indicators of student behavior are showing progress and the school climate has

improved. MSA results have also improved and the project showed small to middle effect size on

the Reading and Mathematics assessment at almost all grades. It is true that teachers at Bates and

elsewhere in the state will face challenges these coming years, as Maryland changes its statewide

assessment. The new assessments are said to be more performance oriented, what may show

even stronger improvements under arts integration.

Improving student learning is an ongoing process

Despite the project successes, Bates still face areas of concern regarding student

academic performance, particularly the gap in performance across student subgroups. Students

from minority and low socio-economic background continue to lag behind their peers from

majority, higher socio-economic families. What works for what groups of students? This is a

question that teachers should strive to answer. As teachers continue looking at the impact of arts

integration on learning of specific standards, it is important to disaggregate results for student

subgroups and understand how the most at-risk students react to the lessons. It is also important

to accept that not everything that is done with so much love and dedication will work, or will

work equally for all students. Student feedback on the lessons will help to gauge interest and

engagement in learning. The benchmark analysis, disaggregated by subgroup, will provide

further evidence on what needs to be retaught through different strategies.


Final words

Probably the greatest success of SAILSS was the creation of a model of Arts Integration

that shows evidence of working .in large middle schools within urban settings. The model is

simple and uses a lot of “in-house” resources, which facilitates its implementation by other

schools. Within Bates, SAILSS has attained a maturity that guarantees its continuity, even

without grant funds. SAILSS is “the Bates way” of teaching. Many of the project-supported

lessons are being used in other schools in the district. AACPS created a district-wide AIS

position to support those schools. Additionally, the partnership that gave impetus to SAILSS is

sustaining the summer institute. Consequently, federal funding was well spent in a project that

showed success, has clear signs that it will be sustained, and is already expanding beyond its

initial boundaries.


REFERENCES

Armstrong, T. (2006). The best schools: How human development research should inform

educational practice. Alexandria, VA: Association for Supervision and Curriculum

Development.

Burnaford, G. (2007). Arts integration frameworks, research and practice: A literature review.

Washington, D.C.: Arts Education Partnership.

Burton, J., Horowitz, R., & Abeles, H. (1999). Learning in and through the arts: Curriculum

Implications. New York: Teachers College, Columbia University., Center for Arts Education

Research.

Fisher, D. L., & Fraser, B. J. (1990, April 16-20). Validity and use of the School-Level

Environment Questionnaire. Paper presented at the Annual Meeting of the American

Educational Research Association, Boston.

Gardner, H. (1983). Frames of mind: The theory of multiple intelligences. New York: Basic

Books.

Glaser, B. G. & Strauss, A. L. (1967). The discovery of grounded theory; strategies for

qualitative research. Chicago, Illinois: Aldine Publication, Co.

Nelson, C. A. (2001). The arts and education reform: Lessons from a four-year evaluation of

the A+ Schools Program, 1995-1999 (Executive summary of the series of seven policy

reports summarizing the four-year pilot of A+ Schools in North Carolina). Winston-Salem,

NC: Thomas S. Kenan Institute for the Arts.

Strauss, A.L. (1987). Qualitative analysis for social scientists. New York, New York: Cambridge

University Press.

U.S. Department of Education. (2008). Guide to U.S. Department of Education Programs.

Washington, D.C.: USED, Office of Communications and Outreach.

Yin, R.K. (2009). Case study research: design and methods. Los Angeles, California: Sage

Publications.


APPENDIX A: EVALUATION DESIGN

Purpose

SAILSS’ external evaluation had a three-fold purpose: (1) provide stakeholders with

periodic assessments of the project’s progress toward goals, (2) collect evidence that informs a

continuous quality improvement process, and (3) assess changes in teacher practices and student

outcomes that are related to the project.

RMC approaches evaluation from a perspective of partnership. Evaluators work closely

with project staff to address their questions, and promptly provide information that can be used

for project’s improvement. To ensure timely and useful feedback, the evaluators and project staff

meet periodically to review the implementation and evaluation plans, adjust timelines, share

information, and discuss evaluation findings and recommendations.

Research Questions

The following questions guided the implementation and evaluation of the project:

How does the arts integration triad model of spiraled teacher professional development,

curriculum writing, and curriculum implementation work to bring quality integrated arts

instruction across the curriculum?

o What are the challenges and successes with arts integration development and

implementation using this model?

o What are the challenges and successes with arts integration growth and sustainability

using this model?

o What are the experiences and perceptions of teachers, students and artists in residence

as they engage in the integrated arts curriculum implementation?

o What are the experiences of the community, including parents and other community

stakeholders, as they engage with a school moving into arts integration?

o What are the perceived meaningful nuances and details of arts integration that support

or detract from implementation success (student, parent, teacher, artist, and

administration)?

How does the integration of arts into mathematics classes affect student achievement in,

and attitudes toward, mathematics?


How does the integration of arts across the curriculum affect overall student achievement

and attitude towards learning?

Evaluation Design

The implementation of the project (formative evaluation) used a case study design (Yin,

2009) to address the first four research questions. The effects of the project on teacher practice

and student learning (outcome evaluation) were assessed through a quasi-experimental design

using a school with similar demographics but without arts integration as the comparison group.

The hypothesis was that students at the treatment school would show greater gains when

compared to students at the comparison school. Indicators used to assess gains were: MSA

mathematics and discipline referrals. Results in the MSA reading were also collected.

At the time of the grant proposal, AACPS chose the treatment and the comparison

schools on the basis of school size, student demographics, and the principals’ interest in the

project. Both schools had expressed their interest in becoming Arts Magnet schools. Bates was to

start the Arts Magnet program in SY 2009-2010, while the comparison school would start the

Magnet Program by the end of the grant. Yet, as SAILSS started, teachers at the comparison

school began attending professional development together with the treatment teachers and the

following year, an arts magnet program started at the school. Therefore, the school had to be

dropped from the study. In SY 2010-11, a new school was chosen for comparison based on

student demographics and size.

The plan was to involve the comparison school in the surveys, and this happened in the

two first years of the grant. When the initial comparison school had to be dropped, the project

was in its third grant year and the new comparison school was not fully committed to the project.

Therefore, comparisons were limited to the analyses of student data, which were collected from

central office.

Data Collection

The following instruments were used to collect data for the study:

Classroom Observation Form – This form was developed by Dr. Elda Franklin for the

North Carolina Blumenthal Performing Arts Center Education Institute, a past AEMDD

project. Blumenthal Performing Arts Center graciously granted permission to use the

form (Franklin, 2005). The instrument consists of a time line with five-minute


observation intervals to assess student engagement and instruction. A rating scale for

assessing the implemented lesson plan for each indicator is also provided.

Arts integration: Classroom Observations for Middle Schools (AICOM) – This

instrument was developed by RMC to be used in conjunction with the Classroom

Observation Form. The form requires visual observation of the classroom environment,

an overview of the lesson in progress and a brief review of student Arts integration

projects.

Professional development log – collects information on who attended which professional

development activity. This log was collected by the AIS and provided to the evaluator on

a quarterly basis.

Arts integration activities log – provides information on curriculum development and

Arts integration activities implemented in the classrooms. Teachers kept a log of these

activities and provided it to the AIS at the end of the each school semester.

The School-level Environment Questionnaire (SLEQ) - the SLEQ measures teachers’

perceptions of psychosocial dimensions in the school environment. It consists of 56

items scored on a five-point scale and organized into eight categories: (1) student

support; (2) affiliation; (3) professional interest; (4) staff freedom; (5) participatory

decision making; (6) innovation; (7) resource adequacy, and (8) work pressure. The

development and validation of the SLEQ is described in Fisher & Fraser (1990), who

granted permission to RMC for use in this evaluation. Burton, Horowitz & Abeles

(1999) used the SLEQ in their research regarding arts-rich versus arts-poor schools. The

instrument was applied to all full time teachers with classroom responsibility (that is,

resource teachers and substitute were not surveyed).

Student Arts integration survey – this instrument was created by RMC after a literature

search indicated that no existing instrument addressed the project needs and interest. The

instrument, created with feedback from the project director, was applied annually.

Table A-1 summarizes the evaluation activities, their target population and focus, the

instruments or protocols used to collect data, and the timeline.


Table A-1: Evaluation Activities, timeline, and instruments

Data Collection Focus Timeline Instrument

Professional

Development

Identify intensity of

treatment and participation Yearlong; all four years Project Director Log

Arts integration

Activities

Track development of

curriculum components and

activities

Yearlong; all four years Teacher Log

Classroom

observation

Assess fidelity and intensity

of implementation. Yearly

Blumenthal Performing

Arts Center Education

Institute (2003)

Form and Arts integration

Form (RMC)

Teacher Inventory Assess teachers’ perception

of school climate May (yearly) -

School-level

Environment

Questionnaires (SLEQ) –

Fisher & Fraser (1990)

Student survey

Gauge students’ interest on

learning and involvement on

Arts integration activities

April or May (yearly) -

Student Arts integration

Survey (Project -

developed)

Student

attendance/

behavior data

Indirect measures of

engagement in learning September, yearly - AACPS database

Student academic

performance data

Assess changes in student

performance on mathematics

(project objective/GPRA)

and reading (GPRA)

Quarterly AACPS Benchmarks

(Mathematics)

Yearly (released end of

summer)

Maryland - MSA results

(in Reading and

Mathematics)

Data management and analysis

Both qualitative and quantitative data were collected as part of the evaluation, and

methods addressing both types of data were utilized in order to elicit pertinent information for

program decision making. Triangulation was used to confirm, explain, or disqualify qualitative

and quantitative responses. Qualitative data were coded thematically using the Grounded Theory

approach (Glaser & Strauss, 1967), and summarized in narrative format. This method allows the

theory to emerge from the data, rather than follow from a hypothesis. The process starts with

breaking the information down into categories as small as recurring words and phrases (Strauss,

1987). Individual codes are derived from the data. They are then categorized under developed

themes and examined for patterns within and across documents to address the specific research

questions.


Quantitative analyses explored the following outcomes: academic performance, as

measured by state assessments, and behavioral data. Changes in performance measures at the

two schools were analyzed longitudinally, using the four baseline years and the project years.

Results from the treatment school (Bates) were compared to those from the comparison school

(School B) using comparison of means tests and Cohen’s d analyses.

Communication of findings

In addition to oral communications made at meetings with project and school staff, the

evaluators completed a number of small reports to present findings from the surveys.

Additionally, yearly evaluation reports were completed and provided to the project director.

These reports detail implementation activities conducted for each project year, as well as results

from surveys, interviews and observations. The final report summarizes findings from a

longitudinal perspective.

IRB review and parental consent issues

All the study procedures were reviewed at RMC Research Corporation and kept in

agreement with its ethics procedures. AACPS does not have a formal IRB process. However, an

agreement was established between evaluators and AACPS that ensures fair and confidential

treatment for all study participants. RMC Research Corporation was granted permission to obtain

student-level data from AACPS, upon consultation with its lawyers. The data is kept in a secure

partition within RMC server and can be accessed only at that location by the evaluation team

members. No copies of the data or transfers over non-secure lines are allowed. Two evaluation

activities involved student participation: classroom observations and an annual survey.

Following AACPS procedures, as the student surveys are anonymous and do not collect data that

can identify students or are of a personal nature, parents were informed of the study but no opt-in

permission was required.

Documents

Project SAILSS: Final Evaluation Report€¦ · 410-570-0315 Submitted by: Sonia Jurich, MD, Ed.D., Senior Research Associate Laura Taylor, M.A., Research Associate RMC Research Corporation