THE IMPACT OF IPADS ON STUDENT LEARNING

PREPARED BY MATTHEW LIPSTEIN , TECHNOLOGY CHAMPIONST. GABRIEL’S CATHOLIC SCHOOL, AUSTIN , TEXAS • AUGUST, 2013

• RESEARCH STUDY •

ABOUT THE RESEARCH STUDYThe data, research and analysis that inform this paper were collected over the course of the 2012-2013 school year. The Research Study was completed in the Summer of 2013. All research and data collection occurred at St. Gabriel’s Catholic School in Austin, Texas.

This project would not have been possible without the generous participation and support of St. Gabriel’s Catholic School administrators, faculty, students and families. Specifically, Interim Head of School, Dr. Judy Knotts, and Director of Technology, Tracy Mulligan were invaluable for their time, input and guidance.

ABOUT THE AUTHORMatthew Lipstein is the Technology Champion for St. Gabriel’s Catholic School. In this role, he manages all aspects of the School’s iPad program in addition to tracking metrics and assessments which help gauge the success of the implementation.

Since 2002, he has worked since 2002 at independent schools in New York City and Austin, Texas providing leadership and vision around student, faculty and administrator use of educational technology.

ABOUT ST. GABRIEL’S CATHOLIC SCHOOLLocated on 32 acres of rolling hill country in Austin, Texas, St. Gabriel’s Catholic School is a community for learning focused on helping children excel academically, grow in virtue, and acquire the leadership skills, experiences and confidence required to form the foundation for successful and meaningful lives. Serving children in grades Junior Kindergarten through 8th, St. Gabriel’s gives children an academic advantage, enabling them to excel in school and to gain a strong, intellectual foundation. The School seeks to create a safe and nurturing community that helps instill in children the virtues, judgment and courage to do what is good and right. Ultimately, St. Gabriel’s is focused on engaging young minds and readying them for leadership at school, in their communities, and in the world.

i

1. Executive Summary 3

2. Introduction 4

3. iPad Program Background 6 4. Methodology 8

5. Findings & Results 12 6. Conclusion & Discussion 20

7. Limitations 22

8. Works Cited 23

CONTENTS

ii

Over the course of the 2012-2013 school year, research and data collection was conducted at St. Gabriel’s Catholic School to gauge the impact on student learning with the introduction of iPads in Grades 2 through 5. The goal of this study was to determine if there existed a correlational relationship, whether it be positive or negative, between a student’s use of an iPad and their performance on assessments. This study did not attempt to establish causality between iPad use and test scores.

Students were grouped into two discreet clusters: Grade 4 & 5 students and Grade 2 & 3 students. Grade 4 & 5 students utilized the eSpark program to provide instruction on targeted content specific to the needs of each student. Assessments were conducted using the Let’s Go Learn program. Because all Grade 4 & 5 students used the eSpark program, and the material studied was differentiated for each student, growth was measured by comparing a student’s assessment data in their eSpark area of focus against the areas of focus where they did not use eSpark.Conversely, Grade 2 & 3 students utilized a variety of Apps and websites that provided instruction for a given Math unit, and their assessments were performed via the TenMarks.com program. During a unit only one of the two grade level classes received iPads to work with, thus establishing a control group. Assessment data was compared between the class that utilized iPads and the corresponding class that did not use iPads during the same Math unit.Key Findings • Grade 4 & 5 students overall showed at least twice the level of growth in their eSpark areas of focus when compared to their growth in areas where they did not use eSpark.• Grade 2 & 3 students who used iPads for a Math unit overall saw slightly higher levels of growth when compared to the class which did not use iPads.Given these results, St. Gabriel’s Catholic School is confident that targeted student use of an iPad results in stronger learning outcomes for students, as demonstrated by assessment data.

EXECUTIVE SUMMARY

1

3

Still in it’s relative infancy, the iPad is a device that is unique both for it’s perceived potential use as a learning tool for students, and for the overwhelming rate of adoption in schools world-wide. The iPad was introduced in Spring of 2010, and was adopted quickly and fervently by educational institutions as a potential game-changer for students. The iPad is just one device in a long list of educational technology tools that have aided student learning. Technology is seen by most as an essential tool students can use to receive instruction, learn, create, communicate, and grow. The White House Council of Economic Advisors (2011) suggests that the payoff of utilizing educational technologies (not specifically iPads), that can lead to better outcomes for students, could be enormous.

Furthermore, the iPad has been lauded as revolutionary in education, and there are many explanations that researchers and educators use to explain why they feel the iPad is more significant than the educational technology tools that preceded it.

For many, the iPad is seen as a robust and efficient tool that combines a variety of pre-existing educational technology tools into one device. Ireland and Woolerton (2010) outlined many pragmatic and pedagogical benefits of this device unification, including audio and video material being controlled by the student rather than the teacher, instant-access to the web, and an institutions library for research purposes, and the potential for students to author his/her own material. Cohen, in his study “Young Children, Apps & iPad” (2011), examined the potential an iPad allows for a new kind of learning to exist. His findings indicate that “there are several types of learning that occur during App play. These include: the tacit learning of the game and how it works; mastering of explicit learning tasks (e.g., matching, counting) embedded in the game narrative; and the use of skills and models learned and applied to other types of games and levels of play. Engaging with creative App activities often shifts the child’s focus away from the subjective experience of winning or losing to a personal-best competition. Additionally, children progress quickly from novice to mastery when using a well-designed App.” According to Manuguerra and Petocz (2011) who write in their study “Promoting Student Engagement by Integrating New Technology into Tertiary Education: The

INTRODUCTION

2

4

Role of the iPad”, that “the iPad has been used as a means to engage, inspire and motivate students through high-level presentation and communication tools. It has changed the pedagogical approach, making the learning experience simpler and yet deeper. The results show that students learn best when technologies are seamlessly integrated into the curriculum to enhance their learning experience.”Perhaps the greatest potential seen with the iPad is the opportunity for true differentiation in the classroom - allowing each student to receive tailored instruction to meet them where they are on the learning continuum. Differentiated learning environments, or Personal Learning Environments as described by Johnson, Adams and Haywood in The NMC Horizon Report: 2011 K-12 Edition (2011), are “conceived as drawing on a variety of discrete tools, chosen by the learner, which can be connected or used in concert in a transparent way” while still allowing for flexibility and customization.

Cohen (2011) explains that by combining software that can accurately assess and adjust the level of challenge accordingly with a high-level of student engagement and optimal content, the resulting environment will sustain a student’s play and learning.

It has been noted by many, including the Department of Education and Early Childhood Development (2010) the iPad alone is simply just a device - another tool in the teaching and learning toolkit. More so, it has been offered by Michel Cohen (2011) as well as the Bill & Melinda Gates Foundation (2012) that even with the high-levels of enthusiasm students have for the iPads, the device itself does not guarantee engagement and learning. Rather, it is the combination of the iPad with effective teachers and pedagogy that allow for the true benefits of the technology for students.There also exists a growing demand for analytics to quantify the impact the iPad can make on a student’s education. Cohen (2011) indicates that analytics should be used in the service of extending and refining learning. In considering the impact of tracking student data, Johnson, Adams and Haywood (2011) explain that by analyzing the results, one would see not only the change in student performance, but would also see results in how educators perceive teaching, learning, and assessment. Suddenly, models of curriculum are more fluid and open to change.

The introduction of iPads to St. Gabriel’s Catholic School afforded an opportunity to systematically create metrics and run assessments to gauge the devices’ impact on student learning, specifically by analyzing data collected via student assessments.

5

St. Gabriel’s Catholic School is a Junior Kindergarten through 8th grade school located in Austin, Texas. In the 2012-2013 school year, the School’s enrollment was approximately 420 students. Through a generous donation to the School, iPads were initially introduced to faculty in the fall of 2012, and then subsequently to students in the winter of 2012. The iPads were distributed in the following manner:1:1 iPad Program in Grades 4 & 5 (92 iPads) Students in these grades each received their own iPad which they were responsible for bringing back and forth to school.Shared iPad Carts (55 iPads) Students in Junior Kindergarten through Grade 3 shared iPads which were available in hallway carts. Each class had five iPads designated for student use, which were typically used during “centers” time. However, it was not uncommon for a teacher to take a full-class set for a specific lesson.Faculty, Administrators & Staff (55 iPads) Each faculty member received their own iPad to be used as a teaching tool. Additionally, select administrators and staff members also received iPads for use at school. It is worth noting that while Middle School students had access to iPads for specific projects or lessons, the decision was made to not deploy iPads to those grades as Middle School students were already engaged in a 1:1 laptop program.The role of the Technology Champion at the School was also introduced with the advent of the iPad distribution. The primary goal of the Technology Champion is to assist faculty and administrators in using iPads as a tool to positively affect teaching and learning outcomes. Additionally, it is the Technology Champion’s responsibility to establish metrics and preform assessments that gauge the effectiveness of the iPad’s impact on education. Beginning in the fall of 2012 and extending through the school year ending in Spring

IPAD PROGRAM BACKGROUND

3

6

2013, research and data collection was preformed by the Technology Champion that ultimately informed the content of this research study. Since the introduction of iPads to the School, much work has been done to train faculty on effective use of the device, including professional development opportunities offered both within and outside of school. Lastly, the iPad implementation was largely guided by the goals of the St. Gabriel’s Wings program at the School. The program has three main tenets:

• The goal of all technology use at St. Gabriel's Catholic School is to have a positive impact on the education of our students• Wherever possible, technology will be used to transform the student experience, rather than replicate it•The use of educational technology will sharpen students' 21st Century skills

By keeping the focus of iPad and technology usage on these tenets, the discussions were guided away from the iPad itself, and shifted to what the device allows teachers and students to accomplish.

7

Given the variety of iPad implementations at different grade levels, different methodologies and data collection techniques were used throughout the school year. Overall the goal was to assess student knowledge of a particular learning objective before the start of a unit, introduce the iPad loaded with targeted Apps and videos to be used as an instructional tool by some or all of the students, and then assess the students’ knowledge again after the unit was complete. It has been a constant goal of this research to avoid any attempts in establishing any causality between iPad use and student behavior. Given the multitude of factors at play during a student’s education, we did not feel comfortable stating that the introduction of an iPad was the sole reason for any changes in student performance. Rather, it has been our focus to simply research if a correlation could be established between iPad use and the impact on a student’s learning.After some consideration, the decision was made to focus on Grades 4 & 5 and Grades 2 & 3 as the sample of students to be analyzed for this research study. The methodology was that these grades, while utilizing different types of iPad implementations, would provide the greatest opportunity to observe the impact of iPads on learning.Below are the approaches to data collection and analysis that we employed at the different grade levels. Grades 4 & 5: iPads and eSparkEvery student in Grades 4 & 5 (ninety-two students total) participated in a 1:1 iPad deployment where they received their own iPad and brought it back and forth to school each day.

In an effort to take advantage of the potential for differentiation in a 1:1 deployment, St. Gabriel’s Catholic School partnered with eSpark. The eSpark program is designed specifically for iPads, leveraging the device as a personal learning environment by creating a custom curriculum for each student. As well, the ongoing data-collection that is part of the eSpark process allowed us to analyze the impact of the eSpark program and the iPads.

METHODOLOGY

4

8

Below is an explanation of the overall steps in the eSpark process for assessing students and creating custom curricula for students.

eSpark Process1. Pre-AssessmentAll Grade 4 & 5 students took a pre-assessment using the adaptive web-based testing tool called Let's Go Learn. This established baseline scores in the areas of reading and math.

2. Goal SettingThe assessment results were sent to eSpark who recommended areas of focus for every student. Each goal aligned to a Common Core Domain. Teachers reviewed these suggestions and chose three areas of focus based on that student's greatest opportunity for development. This teacher review was an important check to confirm if the initial recommendation was accurate and provided a safeguard against any error in the system or assessment.

3. Learning Plans & QuestsEach student then received an individualized Learning Plan consisting of Apps and videos. The St. Gabriel’s Technology Department loaded a total of 4,700 individual Apps - roughly 50 Apps per student, on Grade 4 & 5 iPads.

4. Classroom UsageStudents used eSpark three times a week at school for twenty minutes a session, resulting in one hour per week. During that time they would watch video tutorials and work with Apps that were aligned to their Learning Plan. At the end of a content segment each student recorded a video explaining back what they had learned. 5. Post-AssessmentAt mid and end-year points students took the Let's Go Learn assessment again to measure the effectiveness of the eSpark program on their iPads.

Student Experience

Students worked on their eSpark goals by completing Missions, which consisted of a series of “quests” designed to teach a student the various concepts required for mastery of their area of focus. Each quest took the student through videos to be watched and Apps to be used.

9

1. Starting a MissionAt the start of each new quest, the student was required to watch an introductory video framing the concept to be taught. The student could not advance to the videos or Apps until after watching this introductory video.

2. Working through a MissionAfter watching the introductory video, students completed specific work in a variety of Apps and watched additional videos to reinforce the concept learned. 3. Finishing a MissionOnce all of the work in a Mission was complete, the student recorded a video where he or she explained back the concept they had just learned. This gave the teacher the opportunity to check the student’s understanding and provided additional instruction to reteach the concept if necessary.Grades 2 & 3: iPads used During Math UnitsNinety Five students in Grades 2 and 3 had access to a full class set of iPads that were available as a shared resource in hallway carts. Building off the eSpark model used with Grades 4 & 5, a system was employed to utilize iPads as a supplemental instruction and assessment tool used in Grade 2 & 3 Math classes. There was refinement to the process as the year progressed, but the overall approach was to identify one specific objective in a Math unit and include iPads as supplemental assistance in one of the two classes in each grade.

Process

1. Planning Leading up to a new Math unit the Technology Champion met with teachers to define a typical area of struggle for students in the upcoming material. A specific objective and area of focus, aligned to a TEKS (Texas Essential Knowledge and Skills) standard, was selected.

2. Pre-Test At the start of the unit, all students in the grade took a pre-test on iPads using the TenMarks website to establish base-line knowledge.

3. iPad Usage Then, iPads were brought to one of the two grade-level classrooms to be used as a supporting tool for the Math unit, creating a control group.

10

4. Post-Test At the end of the unit, each class was given a post-test on the same objective, again using TenMarks on the iPad.

The assessment process for Grade 2 and 3 students evolved throughout the year. Initially, assessments were preformed using Learning.com, but in practice, this proved challenging for students to do on an iPad using a Flash-enabled browser. Ultimately, TenMarks.com was a better solution for administering assessments.

It also became clear it was imperative that teachers assist in defining what specific concepts students should focus on for assessments and iPad usage. There were instances when pre-tests showed that most students already had near-mastery of a concept, making any further iPad intervention superfluous.

Lastly, we began by introducing iPads once a week to students; this was increased to twice a week after our first unit. By the close of the school-year we had greatly refined the assessment and data-collection processes.Iowa Test of Basic Skills (ITBS) AssessmentStudents in all grades took the Iowa Test of Basic Skills (ITBS) assessment in September of 2012. This is an annual assessment that is mandatory for schools within the Austin Diocese of Catholic Schools. There is an addendum planned for this research study to be included once the September 2013 ITBS assessment has been completed, and the resulting data has been analyzed.

11

There were separate data-collection processes used for Grade 4 & 5 students and Grade 2 & 3 students. The findings and results included herein have been separated accordingly. Grades 4 & 5: iPads and eSparkThe Let’s Go Learn web-based adaptive assessment tool was used to collect data in Grades 4 & 5. The Let’s Go Learn assessment was administered to students three times over the course of the 2012-2013 school year (October, February, and May).Let’s Go Learn Beginning-of-Year Assessment (October 2012)Establishing Base-line Scores and Goal Areas

The data gathered from students’ initial Let’s Go Learn assessment helped establish a base-line for each student’s strengths and weaknesses as they aligned to the Common Core Domain standards.

It was immediately clear that our students performed higher in the Language Arts domains when compared to Math. Specifically, our Grade 4 & 5 students scored within the Grade 6 & 7 ranges for Language Arts.

From this assessment data we identified each student’s areas of need. We then selected specific learning goals upon which to develop that student’s individualized Learning Plan.

FINDINGS & RESULTS

5

12

Figure 5.1: Base-line Let’s Go Learn assessment scores from October 2012. There were already clear indications of high performance in Language Arts domains.

13

Let’s Go Learn Mid-Year Assessment (February 2012)Tracking Progress in Semester One

Overall, students excelled in part due to their use of eSpark in the first semester.

Assessment data showed that students tested twice as high in the areas where they were using eSpark compared to those areas in which they were not receiving instruction via eSpark.

Because eSpark is customized for each learner, students progressed through the program at different paces. Therefore, not all students had completed each of their Missions when they took the mid-year assessment. This allowed us to see that students who completed more than two-thirds of their Mission, grew almost one-full grade level in four-months. This supports the idea that more eSpark usage resulted in greater understanding of the content.

Looking with a deeper lens at the data, it became apparent that students were seeing more gains in Language Arts when compared to Math. Additionally, Grade 5 students showed greater gains than Grade 4 students.

Figure 5.2: Grade level growth after one-semester of eSpark usage.

14

Figure 5.3: Grade level growth, as indicated by percentage of Mission completed.

Figure 5.4: Grade level growth by subject after one-semester of eSpark usage.

15

Let’s Go Learn End-of-Year Assessment (February 2013)Tracking Progress from Semester One to Semester Two, and Full-Year Analysis

While student growth was not as high in semester two, there were still indications of success. Once again, students saw more growth in the subjects where there was supplemental eSpark use. Specifically three times as much growth when compared to non-goal areas of work.

As in the first semester, Language Arts scores grew higher than Math.

Averaging student growth over the full school year, students scored roughly twice as high in eSpark goal areas when compared to non-goal areas.

An analysis of the data shows that Grade 5 students had an average growth of almost one full grade level in seven months.

Figure 5.5: Semester two growth in goal area vs. non-goal area.

16

Figure 5.6: Grade 5 students consistently showed more growth using eSpark.

Figure 5.7: Full year growth in goal area vs. non-goal area.

17

Grades 2 & 3: iPads used During Math UnitsiPads were used during Grade 2 & 3 Math units both as an assessment tool (ultimately using the TenMarks.com website), and as a supplemental resource for instruction. Each Math unit lasted approximately one month. Below are the results of each Math unit with Grade level indicated.Grade 3 Math Unit: Multiplication Facts 0 to 12

Pre-Test AveragePost-Test Average

Overall Improvement

iPad Class 70% 83% 13%

No iPad Class 63% 77% 14%

Resources used for instruction during math unit: Learning.com videos & curriculum (website), Flash to Pass (App)

Grade 3 Math Unit: Lines, Line Segments, Rays & Angles

Pre-Test Average Post-Test Average Overall Improvement

iPad Class 52% 90% 38%

No iPad Class

53% 81% 29%

Resources used for instruction during Math unit: Math Splash Grade 4 (App), Khan Academy (website), TenMarks practice test (website)

Grade 3 Math Unit: Fractions as Part of a Group

Pre-Test Average Post-Test Average Overall Improvement

iPad Class 77% 84 7%

No iPad Class

72% 77% 5%

Resources used for instruction during Math unit: Fractions: Smart Pirate (App), Pizza Fractions: Beginning with Simple Fractions (App), Match the Fraction (App), Splash Math: Grade 3 Summer Math (App)

18

Grade 2 Math Unit: Comparing Money Amounts

Pre-Test Average Post-Test Average Overall Improvement

iPad Class 82% 90% 5%

No iPad Class

79% 82% 3%

Resources used for instruction during Math unit: Amazing Coin (App), Coin Math (App), Counting Money + (App), Coins Genius (App)

Grade 2 Math Unit: Division as Repeated Subtraction

Pre-Test Average Post-Test Average Overall Improvement

iPad Class 64% 86% 22%

No iPad Class

75% 85% 10%

Resources used for instruction during Math unit: Splash Math Grade 3 (App), Divi-sion Wiz (App), Squeebles DV (App), Flash to Pass

19

iPads made a pivotal contribution to learning for students in Grades 2 through 5, as indicated by the assessment data. This supports the work of many researchers such as Cohen, Ireland and Woolerton, and Manuguerra and Petocz, who have posited that the iPad can be a powerful device in the hands of educators and students by positively influencing learning outcomes.

It is worth noting that it is not simply the introduction of iPads that contributed to the results we have found. As part of this process, efforts were made to find high-quality content and resources for the students to assist their learning. Additionally, we strove to provide a differentiated experience to the students, making their work more individualized, and therefore more impactful. Furthermore, the teachers of the students examined in this study were highly involved in the overall process. The Department of Education and Early Childhood Development (2010) suggest in their study that it is not enough to simply give a student an iPad and expect results. Rather the technology needs to be supported by committed faculty members: “Quality teaching is (as has been found in other studies) the factor which enables the iPad to be used effectively to improve student motivation, engagement and learning outcomes.”

Throughout this process many anecdotal comments were collected that address themes lying outside the scope of this research study. Included here are the prevalent topics that arose in discussing other benefits of the iPads introduction. Both teachers and students referenced the high levels of engagement students had with the iPad. A teacher offered the description “Anytime I said we were going to use the iPads, one-hundred percent of the students were thrilled”.Teachers spoke to how students as young as Junior Kindergarten required little to no instruction on how to use the device on a basic level. One faculty member attributes this to “the fact that they have been raised in a world of technology; it is all already so natural and comfortable.” The power of differentiation was also noted by teachers and administrators, with one

CONCLUSIONS & DISCUSSION

6

20

administrator explaining how the iPads “have made a huge impact on how we teach, what we teach and how we’re able to reach all the individual levels of our students”. Pragmatically, teachers spoke to how the “instant-on” and”always available” nature of the iPad allowed unforeseen benefits. Specifically, teachers no longer needed to bring their classes to the computer lab, wasting precious teaching minutes. As well, teachers voiced that while students used iPads during class they as teachers could “get around and help students on a small group or one-on-one basis”.As the device and software evolve, and as students, teachers, and administrators become more comfortable with the technology, we predict greater gains in teacher and learning outcomes due in part to the iPad.

21

As is the case with any research study, certain limitations were present and observed as a part of our process.

All students in Grades 4 & 5 participated in the eSpark program which provided learning content, and included assessments using Let’s Go Learn. Thusly, we were unable to establish a traditional control group of students in those grades not using eSpark and Let’s Go Learn. Instead, we analyzed students’ assessment data by comparing the change in results for subject areas in which eSpark was employed against those areas where eSpark was not employed.

The methodology and data collection process in Grades 2 & 3 underwent refinement as we worked through the school year. As such, we were not able to gather as much data from these grades as we had hoped.

Lastly, as noted previously in this study, our goal has been to establish either a positive or negative correlation between iPad use and changes in assessment data. There were no attempts to establish causality between iPad use and changes in assessment data. By making this distinction we acknowledge that there are a multitude of factors that influence a student’s performance on an assessment that exist outside of the introduction of iPads.

LIMITATIONS

7

22

Bill & Melinda Gates Foundation. (2012). “Technology & Effective Teaching in the U.S.” Retrieved April 16, 2013 from http://www.setda.org/c/document_library/get_file?folderId=304&name=DLFE-1507.pdf Council of Economic Advisers. (2011). “Unleashing the Potential of Educational Technology.” Executive Office of the President of the United States.

Department of Education and Early Childhood Development. (2010). “iPads for Learning - In Their Hands: Evaluation”. Retrieved April 16, 2013 from http://www.ipadsforeducation.vic.edu.au/ipad-student-trial/ipad-research

Ireland, G. V., & Woollerton, M. (2010). “The impact of the iPad and iPhone on education.” Journal of Bunkyo Gakuin University Department of Foreign Languages and Bunkyo Gakuin College.

Johnson, L., Adams, S., and Haywood, K., (2011). “The NMC Horizon Report: 2011 K-12 Edition.” Austin, Texas: The New Media Consortium.

Manuguerra, M. & Petocz, P. (2011). “Promoting student engagement by integrating new technology into tertiary education: The role of the iPad.” Asian Social Science.Michael Cohen Group & USDOE [US Department of Education]. (2011). “Young Children, Apps & iPad.” New York: Michael Cohen Group.Sergio, F. (2012). “10 Ways that Mobile Learning Will Revolutionize Education”. Retrieved April 16, 2013 from http://www.fastcodesign.com/1669896/10-ways-that-mobile-learning-will-revolutionize-education

WORKS CITED

8

23