Guide Data Online - final - Amazon S3€¦ · Welcome to Assessment Data Online. This Guide to Data Online is intended to help you understand the interactive nature of the Assessment

Data Online

Guide to Assessment

Data Online

Developed and published by Data Recognition Corporation, 13490 Bass Lake Road, Maple Grove, MN 55311. Copyright © 2018 by Data Recognition Corporation. All rights reserved. Only authorized customers may copy, download, and/or print the document, located online at http://csp.drcctb.com. Any other use or reproduction of this document, in whole or in part, requires written permission of the publisher. TerraNova is a trademark of Data Recognition Corporation. All other brand and product names are property of their respective owners and not associated with the publisher of this publication.

Copyright © 2018 Data Recognition Corporation. All rights reserved. i

TABLE OF CONTENTS

ASSESSMENT DATA ONLINE: GUIDE TO DATA ONLINE........................................ 1 INTRODUCTION ........................................................................................................................... 1

DASHBOARD HEADER AND REPORT FILTERS ............................................................................. 2 PDF AND EXCEL REPORT VERSIONS ......................................................................................... 6 2011 TERRANOVA3 NORMS ..................................................................................................... 6

USING THE GUIDE TO DATA ONLINE .............................................................................................. 9

ASSESSMENT DATA ONLINE: CLASS LEVEL .................................................... 10 TERRANOVA3 TAB .................................................................................................................. 10

CLASS DASHBOARD ............................................................................................................... 10 NRT SUMMARY DASHBOARD .................................................................................................. 11

TERRANOVA WITH INVIEW TAB ................................................................................................ 12 SUMMARY DASHBOARD .......................................................................................................... 12 COMBINATION STUDENT ROSTER ............................................................................................ 14

BIBLE TAB .............................................................................................................................. 15 BIBLE SUMMARY DASHBOARD ................................................................................................. 15 BIBLE STUDENT ROSTER ........................................................................................................ 15

LONGITUDINAL TAB ................................................................................................................. 16 TERRANOVA LONGITUDINAL TRACKING AT THE CLASS LEVEL ................................................... 16

ASSESSMENT DATA ONLINE: SCHOOL LEVEL .................................................. 18 TERRANOVA3 TAB .................................................................................................................. 18

NRT SUMMARY DASHBOARD .................................................................................................. 18 OBJECTIVES DASHBOARD ....................................................................................................... 19 ALL GRADES DASHBOARD ...................................................................................................... 20 SCALE SCORE COMPARISON DASHBOARD ................................................................................................. 20 COMPARISON DASHBOARD ............................................................................................................ 21 STUDENT ROSTER REPORT ........................................................................................................................................ 22 SELECTED STUDENT REPORT ................................................................................................. 23 MULTI-SELECTED STUDENT REPORT ...................................................................................... 23 SCHOOL SIZE REPORT .............................................................................................................. 24

TERRANOVA3 WITH INVIEW TAB ............................................................................................... 25 SUMMARY DASHBOARD .......................................................................................................... 25 COMPARISON DASHBOARD .............................................................................................................. 26 STUDENT ROSTER ................................................................................................................... 26

Copyright © 2018 Data Recognition Corporation. All rights reserved. ii

SCHOOL BIBLE ASSESSMENT ................................................................................................... 28 BIBLE SUMMARY DASHBOARD ............................................................................................... 28 BIBLE CLUSTERS DASHBOARD ................................................................................................ 29 BIBLE STUDENT ROSTER ........................................................................................................ 29

LONGITUDINAL TAB ....................................................................................................................... 30 LONGITUDINAL TRACKING AT THE SCHOOL LEVEL .................................................................... 30

RESOURCES TAB ................................................................................................ 32 ASSESSMENT DATA ONLINE FOR PARENTS ...................................................... 33

Copyright © 2018 Data Recognition Corporation. All rights reserved. 1

ASSESSMENT DATA ONLINE: GUIDE TO DATA ONLINE

INTRODUCTION Welcome to Assessment Data Online. This Guide to Data Online is intended to help you understand the interactive nature of the Assessment Data Online dashboards and provide examples of useful ways to organize and analyze your students’ TerraNova, Third Edition, InView, Primary Test of Cognitive Skills (PTCS) (2012-2016 data), and Bible Assessment data. The goal is to help you make meaningful use of your CSP Assessment data in a variety of ways, from reporting to parents, faculty, and your community to identifying insights about the strengths and challenges within your curriculum and instructional programs. This guide has two major sections organized around the structure of Assessment Data Online. One segment is focused on the Class level (Teacher login); the dashboards and reports here provide results for the students in a single class. A single class is defined as all the students submitted under one Group Information Sheet (GIS) for scoring. Note that teachers who submitted two (or more) groups (two or more GISs) for scoring will receive separate logins per group. Each group or class must be accessed separately. The second segment is focused at the School level (School login), with dashboards and reports that summarize data for students across a single grade, form, and level.

Once you have logged on to Assessment Data Online, in the upper right-hand corner of both the Class and School screens, you will see “Welcome” followed by your name. To the right of the Welcome sign is the Logout button, and under the word Logout, the Menu icon. Beneath the Menu icon, a list will show the six tabs that categorize the various dashboards and information available to the Assessment Data Online user. Under each of these tabs you will find several dashboards and options available as you work with the different types of student data at the Class or School levels. Each of these dashboards and options are explained in this Guide to Data Online, with the tabs serving as the headings for each section in this document. A default dashboard will appear when the user logs in as a Teacher (Class level) or School (School level). For the Class level, the default will

be the Class Dashboard; for the School level, the default is the NRT Summary Dashboard. From a default dashboard, the user can select another dashboard to view by clicking on the Menu button, selecting one of the six tabs, and choosing from the options listed in each call-out box. Each dashboard will display important data relevant to its title. For example, under the TerraNova3 tab for the Class level, you can move to the NRT Summary Dashboard. As the title suggests, the norm-referenced results for the students in a class are summarized on this dashboard. Similarly, under the Class Bible tab, the Bible Student Roster lists the students in a class and their Bible Assessment results. Each of the default dashboards provides useful information but may be examined further by using the filters and sorting options available.


DASHBOARD HEADER AND REPORT FILTERS At the top of each dashboard is the dashboard header, a gray bar labeled with the tab names, e.g., TerraNova3 or Bible. Report Filter Options are opened by default upon login. By clicking on this bar you can open or close the filters available for each dashboard. With these filters you can select the report data you wish to review, such as the grade, form/level, and scores

. By clicking on the right side of any option, a drop-down menu of choices under that filter offers specific alternatives, such as each grade level assessed or the range of subtest data. The dashboard can then be customized for the precise information needed. When you have selected the filters to apply, click on the Refresh Report button to submit the dashboard request. The specific report requested will appear within seconds. In addition to defining the basic report to view, filters can help narrow the focus of the data you review, such as limiting the display to the data of only boys or only students who have attended Christian schools for three or more years. For certain dashboards, you may select multiple options within a single filter; for example, you could choose to display both National Percentile (NP) scores and Scale Scores (SS) on a single report.

One caution: as you apply one or more filters, you may significantly limit the data being displayed. While this may be helpful when doing certain analyses, at times it will restrict the amount of data to so few students that the analyses will not meaningfully inform decisions. As your data group is narrowed, there is a greater chance that a few outlier scores (very low or very high) will unduly influence the results you see, especially if you are reviewing mean (average) scores. If the filters you pick result in no students, you will receive the message, “The report is empty.” As you apply the filters and analyze your data, remember to consider group size when drawing conclusions. Decisions that will lead to major changes should be well supported in the data or with multiple data sources. Avoid over-analyzing limited fragments of data, such as the results of one or a few students. The Rank Order option on the Student Roster (School login) or Class Dashboard (Teacher login) will frequently be useful to sort students’ data from high to low according to the selected subtest and score. Such rankings make it easier to note patterns in the data or to identify gaps. Some dashboards will link to other reports, allowing you to drill down to more specific information or to present related data. For example, in the Class InView Student Roster dashboard, you can click on a student’s name to link to that student’s Individual Student Profile report. Drilling down into related data can add richness to your understanding of the details underlying the broader data picture. Be aware that a few School-level dashboards include either or both region and CSP data as well as school data, such as the Comparison Dashboard under the TerraNova with InView tab. Region and/or CSP data will continually fluctuate as new school data are loaded into Assessment Data Online each year. The numbers of students and mean (average) scores are subject to change until the last school has been loaded. The Report Filter options provide flexibility when you disaggregate the special group populations in Assessment Data Online. These filter options allow you to easily separate out special populations or keep them included with the mainstream population.


These options apply only to the data loaded to Assessment Data Online. They do NOT change the way paper reports are sorted. For information on paper reports, please follow directions in your Test Coordinator’s Manual for using Exclusion groups. In order to use these options in Assessment Data Online, you must accurately bubble the student biographical data special codes fields in the Student Data Grid to identify the students you want to disaggregate in the available categories. Please see the section titled “Student-Identifying Information” in the Test Coordinator’s Manual and bubble appropriately for each student you want in these categories. These enhanced Assessment Data Online Report Filter Options are as follows:

1. The filter labeled “Gifted and Talented” (Bubble Special Code column K with 1. See the Test Coordinator’s Manual.) displays the following three options in the drop-down menu:

All GATE Only Not GATE

2. The filter labeled “Special Programs” (Bubble Program on Student Data Grid. See the Test Coordinator’s

Manual.) is a multi-select control with the following six options in the drop-down menu:

IEP Only 504 Only LEP/ELL Only IEP and LEP/ELL 504 and LEP/ELL None

3. The filter labeled “Other Groups” (Bubble Special Code column M on the Student Data Grid as described

in the Other Groups section below.) is a multi-select control with the following four options in the drop-down menu:

Home School International Other None

Below is an example of the Report Filter Options as shown on the NRT Summary Dashboard:


Let’s discuss the filter called Gifted & Talented (GATE).

By filling in Bubble 1 in the Special Code field column K on the student answer document, you identified this student as a GATE student. The choices in the drop-down menu are ALL, GATE Only, or Not GATE. ALL is the default chosen for the Gifted & Talented filter. This selection includes all of the students, unfiltered. That includes students marked as GATE and those not marked or Not GATE. When GATE Only is selected, the population of students returned will include only students marked as Gifted & Talented in the Special Code field K of their answer document. When Not GATE is selected, the population of students returned includes only those students who were not marked as Gifted & Talented in the Special Code field K of their answer document. In other words, Special Code field column K on the student answer document is blank for these students. The next Report Filter Option is the Special Programs.

By filling in the appropriate Program bubble(s) on the student answer document, you identified this student as having one or more of these special programs. This filter category is a multi-select feature. This feature allows you to select multiple Special Program category selections from one to six. The number of selections chosen will be indicated in the list box, such as 5 selected. The population returned will include only the students with the categories selected.


The drop-down menu also includes a None selection. If you choose only this selection, then the population of students returned will not include any of the Special Programs. However, if None is selected along with any other category, 504 for example, this would return all students who were marked as 504 and all of the students who had nothing marked in 504 on their answer document; in other words, the student answer documents that are blank for these Special Programs and the ones that did have a Special Program bubbled. So in our 504 example, to only see the 504 students, you must select 504 Only and uncheck all other selections, including the None selection. The next Report Filter Option is the category of Other Groups.

By filling in the appropriate bubble on the student answer document, Special Codes field M, you can identify your student as being either Home School (bubble 1), International (bubble 2), or Other (bubble 3). See Figure 1 for the location of the Special Codes field column M and associated values for Other Groups. If you did not bubble one of these other special group categories on the student answer document Special Codes field column M, then no Other Group data will exist for reporting. FIGURE 1- STUDENT ANSWER DOCUMENT SPECIAL CODES FIELDS – COLUMN M

Please note that on the student’s answer document you should only bubble one of the choices for Other Group (Special Codes column M) per student. Two or more bubbles in this field column will be considered a multiple-mark.


This filter also includes the multi-select feature that allows you to select a multiple number of the Other Groups categories to be selected, including None. The number of selections chosen will be indicated in the list box, such as 3 selected. The population returned will include only the students with the categories selected. For example, if only Home School is selected then only the students who have the bubble 1 filled in on their answer document, Special Codes field column M, will be identified in the report. To see only a specific population, such as Home School then you must uncheck all other selections, including None. PDF AND EXCEL REPORT VERSIONS On the dashboard header are two buttons for creating a PDF version or an Excel export file from any report created within Assessment Data Online. For example, if you use filters to create a screening list of potential students for the Gifted/Talented program, you can click the PDF button in the upper right corner of the functions bar, and a PDF version of your report will be created. This is a static file that cannot be altered, but can be saved and printed for future use. To the right of the PDF button on the functions bar is an Excel button. By clicking this button the data in any report can be exported as an Excel spreadsheet. After an Excel spreadsheet is saved, the data within that file can be manipulated using Excel functions, such as sorting the scores, calculating NCE or Scale Score means for a subset of students, or creating data graphs. ACCESS TO PAPER REPORT PDFS For Spring 2017 and onwards, PDFs of paper reports are accessed in the DRC INSIGHT Portal. Please refer to the document titled “Steps to Access PDFs in DRC INSIGHT Portal” (page XX, Appendix A, of this guide) for instructions to access PDFs for Spring 2018. For the 2012-2016 administrations, PDF versions of certain TerraNova3 paper reports are also available under each tab at the Class and School levels. All PDFs for a given group are posted under each of the TerraNova tabs and are the same under the TerraNova3 tab, the TerraNova with InView tab, and the TerraNova with PTCS tab. Similarly, certain Bible Assessment reports are available as PDF files.

When you click on PDFs under a Class tab, you will see the four PDF reports available for the teacher’s class and students. These are the Parent Report, the Individual Profile Report, the Group List Report, and the Objectives Performance Report. Under the Bible tab, PDFs of the Student Report and the Roster Report are the two options.

When you click on PDFs under a School tab, you will see the three PDF reports available for the grade, form, and level of TerraNova3 administered at a grade level. These are the Group List Report, the Objectives Performance Summary, and the Assessment Summary. Under the Bible tab, a PDF of the Bible Summary is the single option. Note: The Objectives Performance Summary, Assessment Summary, Group List Report, and Objectives Performance Report were not available for 2017. The content in these reports, however, can be accessed in the Assessment Data Online dynamic screens.

By clicking on the selected PDF report under a tab, the PDF of that report will appear and be available to save or print. The PDF reports are identical to the paper reports that are shipped to the CSP school in the CSP Report Package with the exception of the Parent Report. The Parent Report is available in PDF only and not available as a printed report. It is provided in PDF by Class and by individual student within Assessment Data Online for Parents. There are times when it will be convenient to print a PDF copy of a report from Assessment Data Online. For example, a teacher may want to print a copy of a student’s Individual Profile Report to share at a parent-teacher conference or as a handout at an Intervention Team planning meeting for a particular student. A principal may want extra copies of a grade level’s Objectives Performance Summary to share with teachers at a grade-level curriculum review meeting. 2011 TERRANOVA3 NORMS

The CSP Assessment Program strives to provide the most current norms available to ensure accurate comparisons between CSP students’ achievement and the national picture. As such, beginning in spring 2014, the TerraNova3, CSP Edition norm-referenced achievement results will be updated from 2007 norms to 2011 norms. These revised norms are reported on both the TerraNova3 paper reports as well as the TerraNova3 results in Assessment Data Online.


These TerraNova3 norms are anchored in the 2007 large scale TerraNova3 national standardization study updated with a large convenient sample of students tested in 2011; sophisticated statistical procedures were used to yield precise data. The resulting norms ensure that the comparison of CSP students’ performance to the national norm group reflects current expectations and performance nationwide. Beginning in spring 2014, all CSP TerraNova3 achievement scores reflect the 2011 norms. However, some dashboards in Assessment Data Online have portions that will continue to provide data from the 2007 norms as well. This is primarily found in the Report Filter Option where the user is given a choice of Administration Year for viewing the data. In these instances, when data from 2012 or 2013 is requested, the TerraNova3 achievement scores shown will be based on the 2007 norms. When an administration year of 2014 or later is requested, the TerraNova3 achievement scores shown will be based on the 2011 norms. Longitudinal reports at both the School and Class levels may also show scores from both the 2007 and 2011 TerraNova3 norms. As CSP schools transition from the 2007 norms used in spring 2013 to the 2011 norms in spring 2014, minor shifts in year-to-year TerraNova3 results may be noted. In most cases, the National Percentile (NP) differences will be minor, typically within 1 to 2 percentile points. For example, a third-grade student with a Reading Scale Score of 606 would receive a NP score of 25 on 2007 norms; with 2011 norms, this same Scale Score yields a NP of 23. Similarly, a third-grade Reading Scale Score of 650 would result in a 2007-based NP of 75 and a 2011-based NP of 76. Such variations are expected when updating norms and reflect the most recent fluctuations in national performance. The specific dashboards and PDF reports affected are listed below.

SCHOOL Level Dashboard or Report Prompt Norm Scores

Displayed NRT Summary Dashboard

TerraNova only; TerraNova with InView; TerraNova with PTCS

Note: PTCS data is only available for 2012-2016 in ADOL

Report Filter Option: Admin Year

2012-2013 = 2007 norms 2014 and beyond = 2011

norms

All Grades Dashboard Report Filter Option:

Admin Year

2012-2013 = 2007 norms 2014 and beyond = 2011

norms

Scale Score Comparison Dashboard Report Filter Option:

Admin Year

2012-2013 = 2007 norms 2014 and beyond = 2011

norms Comparison Dashboard




2012-2013 = 2007 norms 2014 and beyond = 2011

norms

Student Roster TerraNova only; TerraNova with InView;

TerraNova with PTCS Note: PTCS data is only available for 2012-2016 in ADOL


2012-2013 = 2007 norms 2014 and beyond = 2011

norms

Individual Student Report (linked from the Student Roster or

Class dashboards)

Depends on Admin Year requested on the Student Roster dashboard or the

Class dashboard

2012-2013 = 2007 norms 2014 and beyond = 2011

norms

Selected Student Report Report Filter Option:

Admin Year

2012-2013 = 2007 norms 2014 and beyond = 2011

norms

Multi-Select Student Dashboard Report Filter Option:

Admin Year

2012-2013 = 2007 norms 2014 and beyond = 2011

norms Student Profile Report

TerraNova with InView; TerraNova with PTCS (linked from the Student Roster dashboard)



2012-2013 = 2007 norms 2014 and beyond = 2011

norms


Scale Score Comparison Dashboard Report Filter Option:

Admin Year

2012-2013 = 2007 norms 2014 and beyond = 2011

norms

School Size Report Report Filter Option:

Admin Year

2012-2013 = 2007 norms 2014 and beyond = 2011

norms

Cohort Summary Longitudinal Dashboard Report Filter Options:

Admin Year and Additional Years

2012-2013 = 2007 norms 2014 and beyond = 2011

norms

Cohort Scale Score Comparison Dashboard Report Filter Options:


2012-2013 = 2007 norms 2014 and beyond = 2011

norms

Longitudinal Summary Dashboard Report Filter Options:


2012-2013 = 2007 norms 2014 and beyond = 2011

norms

SCHOOL Level Dashboard or Report Prompt Norm Scores

Displayed

Longitudinal Roster Report Filter Options:


2012-2013 = 2007 norms 2014 and beyond = 2011

norms

CLASS Level Dashboard or Report Prompt Norm Scores

Displayed

Class Dashboard Report Filter Option:

Admin Year

2012-2013 = 2007 norms 2014 and beyond = 2011

norms NRT Summary Dashboard




2012-2013 = 2007 norms 2014 and beyond = 2011

norms

Student Roster TerraNova only; TerraNova with InView;

TerraNova with PTCS


2012-2013 = 2007 norms 2014 and beyond = 2011

norms

Longitudinal Summary Dashboard Report Filter Option:

Admin Year

2012-2013 = 2007 norms 2014 and beyond = 2011

norms

Longitudinal Roster Report Filter Option:

Admin Year

2012-2013 = 2007 norms 2014 and beyond = 2011

norms

PDF Reports in Assessment Data Online (PDF reports for 2012-2016 only*)

2007 Norms 2011 Norms

On the PDF reports as on the paper reports, TerraNova3 achievement scores will be reported based on 2011 norms; the TerraNova3 objectives data and all InView/PTCS data are reported based on 2007 norms.

Group List Report – TerraNova3 achievement only --- 2011

Group List Report –TerraNova with InView TerraNova summary

--- 2011 TerraNova3 achievement data

Group List Report – TerraNova and InView InView/PTCS summary data

2007 InView/PTCS data

---

Assessment Summary – TerraNova achievement only --- 2011 TerraNova3 achievement data

Assessment Summary – achievement only and Performance on Objectives pages

2007 Objectives data

---

Assessment Summary – TerraNova achievement

--- 2011 TerraNova3 achievement data


Assessment Summary –TerraNova3 with InView or PTCS summary


---

Assessment Summary – combination (Performance on Objectives page)


---

Assessment Summary – combination (Performance Level Summary page)

2007 Performance Level data

---

Objectives Report 2007

Objectives data ---

Objectives Summary Report 2007

Objectives data ---

Individual Profile Report – TerraNova3 Objectives only


---

Individual Profile Report – TerraNova3 achievement only

--- 2011

Individual Profile Report – combination InView and PTCS subtest scores Note: PTCS data is only available for 2012-2016 in ADOL


---

*For Spring 2017 and onwards, PDFs of paper reports are accessed in the DRC INSIGHT Portal. Please refer to “Steps to Access PDFs in DRC INSIGHT Portal” (page 35 of this guide) for instructions to access PDFs for Spring 2018. USING THE GUIDE TO DATA ONLINE This Guide to Data Online document provides an explanation of each dashboard and examples of how a dashboard may be used. The discussion is organized in the following manner:

I. Class-Level Dashboards (accessed through a Teacher log-in) a. TerraNova3 b. TerraNova with InView and TerraNova with PTCS (2012-

2016 data) c. Bible Assessment d. TerraNova Longitudinal

II. School-Level Dashboards (accessed through a School log-in) a. TerraNova3 b. TerraNova with InView and TerraNova with PTCS (2012-

2016 data) c. Bible Assessment d. TerraNova Longitudinal

III. Resources a. Resources common to the Class-Level and School-Level

dashboards (User’s Guide, Score Legend, Guide to Data Online, Performance Levels: Bible, Performance Levels: TerraNova, and the 2012 TerraNova: SAT 10 Linking Study)

b. School-Level Resources: information for longitudinal tracking


ASSESSMENT DATA ONLINE: CLASS LEVEL

TERRANOVA3 TAB CLASS DASHBOARD

The default TerraNova3 Class Dashboard provides a class’s student roster in alphabetical order along with norm-referenced scores and color-coded objectives performance data for each student. By selecting a specific subtest or composite score from the Subtest drop-down menu, you see students’ results for the selected subtest. Additional choices from the various drop-down menus, such as selecting one or more scores or a specific student subgroup, allow you to drill further into the data. Check the Assessment Data Online User’s Guide for directions on using the drop-down menus to create more detailed reports.

The TerraNova3 Class Dashboard is useful for creating disaggregated reports focused on the achievement of specific student groups or producing rank-ordered data reports that are useful in reviewing curriculum and instruction within a content area. The report in the example ranks results by Reading National Percentiles (NPs). These data can then be analyzed based on how students’ performance on Reading objectives changes as NPs decline. This analysis can provide insights for strengthening the Reading curriculum and modifying instruction for students with moderate and low Reading achievement.

This example shows Reading subtest results reported and ranked by Reading-NP. Students’ corresponding Reading objectives performance is shown in the color-coded columns on the right. The three green circles for students with

high NPs indicate strong performance on all Reading objectives.

As NPs decline, the yellow circles for the objective “Analyze Text” indicate this as a challenge area for lower achieving readers.

Using information in the Teacher’s Guide to TerraNova3, the teacher can identify sub-skill areas within the objective “Analyze Text” to strengthen in-classroom Reading instruction.

In this example, the default Class Dashboard displays students’ names in alphabetical order and data for the Reading subtest. Four norm- referenced scores along with the color-coded objectives assessed on the Reading subtest are shown on the default screen.


Clicking on a student’s name links you to the student’s individual TerraNova3 report. Two tables–Relative Objective Strengths and Relative Objective Needs–display five specific objectives each; these strengths and needs are based on the student’s TerraNova3 results. This information is useful in parent- teacher conferences as well as when planning instruction and support services for a student. Below these tables is a bar graph presenting the student’s National Percentile (NP) results for each subtest; a numeric chart to the right of the graph presents subtest results in a variety of scores. Below this chart is the student’s Lexile score and Lexile range. The Lexile data is useful in a variety of ways, including:

determining if a student is reading below, at, or above grade-level expectations; tracking a student’s reading progress across grade levels; and grouping students for reading instruction and selecting appropriate books for students.

Additional information about Lexiles, including a database of books sorted according to Lexiles and Lexile ranges by grade level, can be found at www.lexile.com. NRT SUMMARY DASHBOARD

The NRT Summary Dashboard presents a bar graph of the class’s norm-referenced results for each TerraNova3 subtest and the PLUS tests, if they were administered. The bar graph provides a quick snapshot for this group of students’ achievement in various subtests. These bar graph data are easily analyzed by referencing each bar’s position relative to the national average of 50 NP (National Percentile)/50 NCE (Normal Curve Equivalent). The group’s numeric scores for each subtest are found in the table to the right of the graph. Disaggregated reports for specific subgroups within the class can be created by using the filters in the report header, such as Years in Christian Schools or Ethnicities. By clicking directly on a subtest’s bar in the graph, you can drill down to summarized results for the objectives assessed in that content area. Beginning in the middle column of the resulting Objectives Summary table, there is a breakout of students’ performance for each Mastery Category within each objective.

The percentage of students at High Mastery indicates how well the curriculum and instruction supported students at achieving grade-level expectations on the TerraNova3 content. Clicking on the underlined word High, Moderate, or Low in the Mastery column for an objective leads to a drop-down menu listing the names of students scoring in this category for this objective. This information can be useful when grouping students for differentiated instruction, as well as identifying students for accelerated instruction or intervention support in specific areas.

Note: At the Class level, both the Class and the NRT Summary dashboards have CSP norm scores in addition to national norm scores. CSP norm scores are included under the Score filter option as Mean Local NCE, NP of Mean Local NCE, and/or Local Stanine.

The categories for Objectives Mastery are:

High (at or above grade-level expectations)

Moderate (some skills but not yet performing at grade level)

Low (significantly low for grade level)


TERRANOVA WITH INVIEW TAB SUMMARY DASHBOARD The Summary Dashboard for TerraNova3 with InView is a bar graph display of the mean (average) TerraNova3 achievement results compared to the mean anticipated achievement results based on InView and TerraNova3. A table to the right of the bar graph provides numeric data of the mean scores for this group of students, including the Normal Curve Equivalent (NCE) scores. Obtained achievement- anticipated achievement comparisons are reported for the TerraNova3 subtests listed under the Subtest drop-down menu. You may select one, some, or all of the subtests listed when creating a report. Interpretation of these data primarily centers on analyzing differences between the achievement obtained on TerraNova3 and the anticipated achievement based on InView results.

A Mean NCE higher than the Mean Anticipated Achievement NCE indicates that these students’ TerraNova3 achievement is higher than expected based on the students’ InView cognitive abilities results. A Mean NCE lower than the Anticipated Achievement NCE indicates that TerraNova3 achievement is lower than expected based on the students’ InView cognitive abilities results.

Small differences between obtained and anticipated achievement are typically seen, but when is this difference meaningful? To answer that question, it may be helpful to change the Score filter to Mean NCE and view this graph based on Mean NCEs. A commonly used rule of thumb for a meaningful difference between scores is 7 or more NCEs. There is no drop-down menu for the InView anticipated achievement data on this bar graph; a drop-down menu for the TerraNova3 achievement data is available under the TerraNova3 tab, NRT Summary Dashboard. Disaggregation reports can be created for subgroups within this class by using the filters in the report header, such as Years in Christian Schools or Ethnicities. Mean Cognitive Skills Index (CSI) scores are not found on this dashboard but are available on the PDF/paper report for Group List Report with InView. To access the PDF version of this report, go to the TerraNova with InView tab and select the PDFs option. The Group List Report option is listed under Report Name. Click through to view the report. The InView Student Roster provides a list of students in alphabetical order with each student’s InView results by subtest. The scores on this page are unique to InView with the exception of Scale Score, which is also available for TerraNova3.

Acronyms InView Score Names Explanation of the InView Scores

NPA NSA

National Percentile by Age National Stanine by Age

uses NP or NS to compare the student’s results to other students with the same age, regardless of grade

NPG NSG

National Percentile by Grade

National Stanine by Grade

uses NP or NS to compare the student’s results to other students in the same grade, regardless of age

CSI

Cognitive Skills Index a cognitive abilities score based on the combined results from all InView subtests

Clicking on a student’s name will link you from the InView Student Roster to that student’s individual assessment results. This Student Profile report presents the TerraNova3 and InView data in an easy-to-read graphic and numeric format.


The data on the individual Student Profile is useful in a number of ways. For example: Patterns of NP scores lower than the Anticipated Achievement NP may signal a possible learning

disability. The CSI can be used when screening for special programs, such as Gifted and Talented. The DIFF scores can indicate a possible learning problem or an under-achieving student.

The DIFF score can be useful when exploring whether a student has a learning problem. If there is no significant difference between the student’s TerraNova3 achievement and Anticipated Achievement, the DIFF will be “NA” (not applicable, indicating no significant difference). If the student’s TerraNova3 achievement is significantly higher than anticipated, the DIFF will be a plus (+) sign. If the student’s TerraNova3 achievement is significantly lower than anticipated, the DIFF will be a minus (−) sign. The InView subtests’ bar graphs make it easy to identify areas where students may benefit from additional support. For example, a low score in the InView subtest Sequences may indicate a student who will have difficulty with mathematical patterns and algebraic thinking. Follow-up with hands-on and paper activities focused on distinguishing and developing patterns and sequences may help improve these skills.

The following combination of data is found on an individual Student Profile. 1. A bar graph showing NPs for

each TerraNova3 subtest along with the InView anticipated achievement scores

2. A graphic display of the student’s CSI score on the CSI scale, with the “average range” for CSI scores (84–116) shown by a bold brown line

3. A table with TerraNova Subtest Scores presented in a variety of metrics, such as Scale Score, National Percentile, and DIFF score (an indication that the disparity between the TerraNova3 NP and the Anticipated Achievement NP is significantly different)

4. A bar graph showing NPs for each InView subtest; the scores displayed are NPs by Grade (NPG)


COMBINATION STUDENT ROSTER

The Combination Student Roster provides a list of students in alphabetical order with each student’s TerraNova3 results reported in a variety of score types. The three key scores used when analyzing these data are listed:

Analysis of a student’s NP, AANP, and Diff scores may prove especially insightful when examining the needs of low-achieving and very high-achieving students. Note that if PLUS tests are administered with TerraNova3, the data columns will be wider than the screen; scrolling to the right will bring additional subtests and scores into view. By clicking on a student’s name, you will access the individual’s Student Profile report. This is the same report that can also be accessed from the InView Student Roster and is described above. Note: The Student Profile report is focused on the TerraNova3 subtest information. Students who did not take TerraNova3 will not receive a report. The page will be empty. For example, a student who took InView but not TerraNova3 will have an empty Student Profile.

The dashboards and Student Profile formats are the same for TerraNova3 with Primary Test of Cognitive Skills (PTCS) and TerraNova3 with InView.

The differences are (1) the PTCS subtests will be reported on the PTCS Student Roster and Student Profile reports rather than the InView subtests and (2) Anticipated Achievement scores are not available at the Kindergarten level. Otherwise, the scores reported and the dashboard formats will be identical for the Summary Dashboard, PTCS Student Roster, and Combination Student Roster.

Refer to the TerraNova3 with InView section for information relevant to the TerraNova3 with PTCS reports.


1. NP = National Percentile for each TerraNova3 subtest

2. AANP = Anticipated Achievement NP based on InView results in combination with TerraNova3 results

3. Diff = indicates a significant difference between the NP and AANP for this subtest or Total


BIBLE TAB BIBLE SUMMARY DASHBOARD

The Bible Summary Dashboard provides a quick snapshot of a class’s performance on the CSP Bible Assessment.

A color-coded column chart displays the percentage of students in each Performance Level (PL) on this assessment. By placing the mouse over a colored section on the column, the title of that PL and the percentage of students in this class scoring within that PL will appear.

A bar graph presents the class’s mean (average) percent correct for each Content Cluster and Process Cluster. A quick visual analysis shows which areas students performed well in and those areas where a review of the Bible curriculum and instructional program may be warranted.

A data table provides the numbers underlying the bar graph display, including the number of items per cluster and this group’s mean (average) number correct/percent correct. These additional details provide further clarification of this group’s performance on the Bible Assessment.

The Bible Summary Dashboard is an overall summary report without drop-down menu functions. Disaggregated reports for specific subgroups within the class can be created by using the filters in the report header, such as Years in Christian Schools or Ethnicities.

BIBLE STUDENT ROSTER

The Bible Student Roster presents Bible Assessment results for each student in the class; on the default dashboard, students are listed alphabetically by last and first names. The Bible Assessment total scores are presented first, including Number Correct, Scale Score, and Performance Level, followed by the student’s number and percent correct for each Content Cluster and Process Cluster on this assessment. The cluster information is wider than a single screen and can be accessed by scrolling to the right of the last visible column.

When students are listed alphabetically, it is easy to locate the Bible Assessment results for an individual student. However, it is more difficult to analyze the effectiveness of the Bible curriculum and instruction for these students with this data display.

By changing the Scores and Rank Order filters to PL and clicking Submit, the students’ data are rearranged from highest to lowest according to PL. It is now easier to analyze cluster differences between high-achieving and low-achieving students. In this example, the teacher was able to determine that Basic

and Below Basic students tended to perform better in the Christ and Old and New Testament Stories clusters, but less well in the Bible Organization/Structure and Church clusters.

These data-supported insights can inform future Bible curriculum and instruction.


LONGITUDINAL TAB

TERRANOVA LONGITUDINAL TRACKING AT THE CLASS LEVEL

Assessment Data Online provides longitudinal tracking of individual students’ TerraNova3 results across more than one year. This option is available only for students who are tested on grade level and have valid scores. The information is useful for teachers, administrators, and parents/guardians to follow students’ academic progress as they move across grades. Tracking is available for Grades 1−12, but not Kindergarten since there is no previous year for baseline. This option is also not available for ungraded or adult students. The default longitudinal report compares two years of data: the current year and previous administration year; however, up to four years of TerraNova achievement data may be tracked online. In viewing the longitudinal report, the oldest year will be displayed first, with each more recent year in sequence to the newest (current) year. This allows students’ academic progression to be easily analyzed. While no printed reports of longitudinal results are included in a school’s report package, a PDF can be printed for a longitudinal report created in Assessment Data Online. Longitudinal tracking is only available for TerraNova3 results; no longitudinal results are available for InView, PTCS, or the Bible Assessment. As previously described in the Introduction of this guide, TerraNova3 achievement norms for CSP schools were updated beginning in spring 2014. When reviewing longitudinal data, it is important to recognize that 2012 and 2013 TerraNova3 data are based on the 2007 norms; 2014 and beyond are based on the 2011 norms. Longitudinal tracking at the Class level requires matched student data; a unique 6-digit student ID number enables a student’s data to be tracked from one year to the next. The same exact 6-digit assigned ID number must be used across years and must be pre-coded or entered on student answer documents to enable longitudinal tracking of students’ TerraNova results. Beginning spring 2015, the student ID number has been added to the Longitudinal Roster dashboard. You can download a version of this report (PDF or Excel) to obtain the student ID numbers currently in use. For prior years (2012-2014), student ID numbers, when used, are listed on the School Group List Report (PDF or paper version). Two longitudinal tracking dashboards are available at the Class level:

Longitudinal Summary dashboard: This report will graphically display TerraNova3 class mean scores for the same matched students on all subtests. In other words, in this example, the only students included will be those students who have TerraNova3 data for both Grade 5 (current year) and Grade 4 (previous year). While all TerraNova3 subtests and Plus tests data will be available, a maximum of seven subtests can be displayed at one time.


Longitudinal Roster dashboard: This is a roster list of all “matched” students, that is, current year students who also have scores from the previous years’ TerraNova results. Instead of a graphic display of students’ results, this dashboard presents the numeric scores for each year of TerraNova3 included. As with the Longitudinal Summary dashboard, the data for all TerraNova3 and Plus tests are available, and up to seven subtests can be displayed on a single longitudinal report. Four scores are displayed: Scale Score, NP, NCE, and National Stanine. Beginning spring 2015, the student ID number has been added to the Longitudinal Roster dashboard. You can download a version of this report (PDF or Excel) to obtain the student ID numbers currently in use. For prior years (2012-2014), student ID numbers, when used, are listed on the School Group List Report (PDF or paper version).


ASSESSMENT DATA ONLINE: SCHOOL LEVEL

TERRANOVA3 TAB NRT SUMMARY DASHBOARD

The NRT Summary Dashboard displays a bar graph of norm-referenced subtest results for a grade and level in the school. The default screen will present the data for students tested on-grade level, e.g., Grade 3 students assessed with TerraNova3, Level 13. Data are included from all students at this grade level with valid TerraNova3 scores. If some students in your school were tested off-level for their grade, those results must be viewed separately by using the Grade and Level/Form drop-down menus. The reported score in the default dashboard is the National Percentile of the Mean Normal Curve Equivalent (NP of the Mean NCE). The graph can also display Mean NCE results by changing the Score filter to Mean NCEs. Because of the relationship between NPs and NCEs, the graph will appear the same, but the data in the table of scores will change. When the mouse is moved over each bar, the subtest name is displayed. Disaggregation reports can be created for subgroups within this grade level by using the filters in the report header, such as Years in Christian Schools or Ethnicities. Educators can quickly read the overall academic performance at this grade level by reviewing (1) the relative height of each subtest bar to the other bars and (2) each bar’s position relative to the national average of 50 NPs or 50 NCEs. The data table to the right of the bar graph adds additional information to clarify this picture, including the number of students on this report, the average Scale Score for this group, and the numeric data represented in the bar graph display.

By clicking directly on a subtest’s bar in the graph, you can drill down to summarized results for the objectives assessed in that subtest. Beginning in the middle column of the resulting Objectives Summary table, there is a breakout of students’ performance for each Mastery Category within each objective.

The percentage of students at High Mastery indicates how well the school’s curriculum and instruction supported students in achieving grade-level expectations on the TerraNova3 content. This information can be used to identify specific objectives within the content area where students’ average performance is stronger or weaker. Such insights are useful when reviewing and adjusting the curriculum and instructional program related to this content area. It is possible to identify the names of students in each Objectives Mastery category in one of two ways:

1. Logging in to the Teacher section of Assessment Data Online, going to the TerraNova3 tab and then the NRT Summary dashboard. As a school administrative user, you will be able to access a similar bar graph based on a single class. Follow the directions in the Teacher’s NRT Summary section of this CSP Guide to Data Online to access students’ names within each Objectives Mastery category. This approach is useful if the administrator and grade-level teachers are meeting together so the teachers can apply hands-on analysis of their own students’ data.

2. By following the directions for the School Objectives Dashboard on the next page.

The categories for Objectives Mastery are:

High (at or above grade-level expectations)

Moderate (some skills but not yet performing at grade level)

Low (significantly low for grade level)


Note: At the School Level, the NRT Summary dashboard, as well as the All Grades and Comparison dashboards, have CSP norm scores in addition to national norm scores. CSP norm scores are included under the Score filter option as Mean Local NCE, NP of Mean Local NCE, and/or Local Stanine.

OBJECTIVES DASHBOARD

The Objectives Dashboard is a grade and form/level display of students’ performances on the objectives within a single subtest. If any students were assessed off-grade level, their Objectives Report must be accessed separately by using the Grade and Level/Form drop-down menus. (The desired subtest is selected using the Subtest filter in the header of this dashboard.) A column chart graphically shows three Objectives Mastery levels–High Mastery, Moderate Mastery, and Low Mastery–for each TerraNova3 objective assessed in the selected subtest. The Mastery levels are color-coded in each column. Hovering the mouse over a colored section will display the objective name, the name of the mastery category for the specific color bar chosen, and the percentage of students scoring in that category for this objective. Clicking on a specific color bar will drill down to the names of the students at this grade and form/level who scored at this Objectives Mastery level. In addition to the color graphics bar chart, this dashboard provides a data table with information for each objective, the Mastery levels within each objective, and the percent of students at this grade and form/level who scored in each Mastery level.

The information on this dashboard is useful in a variety of ways. It provides valuable information for reviewing the effectiveness of the current curriculum and instructional

program and targeting specific areas for refinements and updates. Teachers can better group students for the upcoming year using the student specific data. This grouping

can make differentiated instruction and supplementary support or enrichment more effective. This information can also inform professional development plans.

The information on the Objectives Dashboard can be further enhanced in two additional ways.

1. Review the paper Objectives Report to identify the specific Moderate Mastery Range for this level’s objectives. This is helpful when refining instruction or grouping students, as the strategies may differ if students are near the top of the Moderate Mastery Range, at the bottom of this range, or scattered widely throughout this range.

2. Review the Teacher’s Guide to TerraNova3 to identify the sub-skills assessed within an objective at this grade’s test level. The assessed sub-skills can provide a connection to classroom lessons, tests, and assignments.

…a list of students' names

and scores in Moderate

Mastery for the selected objective.

Click on the gold section

(Moderate Mastery) to get…


ALL GRADES DASHBOARD

The All Grades Dashboard provides a look at the performance in a specific subtest across all grade levels assessed. Each grade level is represented by a column on the bar graph; this bar represents the mean score of all students who took TerraNova3 on-level. (Students who took TerraNova3 off-grade level are not included in this dashboard.) When you move the mouse over a column on this graph, the grade level and mean score for this group of students is displayed. The subtest to be reviewed is selected using the Subtest filter in the header of the dashboard. The Score filter can be used to select either the National Percentile of the Mean Normal Curve Equivalent (NP of the Mean NCE) or Mean NCE for the data display. In addition to the bar graph, a table of scores will provide the numeric results for each grade.

Analyses of the All Grades Dashboard are focused on looking for trends and benchmarks. For example:

Do the mean scores in this subtest increase across grade levels, decrease across grade levels, or stay relatively consistent across the grades? Or is there an erratic pattern across grades?

At which grade levels are our average performance above the national average of 50 NCEs/50 NPs? At which grade levels are our average performance below the national average of 50 NCEs/50 NPs? In which subtests are our students, on average, performing above the national average of 50 NCEs/50

NPs? In which subtests are we performing below the national average? The trends seen in the TerraNova3 data can have implications for refinements to the school’s curriculum and instructional program.

SCALE SCORE COMPARISON DASHBOARD

TerraNova3 is a vertically-scaled test, meaning that each level of TerraNova3 is statistically linked to the level below and the level above. This linkage spans the grade levels and creates a scale that describes achievement on a continuum across the grades. The Scale Score is the basis for norm-referenced scores and allows TerraNova3 users to track progress across grade levels. This dashboard presents a line graph showing the school’s Mean Scale Score for each grade level at which TerraNova3 is administered. One content area is displayed at a time; that content area is selected using the Subtest filter in the dashboard header. The school’s Mean Scale Score is shown by a blue line; the red comparison line is the National Mean Scale Score. A corresponding table of scores is displayed to the right of the line graph. This table shows the numeric data for the graph (School’s Mean Scale Score; National Norm Group’s Mean Scale Score) for each grade as well as the number tested at each grade in this school. The numeric data


for each point on the line graph can also be found by moving the mouse over any circle on the line graph; the data source (school or national norm group) and the 3-digit Scale Score value will appear.

Analysis of this dashboard centers on comparing where the school’s Mean Scale Score at each grade level is relative to the national norm group. In addition, you will look for trends in the data. For example, are all grade levels consistently above the national comparison group or do some grades dip below the national norm group? If gaps are observed between the school’s average Scale Score and the national group’s average Scale Score, do those gaps increase or decrease across the grade levels? The outcome of these comparisons and trends analyses is to target specific issues within content areas or certain grade levels where achievement improvement is indicated. Data from TerraNova3 paper reports and other Assessment Data Online dashboards will be helpful in clarifying details and illuminating next steps. COMPARISON DASHBOARD

The Comparison Dashboard provides a look at your school’s achievement at a grade level compared to your CSP region and all CSP schools. One, some, or all content areas can be selected for this review. In addition to the graphic display, a table of scores is presented to the right of the bar graph. Information about the school, region, and CSP is shown: the number of students assessed at your school and the group scores (NP of the Mean NCE or Mean NCE) for your school, the CSP region, and CSP. There is one factor to keep in mind as you use the Comparison Dashboard during the spring of the year. The region and CSP data accumulate over several weeks as the tests for schools across the CSP regions are scored by DRC.


The timeframe for this process is approximately April through May each year. That means the data on which the region and CSP means are computed continue to build and change over several weeks. In other words, from day to day, you may see different averages as you view this dashboard. Thus, comparisons may vary somewhat during April and May until all scoring is finalized for each school year, typically around early June. STUDENT ROSTER REPORT

This dashboard is designed to let you customize the grade-level TerraNova3 norm-referenced or Lexile data you want in a report. You select which subtests you want to see (one, some, or all) and which of five scores you want to review (Scale Score, National Percentile, National Stanine, Normal Curve Equivalent, and/or Lexiles–choose one, some, or all). You can choose to present the data alphabetically or in rank order by one of the scores. The Student Roster Report is designed to let you be in the driver’s seat for creating the TerraNova3 norm-referenced or Lexile report to meet your needs. A simple but very useful Student Roster Report is a Lexile report. This is easy to create:

1. Select the grade level for this Lexile report. 2. Select Reading for the Subtest filter. 3. Select Lexile for the Scores filter. 4. Select Reading-Lexile for the Rank Order filter. 5. Click Submit.

The resulting Lexile report will list all the students at the chosen grade level in rank order by Lexiles, from the highest to the lowest. This information has a number of instructional uses, as illustrated by the following questions. Which students read with comprehension at or above grade level? The data in this example is for Grade 8. Depending on the reference used, the 8th-grade Lexile reading range is considered approximately 900–1100 Lexiles. The first three students on this list, reading at 920–980 Lexiles, would be considered performing at grade level.

Which students fit together well in the same instructional reading group? A general rule of thumb is to group students who are within 50–100 Lexile points of each other for reading instruction. Following this guideline, the first four students might make a good instructional group (880–980 Lexile range). How do I put together appropriate summer reading lists for students? It is recommended that independent reading be in the “not too easy-not too hard” range. When using Lexiles it is often suggested to stay within 50 Lexile points of the student’s Lexile score for independent reading. In this example, a student with a Lexile score of 800 would look for books within the 750–850 Lexile range (50 Lexile points lower and higher than his/her Lexile score of 800). Additional information about Lexiles, including a database of books sorted according to Lexiles and Lexile ranges by grade level, can be found at www.lexile.com.


When planning a Student Roster Report, think first: What is my purpose? As with the Lexile example above, the questions you are trying to answer and your purposes for using the data will drive the report you create. SELECTED STUDENT REPORT The Selected Student Report allows you to create a report targeted to a specific TerraNova3 score or range of scores. This is helpful when you are screening for a given purpose, such as identifying students who might need summer school or intervention services, screening students for an academically talented program, or identifying students reading above-grade level and in need of an accelerated−reading program. Once your purpose for selecting students is clear, you are ready to begin. When planning a Selected Student Report, first focus on the criteria and score to be used. If you are screening for low-achieving students who may need reading intervention support, you would select the Subtest Reading and could use a range of 1 National Percentile (NP) as a minimum score and 39 NP as the maximum score. (The default minimum score is 1 and the default maximum score is 99 when you open the Selected Student Report dashboard.) If you are screening for academically talented students in Mathematics, you would select the Subtest Mathematics and could use 97 NP as the minimum score and 99 NP as the maximum score. The number of digits in the minimum and maximum scores depends upon the score being used. National Stanines will have only one digit in the Min Score and Max Score fields; NPs and Normal Curve Equivalents (NCEs) will have one or two digits; Scale Scores and Lexiles will have three digits. In addition, you can select the scores to be reported. While you can have only one Subtest and Score Selection Criteria at a time, you may report as many as five scores on the Selected Student Report. In a way, the analysis for a Selected Student Report is done up front, when you are focusing on the purpose or need for this report and the criteria to use. Once the report has been generated, it provides the information to answer your question or address the identified need. MULTI-SELECTED STUDENT REPORT The Multi-Selected Student Report is similar to the Selected Student Report; however, this version allows you to not only target students within a specific TerraNova3 score or score range, but to view up to three subtest scores at a time. The selected students will be those who have a score within the minimum and maximum range for all selected subtests, not just one of the subtests. For example, to identify students in need of intervention in core content areas, you could target students scoring below the 40th NP in Reading, Language, and Mathematics. The default minimum score is 1 NP; the default maximum score is 99 NP. (Change the maximum score to “39” in the filter options to show only students scoring below the 40th NP.) The subtests selected would be Reading, Language, and Mathematics. The resulting report will be an alphabetical list of students scoring below 40th NP in all three core subtests specified.

In this Multi-Selected Student Report, data were screened for all students with Reading and Language scores below 40 NPs. The students with both scores below 40 NPs are listed alphabetically.


As with the Selected Student Report, only one score selection type, such as scores between 1−39 NPs, can be designated for identifying the students within the targeted group, but up to four scores may be displayed on a report, e.g., Stanine, NP, NCE, and Scale Score. SCHOOL SIZE REPORT The School Size Report allows you to compare your students’ mean TerraNova3 results with CSP schools of different sizes. All CSP schools are included in one of four categories based on the total enrollment of the school. The categories are as follows:

Below 75 students 75–199 students 200–399 students 400 or more students

The TerraNova3 results for all students tested on-grade level from all schools in the same category are combined, a Mean Normal Curve Equivalent (NCE) is calculated, and the National Percentile (NP) of that Mean NCE is displayed. This is computed for each subtest on TerraNova3 and the PLUS tests as well as the composite scores. Note that since this report includes data from all CSP schools, the numbers and scores can fluctuate until the last school is loaded to Assessment Data Online. You generate a School Size Report by selecting one of the fourteen subtests or composite scores with which you wish to compare your results to other schools. All grades are displayed in a single table, from Kindergarten through 12th Grade as well as Ungraded and Post-12th or Adult levels. The report does not display your school’s data separately; instead, your school’s data is included in the average computed for all schools of similar enrollment. A primary use of this report is to check if students at your school achieved as well as or better than students at CSP schools of similar size. This information can be useful when reporting your school’s achievement to faculty, parents, or your school community. It can also be helpful in setting goals if your students’ results were not as high as the achievement of similar schools.


TERRANOVA3 WITH INVIEW TAB

SUMMARY DASHBOARD

The Summary Dashboard for TerraNova3 with InView is a bar graph display of the mean (average) TerraNova3 achievement results compared to the mean anticipated achievement results from InView for students at the selected grade and test level/form. The default dashboard displays results for students who took TerraNova3 and InView on-grade level. If any students at the selected grade were tested off−grade level, e.g., 3rd-graders who took TerraNova3, Level 12 instead of the on-grade TerraNova3, Level 13, their results will be viewed on a separate dashboard by choosing from the Grade and Form/Level menu options. Obtained achievement-anticipated achievement comparisons are reported for the TerraNova3 subtests shown in the Subtest drop-down menu; you can choose one, some, or all subtests for a report. A table to the right of the bar graph provides numeric data as the mean scores for this group of students. Interpretation of these data primarily centers on analyzing differences between the school’s achievement obtained on TerraNova3 and the school’s anticipated achievement based on InView results. A Mean Normal Curve Equivalent (NCE) higher than the Mean Anticipated Achievement NCE indicates that

the TerraNova3 achievement of the students at this grade level is higher than expected based on these students’ InView cognitive abilities results.

When the school’s Mean NCE is lower than the school’s Anticipated Achievement NCE, it indicates that TerraNova3 achievement is lower than expected based on these students’ InView cognitive abilities results.

Small differences between obtained and anticipated achievement are typically seen, but when is this difference meaningful? To answer that question, it may be helpful to change the Score filter to Mean NCE and view this graph based on Mean NCEs. A commonly used rule of thumb for a meaningful difference between scores is 7 or more NCEs. There is no drop-down menu for the InView anticipated achievement data on this bar graph; a drop-down menu for the TerraNova3 achievement data is available under the TerraNova3 tab, NRT Summary Dashboard. Disaggregation reports can be created for subgroups within this school by using the filters in the report header, such as Years in Christian Schools or Ethnicities. Mean Cognitive Skills Index (CSI) scores are not found on this dashboard but are available on the PDF/paper report Assessment Summary with InView and Group List Report with InView. To access the PDF version of these reports, go to the TerraNova3 with InView tab and select the PDFs option. The Assessment Summary and Group List Report options are listed under Report Name. Click through to the report page you want.


COMPARISON DASHBOARD The Comparison Dashboard displays the school’s TerraNova3 achievement at a selected grade and level compared to the overall TerraNova3 achievement for all CSP schools at this grade and level. The Anticipated Achievement (AA) based on InView results is also shown for the school and for all CSP schools. This graphic display is color coded, making it easy to distinguish the school’s results from the CSP results. The numeric data will also be displayed when holding the mouse over a bar on the graph. A table of scores to the right of the bar graph displays the numeric data from the graph.

When selecting the data to compare, select the desired subtests or composite scores from the Subtests filter. You may also select one of two scores to display: National Percentile of the Mean AA Normal Curve Equivalent (NP of the Mean AANCE) or the Mean AANCE. Because National Percentiles are often easier to understand, this is the default score. However, if you are doing any mathematical calculations, such as computing the difference between your school’s TerraNova3 and Anticipated Achievement and that of all CSP schools, it is more accurate to use the Mean AANCE score. It is important to note that the TerraNova3 achievement scores for comparing all CSP schools are from a research study completed in 2012; these results are based on 2007 TerraNova3 norms. Beginning in 2014, individual CSP schools will receive TerraNova3 scores based on the 2011 norms. There may be minor shifts, typically in the range of 1-to-2 NPs, between scores based on 2007 norms and those based on 2011 norms. STUDENT ROSTER The Student Roster provides a list of students at this grade and TerraNova3 level in alphabetical order, with each student’s TerraNova3 results reported in a variety of score types. Three key scores are used when analyzing these data:

Analysis of a student’s NP, AANP, and Diff scores may prove especially insightful when examining the needs of low-achieving and very high-achieving students. Note that if PLUS tests are administered with TerraNova3, the data columns will be wider than the screen; scrolling to the right will bring additional subtests and scores into view.

1. NP = National Percentile for each TerraNova3 subtest

2. AANP = Anticipated Achievement NP based on InView results in combination with TerraNova3 results

3. Diff = indicates a significant difference between the NP and AANP for this subtest or Total Score

2

3

12

3

1


ACSI Data Online: Guide to Data Online 17

Clicking on a student’s name will link you from the Student Roster to that student’s individual assessment results. This Student Profile report presents the TerraNova3 and InView data in an easy-to-read graphic and numeric format.

The data on the individual Student Profile is useful in a number of ways. For example:

Patterns of NP scores lower than the Anticipated Achievement NP may signal a possible learning disability or an under-achieving student.

The CSI can be used when screening for special programs, such as Gifted and Talented. The DIFF scores can indicate a possible learning problem or under-achieving student.

The InView subtests’ bar graphs make it easy to identify areas where students may benefit from additional support. For example, a low score on the InView subtest Sequences may indicate a student who will have difficulty with mathematical patterns and algebraic thinking. The teacher can follow up with hands-on and paper activities focused on distinguishing and developing patterns and sequences to help improve these skills.

The dashboards and Student Profile formats are the same for TerraNova3 with Primary Test of Cognitive Skills (PTCS) and TerraNova3 with InView.

The only differences are (1) the PTCS subtests will be reported on the PTCS Student Roster and Student Profile reports rather than the InView subtests and (2) Anticipated Achievement scores are not reported for Level 10 (Kindergarten). The other scores reported and the dashboard formats will be identical for the Summary Dashboard, Comparison Dashboard, and the PTCS Student Roster. Refer to the TerraNova3 with InView section for information relevant to the TerraNova3 with PTCS reports.

Note: PTCS data is only available for 2012-2016 in ADOL.

1

2

3 4

The following combination of data is found on an individual Student Profile. 1. A bar graph showing NPs for

each TerraNova3 subtest along with the InView anticipated achievement scores

2. A graphic display of the student’s CSI score on the CSI scale, with the “average range” for CSI scores (84–116) shown by a bold brown line

3. A table with TerraNova Subtest Scores presented in a variety of metrics, such as Scale Score, National Percentile, and DIFF score (an indication that the disparity between the TerraNova3 NP and the Anticipated Achievement NP is significantly different)

4. A bar graph showing NPs for each InView subtest; the scores displayed are NPs by Grade (NPG)

1

2

3

4


SCHOOL BIBLE ASSESSMENT BIBLE SUMMARY DASHBOARD

The Bible Summary Dashboard presents a bar graph displaying the percent of students for each of three groups (school, the school’s region, and all CSP schools) according to four Performance Levels (PLs): Advanced, Proficient, Basic, and Below Basic. The data presented is specific to the selected grade and level. If some students took the Bible Assessment off-level, their data will be presented on a separate display by selecting the appropriate Grade and Level options. By placing the mouse over a colored bar, the title of that PL and the percentage of students scoring within that PL will appear. A table of scores to the right of the graphic displays the numeric data detailing the bar graph information. Note that since this report includes data from all schools in your region, as well as the CSP organization as a whole, the numbers and percentages can fluctuate until the last school is loaded to Assessment Data Online.

While the goal is for all students to score in the Proficient or Advanced PLs, in reality scores are often spread across a range of levels. Interpreting the information on this dashboard typically centers around two basic questions:

1. Is the percentage of our students in the Advanced and Proficient Performance Levels the same or higher than the percentages for the region and all CSP schools?

2. Is the percentage of our students in the Basic and Below Basic Performance Levels the same or lower than the percentages for the region and all CSP schools?

You can create a variety of disaggregated reports to further explore the answers to these questions. For example, you can compare the performances of girls or boys compared to the region and all CSP schools, or evaluate based on the number of years students have been enrolled in Christian schools. Information from the Bible Clusters Dashboard and the Bible Student Roster can provide additional information to identify possible enhancements for future Bible curriculum and instructional programs.


BIBLE CLUSTERS DASHBOARD

The Bible Clusters Dashboard provides a quick snapshot of students’ performance on the CSP Bible Assessment at the selected grade and level. A bar graph presents the mean percent correct for each Content Cluster and Process Cluster. A quick visual analysis shows in which areas students performed well and those areas where a review of the Bible curriculum and instructional program may be warranted. For example, on this Bible Clusters dashboard, the Church cluster has noticeably lower achievement than the other Content Clusters or the Process Clusters.

A data table provides the numbers underlying the bar graph display, including the number of items per cluster, points possible, the Mean Number Correct, and the Mean Percent Correct for the selected grade and level. These additional details provide further clarification of this group’s performance on the Bible Assessment. The Bible Clusters Dashboard is an overall summary report without drop-down menu functions. Disaggregated reports for specific subgroups within the school can be created by using the filters in the report header, such as Years in Christian Schools or Ethnicities, to provide more in-depth information. BIBLE STUDENT ROSTER

The default Bible Student Roster shows an alphabetical list of students in the selected grade and level with three Bible scores for each student: the Number Correct (NCR); the Scale Score (SS); and the Performance Level (PL). The alphabetical list is helpful in quickly locating information for an individual student. There are also two additional approaches commonly used to further explore these data.

.

One approach is to create a rank-order list using specific scores. In this example, two scores were requested–the SS and the PL–and the data were rank-ordered by SS. With this list, it is easy to see which students are in each Performance Level and what the spread of Scale Scores is among students within a performance range. This information can be useful when examining results within each PL. Disaggregated reports can also provide more depth to these analyses, e.g., checking the PLs based on number of years in Christian schools.


The administrator could then focus on these areas, such as reviewing these parts of the Bible curriculum for possible enhancements and giving more emphasis on hands-on and interactive instruction in these areas. The combination of analyzing the default dashboard, creating disaggregated reports, and drilling deeper into available information can provide a wealth of information and insights for refining the Bible curriculum and instructional program and enhancing students’ future achievement.

LONGITUDINAL TAB

LONGITUDINAL TRACKING AT THE SCHOOL LEVEL

Assessment Data Online provides two types of school-level longitudinal tracking:

1. individual students’ matched data across two-to-four years; and 2. grade-level cohorts’ TerraNova3 results across two-to-four years.

Tracking of individual students’ data is available only for students who are tested on grade level and have valid scores for all years being tracked. Grade-level cohort data track academic results for non-matched groups. As an example, this year’s 5th-grade cohort will be compared to last year’s 5th-grade cohort, regardless of the number of students in each group. Longitudinal information is useful for teachers, administrators, and parents/guardians to follow students’ and grade-level groups’ academic performance as they move across years. Tracking is available for Grades 1−12, but not Kindergarten since there is no previous year for baseline. This option is also not available for ungraded or adult students. The default longitudinal report compares two years of data, the current year and previous administration year; however, up to four years of TerraNova achievement data may be tracked online. In viewing the longitudinal report, the oldest year will be displayed first, with each more recent year in sequence to the newest (current) year. This allows students’ or grade-levels’ academic progression to be easily analyzed. Longitudinal tracking is only available for TerraNova3 results; no longitudinal results are available for the Bible Assessment, InView, or PTCS. There are no printed longitudinal reports as part of a school’s report package, but a PDF may be printed for any longitudinal report created within Assessment Data Online. As previously described in the Introduction of this guide, the 2011 TerraNova3 norms for CSP schools are used beginning in spring 2014. When reviewing longitudinal data, it is important to recognize that 2012 and 2013 TerraNova3 data are based on the 2007 norms; the data for 2014 and beyond are based on the 2011 norms. You may notice minor shifts between 2013 and 2014 TerraNova3 data, typically in the range of one or two national percentile points. Such variations are common when norms are updated and may reflect the most recent fluctuations in national performance rather than changes in local performance.

Another approach for more in-depth information about the effectiveness of the Bible curriculum and instruction is to review the results of targeted individuals. In this example, the administrator referenced the CSP Bible Performance Level Information under the Resources tab to look up Grade 3 Scale Score ranges for Performance Levels. Using the knowledge that the Basic PL=549−599, he identified students at the high end of the Basic PL--in other words, students who were almost in the Proficient PL. When examining several of these students' data, a pattern emerged, as illustrated in this student's data: a number of the Basic students had lower achievement in the "Church" Content Cluster


Two longitudinal tracking dashboards are available for individual student’s data and both require matched student data. A unique 6-digit student ID number enables a student’s data to be tracked from one year to the next. The same exact 6-digit ID number must be used across years and must be pre-coded or entered on student answer documents in order for a student’s results to be matched. If you need a listing of the student ID numbers already in use for your school or class, beginning spring 2015, the student ID number has been added to TerraNova3 Student Roster and to the Longitudinal Roster. For prior years, student ID numbers, when used, are listed on the School Group List Report (PDF or paper version). The School-level longitudinal tracking options based on matched individual student data are similar to those described on page 10 in the Class-level section of this guide. However, the School-level longitudinal dashboards present data for an entire grade level rather than a single class. These two dashboards at the School level are: Longitudinal Summary dashboard, which presents grade-level mean scores for the matched student

group; and Longitudinal Roster dashboard, listing all students in the current year who have matched data to the

previous year/s being tracked.

Two additional longitudinal tracking dashboards are available for grade-level cohort data across a two-to-four year span. As previously stated, cohorts are not matched groups and include all students who have valid on-level TerraNova3 scores in any year being tracked. The number of students included in one cohort group may vary widely from other comparison cohorts. Cohort Summary dashboard: This shows the school’s mean score for one grade level across multiple

years. This is a same grade comparison, not a same student comparison; for example, it can show 4th grade TerraNova performance across three consecutive years (a 4th grade-to-4th grade-to-4th grade comparison). The data in this dashboard reflects TerraNova3 subtests that are consistently given across all years being tracked. In other words, if Plus tests were administered one year, but not the following year, no Plus test data will be displayed.


Cohort Scale Score Comparison dashboard: The school’s mean scores for all grades are displayed graphically one subtest at a time to compare the school’s results to the national norm group. TerraNova3 data for two, three, or four years may be compared. Again, this dashboard is non-matched data and includes all students in a grade group who took TerraNova on-level and have valid scores. All years being tracked must have the same subtest data in order for a subtest’s results to be displayed. For example, if the Science subtest was given one year but not the following year, no longitudinal results for Science will be displayed.

RESOURCES TAB

Listed below are some of the resources available within Assessment Data Online. Periodically, additional resource documents are identified and added here for your use. User’s Guide – A PDF document explaining the functionality of the Assessment Data Online, including how to log in and access the various dashboards, save reports from these dashboards, and drill down into data. Score Legend – A list of score acronyms with their full names. Guide to Data Online – A PDF of this document with helpful information for interpreting test results. Performance Levels – Bible – A PDF showing Performance Level (PL) score ranges and descriptors for the Bible Assessment. This information can be useful when analyzing the PLs of your students’ Bible Assessment results. Performance Levels – TerraNova – A PDF showing TerraNova3 Performance Level score ranges by grade and test level. This document provides additional information related to whether students are performing at or above grade-level expectations in key content areas. Grade-level expectations are determined based on the following key:

Grade-Level Expectation

TerraNova3 Performance Levels

At or Above Grade Level Advanced Proficient

Not Yet at Grade Level Nearing Proficiency Progressing

Well-Below Grade Level Step 1 (grades 3–12) Starting Out (grades 1–2)


ASSESSMENT DATA ONLINE FOR PARENTS

Assessment Data Online for Parents is a segment of the Assessment Data Online system where parents can log in and view their child’s assessment results and access resources to support their child’s learning. Initially, parents receive an Activation Code used to log in and create an account; parents must have an email account where they can access information related to Assessment Data Online for Parents. Once their account is created, parents can access their child’s TerraNova and Bible Assessment results and all available resources on this parent site. If parents have more than one student enrolled in an CSP school, they will log in one time to see all of their children’s assessment results; however, they will receive a separate Activation Code for each child.

Two dashboards and a resources tab are available on Assessment Data Online for Parents. When parents log on, they land on the TerraNova3 Student Report dashboard.

This dashboard is a bar graph that shows the individual student’s TerraNova results for all subtests completed, including PLUS tests, if administered. No InView or PTCS results are shown.


The Bible Student Report dashboard displays a graphic of the student’s performance on the Bible Assessment, if the student completed that test.

This dashboard explains the student’s Bible Assessment results in terms of Scale Scores and graphically displays the student’s Bible Performance Level. Each of the four Bible Performance Levels is explained.


STEPS TO ACCESS THE PDFS POSTED IN THE DRC INSIGHT PORTAL

Beginning in 2017, PDF Reports are posted in the DRC INSIGHT Portal for you to access and download. To access PDF Reports from prior years (2012-2016), please refer to the instructions on page 6 of this guide. The DRC INSIGHT Portal is the same portal that your school used to complete Online Enrollment for machine scoring. To log in to the DRC INSIGHT Portal, please click https://www.drcedirect.com/all/eca-portal-ui/welcome/DRCPORTAL. To access the system, you will need to log in with the same username and password that you used for Online Enrollment. If the username or password has been forgotten, please use the “Forgot your username or password?” link to retrieve it (screenshot below).

Once you are logged in, click on ‘All Applications’ located at the upper left hand corner of the screen:

Under ‘All Applications’, click on ‘Report Delivery’:


Now click on ‘View Reports’:

You will see the following screen below and first must select the Administration for which you would like to view reports. For Spring, 2017 reports, please select the ‘TERRANOVA’ administration from the drop-down menu. To access Spring, 2018 reports (when they become available), please select the ‘TerraNova Spring 2018’ administration. After the Administration is selected, your Region and School information will be pre-populated in the ‘District’ and ‘School’ fields. Under the ‘Report’ tab, you will see the reports available in the drop-down menu:

Select the Report that you would like to view, and then click on ‘Show Reports’:


You will see the information below appear, and under the ‘Action’ column (far right) you can either open the PDF file or save the PDF file to your computer. This can be done for each report available under the ‘Report’ drop-down menu.

Please note that only the following reports are available for 2017 under the Report drop-down menu: Bible Roster Report, Bible Student Report, Home Report, and Individual Profile Report. Note: The screenshots pictured in this user guide are for demonstration purposes only and contain sample information. The dates shown on the screenshots may be different than the actual dates for the 2018 Christian School Program administration. If you have any questions, please contact the ACSI Member Care at 800-367-0798.

Documents

Guide Data Online - final - Amazon S3€¦ · Welcome to Assessment Data Online. This Guide to Data Online is intended to help you understand the interactive nature of the Assessment