Part V

Scoring Dimensions of the APA RubricComplexityModule VIII112014-2015 NJ APA Teacher Training - Module VIII. This PowerPoint in its entirety is for the exclusive use of New Jersey educators. All other uses of this document are forbidden without written permission from the NJDOE.

In this portion of the training, the scoring dimensions of the APA Scoring Rubric will be introduced, and examples and non-examples of each dimension will be provided. PowerPoint Modules VIII, IX, and X should be viewed together within one session.

Up to this point, the training modules have covered basic test design, selecting CPI Links for assessment, and producing APA instructional evidence that meets the Universal Scoring Rules. You should also have reviewed and become familiar with Chapter 5 of the Procedures Manual on the use of prompts and the prompt hierarchy prior to reviewing these modules on scoring, as the level of independence is reflected in the scoring dimensions of the APA rubric.

If you are still absorbing the information provided thus far, please stop and take time to review the presentations and consider the information presented up to this point. If you are comfortable with these topics, continue the presentation. The next main topic is scoring of the evidence, both by the teacher and by the APA Scoring staff. Module VIII will review the Complexity dimension of the APA Scoring Rubric.APA Scoring RubricThere are three scoring dimensions in the APA Scoring Rubric:ComplexityPerformanceIndependence22014-2015 NJ APA Teacher Training - Module VIII

Each entry will receive three scores: a score for Complexity, a score for Performance, and a score for Independence. Scoring DimensionsDimension01234ComplexityEvidence provided is unscorable; all dimensions will receive a score of zeroCPI Link was assessed, but there are major flaws in the evidenceCPI Link is a Far Link to the grade-level indicatorCPI Link is a Near Link to the grade-level indicatorCPI Link is a Matched Link to the grade-level indicator Performance Evidence is not clear or all items are not marked as correct/incorrectAccuracy of work is 0-39% based on the final activity ORSecond activity includes more intrusive prompt Accuracy of work is 40-59% based on the final activity Accuracy of work is 60-80% based on the final activity Accuracy of work is 81-100% based on the final activity Independence Evidenceis not clear or all items are not marked for Independence/ prompt levelStudent completed items/tasks independently0-39% of the time based on the final activity Student completed items/tasks independently 40-59% of the time based on the final activityStudent completed items/tasks independently 60-80% of the time based on the final activityStudent completed items/tasks independently 81-100% of the time based on the final activity32014-2015 NJ APA Teacher Training - Module VIII

The APA Scoring Rubric is used to score each completed entry of the APA. Each entry is scored based on these three dimensions. It is important to understand how portfolios are scored when collecting evidence for a students portfolio; therefore, each of these dimensions will be addressed in detail, starting with Complexity in this module, Performance in Module IX, and Independence in Module X. Scoring Rubric and APA FormsThe APA Scoring Rubric found on the previous slide is also found in the 2014-2015 APA Procedures Manual. Dont forget to review the Procedures Manual, which includes the APA Scoring Rubric, scoring clarifications, and other topics.42014-2015 NJ APA Teacher Training - Module VIII

It would be helpful to have the current Procedures Manual available to review in conjunction with this scoring portion of the training.Complexity evaluates the CPI Link assessed and how closely the complexity and difficulty (Matched, Near, Far) links to the New Jersey Core Curriculum Content Standards (NJ CCCS) and grade-level cumulative progress indicators (CPI) for science.

5Using the APA Scoring RubricComplexity Dimension2014-2015 NJ APA Teacher Training - Module VIII

The first dimension of the APA Scoring Rubric is Complexity. This dimension evaluates the complexity and difficulty of the CPI Link assessed in relation to the grade-level standard. Throughout this module, we will review the rubric and some examples and non-examples for this dimension. Remember that in order to get to this point in scoring, the entry has to have already met all of the requirements of the Universal Scoring Rules. If the entry does not meet the requirements for the Universal Scoring Rules, the entry will score a zero for all dimensions. 601234Evidence provided is unscorable; all dimensions will receive a score of zero

CPI Link was assessed, but there are major flaws in the evidence

CPI Link is a Far Link to the grade-level indicator

CPI Link is a Near Link to the grade-level indicator

CPI Link is a Matched Link to the grade-level indicator

Complexity Dimension2014-2015 NJ APA Teacher Training - Module VIII

The Complexity dimension corresponds to some degree to the CPI Link chosen for assessment in that particular entry. If the Link assessed is from the Far Link column, the entry will score a 2 in Complexity. If the Link is from the Near Link column, then the entry will score a 3 in Complexity. If the Link assessed is chosen from the Matched Link column, the entry will score a 4 in Complexity. However, if there is a major flaw in the evidence, regardless of the Link level, the entry will score a 1 in Complexity. Complexity Score of 2, 3, or 4A score of 2, 3, or 4 is obtained if the CPI Link is assessed correctly and a different activity is used for the initial and final piece of evidence.A score of 2 is obtained when a Far Link is assessed correctly.A score of 3 is obtained when a Near Link is assessed correctly.A score of 4 is obtained when a Matched Link is assessed correctly.72014-2015 NJ APA Teacher Training - Module VIII

Before an entry can score a 2, 3, or 4 for Complexity, the CPI Link chosen must be assessed correctly and a different activity must be used for the final piece of evidence than was used for the initial piece of evidence. Two activities are required when submitting evidence for an entry.Both of these activities must assess the same CPI Link.The two activities must be different activities in order to avoid a major flaw.An activity is considered to be different than another when the context of the instruction of the CPI Link is different.An activity is also considered to be different from another when the application of the skill in the CPI Link is different.8What Makes an Activity Different?2014-2015 NJ APA Teacher Training - Module VIII

Remember, in the first portion of the training we discussed the required components of an entry. Each entry must contain 2 pieces of evidence: an initial piece collected between September 2 and November 14, 2014 and a final piece that must be collected between December 8, 2014 and February 13, 2015. Each of these pieces of evidence must be collected within the collection periods, and since the use of different instructional activities can affect the score of an entry, we will look at what constitutes 2 different activities. While the skills assessed and the supports needed to perform the skills remain the same, an activity is considered different:With a different context in which the framework or events within which the student is demonstrating the skill changes and/orWith a different application in which the types of questions used, or the way in which the student demonstrates the skill changes

9What Makes an Activity Different?2014-2015 NJ APA Teacher Training - Module VIII

An activity will be considered different when there is a change in the context and/or the application from one activity to the next. A change does not have to occur in both context and application, but it can. What must remain consistent throughout the activities are the skills and concepts being assessed as well as the support needs of the student.Examples of two activities with different contexts:The student is working on using internal and external characteristics to compare and contrast two animals from different classes of the animal kingdomby comparing and contrasting a bear and an alligatorby comparing and contrasting a salmon and an eagle. The student is working on comparing the mass of substances before and after chemical reactionsby completing an experiment involving baking soda and vinegarby completing an experiment involving heavy whipping cream and vinegar The student is working on completing a Punnett square and identifying the resultsby comparing eye color in parentsby comparing the height of pea plants10What Makes an Activity Different?2014-2015 NJ APA Teacher Training - Module VIII

Here are a few examples of how activities are viewed as different through a different context. While the skills assessed and the supports needed to perform the skills remain the same, the framework or events within which the student is demonstrating the skill changes. That is the context of the activity. This slide presents some examples of how context can change, making the activities different. Examples of two activities with different applications:The student is working on classifying types of planetsby using a graphic organizerby matchingThe student is working on determining the mass of various substancesby using a balance to determine the mass of five substances then logging the mass of each item in a computer spreadsheetby using a balance to determine the mass of five substances then writing the mass of each item in his/her science journalThe student is working on labeling the steps of the water cyclethe student is given worksheet with a diagram of the water cycle and pastes the name of each step onto the diagram using word stripsthe student is given a diagram of the water cycle on the SMARTboard and drags each steps name onto the diagram11What Makes an Activity Different?2014-2015 NJ APA Teacher Training - Module VIII

Here are a few examples of how activities are viewed as different through a different application. While the skills assessed and the supports needed to perform the skills remain the same, the types of questions used, or the way in which the student demonstrates the skill changes. That is a different application of a skill within an activity. This slide presents some examples of how applications can change, making the activities different. Example of 2 Different Activities (Different Context)122014-2015 NJ APA Teacher Training - Module VIIIINITIAL ACTIVITY5.10.12B2 Near Link: Classify positive and negative ways humans affect the environment*Activity Description: After discussing positive and negative ways humans affect the environment, the student reviewed the pictures and then classified each one as negative or positive by labeling.

Here is an example of an initial piece of evidence for classifying the positive and negative ways humans affect the environment. Notice that the activity description provides information about the context of the activity, and that after discussion, the student classified 5 examples of human impact as positive or negative by labeling the picture.13Example of 2 Different Activities (Different Context)2014-2015 NJ APA Teacher Training - Module VIIIFinal Activity5.10.12B2 Near Link: Classify positive and negative ways humans affect the environment*Activity Description: After researching ways humans affect the environment, the student classified the pictures as Positive or Negative by labeling.

Here is an example of a final piece of evidence for this entry. The final activity may appear similar to the initial activity because the student is classifying human impact as positive and negative by labeling. However, in this activity, the student labeled different human impacts than were used in the initial piece of evidence. Therefore, this final piece of evidence is considered a different activity/different context from the initial activity based upon the different pictures representing different ways humans impact the environment.14Example of 2 Different Activities (Different Application)5.9.8B1 Near Link: Classify planets and other objects in our solar system as terrestrial (rocky) or gaseous*

INITIAL ACTIVITY2014-2015 NJ APA Teacher Training - Module VIII

In this initial piece of evidence, the student is classifying planets and other objects as terrestrial or gaseous using a T-chart.Example of 2 Different Activities (Different Application)155.9.8B1 Near Link: Classify planets and other objects in our solar system as terrestrial (rocky) or gaseous*Final Activity2014-2015 NJ APA Teacher Training - Module VIII

In this final piece of evidence, the student is classifying planets and other objects in our solar system as either terrestrial or gaseous by answering multiple choice questions. Multiple choice questions are considered a different application of the skill than using a graphic organizer or T-chart.You Must Think about the Two Activities in ConjunctionAs you plan for instruction, you should not think about the initial and final activity in isolation. Things to think about:What type of activities might be easier and harder for the student? Do not use the easier activity as the final activity.What type of prompting, if any, might the student need on the final activity? Do not use a more intrusive prompt level on the final activity than was used on the initial activity.How can I keep the student engaged in the various activities? How will the initial activity be different than the final activity and still engage the student?

162014-2015 NJ APA Teacher Training - Module VIII

As you plan for the instruction of the CPIs and assessment of the CPI Links, you need to think about all the different aspects of the lesson: the supports in place for the student, the different ways a student may be able to demonstrate the skill, the different contexts within which the student can use the skill, as well as the specific activities and supports used when first introducing a skill (initial activity) versus those used when demonstrating best performance of the skill (final activity). 17Complexity: Major FlawsAn entry that demonstrates work in a CPI Link but has a major flaw will score a 1 in Complexity. A major flaw includes 1. Assessing the same activity for both pieces of evidence2. Assessing only part of the CPI LinkThis may be found in the work sample, and both pieces of evidence must be assessing the same part of the link.

2014-2015 NJ APA Teacher Training - Module VIII

If an entry receives a score point of 1 for Complexity, it is because the entry contains a major flaw. There are two types of major flaws:In one flaw, the student is assessed using the same activity for both pieces of evidence. The other flaw occurs when the student is assessed on only part of a CPI Link. Suppose a student was assessed using the CPI Link Identify the structure and function of plant cells. Assessment of the entire CPI Link must include both structure and function of plant cells in each piece of evidence. So, if both pieces of evidence consisted of just identifying structure of plant cells or both pieces of evidence consisted of just identifying the function of plant cells, the evidence would be assessing only part of the link for that particular CPI. The rest of this module will provide examples of each of the major flaws that will result in a score of 1 for Complexity.18

Complexity Major Flaw Example(Same Activity)

2014-2015 NJ APA Teacher Training - Module VIII

5.5.8B1 Matched Link: Using internal and external characteristics compare and contrast two animals from different classes of the animal kingdom*

Just changing the graphic organizer does not change the activity. Both activities are comparing and contrasting two animals from different classes by cut and pasting characteristics. It isnt different because the initial activity uses a three column T-chart and the final activity uses a Venn diagram. These two activities represent the same application of the skill gluing characteristics into a graphic organizer and the same context using the same organisms and characteristics. So again, an entry containing this major flaw would score a 1 for Complexity. The scores for Performance and Independence would be determined based on their respective criteria using the APA Scoring Rubric.

Providing a different context such as a different purpose for performing the skill or different organisms when performing the skill could be a way to plan a different activity, or providing a different application for how the student demonstrates the skill could be another way to plan a different activity.

Please review the current Procedures Manual for more information on different activities. 192014-2015 NJ APA Teacher Training - Module VIII5.8.4B3 Far Link: Identify tools and their purposes for measuring weather*

Complexity Major Flaw Example(Activity assesses only part of a link)

The evidence in this entry does not demonstrate the student working on the full CPI Link, only part of the link. The evidence only assesses the student identifying the weather instrument; it does not assess identifying the purpose of that instrument. The student is only completing part of the CPI Link. 202014-2015 NJ APA Teacher Training - Module VIII5.8.4B3 Far Link: Identify tools and their purposes for measuring weather*

Complexity Major Flaw Example(Activity assesses only part of a link)

Notice that the same part of the Link (identifying the instrument only) is assessed in both pieces of evidence; therefore, this entry will score a 1 in Complexity. The scores for Performance and Independence would be determined based on their respective criteria using the APA Scoring Rubric. Score of Zero for ComplexityWhen the Universal Scoring Rules are not adhered to, the entry will receive a score of zero for Complexity, Performance, and Independence.Please review Module VII: Universal Scoring Rules for more information.212014-2015 NJ APA Teacher Training - Module VIII

When the Universal Scoring Rules are not adhered to, the entry will receive a score of zero for Complexity, Performance, and Independence. Information to answer these questions is found throughout Module VIII of the training. You may go back to any part of the training or to your 20142015 Procedures Manual to find the answers. You should not continue with the training until you can answer all of these questions correctly. Review222014-2015 NJ APA Teacher Training - Module VIII

Please answer the questions presented on the next slide as a review of the material introduced in Module VIII.

What are two ways activities can be different while assessing the same CPI Link skill(s)?What are the two Major Flaws that would cause an entry to receive a score of 1 for Complexity?For a link that contains two skills, is it acceptable to assess one skill in the first piece of evidence and the other skill in the final piece of evidence?

Review232014-2015 NJ APA Teacher Training - Module VIII

Stop and answer the questions. Once you have answered all questions, move to the next slide.

Activities are different when they have different contexts and/or different applications.The two Major Flaws are: 1) assessing the same activity for both pieces of evidence and 2) assessing only part of the link.No. The full link must be assessed in both pieces of evidence. If only part of the link is assessed, and that SAME part is assessed in both pieces of evidence, the entry will score a 1 in Complexity as a Major Flaw. If one part of the link is assessed in the first piece of evidence and another part of the link is assessed in the final piece of evidence, the entry will score a 0 since different skills were assessed from one piece of evidence to the other.

Review Module VIII Answers242014-2015 NJ APA Teacher Training - Module VIII

Check your answers. Review Module VIII and the current Procedures Manual for any topics in which your answers were incorrect or you are unclear.Scoring Dimensions25Dimension01234ComplexityEvidence provided is unscorable; all dimensions will receive a score of zeroCPI Link was assessed, but there are major flaws in the evidenceCPI Link is a Far Link to the grade-level indicatorCPI Link is a Near Link to the grade-level indicatorCPI Link is a Matched Link to the grade-level indicator Performance Evidence is not clear or all items are not marked as correct/incorrectAccuracy of work is 0-39% based on the final activity ORSecond activity includes more intrusive prompt Accuracy of work is 40-59% based on the final activity Accuracy of work is 60-80% based on the final activity Accuracy of work is 81-100% based on the final activity Independence Evidenceis not clear or all items are not marked for Independence/ prompt levelStudent completed items/tasks independently0-39% of the time based on the final activity Student completed items/tasks independently 40-59% of the time based on the final activityStudent completed items/tasks independently 60-80% of the time based on the final activityStudent completed items/tasks independently 81-100% of the time based on the final activity2014-2015 NJ APA Teacher Training - Module VIII

Please proceed to Module IX: Performance for information on the next dimension of the APA Scoring Rubric and how an entry is scored for Performance.