Smarter Balanced Assessment SystemDr. Cristi Alberino, ELA Assessment Consultant,

CSDE

Abe Krisst, CMT Education Consultant, CSDE

Education Institutes of Higher Education Symposium

April 27, 2012

The Assessment Challenge

How do we get from here... ...to here?

All studentsleave high school

college and career ready

All studentsleave high school

college and career ready

Common Core State Standards

specify K-12 expectations for

college and career readiness

Common Core State Standards

specify K-12 expectations for

college and career readiness

...and what can an assessment system do

to help?

Summative Assessments Today

Each state procures its own assessment

systemMeasure proficiency against state

standards, not agreed-upon standardsUsually heavy reliance

on multiple choice questions

Results often delivered months after tests are

givenAccommodations for

special education and ELL students vary

Most administered on paper

Next Generation Assessments

• More rigorous tests measuring student progress toward “college and career readiness”

• Have common, comparable scores across member states

• Provide achievement and growth information to help make better educational decisions and professional development opportunities

• Assess all students, except those with “significant cognitive disabilities”

• Administer online, with timely results• Use multiple measures

Source: Federal Register / Vol. 75, No. 68 / Friday, April 9, 2010 pp. 18171-85

A National Consortium of States

• 27 states representing 43% of K-12 students

• 21 governing, 6 advisory states

• Washington state is fiscal agent

The Purpose of the Consortium

• To develop a comprehensive and innovative assessment system for grades 3-8 and high school in English language arts and mathematics aligned to the Common Core State Standards, so that...

• ...students leave high school prepared for postsecondary success in college or a career through increased student learning and improved teaching

[The assessments shall be operational across Consortium states in the 2014-15 school year]

Remediation in Higher Education

•Nationally, 42% of students at 2 year institutions took a remedial course

•70% of students who took 1 or more remedial reading classes did not get a college degree within 8 years

•51% of students who took the ACT were not ready for college level reading

Addressing Higher Education Concerns

Comparability

Relevance

Stability

Quality

Utility

Higher education faculty involved in assessment design to ensure that the assessments are true to Common Core standards and higher education expectations.

Reaching the Goal: Expectations of Higher Education

Smarter BalancedDesign

A Balanced Assessment System

Common Core State Standards specify

K-12 expectatio

ns for college and

career readiness

Common Core State Standards specify

K-12 expectatio

ns for college and

career readiness

All students

leave high

school college

and career ready

All students

leave high

school college

and career ready

Teachers and schools have information and tools

they need to improve

teaching and learning

Interim assessments

Flexible, open, used for actionable

feedback

Summative assessments

Benchmarked to college and career readiness

Teacher resources for

formative assessment

practicesto improve instruction

Test Question Types and Design

• Items and the collection of items MUST measure deeper understanding and application of conceptions.

• Therefore a variety of item types are necessary:– selected response, short constructed response,

extended constructed response, technology enhanced, and performance tasks

Changes to English Language Arts Assessment

ELA Reading Deficiencies

• Inadequate understanding of words

• Inadequate background knowledge

• Lack of knowledge about writing

conventions

• Inability to recall information given

verbally

• Inability to understand implicit information

Education Policy Improvement Center (EPIC)

• Survey of 1897 postsecondary

instructors

– both 2 and 4 year schools– variety of subject areas: ELA, Math,

science, social science, business, computer technology, healthcare

• CCSS English Language Arts and Literacy standards are applicable and important for postsecondary readiness

Are the ELA standards as a whole a coherent representation of the fields of knowledge necessary for success in your course?

Reading and writing Grounded in Evidence From Text

●Students should be taking stances and using evidence from sources to support their positions (verbally and in writing)

●Writing prompts should be tied to texts●In other words, the student must have read and analyzed a text to respond to the prompt.

Several Key Shifts

● Building knowledge through content-rich nonfiction and informational texts

● Reading and writing grounded in evidence from text

● Regular practice with complex text and its academic vocabulary

ACT Reading Between the Lines

The complexity of what students can read is the greatest predictor of success in college

Building Knowledge Through Content-Rich Nonfiction and

Informational TextsRange of Text Types

●Make sure that students are exposed to a wide range of text types including informational text (HS – 70%/30%).

Text Complexity

●Make sure students are exposed to grade level text complexity – gradient increase with support.

Preparing Teachers to Use CCSS

Text Complexity is the key!

Regular practice with complex text and academic vocabulary

●Tier 2 Vocabulary: Words that add to students’ language ability (e.g., maintain, fortunate, required, tend, contrast/compare, insisted, summarize)

●Tier 3 Vocabulary: Words that are needed in a content area (e.g., isotope, peninsula, photosynthesis, cubism, isosceles triangle)

Preparing Teachers to Use CCSS

• Do not “fragment standards”• Do not arrange standards into new

categories• Use cluster headings and domain

names– Writing: Production and Distribution of Writing– Research to Build Knowledge

Assessment Claims for English Language Arts/Literacy(a/o 12/27/11)

Total English language arts / literacy (Grades 3-8)“Students can demonstrate progress toward college and career readiness in English language arts and literacy.”

Total English language arts / literacy (High School) “Students can demonstrate college and career readiness in English language arts and literacy.”

Assessment Claims for English Language Arts/Literacy

Reading

Writing

Speaking/Listening

Research/Inquiry

(a/o 1/6/11)

Understanding the ProcessOverarching Claims 3-8 and High

School Overarching Claims 3-8 and High

School

TargetsTargets

StandardsStandards

Assessment Claims

Assessment Claims

Claim 1: Reading Informational Text

Target 12: Analysis Within or Across Texts

Model is based on standards from Listening/Speaking, Writing, Reading Literature and Informational Text

Evidence: • students conduct research• evaluate sources, analyze and

integrate • write a short report with evidence cited• use precise language for purpose and

audience• deliver oral presentation

How Does this Task Contribute to the Sufficient Evidence for the

Claim?In order to complete the PT, students

1. Make inferences and summarize using key details in text.

2. Analyze information presented in multiple texts.3. Analyze information delivered orally and

visually4. Conduct short research on a topic, analyze and

interpret the information, and cite evidence about how it supports a concept

5. Organize, compose, and deliver oral presentations using precise language appropriate to purpose and audience.

6. Use visual or audio information to enhance oral presentations

Mathematics

(not yet released)

Total mathematics (Grades 3-8)“Students can demonstrate progress toward college and career readiness in mathematics.”

Total mathematics (High School) “Students can demonstrate college and career readiness in mathematics.”

Assessment Claims for Mathematics

Concepts and Procedures

Problem Solving

Communicating Reasoning

Data Analysis and Modeling

(a/o Round 1 – released 8/29/11)

Shifts Evident in CCSSM

• FOCUS• FOCUS• FOCUS• Coherence• Fluency• Deep Understanding• Application• Intensity

Key FluenciesGrade Required Fluency

K Add/subtract within 5

1 Add/subtract within 10

2Add/subtract within 20

Add/subtract within 100 (pencil and paper)

3Multiply/divide within 100

Add/subtract within 1000

4 Add/subtract within 1,000,000

5 Multi-digit multiplication

6Multi-digit division

Multi-digit decimal operations

7 Solve px + q = r, p(x + q) = r

8 Solve simple 22 systems by inspection

• Mile-wide/inch-deep traditions

• Attaining CCSSM balance

– conceptual understanding

– procedural skill

– application

• Connecting content and practices

• Grain size in CCSSM

• Psychometric culture

Implementation Challenges

Changes to Mathematics Assessment

Assessment

● What will happen to CMT and CAPT?

● CT has submitted an application for an NCLB waiver.

● School year 2014/2015, SMARTER Balanced Assessment Consortium (SBAC) assessment system operational for students in Grades 3-8 and 11.

Item Exemplars: Performance Task

Performance Task drawn from the Ohio Performance Assessment Project.

Item Exemplars: Performance Task (cont’d)

Performance Task drawn from the Ohio Performance Assessment Project.

Item Exemplars: Performance Task (cont’d)

Performance Task drawn from the Ohio Performance Assessment Project.

Essential Properties of Tasks That Assess Claim 1

Conceptual Understanding and Procedural Fluency

Tasks assessing this category will include short items, multiple-choice, other selected-response, and short constructed-response items, that focus on a particular skill or concept.

They will also include items that require students to translate between representations of concepts (words, diagrams, symbols) and items that require the identification of structure.

Aspects of Claim 1 can also be examined in the context of student work on more complex tasks.

Sample Item

Sample Item

Sample Item

Sample Item

Sarah is 12 years old. • George is g years old. • Sarah is 3 times as old as George.

For numbers 1a – 1c, choose Yes or No to indicate whether each statement is true.

1a. George’s age, in years, can be representedby the expression 12 ÷ 3.

1b. George is 15 years old.

1c. Sarah’s age, in years, can be represented by the equation 12= 3 × g.

OECD, 2009. Sample questions from OECD’s PISA Assessments. ISBN 978-92-64-05080-8.

Essential Properties of Tasks That Assess Claim 2Problem Solving

Evidence for Claim 2 depends on tasks that

• present non-routine problems where a substantial part of the challenge is in deciding what to do, and which mathematical tools to use;

• involve chains of autonomous reasoning, taking a successful student at least 5 to 10 minutes (depending on the age of the student and complexity of the task), including explanation of assumptions and conclusions as well as the use of representational and procedural skills.

Essential Properties of Tasks That Assess Claim 3

Communicating ReasoningEvidence for Claim 3 depends on tasks that

• present a situation in which either propositions are given or students are encouraged to make their own conjectures;

• ask students to test propositions or conjectures with specific examples;

• ask students to construct, autonomously, chains of reasoning that will justify or refute the propositions or conjectures; these chains should typically take a successful student 10 minutes or more. (Times will be somewhat shorter for younger students, but still giving them time to think and explain.)

Gavin, M. K., Casa, T. M., Chapin, S., Copley, J. V., & Sheffield, L. J. (2008). Project M2: Using Everyday Measures: Measuring with the Meerkats from Project M2: Mentoring Young Mathematicians series.

Sample Item

Essential Properties of Tasks That Assess Claim 4

Mathematical ModelingEvidence for Claim 4 depends on tasks that

• present non-routine problems from the real world where the solution involves some or all of the phases of the modeling cycle;

• for some tasks, a substantial part of the challenge is in formulating an approach: deciding what to do, and which mathematical tools to use;

• involve substantial chains of autonomous reasoning, taking a successful student at least 10 minutes (less for younger students), and call for explanation of assumptions, interpretations, evaluations, and conclusions as well as reliable representational and procedural skills.

There is Not Necessarily a Simple Correspondence Between

Standards, Claims, and TasksSome items will assess student understanding of particular content-related standards. For example, the task

“If x and y are positive integers, and 3x + 2y = 13, what could be the value of y? Write all possible answers.”

This item addresses Content Standard EE-8.1 and Claim 1.But, consider the following problem, “Hurdles Race.”

“Hurdles Race”

Think of the Content Involved

• Interpreting distance-time graphs in a real-world context

• Realizing “to the left” is faster

• Understanding points of intersection in that context (they’re tied at the moment)

• Interpreting the horizontal line segment

• Putting all this together in an explanation

Think of the Practices Involved

• Make sense of problems and persevere in solving them.

• Reason abstractly and quantitatively.

• Construct viable arguments.

• Model with mathematics.

• Use appropriate tools strategically.

• Attend to precision.

• Look for and make use of structure.

• Look for regularity in repeated reasoning.

Resources for CT Districts

Useful websites for Text Complexity

• Connecticut State Department of Education: http://www.sde.ct.gov/sde/site/default.asp

• Council of Chief State School Officers: http://www.ccsso.org/Resources/Digital_Resources/Common_Core_Implementation_Video_Series.html

• Kansas State Department of Education: http://www.ksde.org/Default.aspx?tabid=4778#TextRes

• Lexile Analyzer: www.lexile.com/findabook• Maine Department of Education:

http://www.maine.gov/education/lres/commoncore/ • National PTA:

http://www.pta.org/common_core_state_standards.asp

• The Hunt Institute (video series): http://www.youtube.com/user/TheHuntInstitute#g/u

Find Out More

Smarter Balanced can be found online at:

SmarterBalanced.org

Contact Information

Cristi Alberino, Ph.D.English Language Arts ConsultantELA Lead, SBAC Item Writing and

Performance Task Work Groupcristi.alberino@ct.gov

Abe KrisstCMT Assessment Consultant

Reporting Work Group and Technology Readiness State Leadabe.krisst@ct.gov