Montana’s Formative Science Repository

Montana Data Use Conference (MDUC)

Tuesday, September 26, 2017

Radisson Hotel in Helena, MT 59601

Ashley McGrath & Michelle McCarthy

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

Session Objectives

2

Standards Context• History of Science Standards in MT

Assessment Context• Summative Plans• SCILLSS Partnership• ACESSE Partnership

Methods• PAO Science Mission• PAO Timeline• Example of PAO Forms & Process

Results• PAO Phase I Findings

Future Work• Example of Repository Materials

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

Standards Context

3

Foundation Boxes

What students should KNOW and be able to DO

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

2012:

Publication of A Framework for

K – 12 Science Education

2013: Publication of Next Generation Science Standards

2013:

A critical look at NGSS for Montana and verification of gap analysis.

2014:

NAEP Item Classification Study

2014:

NRC Report Published on Assessments for NGSS identifies NAEP as a potential exemplar

History of Science Standards in MT

• Last Science Standards adoption was in 2006. • In 2009, the Essential Learning Expectations (ELE’s) were

written in an effort to provide guidance and support for teachers.

• Note: Standards review period is approximately every five years.

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

NGSS Lead State

5

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

2015: OPI begins standard revision process

2016: Adopted NGSS-Aligned Standards

2017: Implementation of Standards July 1, 2017.

2017: PAO Science Workshop

2020: New Assessment

History of Science Standards in MT Cont…

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

NGSS and Framework-Based State Adoptions

7

Updated last as of April 2017

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

Assessment Context

8

Teachers and schools have

information and tools they need to improve teaching

and learning

Summative

Assessments

Benchmarked to college

And career readiness

Teacher resources for

Formative

Assessment Practices

To improve instruction

and student learning

Interim

Assessments

Flexible, open, used for

actionable feedback

Montana Science

Performance Standards

(NGSS-aligned)

for college and career readiness

All students leave high

school college and

career ready

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

Design Requirements

• Administered not less than one time during—• grades 3 through 5;• grades 6 through 9; and• grades 10 through 12. (ESSA Sec.1111 (b)(2)(B)(v)).

• Assessment and instruction are inseparable• Worth Taking• High-Quality• Time-limited• Fair and Supportive • One of multiple measures• Ties to improved learning

9

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

Summative Science Plans

• Since FY 2008 Montana has used the MontCAS CRT for Science.

• Two year contract extension for the MontCAS CRT-Science.

• Plan to measure the three-dimensional standards beginning in the spring of 2020.

10

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

Partnerships

• Montana is a partner state in the “Strengthening Claims-Based Interpretations and Uses of Local and Large-Scale Science Assessments” (SCILLSS) Educational Assistance Grant (EAG) funded by the US Department of Education.

• The Advancing Coherent and Equitable Systems of Science Education (ACESSE, or “access”) project brings together partners from educational research and practice to improve equity by building coherence in science education. - Council of State Science Supervisors

11

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

What to Measure?

Standards Assessment

Standards

Assessment

Assessment

Standards

StandardsAssessment

Items

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

Montana’s Theory of Action

13

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

What is Formative Assessment?

14Source: https://www.smarterbalanced.org/wp-content/uploads/2015/09/Digital-Library-Factsheet.pdf

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

Balanced Assessment for Science

15

Website:https://www.smarterbalancedlibrary.org/

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

Benefits of NAEP Items

Free and Accessible

Rigorous

Inclusion of science

practices

Available performance

data

Available scoring Guides

Variety of Formats

Why NAEP?

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

What is PAO Science?

17

Last Row: Marshall Lagge, Bruce DudekSixth Row: Amanda Obery, Emily Currier, Karla Miller, Jessica Eilertson, John Deming, Jared BetzFifth Row: Chris DeWald, Jennifer Stadum, Sue Mohr, Lily Haines, Nicole KirschtenFourth Row: Ashley McGrath, Katie Burke, Melissa Johnson, Lindsay Manzo, Debbie HansonThird Row: Maureen Karlin, Jacqueline Marshall, Michelle McCarthy, Brian WilliamsSecond Row: Jodi Hall, Audrey Howard, Roni Sells, Monica Tomayer, Katherine AuneFront: Karen Pollari, Marcy Fortner, Yvonne Field, Molly Ward, Mary Williams, Summer Graber

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

PAO Formative Assessment

18

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

Methods

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

PAO Science MissionThe following statements define the significance of this work:1. PAO provides a model for taking existing Framework-aligned

assessment materials and methods to examine them for formative purposes.

2. PAO provides a step-wise Understanding by Design (UbD) process using carefully crafted forms to consider resources and instructional strategies for instructional intention, clarity, evidence production, feedback, and closing the learning gap.

3. PAO offers guidance on accessibility through identifying areas of existing accessibility and areas where additional support or modification may be needed to meet the needs of all students.

4. PAO provides a state-specific form to increase the student engagement, authenticity, and connections to the community through the cultural relevance and significance section of this project.

5. PAO provides rubrics and processes for reviewing and considering materials for inclusion or exclusion in the online repository.

6. PAO Groups identify formative materials for inclusion on the OPI’s online repository which will be freely shared with Montana educators across the state.

20

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

PAO Phase I - Pilot

21

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

PAO Data Collection Forms

22

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

Results

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

PAO Phase I Findings

Facts:

• 25 teachers participated in this science formative development project.

• There were several teachers who participated on the team that wrote the Montana Science Standards (2016)

• Several informal science educators (e.g., educators form museums, science centers, etc. who work with teachers throughout the school year) were also participants.

24

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

PAO Phase I Findings

Benefits to State:

• This workshop increased teacher’s knowledge of the new science standards and around developing formative assessments that measure these new standards.

25

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

PAO Phase I Findings

Benefits to Teachers:

• “The course overall helped me grow. I appreciated the face-to-face experience the most, truly my best learning style”. – PAO Teacher Participant

26

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

PAO Phase I Findings

Benefits to Students:

• Students of teachers who were a part of this workshop will have teachers who have more confidence teaching the new science standards.

27

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

Future Work

28

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

Goals for PAO Phase II

Use the Hub evaluations and PAO in-person evaluation to revise the Hub Course materials for future re-launch.

Use the evaluation feedback to revise “PAO Forms”, “Review Rubric” and “Cover Profile” for clarity and usability.

Continue to expand our formative resource repository using the established training, development process, and review process.

Continue to build statewide capacity through opportunities listed in “Future Interest Survey”.

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

PAO Repository Material

30

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

Repository Timeline

Aug – Nov

Assemble PAO Recommended Resources for

OPI Review

Aug – TBD

Interim

“Holding Tank”

Aug – Dec

OPI Resource Review

Oct – Dec

Publishing Review

Dec – Feb Publishing

Materials for Teacher Access

31

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

Build Statewide Capacity

What Next?

• For example:• Resource in Practice - teachers use this work during

the year and submit evidence of student work, personal reflections, and/or activity in action (video).

• Feedback on Resources – pilot them and provide personal experience or modifications of resources.

• Contribute to the Pool – submit more formative resources to the repository.

See the Hub Catalog for ideas and additional courses!

• Formative Assessment - is a deliberate process used by teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.

Questions?

33

Ashley McGrath

NAEP State Coordinator

Montana Office of Public Instruction

Phone: 406.444.3450

E-mail: amcgrath@mt.gov

Website: http://opi.mt.gov/Leadership/Assessment-

Accountability/NAEP

Michelle McCarthy

Science Instructional Coordinator

Montana Office of Public Instruction

Phone: 406.444.3537

E-mail: MMcCarthy5@mt.gov

Website: http://opi.mt.gov/Educators/Teaching-Learning/K-

12-Content-Standards-Revision