View
213
Download
0
Category
Tags:
Preview:
Citation preview
1
Oregon Content Standards and Assessment System Evaluation
Prepared for the Oregon Department of Education
by WestEd
Dr. Stanley Rabinowitz
Dr. Edynn Sato
June 22, 2007
2
Background: Overview
One of several key projects funded by the Gates Foundation
Independent evaluation by WestEd, a non-profit educational research, development, and service agency
3
Content Standards and Assessment System Evaluation: What?
WestEd will: Review the content standards
(all grades, academic content areas) Evaluate the structure of the content
standards Evaluate alignment between the state
assessments and the content standards Make recommendations for improvement of
structures and systems
4
Key Deliverables
Preliminary report of an initial review of selected content standards and grades using initial protocol/criteria
Final report for the comprehensive review of the content standards
Final report for the alignment of assessments to content standards
Final report on the structure and quality of Oregon’s content standards and assessments
5
Comprehensive Evaluation of Content Standards
• English language arts• Mathematics• Science• Social sciences• Arts• Second languages• Physical education• Health education• Technology• English language proficiency
6
Standards Evaluation Studies
The analyses aimed to address the following key questions:
1. Do Oregon’s content standards adequately represent the knowledge and skills that all students should know and be able to do?
2. Do Oregon’s content standards reflect the appropriate breadth and depth of the content area?
3. Do Oregon’s content standards have the clarity and consistency needed to adequately guide instruction and assessment?
7
External Referents
CONTENT AREA LEVELS STATE REFERENT NATIONAL REFERENT
English Language Arts
10 GradesK-8 and CIM
Indiana Draft 2009 NAEP Reading Framework and 2011 NAEP Writing Framework (grades 4, 8, and 12), McREL Speaking and Listening (grades K-3, 5-7, CIM)
Mathematics 10 GradesK-8 and CIM
Indiana NCTM Principals and Standards for School Mathematics (2000) and Curriculum Focal Points for K-8 Mathematics (2006)
Science 10 GradesK-8 and CIM
Indiana AAAS benchmarks (2001)
8
External Referents (continued)
CONTENT AREA LEVELS STATE REFERENT NATIONAL REFERENT
Social Sciences: Geography
Benchmarks 1 (K-3), 2 (4-5), 3 (6-8), CIM
Washington NAEP Geography Framework (2001)
Social Sciences: History
Benchmarks 1 (K-3), 2 (4-5), 3 (6-8), CIM
Washington NAEP History Framework (2006)
Social Sciences: Civics
Benchmarks 1 (K-3), 2 (4-5), 3 (6-8), CIM
Washington NAEP Civics Framework (2006)
Social Sciences: Geography
Benchmarks 1 (K-3), 2 (4-5), 3 (6-8), CIM
Washington NAEP Economics Framework (2006)
9
External Referents (continued)
CONTENT AREA LEVELS STATE REFERENT NATIONAL REFERENT
Arts Benchmarks 1 (K-3), 2 (4-5), 3 (6-8), CIM
Washington Compendium Arts Standards developed by McREL (2006)
Physical Education Benchmarks 1 (K-3), 2 (4-5), 3 (6-8), CIM
Indiana Moving into the Future: National Standards for Physical Education (2006)
Health Benchmarks 1 (K-3), 2 (4-5), 3 (6-8), CIM
Indiana National Health Education Standards: Achieving Health Literacy
Educational Technology
1 Level (CCGs) Washington ISTE Standards for Students (2000)
10
External Referents (continued)
CONTENT AREA LEVELS STATE REFERENT NATIONAL REFERENT
Second Languages Benchmarks 1 (K-3), 2 (4-5), 3 (6-8), CIM
Indiana American Council on the Teaching of
Foreign Languages (1999)
English Language Proficiency: Forms and Functions
5 Levels of Language Forms and Functions
NA Current National Research on Forms
and Functions
11
Standards Evaluation Studies (continued)
Overall Quality Oregon’s English Language Arts (ELA) standards overall are of good
quality and provide breadth and depth of coverage of Reading, Literature, Writing, Speaking, and Listening. The standards generally are clearly written, focus on important skills/concepts for instruction, and are assessable.
Oregon’s mathematics standards overall are of good quality and reflect a breadth and depth of content coverage. The standards generally are clearly written, focus on important skills/concepts for instruction, and are assessable.
Oregon’s science standards overall are of good quality in terms of clarity and importance for instruction. However, in all grades except for grade 8, analysts determined that the standards did not reflect a range of depth of knowledge. For all grades except for grade 1, analysts determined that the standards did not reflect a breadth of knowledge. With regard to consistency, analysts determined that standards were inconsistent (language, skills, knowledge) in grades 4, 8, and CIM. And, in all grades except for grades 3, 5, 8, and CIM, there were issues of assessability.
12
Standards Evaluation Studies (continued)
Overall Quality (continued) Oregon’s social sciences standards overall are of good quality
in terms of depth, breadth, consistency, importance for instruction, and assessability. However, overall these standards lack clarity.
Oregon’s arts standards overall are of good quality and cover a breadth and depth of content. The standards generally are clearly written, focus on important skills/concepts for instruction and are assessable.
Oregon's physical education standards overall are of good quality in terms of breadth and depth of content coverage, consistency across benchmark grades, importance for instruction, and degree of assessability. However, analysts determined that these standards are not easily understood due to lack of clarity and precision of language.
13
Standards Evaluation Studies (continued)
Overall Quality (continued) Oregon’s health standards overall are of good quality in terms
of depth, consistency, importance for instruction, and assessability. For all benchmark levels except for Benchmark 2, the standards overall are of good quality in terms of breadth of content. And, overall these standards lack clarity.
Oregon’s educational technology standards overall are of good quality in terms of depth, breadth, consistency, importance for instruction, and assessability. However, overall these standards lack clarity.
Oregon’s second languages standards overall are of good quality in terms of depth, breadth, clarity, consistency, importance for instruction, and assessability.
14
Standards Evaluation Studies (continued)
Overall Quality (continued) Generally, Oregon’s English language proficiency (ELP)
forms and functions are presented in a clear and useful format. The proficiency level descriptors clearly describe and differentiate the language skills of students at each level. However, the definitions of language function and forms of language appear incomplete. All the language functions except one are appropriate—literary analysis is typically considered English-Language Arts content rather than a language function. All the language forms except one are appropriate—the language of propaganda is not a form consistent with other elements of form presented by the state; rather it appears to be a genre.
15
Standards Evaluation Studies (continued)
Alignment to External Referents Oregon’s ELA standards generally aligned to the Indiana, NAEP
and McREL standards in terms of overall content order/sequence and depth of content. However, while the breadth of the Oregon standards appeared comparable to the NAEP and McREL standards, they were not as comparable to the Indiana standards.
Oregon’s mathematics standards were compared to Indiana’s mathematics standards as well as to the NCTM Principles and Standards for School Mathematics (2000) and Curriculum Focal Points for K–8 Mathematics (2006). Oregon’s mathematics standards generally aligned to the NCTM standards and to the Indiana standards with regard to overall depth and breadth of content coverage. However, there was not overall congruence between Oregon and Indiana’s content ordering and sequencing.
16
Standards Evaluation Studies (continued)
Alignment to External Referents (continued) Oregon’s science standards were compared to Indiana’s science
standards and the AAAS benchmarks (2001). – Generally, the Earth and Space Science standards aligned in
terms of overall depth of content; however, they did not align in terms of content order/sequence or overall breadth of content.
– Generally, the Physical Science standards aligned to the Indiana standards in terms of content order/sequence, overall depth, and overall breadth of content. The Oregon standards also aligned to the AAAS standards in terms of overall depth and breadth of content; however, they did not align in terms of content order/sequence.
– Generally, the Life Science standards aligned to the Indiana standards in terms of content order/sequence, overall depth, and overall breadth of content. However, they did not align to the AAAS standards in terms of content order/sequence, overall breadth, or overall depth of content.
17
Standards Evaluation Studies (continued)
Alignment to External Referents (continued) Oregon’s social sciences standards were compared to Washington’s
social science standards and the NAEP frameworks.
– Generally, the Geography standards aligned in terms of overall content order/sequence with Washington’s standards. However, they did not align in terms of content order/sequence with the NAEP framework or in terms of the overall depth or breadth of content with the Washington standards or the NAEP framework.
– Generally, the History standards did not align to the NAEP framework in terms of content order/sequence, depth or breadth of content. Nor did these standards align in terms of overall breadth with the Washington U.S. History and World History standards. However, the Oregon standards aligned to the Washington standards in terms of content order/sequence and overall depth.
18
Standards Evaluation Studies (continued)
Alignment to External Referents (continued) Oregon’s social sciences standards (continued).
– Generally, the Civics standards aligned in terms of overall content order/sequence with Washington’s standards and the NAEP framework. However, they did not align in terms of overall depth or breadth of content with either the Washington standards or the NAEP framework.
– Generally, the Economics standards aligned to the Washington standards and the NAEP framework in terms of content order/sequence, depth and breadth of content.
At the broad level of the content standard, there was a high degree of alignment between Oregon’s arts standards and those of Washington and McREL. In terms of content order / sequence, Washington’s standards spiral content across different benchmarks in a manner similar to Oregon; many of McREL’s benchmark level standards are not spiraled across different benchmarks. Both external referents show similar depth of content to Oregon’s standards.
19
Standards Evaluation Studies (continued)
Alignment to External Referents (continued) Generally, Oregon's physical education standards, the National
Association of Sports and Physical Education Standards (NASPE), and the Indiana standards share a common purpose. In broad terms, the nature of the content knowledge/cognitive information and the types of skills students are expected to learn and be able to demonstrate are comparable across the three sets of standards. The Oregon differs in depth of content coverage from the NASPE and Indiana State standards.
Oregon’s health standards were compared to Indiana’s health standards as well as to the National Health Education Standards (NHES). Oregon’s standards generally aligned to the Indiana and NHES standards with regard to overall content order/sequence and breadth of content coverage. However, there was not overall congruence in terms of depth of content.
20
Standards Evaluation Studies (continued)
Alignment to External Referents (continued) Oregon’s educational technology standards generally aligned to the
Washington and NETS standards in terms of overall content order/sequence, depth of content and breadth of content.
While Oregon and Indiana both have the same number of proficiency levels, a comparison of the depth and breadth of the second languages standards across levels in both states shows that the Indiana standards start at a slightly higher level than those in Oregon, and reach a higher level. Oregon’s proficiency levels appear to be based on ACTFL levels Novice-Low through Intermediate-Mid. The standards statements differ in level of detail. Generally, Oregon standards are written more broadly, and Indiana’s are more detailed and specific. Oregon’s set of Functions Supporting Standards describe a similar level of detail as Indiana’s standards.
21
Standards Evaluation Studies (continued)
Alignment to External Referents (continued) Oregon’s English language proficiency (ELP) forms and
functions were compared to current national research and literature on forms and functions (e.g., TESOL, CALLA, Butler, et al.). All but one of Oregon’s language functions (literary analysis) are appropriate and relevant for ELD instruction. The external referents include functions that are currently not present among those in the Oregon ELP standards.
22
Standards Evaluation StudiesGeneral Recommendations
Evaluate standards for redundancy within a grade. Evaluate standards for level of detail—some standards
have elaborate detail and others do not. Differentiate between detail that describes the curriculum
to be taught and the intended learning outcomes (content skills, knowledge).
Evaluate coherence across levels of specificity for a given standard—e.g., strand, common curriculum goals (CCGs), grade-level standards—in terms of appropriateness of progression of specificity (general/broad to specific) and clarity of relationships.
23
General Recommendations(continued)
Verify that expansion/extension of content reflected in standards (e.g., references that include student’s knowledge of the community, knowledge of historical periods) is reasonable and intentional.
Evaluate the clarity and effectiveness of small variations in the wording/language of specific standards across the grade levels—verify that the differences are intentional and reflect clear, real differences in expectations of students vis-à-vis the content/skill.
24
General Recommendations(continued)
Embed definitions of key content terminology, if interpretations of terminology can be varied AND the intent of the state is to promote consistency
Make sure the language clearly conveys expected skills and complexity
Evaluate coherence and consistency of content emphases across grades
25
Core Standards “Content Standards” define the knowledge and skills that a
state’s students are supposed to learn The definition of “Core Standards” includes/reflects:
– Knowledge and skills central to a content area– Significant/essential content/skill “targets” in a content
area– State-specific priorities in a content area– Learning expectations and performance goals for all
students Core standards subsume other standards and/or provide a
super-structure for a coherent system of content standards Some states identify core standards in order to ensure that
these standards are addressed in each grade and developed across grades—local curricula are expected to align to these standards
26
Core Standards (continued)Considerations: Will the standard provide students with
knowledge and skills that will be of value beyond a single test date? (What endures?)
Will the standard provide knowledge and skills that will be of value in multiple disciplines? (What has leverage?)
Will the standard provide students with essential knowledge and skills that are necessary for success in the next level of instruction? Beyond school? (What is really necessary for “success”?)
[Based on Reeves, 2004]
27
Essential Skills “Essential Skills” refer to the core abilities/process
skills that all students are expected to develop and apply across disciplines
Essential skills typically span content areas and facilitate the effective delivery of content knowledge across disciplines
Essential skills typically are expected to be demonstrated in a variety of ways and in different contexts
Essential skills reflect state’s priorities Local curricula are expected to incorporate these
skills—the skills are frequently embedded within existing learning standards, curricula, and assessments.
28
Essential Skills (continued)Considerations: Will the skills be of value beyond a single
test date? (What endures?) Will the skills be of value in multiple
disciplines? (What has leverage?) Will the skills be necessary for success in
the next level of instruction? Beyond school? (What is really necessary for “success”?)
[Adapted from Reeves, 2004]
29
Next Steps Additional alignment analyses and
information Additional information regarding
assessments and standards (ODE questions)
Analyses of “core standards” Analyses of “essential skills” Continued consideration of
feedback/information provided by key stakeholders through surveys, meetings, etc.
Recommended