Forum - 1 Assessments for Learning: A Briefing on Performance-Based Assessments Eva L. Baker Director National Center for Research on Evaluation, Standards,

Forum - 1

Assessments for Learning: A Briefing on Performance-Based Assessments

Eva L. BakerDirectorNational Center for Research on Evaluation, Standards, & Student Testing

Graduate School of Education & Information StudiesUniversity of California, Los Angeles

The Forum for Education and DemocracyOctober 20, 2008

© Regents of the University of California

Forum - 2

Performance Assessment

• Why now?

• What’s different?

• What about quality, fairness and cost?


Forum - 3 © Regents of the University of California

Forum - 4

Why Performance Assessment? 21st Century Skills!

• Adaptive problem-solving

• Situation awareness and risk assessment

• Decision-making

• Self-management

• Teamwork

• Learning to learn

• Communication

• Application of knowledge to new settings and situations


Forum - 5

What’s Changed inPerformance Assessments

Correlations = Exchangeability Learning requires active patterns and performance

Subjectivity Raters can be trained to agree almost perfectly

Arbitrary and mindless, Not serious content

Map to detailed specs of content, criteria

Not fair to minorities Because of sensitivity to instruction, evidence that all children can learn complex outcomes

High cost Technology-based design reduces cost more than order of magnitude

Myth

Truth


Forum - 6

How?

• Collaborate on detailed domains (ontologies) underlying only key standards

• Focus on important outcomes rather than covering a thin layer of standards

• Connect during up-front design to instruction and curriculum as well as measures

• Include specific measures of transfer and adaptation as well as information for improvement


Forum - 7

Algebra Ontology


Forum - 8

Physics Ontology

NEWTON'SLAWS

Third Law

Second Law

First LawA body in motionremains in motion

unless...

Forces betweeninteracting bodies:equal but opposite

is

is

is

areForce equals Masstimes Acceleration

(F=MA)


Forum - 9

Novice Level 1 Flooding Ontology

Situation Awareness and Communications


Forum - 10

Collaborate on Careful Specifications of Domain

• Assure coverage of the key standards

• Provide transparent map of relationships for teaching, learning, and testing

• Guide professional development

• Allow task sampling for test to be systematic and increasing complex up the grade levels

• Allows weighting of tasks by importance, complexity, skill level so the test reflects proficiency


Forum - 11

Technology’s Role

• Design reusable task components, cost sensitive formats, scoring criteria, and reporting

• Many performance assessments use computers for scoring and instant feedback

• Some approaches use well-trained raters to score, and some use mixed models

• Psychometric methods can make predictions about who needs what during instruction to make teaching more individual and adaptive

• Technology is repeatable and easily improved


Forum - 12

Validity of Measures and Accountability Systems: What Evidence is Needed?

• Transparent and fair to all

• Multiple measures of learning

• 21st century focus

• Sensitivity to good teaching

• Measures of classroom practice

• Accurate for all purposes

• Valid classification cuts

• Progress and improvement

• Technology and cost© Regents of the University of California

Forum - 13

Changing Educational Practices

• Diminishing attention to tests, more to problem solving, and evidence of actual performance in universities and businesses

• International competitors, e.g., South Korea, Singapore, are concerned with innovation, adaptation, and creativity, areas where U.S. students have historically been stronger


Forum - 14 © Regents of the University of California

1. We cannot tolerate accountability systems that do not push our students to serious performance needed for satisfaction and for our own society’s competitive position in the world

2. Nor weak and indefensible low achievement in middle and secondary schools

3. Start here by changing NCLB measures to focus on applied as well as academic skills at the secondary level

Where to Start?

Forum - 15

http://www.cse.ucla.edu

Eva L. Baker

[email protected]

voicefax

email


Forum - 16

Back Up


Forum - 17

POWERSOURCE© Pre-Algebra

• Learning-based assessment

• Content and cognitive ontology

• Tasks

• Schema or pattern development

• Explanation

• Transfer and application

• Formative and certification use

• Supportive instruction and PD

• Small investment, big change


Forum - 18

CRESST PA Experiences1. Ages of students using range from 4-year-olds to post graduates

and military

2. Special efforts with English Language Learners and underperforming students to assure fairness

3. Evidence of scale in multiyear performance assessment in Los Angeles (300K+ students annually)

4. Now preparing performance assessments in procedural and complex decision-making in military for critical tasks

5. Presently conducting studies in pre-algebra and algebra with positive early effects

6. Engaged in simulation and gaming forms of assessment

7. Roadmap for the design of performance assessments for accountability and other purposes


Documents

Forum - 1 Assessments for Learning: A Briefing on Performance-Based Assessments Eva L. Baker Director National Center for Research on Evaluation, Standards,