1

Aligning Assessment Methods with Learning Outcome Statements and Curricular Design

Presented atCCRIApril 8, 2005PeggyMaki@aol.com

Material from Maki, P. (2004). Assessing for Learning: Building a Sustainable Commitment Across the Institution. Stylus Publishing and AAHE.

2

Alignment—degree to which learning outcome statements match how and what we teach; degree to which assessment methods match or build upon our collective educational practices

What methods of assessment capture desired student learning--methods that align with pedagogy, content, and curricular design?

3

“The tasks to which students are asked to

respond on an assessment are not arbitrary. They must be carefully designed to provide evidence that is linked to the cognitive model of learning and to support the kinds of inferences and decisions that will be based on the assessment results.”

National Research Council. Knowing what students know: The science and design of educational assessment . Washington, D.C.: National Academy Press, 2001, p. 47.

4

Assumptions UnderlyingTeaching

Actual Practices

Assumptions UnderlyingAssessment Tasks

Actual Tasks

5

When Will You Seek Evidence?

Formative—along the way? For example, to ascertain progress or development

Summative—at the end? For example, to ascertain mastery level of achievement

6

What Tasks Elicit Learning You Desire?

Tasks that require students to select among possible answers (multiple choice test)?

Tasks that require students to construct answers (students’ problem-solving and thinking abilities)?

7

Direct Methods

Focus on how students represent or demonstrate their learning (meaning making)

Align with students’ learning and assessment experiences

Align with curricular-and co-curricular design verified through mapping

8

Invite collaboration in design (faculty, students, tutors)

9

Standardized Instruments

Psychometric approach—values quantitative methods of interpretation

History of validity and reliability

Quick and easy adoption and efficient scoring

One possible source of evidence of learning

10

Do Not Usually ProvideEvidence of strategies, processes, ways of

knowing, understanding, and behaving that students draw upon to represent learning

Evidence of complex and diverse ways in which humans construct and generate meaning

Highly useful results that relate to pedagogy, curricular design, sets of educational practices

11

Authentic, Performance-based Methods

Focus on integrated learning

Directly align with students’ learning and previous assessment experiences

Provide opportunity for students to generate responses as opposed to selecting responses

12

Provide opportunity for students to reflect on their performance—strengths, weaknesses, repositioned learning

13

Do Not Provide

Immediate reliability and validity (unless there has been a history of use)

Usually do not provide easy scoring unless closed-ended questions are used.

14

Some Options for Alternative Methods

E-Portfolios

Capstone projects (mid-point and end-point)

Performances, productions, creations

Visual representations (mind mapping, charting, graphing)

15

Case studies

Disciplinary or professional practices

Agreed upon embedded assignments

Selection of assignments students hand in

Writing to speaking to visual presentation

16

Team-based or collaborative projects

Internships and service Projects

Oral examinations/questions

Critical incidents

17

Externally or internally juried review of student projects

Externally reviewed internship

Performance on a case study/problem

Performance on case study accompanied with students’ analysis

18

Locally developed tests

Pre-and post-tests

Learning Logs or Journals

Videotaping over time

19

Simulations—virtual labs, scenarios that track decision making and actions

Magic box—problem solving over time

20

Indirect Methods-- May Be Combined with Direct Methods

Focus group (representative of the population)

Interviews (representative of the population)

Surveys

Transcript analyses

21

Other sources of information that contribute to your inference making: CCSSE results, grades, participation rates or persistence in support services, course-taking patterns, majors

22

Identify Methods to Assess Outcomes

Using the handout, identify both direct and indirect methods you might use to assess several of your outcomes. Determine the kinds of inferences you will be able to make based on each method.

23

Examples of Changes:

Increased attention to weaving experiences across the institution, a program, or a department to improve student achievement

Changes in advising based on assessment results

Closer monitoring of student achievement--tracking

24

Faculty and staff development to learn how to integrate experiences that contribute to improved student learning

Changes in pedagogy and curricular and co-curricular design

Development of modules to assist learning; use of technology; self-paced learning, supplemental learning

25

Mission/Purposes

Learning Outcomes

How well dowe achieve

our outcomes?

Gather Evidence

Interpret Evidence

Enhance teaching/ learning;

inform institutional decision-

making, planning, budgeting

26

“What and how students learn depends to a major extent on how they think they will be assessed.”

John Biggs, Teaching for Quality Learning at University: What The Student Does. Society for Research into Higher Education & Open University Press, 1999, p. 141.

27

Works Cited

Biggs, J. (1999). Teaching for Quality Learning at University: What The Student Does. Society for Research into Higher Education & Open University Press, 1999, p. 141.

Maki, P. (forthcoming, 2004., May). Assessing for Learning: Building a Sustainable Commitment Across the Institution. Sterling, VA: Stylus Publishing, LLC, and the American Association for Higher Education.

National Research Council. 2001. Knowing What Students Know: The Science and Design of Educational Assessment. Washington, D.C.: National Academy Press