Knowledge Surveys Knowledge Surveys and SRLcurriculumredesign.org/wp-content/uploads/Wirth-OECD-Metacognition.pdf · Interdisciplinarity, Metacognition and Self-Regulated Learning

CCER, BIAC, OECD/CERI Colloquium 14-15 October 2013

1

A Metacurriculum on Metacognition:

What Instructors and Students Learn

From Thinking About Learning

Karl WirthMacalester College

CCR Colloquium on Interdisciplinarity, Meta-

Cognition and Self-Regulated Learning

14-15 October 2013, OECD, Paris

Knowledge Survey

Interdisciplinarity, Metacognition and Self-Regulated Learning

Pre Post

1 1I can list the essential characteristics of a knowledge

survey.

2 2

I can describe two different ways that technology can be

used to foster metacognition in advanced learning

technologies.

3 2

I can use a knowledge survey to ascertain background

knowledge and learning gains related to a unit of

instruction.

4 3

I can demonstrate an understanding of the concept

“learning to learn,” it´s meaning and it´s different

components

5 4I can describe and analyze how “learning to learn” could

be measured and the challenges related to it

6 4

I can critically analyze how the concept of “learning to

learn” can be used to support learning in different

contexts.

7 4

I can compare and contrast two different interdisciplinary

methods for detecting, tracking, and modeling

metacognition in advanced learning technologies.

8 6

I can design and implement metacognitive instructional

activities (e.g., learning to learn, knowledge surveys) for

for the purposes of assessing student learning, promoting

metacognitive skills, and for improving instruction.

1 I can not complete this task at the present time

2 I can partially complete this task at the present time

3 I can fully complete this task at the present time

Item Knowledge or Ability StatementBloom

Level

Self-Assessment

Response Scale:

Let’s Take a Quiz

Respond with:

1. I don’t know

2. I know some

3. I know it

Knowledge Surveys

Knowledge Surveys:

• Introduced by Nuhfer (1993, 1996)

• Content & skill objectives of course

• Span levels of thinking (Bloom levels)

• Students indicate perceived knowledge/ability

• Goal-Setting

• Monitoring

• Self-Assessment

Knowledge Surveys and SRL


2

Expert Learners

Modified from Ertmer and Newby (1996)

Evaluate Monitor

Plan

Reflection

Metacognitive Control

(self-regulation)

Metacognitive Knowledge

(declarative, procedural, conditional)

Personal

Resources

Prior Knowledge

Available Strategies

Task

Requirements

Type of Learning

Appropriate Strategies

GoalsBeliefs

AttitudesMotivation

Reflection Reflection

Reflection

Example Survey Items

Items Level

I can recite the definition of a mineral. Remember

I can describe the cooling of a basaltic magma

using Bowen’s Reaction Series.Understand

I can calculate the relief of a region using a

map.Apply

I can compare the generation of melts along

convergent and divergent plate boundaries.Analyze

I can evaluate three potential sites for

suitability to host a municipal waste landfill.Evaluate

I can construct a model of the origin of a suite

of rocks from the Sonju Lake Intrusion.Create

Survey Items Administering Knowledge Surveys

Single class, Unit, Course, Curriculum

Paper (in-class)

Electronic (in-class with clickers; out-of-

class with learning management system)

Beginning of Course

End-of-Course

Pre-Exam

Graduating Seniors

Alumni


3

Now it is time to learn.

Pay attention!

You are going to be tested laterG

(but you already know what’s on the test)

A Metacurriculum on Metacognition:

What Instructors and Students Learn

From Thinking About Learning

Karl WirthMacalester College

CCR Colloquium on Interdisciplinarity, Meta-

Cognition and Self-Regulated Learning

14-15 October 2013, OECD, Paris

Pre-Instruction Self-Reported Knowledge Self-Reported Learning Gains


4

Prior Knowledge and Gains by Item KS Results - Individuals

Exam & Predicted Scores Exam & Predicted Scores


5

Exam & Predicted Scores Learning Gains

Pre-

diction

Know.

Surv.

Post-

dictionScore Signif.

Prediction 1.00 <0.0001

KS 0.44 1.00 <0.0001

Post-diction 0.56 0.71 1.00 <0.0001

Score 0.25 0.61 0.48 1.00 <0.0001

Comparison of Self-Assessments “Map” of Learning for Dynamic EarthPre-Course Survey

Stu

dents

Survey Items


6

Pre-Course Survey

Pre-Exam I Survey

“Map” of Learning for Dynamic EarthPre-Course Survey

Pre-Exam I Survey

Pre-Exam II Survey

“Map” of Learning for Dynamic Earth

Pre-Course Survey

Pre-Exam I Survey

Pre-Exam II Survey

Pre-Exam III Survey

“Map” of Learning for Dynamic Earth Utility of Knowledge SurveysCourse Design

Clarification of course objectives and structureImproved organization and preparation

StudentsFull disclosure of course objectives and expectationsStudy guideFormative assessment toolDevelopment of self-assessment skills

InstructorsAssessment of learning gainsCourse assessmentAssessment of instructional practices

ProgramsProgram ObjectivesStudent Learning


7

Metacurriculum for Metacognition

Activity Knowledge or Skills

Knowledge Surveys Goal-setting, Monitor. & Eval.

Reading Reflections Reflection & Monitoring

How I Earned an “A” Goal-setting & Monitoring

Exam Wrappers Refl., Monitoring & Evaluation

Learning Reflections Evaluation & Goal-setting

Critical Thinking Strategies for Thinking

Expert Learners - Reflection

Modified from Ertmer and Newby (1996), Butler (1997), Winne

and Hadwin (1998), Pintrich (2000), Lovett (2008)

Evaluate Monitor

Plan

Reflection

Metacognitive Control

(self-regulation)

Metacognitive Knowledge

(declarative, procedural, conditional)

Personal

Resources

Prior Knowledge

Available Strategies

Task

Requirements

Type of Learning

Appropriate Strategies

GoalsBeliefs

AttitudesMotivation

Reflection Reflection

Reflection

Reading Reflections:

Reflection and Evaluation

Reading and Reflecting

Reading Reflections:

• Completed after each reading assignment

• Short responses to a few questions

• Submitted online before class

• Credit awarded for “reflective” submissions

• What is the main point of this reading?

• What did you find surprising? Why?

• What did you find confusing? Why?


8

Survey of Reading Strategies

Depth of reading

Environment

Reading Strategies

• Pre-Reading

• Reading

• Post-Reading

Learning Reflections:

Goal Setting/Reflection-for-Action

Planning and Goal-Setting

The Art of Possibility (Zander and Zander, 2000)

Journal Activity:

Write a letter describing what you

accomplished in this course. The letter

should be dated for the end of the semester

and written in the past tense. Tell me what

you did, how you did it, and how your

thinking and understanding changed as a

result of it.

Begin your letter with:

I earned an “A” in this course because4


9

Wrappers:

A Structured Reflection-on-Action

Exam

“Wrapper”

Achacoso (2004)

Lovett (2008)

• Self Evaluation

• Preparation Strategies

• Performance Analysis

• Planning

Exam Preparation Exam “Wrapper” Results

Study Strategies

Analysis of Errors


10

Reflection and closing thoughtsG?Reflection and closing thoughtsG?

Learning Reflections:

Supporting Learning withReflection-in-Action

Learning to Learn

• Purpose of Education

• Levels of Thinking

• Affective Domain

• Significant Learning

• Meanings of Learning

• Research on The Brain

• Intellectual Development

• Critical Thinking

• Metacognition

• Behavioral Dimensions of Grades

Reflecting on Learning

• What are the three most important things

you have learned? Why?

• Describe the learning

strategies that you are using.

• How might they be adapted

for more effective learning?

• How does learning in this course relate to

other courses?


11

• What do you MOST wish your students

understood about their own thinking and

learning?

Reflective Prompt

The Goal of Higher Education isG

“to help college students become Intentional

Learners who can adapt to new environments,

integrate knowledge from different sources, and

continue learning throughout their lives.”

Greater Expectations

(2002 AAC&U Report)

Intentional Learners

Greater Expectations

(2002 AAC&U Report)

Becoming an intentional learner means:

developing self-awareness about the

reason for study, the learning process

itself, and how education is used.

Intentional learners are integrative

thinkers who see connections in

seemingly disparate information to inform

their decisions.


12

Intentional Learners are also

“Self-Directing”

Savin-Baden and Major (2004)

Self-directing learners are highly

motivated, independent, and strive toward

self-direction and autonomy. They take

the initiative to diagnose their learning

needs, formulate learning goals, identify

resources for learning, select and

implement learning strategies, and

evaluate learning outcomes.

Three Principles of Learning

1. Recognize/Address Preconceptions

2. Expert Knowledge

• Deep foundation

• Contextual framework

• Organizational structure

3. Metacognition is Essential

Bransford et al. (2000)

Metacognition Involves Reflection

• What kind of problem is this?

• What is the best strategy for solving it?

• How will I know if I solved it correctly?

• How could I do it better next time?

• What additional information do I need?

• What use is this new information?

• How can I use my new understanding to

solve different kinds of problems?

Pedagogical Challenge

• Metacognition is a “self-imposed internal

conversation”

• Shown to improve transfer (Bransford et al. 2000)

• Easily assumed that students are doing it, or can

develop on own; both assumptions are wrong

• Challenge is to keep students in constant contact

with their metacognition

• Instruction must be explicit (Pintrich, 2002)


13

Affect – Beliefs About Intelligence

• “fixed” versus “growth”

theories of intelligence

• Affects motivation to

learn and persistence

• Students taught study

skills and brain

plasticity outperform

control groups

Dweck’s “Fixed” vs “Growth” Mindsets

• avoid challenges

• give up easily

• see effort as fruitless

• ignore feedback

• be threatened by

success of others

• embrace challenges

• persist in face of setbacks

• see effort as path to mastery

• Iearn from criticism

• find lessons and inspiration in

success of others

Reflection & Learning

Dimension Description

Habitual Action

Minimal thought and engagement; memorization is

emphasized; correlated with surface learning; tasks treated

as unrelated activities; an attitudinal state of unreflectiveness

Understanding

Focuses on comprehension without relation to one’s personal

experience or other learning situations; book learning that is

understanding-oriented; learning stays within boundaries of

preexisting perspectives

Reflection

Learning is related to personal experience and other

knowledge; involves challenging assumptions, seeking

alternatives, identifying areas of improvement; active

engagement; characteristic of deep approaches to learning

Critical or

Intensive

Reflection

Highest level of reflective learning; learners are aware of why

they think, perceive, or act as they do; as a result, learner

likely alters or changes firmly held personal beliefs and ways

of thinking

Modified from Mezirow (1991) by Kember et al. (2000)

Stages of Learning

Stages Representation of Learning

Transformative

Learning

Meaningful, reflective, restructured by

learner – idiosyncratic or creative

Working with

MeaningMeaningful, reflective, well-structured

Making

MeaningMeaningful, well-integrated, ideas linked

Making Sense Reproduction of ideas, ideas not well linked

Noticing Memorizing representations

Moon (1999)

Documents

Knowledge Surveys Knowledge Surveys and SRLcurriculumredesign.org/wp-content/uploads/Wirth-OECD-Metacognition.pdf · Interdisciplinarity, Metacognition and Self-Regulated Learning