Optimizing Performance-Enhancing Tools for Teaching and Assessment Capturing, representing, and...

Optimizing Performance-Enhancing Tools for Teaching and Assessment

Capturing, representing, and using classroom data to enhance workflow, diagnosis, decision making, and

instructional improvement

OPerETTA TeamCenter for Technology in Learning

SRI InternationalJuly 18, 2007

Contact: Mark.Schlager@sri.com

This is what early air traffic control looked like

This is how teaching looked . . .

Air traffic control today

Teaching today . . .

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

• Cognitive • Task• Communicative• Temporal

High-Performance Professionals Rely on Real-Time Data

…to increase efficiency, optimize performance, and improve their own practice.

Technology for Teaching & Learning

• Teachers perform highly complex, knowledge-intensive work in time-pressured conditions

• Teachers need a classroom performance-support infrastructure

– Automatically captures simple and complex data in real time

– Provides visual “pattern-recognition” feedback for decision-making and instruction

– Addresses workload and workflow problems

– Creates a powerful learning environment

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Common Instructional Challenges

•Time on task is lost every period– “Overhead” tasks with little learning payoff

•Don’t know what all students know– Student assessment data are too little too late

•Can’t individualize instruction– No time to create, deliver, grade different work, assessments– Top and bottom students tune out

•Teacher learning/improvement opportunities are lost– No feedback on their own performance to hone their practice– No access to classroom data patterns over time

•Wireless classroom infrastructure can help

Can Real-time Feedback Improve Learning and Teaching?

Engagement (S)

Participation (S)

Decision making (T)

Diagnostic power (T)

Time on task (S&T)

Metacognition (S&T)

Accountability (T&S)

Building-blocks of Cognitive and Workflow Performance

Cognitive

DensityIncrease quality and bandwidth

Enable simultaneous channels

Enable message buffering Communicative

Density

Automate routine procedures

Reduce down time

Enable timely feedback

Differentiate task start/stop times

TemporalDensity

Capture work process & product

Resource access and variety

More & better student feedback

Feedback to teacher

Task/Content

Density

Instructional Improvement

Capturing, Representing, and Presenting Data to Optimize Activity Density

Framework for OPerETTA Classroom

Students Density Teacher

Communicative

Temporal

Task/Content

Cognitive

Workflow: Minutes lost in every class

• Problem:

Here!

X + 27 = 54 x = ?

Diagnosis: Knowing what students know

• Problem:No questions?

OK let’s move on…Here’s your quiz

from two weeks ago.

I don’t understand this, but I don’t want to sound

stupid asking for help.

She coveredthis last week!I wonder what’s

for lunch today?

Huh?

Analyzing Performance: Teacher Improvement

“Analyze student work,” they tell us. I’d like to see themanalyze all this and

make sense of it

Homework Results

0 20 40 60 80 100

5

4

3

2

1

Problem #

% CorrectHomework Results

0

20

40

60

80

100

5 4 3 2 1

Problem #

% Correct

Attendance

28 students present2 students absent

•Alice•Mike

1 student excused due to illness•Sam

Problem #5

27 + x = 53

X = ?

Improved Workflow

Great, I see everyone did their

homework.Looks like problem #5 was challenging.

The first minute of class

Knowing what students know

OK, I want everyone to work on this

problemx + 33 = 54

x = ?

Jane, Can you help

me?

Subtract 33 from

54

Knowing what students know

Quiz Results

x = 21………49%

x ≠ 21………52%

Knowing what students know

OK, everyone is done. Let’s take a look at the results.

x + 33 = 54

x = ?X = 21

Looks like half the class has a misconception.Let’s talk about

that.

Warm-up Results

0 20 40 60 80 100

4

3

2

1

Problem #

% Correct

Individualizing instruction

OK, I’m going tosend out problems

based on the results from the warm-up

Set

B

Set

A Correct

Andy

Brian

Cathy

Jim

Susan

Tom

Valerie

Not Correct

Alan

Billy

Debra

James

Sandy

Tina

Vickie

Correct

Andy

Brian

Cathy

Jim

Susan

Tom

Valerie

Set

A

Individualizing instruction

Set

B

Set

A Correct

Andy

Brian

Cathy

Jim

Susan

Tom

Valerie

Not Correct

Alan

Billy

Debra

James

Sandy

Tina

Vickie

Not Correct

Alan

Billy

Debra

James

Sandy

Tina

Vickie

Correct

Andy

Brian

Cathy

Jim

Susan

Tom

Valerie

Set

B

Individualizing instruction

Teaching Improvement

Master %

0 20 40 60 80 100

5

4

3

2

1

Problem #

% Correct

Student Wait Time

0 20 40 60 80 100

5

3

1

Problem #

Time

Lesson Plan

Pop quizResultsChallenge ProblemsReview Chapter 9Homework Assignment

Male to Female Ratio

Girls

Boys

Hmm, let’s compare my 3

algebra classestoday on #5

Whole is greater than the sum of current tools• Learner-centered content-focused simulation, modeling,

calculation, gaming

• Classroom communication/student response

• Formative assessment

• Lesson planning

• Productivity, student information, grading,and accountability.• 1-to-1 computing

• WIFI

• Web 2.0

How can we integrate these tools and capture the data they generate?

How can we interpret and represent the results as patterns that can be recognized and acted on in real time?

Infrastructure for High-Performance Teaching

Air traffic controllers without automation?Medical clinicians without automation?

Teachers with automation?What do you think?

If Medical Clinicians Worked under Classroom Conditions

I’ve only got time for one question. If you can’tdiagnose yourself, ask the

person beside you

I’m gonna ace this steroid test

By the time Iget the test results,they’ll be in ICU

I want a CT scan for everyone to rule out

head trauma. Stat!

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

If Air Traffic Controllers Worked under Classroom Conditions

Work on your incursionsBilly; I’ll check them and

give you feedback next week

I don’t have time to lookat all these flight plans.I’ve got to move metal!

I have a mental modelof where all the planesare...except those two.

OK, everyone’s in a holding pattern while I

help UA34 get backon course