Understanding and Promoting Interaction in the Classroom Ph.D. Defense Steven A. Wolfman Computer Science & Engineering University of Washington UNIVERSITY

Understanding and Promoting Interaction in the Classroom

Ph.D. Defense

Steven A. Wolfman

Computer Science & Engineering

University of Washington

UNIVERSITY LEVEL

Steve Wolfman Understanding and Promoting Interaction 2

t

The Blackboard

“…in the winter of 1813 & '14 … I attended a mathematical school kept in Boston…On entering [the] room, we were struck at the appearance of an ample Black Board suspended on the wall… I had never heard of such a thing before.” [Samuel J. May, 1855]

BENEFITS: successful ed tech

CHALLENGES: few such successes

LEVERAGE PNT: public mediating artifact

BIG EXCEPTION: PowerPoint


t

Mediating Artifact [Vygotsky]

An external object or structure that participates in cognition by supporting or

shaping thought.

Vygotsky: thought is socialDISTRIBUTED COGNITION


t

Slides as Mediating Artifact

• In the classroom:– facilitate communication– structure discussion

• Outside the classroom:– used as memory aid– used as study guide

• Across terms:– reify course knowledge

Persistent context for communication!


t

Thesis Question

How can computer technology exploit the mediating nature of

presentation slides to support and shape interactive learning and

teaching?UNIVERSITY LEVEL


tDistance & Large Class StudiesClassroom

Presenter

Gestural Modelof Ink

ClassroomFeedbackSystem

Feedback PatternsStructured Interaction Presentation system

Research History


t

Design Experiment Methodology [Brown]

Class studies Theoretical framework

SystemdesignEvaluation &

user-centered design


t

Outline

• Introduction

• Classroom Presenter

• Classroom Feedback System (CFS) and feedback patterns

• Structured Interaction Presentation System (SIP)

• Conclusions


t

Explore slides as mediating artifact

Classroom Presenter Goals

• Maintain strengths of slideware (organization, preparation, sharing, execution)

• Mitigate weaknesses of slideware(inflexibility, immobility, passivity)

• Secure classroom adoption

• Prepare for more ambitious systems

Response to dist stud: INK in CONTEXTTRANS: benefit from separating views (fract med art)

Prelim studies indicate need: flexibility/contextual writing

PETTT studies suggest like: organization / preparation


t

benefit from separating views (fract med art)

Instructor view with notes

Displayed view without notes

COMPARE WRITTEN NOTES WITH INK!


t

Innovations from User-Centered Design

• Instructor notes

• Filmstrip and slide previews

• “Whiteout”

• “TV Talk Show” Tablet

• Collective brainstormingFlexibility/interaction enabled by exploiting:SEPARATION OF VIEWS, INK IN CONTEXT, SLIDES AS MED ART


t

Classroom Deployments

Surveyed since Spring ’02:– 37 courses– 21 instructors– 2,000+ students– CSE courses: introductory to Master’s level– UW, U. of Virginia, & U. of San Diego


t

Survey Results

• Positive comments and repeat use by instructors• Instructor survey: N = 9

• Student surveys: N = 479Attention to lecture 10% less 35% no change 55% more

Encourage future use 8% disc. 22% neutral 69% enc.

Students engaged in lecture

0% less 44% no change 56% more

Use in future 0% no 33% maybe 67% yes

Omits all project participants.


t

Contributions (Presenter)

• Combined strengths of slides w/increased flexibility, mobility, potential for interaction

• Developed features that exploit slides as mediating artifact:– Ink in context– Separation of views

• Secured broad adoption

• Established basis for student extensions


t

Outline

• Introduction




• Conclusions


t

CFS Goals

• Understand challenges to interaction

• Develop system exploiting slides as mediating artifact to respond to challenges

• Evaluate impact of feedback system

• Understand how feedback system changes interaction

NOT FOCUS today.


t

Challenge to Interaction:Feedback Lag

A student hesitates to pose a question until the instructor finishes a point. When

the instructor moves on, the question seems out of place and is left unasked.EVIDENCE: Survey/focus group responses (3/12 in pilot; 5/11 in final)Lagged questions in video archives; Personal experience

TRANS: built CFS in response to challenges WE identified like this one


t

Leaving Feedback on Current Slide


t

Leaving Feedback on Last Slide


t

Instructor View


t

“Retrospective” Feedback

Students’ response (n=12; 7 sessions; 150 students total):– 29 episodes of retrospective feedback– CFS helped all who reported feedback lag

Instructor’s response:– Retro. feedback is important; often responded– Retro. feedback upset pacing

Adoption lessons for classroom tech.Not enough retrospect

So, did it work?

Imagine n=150!


t

“Prospective” Feedback

What if a student leaves feedback ahead of the discussion?

“…if I’m smooth enough… the class will just think ‘Oh, he’s going to talk about [that] now.’ … [To them,] here’s something that for some reason I decided to talk about towards the end of the slide.”


t

Prospective Feedback Episode


t

Contributions (CFS)

• Identified interaction challenges• Proposed slide context as interaction medium• Developed contextual feedback system• Established potential for student feedback• Discovered novel interaction patterns

– retrospective feedback: addressing feedback lag– prospective feedback: enabled by computer-mediated

communication


t

Outline

• Introduction




• Conclusions


t

“Conductor-of-Performances” Model

Computer-Supported Collaborative Learning (CSCL) has been “a move from ‘sage-on-the-stage’ … to ‘guide-by-the-side’”.

New CSCL systems will be “much more like the ‘conductor-of-performances’ for an orchestra: students … [will contribute] to an overall performance.”

[Roschelle & Pea]Leader-of-theater?


t

Goals of Structured Interaction Presentation System (SIP)

• Mitigate slides’ passivity, oversimplification

• Maintain intuitive, flexible design

• Explore enabled interactions

• Understand how integrated exercises affect the classroom

Support the design, use, sharing, and reflection on the “orchestra’s” score.

interactive exercises +rich student data

w/PPT-style design+ widgets


t

EXPLAINING SIP “by example”Experimental class on “Risk Assessment”24 “students”50 minutes6 SIP exercisesTook ~1 hour to convert static => interactiveDesigned INSIDE PPT USING SIP, etc.


t

SIP Architecture

Presentationdesign

environment

Presentation/Widget

database

Instructor view

ViewerscrnshtViewer

scrnshtViewerscrnshtStudent views

Interactive widget designenvironment

Presenter + SIP exercises

SQL back-end for reliability/archival reuse.

Pluggable widgets. Support interactive version of PPT vision.


tREMIND: exercises from experimentalclass; REAL DATA




t


t

Distributed Human Computation (DHC)

• Are these on the same or distinct topics?

• Which would you rather discuss?

Of those who died from receiving the vaccine, what percentage had compro-mised immune systems?

What are the death rates for specific groups who received this vaccine?


t


t

DHC Results: Instructor’s View

Group members

Group “winners”


t

Experimental Class Results

• Interaction analysis (video/audio/logs): substantial engagement by students

• Student survey results– Factors supporting interaction:

highlights particular strengths of integration– Factors hindering interaction:

highlights important design lessons


t

Interaction Analysis

0:00:00 0:10:00 0:20:00 0:30:00 0:40:00 0:50:00 1:00:00

teacher talk

student talk

studentdiscussion

studentthinking

other

98% participation in SIP exercises4-7 interactions reported/student (median)3 recorded voicings/student (72 total)

68% K-12 norm

(62%)

(15%)

(13%)

(5%)

(5%)


t

Factors Supporting Interaction (n=18)

Category % students

Sharing responses w/ whole class 39

Participatory feel 33

Novelty 22

Anonymity 17

Forced to participate 17

Helps follow instructor 11

Neighbor discussion 6

None 6

Did not respond 6“anonymity encouraged honest participation”“I felt as though I … didn’t have much choice but to participate … I think this is a good thing.”

78% cited supporting factors EXCLUDING NOVELTY


t

Factors Hindering Interaction (n=18)

Category % students

None 56

Distracting applications 22

Distracted looking at my slides 17

Did not respond 11

Reduced coverage 11

Lack of student control 11

Pace too fast 6

Pair discussions 6

Anti-climactic exercise 6

50% cited aspect hindering interaction


t

Contributions (SIP)

• Proposed “score” as role of slides as mediating artifact in “orchestra” CSCL model

• Developed prototype SIP system

• Designed novel interactive exercises (e.g., DHC, “sampled quiz”)

• Identified advantages and pitfalls of integrated interactivity


t

Conclusions

• Developed and evaluated widely-adopted Classroom Presenter system

• Developed and evaluated Classroom Feedback System

• Developed and evaluated Structured Interaction Presentation System

Demonstrated how to exploit slides as mediating artifact across all three systems (e.g., separated views, contextual feedback, “forced” participation)


t

Related Systems

• Ubiquitous Computing:– eClass/Classroom 2000 [Abowd & Brotherton]– ActiveClass [Griswold, Ratto, Truong, et al.]– Cell-phone feedback [Brittain]

• Education/Educational Technology:– ClassTalk [Dufresne]– Debbie/DyKnow [Berque]– WILD [Roschelle & Pea]

• HCI: Pebbles [Myers]

DISCUSS HOW CP/CFS/SIP FIT IN


t

Related Pedagogy

• Active learning [Bonwell & Eison]

• Active learning in CS [McConnell]

• Classroom Assessment Techniques “CATs” [Angelo & Cross]

• CATs in CS [Schwarm & VanDeGrift]

• Collaborative Learning [Johnson & Johnson]


t

Acknowledgments

• Richard Anderson and the Committee• Rachel Pottinger• Education and Educational Technology Group• Microsoft Research LST Group• Experiment participants• Faculty, staff, and students of UW CSE• Intel, MERL, Microsoft, and NSF for funding• Everyone

SPARES


t

Modern Pedagogy vs. Modern PracticeWARNING: overgeneralization, but borne out by research

active learning

participatory

interactive

student-directed

lecture

instructor-dominated

passive

disconnected

~80-90% lectures Thielens, 1987Many ways to resolve tension: reduce class size, retrain instructors, etc.QUOTE ON RQ SLIDE


t

Pedagogy of Active Learning

• Encourage “connected” learning– Constructivism [Bruner]– Social learning [Lave]

• Recapture flagging attention– Attention studies [Stuart & Rutherford]– Heart rate/memory [Bligh]– Skin conductivity [Picard]

• Address varied learning styles – Index of Learning Styles [Felder & Silverman]– Bloom’s taxonomy

Smiley plot


t

Large class challenges

• Maintaining attention

• Communication/Feedback

• Spontaneous discussion

• Management of class activities

Bruner (constr.) Lave (soc. Learning)Attention studies [Stuart & Rutherford]Heart rate/memory [Bligh]Skin conductivity [Picard]ILS [Felder & Silverman]Bloom’s taxonomy

CFS: 2.4 voicngs/class, 90-120 studs

Tech problems in distance

Our experience & McConnell


t

Slides as Externalization/Mediating Artifact

Saljo: the significance of new technologies does not lie in their enhancing learning in a linear sense… the important point about new technologies is that they, if they are powerful enough, transform basic features of how people communicate knowledge and skills in society and how information is organised. In this sense, new media may imply that learning will become different. Technologies are ultimately about the regulation and improvement of human relationships Draw mental arith– paper and pencil – mem

Elec calculator --alg:communicates in familiar symbolic representation

HK Jade market : comm burden


t

CP


t

Slideware Strengths/Weaknesses

Strengths• Organization [PETTT]• Preparation [PETTT]• Sharing [Bligh]• Easy execution [Bligh]

Weaknesses• Passivity [PETTT]• Simplified ideas [Tufte]• Inflexibility [VanDeGrift]• Immobility

Preliminary studies indicated instructors needed: flexibility and contextual writing

PETTT studies suggest students like: organization and preparation.


t

Slide previews with navigation


t

CFS


t

Design Experiment [Brown]

• Discover what inhibits interaction

• Understand what makes a good design

• Design intervention

• EvaluatePIPE DREAM!Iterative process: in particular, Ann Brown’s“design experiment” styleStill, will address the steps in this order.


t

Feedback on Student View


t

Instructor View (2/3)


t

Instructor View (3/3)


t

CFS Evaluation

Intro. programming course, summer 2002:– 150 students, 12 participants w/laptops– 9 week course, 3 weeks with CFS

Data: observations, surveys, focus groups, interview w/instructor, logs

Final evaluation iteration


t

CFS increased interaction

Voicings pre-CFS

Voicings with CFS

All inter-actions

All but “Got it”

# per class

2.4 2.6 15.9 7.9

p-value .91 .04* .14


t

SIP




t


t

TRANS: DESIGN


t


t


t


t

Experimental SIP Class

Topic: Risk Assessment

Duration: 50 minutes

Students: 24, CS stud-ents, faculty, staff, each w/a Tablet PC

Presentation included six SIP exercises.

Results:• System successful• 98% participation in

SIP exercises• 4-7 interactions per

student (median)• 62% “teacher-talk”

Took ~1 hour to convert static => interactiveDesigned INSIDE PPT USING SIP, etc.62% T-T (compare to 68% K-12 norm)Median of 3-5 interactions through SIP


t

Design Lessons

• SIP’s integrated exercises create “participatory feel”• Students’ social expectations support participation• SIP anonymity policies should be easy to specify

and understand• Student interface should present few distractions• Student interface should provide clear value

(independent navigation and notetaking)• Instructors must still motivate interactive pedagogy


t

Documents

Understanding and Promoting Interaction in the Classroom Ph.D. Defense Steven A. Wolfman Computer Science & Engineering University of Washington UNIVERSITY