March 26, 2007 Microsoft Research India 1
Tutored Video Instruction and Course Export
Richard Anderson
University of Washington
March 26, 2007 Microsoft Research India 2
Course Export
• Offer established courses at external institutions
• Remote institution is not in a position to offer the course
March 26, 2007 Microsoft Research India 3
Tutored Video Instruction
• Recorded video basis of instruction• Video watched in a group setting with a
facilitator• Model – alternate between video and
discussion• Technology has radically changed costs
– Capture– Distribution– Replay
March 26, 2007 Microsoft Research India 4
TVI Projects
• Gibbons, 1974-1980– Stanford University and HP– Master’s Level Engineering Courses
• UW, 1997-2000– University of Washington and Community Colleges– Introductory Computer Programming
• Digital Study Hall, 2005-– India, Urban and Rural Schools– Primary education
• UW, 2006– University of Washington and Beihang University, Beijing– Senior level Computer Science Course (Algorithms)
March 26, 2007 Microsoft Research India 5
What factors influence success or failure of TVI?
• Initial measures of success:– Impact on students– Sustainability
• Additional measures of success:– Impact on facilitators– Institutional relationships
March 26, 2007 Microsoft Research India 6
Gibbons, Stanford
• Master’s level courses offered between Stanford University and HP Engineering sites– Offer Stanford classes to remote degree candidates– Students received Stanford Credits– Centralized grading
• Courses recorded live– Single camera– Distributed by videotape– Video quality poor – e.g., blackboard writing illegible– Quality of instructor considered important
March 26, 2007 Microsoft Research India 7
Stanford
• Small sections led by facilitators– 3-10 students– Model – stop video frequently for discussion
• Once every five minutes or when question• Goal – student initiated discussion
– Peer facilitation• Theory that discussion would be better with peer
facilitation• Student who took the course the previous year
March 26, 2007 Microsoft Research India 9
UW-Community Colleges
• Introductory Computer Science courses from UW offered at Community Colleges
• State of Washington Higher Education Model
UW
Branch Campuses
WSU
CC CC CC CC CC CC
March 26, 2007 Microsoft Research India 10
UW - CC
• Problems being addressed – Lack of qualified CS Instructor’s at CC’s– Courses not available at CC’s– Transfer students not having an equivalent
background
• Political sensitivities between UW and CC’s
March 26, 2007 Microsoft Research India 11
UW-CC TVI
• Materials recorded from Live UW lectures– Talking head + slides– Shown in CC courses with CC instructors as
facilitators– Wanted to have instructors be “peers”
• Grading done by UW
March 26, 2007 Microsoft Research India 12
UW-CC
• Two phases of project
• Phase I– Live materials recorded at UW– Grading Centralized– Substantial management from UW
• Phase II– Studio created materials– Grading handled by CC’s
March 26, 2007 Microsoft Research India 13
Digital StudyHall
• Facilitated video instructor for primary education in rural India– Severe resource constraints– Limited teachers
• Hub and spoke model– Multiple sites (Lucknow, Bangalore, Pune)
• Capture of live-staged lectures
March 26, 2007 Microsoft Research India 14
UW - Beihang
• Pilot course offering
• Algorithms course taught at UW and Beihang University
• Beihang students received Beihang credit for the course
• Goal was to offer course using US pedagogy and content
March 26, 2007 Microsoft Research India 15
UW-Beihang
• Materials captured from live classes– Slides, talking, head,
digital ink
• Classroom Technology– Students used Tablet
PCs to participate in classroom activities
– Tablets PCs used both at Beihang and UW
March 26, 2007 Microsoft Research India 16
Facilitation
• Three sections of 25 students each in Beihang
• Teaching Assistants led each section
• Instructional materials in English, but much of the discussion was in Chinese
March 26, 2007 Microsoft Research India 17
High degree of UW involvement
• Initial visit– Set up– Training– Introduction to the students
• Second visit– Observation– Deliver class
• Support materials– Lecture summaries
March 26, 2007 Microsoft Research India 19
General Lessons
• Were these courses successful?• Stanford
– Considered successful in terms of outcomes
• UW-CC– Initial offerings had mixed results– Some of the sections could be considered successful– Program was not sustainable
• UW-Beihang– Pilot offering, student outcomes and evaluations were
positive
March 26, 2007 Microsoft Research India 20
Institutional Relationship
• Student View– Stanford – favorable– UW CC – negative, fear of competition– UW Beihang – favorable
• Facilitator View– UW CC – mixed
• Resentful• Supportive of program• Personal advancement• Positive because course couldn’t be offered without support
– UW Beihang• Favorable
March 26, 2007 Microsoft Research India 21
Facilitation I
• Peer vs. non-peer– Stanford – peer– UW CC – non-peer– Beihang - peer
• Often extensive involvement of source site– Negative: Limits scaling– Positive: Mentorship
March 26, 2007 Microsoft Research India 22
Facilitation II
• Relationship between facilitator and external teacher– UW CC: Mixed
• Competitive or supportive• Reflected general attidudes
– Beihang• Teacher / TA
March 26, 2007 Microsoft Research India 23
Facilitation III
• Gibbons – Encourage students to ask questions to figure
out the material
• UW– Co-teaching with recorded materials– Facilitators initiated interaction (not students)
• Broad range of techniques
• Preparation by facilitators important
March 26, 2007 Microsoft Research India 24
Institutional Course Adoption
• Transfer of curriculum– Institution adopts curriculum or teaching style
• Transfer of expertise– Facilitators gaining experience to teach the
course
March 26, 2007 Microsoft Research India 25
Delivery Mechanics
• Course materials – Gibbons – capture of live materials– UW CC
• Version 1 – Live• Version 2 – Studio
– UW Beihang• Live – designed for TVI
• Supporting materials– UW CC – classroom activities– UW Beihang – lecture summaries
March 26, 2007 Microsoft Research India 26
Evaluation
• Learning outcome evaluation– Established deployments
• Robust technology, methodology, programs
– Autonomous– Multiple instances– Control– Education experts
March 26, 2007 Microsoft Research India 27
Additional approaches to evaluation
• Learning outcomes are not the only metric – Broader range of impact– Richer understanding
March 26, 2007 Microsoft Research India 28
Evaluation
• Student outcomes– How do students perform– Attendance
• Student reactions– Survey
• Student satisfaction• Evaluation of different components
– Short answer
• Facilitator reaction– Interviews
March 26, 2007 Microsoft Research India 29
Understanding classroom behavior
• The TVI methodology has certain classroom goals– Are these achieved?
• Observation• Data logging
– Observational (e.g., rounds of communication)– Technology logs
• Digital artifacts
March 26, 2007 Microsoft Research India 30
What I have learned about TVI
• Relationship between institutions/partners is absolutely critical– Inherently asymmetric– Motivations open to question
• Role of “source” site– Develop materials that are suitable for target
population and facilitated playback• No general guidelines
– Important to support facilitators• Training• Support materials
March 26, 2007 Microsoft Research India 31
What I learned about TVI
• Potential long term impacts on facilitators and institutional relationships
• Facilitation– Co-instruction by facilitator and video (not
Peer instruction as proposed by Gibbons)– Wide range of practices adopted by
facilitators– Facilitator training and attitude important
March 26, 2007 Microsoft Research India 32
• For more information, contact Richard Anderson ([email protected])
• UW-Beihang course website:
• Papers
• This work has been supported by Microsoft External Research and Programs, University of Washington, and Microsoft Research Asia
Questions?
http://www.cs.washington.edu/education/courses/cse421/06au/
http://www.cs.washington.edu/education/dl/presenter/papers.html