Diana LaurillardLondon Knowledge LabInstitute of Education

Teaching as a design science: Designing and assessing the effectiveness

of the pedagogy in learning technologies

eLearning Forum Asia 2013Hong Kong Baptist University29-31 May 2013

The issues

• Global demand for HE

• The aims of HE

• The roles of TEL

• Modelling learning technologies

• Teachers as innovators

The Challenges to Higher Education

• Teachers to have the capacity to plan and deliver ICT-enriched learning experiences for students to become self-directed and collaborative learners... (Singapore ICT Masterplan, 2008)

• Teacher capacities will be developed in instructional design, selection and critical evaluation of digital content, and strategies for effective use of digital content to enhance student learning. (India Government, 2009)

• Professional educators connected by technology to empower, and inspire effective teaching (US Plan 2010)

• … greater prioritisation of teaching partnerships between technologists, learning support specialists and academics, and an end to the ‘not invented here’ syndrome… Good practice must also be shared. (HEFCE OLTF, 2011)

• Professional development is needed for all school staff to support the process of ICT integration in schools (UNESCO, ICT in Primary, 2012)

The policy context

The global demand for HE

The new UNESCO goals for education:• Every child completes a full 9 years of

free basic education … • Post-basic education expanded to meet

needs for knowledge and skills … (Draft for UNESCO post 2015 goals)

By 2025, the global demand for higher education will double from 100m to ~200m per year, mostly from emerging economies (NAFSA 2010)

- Implying significant teacher training needs for HE

25:1 student:staff ratio = 4m new teachers??

The aims and purpose of HE

The purposes of higher education:• To inspire and enable individuals to develop their

capabilities to the highest…• To increase knowledge and understanding for their

own sake…• To serve the needs of an adaptable, sustainable

knowledge-based economy…• To play a major role in shaping a democratic,

civilised, inclusive society (NCIHE, 1997)

How could mass HE achieve that nurturing of the individual, while reducing the current 25:1 student:staff ratio for student support?

An example: The Duke MOOC

Bioelectricity: A Quantitative Approach Taught in class for over 20 yearsExperimental move to a free and open MOOC12,000 students enrolled from >100 countries• 8 weeks long • 97 ~6 min videos • 22 GB of data • 1052 files • 18 graded exercises, including a peer-graded writing

assignment and final exam (Duke University 2013)

The Duke MOOC

Not for undergraduates

Potential undergraduates

Enrolled students

The Duke MOOC

Not for the faint-hearted

Comparable with normal online u/g

courses

420 hours to develop materials and course design

Basic MOOCs vs the Duke MOOC: Comparing the learning experience

Duke: 8 weeks, providing 50 hours learning time including support:Videos and pdfsQuizzesWikiPeer discussionsPeer gradingTutored discussionsSummative assessment

High on prep timeZero contact for 42 hours

Low on prep timeHigh contact for 8 hours

200 hours to support ~500 students for 8 hours = 20:1 student:staff ratio

Basic: 8 weeks, providing 50 hours learning time, no teacher support

Report at http://bit.ly/ZRMbjp

Comparing teacher hours for a basic MOOC and the Duke MOOC (48 hour course)

Support time 50 500 5000Duke MOOC 20 hrs 200 hrs 2000 hrsBasic MOOC 0.00 0.00 0.00

Teacher support time rises to 2000 hours for 5000 students.

2000 hours = 1 year of a tutor for a 5 credit course.

= 24 FT tutors for 120 credit course.

50 500 50000

500

1000

1500

2000

2500

3000

Duke MOOCBasic MOOC

50 500 50000

500

1000

1500

2000

2500

3000

Duke MOOCBasic MOOC

Total teacher time

Prep time = 420 hrs

Prep time

200

Modelling the benefits and costs

• We need to understand the pedagogical benefits and teacher time costs of online HE

• What are the new digital pedagogies that will address the 25:1 student support conundrum?

• Who will innovate, test, and build the evidence for what works at scale online?

TEACHERS!

Pedagogies for supporting large classes

Concealed MCQs

The virtual Keller Plan

The vicarious master class

Pyramid discussion groups

Tutorial for 5 representative studentsQuestions and guidance represent all students’ needs

Conceal answers to questionAsk for user-constructed input Reveal multiple answersAsk user to select nearest fit

240 individual students produce response to open questionPairs compare and produce joint responseGroups of 4 compare and produce joint response and post as one of 10 responses...6 groups of 40 students vote on best responseTeacher receives 6 responses to comment on

Introduce contentSelf-paced practiceTutor-marked testStudent becomes tutor for creditUntil half class is tutoring the rest

Digital pedagogies for scaling up to higher student:staff ratios without losing quality

Like MCQs, no teacher supportMore challenging learning experienceConcealed MCQs

The virtual Keller Plan

The vicarious master class

Pyramid discussion groups

Little teacher supportEvery learner receives 1-1 support

One teacher guides 5 students They represent all students’ needs

Teacher reacts to 6 student outputsAll students contribute, discuss, vote

First things first – how do students learn?

How does technology help?

Tools for teachers to build pedagogic knowledge

Teachers as design scientists…

How might technology help teachers?

The learner learning

LC

Teacher concepts

LC

LP

LP

Learner concepts

Learner practice

GenerateModulate

Learning through acquisition, instruction

Learning through inquiry

Acquiring

Inquiring

LC

Teacher concepts

Learning environment

LC

LP

LP

Learner concepts

Learner practice

GenerateModulate

Learning through practice with meaningful intrinsic feedback

Task

Actions

GenerateModulate

Feedback

The learner learning

LC

Teacher concepts

Peer concepts

Peer practice

Learning environment

LC

LP

LP

Learner concepts

Learner practice

GenerateModulate

GenerateModulate

GenerateModulate

Actions

Ideas, questions

Ideas, questions

Outputs

OutputsTask/Feedback

Acquiring

Inquiring

Learning through discussion from peers’ ideas, questionsLearning through sharing from peers’ practice

The learner learning

LC

Teacher concepts

Peer concepts

Peer practice

Learning environment

LC

LP

LP

Teacher communication

cycle

Peer communication

cycle

Teacher modelling

cycle

Peer modelling

cycle

Learner concepts

Learner practice

GenerateModulate

Generate

Inquiring Discussing

Acquiring

Practising Sharing

Producing

The teacher needs to design for all these types of learning

Representing types of learning

Modulate

Deploying learning technologies

LC

Teacher concepts

Peer concepts

Peer practice

Learning environment

LC

LP

LP

Teacher communication

cycle

Peer communication

cycle

Teacher modelling

cycle

Peer modelling

cycle

Learner concepts

Learner practice

GenerateModulate

GenerateModulate

Library… Web Tutorial… Facebook

Book… Podcast

Lab… Simulation

Project… VLE

Essay… Model

Libraries, journals, repositories,

databases, Web…

Tutorials, seminars, small groups, online

forums, social media…

Books, lectures, papers, websites,

videos, AR, podcasts…

Essays, designs, performance, programs, videos, ppts, digital assets, models, e-portfolios…

Project groups, teamwork, Google

groups, wikis, VLEs…

Labs, exercises, problems, projects, serious games,

models, simulations…The teacher needs to use all types of learning

technology within the Conversational Framework

Collecting learning analytics

LC

Teacher concepts

Peer concepts

Peer practice

Learning environment

LC

LP

LP

Teacher communication

cycle

Peer communication

cycle

Teacher modelling

cycle

Peer modelling

cycle

Learner concepts

Learner practice

GenerateModulate

GenerateModulate

Library… Web Tutorial… Facebook

Book… Podcast

Lab… Simulation

Project… VLE

Essay… Model

Questions asked Social interaction patterns

What accessed in what sequence

Analysis of essay content, quiz scores, game scores,

accuracy of model

Tracked group outputs, peer assessment

Tracked inputs, reaction times,

sequence

Forms of TEL/online learning activities

• Guided TEL resources (model, AR)

• Access to expositions (lecture videos)

• Automated grading (MCQs, models)

• Readings (pdfs)

• Guided collaboration activities (wiki, AR)

• Peer group discussion (forum)

• Peer grading against criteria (share)

• Tutored discussion (forum)

• Tutor feedback (assignments)

• Tutor-based assessment (exams)

Learning activities for online courses

What are effective ways

of combining and

sequencing these

activities for learning and

formative assessment,

that will help students

achieve the intended

outcomes?

Teachers as designers need the tools for innovation

Tools for teachers as learning designers

To find or create new

ideas

Adopt

Adapt

Test

To collect learning analytics

Redesign

Analyse

Publish

Creating knowledge about effective online pedagogies for specific learning outcomes

Laurillard, D. (2012). Teaching as a Design Science: Building Pedagogical Patterns for Learning and Technology. New York and London: Routledge.

Tools for teachers as learning designers

A library of patterns to

inspect

Tools for teachers as designers - PPC

Colour-coded text identifies content

parametersBlack text expresses

pedagogy design

Tools for teachers as designers - PPC

Each pattern is designed to deliver a specific type of

learning outcome

The Pedagogical Patterns Collector Transferring the pedagogy across topics - PPC

The Pedagogical Patterns Collector Transferring the pedagogy across topics - PPC

Transferring the pedagogy across topics - PPC

Read, Watch, ListenInquireDiscussPracticeShareProduce

Adjust the type of learning activity.Edit the instructions.

Check the feedback on the overall distribution of learning activity

Fully specifies the pedagogy for others to adopt, adapt, test, and share

Export to Word

[Moodle]

Represent the teacher as

present or notSpecify duration of the activity in mins

Add link to an OER, e.g. a digital tool for practice

Teacher adopts and adapts a design

Explain how to optimise the inputs to a patient simulator to achieve the ideal blood pressure

With your partner select different inputs to the patient simulator – can you improve on your previous results?

Design for Med students in PPC

Export to Moodle for Med students

Re-design for Business students in PPC

Explain how to optimise the inputs to a patient simulator to achieve the ideal blood pressure

With your partner select different inputs to the patient simulator – can you improve on your previous results?

Explain how to optimise the inputs to a business model to achieve the optimal cash flow

With your partner select different inputs to the business model – can you improve on your previous results?

Export to Moodle for Med students

The process of professional collaboration:

Find effective patterns for a learning objective – adopt, adapt, improve – export to Moodle – test with students – improve – test - publish

Export to Moodle for Business students

• [The pie-chart] is one of the most useful features … it gives a good overview of the balance between different learning experiences

• I rarely consider how the students' time is apportioned … it's good to be made to think about this.

• Seeing how the sessions are shaping up in such a visual medium …. would probably make me think more carefully about providing a mix of activities

Comments on the PPC

Tools for teachers as learning designers

教学设计库

欢迎 这套教学设计库工具包使教师能够分享他们好的教学理念。其目的是帮助学科教师了解某个特定教学设计可以如何成功地应用于不同的教学主题 。系統中有教学设计样本供浏览和编辑，你也可以从头开始自行设计。这是由ESRC- EPSRC 资助的TLRP-TEL 研究项目之‘教学设计支持环境’的一项成果。

PPC 浏覽器

提供了一些通用的教学设计及其相关的实例，你可以参考或改编这些设计用于自己的教学设计之上

浏覽器

设计师 PPC 设计师

展示教学设计模板来帮助你描述自己某节课的教学理念（如学生准备、课堂活动和家庭作业）

[Chinese version coming soon…]

With a computational model, we can estimate the effects of the design decisions on quality of learning experience AND teaching cost:

consequences for the pedagogical benefits

comparative costs of teachers’ workload

Then check actual outcomes using targeted learning analytics

Modelling learning experience and teacher workload

AcquisitionInquiryDiscussionPracticeProduction

Acquisition

Inquiry

Discussion

Practice

Production

Conventional

Blended

Categorised learning activities

Analysis shows more active learning

A computational representation can analyse how much of each activity has been designed in

Enhancement: Modelling the pedagogic benefits

Efficiency: Modelling teacher time

Learning activities for online courses Preparation time (fixed costs)

Support time (variable costs)

• Guided TEL resources (model, AR)

• Access to expositions (lecture videos)

• Automated grading (MCQs, models)

• Readings (pdfs)

• Guided collaboration activities (wiki, AR)

• Peer group discussion (forum)

• Peer grading against criteria (share)

• Tutored discussion (forum)

• Tutor feedback (assignments)

• Tutor-based assessment (exams)

Modelling the teacher’s workload for increasing student cohort size

Year 1 Year 2 Stable state

0

200

400

600

800

1000

1200

1400

SupportPreparation

Year 1 Year 2 Stable state

0100200300400500600700800900

40 40 40

-£13000 -£600 £2000

40 80 160 Students

-£13000 £19000 £61000 Profit/Loss

Planning teacher time for a new online course at IOE

50 500 50000

500

1000

1500

2000

2500

3000

Duke MOOCBasic MOOC

Total teacher time in hours

Cohort size

Scaling up will never improve the student support costs… unless…

Modelling the teacher’s workload for increasing student cohort size

Time for student support

Time for preparation of resources

From the Duke MOOC

Modelling the costs for increasing student cohort size

… we develop some clever pedagogical patterns that support students at better

than the 25:1 student:staff ratio

The question is – what are they, and how do we develop and share them?

Invest in the teachers who can innovate with learning technologies!

Rethinking University Teaching: A Conversational Framework for the Effective Use of Learning Technologies (Routledge, 2002)

(Chinese edition ECNU Press)

Teaching as a Design Science: Building pedagogical patterns for learning and technology(Routledge, 2012)

d.laurillard@ioe.ac.uk

Further details…

tinyurl.com/ppcollector

The global demand for HE requires investment in pedagogic innovation to deliver high quality at scale

Technology-based pedagogic innovation must support students at a better than 25:1 student:staff ratio

Teachers need the tools to design, test, gather the evidence of what works, model benefits and costs

Teachers are the engine of innovation – designing, testing, sharing their best pedagogic ideas

Teaching as a design science: SUMMARY

The Learning Designer A TLRP-TEL project

The project partners

IOE/LKLBrock Craft (RF)Diana Laurillard (PI)Dejan Ljubojevic (RF)

OxfordLiz Masterman (CoPI)Marion Manton (CoPI)Joanna Wild (RF)

Birkbeck/LKLGeorge Magooulas (CoPI)Patricia CharltonDionisis Dimakopoulos

LondonMetTom Boyle (CoPI)

LSESteve Ryan (CoPI)Ed WhitleyRoser Pujadas (PhD Student)

RVCKim Whittlestone (CoPI)Stephen MayCarrie Roder (PhD Student)

Project website at www.ldse.org.uk PPC at web.lkldev.ioe.ac.uk/PPC/live/ODC.html

ALTSeb SchmollerRachel Harris