E xploring MOOC Pedagogies

Karen Swan, Scott Day, Leonard Bogle & Traci van Prooyen

University of Illinois at Springfield

Exploring MOOC Pedagogies

American Council on Education’s (ACE) College Credit Recommendation Service isevaluating MOOCs for content and approving them for specific college credits if students pass ACE CREDIT tests ($150).ACE CREDIT evaluates MOOCs for their content; we are reviewing MOOCs for their pedagogical approaches.

CONTEXT

We developed an instrument to characterize the pedagogical approaches taken in MOOCs --

modeled on a similar instrument proposed by Tom Reeves (1996) for describing the pedagogical approaches found in computer-based instruction --“Pedagogical dimensions are concerned with those aspects of design and implementation . . . that directly affect learning.”

AMP (Assessing MOOC Pedagogy) tool

pedagogical category

ratings

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)focus of activities (convergent

) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5 (frequent,

constructive)cooperative learning (unsupporte

d) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupporte

d) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

AMP TOOL

METHODSBy Fall 2013, ACE evaluated and approved 5 Coursera, 1 EdX, and 7 Udacity MOOCs -- College Algebra, BioElectricity, Genetics, Pre-Calculus, and Single Variable Calculus, Circuits, Introduction to Artificial Intelligence, Introduction to Computer Science, Introduction to Physics, Introduction to Statistics, Introduction to Parallel Programming, 3-D Modeling and HTML 5 Game Development

We reviewed the same 13 MOOCs and four other Coursera MOOCs that covered non-STEM courses.

METHODSWe iteratively revised the AMP tool as we tested its efficacy on the approved MOOCs, first by reducing the scale of each dimension from ten to five, and then by establishing specific criteria for each dimension, to improve inter-rater reliability. We reviewed all the approved MOOCs using the revised instrument.

Four researchers reviewed each MOOCs. Inter-rater reliability was >80% on all the reviews before meeting to achieve consensus on the ratings.

The approved COURSERA courses followed a format that resembled the traditional lecture/text/testing routine of traditional university courses, including cohorts and weekly hard deadlines.


ratings



centered)

focus of activities (convergent) 1 2 3 4 5 (divergent)


structure) approach to

content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5

(frequent, constructive

)cooperative

learning(unsupport

ed) 1 2 3 4 5 (integral)


differences(unsupport

ed) 1 2 3 4 5 (multifaceted)



college algebra

bioelectricity


ratings



centered)







)cooperative

learning(unsupport








ratings



centered)







)cooperative

learning(unsupporte

d) 1 2 3 4 5 (integral)






genetics & evolution

pre-calculuspedagogical

category ratings



centered)







)cooperative

learning(unsupport







single variable calculus pedagogical

category ratings




) 1 2 3 4 5 (divergent)





)cooperative

learning(unsupporte

d) 1 2 3 4 5 (integral)







ratings




) 1 2 3 4 5 (divergent)





d) 1 2 3 4 5 (integral)






college algebra bioelectricity genetics & evolution

pre-calculus single variable calculus

circuitspedagogical

category ratingsepistemology (objectivist) 1 2 3 4 5 (constructivist)


centered)focus of activities (convergent) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more structure)

approach to content (concrete) 1 2 3 4 5 (abstract)


constructive)cooperative

learning(unsupporte

d) 1 2 3 4 5 (integral)






The Coursera courses approved for ACE credit, as well as EdX’s Circuits, tended to be objectivist, teacher-centered, convergent, highly structured, more abstract than concrete, with minimal feedback. The courses tended to fall somewhere in the middle between supporting and not supporting cooperative learning, accommodating and not accommodating individual differences, artificial and authentic activities/assessment, and passive and active user roles.


ratings



centered)





constructive)

cooperative learning (unsupported) 1 2 3 4 5 (integral)






college algebra bioelectricity genetics & evolutionpre-calculus single variable calculus circuits

UDACITY courses followed a format that resembled programmed learning, including self-pacing and embedded quizzes.

interactive 3-d graphicspedagogical







constructive)







html 5 graphicspedagogical

category ratings



centered)






)cooperative

learning(unsupporte

d) 1 2 3 4 5 (integral)






pedagogical category ratings








constructive)cooperative

learning (unsupported) 1 2 3 4 5 (integral)


differences(unsupported) 1 2 3 4 5 (multifaceted)



introduction to artificial intelligence

introduction to computer sciencepedagogical




structure (less structure) 1 2 3 4 5 (more structure)

approach to content (concrete) 1 2 3 4 5 (abstract)


constructive)cooperative learning (unsupported) 1 2 3 4 5 (integral)


differences(unsupported) 1 2 3 4 5 (multifaceted)



introduction to parallel processingpedagogical

category ratings



centered)






)cooperative

learning(unsupporte

d) 1 2 3 4 5 (integral)






introduction to physicspedagogical

category ratings




) 1 2 3 4 5 (divergent)





)cooperative

learning(unsupporte

d) 1 2 3 4 5 (integral)






introduction to statisticspedagogical

category ratings



centered)







)cooperative

learning(unsupport







Udacity courses tended to be neither objectivist nor constructivist, slightly less teacher-centered and convergent than Coursera courses, highly structured, halfway between abstract and concrete, with immediate, clear, and constructive feedback. They also tended not to support cooperative learning, but because of their self-directed approach were quite accommodating of individual differences. Udacity also made an effort to develop authentic activities, and perhaps because of the large number of CS courses, supported a more generative user role than the ACE-approved courses from Coursera and EdX.


ratings



centered)





constructive)







interactive 3-d graphics HTML 5 graphics intro to artificial intelligence intro to computer science intro to parallel processing

intro to physics intro to statisics

EDX COURSERA UDACITYepistemology 1.0 1.0 2.4role of teacher 1.0 1.4 2.0

focus of activities 1.0 1.0 1.9structure 5.0 5.0 4.9

approach to content 4.0 3.6 3.0feedback 3.0 2.0 4.3

cooperative learning 2.0 2.8 2.1accommodation of individual

differences 2.0

2.6 3.0activities/assessment 1.0 2.6 3.3

user role 2.0 3.0 3.1average ratings across platforms


ratings




) 1 2 3 4 5 (divergent)





d) 1 2 3 4 5 (integral)






Coursera EdX Udacity

comparisons of ACE approved courses across platforms

art & inquirypedagogical

category ratings




) 1 2 3 4 5 (divergent)





d) 1 2 3 4 5 (integral)







ratings




) 1 2 3 4 5 (divergent)





d) 1 2 3 4 5 (integral)






comic books & graphic novels


ratings




) 1 2 3 4 5 (divergent)





d) 1 2 3 4 5 (integral)






jazz improvisation


ratings




) 1 2 3 4 5 (divergent)





d) 1 2 3 4 5 (integral)






the music of the Beatles


ratings




) 1 2 3 4 5 (divergent)





d) 1 2 3 4 5 (integral)






art & inquiry comic books & graphic novelsjazz improvisation the music of the Beatles

non-stem


ratings




) 1 2 3 4 5 (divergent)





d) 1 2 3 4 5 (integral)






art & inquiry comic books & graphic novelsjazz improvisation

non-stem

STEM BEATLES NON-STEMepistemology 1.0 1.0 4.7role of teacher 1.4 1.0 3.0

focus of activities 1.0 1.0 4.3structure 5.0 4.0 3.7

approach to content 3.6 2.0 2.3feedback 2.0 2.0 3.7

cooperative learning 2.8 2.0 3.0accommodation of individual

differences 2.6 2.0 2.7activities/assessment 2.6 1.0 4.7

user role 3.0 2.0 4.3average ratings across domains


ratings




) 1 2 3 4 5 (divergent)





d) 1 2 3 4 5 (integral)






STEM Beatles non-STEM

(objectivist) 1 2 3 4 5 (constructivist)

(teacher-centered) 1 2 3 4 5 (student-

centered)(convergent

) 1 2 3 4 5 (divergent)

(less structure) 1 2 3 4 5 (more

structure) (concrete) 1 2 3 4 5 (abstract)

(infrequent, unclear) 1 2 3 4 5

(frequent, constructiv

e)(unsupporte

d) 1 2 3 4 5 (integral)

(unsupported) 1 2 3 4 5 (multifacet

ed)(artificial) 1 2 3 4 5 (authentic)

(passive) 1 2 3 4 5 (generative)

two metaphors for learning (Sfard, 1998)

acquisition

participation

CONCLUSIONS• AMP tool can effectively characterize

MOOC pedagogies• At least two distinct pedagogical

approaches can be distinguished• Future research will explore these

distinctions, as well as MOOCs offered on different platforms & differently organized

• Future research will try to link pedagogical to learning outcomes

[email protected] [email protected]

[email protected]@uis.edu

mailto:[email protected]




Documents

E xploring MOOC Pedagogies