Measuring the Effectiveness of Online Tutorials: A Pragmatic Approach

Measuring the Effectiveness Measuring the Effectiveness of Online Tutorials: of Online Tutorials:

A Pragmatic ApproachA Pragmatic Approach

Curt G. FriehsCurt G. FriehsBusiness LibrarianBusiness Librarian

Kansas City, Kansas Public LibraryKansas City, Kansas Public Library

Cindy L. CraigCindy L. CraigAssistant Professor and Social Sciences LibrarianAssistant Professor and Social Sciences Librarian

Wichita State UniversityWichita State University

A Bear with a Bad Idea

Occasionally People Get Bad Ideas Too…

That Includes Librarians “Interactivity is the online hallmark of active

learning.” ~Nancy Dewald Technology is naturally intuitive to college

students and young adults. Tutorials are too hard to make and “I’m too old.” Why can’t we just use the vendor tutorials? “I’ll just put my handouts online. It’s the same

thing as an online tutorial.” “Online tutorials are going to make my job

obsolete.”

Online tutorials can supplement (but not completely replace) live library instruction unless you’re this guy....

Our Conclusions

Replacement Librarian and Library Fine Negotiator “iBender”

Our Conclusions, Continued

Students learn more from video tutorials than from HTML tutorials.

Interactivity is not necessary for learning (and may hinder it).

Research: Part IResearch: Part I

Surveyed 140 finance students after watching Surveyed 140 finance students after watching Value Line Online tutorial. Value Line Online tutorial.

Tutorial left students wanting to know even Tutorial left students wanting to know even more.more.

But more research was needed. Would But more research was needed. Would students outside of a brutally difficult finance students outside of a brutally difficult finance class have different learning needs?class have different learning needs?

HTML vs. Streaming Media Tutorials

Nadaleen Tempelman-Kluit. (2006). Multimedia Learning Theories…

Principle HTML Tutorial Streaming Media Tutorial

Modality Effect Uses visual only Visual & verbal used

Dual Coding Theory

Not used (visual only)

Yes (visual & verbal)

Constructivism Navigation allows for choice (interactivity)

Navigation is linear (minimal interactivity)

Creating the Tutorials

HTML Tutorial

Video Tutorial

Confidence BEFORE tutorial- Interactive: 64%- Video: 47%

Confidence AFTER tutorial- Interactive: 89%- Video: 91%

Survey Results

Video tutorial students reported greater confidence boost after viewing.

Interactive n=53 Video n=54

Question 1- Interactive: 29 (55%)- Video: 42 (78%)

Question 2- Interactive: 39 (74%)- Video: 51 (94%)

Question 3- Interactive: 20 (38%)- Video: 26 (48%)

Quiz Results: Correct AnswersInteractive n=53 Video n=54

Quiz Results: Correct AnswersInteractive n=53 Video n=54

Question 4

- Interactive: 29 (55%)

- Video: 38 (70%) Question 5

- Interactive: 35 (66%)

- Video: 47 (87%)

Video tutorial students scored higher on every quiz question.

Laura Baas Jennifer McCoy Peggy Ward John Sweller South Central Kansas Library System North East Kansas Library System

Special Thanks

Curt Friehs Cindy Craigcfriehs@kckpl.lib.ks.us cindy.craig@wichita.edu